Image sensor, imaging device, and image sensor control method

Abstract

An image sensor of the present invention, which switches conversion gain, enhances image quality and comprises a pixel array unit, an analog-to-digital conversion unit, and an estimation control unit. In the pixel array unit of the image sensor, a designated number of pixels are arranged in a 2D lattice shape. The analog-to-digital conversion unit converts a pixel signal into a digital signal for each pixel from the designated number of pixels. The control unit determines, for each pixel, an estimated value of conversion gain for when an electrical charge is converted into voltage inside the pixel and controls said conversion gain to the estimated value.

Description

Image sensor, imaging device, and method for controlling image sensor

[0001] The present technology relates to an image sensor, and more particularly to an image sensor with variable conversion efficiency when converting to voltage, an imaging device, and a method for controlling the image sensor.

[0002] Conventionally, image sensors have utilized a driving method in which the conversion efficiency when converting electric charge to voltage is switched between two stages in order to expand the dynamic range. Hereinafter, a conversion efficiency higher than a predetermined value is referred to as "HCG (High Conversion Gain)," and a conversion efficiency lower than that predetermined value is referred to as "LCG (Low Conversion Gain)." When the conversion efficiency is switched between two stages, the readout time doubles compared to when the conversion efficiency is not switched. Therefore, an image sensor has been proposed in which the reset level is read out only using HCG, and the conversion efficiency is adaptively switched depending on the illuminance when reading out the signal level (see, for example, Patent Document 1).

[0003] US Patent Application Publication No. 2022 / 0124272

[0004] In the above-described conventional technology, the readout speed is increased by omitting the readout of the reset level in the LCG and switching the conversion efficiency when reading out the signal level. However, in the above-described image sensor, in the case of high illuminance, the reset level is read out by the HCG while the signal level is read out by the LCG. In this case, the correlation between the reset level and the signal level becomes poor, which causes a problem of deterioration in FD (Floating Diffusion) noise, kTC noise, and offset noise, resulting in a deterioration in the image quality of the image data.

[0005] This technology was developed in light of these circumstances, and aims to improve image quality in image sensors that switch conversion efficiencies.

[0006] The present technology has been made to solve the above-mentioned problems, and a first aspect thereof is an image sensor including a pixel array unit in which a predetermined number of pixels are arranged in a two-dimensional lattice pattern, an analog-to-digital converter unit that converts pixel signals of the predetermined number of pixels into digital signals, and a controller that calculates, for each pixel, a predicted value of conversion efficiency when converting electric charge into a voltage within the pixel and controls the conversion efficiency to the predicted value, and a control method thereof, thereby improving image quality.

[0007] In addition, in this first aspect, the pixel array unit may further include a vertical drive unit that sequentially selects and drives rows in the pixel array unit, and the control unit may include a determination unit that determines whether or not illuminance for each pixel is below a predetermined determination threshold each time the row is selected and outputs the determination result, and a prediction control unit that obtains the predicted value for the next row based on the determination result each time the row is selected and controls the conversion efficiency to the predicted value. This brings about the effect of predicting the conversion efficiency for each row.

[0008] In this first aspect, the determination unit may further determine for each pixel whether a critical error, in which the predicted value is an incorrect value, has occurred, and the prediction control unit may control the conversion efficiency to a value lower than a predetermined value for a predetermined pulse period when the critical error has occurred, thereby attenuating the pixel signal.

[0009] In addition, in the first aspect, the image processing device may further include a digital signal processing unit that, when the critical error occurs, performs a predetermined interpolation process on the digital signal of the pixel in which the critical error occurs, thereby improving image quality.

[0010] In the first aspect, the vertical drive section may simultaneously drive two rows in the pixel array section, thereby improving the readout speed.

[0011] In this first aspect, the predetermined number of pixels may include first and second pixels that share a floating diffusion layer, thereby providing an effect of reducing the number of elements per pixel.

[0012] In this first aspect, the pixel array unit may be divided into a predetermined number of groups, each group including a plurality of pixels, and the prediction control unit may obtain the predicted value for each group by performing a predetermined logical operation on the determination results within the group. This brings about an effect of predicting the conversion efficiency for each group.

[0013] In this first aspect, the pixel array unit may be divided into different groups for each column, thereby providing an effect that the conversion efficiency can be predicted for each column.

[0014] In the first aspect, the pixel array unit may be divided into different groups every two columns, thereby providing an effect that the conversion efficiency can be predicted for every two columns.

[0015] In the first aspect, the pixel array unit may be divided into different groups for each color, thereby providing an effect that the conversion efficiency can be predicted for each color.

[0016] In this first aspect, the control unit may include an illuminance determination unit that determines for each pixel whether or not illuminance is below a predetermined threshold based on the digital signal and outputs the determination result, a data buffer that stores each of the determination results, a prediction control unit that calculates the predicted value for the next frame based on the determination result each time a frame in which the digital signals are arranged is generated, and a logic circuit that controls the conversion efficiency to the predicted value, thereby providing the effect of predicting the conversion efficiency for each frame.

[0017] According to a second aspect of the present technology, there is provided an image sensor including: a pixel array unit in which a predetermined number of pixels are arranged in a two-dimensional lattice; an analog-to-digital conversion unit that converts pixel signals of the predetermined number of pixels into digital signals; a determination unit that determines, for each pixel, whether or not illuminance is below a predetermined determination threshold based on a reset level when the pixel is initialized; and a control unit that controls a conversion efficiency when converting charge into voltage in the pixel to a value lower than a predetermined value over a predetermined pulse period when the illuminance is not below the determination threshold, thereby achieving an effect of reducing the circuit size.

[0018] A third aspect of the present technology is an imaging device including a pixel array unit in which a predetermined number of pixels are arranged in a two-dimensional lattice pattern, an analog-to-digital conversion unit that converts pixel signals of the predetermined number of pixels into digital signals, a control unit that calculates, for each pixel, a predicted value of conversion efficiency when converting electric charge into a voltage within the pixel and controls the conversion efficiency to the predicted value, and a digital signal processing unit that performs predetermined signal processing on the digital signals. This brings about an effect of improving image quality in the imaging device.

[0019] 1 is a block diagram showing an example of a configuration of an imaging device according to a first embodiment of the present technology. FIG. 2 is a block diagram showing an example of a configuration of an image sensor according to the first embodiment of the present technology. FIG. 3 is a circuit diagram showing an example of a configuration of a vertical driver, a horizontal driver, and a pixel according to the first embodiment of the present technology. FIG. 4 is a block diagram showing an example of a configuration of a column signal processing circuit according to the first embodiment of the present technology. FIG. 5 is a circuit diagram showing an example of a configuration of a logic circuit according to the first embodiment of the present technology. FIG. 6 is a circuit diagram showing an example of a configuration of a conversion efficiency control circuit according to the first embodiment of the present technology. FIG. 7 is a timing chart showing an example of a readout operation of an image sensor when a prediction according to the first embodiment of the present technology is incorrect. FIG. 8 is a timing chart showing an example of a readout operation of an image sensor when a prediction according to the first embodiment of the present technology is correct. FIG. 9 is a graph showing an example of noise characteristics according to the first embodiment of the present technology. FIG. 10 is a diagram showing an example of a signal list according to the first embodiment of the present technology. FIG. 11 is a diagram for explaining a method of predicting conversion efficiency according to the first embodiment of the present technology. FIG. 12 is a diagram for explaining an example of a prediction algorithm according to the first embodiment of the present technology. FIG. 13 is a diagram for explaining another example of a prediction algorithm according to the first embodiment of the present technology. FIG. 14 is a diagram for explaining an influence of a prediction error according to the first embodiment of the present technology. 1 is a flowchart showing an example of operation of an image sensor according to a first embodiment of the present technology; FIG. 2 is a block diagram showing an example configuration of an image sensor according to a first modified example of the first embodiment of the present technology; FIG. 3 is a diagram showing an example of a prediction algorithm according to a first modified example of the first embodiment of the present technology; FIG. 4 is a diagram showing another example of a prediction algorithm according to a first modified example of the first embodiment of the present technology; FIG. 5 is a circuit diagram showing an example configuration of an FD sharing block according to a second modified example of the first embodiment of the present technology; FIG. 6 is a diagram showing an example of a prediction algorithm according to a second modified example of the first embodiment of the present technology;FIG. 10 is a diagram showing an example of a prediction algorithm in the case of grouping by color in a third modified example of the first embodiment of the present technology. FIG. 11 is a circuit diagram showing an example of a configuration of an FD sharing block in a fourth modified example of the first embodiment of the present technology. FIG. 12 is a diagram showing an example of a prediction algorithm in a fourth modified example of the first embodiment of the present technology. FIG. 13 is a circuit diagram showing an example of a configuration of a logic circuit in a second embodiment of the present technology. FIG. 14 is a timing chart showing an example of a readout operation of an image sensor in the second embodiment of the present technology. FIG. 15 is a circuit diagram showing an example of a configuration of a logic circuit in a third embodiment of the present technology. FIG. 16 is a block diagram showing an example of a configuration of a digital signal processing unit in a third embodiment of the present technology. FIG. 17 is a diagram showing an example of a prediction algorithm in the third embodiment of the present technology. FIG. 18 is a block diagram showing an example of a schematic configuration of a vehicle control system. FIG. 19 is an explanatory diagram showing an example of installation positions of an outside vehicle information detection unit and an imaging unit.

[0020] Hereinafter, modes for carrying out the present technology (hereinafter referred to as embodiments) will be described. The description will be given in the following order: 1. First embodiment (example of predicting conversion efficiency on a row-by-row basis) 2. Second embodiment (example of controlling conversion efficiency on a row-by-row basis) 3. Third embodiment (example of predicting conversion efficiency on a frame-by-frame basis) 4. Example of application to a moving body

[0021] 1 is a block diagram showing an example of the configuration of an imaging device 100 according to a first embodiment of the present technology. The imaging device 100 is a device that captures image data, and includes an imaging lens 110, an image sensor 200, a recording unit 120, and an imaging control unit 130. The imaging device 100 may be a digital camera such as an IoT camera, or an electronic device with an imaging function (such as a smartphone or a personal computer).

[0022] The image sensor 200 captures image data under the control of the imaging control unit 130. The image sensor 200 supplies the image data to the recording unit 120 via a signal line 209.

[0023] The imaging lens 110 focuses light and guides it to the image sensor 200. The imaging control unit 130 controls the image sensor 200 to capture image data. The imaging control unit 130 supplies an imaging control signal including, for example, a vertical synchronization signal VSYNC to the image sensor 200 via a signal line 139. The recording unit 120 records the image data.

[0024] Here, the vertical synchronization signal VSYNC is a signal that indicates the timing of imaging, and a periodic signal with a constant frequency (such as 60 Hz) is used as the vertical synchronization signal VSYNC.

[0025] It should be noted that although the imaging device 100 records image data, the image data may be transmitted to an external device. In this case, an external interface for transmitting the image data is further provided. Alternatively, the imaging device 100 may further display the image data. In this case, a display unit is further provided.

[0026] 2 is a block diagram showing an example of the configuration of an image sensor 200 according to the first embodiment of the present technology. The image sensor 200 includes a vertical drive unit 210, a pixel array unit 220, a timing control circuit 241, a DAC (Digital to Analog Converter) 242, a horizontal drive unit 250, and a column signal processing circuit 260. A predetermined number of pixels 230 are arranged in a two-dimensional lattice in the pixel array unit 220.

[0027] Hereinafter, a group of pixels 230 arranged in the horizontal direction will be referred to as a "row," and a group of pixels 230 arranged in a direction perpendicular to the rows will be referred to as a "column."

[0028] The timing control circuit 241 controls the operation timing of the vertical drive unit 210 , the DAC 242 , and the column signal processing circuit 260 in synchronization with a vertical synchronization signal VSYNC from the imaging control unit 130 .

[0029] The DAC 242 generates an analog reference signal that varies over time by digital-to-analog (DA) conversion. For example, a sawtooth ramp signal is used as the reference signal. The DAC 242 supplies the generated reference signal to the column signal processing circuit 260.

[0030] The vertical drive unit 210 sequentially selects and drives rows to output analog pixel signals. The pixels 230 photoelectrically convert incident light to generate analog pixel signals. The pixels 230 supply the pixel signals to a column signal processing circuit 260.

[0031] The horizontal driving section 250 drives specific transistors in each column under the control of the column signal processing circuit 260. The horizontal driving section 250 will be described in detail later.

[0032] The column signal processing circuit 260 performs signal processing such as AD conversion on pixel signals for each column. The column signal processing circuit 260 supplies image data in which the processed signals are arranged as a frame to the recording unit 120. The column signal processing circuit 260 is an example of a signal processing circuit as defined in the claims.

[0033] [Configuration Example of Pixel] FIG. 3 is a circuit diagram showing a configuration example of the vertical drive section 210, the horizontal drive section 250, and the pixel 230 according to the first embodiment of the present technology.

[0034] The vertical drive unit 210 has drivers 211, 212, and 213 arranged for each row. The driver 211 supplies a reset signal RST to the corresponding row, the driver 212 supplies a transfer signal TRG to the corresponding row, and the driver 213 supplies a selection signal SEL to the corresponding row.

[0035] A driver 251 is arranged for each column in the horizontal drive unit 250. A control line 228 and a vertical signal line 229 are wired vertically for each column in the pixel array unit 220. A control signal FDG[i] from a column signal processing circuit 260 is input to the driver 251 via a control line 259. i is an integer from 0 to I-1, and I indicates the total number of columns. The driver 251 amplifies the control signal FDG[i] and supplies it to the corresponding column via the control line 228.

[0036] The pixel 230 includes a photodiode 231, a transfer transistor 232, a reset transistor 233, a conversion efficiency control transistor 234, a capacitance element 235, a floating diffusion layer 236, an amplification transistor 237, and a selection transistor 238. Each transistor in the pixel 230 is, for example, an nMOS (n-channel Metal Oxide Semiconductor) transistor.

[0037] The photodiode 231 photoelectrically converts incident light to generate electric charges. The transfer transistor 232 transfers electric charges from the photodiode 231 to the floating diffusion layer 236 in accordance with a transfer signal TRG. The reset transistor 233 initializes the amount of electric charge in the floating diffusion layer 236 in accordance with a reset signal RST. The floating diffusion layer 236 accumulates electric charges and generates a voltage according to the amount of electric charge.

[0038] The conversion efficiency control transistor 234 opens and closes the path between the capacitance element 235 and the floating diffusion layer 236 in accordance with a control signal FDG[i].

[0039] The amplification transistor 237 forms a source follower circuit and outputs a signal at a level corresponding to the voltage of the floating diffusion layer 236. The selection transistor 238 outputs the signal of the amplification transistor 237 as a pixel signal SIG to the column signal processing circuit 260 via the vertical signal line 229 in accordance with a selection signal SEL.

[0040] The circuit of the pixel 230 is not limited to the configuration exemplified in the figure, as long as it can generate a pixel signal by photoelectric conversion and can switch the conversion efficiency.

[0041] The potential of the vertical signal line 229 when the floating diffusion layer 236 is initialized (in other words, the level of the pixel signal SIG) is referred to as the “P phase” or “reset level.” The level of the pixel signal when the signal charge is transferred is referred to as the “D phase” or “signal level.”

[0042] Furthermore, the conversion efficiency when the conversion efficiency control transistor 234 is in the off state is higher than when the conversion efficiency control transistor 234 is in the on state. Hereinafter, the higher conversion efficiency will be referred to as "HCG" and the lower conversion efficiency will be referred to as "LCG".

[0043] At the end of exposure using the rolling shutter method, the pixel 230 converts the charge into a voltage with the conversion efficiency of either HCG or LCG in accordance with the control signal FDG[i], and sequentially generates a reset level and a signal level. The downstream column signal processing circuit 260 sequentially converts the reset level and the signal level into digital signals, respectively, and performs CDS (Correlated Double Sampling) to determine the difference between them.

[0044] 4 is a block diagram showing an example of the configuration of the column signal processing circuit 260 according to the first embodiment of the present technology. The column signal processing circuit 260 includes a plurality of column circuits 270 and a digital signal processing unit 280. The column circuits 270 are arranged for each column. Each of the column circuits 270 includes an ADC 275 and a logic circuit 300.

[0045] Furthermore, a load MOS circuit 254 that supplies a constant current is connected to each of the vertical signal lines 229 .

[0046] The ADC 275 converts the analog pixel signal SIG into a digital signal Dout. The ADC 275 includes auto-zero switches 271 and 272, a comparator 273, and a counter 274. The circuit made up of the ADC 275 for each column is an example of the analog-to-digital conversion unit described in the claims.

[0047] A non-inverting input terminal (+) of the comparator 273 is connected to the vertical signal line 229 via a capacitance element. Furthermore, an inverting input terminal (−) of the comparator 273 is connected to the DAC 242 via a capacitance element. The comparator 273 compares the pixel signal SIG input via the vertical signal line 229 with the ramp signal Rmp from the DAC 242, and outputs the comparison result VCO[i] to the logic circuit 300 and the counter 274.

[0048] The counter 274 counts the count value over the period until the comparison result VCO[i] is inverted. The counter 274 supplies a digital signal Dout indicating the count value to the digital signal processing unit 280.

[0049] For example, the counter 274 counts down during the readout period of the reset level and counts up during the readout period of the signal level. This allows CDS processing to be performed to determine the difference between the reset level and the signal level. Note that the ADC 275 may be configured to perform only AD (Analog to Digital) conversion, and the downstream digital signal processing unit 280 may perform CDS processing.

[0050] Here, the pixel array unit 220 is divided into a predetermined number of groups, each including a plurality of pixels. These groups are used as units for predicting and controlling conversion efficiency. The pixel array unit 220 is divided into groups, for example, by column. The number of columns in each group is set to, for example, N (N is an integer).

[0051] The i-th column logic circuit 300 receives N-1 determination results is_dim from each of the column circuits 270 other than the i-th column in the same group. The i-th column logic circuit 300 also outputs a determination result is_dim[i] to each of the N-1 column circuits 270 other than the i-th column in the same group. A method for generating the determination result is_dim will be described later.

[0052] The logic circuit 300 for the i-th column determines whether the illuminance of light incident on the pixel for the i-th column is below a predetermined determination threshold based on the comparison result VCO[i]. Furthermore, each time a row is selected by the vertical drive unit 210, the logic circuit 300 for the i-th column predicts the conversion efficiency of the next row based on the determination result within the group, and controls the conversion efficiency of the i-th column using a control signal FDG[i]. Furthermore, the logic circuit 300 supplies the illuminance determination result to the digital signal processing unit 280. The circuit composed of the logic circuits 300 for each column is an example of a control unit as defined in the claims.

[0053] The digital signal processing unit 280 performs various signal processing on the digital signal Dout and supplies the recording unit 120 with frames in which the processed signals are arranged.

[0054] As described above, the column signal processing circuit 260 can expand the dynamic range by switching the conversion efficiency according to the illuminance for each pixel. Furthermore, the readout speed can be increased compared to when the HCG and LCG are read out sequentially for each row. This allows for improved image quality while also improving the readout speed.

[0055] 5 is a circuit diagram showing an example of the configuration of a logic circuit 300 according to the first embodiment of the present technology. The logic circuit 300 includes a determination circuit 310 and a prediction control circuit 320.

[0056] Each time a row is selected by the vertical drive unit 210, the determination circuit 310 determines whether the illuminance of the i-th column is below a predetermined determination threshold and outputs the determination result. This determination circuit 310 includes latch circuits 311, 312, and 313, inverters 321, 322, and 323, AND (logical product) gates 331, 332, and 333, and an OR (logical sum) gate 351. Note that the circuit formed by the determination circuits 310 for each column is an example of the determination unit described in the claims.

[0057] Each time a row is selected by the vertical drive unit 210, the prediction control circuit 320 predicts the next conversion efficiency based on the determination result of the illuminance within the group and controls the i-th column based on the predicted value. This prediction control circuit 320 includes a latch circuit 314, a conversion efficiency control circuit 340, and an AND gate 334. Note that the circuit consisting of the prediction control circuit 320 for each column is an example of a prediction control unit described in the claims.

[0058] The latch circuit 311 holds the comparison result VCO[i] in accordance with the enable signal en1 from the timing control circuit 241. When the enable signal en1 is at a high level, the latch circuit 311 updates the held value with the comparison result VCO[i], and when the enable signal en1 is at a low level, the latch circuit 311 holds the state. The latch circuit 311 then supplies the held value to the inverter 321 as a latch signal Lat1.

[0059] The latch circuit 312 holds the comparison result VCO[i] in accordance with the enable signal en2 from the timing control circuit 241. When the enable signal en2 is at a high level, the latch circuit 312 updates the held value with the comparison result VCO[i], and when the enable signal en2 is at a low level, the latch circuit 312 holds the state. The latch circuit 312 then supplies the held value to the AND gate 332 as a latch signal Lat2.

[0060] The latch circuit 313 holds the comparison result VCO[i] in accordance with the enable signal en3 from the timing control circuit 241. When the enable signal en3 is at a high level, the latch circuit 313 updates the held value with the comparison result VCO[i], and when the enable signal en3 is at a low level, the latch circuit 313 holds the state. The latch circuit 313 then supplies the held value to the AND gate 333 as a latch signal Lat3.

[0061] The inverter 321 inverts the latch signal Lat 1 and supplies the inverted signal to the AND gate 331 .

[0062] The AND gate 331 supplies the logical product of the inverted signal from the inverter 321 and the predicted value cg[i] from the latch circuit 314 to the inverter 322 and the digital signal processing unit 280 as an error flag ce[i].

[0063] The AND gate 332 supplies the logical product of the latch signal Lat2 and the predicted value cg[i] to the OR gate 351 as the determination result hcg[i].

[0064] The inverter 322 inverts the error flag ce[i] and supplies it to the OR gate 351 .

[0065] The inverter 323 inverts the predicted value cg[i] and supplies it to the AND gate 333 .

[0066] The AND gate 333 supplies the logical product of the latch signal Lat3 and the inverted signal from the inverter 323 to the OR gate 351 as the determination result Lcg[i].

[0067] The OR gate 351 outputs the logical sum of the determination results Lcg[i] and hcg[i] and the inverted signal from the inverter 322 as a determination result is_dim[i] to the AND gate 334. The determination result is_dim[i] is also output to each of the N-1 logic circuits 300 other than the i-th column in the same group.

[0068] The AND gate 334 supplies the logical product of the determination results is_dim for the N columns in the same group to the latch circuit 314 as the determination result grp[i].

[0069] The latch circuit 314 holds the determination result grp[i] in accordance with the enable signal en4 from the timing control circuit 241. When the enable signal en4 is at a high level, the latch circuit 314 updates the held value with the determination result grp[i], and when the enable signal en4 is at a low level, the latch circuit 314 holds the state. The latch circuit 314 then supplies the held value as a predicted value cg[i] to the AND gate 331, the digital signal processing unit 280, the AND gate 332, the conversion efficiency control circuit 340, and the inverter 323.

[0070] The conversion efficiency control circuit 340 generates a control signal FDG[i] and supplies it to the horizontal drive unit 250 .

[0071] 6 is a circuit diagram showing an example of the configuration of the conversion efficiency control circuit 340 according to the first embodiment of the present technology. The conversion efficiency control circuit 340 includes AND gates 341 and 342, an inverter 343, and an OR gate 344.

[0072] The AND gate 341 supplies the OR gate 344 with the logical product of the error flag ce[i] from the AND gate 331 and the control signal fdg_c 2 from the timing control circuit 241 .

[0073] The inverter 343 inverts the predicted value cg[i] from the latch circuit 314 and supplies it to the AND gate 342 and the OR gate 344 .

[0074] The AND gate 342 supplies the OR gate 344 with the logical product of the control signal fdg_c 1 from the timing control circuit 241 and the inverted signal from the inverter 343 .

[0075] The OR gate 344 outputs the logical sum of the signals from the AND gates 341 and 342 and the inverter 343 to the horizontal drive section 250 as a control signal FDG[i].

[0076] 7 and 8, the readout operation of the image sensor 200 will be described. Exposure is performed sequentially row by row in synchronization with the horizontal synchronization signal XHS using the rolling shutter method, and at the end of exposure of each row, the reset level and signal level of that row are read out sequentially.

[0077] FIG. 7 is a timing chart showing an example of a readout operation of the image sensor 200 when the prediction according to the first embodiment of the present technology is incorrect.

[0078] At timing T11, the timing control circuit 241 causes the horizontal synchronization signal XHS to fall, thereby starting readout of the i-th row.

[0079] The vertical drive unit 210 supplies a reset signal RST to the selected row over a pulse period from timing T11. Here, it is assumed that the current conversion efficiency is predicted based on the result of the previous illuminance determination, and HCG is set.

[0080] Furthermore, the timing control circuit 241 supplies a high-level control signal fdg_c1 to the logic circuit 300 of each column over the pulse period from timing T11. In response to this signal, the logic circuit 300 supplies a high-level control signal FDG[i] to the i-th column over the pulse period from timing T11.

[0081] When the timing control circuit 241 returns the control signal fdg_c1 to low level after the pulse period has elapsed, charge injection charges flow into the HCG floating diffusion layer 236. The amount of this charge injection varies due to manufacturing variations in transistors. HCG pixels, in particular, have small capacitance values ​​and are therefore susceptible to variations, potentially resulting in fixed pattern noise (FPN). Therefore, the logic circuit 300 returns the control signal FDG[i] to low level after supplying a pulse of the control signal fdg_c1 between timings T16 and T17 (described below). As a result, since there is a correlation between the amount of charge injection occurring immediately after timing T11 and at timing T17, FPN can be removed by CDS processing.

[0082] However, when the control signal FDG[i] is returned to the low level, there is no correlation between the kTC noise that occurs immediately after timing T11 and that occurs at timing T17, and therefore the noise cannot be completely removed by CDS processing. However, in pixels with high illuminance, the optical shot noise is greater than the kTC noise, and therefore the deterioration in image quality due to the inability to completely remove the kTC noise is minor.

[0083] The high-level reset signals RST and FDG[i] at timing T11 initialize the pixels in the selected row, generating a P-phase level (i.e., reset level). The dashed-dotted line in the figure indicates the potential of the vertical signal line 229. The potential immediately after timing T11 corresponds to the reset level.

[0084] Then, the ramp signal Rmp gradually drops over the reset level readout period, and the reset level is read out.

[0085] During the period from timing T13 to T14 immediately after reading out the reset level, the vertical drive unit 210 supplies a high-level transfer signal TRG to the selected row. This causes the potential of the vertical signal line 229 to drop according to the signal amount, generating a D-phase level (i.e., signal level). The difference between the signal level and the reset level is denoted as Δvsl.

[0086] Then, during the period from timing T14 to T15, the DAC 242 decreases the ramp signal Rmp. If the difference in level between the ramp signal Rmp during auto-zero and the ramp signal Rmp during the decrease is ΔRmp1, then ΔRmp1 can be expressed by the following equation, for example: ΔRmp1=TH2*u ＨＣＧ In the above equation, TH2 represents a threshold value used to determine the illuminance. ＨＣＧ indicates the set value of HCG. The unit of TH2 is, for example, e, and u ＨＣＧ The unit of is, for example, V / e. If the parasitic capacitance of the floating diffusion layer 236 is fd1, then u ＨＣＧ is expressed by the following formula: ＨＣＧ = 1e － / fd1 ... Formula 2

[0087] Furthermore, the timing control circuit 241 supplies a high-level enable signal en1 to the logic circuit 300 of each column over the period from just before timing T15 until timing T15. At this time, if Δvsl is equal to or greater than ΔRmp1, the logic circuit 300 determines that the illuminance is high, not below the determination threshold corresponding to TH2. On the other hand, if Δvsl is smaller than ΔRmp1, the logic circuit 300 determines that the illuminance is low, below the determination threshold. In the figure, it is assumed that the illuminance is high.

[0088] When the illuminance is high, the conversion efficiency is preferably LCG. On the other hand, if the illuminance is high but the prediction is HCG, the prediction is incorrect, and this error is called a "critical error." In the figure, it is assumed that a critical error has occurred in column i.

[0089] The timing control circuit 241 then supplies a high-level control signal fdg_c2 to the logic circuit 300 of each column over the period from timing T16 to timing T17. Because a critical error occurred in column i during this period, the logic circuit 300 supplies a high-level control signal FDG[i] to column i. As a result, column i is temporarily controlled by LCG, and the amplitude of the potential of the vertical signal line 229 is attenuated according to the LCG / HCG ratio. This control is referred to as "analog repair." This analog repair can suppress degradation of image quality when a critical error occurs. In addition to analog repair, the digital signal processing unit 280 can also perform digital repair, which will be described later.

[0090] Furthermore, in the case of high illuminance, even if the charges are transferred during the period from timing T13 to T14, the floating diffusion layer 236 may not be able to receive all of the charges, and the charges may remain in the photodiode 231. For this reason, the vertical drive unit 210 supplies a high-level transfer signal TRG to the selected row during the period from timing T16 immediately before the signal level is read out until immediately before timing T17. This allows the charges to be completely transferred.

[0091] The ramp signal Rmp gradually drops over the signal level readout period from timing T17 onwards, and the signal level is read out during this period.

[0092] After reading out the signal level, the DAC 242 increases the ramp signal Rmp. Then, during the period from just before timing T18 during the increase until timing T18, the timing control circuit 241 supplies a high-level enable signal en2 to the logic circuit 300 of each column. If the difference in level of the ramp signal Rmp between the time of auto-zero and timing T18 is ΔRmp2, then this ΔRmp2 can be expressed by, for example, the following equation: ΔRmp2=TH3*u ＨＣＧIn the above formula, TH3 is a threshold value lower than TH2, and the unit is, for example, e.

[0093] The pulse of the enable signal en2 determines whether the signal amount of the pixel for which HCG is set is smaller than TH3. If the signal amount is equal to or greater than TH3, it indicates that the illuminance of the pixel is high and does not fall below the determination threshold value corresponding to TH3. This determination result is used to predict the conversion efficiency.

[0094] After timing T18, the DAC 242 further increases the ramp signal Rmp. Then, during the period from just before timing T19 during the increase until timing T19, the timing control circuit 241 supplies a high-level enable signal en3 to the logic circuit 300 of each column. If the difference in level of the ramp signal Rmp between the time of auto-zero and timing T19 is ΔRmp3, this ΔRmp3 can be expressed by, for example, the following equation: ΔRmp3=TH3*u ＬＣＧ ...Equation 4 In the above equation, u ＬＣＧ indicates the setting value of the LCG. ＬＣＧ The unit of is, for example, V / e. When the parasitic capacitance of the capacitance element 235 is fd2 and the gate capacitance of the conversion efficiency control transistor 234 in the ON state is Cg, u ＬＣＧ is expressed by the following formula: ＬＣＧ = 1e － / (fd1+fd2+Cg) ... Equation 5

[0095] The pulse of the enable signal en3 determines whether the signal amount of the pixel for which LCG is set is smaller than TH3 or not. The result of this determination is used to predict the conversion efficiency.

[0096] Then, from timing T20 over the pulse period, the timing control circuit 241 supplies a high-level enable signal en4 to the logic circuit 300 of each column, thereby generating and holding the next predicted value cg[i] of the conversion efficiency.

[0097] 8 is a timing chart showing an example of a readout operation of the image sensor 200 in which the prediction according to the first embodiment of the present technology is correct. In FIG. 8, it is assumed that the same conversion efficiency as in FIG. 7 is predicted and HCG is set in the i-th column. Meanwhile, in FIG. 8, unlike FIG. 7, it is assumed that the i-th column is determined to have low illuminance.

[0098] In this case, since the prediction was correct, no critical error occurs, and the control signal fdg_c2 is kept at low level for the logic circuit 300 in the i-th column during the period from timing T15 to timing T17. Therefore, the amplitude of the potential is not attenuated.

[0099] 9 is a graph showing an example of noise characteristics in the first embodiment of the present technology. The vertical axis in the figure represents the amount of noise, and the horizontal axis represents the amount of signal. In the figure, white circles represent plots of measured values ​​of optical shot noise. Gray circles represent readout noise when read out using HCG. Black circles represent readout noise when read out using LCG.

[0100] In the high-illuminance region where the signal level is TH2 or higher, optical shot noise is dominant, and in this region, it is preferable to set the conversion efficiency to LCG. In the low-illuminance region where the signal level is less than TH1, circuit noise is dominant, and in this region, it is preferable to set the conversion efficiency to HCG. In the medium-illuminance region where the signal level is greater than TH1 and less than TH2, it does not matter whether the conversion efficiency is set to LCG or HCG.

[0101] The threshold value TH3 is set to a value that satisfies the following formula, for example: TH1<TH3<TH2 (Formula 6)

[0102] 10 is a diagram illustrating an example of a signal list according to the first embodiment of the present technology. The latch signal Lat1 indicates the first determination result within one AD period. This determination result is used to determine whether or not to perform analog repair. For example, when Δvsl is TH2*u ＨＣＧ On the other hand, when Δvsl is less than TH2*u ＨＣＧIf this is the case (in other words, if the illuminance is high), the latch signal Lat1 is set to a logical value of "0." If HCG is set despite the high illuminance, it is determined that a critical error has occurred, and analog repair is performed.

[0103] The latch signal Lat2 indicates the second determination result within one AD period. For example, when Δvsl is TH3*u ＨＣＧ On the other hand, when Δvsl is less than TH3*u, the logical value of the latch signal Lat2 is set to "1". ＨＣＧ If this is the case, the latch signal Lat2 is set to a logical value "0".

[0104] The latch signal Lat3 indicates the third determination result within one AD period. For example, when Δvsl is TH3*u ＬＣＧ On the other hand, when Δvsl is less than TH3*u, the logical value of the latch signal Lat3 is set to "1". ＬＣＧ If this is the case, the latch signal Lat3 is set to the logical value "0." The results of the second and third determinations are used to predict the next conversion efficiency.

[0105] The predicted value cg[i] indicates the predicted value of the conversion efficiency of the i-th column. For example, if an HCG is predicted, the predicted value cg[i] is set to a logical value of "1," and if an LCG is predicted, the predicted value cg[i] is set to a logical value of "0."

[0106] The error flag ce[i] indicates whether or not there is a critical error in column i. For example, if there is a critical error, the error flag ce[i] is set to a logical value of "1," and if there is no critical error, the error flag ce[i] is set to a logical value of "0."

[0107] The determination result hcg[i] indicates whether the incident light of the pixel set to HCG is low illuminance or not. For example, if the incident light is low illuminance, the logical value "1" is set to hcg[i], and if the incident light is high illuminance, the logical value "0" is set to hcg[i].

[0108] The determination result Lcg[i] indicates whether the incident light of the pixel set in the LCG is low illuminance or not. For example, if the incident light is low illuminance, the logical value "1" is set to Lcg[i], and if the incident light is high illuminance, the logical value "0" is set to Lcg[i].

[0109] The determination result is_dim[i] indicates whether the incident light on the pixel in column i is low illuminance or not. For example, if the incident light is low illuminance, the logical value "1" is set to is_dim[i], and if the incident light is high illuminance, the logical value "0" is set to is_dim[i].

[0110] The determination result grp[i] indicates whether the incident light of all pixels in the group including the i-th column is low illuminance. For example, if all pixels are low illuminance, the logical value "1" is set to grp[i]. If one or more pixels are high illuminance, the logical value "0" is set to grp[i].

[0111] 11 is a diagram illustrating a method for predicting conversion efficiency according to the first embodiment of the present technology. The logic circuit 300 for the i-th column determines whether or not the output of one or more pixels in a group including the i-th column is equal to or greater than TH3 at the end of the readout period for the j-th row (j is an integer). In other words, it is determined whether or not the illuminance of one or more pixels in the group is below a determination threshold.

[0112] If the output of one or more pixels in the group of row j is equal to or greater than TH3, the logic circuit 300 sets the predicted value of the conversion efficiency of the (j+1)th row of all pixels in that group as LCG.

[0113] On the other hand, if the output of all pixels in the group in the jth row is less than TH3, the logic circuit 300 sets the predicted value of the conversion efficiency of all pixels in the group in the (j+1)th row as HCG.

[0114] As illustrated in the figure, each time a row is selected, the logic circuit 300 determines whether the illuminance of the pixel in column i is below the determination threshold, and predicts the next conversion efficiency based on the determination result. As illustrated in Figure 5, the AND gate 334 performs a logical operation on the determination result is_dim[i] for each column to obtain a predicted value cg[i].

[0115] 12 is a diagram showing an example of a prediction algorithm in the first embodiment of the present technology. In a, b, and c in the figure, gray blocks indicate pixels for which readout has been completed. White blocks indicate pixels before or during readout. a1, b1, a2, and b2 indicate group identifiers. The same applies hereinafter.

[0116] For example, the pixel array unit 220 is divided into different groups for each column. Group a1 includes column i and column (i+2). Group b1 includes column (i+1) and column (i+3). Group a2 includes column (i+4) and column (i+6). Group b2 includes column (i+5) and column (i+7).

[0117] As shown in FIG. 1A, the vertical drive unit 210 selects and drives row j, and then as shown in FIG. 1B, the vertical drive unit 210 selects and drives row (j+1), and then selects and drives row (j+2).

[0118] The logic circuit 300 of group a1 predicts the conversion efficiency of group a1 in the (j+1)th row based on the determination result is_dim[i] of group a1 in the jth row. The logic circuit 300 of group b1 predicts the conversion efficiency of group b1 in the (j+1)th row based on the determination result is_dim[i] of group b1 in the jth row.

[0119] The logic circuit 300 of group a1 predicts the conversion efficiency of group a1 in the (j+2)th row based on the determination result is_dim[i] of group a1 in the (j+1)th row. The logic circuit 300 of group b1 predicts the conversion efficiency of group b1 in the (j+2)th row based on the determination result is_dim[i] of group b1 in the (j+1)th row.

[0120] The conversion efficiencies of the (j+2)th row and onward are predicted using the same algorithm. The conversion efficiencies of groups a2 and b2 are also predicted using the same algorithm.

[0121] As shown in the figure, for each group including a plurality of adjacent pixels whose inter-pixel distance is less than a predetermined value, the next conversion efficiency is predicted based on the judgment results obtained within that group. Since the outputs of adjacent pixels are often correlated, the prediction algorithm shown in the figure can predict an appropriate conversion efficiency. This allows for improved image quality of image data compared to Patent Document 1.

[0122] As shown in FIG. 13, the pixel array unit 220 can be divided into different groups every two columns. In a, b, and c in the figure, group a1 includes column i and column (i+1). Group b1 includes column (i+2) and column (i+3). Group a2 includes column (i+4) and column (i+5). Group b2 includes column (i+6) and column (i+7).

[0123] Also, although the number of columns per group is two, it may be three or more.

[0124] 14 is a diagram for explaining the influence of a prediction error in the first embodiment of the present technology. When the actual illuminance of a pixel is low, as described above, the conversion efficiency is preferably HCG. Therefore, when HCG is predicted, no problem occurs. On the other hand, when LCG is predicted, the read data can be used as valid data, although some noise occurs. Therefore, this error can be evaluated as a non-critical error.

[0125] Furthermore, when the actual illumination of a pixel is medium, there is no problem whether HCG or LCG is predicted.

[0126] Furthermore, when the actual illuminance of the pixel is high, it is preferable that the conversion efficiency is LCG, as described above. Therefore, if LCG is predicted, no problems occur. On the other hand, if HCG is predicted, the read data will become invalid unless measures such as analog repair are taken. Therefore, the logic circuit 300 detects a critical error.

[0127] When a critical error occurs, the logic circuit 300 performs the analog repair described above. Furthermore, the logic circuit 300 supplies the error flag ce[i] to the digital signal processing unit 280, allowing the digital signal processing unit 280 to identify the pixel in which the critical error occurred. The digital signal processing unit 280 then interpolates the digital signal of the pixel in which the error occurred using the digital signals of adjacent pixels of the same color. This interpolation process is called "digital repair." While digital repair has a smaller effect on improving image quality than analog repair, it can be implemented using simpler circuitry than analog repair.

[0128] Furthermore, as described above, even if a critical error occurs in a certain row, analog repair is performed on that pixel. Therefore, even if a critical error occurs in a pixel in the next row, valid data can be obtained from the pixels adjacent to that pixel after analog repair. Therefore, in digital repair, the digital signal processing unit 280 can interpolate the pixel with the critical error using valid data from the adjacent pixels.

[0129] Although the image sensor 200 performs both analog and digital redundancy, the present invention is not limited to this configuration. For example, the logic circuit 300 may perform analog redundancy while the digital signal processing unit 280 does not perform digital redundancy. Alternatively, the logic circuit 300 may not perform analog redundancy, but the digital signal processing unit 280 may perform digital redundancy.

[0130] FIG. 15 is a diagram for explaining digital correction according to the first embodiment of the present technology. The vertical axis in the figure indicates the output value, which is the value of the digital signal. The horizontal axis in the figure indicates the signal amount, which is the amount of signal charge. The thin diagonal lines indicate the characteristics of the digital signal read out by the HCG. The dashed-dotted line indicates the characteristics of the digital signal read out by the LCG without analog rescue. The thick solid line indicates the characteristics of the digital signal read out by the LCG with analog rescue performed. The rough dotted line indicates the characteristics after digital correction.

[0131] As shown in the figure, the LCG has a lower conversion efficiency than the HCG, resulting in a smaller slope. When analog repair is performed, the slope differs slightly compared to when analog repair is not performed, due to the gate capacitance of the conversion efficiency control transistor 234. For this reason, the digital signal processing unit 280 must perform digital correction according to the ratio of these slopes.

[0132] The digital signal processing unit 280 receives the predicted value cg[i] and the error flag ce[i] from the logic circuit 300. The predicted value cg[i] indicates an HCG or an LCG, and the error flag ce[i] indicates whether a critical error has occurred, in other words, whether analog repair is possible.

[0133] When the predicted value cg[i] is an LCG and there is no analog remedy, the digital signal processor 280 calculates the digital gain u ＨＣＧ / u ＬＣＧ２ Digital correction is performed by multiplying the digital signal by u. ＬＣＧ２ is expressed by the following formula: ＬＣＧ２ = 1e － / (fd1+fd2) ...Equation 7

[0134] Furthermore, when the predicted value cg[i] is an LCG and analog relief is provided, the digital signal processing unit 280 calculates the digital gain u ＨＣＧ / u ＬＣＧ Digital correction is performed by multiplying the digital signal by

[0135] When multiplied by the digital gain, the quantization noise of the comparator 273, the DAC 242, and the counter 274 is amplified, but in the case of high illuminance, the optical shot noise is dominant. Therefore, even if the quantization noise generated in the circuit increases, the impact on the overall noise is negligible.

[0136] 16 is a flowchart showing an example of the operation of the image sensor 200 according to the first embodiment of the present technology. This operation is started, for example, when a predetermined application for capturing image data is executed.

[0137] The vertical driving unit 210 selects a row and starts exposure of the selected row (step S901). Then, just before the exposure ends, the column signal processing circuit 260 reads out the reset level with the predicted conversion efficiency (step S902).

[0138] The logic circuit 300 of the i-th column determines whether a critical error has occurred in that column (step S903). If a critical error has occurred (step S903: Yes), the logic circuit 300 performs analog repair to attenuate the amplitude of the potential (step S904). Note that steps S903 and S904 are performed in parallel for each column.

[0139] If no critical error has occurred (step S903: No), or after step S904, the column signal processing circuit 260 reads the signal level at the predicted conversion efficiency (step S905).

[0140] Then, the logic circuit 300 predicts the conversion efficiency of the next row and stores the predicted value (step S906).The digital signal processor 280 performs digital correction using a digital gain (step S907).

[0141] The image sensor 200 determines whether all rows have been read out (step S908). If all rows have not been read out (step S908: No), the image sensor 200 repeats step S901 and subsequent steps.

[0142] On the other hand, if all rows have been read out (step S908: Yes), the digital signal processor 280 performs digital restoration by pixel interpolation (step S909). After step S909, the image sensor 200 executes subsequent processing as necessary and ends the operation for capturing an image.

[0143] When data for a plurality of images is to be read out successively, steps S901 to S909 are repeatedly executed in synchronization with the vertical synchronization signal VSYNC.

[0144] Thus, according to the first embodiment of the present technology, the logic circuit 300 predicts the conversion efficiency of the next (j+1)-row group based on the judgment result is_dim[i] of the j-row group, thereby enabling the image quality to be improved through appropriate prediction.

[0145] [First Modification] In the first embodiment described above, the image sensor 200 reads out one row at a time, but it is preferable to further improve the readout speed. The image sensor 200 in this first modification of the first embodiment differs from the first embodiment in that it reads out two rows simultaneously.

[0146] 17 is a block diagram showing an example configuration of an image sensor 200 according to a first modified example of the first embodiment of the present technology. The image sensor 200 according to the first modified example of the first embodiment differs from the first embodiment in that it further includes a horizontal drive unit 255 and a column signal processing circuit 265.

[0147] The circuit configuration of the horizontal driver 255 is the same as that of the horizontal driver 250, except that the rows that it drives are different from those of the horizontal driver 250. Furthermore, the vertical driver 210 can drive two rows simultaneously.

[0148] The circuit configuration of the column signal processing circuit 265 is the same as that of the column signal processing circuit 260, except that the row that is read out differs from that of the column signal processing circuit 260. These circuits enable two rows to be read out simultaneously.

[0149] 18 is a diagram illustrating an example of a prediction algorithm in a first modified example of the first embodiment of the present technology. Group a1 includes columns i and (i+2) in odd-numbered rows. Group b1 includes columns i and (i+2) in even-numbered rows. Group c1 includes columns (i+1) and (i+3) in odd-numbered rows. Group d1 includes columns (i+1) and (i+3) in even-numbered rows.

[0150] Group a2 includes columns (i+4) and (i+6) in odd-numbered rows. Group b2 includes columns (i+4) and (i+6) in even-numbered rows. Group c2 includes columns (i+5) and (i+7) in odd-numbered rows. Group d2 includes columns (i+5) and (i+7) in even-numbered rows.

[0151] As shown in FIG. 1A, the vertical drive unit 210 selects and drives rows j and (j+1) simultaneously, then as shown in FIG. 1B, the vertical drive unit 210 selects and drives rows j+2 and (j+3) simultaneously, and then as shown in FIG. 1C, the vertical drive unit 210 selects and drives rows j+4 and (j+5) simultaneously.

[0152] The logic circuit 300 of group a1 predicts the conversion efficiency of group a1 in the (j+2)th row based on the determination result of group a1 in the jth row. The logic circuit 300 of group b1 predicts the conversion efficiency of group b1 in the (j+3)th row based on the determination result of group b1 in the (j+1)th row. The logic circuit 300 of group c1 predicts the conversion efficiency of group a1 in the (j+2)th row based on the determination result of group c1 in the jth row. The logic circuit 300 of group d1 predicts the conversion efficiency of group b1 in the (j+3)th row based on the determination result of group d1 in the (j+1)th row.

[0153] The conversion efficiencies of the (j+4)th row and onward are predicted using the same algorithm. The conversion efficiencies of the groups a2, b2, c2, and d2 are also predicted using the same algorithm.

[0154] 19A, 19B, and 19C, the pixel array section 220 can be divided into different groups for each column. Also, the pixel array section 220 can be divided into different groups for every two columns.

[0155] 20A, 20B, and 20C, the pixel array unit 220 can also be divided into different groups for each color. For example, the pixel array unit 220 has R (Red) pixels, Gr pixels, B (Blue) pixels, and Gb pixels arranged in a Bayer array. The R pixels are pixels that receive red visible light, and the Gr and Gb pixels are pixels that receive green visible light. The B pixels are pixels that receive blue visible light.

[0156] Group R1 includes R pixels in columns i and (i+2). Group Gb1 includes Gb pixels in columns i and (i+2). Group Gr1 includes Gr pixels in columns (i+1) and (i+3). Group B1 includes B pixels in columns (i+1) and (i+3).

[0157] Group R2 includes R pixels in columns (i+4) and (i+6). Group Gb2 includes Gb pixels in columns (i+4) and (i+6). Group Gr2 includes Gr pixels in columns (i+5) and (i+7). Group B2 includes B pixels in columns (i+5) and (i+7).

[0158] As described above, according to the first modification of the first embodiment of the present technology, the column signal processing circuits 260 and 265 simultaneously read out two rows, thereby improving the readout speed.

[0159] [Second Modification] In the first embodiment described above, a floating diffusion layer 236 and the like are provided for each pixel, but it is preferable to reduce the number of elements per pixel. The image sensor 200 in this second modification of the first embodiment differs from the first embodiment in that two pixels share the floating diffusion layer 236.

[0160] 21 is a circuit diagram showing a configuration example of an FD sharing block 239 in a second modified example of the first embodiment of the present technology. A predetermined number of FD sharing blocks 239 are arranged in a pixel array unit 220 in the second modified example of the first embodiment.

[0161] The FD shared block 239 includes photodiodes 231-1 and 231-2, and transfer transistors 232-1 and 232-2. The FD shared block 239 further includes a reset transistor 233, a conversion efficiency control transistor 234, a capacitance element 235, a floating diffusion layer 236, an amplification transistor 237, and a selection transistor 238.

[0162] The transfer transistor 232-1 transfers charges from the photodiode 231-1 to the floating diffusion layer 236 in accordance with a transfer signal TRG1. The transfer transistor 232-2 transfers charges from the photodiode 231-2 to the floating diffusion layer 236 in accordance with a transfer signal TRG2. The FD sharing block 239 functions as two pixels that share the floating diffusion layer 236. The two pixels that share the floating diffusion layer 236 are an example of the first and second pixels described in the claims.

[0163] 22 is a diagram illustrating an example of a prediction algorithm according to a second modification of the first embodiment of the present technology. For example, two adjacent pixels arranged in the vertical direction share a floating diffusion layer 236. The portion surrounded by a bold frame in the figure indicates an FD sharing block 239.

[0164] As shown in a, b, and c in the figure, for example, the pixels are divided into different groups for each column. It is also possible to divide the pixels into different groups every two columns. It is also possible to divide the pixels into different groups for each color.

[0165] As described above, according to the second modification of the first embodiment of the present technology, the floating diffusion layer 236 is shared by two pixels, so that the number of elements per pixel can be reduced.

[0166] [Third Modification] In the second modification of the first embodiment described above, two pixels share the floating diffusion layer 236. However, the first modification in which two rows are read out simultaneously can also be applied to this second modification. The image sensor 200 in this third modification of the first embodiment differs from the second modification of the first embodiment in that two rows are read out simultaneously.

[0167] 23 is a diagram illustrating an example of a prediction algorithm in a third modified example of the first embodiment of the present technology. As illustrated in a, b, and c in the figure, each column is divided into different groups, and two rows are read out at a time. Note that each two columns may also be divided into different groups.

[0168] Also, as shown in a, b, and c in FIG. 24, the colors can be divided into different groups.

[0169] As described above, according to the third modified example of the first embodiment of the present technology, the floating diffusion layer 236 is shared by two pixels and two rows are read out simultaneously, so that the number of elements per pixel can be reduced and the readout speed can be improved.

[0170] [Fourth Modification] In the third modification of the first embodiment described above, two pixels share the floating diffusion layer 236, but the number of pixels that share is not limited to two. The image sensor 200 in this fourth modification of the first embodiment differs from the third modification of the first embodiment in that four pixels share the floating diffusion layer 236.

[0171] 25 is a circuit diagram showing a configuration example of an FD shared block 239 in a fourth modified example of the first embodiment of the present technology. The FD shared block 239 in the fourth modified example of the first embodiment differs from the third modified example of the first embodiment in that it further includes photodiodes 231-3 and 231-4 and transfer transistors 232-3 and 232-4.

[0172] The transfer transistor 232-3 transfers charges from the photodiode 231-3 to the floating diffusion layer 236 in accordance with a transfer signal TRG3. The transfer transistor 232-4 transfers charges from the photodiode 231-4 to the floating diffusion layer 236 in accordance with a transfer signal TRG4. The FD sharing block 239 functions as four pixels that share the floating diffusion layer 236.

[0173] The number of pixels sharing the floating diffusion layer 236 is not limited to two or four pixels, but may be eight pixels or the like.

[0174] 26 is a diagram illustrating an example of a prediction algorithm according to a third modified example of the first embodiment of the present technology. As illustrated in a, b, and c in the figure, four pixels arranged in two rows and two columns supply a floating diffusion layer 236.

[0175] For example, as illustrated in a in the same figure, the vertical drive unit 210 simultaneously drives the Gb pixel in row j, the Gr pixel in row (j+1), the B pixel in row (j+4), and the R pixel in row (j+5).

[0176] As shown in FIG. 1B, the vertical drive unit 210 simultaneously drives the Gb pixel in row (j+2), the Gr pixel in row (j+3), the B pixel in row (j+6), and the R pixel in row (j+7). Based on the results of the determination of the adjacent R pixels read out in FIG. 1A, the conversion efficiencies of the adjacent R pixels in FIG. 1B are predicted. The same applies to the Gr, Gb, and B pixels.

[0177] Then, as illustrated in c in the figure, the vertical drive unit 210 simultaneously drives the Gb pixel in row (j+4), the Gr pixel in row (j+5), the B pixel in row (j+8), and the R pixel in row (j+9). The same applies to subsequent readouts.

[0178] As described above, according to the fourth modification of the first embodiment of the present technology, the floating diffusion layer 236 is shared by four pixels, and the number of elements per pixel can be further reduced.

[0179] 2. Second Embodiment In the first embodiment described above, the logic circuit 300 predicts the conversion efficiency for each row, but in this configuration, a latch circuit and a logic gate are required for each column to predict the conversion efficiency. The image sensor 200 in this second embodiment differs from the first embodiment in that the conversion efficiency is controlled according to the illuminance for each row.

[0180] 27 is a circuit diagram showing a configuration example of a logic circuit 300 according to the second embodiment of the present technology. The logic circuit 300 according to the second embodiment includes a determination circuit 310 and a conversion efficiency control circuit 340. The determination circuit 310 includes a latch circuit 311 and an inverter 321. The conversion efficiency control circuit 340 includes an AND gate 341 and an OR gate 344. Note that a circuit including the conversion efficiency control circuits 340 for each column is an example of a conversion efficiency control unit described in the claims.

[0181] In the second embodiment, the inverter 321 inverts the latch signal Lat1 and outputs it as the error flag ce[i] to the AND gate 341 and the digital signal processing unit 280. Similarly to the first embodiment, the AND gate 341 outputs the logical product of the error flag ce[i] and the control signal fdg_c2 to the OR gate 344. The OR gate 344 outputs the logical sum of the control signal fdg_c1 and the output signal of the AND gate 341 to the horizontal drive unit 250 as the control signal FDG[i].

[0182] FIG. 28 is a timing chart showing an example of a readout operation of the image sensor 200 according to the second embodiment of the present technology.

[0183] The vertical drive unit 210 supplies a reset signal RST to the selected row over a pulse period from timing T11. In the initial state, the conversion efficiency of all pixels is set to HCG.

[0184] Furthermore, the timing control circuit 241 supplies a high-level control signal fdg_c1 to the logic circuit 300 of each column over the pulse period from timing T11. In response to this signal, the logic circuit 300 supplies a high-level FDG[i] to the i-th column over the pulse period from timing T11.

[0185] Then, the timing control circuit 241 supplies a high-level enable signal en1 to the logic circuit 300 of each column over the period from just before timing T15 until timing T15. At this time, if Δvsl is equal to or greater than ΔRmp1, the logic circuit 300 determines that the illuminance is high, not below the determination threshold corresponding to TH2. On the other hand, if Δvsl is smaller than ΔRmp1, the logic circuit 300 determines that the illuminance is low, below the determination threshold. In the figure, it is assumed that the illuminance is high.

[0186] The timing control circuit 241 then supplies a high-level control signal fdg_c2 to the logic circuit 300 of each column over the period from timing T16 to timing T17. During this period, the logic circuit 300 of the i-th column determines that the illuminance is high, and therefore supplies a high-level control signal FDG[i] to the i-th column. As a result, the i-th column is temporarily controlled by LCG, and the amplitude of the potential of the vertical signal line 229 is attenuated by the ratio LCG / HCG.

[0187] On the other hand, if it is determined that the illuminance is low, the logic circuit 300 of the i-th column keeps the control signal FDG[i] at a low level during the period from timing T16 to timing T17.

[0188] Furthermore, the vertical drive unit 210 supplies a high-level transfer signal TRG to the selected row during the period from timing T16 to immediately before timing T17, as in the first embodiment, to completely transfer the charges.

[0189] The ramp signal Rmp gradually drops over the signal level readout period from timing T17 onwards, and the signal level is read out during this period.

[0190] In the first embodiment, the logic circuit 300 predicts the conversion efficiency and performs analog repair by attenuating the amplitude of the potential only in the case of a prediction error.

[0191] In contrast, in the logic circuit 300 of the second embodiment, the determination circuit 310 determines whether the illuminance is high enough not to fall below a determination threshold value corresponding to TH2. If the illuminance is determined to be high, the conversion efficiency control circuit 340 performs analog redundancy by setting the control signal FDG[i] to a high level over the pulse period. With this control, analog redundancy is performed on all pixels with high illuminance, which may result in a decrease in image quality compared to the first embodiment, but the control is simplified and the circuit scale can be reduced.

[0192] As described above, according to the second embodiment of the present technology, the logic circuit 300 sets the control signal FDG[i] to a high level over the pulse period when the illuminance is high, thereby enabling a reduction in circuit size compared to the first embodiment.

[0193] 3. Third Embodiment In the above-described first embodiment, the logic circuit 300 predicts the conversion efficiency for each row, but it is preferable to reduce the circuit scale of the logic circuit 300. The image sensor 200 in this third embodiment differs from the first embodiment in that the digital signal processing unit 280 predicts the conversion efficiency for each frame.

[0194] 29 is a circuit diagram showing a configuration example of a logic circuit 300 according to the third embodiment of the present technology. The logic circuit 300 according to the third embodiment includes a determination circuit 310 and a conversion efficiency control circuit 340. The determination circuit 310 also includes a latch circuit 311, an inverter 321, and an AND gate 331. The AND gate 331 receives an inverted signal from the inverter 321 and a predicted value cg[i] from the digital signal processing unit 280. The AND gate 331 supplies the logical product thereof as an error flag ce[i] to the digital signal processing unit 280 and the conversion efficiency control circuit 340.

[0195] 30 is a block diagram showing an example configuration of a digital signal processing unit 280 according to the third embodiment of the present technology. The digital signal processing unit 280 includes a digital correction unit 281, an image processing unit 282, an illuminance determination unit 283, a data buffer 284, and a prediction control unit 285.

[0196] The digital correction unit 281 performs digital correction in the same manner as in the first embodiment. The digital correction unit 281 receives as input the predicted value cg[i] from the prediction control unit 285, the error flag ce[i] from the logic circuit 300, and the digital signal Dout[i] from the ADC 275. The digital correction unit 281 performs correction using digital gain on the digital signal Dout[i] based on the predicted value cg[i] and the error flag ce[i], and supplies the corrected digital signal Dout′[i] to the image processing unit 282 and the illuminance determination unit 283.

[0197] The image processing unit 282 performs various image processing, including digital redundancy as necessary, on the frame in which the digital signal Dout'[i] is arranged, and outputs the processed frame.

[0198] The illuminance determination unit 283 determines whether the illuminance of each pixel is below a determination threshold based on whether the digital signal Dout'[i] is equal to or greater than a predetermined value. The illuminance determination unit 283 stores the determination results for one frame in the data buffer 284.

[0199] The prediction control unit 285 refers to the data buffer 284 each time a frame is generated, and predicts the conversion efficiency of the current frame for each pixel based on the determination result of the previous frame. If the determination result for column i and row j of the previous frame is high illuminance, LCG is predicted as the conversion efficiency for column i and row j of the current frame. On the other hand, if the determination result for column i and row j of the previous frame is low illuminance, HCG is predicted as the conversion efficiency for column i and row j of the current frame. The prediction control unit 285 generates a predicted value cg[i] and supplies it to the logic circuit 300 and the digital correction unit 281.

[0200] Note that, although the determination results for one frame are stored in the data buffer 284, this configuration is not limiting. For example, the prediction control unit 285 can perform a logical operation similar to the AND gate 341 in FIG. 5 on the determination results for each M (M is an integer smaller than the total number of pixels) pixels to generate the determination result grp[i]. In this case, it is sufficient to store grp[i] for each group in the data buffer 284, so the size of the data buffer 284 can be reduced. However, it should be noted that this is a trade-off with prediction accuracy.

[0201] Furthermore, the circuit made up of the digital signal processing unit 280 and the logic circuit 300 for each column is an example of a control unit set forth in the claims.

[0202] 31 is a diagram showing an example of a prediction algorithm according to the third embodiment of the present technology. The column circuit 270 converts the pixel signal of each pixel in the frame F1 into a digital signal, and also generates an error flag ce[i] and supplies it to the digital signal processing unit 280 together with the digital signal.

[0203] The digital signal processor 280 corrects the digital signal and determines the illuminance for each pixel based on the corrected signal. Then, based on the determination result for frame F1, the conversion efficiency for the next frame F2 is predicted. For example, if the pixel at column i, row j in frame F1 is determined to have high illuminance, LCG is predicted as the conversion efficiency for column i, row j in frame F2.

[0204] As shown in the figure, in a configuration in which the digital signal processing unit 280 predicts the conversion efficiency on a frame-by-frame basis, the logic circuit 300 does not need to predict the conversion efficiency, which makes it possible to reduce the circuit size of the logic circuit 300. The method of the third embodiment is particularly effective when there is little change in scene movement or illuminance between frames.

[0205] It should be noted that the first, second, third and fourth modifications of the first embodiment can be applied to the third embodiment.

[0206] As described above, according to the third embodiment of the present technology, the digital signal processing unit 280 predicts the conversion efficiency of the current frame based on the determination result of the previous frame, so that the circuit scale of the logic circuit 300 can be reduced.

[0207] 4. Application Examples to Mobile Bodies The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of mobile body, such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, or a robot.

[0208] FIG. 32 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile object control system to which the technology according to the present disclosure can be applied.

[0209] The vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001. In the example shown in Fig. 32, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside-vehicle information detection unit 12030, an inside-vehicle information detection unit 12040, and an integrated control unit 12050. Also shown as functional components of the integrated control unit 12050 are a microcomputer 12051, an audio / video output unit 12052, and an in-vehicle network I / F (Interface) 12053.

[0210] The drivetrain control unit 12010 controls the operation of devices related to the drivetrain of the vehicle in accordance with various programs. For example, the drivetrain control unit 12010 functions as a control device for a drive force generating device for generating a drive force of the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking device for generating a braking force of the vehicle.

[0211] The body system control unit 12020 controls the operation of various devices equipped in the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as headlamps, backup lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves transmitted from a portable device that serves as a key or signals from various switches can be input to the body system control unit 12020. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock device, power window device, lamps, etc.

[0212] The outside-vehicle information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the outside-vehicle information detection unit 12030. The outside-vehicle information detection unit 12030 causes the imaging unit 12031 to capture images outside the vehicle and receives the captured images. The outside-vehicle information detection unit 12030 may perform object detection processing or distance detection processing for people, cars, obstacles, signs, characters on the road surface, etc. based on the received images.

[0213] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output the electrical signal as an image or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0214] The in-vehicle information detection unit 12040 detects information inside the vehicle. For example, a driver state detection unit 12041 that detects the state of the driver is connected to the in-vehicle information detection unit 12040. The driver state detection unit 12041 includes, for example, a camera that captures an image of the driver, and the in-vehicle information detection unit 12040 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041, or may determine whether the driver is dozing off.

[0215] The microcomputer 12051 can calculate control target values ​​for the driving force generating device, steering mechanism, or braking device based on the information inside and outside the vehicle acquired by the outside-vehicle information detection unit 12030 or the inside-vehicle information detection unit 12040, and output control commands to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing the functions of an ADAS (Advanced Driver Assistance System), including vehicle collision avoidance or impact mitigation, following driving based on the distance between vehicles, maintaining vehicle speed, vehicle collision warning, vehicle lane departure warning, etc.

[0216] In addition, the microcomputer 12051 can perform cooperative control for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on driver operation, by controlling the driving force generating device, steering mechanism, braking device, etc. based on information about the surroundings of the vehicle obtained by the outside vehicle information detection unit 12030 or the inside vehicle information detection unit 12040.

[0217] Furthermore, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information outside the vehicle acquired by the outside information detection unit 12030. For example, the microcomputer 12051 can control the headlamps according to the position of a preceding vehicle or an oncoming vehicle detected by the outside information detection unit 12030, and perform cooperative control aimed at preventing glare, such as switching from high beams to low beams.

[0218] The audio / video output unit 12052 transmits at least one of audio and video output signals to an output device capable of visually or audibly notifying the passengers of the vehicle or the outside of the vehicle of information. In the example of Fig. 32, the output devices are exemplified by an audio speaker 12061, a display unit 12062, and an instrument panel 12063. The display unit 12062 may include, for example, at least one of an on-board display and a head-up display.

[0219] FIG. 33 is a diagram showing an example of the installation position of the imaging unit 12031.

[0220] In FIG. 33, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0221] The imaging units 12101, 12102, 12103, 12104, and 12105 are provided, for example, at positions such as the front nose, side mirrors, rear bumper, back door, and the top of the windshield inside the vehicle cabin of the vehicle 12100. The imaging unit 12101 provided on the front nose and the imaging unit 12105 provided on the top of the windshield inside the vehicle cabin mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 provided on the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 provided on the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 provided on the top of the windshield inside the vehicle cabin is mainly used to detect preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, etc.

[0222] 33 shows an example of the imaging ranges of the imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of the imaging unit 12104 provided on the rear bumper or back door. For example, by overlaying the image data captured by the imaging units 12101 to 12104, an overhead image of the vehicle 12100 viewed from above can be obtained.

[0223] At least one of the image capturing units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the image capturing units 12101 to 12104 may be a stereo camera made up of multiple image capturing elements, or may be an image capturing element having pixels for phase difference detection.

[0224] For example, based on the distance information obtained from the imaging units 12101 to 12104, the microcomputer 12051 can calculate the distance to each three-dimensional object within the imaging ranges 12111 to 12114 and the change in this distance over time (relative speed with respect to the vehicle 12100), thereby extracting as a preceding vehicle, in particular, the three-dimensional object that is the closest three-dimensional object on the path of the vehicle 12100 and traveling in approximately the same direction as the vehicle 12100 at a predetermined speed (e.g., 0 km / h or higher). Furthermore, the microcomputer 12051 can set a vehicle-to-vehicle distance to be maintained in advance in front of the preceding vehicle, and perform automatic braking control (including follow-up stop control), automatic acceleration control (including follow-up start control), etc. In this way, cooperative control can be performed for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on driver operation.

[0225] For example, the microcomputer 12051 classifies and extracts three-dimensional object data regarding three-dimensional objects into two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on distance information obtained from the imaging units 12101 to 12104, and can use the data for automatic obstacle avoidance. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines a collision risk that indicates the risk of collision with each obstacle, and when the collision risk is equal to or greater than a set value and a collision is possible, the microcomputer 12051 can provide driving assistance for collision avoidance by outputting an alarm to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or avoidance steering via the drive system control unit 12010.

[0226] At least one of the image capturing units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether a pedestrian is present in the images captured by the image capturing units 12101 to 12104. Such pedestrian recognition is performed, for example, by extracting feature points from the images captured by the image capturing units 12101 to 12104 as infrared cameras and performing pattern matching on a series of feature points that indicate the outline of an object to determine whether the object is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the image capturing units 12101 to 12104 and recognizes the pedestrian, the audio / image output unit 12052 controls the display unit 12062 to superimpose a rectangular outline on the recognized pedestrian for emphasis. The audio / image output unit 12052 may also control the display unit 12062 to display an icon or the like indicating the pedestrian at a desired position.

[0227] An example of a vehicle control system to which the technology according to the present disclosure can be applied has been described above. The technology according to the present disclosure can be applied to, for example, the image capturing unit 12031 of the above-described configuration. Specifically, the image capturing device 100 of FIG. 1 can be applied to the image capturing unit 12031. By applying the technology according to the present disclosure to the image capturing unit 12031, it is possible to obtain a captured image that is easier to see, thereby reducing driver fatigue.

[0228] Note that the above-described embodiment shows an example for realizing the present technology, and the matters in the embodiment and the matters specifying the invention in the claims correspond to each other. Similarly, the matters specifying the invention in the claims and the matters in the embodiment of the present technology having the same name correspond to each other. However, the present technology is not limited to the embodiment, and can be realized by applying various modifications to the embodiment within the scope of the gist thereof.

[0229] The effects described in this specification are merely examples and are not limiting, and other effects may also be obtained.

[0230] The present technology may also be configured as follows: (1) An image sensor comprising: a pixel array unit in which a predetermined number of pixels are arranged in a two-dimensional lattice pattern; an analog-to-digital conversion unit that converts pixel signals of the predetermined number of pixels into digital signals; and a control unit that calculates, for each pixel, a predicted value of conversion efficiency when converting electric charge into a voltage in the pixel and controls the conversion efficiency to the predicted value. (2) The image sensor according to (1), further comprising: a vertical drive unit that sequentially selects and drives rows in the pixel array unit, wherein the control unit comprises: a determination unit that determines, each time the row is selected, whether illuminance for each pixel is below a predetermined determination threshold and outputs the determination result; and a prediction control unit that calculates the predicted value for the next row based on the determination result each time the row is selected, and controls the conversion efficiency to the predicted value. (3) The image sensor according to (2), wherein the determination unit further determines for each pixel whether a critical error, in which the predicted value is an incorrect value, has occurred, and the prediction control unit controls the conversion efficiency to a value lower than a predetermined value for a predetermined pulse period when the critical error has occurred. (4) The image sensor according to (2) or (3), further comprising a digital signal processing unit that, when the critical error has occurred, performs a predetermined interpolation process on the digital signal of the pixel in which the critical error has occurred. (5) The image sensor according to any of (2) to (4), wherein the vertical drive unit simultaneously drives two rows in the pixel array unit. (6) The image sensor according to any of (2) to (5), wherein the predetermined number of pixels include first and second pixels that share a floating diffusion layer. (7) The image sensor according to any of (2) to (6), wherein the pixel array unit is divided into a predetermined number of groups, each including a plurality of pixels, and the prediction control unit calculates the predicted value for each group by performing a predetermined logical operation on the determination result within that group. (8) The image sensor according to (7), wherein the pixel array section is divided into different groups for each column. (9) The image sensor according to (7), wherein the pixel array section is divided into different groups for every two columns.(10) The image sensor according to (7), wherein the pixel array unit is divided into different groups for each color. (11) The image sensor according to (1), wherein the control unit comprises: an illuminance determination unit that determines, for each pixel, whether or not illuminance is below a predetermined threshold based on the digital signal and outputs the determination result; a data buffer that stores each of the determination results; a prediction control unit that calculates the predicted value for the next frame based on the determination result each time a frame in which the digital signals are arranged is generated; and a logic circuit that controls the conversion efficiency to the predicted value. (12) An image sensor comprising: a pixel array unit in which a predetermined number of pixels are arranged in a two-dimensional lattice pattern; an analog-to-digital conversion unit that converts each pixel signal of the predetermined number of pixels into a digital signal; a determination unit that determines, for each pixel, whether or not illuminance is below a predetermined determination threshold based on a reset level when the pixel is initialized; and a control unit that controls the conversion efficiency when converting charge to voltage in the pixel to a value lower than a predetermined value for a predetermined pulse period when the illuminance is not below the determination threshold. (13) An imaging device comprising: a pixel array unit in which a predetermined number of pixels are arranged in a two-dimensional lattice pattern; an analog-to-digital conversion unit that converts each pixel signal of the predetermined number of pixels into a digital signal; a control unit that obtains, for each pixel, a predicted value of conversion efficiency when converting electric charge into a voltage in the pixel and controls the conversion efficiency to the predicted value; and a digital signal processing unit that performs predetermined signal processing on the digital signal. (14) A control method for an image sensor comprising: an analog-to-digital conversion procedure that converts each pixel signal of a predetermined number of pixels arranged in a two-dimensional lattice pattern in the pixel array unit into a digital signal; and a control procedure that obtains, for each pixel, a predicted value of conversion efficiency when converting electric charge into a voltage in the pixel and controls the conversion efficiency to the predicted value.

[0231] 100 Imaging device 110 Imaging lens 120 Storage unit 130 Imaging control unit 200 Image sensor 210 Vertical drive unit 211, 212, 213, 251 Driver 220 Pixel array unit 230 Pixel 231, 231-1, 231-2, 231-3, 231-4 Photodiode 232, 232-1, 232-2, 232-3, 232-4 Transfer transistor 233 Reset transistor 234 Conversion efficiency control transistor 235 Capacitor element 236 Floating diffusion layer 237 Amplification transistor 238 Selection transistor 239 FD sharing block 241 Timing control circuit 242 DAC 250, 255 Horizontal drive unit 254 Load MOS circuit 260, 265 Column signal processing circuit 270 Column circuit 271, 272 Auto-zero switch 273 Comparator 274 Counter 275 ADC 280 Digital signal processing unit 281 Digital correction unit 282 Image processing unit 283 Illuminance determination unit 284 Data buffer 285 Prediction control unit 300 Logic circuit 310 Determination circuit 311, 312, 313, 314 Latch circuit 320 Prediction control circuit 321, 322, 323, 343 Inverter 331, 332, 333, 334, 341, 342 AND (logical product) gate 340 Conversion efficiency control circuit 344, 351 OR (logical sum) gate 12031 Imaging unit

Claims

1. An image sensor comprising: a pixel array section in which a predetermined number of pixels are arranged in a two-dimensional grid; an analog-to-digital conversion section that converts the pixel signals of each of the predetermined number of pixels into digital signals; and a control section that obtains, for each pixel, a predicted value of conversion efficiency when converting electric charge into voltage within the pixel and controls the conversion efficiency to the predicted value.

2. An image sensor as described in claim 1, further comprising: a vertical drive unit that sequentially selects and drives rows in the pixel array unit, wherein the control unit comprises: a determination unit that determines whether the illuminance of each pixel is below a predetermined determination threshold each time the row is selected and outputs the determination result; and a prediction control unit that determines the predicted value for the next row based on the determination result each time the row is selected, and controls the conversion efficiency to the predicted value.

3. The image sensor according to claim 2, wherein the determination unit further determines for each pixel whether a critical error, in which the predicted value is an incorrect value, has occurred, and the prediction control unit controls the conversion efficiency to a value lower than a predetermined value for a predetermined pulse period if the critical error has occurred.

4. The image sensor according to claim 2, further comprising a digital signal processing unit that, when the critical error occurs, performs a predetermined interpolation process on the digital signal of the pixel in which the critical error occurs.

5. The image sensor according to claim 2, wherein the vertical drive section simultaneously drives two rows in the pixel array section.

6. The image sensor according to claim 2, wherein the predetermined number of pixels includes first and second pixels that share a floating diffusion layer.

7. The image sensor according to claim 2, wherein the pixel array section is divided into a predetermined number of groups, each group including a plurality of pixels, and the prediction control section obtains the predicted value for each group by performing a predetermined logical operation on the judgment results within that group.

8. The image sensor according to claim 7, wherein the pixel array section is divided into different groups for each column.

9. The image sensor according to claim 7, wherein the pixel array section is divided into different groups every two columns.

10. The image sensor according to claim 7, wherein the pixel array section is divided into different groups for each color.

11. The image sensor of claim 1, wherein the control unit comprises: an illuminance determination unit that determines for each pixel whether or not the illuminance is below a predetermined threshold based on the digital signal and outputs the determination result; a data buffer that stores each of the determination results; a prediction control unit that calculates the predicted value for the next frame based on the determination result each time a frame in which the digital signals are arranged is generated; and a logic circuit that controls the conversion efficiency to the predicted value.

12. An image sensor comprising: a pixel array section in which a predetermined number of pixels are arranged in a two-dimensional lattice; an analog-to-digital conversion section that converts each pixel signal of the predetermined number of pixels into a digital signal; a determination section that determines for each pixel whether or not the illuminance is below a predetermined determination threshold based on a reset level when the pixel is initialized; and a control section that, if the illuminance is not below the determination threshold, controls the conversion efficiency when converting charge to voltage in the pixel to a value lower than a predetermined value over a predetermined pulse period.

13. An imaging device comprising: a pixel array section in which a predetermined number of pixels are arranged in a two-dimensional lattice; an analog-to-digital conversion section that converts each pixel signal of the predetermined number of pixels into a digital signal; a control section that obtains, for each pixel, a predicted value of conversion efficiency when converting electric charge into voltage within the pixel and controls the conversion efficiency to the predicted value; and a digital signal processing section that performs predetermined signal processing on the digital signal.

14. A control method for an image sensor comprising: an analog-to-digital conversion procedure for converting into a digital signal each pixel signal of a predetermined number of pixels arranged in a two-dimensional lattice pattern within a pixel array; and a control procedure for calculating, for each pixel, a predicted value of conversion efficiency when converting electric charge into voltage within the pixel, and controlling the conversion efficiency to the predicted value.

Images