Imaging system, image sensor, and control method for imaging system

Abstract

The present invention sets power consumption and correction accuracy to appropriate values according to an operating status in an imaging system that corrects a black level.　A dark current amount generated in a pixel is estimated on the basis of an exposure time EX set in an image sensor and a temperature T measured at a timing synchronized with a vertical synchronization signal, and an operation of black level correction is changed according to the estimated value. The present invention operates in a mode for reducing power consumption in a situation where the dark current amount is small, and operates in a mode for correcting a temporal and spatial temperature difference in a situation where the dark current amount is large.

Description

Imaging system, image sensor, and control method for imaging system

[0001] The present technology relates to an imaging system. More specifically, it relates to an imaging system that performs black level correction, an image sensor, and a control method for an imaging system.

[0002] Conventionally, it has been known that in image data output from an image sensor (in other words, frames), an optical black level fluctuates due to noise such as dark current. Accordingly, image sensors are provided with a function of correcting the black level for each frame based on the average value of signals in a light-shielded region (see, for example, Patent Document 1). Fluctuations in black level occur due to various factors such as pixel characteristics, readout order, and circuit configuration, and these are comprehensively handled and corrected.

[0003] US Patent Application Publication No. 2005 / 0237402

[0004] In the above-mentioned conventional technology, image quality is improved by correcting the black level for each frame based on an average value. However, depending on the amount of dark current-induced noise per frame, correction using the average value may be inappropriate in some situations. In situations where the amount of dark current is small, it can be considered that the average value does not change per frame, so calculating the average value for each frame results in surplus power consumption. Furthermore, in situations where the amount of dark current is large, temporal and spatial differences in the amount of dark current generated are not taken into account, resulting in insufficient correction accuracy. By individually handling dark current in black level correction, there is room for optimizing power consumption and correction accuracy according to operating conditions as an image sensor system.

[0005] The present technology has been developed in view of such circumstances, and an object thereof is to adjust power consumption and correction accuracy to appropriate values according to operating conditions in an imaging system that performs black level correction.

[0006] This technology was developed to solve the aforementioned problems, and its first aspect is an imaging system comprising a pixel array section in which light-shielded OPB (optical black) pixels and light-shielded photosensitive pixels are arranged, and a dark current estimation section that estimates the amount of dark current in the pixel array section based on the exposure time and the temperature measured at a timing synchronized with a predetermined vertical synchronization signal, and outputs it as an estimated value, and a control method for the imaging system. This results in the power consumption and correction accuracy being controlled to appropriate values ​​according to the operating conditions.

[0007] Furthermore, the first aspect may further include a conversion unit that converts each of the temperatures measured at multiple timings during the exposure period into a dark current amount and outputs it as a temperature measurement point dark current amount, a dark current amount correction unit that corrects the pixel signals from the OPB pixels and the photosensitive pixels using the temperature measurement point dark current amount, and a mode determination unit that transitions to a predetermined normal mode if the estimated value is within a range from a predetermined first threshold to a predetermined second threshold, and transitions to a low-power mode with lower power consumption than the normal mode if the estimated value is smaller than the first threshold, and the dark current correction unit may further include an average value calculation unit that calculates the average value of the pixel signals from the OPB pixels when transitioning from the low-power mode to another mode, an average value holding unit that holds the average value, and a clamping processing unit that corrects the pixel signals from the photosensitive pixels based on the average value calculated by the average value calculation unit when transitioning to another mode, and corrects the pixel signals based on the average value held by the average value holding unit when transitioning to the low-power mode. This results in the power consumption and correction accuracy being optimized according to the operating conditions.

[0008] Furthermore, in this first aspect, an analog clamp circuit may be provided to perform analog clamping when switching from the low-power mode to other modes. This has the effect of suppressing the deterioration of image quality when returning from the low-power mode.

[0009] Furthermore, in this first aspect, the low-power mode may be the default mode. This results in reduced power consumption.

[0010] Furthermore, in this first aspect, the average value holding unit may hold the average value for each driving mode of the image sensor. This eliminates the need to read out the pixel signals of the OPB pixels or calculate the average value even when the driving mode of the image sensor is switched, thus maintaining the low-power mode and extending the low-power consumption effect.

[0011] Furthermore, in this first aspect, the low-power mode includes a first low-power mode and a second low-power mode with different power consumption, and in the second low-power mode, only a portion of a predetermined number of OPB rows in which the OPB pixels are arranged may be read. This results in the black level correction operation being able to achieve gradual power reduction.

[0012] Furthermore, in this first aspect, the system may further include a sensor signal processing unit that determines whether the scene has changed based on a signal from a Time of Flight (ToF) sensor or an Event-based Vision Sensor (EVS), and supplies the determination result to the mode determination unit. This provides the functionality to switch modes using signals from the ToF sensor or EVS.

[0013] Furthermore, in this first aspect, the photosensitive pixel includes a phase difference pixel for detecting a phase difference, and the mode determination unit determines whether the scene has changed based on the amount of defocus obtained by converting the phase difference signal from the phase difference pixel, and if the scene has changed, it may switch from the low-power mode to the normal mode. This provides the effect of being able to switch modes using a phase difference signal.

[0014] Furthermore, in this first aspect, if the estimated value is greater than a predetermined second threshold, the system may further include a conversion unit that converts each of the temperatures measured at multiple timings during the exposure period into dark current amounts and outputs them as temperature measurement point dark current amounts; a dark current amount correction unit that corrects the pixel signals from the OPB pixels and the photosensitive pixels using the temperature measurement point dark current amounts; and an average value calculation unit that calculates the average value of the corrected OPB pixel signals. This provides the effect of correcting the generation of dark current due to temperature fluctuations during exposure.

[0015] Furthermore, in this first aspect, the device may further include a plurality of thermometers arranged directly below the photosensitive area of ​​the pixel array with the light-receiving side facing upwards, and the conversion unit may convert the temperature measured by each of the plurality of thermometers into the amount of dark current. This provides the effect of being able to correct for variations in dark current depending on the position.

[0016] Furthermore, in this first aspect, the system may further include a plurality of thermometers arranged directly below the photosensitive area of ​​the pixel array with the light-receiving side facing upward, and a weighting processing unit that weights the pixel signals of the OPB pixels and the photosensitive pixels according to the weights corresponding to the temperatures measured by each of the plurality of thermometers. This results in the pixel signals being weighted according to the weights corresponding to the temperature.

[0017] Furthermore, in this first aspect, the system may further include a signal processing unit that corrects the pixel signals from the photosensitive pixels based on the average value of the pixel signals from the OPB pixels, and a mode determination unit that switches to one of a plurality of different correction modes of the signal processing unit based on the estimated value. The pixel array unit, the dark current estimation unit, and the signal processing unit are located on the image sensor, the mode determination unit is located on the application processor, and the image sensor outputs the dark current estimation result to the application processor. This allows the dark current amount estimation data to be used for system operation determination.

[0018] Furthermore, in this first aspect, the application processor may control the image sensor and external devices based on the determination result of the mode. This results in the effect of considering the impact on image quality based on the amount of dark current of the image sensor, and controlling the power consumption and temperature of the entire system.

[0019] Furthermore, in this first aspect, the application processor may supply a control signal to the image sensor for controlling the black level correction operation of the image sensor. This allows the application processor to control the mode of the black level correction operation.

[0020] Furthermore, in this first aspect, the device further comprises a white point correction unit that corrects the white point based on the pixel signals within a predetermined area that includes the pixel signal of interest from the photosensitive pixels, and the size of the area may be set to a larger value as the estimated value increases. This results in the power and correction accuracy being optimally adjusted.

[0021] Furthermore, in this first aspect, the device further includes a white point correction unit that detects and corrects white points based on whether the difference between the average value of the pixel signals within a predetermined region, which includes the pixel signal of interest, and the pixel signal of interest exceeds a predetermined detection threshold, and the detection threshold may be set to a larger value as the estimated value increases. This results in the power and correction accuracy being optimally adjusted.

[0022] Furthermore, the second aspect of this technology is an image sensor comprising a pixel array section in which light-shielded OPB pixels and light-sensitive photosensitive pixels are arranged, and a dark current estimation section that estimates the amount of dark current in the pixel array section based on the exposure time and the temperature measured at a timing synchronized with a predetermined vertical synchronization signal, and outputs it as an estimated value. This results in the power consumption and correction accuracy being controlled to appropriate values.

[0023] This is a block diagram showing an example configuration of an imaging system in the first embodiment of this technology. This is a diagram showing an example of the pixel array section and the signal after AD (Analog to Digital) conversion in the first embodiment of this technology. This is a block diagram showing an example configuration of a control section in the first embodiment of this technology. This is a diagram showing an example of the determination result of the mode determination section in the first embodiment of this technology. This is a block diagram showing an example configuration of a peripheral circuit and a column circuit in the first embodiment of this technology. This is a block diagram showing an example configuration of a signal processing section in the first embodiment of this technology. This is a block diagram showing an example of the column circuit configuration in the first embodiment of this technology. This is a diagram showing an example of the correspondence between temperature and the dark current amount at the temperature measurement point in the first embodiment of this technology. This is a diagram showing an example of low-power mode control in the first embodiment of this technology. This is a timing chart showing an example of normal mode and low-power mode control in the first embodiment of this technology. This is a timing chart showing an example of temperature fluctuation correction mode control in the first embodiment of this technology. This is a flowchart showing an example of image sensor operation in the first embodiment of this technology. This is a flowchart showing an example of temperature fluctuation correction processing in the first embodiment of this technology. This is a timing chart showing an example of the mode during HDR (High-Dynamic-Range) synthesis and preview in the first embodiment of this technology. This is a diagram for explaining the mode during fixed-point motion imaging and object detection in the first embodiment of this technology. This is a diagram showing an example of the data held in the average value holding unit in the second embodiment of this technology. This is a timing chart showing an example of the control of the normal mode and low-power mode in the second embodiment of this technology. This is a diagram showing an example of the determination result of the mode determination unit in the third embodiment of this technology. This is a timing chart showing an example of the control of the normal mode and low-power mode in the third embodiment of this technology. This is an example of a plan view of the pixel array unit in the fourth embodiment of this technology. This is a block diagram showing an example of the configuration of the signal processing unit in the fourth embodiment of this technology. This is a timing chart showing an example of the control of the normal mode and low-power mode in the fourth embodiment of this technology.This is a block diagram showing an example configuration of an imaging system in a modified version of the fourth embodiment of this technology. This is a block diagram showing an example configuration of an application processor in a modified version of the fourth embodiment of this technology. This is a block diagram showing an example configuration of an imaging system in a fifth embodiment of this technology. This is a block diagram showing an example configuration of a control unit in a fifth embodiment of this technology. This is a block diagram showing an example configuration of an application processor in a fifth embodiment of this technology. This is a plan view showing an example of a thermometer arrangement in a sixth embodiment of this technology. This is a block diagram showing an example configuration of a signal processing unit in a sixth embodiment of this technology. This is a block diagram showing another example of a signal processing unit in a sixth embodiment of this technology. This is a block diagram showing an example of a signal processing unit in a seventh embodiment of this technology. This is a diagram for explaining a white spot correction method in a seventh embodiment of this technology. This is a diagram for explaining a white spot correction method in a modified version of the seventh embodiment of this technology. This is a block diagram showing an example of a schematic configuration of a vehicle control system. This is an explanatory diagram showing an example of the installation position of the external information detection unit and the imaging unit.

[0024] The following describes the embodiments for implementing this technology (hereinafter referred to as "embodiments"). The description will proceed in the following order: 1. First embodiment (Example of determining the mode by estimating the amount of dark current) 2. Second embodiment (Example of determining the mode by estimating using the amount of dark current and maintaining the average OPB value for each drive mode) 3. Third embodiment (Example of a stepped low-power mode in which black level correction is partially suspended spatially and temporally) 4. Fourth embodiment (Example of utilizing scene changes to determine the low-power operation mode of the black level correction circuit) 5. Fifth embodiment (Example of outputting dark current data to an external processor, etc., and having the external device select the black level correction operation mode) 6. Sixth embodiment (Example of correcting the pixel signal using the amount of dark current obtained by converting the temperatures measured by multiple thermometers) 7. Seventh embodiment (Example of determining the mode by estimating the amount of dark current and correcting the white dot) 8. Application examples to moving objects

[0025] <1. First Embodiment> [Example of Imaging System Configuration] Figure 1 is a block diagram showing an example configuration of an imaging system 100 in the first embodiment of this technology. This imaging system 100 is for capturing image data and includes an image sensor 200 and an application processor 110. The image sensor 200 also includes a vertical scanning circuit 210, a control unit 220, a thermometer 230, a pixel array unit 240, a peripheral circuit 250, a column circuit 260, a signal processing unit 280, and an input / output interface 290. The imaging system 100 can be installed in various imaging devices such as digital still cameras, in-vehicle cameras, and smartphones.

[0026] The pixel array section 240 has multiple pixels (not shown) arranged in a two-dimensional grid. The vertical scanning circuit 210 drives the rows of the pixel array section 240 sequentially and outputs the analog signals as pixel signals to the column circuit 260.

[0027] The control unit 220 controls the operation of various circuits, including the vertical scanning circuit 210, the thermometer 230, the peripheral circuit 250, the column circuit 260, the signal processing unit 280, and the input / output interface 290.

[0028] The thermometer 230 measures the temperature according to the control of the control unit 220. The peripheral circuit 250 is equipped with various circuits such as a DAC (Digital to Analog Converter).

[0029] The column circuit 260 performs AD conversion on the pixel signals of each column and supplies them to the signal processing unit 280.

[0030] The signal processing unit 280 performs various signal processing, such as black level correction, on the pixel signals from the column circuit 260. The image data (frame) formed by the arrangement of the processed pixel signals is output to the application processor 110 via the input / output interface 290.

[0031] The application processor 110 performs various processes on image data, such as image recognition.

[0032] [Example of Pixel Array Configuration] Figure 2 shows an example of the pixel array unit 240 and the signal after AD conversion in the first embodiment of this technology. In the figure, a is an example of a plan view of the pixel array unit 240.

[0033] As illustrated in Figure a, the pixel array 240 includes a VOPB region 241 and a photosensitive region 242. The VOPB region 241 is a light-shielded region in which a predetermined number of rows of OPB pixels 243 arranged horizontally are arranged vertically. These rows are referred to as "OPB rows". For example, in Figure a, there are two OPB rows. Note that the number of OPB rows is not limited to two, but may be three or more.

[0034] The photosensitive area 242 is an area that is not shielded from light, and multiple photosensitive pixels 244 are arranged in a two-dimensional grid. A row within the photosensitive area 242 is called a "photosensitive row".

[0035] Furthermore, the pixel signals from the OPB pixels 243 and the photosensitive pixels 244 are converted to AD by a column circuit 260 (not shown).

[0036] In the figure, b shows an example of the pixel signal Do after AD conversion. Let I be the number of rows (where I is an integer) and J be the number of columns (where J is an integer). Do is the pixel signal of row i (where i is an integer from 1 to I) and column j (where j is an integer from 1 to J). ｉｊ Let's assume that among these pixel signals, the pixel signal of the OPB pixel is Do ｏｐｂ ｉｊ The pixel signal of the photosensitive pixel 244 is Do ｅｘｐ ｉｊ Let's assume that.

[0037] [Example of Control Unit Configuration] Figure 3 is a block diagram showing an example configuration of the control unit 220 in the first embodiment of this technology. This control unit 220 includes a dark current amount estimation unit 221, a mode determination unit 222, and a temperature measurement control unit 223.

[0038] The dark current estimation unit 221 estimates the dark current of the pixel array unit 240. This dark current estimation unit 221 receives the temperature T measured from the thermometer 230 at a timing synchronized with the vertical synchronization signal VSYNC, and estimates the dark current based on the temperature T and the exposure time EX. The frequency of the vertical synchronization signal VSYNC is, for example, 30 Hz or 60 Hz. For example, the dark current is estimated by a predetermined function that returns a larger dark current for higher temperatures T and a larger dark current for longer exposure times EX. The dark current estimation unit 221 returns an estimated value I of the dark current. ｅｓｔ This is supplied to the mode determination unit 222.

[0039] The mode determination unit 222 determines the estimated value I of the dark current amount. ｅｓｔ Based on this, the mode to which the image sensor 200 should transition is determined. This mode includes, for example, a low-power mode, a normal mode, and a temperature fluctuation compensation mode. The low-power mode is a mode that consumes less power than the normal mode and the temperature fluctuation compensation mode. The temperature fluctuation compensation mode is a mode in which the amount of dark current is determined from the temperature measured at each of several different timings during the exposure period, and the pixel signals of the OPB pixels and photosensitive pixels are corrected based on that amount of dark current. The mode determination unit 222 then controls each circuit in the image sensor 200 based on the mode determination result. The mode determination unit 222 also supplies the determination result to the temperature measurement control unit 223. The default mode may be the low-power mode. In this case, low-power operation is basically performed in order to obtain a power reduction effect.

[0040] The temperature measurement control unit 223 controls the measurement operation of the thermometer 230 based on the mode determination result. In low-power mode and normal mode, the temperature measurement control unit 223 measures the temperature at a timing synchronized with the vertical synchronization signal VSYNC. On the other hand, in temperature fluctuation correction mode, the temperature measurement control unit 223 measures the temperature periodically at multiple different timings within the exposure period. For example, in temperature fluctuation correction mode, the temperature measurement control unit 223 measures the temperature at the start of exposure for the OPB row and the photosensitive row, synchronized with the horizontal synchronization signal HSYNC.

[0041] It should be noted that, in the temperature fluctuation correction mode, the temperature measurement period does not have to be the same as the period of the horizontal synchronization signal HSYNC. For example, the temperature measurement control unit 223 may also cause temperature measurement to be performed at a period longer than the period of the horizontal synchronization signal HSYNC.

[0042] FIG. 4 is a diagram illustrating an example of a determination result of the mode determination unit 222 in the first embodiment of the present technology.

[0043] For example, when the estimated dark current amount I ｅｓｔ is within a range from the threshold value Th1 to the threshold value Th2, the mode determination unit 222 determines that the mode to be shifted to is the normal mode, and shifts to that mode. Furthermore, when the estimated dark current amount I ｅｓｔ is smaller than the threshold value Th1, the mode determination unit 222 determines that the mode to be shifted to is the low power mode, and shifts to that mode. Furthermore, when the estimated dark current amount I ｅｓｔ is larger than the threshold value Th2, the mode determination unit 222 determines that the mode to be shifted to is the temperature fluctuation correction mode, and shifts to that mode.

[0044] Then, the mode determination unit 222 controls each circuit in the image sensor 200 based on the mode determination result. For example, when shifting to the normal mode, the mode determination unit 222 controls the vertical scanning circuit 210 to sequentially drive OPB rows and photosensitive rows. Furthermore, in the normal mode, the mode determination unit 222 operates the peripheral circuit 250 and the column circuit 260 within the readout period of the OPB rows and photosensitive rows. Furthermore, in the normal mode, the mode determination unit 222 controls the signal processing unit 280 to perform calculations such as calculation of the average value of pixel signals of OPB pixels. Furthermore, in the normal mode, the mode determination unit 222 controls the input / output interface 290 to output pixel signals, and stops the output within the vertical blanking period.

[0045] On the other hand, when switching to low-power mode, for example, the mode determination unit 222 controls the vertical scanning circuit 210 to stop the OPB row drive operation. Also, in low-power mode, the mode determination unit 222 stops the peripheral circuit 250 and column circuit 260 during the OPB row read period. Also, in low-power mode, the mode determination unit 222 controls the signal processing unit 280 to stop the calculation operation of the average value of the pixel signals of the OPB pixels. Also, in low-power mode, the mode determination unit 222 controls the input / output interface 290 to extend the data output stop period. In addition, in low-power mode, the mode determination unit 222 can also reduce the frame rate and the data size of the frame.

[0046] Furthermore, when the system switches to temperature fluctuation correction mode, the mode determination unit 222 causes the vertical scanning circuit 210, peripheral circuit 250, column circuit 260, and input / output interface 290 to operate in the same manner as in normal mode. In addition, in temperature correction operation mode, the mode determination unit 222 controls the signal processing unit 280 to correct the pixel signals of the OPB pixels and photosensitive pixels based on the amount of dark current obtained from the temperature. Details of this process will be described later.

[0047] [Example of Peripheral Circuit and Column Circuit Configuration] Figure 5 is a block diagram showing an example configuration of the peripheral circuit 250 and column circuit 260 in the first embodiment of this technology. The peripheral circuit 250 includes a DAC 251 and a reference level generation circuit 252. The column circuit 260 includes a plurality of load MOS current sources 261 and a plurality of ADCs 270. The load MOS current sources 261 and ADCs 270 are provided for each column, and if the number of columns is J, J of each are provided.

[0048] The DAC251 generates a ramp signal RMP by DA (Digital to Analog) conversion according to the control of the control unit 220 and supplies it to each of the ADC270s.

[0049] The reference level generation circuit 252 generates a reference voltage and current and supplies them to the column circuit 260 and the like.

[0050] Furthermore, vertical signal lines VSL are wired to each row of the pixel array 240. The load MOS current source 261 supplies a constant load MOS current to the vertical signal line VSL of the corresponding row.

[0051] The ADC 270 is a single-slope ADC and includes, for example, a comparator 271 and a counter 272. Each of the ADC 270 receives the analog pixel signal Ai from the OPB pixels via the corresponding vertical signal line VSL. оｐｂ And, the analog pixel signal Ai from the photosensitive pixel ｅｘｐ The following are entered in order:

[0052] Comparator 271 takes the ramp signal RMP from DAC 251 and the analog signal (Ai) from the corresponding vertical signal line VSL. оｐｂ or Ai ｅｘｐ This compares the two values. The comparator 271 supplies the comparison result to the counter 272.

[0053] Counter 272 counts the count value in synchronization with the clock signal CLK for the period until the comparison result reverses. This counter 272 outputs a digital signal indicating the count value to the signal processing unit 280. оｐｂ Digital signal corresponding to Do оｐｂ This is the pixel signal of the OPB pixel after AD conversion, and Ai ｅｘｐ Digital signal corresponding to Do ｅｘｐ This is the pixel signal of the photosensitive pixel after AD conversion.

[0054] While the ADC270 uses a single-slope type ADC, it is also possible to use other types of ADCs, such as SARADC (Successive Approximation Register ADC).

[0055] [Example of Signal Processing Unit Configuration] Figure 6 is a block diagram showing an example of the configuration of the signal processing unit 280 in the first embodiment of this technology. This signal processing unit 280 includes a temperature-to-dark current conversion unit 281, a dark current correction unit 282, an average value calculation unit 283, an average value holding unit 284, and a clamping unit 285.

[0056] The temperature-dark current conversion unit 281, when it switches to temperature fluctuation correction mode, converts the temperature measured at multiple timings during the exposure period into the dark current at the time of measurement and outputs it to the dark current correction unit as the temperature measurement point dark current. For example, if the temperature is measured N times (N is an integer) within the access period from the start of exposure for the first row to the start of exposure for the last row, and the temperature measurement value of the nth time (n is an integer from 1 to N) is T ｎ Let's assume that T ｎ The corresponding temperature measurement point dark current is I ｄａｒｋ ｎ The temperature-dark current conversion unit 281 is an example of a conversion unit described in the claims.

[0057] The dark current amount correction unit 282 adjusts the dark current amount I at the temperature measurement point. ｄａｒｋ ｎ This method corrects the pixel signal Do after AD conversion. Exposure is performed using a rolling shutter method, and in the rolling shutter method, from the start of exposure of the i-th photosensitive row to the end of I ｍａｘ The period until the start of exposure for the second photosensitive row is the access period P. ｉ This is the access period P. ｉ The amount of dark current Q generated within the system ｄａｒｋ ｉ This can be expressed, for example, by the following formula:

[0058] In the above equation, P is the period from the start of exposure for the first row to the end of exposure for the last row. N is the number of temperature measurements taken during that period P. The temperature is measured for each row in synchronization with the horizontal synchronization signal HSYNC, and the number of rows is I ｍａｘ In the case of I ｍａｘ The same value is set to N. The frequency of the horizontal sync signal HSYNC is higher than that of the vertical sync signal VSYNC, for example, between 10 and 500 kilohertz (kHz).

[0059] Furthermore, if the temperature measurement cycle is lower than the horizontal synchronization signal HSYNC, N is I ｍａｘ This value is smaller than the given value. In this case, the dark current at the nth measurement timing, which is closest to the exposure start timing of the i-th row, is used in Equation 1.

[0060] The dark current amount correction unit 282 calculates the dark current amount Q in the temperature fluctuation correction mode according to Equation 1. ｄａｒｋ ｉ The dark current amount correction unit 282 then calculates the pixel signal Do of row i and column j. ｉｊ The following formula is used to correct the pixel signal Do' of the corrected pixel signal Do' of the OPB pixel. оｐｂ This is output to the average value calculation unit 283, and the pixel signal Do of the photosensitive pixel ｅｘｐ This is output to the clamping processing unit 285. ｉｊ =Do ｉｊ -Q ｄａｒｋ ｉ ...Formula 2

[0061] In normal mode or low power mode, the temperature-dark current conversion unit 281 stops its conversion operation, and the dark current correction unit 282 supplies the pixel signals of the OPB pixels to the average value calculation unit 283 without correction. Similarly, the pixel signals of the photosensitive pixels are also supplied to the clamping processing unit 285 without correction.

[0062] The average value calculation unit 283 calculates the average value of the pixel signals of the target OPB pixels. When the system switches to normal mode, the average value calculation unit 283 calculates the average value AVE of the pixel signals of the OPB pixels and supplies it to the average value holding unit 284 and the clamping processing unit 285.

[0063] Furthermore, when the system switches to temperature fluctuation correction mode, the average value calculation unit 283 calculates the pixel signal Do of the corrected OPB pixel. ｏｐｂ ｉｊ The average value is calculated and supplied to the average value holding unit 284 and the clamping processing unit 285. Furthermore, when the system switches to low-power mode, the average value calculation unit 283 stops its calculation operation.

[0064] The average value holding unit 284 stores the average value calculated in normal mode or temperature compensation mode as AVE_P. The data in the average value holding unit 284 is updated for each frame.

[0065] The clamping processing unit 285 uses the average value of the pixel signals of the OPB pixels to determine the pixel signal Do of the photosensitive pixels. ｅｘｐThis is a clamping process that corrects the signal. This clamping process is performed after AD conversion and is therefore considered a digital clamping process. When switching to normal mode, the clamping processing unit 285 performs the clamping process using the average value AVE calculated by the average value calculation unit 283. Specifically, the clamping processing unit 285 processes the pixel signal Do of row i and column j. ｅｘｐ ｉｊ Subtracting the average value AVE from this, the pixel signal Do' after clamping is obtained. ｅｘｐ ｉｊ Output as follows.

[0066] When the system switches to temperature compensation mode, the clamping unit 285 performs clamping using the average value AVE calculated by the average value calculation unit 283. Specifically, the clamping unit 285 uses the pixel signal Do of row i and column j. ｅｘｐ ｉｊ Subtract the average value AVE from this, and the pixel signal Do' ｅｘｐ ｉｊ Output as follows.

[0067] When the system switches to low-power mode, the clamping unit 285 performs clamping using the average value AVE_P held in the average value holding unit 284. This makes it possible to correct optical black level deviations caused by factors other than dark current.

[0068] Furthermore, when switching from low-power mode to other modes, the black level correction value may increase, so it is preferable to perform analog clamping. In this case, for example, as illustrated in Figure 7, an analog clamping circuit 262 is added to the column circuit 260.

[0069] Furthermore, the signal processing unit 280 can perform various signal processing tasks in addition to clamping, such as white balance correction and demosaicing. Also, as will be described later, some of the processing within the signal processing unit 280 can be performed outside the image sensor 200.

[0070] As described above, if the temperature of the image sensor fluctuates during exposure using the rolling shutter method, the amount of dark current generated may differ for each readout row. However, the signal processing unit 280 can suppress the decrease in the accuracy of black level correction due to temperature fluctuations by converting the temperature measured at multiple timings during the exposure period into the amount of dark current at the temperature measurement point and using it for correction.

[0071] Furthermore, as mentioned above, in the rolling shutter method, the amount of dark current decreases from the first row to the last row. This fluctuation in dark current can be calculated using Equation 1, and by correcting the pixel signal with this value, the decrease in the accuracy of black level correction due to temperature drop during readout can be suppressed.

[0072] Figure 8 shows an example of the correspondence between temperature and the dark current at the temperature measurement point in the first embodiment of this technology. A table associating temperature with the dark current at the temperature measurement point, as illustrated in the figure, is pre-stored in a register or the like within the image sensor 200. The temperature-dark current conversion unit 281 converts the measured temperature into the dark current at the temperature measurement point by referring to this table.

[0073] As illustrated in the figure, the conversion from temperature to dark current at the temperature measurement point is performed for each temperature measurement point using a table that establishes a one-to-one correspondence between the two. Therefore, the image sensor 200 can handle both rising and falling temperatures, and can also handle nonlinear temperature fluctuations.

[0074] [Example of Image Sensor Operation] Figure 9 shows an example of low-power mode control in the first embodiment of this technology. The gray area in the figure shows the circuit that stops operating in low-power mode.

[0075] When switching to low-power mode, the control unit 220 stops the peripheral circuit 250 and the column circuit 260 during the OPB row reading period. In addition, in low-power mode, the control unit 220 controls the input / output interface 290 to extend the data output stop period. These controls result in lower power consumption of the image sensor 200 compared to normal mode.

[0076] Figure 10 is a timing chart showing an example of control for the normal mode and low-power mode in the first embodiment of this technology. The thick lines in the figure indicate the exposure start timing for each row. The dashed lines indicate the exposure end and readout timings for each row.

[0077] A vertical synchronization signal VSYNC rises at timings T2 and T6, and the thermometer 230 measures the temperature at timings T1 and T5, which are synchronized with the vertical synchronization signal VSYNC.

[0078] Furthermore, when the temperature is measured, the dark current estimation unit 221 estimates the dark current based on that temperature and the exposure time. At timing T2, if the estimated value of the dark current is within the range of Th1 to Th2, the mode determination unit 222 determines that it is in normal mode.

[0079] In normal mode, during the readout period from timing T2 to T3, the ADC270 of each column reads a predetermined number of OPB rows sequentially in synchronization with the clock signal CLK. During the readout period from timing T3 to T4, the ADC270 of each column reads a predetermined number of photosensitive rows sequentially in synchronization with the clock signal CLK.

[0080] Then, at timing T6, the estimated value of the dark current becomes smaller than Th2, and the mode determination unit 222 determines that it is in low-power mode.

[0081] In low-power mode, the ADC270 in each column reads a predetermined number of photosensitive rows sequentially in synchronization with the clock signal CLK during the readout period from timing T7 to T8. During the readout period for the OPB row, from timing T6 to T7, the ADC270 in each column stops, and the supply of the clock signal CLK is also stopped. This reduces power consumption compared to normal mode.

[0082] In particular, in bright imaging environments where frames with low dark current are consecutive, the low-power mode period is extended, resulting in greater power savings. Furthermore, since the OPB pixel signals are not read out in low-power mode, it may be possible to increase the frame rate.

[0083] Figure 11 is a timing chart showing an example of the control of the temperature fluctuation compensation mode in the first embodiment of this technology. The thick lines in the figure indicate the exposure start timing for each row. The dashed lines indicate the exposure end and readout timings for each row.

[0084] As illustrated in the figure, exposure of the first row begins at timing T10, and exposure of the last row begins at timing T11. When the system switches to temperature fluctuation correction mode, the thermometer 230 periodically measures the temperature N times within the access period from timing T10 to T11. The temperature-dark current conversion unit 281 converts these temperatures into the dark current values ​​at the time of measurement. By using these dark current values ​​to correct the pixel signals of the OPB pixels and photosensitive pixels, noise such as vertical shading that occurs when the temperature fluctuates during exposure can be eliminated.

[0085] In the range where the exposure periods of each row from timing T11 to T12 overlap, all rows are affected equally even with temperature fluctuations. Therefore, in corrections using equations 1 and 2, only the access period is corrected, and the bit width and dark current variance values ​​used in the calculation are treated as small.

[0086] Then, during the readout period from timing T12 to T13, the ADC270 of each column reads out a predetermined number of OPB rows and a predetermined number of photosensitive rows in synchronization with the clock signal CLK. Each time a row is read out during this period, the operation of the circuit for that row stops and the heat generation subsides, causing the temperature to drop. The fluctuation in the amount of dark current due to this temperature drop can be calculated using Equation 1, and by correcting the pixel signals of the OPB pixels and photosensitive pixels with this value, the decrease in the accuracy of black level correction due to the temperature drop during readout can be suppressed. As a result, the accuracy of black level correction can be increased and image quality can be improved compared to when the pixel signals are not corrected.

[0087] Furthermore, the thermometer 230 periodically measures the temperature N times during the readout period from timing T12 to T13. The temperature-to-dark current conversion unit 281 converts these temperatures into dark current values ​​at the time of measurement. These dark current values ​​are also used to correct the pixel signals of the OPB pixels and photosensitive pixels. Note that the thermometer 230 and the temperature-to-dark current conversion unit 281 measure the temperature and convert it to dark current during both the access period and the readout period, but measurement and conversion may be performed only during one of the access period or the readout period.

[0088] The processing in the temperature fluctuation compensation mode described above is particularly effective for long-duration shooting in dark places and for shooting in high-temperature environments in dark places.

[0089] Figure 12 is a flowchart illustrating an example of the operation of the image sensor 200 in a first embodiment of this technology. This operation is initiated, for example, when a predetermined application for capturing image data is executed.

[0090] The thermometer 230 inside the image sensor 200 measures the temperature at a timing synchronized with the vertical synchronization signal VSYNC (step S901), and the control unit 220 estimates the amount of dark current from that temperature and the exposure period (step S902).

[0091] The control unit 220 determines whether to switch to low-power mode based on the estimated dark current amount (step S903). If it switches to low-power mode (step S903: Yes), the vertical scanning circuit 210 drives the photosensitive row to be read, and the ADC 270 in each column reads the photosensitive row (step S904). Then, the signal processing unit 280 performs clamping (step S905). In low-power mode, the average value held in normal mode, etc., is read out and used in step S905.

[0092] The vertical scanning circuit 210 then determines whether or not all rows have been read (step S906). If all rows have not been read (step S906: No), the image sensor 200 repeats steps S904 onward. If all rows have been read (step S906: Yes), the image sensor 200 terminates its imaging operation.

[0093] Furthermore, if the system does not transition to low-power mode (step S903: No), the control unit 220 determines whether or not to transition to temperature fluctuation correction mode (step S907). If the system transitions to normal mode (step S907: No), the vertical scanning circuit 210 drives all OPB rows sequentially, and the ADC 270 in each column reads those OPB rows (step S908). The signal processing unit 280 then calculates and stores the average value of the pixel signals of the target OPB pixels (step S909). Then, steps S904 onwards are executed.

[0094] Furthermore, if the system transitions to temperature fluctuation correction mode (step S907: Yes), the image sensor 200 performs temperature fluctuation correction processing to correct the pixel signal (step S910). Then, steps S904 and onward are executed.

[0095] When capturing multiple image data consecutively, steps S901 to S906 are repeatedly executed in synchronization with the vertical synchronization signal VSYNC.

[0096] Figure 13 is a flowchart illustrating an example of temperature fluctuation correction processing in the first embodiment of this technology. The thermometer 230 in the image sensor 200 performs N temperature measurements within the access period (step S911). The signal processing unit 280 converts each temperature into a dark current at the temperature measurement point (step S912). The vertical scanning circuit 210 drives all OPB rows sequentially, and the ADC 270 in each column reads out those OPB rows (step S913). The signal processing unit 280 then corrects the pixel signals of the OPB pixels and photosensitive pixels using the dark current at the temperature measurement point (step S914), and calculates and stores the average value of the corrected OPB pixel signals (step S915). After step S915, the image sensor 200 terminates the temperature fluctuation correction processing.

[0097] The low-power mode described above can be used for a variety of applications.

[0098] For example, consider the case where the image sensor 200 performs HDR synthesis, as illustrated in Figure 14a. At timing T15, the image sensor 200 captures frame 501 with an exposure time longer than a predetermined time. Then, at timing T16, the image sensor 200 captures frame 502 with an exposure time shorter than a predetermined time. Finally, the image sensor 200 synthesizes frames 501 and 502.

[0099] At timing T15, the estimated dark current becomes large due to the relatively long exposure time, so the mode determination unit 222 determines it to be either normal mode or temperature fluctuation mode (normal mode in the figure). On the other hand, at timing T16, the estimated dark current becomes small due to the relatively short exposure time, so unless it is a high-temperature environment, the mode determination unit 222 is likely to switch to low-power mode. In the figure, it is assumed that the system switches to low-power mode at timing T16. In this way, power consumption can be reduced by switching the image sensor 200 to low-power mode when performing HDR synthesis.

[0100] Furthermore, consider the case where the image sensor 200 performs a preview before imaging, as illustrated in Figure 14b. Before timing T17, the image sensor 200 performs a preview operation. In this preview operation, the image sensor 200 displays image data on the display before imaging. When performing the preview operation, the control unit 220 determines, for example, that it is in low-power mode regardless of the estimated value of the dark current. Then, in response to pressing the shutter button or the like, the image sensor 200 captures image data at timing T17. When imaging is performed, the control unit 220 determines that it is in normal mode or temperature fluctuation correction mode (normal mode in the same figure).

[0101] Furthermore, the control unit 220 can also facilitate the transition to low-power mode during preview operation by using a higher threshold Th1 than that used during imaging.

[0102] During preview operation, as mentioned above, switching to low-power mode allows for prioritizing power reduction while tolerating noise.

[0103] Furthermore, consider the case where the imaging system 100 performs fixed-point motion imaging at a relatively low frame rate (e.g., 1 to 2 frames per second), such as when capturing time-lapse videos, as illustrated in Figure 15a. In this case, since the situation does not change significantly from frame to frame, fluctuations in the amount of dark current have little effect on image quality. Therefore, the control unit 220 determines that it is in low-power mode when performing fixed-point video imaging, regardless of the estimated amount of dark current. Alternatively, the control unit 220 uses a higher threshold Th1 during fixed-point video imaging than during other imaging to facilitate the transition to low-power mode.

[0104] Furthermore, consider the case where the image sensor 200 detects an object such as a face, as illustrated in figure b. In such sensing applications, image quality may be sacrificed. For this reason, the control unit 220 determines that it is in low-power mode when an object is detected, regardless of the estimated value of the dark current. Alternatively, the control unit 220 uses a higher threshold Th1 when an object is detected than when no object is detected, making it easier to switch to low-power mode.

[0105] Thus, according to the first embodiment of this technology, the control unit 220 estimates the amount of dark current in the pixel array based on the exposure time and temperature, and switches to a low-power mode or a temperature fluctuation correction mode based on the estimated value, thereby enabling the power consumption and correction accuracy to be set to appropriate values.

[0106] <2. Second Embodiment> In the first embodiment described above, the image sensor 200 did not perform pixel binning in each mode, but pixel binning can be performed for purposes such as improving sensitivity. The image sensor 200 in this second embodiment differs from the first embodiment in that it performs pixel binning as needed.

[0107] Figure 16 shows an example of data held in the average value holding unit 284 in the second embodiment of this technology.

[0108] In the second embodiment, the image sensor 200 can perform pixel addition by changing the drive mode of the pixel array section 240. Of the drive modes, the mode in which no pixel addition is performed is designated as the first drive mode, and the mode in which four pixels in a 2x2 arrangement are added is designated as the second drive mode. By changing the number of pixels to be added, it is also possible to switch between three or more drive modes. These drive modes can be set independently of the modes that are switched according to the amount of dark current (i.e., normal mode, low power mode, and temperature fluctuation compensation mode).

[0109] The average value holding unit 284 of the second embodiment holds the average value of the pixel signal of the OPB pixel for each drive mode. The average value holding unit 284 holds the average value of the first drive mode as the first drive mode average value, and holds the average value of the second drive mode as the second drive mode average value.

[0110] Figure 17 is a timing chart showing an example of control for the normal mode and low-power mode in a second embodiment of this technology. At timing T21, the image sensor 200 determines that it is in normal mode based on the estimated dark current amount and captures a frame 501 in a non-additive first drive mode. The average value of the first drive mode is also held in the average value holding unit 284.

[0111] Then, at timing T22, the image sensor 200 remains in normal mode, but switches its drive mode to a second drive mode that performs pixel addition, and captures a frame 502. In addition, the average value of the second drive mode is further stored in the average value holding unit 284.

[0112] Then, at timing T23, the image sensor 200 determines that it is in low-power mode and switches the drive mode to the first drive mode. At this time, the signal processing unit 280 reads the average value of the first drive mode from the average value holding unit 284 and performs clamping.

[0113] Then, at timing T24, the image sensor 200 determines that it is in low-power mode and switches the drive mode to the second drive mode. At this time, the signal processing unit 280 reads the average value of the second drive mode from the average value holding unit 284 and performs clamping.

[0114] As mentioned above, by maintaining an average value for each drive mode, low-power operation can be maintained even when switching drive modes.

[0115] Thus, according to the first embodiment of this technology, since the average value holding unit 284 holds an average value for each drive mode, the image sensor 200 can continue low-power operation even when the drive mode is switched.

[0116] <3. Third Embodiment> In the first embodiment described above, the control unit 220 changed the power consumption in two stages, but it can also be changed in three or more stages. The image sensor 200 in this third embodiment differs from the first embodiment in that it changes the power consumption in more stages.

[0117] Figure 18 shows an example of the determination result of the mode determination unit 222 in the third embodiment of this technology.

[0118] In the third embodiment, the estimated value I of the dark current ｅｓｔ If the value is less than the threshold Th1_1, the mode determination unit 222 determines that the mode to transition to is the first low-power mode and transitions to that mode. Estimated value I ｅｓｔIf the value is greater than or equal to threshold Th1_1 and less than threshold Th1_2, the mode determination unit 222 determines that the mode to transition to is the second low-power mode and transitions to that mode. In the second low-power mode, the control unit 220 controls the power consumption to a value between the first low-power mode and the normal mode. By transitioning to the second low-power mode, it is possible to improve the black level correction accuracy compared to the first low-power mode while reducing power consumption compared to the normal mode.

[0119] Estimated value I ｅｓｔ If the value is within the range from threshold Th1_2 to threshold Th2, the mode determination unit 222 determines that the mode to transition to is the normal mode and transitions to that mode. Also, the estimated value I ｅｓｔ If the value is greater than the threshold Th2, the mode determination unit 222 determines that the mode to be transitioned to is the temperature fluctuation correction mode and transitions to that mode.

[0120] Figure 19 is a timing chart showing an example of control for the normal mode and low-power mode in a third embodiment of this technology. The thick lines in the figure indicate the exposure start timing for each row. The dashed lines indicate the exposure end and readout timings for each row.

[0121] A vertical synchronization signal VSYNC rises at timings T2 and T6, and the thermometer 230 measures the temperature at timings T1 and T5, which are synchronized with the vertical synchronization signal VSYNC.

[0122] At timing T2, if the estimated value of the dark current is within the range of Th1_2 to Th2, the mode determination unit 222 determines that it is in normal mode.

[0123] In normal mode, within the readout period from timing T2 to T4, the ADC270 of each column reads out a predetermined number of OPB rows and a predetermined number of photosensitive rows in synchronization with the clock signal CLK.

[0124] Then, the amount of dark current decreases, and at timing T6, the mode determination unit 222 determines that it is in the first low-power mode.

[0125] In the first low-power mode, the ADC270 of each column reads a predetermined number of photosensitive rows sequentially in synchronization with the clock signal CLK during the readout period from timing T7 to T8. During the readout period for the OPB row, from timing T6 to T7, the ADC270 of each column stops, and the supply of the clock signal CLK is also stopped.

[0126] Then, the amount of dark current increases slightly, and at timing T31, the mode determination unit 222 determines that it is in the second low-power mode.

[0127] In the second low-power mode, during the readout period from timing T31 to T32, the ADC 270 of each column sequentially reads a portion of the OPB rows in synchronization with the clock signal CLK. During the readout period for the remaining OPB rows, from timing T32 to T33, the ADC 270 of each column stops, and the supply of the clock signal CLK is also stopped. Then, during the readout period from timing T33 to T34, the ADC 270 of each column sequentially reads a predetermined number of photosensitive rows in synchronization with the clock signal CLK. The signal processing unit 280 performs clamping using the average value of these pixel signals.

[0128] As illustrated in the figure, in the second low-power mode, only a portion of the OPB row is read. This improves the black level correction accuracy compared to the first low-power mode while reducing power consumption compared to the normal mode.

[0129] Furthermore, the control unit 220 can also change the power consumption in four or more stages. Additionally, the third embodiment can be applied to the second embodiment.

[0130] Thus, according to the third embodiment of this technology, the mode determination unit 222 transitions to one of the modes, including the first low-power mode, the second low-power mode, and the normal mode, based on the amount of dark current, and the power consumption can be changed in three or more stages.

[0131] <4. Fourth Embodiment> In the first embodiment described above, the control unit 220 determined the mode based on the amount of dark current, but it is also possible to use a flag indicating whether or not the scene being captured has changed for the determination. The image sensor 200 in this fourth embodiment differs from the first embodiment in that it determines the mode based on whether or not the scene has changed.

[0132] Figure 20 is an example of a plan view of the pixel array section 240 in the fourth embodiment of this technology. In the photosensitive region in the fourth embodiment, a portion of the photosensitive pixels are used to receive one of a pair of pupil-divided lights, and these photosensitive pixels are designated as phase-difference pixels 244-2. Photosensitive pixels that do not correspond to phase-difference pixels 244-2 are designated as effective pixels 244-1.

[0133] Figure 21 is a block diagram showing an example configuration of the signal processing unit 280 in the fourth embodiment of this technology. The signal processing unit 280 of the fourth embodiment differs from the first embodiment in that it further comprises a focus detection unit 286.

[0134] Pixel signal Do of photosensitive pixels (effective pixels and phase difference pixels) ｅｘｐ Of these, the pixel signal Do of the effective pixels ｖａｌｉｄ This is input to the clamping processing unit 285. The pixel signal of the phase difference pixel is the phase difference signal Do ａｆ This is input to the focus detection unit 286.

[0135] The focus detection unit 286 converts the phase difference signal into a defocus amount. The defocus amount is the distance between the current position of the focus lens (not shown) and the in-focus position. The focus detection unit 286 supplies the defocus amount to the control unit 220. The defocus amount is also used to move the focus lens to the in-focus position.

[0136] When the control unit 220 switches to low-power mode, the mode determination unit 222 periodically acquires the defocus amount in synchronization with the vertical synchronization signal VSYNC. The mode determination unit 222 then determines whether the scene has changed based on whether the change in the defocus amount from the previous value exceeds a predetermined value. If the scene has changed, the mode determination unit 222 switches from low-power mode to normal mode.

[0137] Figure 22 is a timing chart showing an example of control for the normal mode and low-power mode in the fourth embodiment of this technology. The image sensor 200 is assumed to have switched to low-power mode before timing T41.

[0138] The image sensor 200 captures frames 511 and 512 in synchronization with the vertical synchronization signal VSYNC at timings T41 and T42. The mode determination unit 222 determines whether the scene has changed in synchronization with the vertical synchronization signal VSYNC based on the defocus amount obtained by converting the phase difference signal. A flag indicating this determination result is designated as the scene change flag.

[0139] Then, frame 513 is captured at timing T43. The distance to the subject in frame 513 has changed compared to frames 511 and 512. As a result, the amount of defocus changes, and the mode determination unit 222 determines that the scene has changed and switches to normal mode.

[0140] As described above, when the scene changes after switching to low-power mode, the mode determination unit 222 switches to normal mode, thereby enabling appropriate mode control in response to scene changes.

[0141] Furthermore, the fourth embodiment can also be applied to the second and third embodiments.

[0142] Thus, according to the fourth embodiment of this technology, the mode determination unit 222 determines the mode based on whether or not the scene has changed, and therefore the mode can be appropriately controlled in response to the change in the scene.

[0143] [Modified Version] In the fourth embodiment described above, the image sensor 200 determined whether or not the scene had changed based on the amount of defocus obtained by converting the phase difference signal, but the configuration is not limited to this. The imaging system 100 in this modified version of the fourth embodiment differs from the fourth embodiment in that it determines whether or not the scene has changed based on a signal from an external sensor to the image sensor 200.

[0144] Figure 23 is a block diagram showing one configuration example of an imaging system 100 in a modified example of the fourth embodiment of the present technology. The imaging system 100 in the modified example of the fourth embodiment differs from the fourth embodiment in that it further includes a ToF sensor 120.

[0145] The ToF sensor 120 is a sensor that detects the distance to an object using the ToF method. This ToF sensor 120 supplies a sensor signal containing a predetermined number of distance data to the application processor 110, for example.

[0146] Figure 24 is a block diagram showing an example configuration of an application processor 110 in a modified version of the fourth embodiment of the present technology. The application processor 110 includes a sensor signal processing unit 111 and an image processing unit 112.

[0147] The sensor signal processing unit 111 determines whether or not the scene has changed based on the sensor signal from the ToF sensor 120. The sensor signal processing unit 111 generates a scene change flag indicating the determination result and supplies it to the mode determination unit 222 (not shown) via the input / output interface 290. The mode determination unit 222, similar to the fourth embodiment, switches to normal mode when the scene changes after switching to low-power mode.

[0148] The image processing unit 112 performs various image processing operations, such as object recognition, on image data received via the input / output interface 290.

[0149] In Figures 23 and 24, the application processor 110 determines whether the scene has changed based on the sensor signal from the ToF sensor 120, but the configuration is not limited to this. Instead of the ToF sensor 120, an EVS can be provided, and the application processor 110 can determine the scene change based on the signal from the EVS.

[0150] By using signals from the ToF sensor 120 and EVS, even an imaging system 100 without phase-difference pixels can determine whether or not the scene has changed and use this information to determine the mode.

[0151] Furthermore, modifications of the fourth embodiment can also be applied to the second and third embodiments.

[0152] Thus, according to a modified version of the fourth embodiment of this technology, the application processor 110 determines whether or not the scene has changed based on signals from the ToF sensor 120 and the EVS.

[0153] <5. Fifth Embodiment> In the first embodiment described above, the image sensor 200 determined the mode, but an external device (such as the application processor 110) can also determine the mode. The imaging system 100 in this fifth embodiment differs from the first embodiment in that the mode is determined outside the image sensor 200.

[0154] Figure 25 is a block diagram showing an example configuration of the imaging system 100 in a fifth embodiment of this technology. The imaging system 100 of this fifth embodiment differs from the first embodiment in that it further comprises a predetermined number of devices such as devices 131 and 132. In addition, the application processor 110 of the fifth embodiment estimates the dark current I from the image sensor 200. ｅｓｔ The system further receives the signal to determine the mode and generates a mode signal MODE indicating the determination result, which is then supplied to the image sensor 200. The mode signal MODE is an example of a control signal as described in the claims.

[0155] Figure 26 is a block diagram showing an example configuration of the control unit 220 in a fifth embodiment of the present technology. The control unit 220 in this fifth embodiment differs from the first embodiment in that it includes a sensor-side control unit 224 instead of a mode determination unit 222.

[0156] In the fifth embodiment, the dark current amount estimation unit 221 estimates the dark current I ｅｓｔ This is supplied to the application processor 110 via the input / output interface 290.

[0157] Furthermore, the sensor-side control unit 224 receives a mode signal MODE from the application processor 110 via the input / output interface 290. The sensor-side control unit 224 controls each of the circuits within the image sensor 200 based on the mode indicated by the mode signal MODE.

[0158] Figure 27 is a block diagram showing an example configuration of an application processor 110 in a fifth embodiment of this technology. The application processor 110 in this fifth embodiment includes a mode determination unit 113 and an image processing unit 112.

[0159] The mode determination unit 113 receives an estimated value of dark current I via the input / output interface 290. ｅｓｔ The system receives this value and determines the mode based on it. The mode determination unit 113 generates a mode signal MODE indicating the determination result and supplies it to the image sensor 200 via the input / output interface 290.

[0160] Furthermore, the mode determination unit 113 controls each device within the image sensor 200, such as devices 131 and 132, based on the determination result. In low-power mode, the mode determination unit 113 reduces, for example, the frame rate of the image sensor 200 or the data size of the frame. Also, in low-power mode, the mode determination unit 222 can stop at least one of the devices, such as the image sensor 200, device 131, or device 132. For example, if the imaging system 100 is a two-lens system, and the image sensor 200 and its external device 131 each capture frames individually, one of them can be stopped.

[0161] Although the application processor 110 determines the mode, the configuration is not limited to this. Any device outside the image sensor 200, other than the application processor 110, can also determine the mode.

[0162] Furthermore, the fifth embodiment can be applied to the second, third, and fourth embodiments, as well as to variations of the fourth embodiment.

[0163] Thus, according to the fifth embodiment of this technology, since the mode is determined outside the image sensor 200, external devices of the image sensor 200 can also be controlled, thereby reducing the overall power consumption of the imaging system 100.

[0164] <6. Sixth Embodiment> In the first embodiment described above, only one thermometer 230 was provided inside the image sensor 200. However, due to the devices surrounding the image sensor 200 and the circuit arrangement directly below the pixel array 240, the temperature distribution of the pixel array 240 may not be uniform. Due to this bias in temperature distribution, the amount of dark current generated differs depending on the location and needs to be corrected. The image sensor 200 in this sixth embodiment differs from the first embodiment in that it converts the temperature measured by multiple thermometers arranged directly below the pixel array 240 into a dark current.

[0165] Figure 28 is a plan view showing an example of a thermometer arrangement in the sixth embodiment of this technology. The circuitry within the image sensor 200 is distributed across stacked upper chips (not shown) and lower chips (not shown), with the light-receiving side facing upwards. The upper chip contains a pixel array 240. On the lower chip, multiple thermometers, such as thermometers 231, 232, and 233, are arranged in a two-dimensional grid directly below the photosensitive area 242 of the pixel array 240. The number of rows of these thermometers is K (where K is an integer), and the number of columns is L (where L is an integer). The number of rows and columns of the thermometers is, for example, less than the number of rows and columns of the photosensitive pixels.

[0166] Figure 29 is a block diagram showing an example configuration of the signal processing unit 280 in the sixth embodiment of this technology. In this sixth embodiment, each of the thermometers such as thermometers 231, 232, and 233 supplies the measured temperature to the temperature-dark current conversion unit 281. The temperatures from the thermometers with k rows (where k is an integer from 1 to K) and l columns (where l is an integer from 1 to L) are converted to T ｋｌ Let's assume that.

[0167] The temperature-dark current conversion unit 281 converts temperature T ｋｌ Each of these measures temperature and dark current I ｄａｒｋ ｋｌ The dark current is converted and supplied to the dark current correction unit 282. The dark current correction unit 282 uses these dark currents to correct the pixel signals of the OPB pixels and photosensitive pixels. This allows for correction even if the dark current fluctuations differ depending on the position.

[0168] Each thermometer, such as thermometer 231, measures the temperature N times per frame, similar to the first embodiment. Alternatively, each thermometer can measure the temperature at its respective location only once per frame. In this case, a weighting processing unit 288 is provided, as illustrated in Figure 30. This weighting processing unit 288 weights the pixel signals of pixels near the measurement location using weights corresponding to the measured temperature, and supplies the weighted pixel signals to the average value calculation unit 283 and the clamping processing unit 285.

[0169] Furthermore, the sixth embodiment can also be applied to the second, third, and fourth embodiments, the modified version of the fourth embodiment, and the fifth embodiment, respectively.

[0170] Thus, according to the sixth embodiment of this technology, the dark current amount correction unit 282 corrects the pixel signal using the dark current amount obtained by converting the temperatures measured by multiple thermometers directly below the pixel array unit 240, so that even if the fluctuation of the dark current differs depending on the position, it can be corrected.

[0171] <7. Seventh Embodiment> In the first embodiment described above, the signal processing unit 280 performed clamping, but it can also perform white spot correction. The image sensor 200 in this seventh embodiment differs from the first embodiment in that it uses an estimated value of the dark current amount when correcting the white spot.

[0172] Figure 31 is a block diagram showing an example configuration of the signal processing unit 280 in the seventh embodiment of this technology. The signal processing unit 280 in this seventh embodiment differs from the first embodiment in that it further includes a white point correction unit 287. Note that in this figure, the temperature-dark current conversion unit 281, the dark current correction unit 282, and the average value holding unit 284 are omitted.

[0173] The white spot correction unit 287 detects white spots in the frame arranged from the pixel signals Do from the column circuit 260, corrects these white spots, and outputs the corrected pixel signal. The pixel signal Do after white spot correction for the OPB pixels. ＯＰＢ This is output to the average value calculation unit 283, and the pixel signal Do after white point correction of the photosensitive pixel is ｅｘｐ This is output to the clamping processing unit 285. In addition to the pixel signal Do from the column circuit 260, the estimated dark current I from the control unit 220 is also output to the white dot correction unit 287. ｅｓｔ The following is entered.

[0174] Figure 32 is a diagram illustrating a white spot correction method in a seventh embodiment of the present technology. The white spot correction unit 287 sequentially focuses on pixel signals within the frame and detects white spots based on whether the difference between the average value of the pixel signals in a predetermined area including the focused pixel signal and the value of the focused pixel signal exceeds a predetermined detection threshold Thw.

[0175] For example, an R (where R is an odd number) x C (where C is an odd number) region centered on the pixel signal of interest is used for white spot detection. The larger the dark current, the greater the noise, resulting in more white spots. Therefore, the estimated dark current I ｅｓｔ The larger the value of the region (in other words, the number of taps), the more preferable it is to increase the size of the region. Let R be "1", for example, and the estimated value I ｅｓｔ The larger the value of , the larger C will be. This region can also be called a "filter" or "kernel".

[0176] Estimated value I ｅｓｔ When the value is relatively small, as illustrated in figure a, the white point correction unit 287 refers to the pixel signals of a 1x3 region enclosed by a dotted line, for example, and calculates their average value. The pixel signal of interest is Do ３３ So, Do ３２ , Do ３３ and Do ３４ The average value is calculated. Then, the white point correction unit 287 uses that average value and the pixel signal Do of interest. ３３ The difference between the two values ​​is calculated, and it is determined whether or not that difference exceeds the detection threshold Thw. If the difference exceeds the detection threshold Thw, a white spot is detected. When a white spot is detected, the white spot correction unit 287 outputs the calculated average value signal as the pixel signal after white spot correction.

[0177] On the other hand, the estimated value I ｅｓｔ When the value is relatively large, as illustrated in figure b, the white point correction unit 287 refers to the pixel signals of a 1x5 region enclosed by a dotted line, for example, and calculates their average value. The pixel signal of interest is Do ３３ So, Do ３１ , Do ３２ , Do ３３ , Do ３４ and Do ３５ The average value is calculated. Then, the white point correction unit 287 uses that average value and the pixel signal Do of interest. ３３The difference between the two values ​​is calculated, and it is determined whether or not that difference exceeds the detection threshold Thw. If the difference exceeds the detection threshold Thw, a white spot is detected. When a white spot is detected, the white spot correction unit 287 outputs the calculated average value signal as the pixel signal after white spot correction.

[0178] As illustrated in figures a and b, by changing the size of the region (number of taps) to an optimal value according to the amount of dark current, it is possible to reduce unnecessary power consumption while improving the accuracy of white spot correction.

[0179] Furthermore, the seventh embodiment can also be applied to the second, third, and fourth embodiments, the modified version of the fourth embodiment, the fifth embodiment, and the sixth embodiment, respectively.

[0180] Thus, according to the seventh embodiment of this technology, the white spot correction unit 287 changes the size (number of taps) of the area used for correction according to the amount of dark current, thereby reducing unnecessary power consumption while improving the accuracy of white spot correction.

[0181] [Modification] In the seventh embodiment described above, the white spot correction unit 287 detected white spots based on whether the difference between the average value of the pixel signals in a predetermined area and the value of the pixel signal of interest exceeded a predetermined detection threshold Thw. However, the detection threshold Thw can also be changed according to the amount of dark current. The image sensor 200 in this modification of the seventh embodiment differs from the seventh embodiment in that it changes the detection threshold Thw according to the amount of dark current.

[0182] Figure 33 is a diagram illustrating a method for correcting white spots in a modified example of the seventh embodiment of the present technology.

[0183] The larger the dark current, the greater the noise, and therefore the larger the difference between the average value of the surroundings and the white spot. For this reason, in the modified version of the seventh embodiment, the estimated value of the dark current I ｅｓｔ The larger the value, the larger the detection threshold will be set to.

[0184] Estimated value I ｅｓｔWhen the estimated value I is relatively small, as exemplified by a in the figure, the white point correction unit 287 refers to the pixel signals in the 1-row×3-column area surrounded by the dotted line, for example, and calculates the average value of these pixel signals. Then, the white point correction unit 287 compares the average value with the pixel signal Do of interest ３３ and calculates the difference therebetween, and determines whether the difference exceeds the detection threshold Thw1. A white point is detected when the difference exceeds the detection threshold Thw1.

[0185] On the other hand, when the estimated value I ｅｓｔ is relatively large, as exemplified by b in the figure, the white point correction unit 287 refers to the pixel signals in the 1-row×3-column area surrounded by the dotted line, for example, and calculates the average value of these pixel signals. Then, the white point correction unit 287 compares the average value with the pixel signal Do of interest ３３ and calculates the difference therebetween, and determines whether the difference exceeds the detection threshold Thw2. Thw2 is a value larger than Thw1. A white point is detected when the difference exceeds the detection threshold Thw2.

[0186] Although the number of taps is fixed and the detection threshold is variable, the white point correction unit 287 can also change both the number of taps and the detection threshold according to the amount of dark current.

[0187] In addition, the modification of the seventh embodiment can also be applied to each of the second, third, and fourth embodiments, the modification of the fourth embodiment, the fifth embodiment, and the sixth embodiment.

[0188] As described above, according to the modification of the seventh embodiment of the present technology, since the white point correction unit 287 changes the detection threshold used for correction according to the amount of dark current, the white point correction accuracy can be improved.

[0189] <8. Application Example to Moving Bodies> The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure may be implemented as a device mounted on any type of moving body such as automobiles, electric vehicles, hybrid electric vehicles, motorcycles, bicycles, personal mobility devices, airplanes, drones, ships, robots, and the like.

[0190] Figure 34 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile control system to which the technology described herein may be applied.

[0191] The vehicle control system 12000 comprises a plurality of electronic control units connected via a communication network 12001. In the example shown in Figure 35, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an external information detection unit 12030, an internal information detection unit 12040, and an integrated control unit 12050. The functional configuration of the integrated control unit 12050 is shown in the figure, which includes a microcomputer 12051, an audio / image output unit 12052, and an in-vehicle network interface 12053.

[0192] The drivetrain control unit 12010 controls the operation of devices related to the vehicle's drivetrain according to various programs. For example, the drivetrain control unit 12010 functions as a control device for a drivetrain generating device that generates driving force for the vehicle, such as an internal combustion engine or a drive motor; a drivetrain transmission mechanism that transmits driving force to the wheels; a steering mechanism that adjusts the steering angle of the vehicle; and a braking device that generates braking force for the vehicle.

[0193] The body system control unit 12020 controls the operation of various devices mounted on the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window system, or various lamps such as headlights, reverse lights, brake lights, turn signals, or fog lights. In this case, the body system control unit 12020 may receive radio waves transmitted from a portable device that replaces a key or signals from various switches. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock system, power window system, lamps, etc.

[0194] The external information detection unit 12030 detects information from outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the external information detection unit 12030. The external information detection unit 12030 causes the imaging unit 12031 to capture images of the outside of the vehicle and receives the captured images. Based on the received images, the external information detection unit 12030 may perform object detection processing such as detecting people, cars, obstacles, signs, or characters on the road surface, or distance detection processing.

[0195] The imaging unit 12031 is a light sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output the electrical signal as an image or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0196] The in-vehicle information detection unit 12040 detects information inside the vehicle. The in-vehicle information detection unit 12040 is connected to, for example, a driver status detection unit 12041 that detects the driver's state. The driver status detection unit 12041 includes, for example, a camera that captures images of the driver, and the in-vehicle information detection unit 12040 may calculate the driver's level of fatigue or concentration, or determine whether the driver is drowsy, based on the detection information input from the driver status detection unit 12041.

[0197] The microcomputer 12051 can calculate control target values ​​for the drive force generator, steering mechanism, or braking device based on information inside and outside the vehicle acquired by the external information detection unit 12030 or the internal information detection unit 12040, and output control commands to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing ADAS (Advanced Driver Assistance System) functions, including collision avoidance or impact mitigation, following driving based on distance between vehicles, maintaining vehicle speed, vehicle collision warning, or vehicle lane departure warning.

[0198] Furthermore, the microcomputer 12051 can perform cooperative control for purposes such as autonomous driving, where the vehicle drives autonomously without driver intervention, by controlling the drive force generating device, steering mechanism, or braking device, etc., based on information about the vehicle's surroundings acquired by the external information detection unit 12030 or the internal information detection unit 12040.

[0199] Furthermore, the microcomputer 12051 can output control commands to the body system control unit 12020 based on external information acquired by the external information detection unit 12030. For example, the microcomputer 12051 can control the headlights according to the position of a preceding or oncoming vehicle detected by the external information detection unit 12030, and perform coordinated control aimed at reducing glare, such as switching from high beams to low beams.

[0200] The audio-image output unit 12052 transmits at least one of audio and image output signals to an output device capable of visually or audibly notifying information to the vehicle's occupants or to those outside the vehicle. In the example shown in Figure 34, the output devices are exemplified as an audio speaker 12061, a display unit 12062, and an instrument panel 12063. The display unit 12062 may include, for example, at least one of an onboard display and a head-up display.

[0201] Figure 35 shows an example of the installation position of the imaging unit 12031.

[0202] In Figure 35, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0203] The imaging units 12101, 12102, 12103, 12104, and 12105 are installed, for example, on the front nose, side mirrors, rear bumper, back door, and the upper part of the windshield inside the vehicle 12100. The imaging unit 12101 installed on the front nose and the imaging unit 12105 installed on the upper part of the windshield inside the vehicle mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 installed on the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 installed on the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 installed on the upper part of the windshield inside the vehicle is mainly used for detecting preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, or lanes.

[0204] Figure 35 shows an example of the imaging range of imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of imaging unit 12101 located on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of imaging units 12102 and 12103 located on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of imaging unit 12104 located on the rear bumper or back door. For example, by superimposing the image data captured by imaging units 12101 to 12104, an overhead view image of the vehicle 12100 can be obtained.

[0205] At least one of the imaging units 12101 to 12104 may have a function for acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera consisting of multiple image sensors, or an image sensor having pixels for phase difference detection.

[0206] For example, the microcomputer 12051, based on distance information obtained from the imaging units 12101 to 12104, can determine the distance to each object within the imaging range 12111 to 12114 and the temporal change of this distance (relative speed to the vehicle 12100). In particular, it can extract the closest object on the vehicle 12100's path that is traveling in approximately the same direction as the vehicle 12100 at a predetermined speed (e.g., 0 km / h or more) as the preceding vehicle. Furthermore, the microcomputer 12051 can set a predetermined distance to be maintained before the preceding vehicle and perform automatic braking control (including follow-and-stop control) and automatic acceleration control (including follow-and-start control), etc. In this way, cooperative control aimed at autonomous driving, where the vehicle drives autonomously without driver intervention, can be performed.

[0207] For example, the microcomputer 12051 can use distance information obtained from imaging units 12101 to 12104 to classify and extract three-dimensional object data related to three-dimensional objects, such as motorcycles, passenger cars, large vehicles, pedestrians, utility poles, and other three-dimensional objects, and use this data for automatic obstacle avoidance. For example, the microcomputer 12051 identifies obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines the collision risk, which indicates the degree of risk of collision with each obstacle. If the collision risk is above a set value and there is a possibility of collision, the microcomputer 12051 can provide driving assistance to avoid collisions by outputting a warning to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or evasive steering via the drive system control unit 12010.

[0208] At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared light. For example, the microcomputer 12051 can recognize pedestrians by determining whether or not pedestrians are present in the images captured by the imaging units 12101 to 12104. Such pedestrian recognition is performed, for example, by a procedure to extract feature points from the images captured by the imaging units 12101 to 12104 as infrared cameras, and a procedure to perform pattern matching on a series of feature points that indicate the contour of an object to determine whether or not it is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the imaging units 12101 to 12104 and recognizes a pedestrian, the audio-image output unit 12052 controls the display unit 12062 to superimpose a rectangular contour line for emphasis on the recognized pedestrian. The audio-image output unit 12052 may also control the display unit 12062 to display an icon indicating a pedestrian at a desired position.

[0209] The above describes an example of a vehicle control system to which the technology described herein may be applied. The technology described herein can be applied to, for example, the imaging unit 12031 of the configuration described above. Specifically, the image sensor 200 in Figure 1 can be applied to the imaging unit 12031. By applying the technology described herein to the imaging unit 12031, the accuracy of black level correction can be improved, and a more easily viewable captured image can be obtained, thereby reducing driver fatigue.

[0210] The embodiments described above are merely examples of how to realize this technology, and there is a corresponding relationship between the matters in the embodiments and the inventive features in the claims. Similarly, there is a corresponding relationship between the inventive features in the claims and the matters in the embodiments of this technology that bear the same name. However, this technology is not limited to the embodiments and can be realized by making various modifications to the embodiments without departing from the gist of the technology.

[0211] The effects described herein are merely illustrative and not limited to those described herein, and other effects may also occur.

[0212] Furthermore, this technology can also be configured as follows: (1) An imaging system comprising a pixel array unit in which light-shielded OPB (optical black) pixels and light-shielded photosensitive pixels are arranged, and a dark current amount estimation unit that estimates the amount of dark current in the pixel array unit based on the exposure time and the temperature measured at a timing synchronized with a predetermined vertical synchronization signal, and outputs it as an estimated value. (2) The imaging system according to (1), further comprising: a conversion unit that converts each of the temperatures measured at multiple timings during the exposure period into a dark current amount and outputs it as a temperature measurement point dark current amount; a dark current amount correction unit that corrects the pixel signals from the OPB pixel and the photosensitive pixel using the temperature measurement point dark current amount; and a mode determination unit that transitions to a predetermined normal mode if the estimated value is within a range from a predetermined first threshold to a predetermined second threshold, and transitions to a low-power mode with lower power consumption than the normal mode if the estimated value is less than the first threshold, wherein the dark current correction unit comprises: an average value calculation unit that calculates the average value of the pixel signals from the OPB pixel when transitioning from the low-power mode to another mode; an average value holding unit that holds the average value; and a clamping processing unit that corrects the pixel signals from the photosensitive pixel based on the average value calculated by the average value calculation unit when transitioning to another mode, and corrects the pixel signals based on the average value held by the average value holding unit when transitioning to the low-power mode. (3) The imaging system according to (2), further comprising an analog clamp circuit that performs analog clamping when switching from the low-power mode to another mode. (4) The imaging system according to (2) or (3), wherein the low-power mode is the default mode. (5) The imaging system according to any one of (2) to (4), wherein the average value holding unit holds the average value for each driving mode of the image sensor. (6) The imaging system according to any one of (2) to (5), wherein the low-power mode includes a first low-power mode and a second low-power mode with different power consumption, and in the second low-power mode, only a portion of a predetermined number of OPB rows in which the OPB pixels are arranged is read.(7) The imaging system according to any one of (2) to (6), further comprising a sensor signal processing unit that determines whether the scene has changed based on a signal from a ToF sensor or EVS and supplies the determination result to the mode determination unit. (8) The imaging system according to any one of (2) to (6), wherein the photosensitive pixel includes a phase difference pixel for detecting a phase difference, and the mode determination unit determines whether the scene has changed based on a defocus amount obtained by converting the phase difference signal from the phase difference pixel, and transitions from the low power mode to the normal mode if the scene has changed. (9) The imaging system according to (1), further comprising: a conversion unit that converts each of the temperatures measured at multiple timings within the exposure period into a dark current amount and outputs it as a temperature measurement point dark current amount when the estimated value is greater than a predetermined second threshold; a dark current amount correction unit that corrects the pixel signals from the OPB pixel and the photosensitive pixel using the temperature measurement point dark current amount; and an average value calculation unit that calculates the average value of the corrected OPB pixel's pixel signal. (10) The imaging system according to (9), further comprising a plurality of thermometers arranged directly below the photosensitive area of ​​the pixel array with the light-receiving side facing upward, wherein the conversion unit converts the temperature measured by each of the plurality of thermometers into the dark current amount. (11) The imaging system according to (1), further comprising a plurality of thermometers arranged directly below the photosensitive area of ​​the pixel array with the light-receiving side facing upward, and a weighting processing unit that weights the pixel signals of the OPB pixels and the photosensitive pixels according to the weights corresponding to the temperatures measured by each of the plurality of thermometers. (12) The imaging system according to any one of (1) to (11), further comprising: a signal processing unit that corrects the pixel signal from the photosensitive pixel based on the average value of the pixel signal from the OPB pixel; and a mode determination unit that transitions to one of a plurality of different modes of correction operation of the signal processing unit based on the estimated value, wherein the pixel array unit, the dark current estimation unit and the signal processing unit are arranged on an image sensor, the mode determination unit is arranged on an application processor, and the image sensor outputs the dark current estimation result to the application processor.(13) The imaging system according to any one of (12), wherein the application processor controls the image sensor and an external device of the image sensor based on the determination result of the mode. (14) The imaging system according to any one of (12) or (13), wherein the application processor supplies a control signal to the image sensor for controlling the black level correction operation of the image sensor. (15) The imaging system according to any one of (1) to (13), further comprising a white point correction unit that corrects white points based on pixel signals in a predetermined area including a pixel signal of interest from the pixel signals of the photosensitive pixels, wherein the size of the area is set to a larger value as the estimated value increases. (16) The imaging system according to any one of (1) to (15), further comprising a white point correction unit that detects and corrects white points based on whether the difference between the average value of pixel signals in a predetermined area including a pixel signal of interest from the pixel signals of the photosensitive pixels and the pixel signal of interest exceeds a predetermined detection threshold, wherein the detection threshold is set to a larger value as the estimated value increases. (17) An image sensor comprising a pixel array section having light-shielded OPB pixels and light-free photosensitive pixels arranged in a row, and a dark current amount estimation section that estimates the dark current amount of the pixel array section based on the exposure time and the temperature measured at a timing synchronized with a predetermined vertical synchronization signal, and outputs it as an estimated value. (18) A control method for an imaging system comprising a dark current amount estimation procedure that estimates the dark current amount of a pixel array section having light-shielded OPB pixels and light-free photosensitive pixels arranged in a row, based on the exposure time and the temperature measured at a timing synchronized with a predetermined vertical synchronization signal, and outputs it as an estimated value.

[0213] 100 Imaging system 110 Application processor 111 Sensor signal processing unit 112 Image processing unit 113, 222 Mode determination unit 120 ToF sensor 131, 132 Device 200 Image sensor 210 Vertical scanning circuit 220 Control unit 221 Dark current amount estimation unit 223 Temperature measurement control unit 224 Sensor side control unit 230, 231, 232, 233 Thermometer 240 Pixel array unit 241 VOPB area 242 Photosensitive area 243 OPB pixel 244 Photosensitive pixel 244-1 Effective pixel 244-2 Phase difference pixel 250 Peripheral circuit 251 DAC 252 Reference level generation circuit 260 Column circuit 261 Load MOS current source 262 Analog clamp circuit 270 ADC 271 Comparator 272 Counter 280 Signal Processing Unit 281 Temperature-Dark Current Conversion Unit 282 Row-by-Row Dark Current Correction Unit 283 Average Value Calculation Unit 284 Average Value Holding Unit 285 Clamping Unit 286 Focus Detection Unit 287 White Point Correction Unit 288 Weighting Unit 290 Input / Output Interface 12031 Imaging Unit

Claims

1. An imaging system comprising: a pixel array section having light-shielded OPB (optical black) pixels and light-shielded photosensitive pixels arranged in a row; and a dark current estimation section that estimates the amount of dark current in the pixel array section based on the exposure time and the temperature measured at a timing synchronized with a predetermined vertical synchronization signal, and outputs the estimated value.

2. The imaging system according to claim 1, further comprising: a conversion unit that converts each of the temperatures measured at multiple timings during the exposure period into a dark current amount and outputs it as a dark current amount at a temperature measurement point; a dark current amount correction unit that corrects the pixel signals from the OPB pixel and the photosensitive pixel using the dark current amount at the temperature measurement point; and a mode determination unit that transitions to a predetermined normal mode if the estimated value is within a range from a predetermined first threshold to a predetermined second threshold, and transitions to a low-power mode with lower power consumption than the normal mode if the estimated value is less than the first threshold, wherein the dark current correction unit further comprises: an average value calculation unit that calculates the average value of the pixel signals from the OPB pixel when transitioning from the low-power mode to another mode; an average value holding unit that holds the average value; and a clamping processing unit that corrects the pixel signals from the photosensitive pixel based on the average value calculated by the average value calculation unit when transitioning to another mode, and corrects the pixel signals based on the average value held by the average value holding unit when transitioning to the low-power mode.

3. The imaging system according to claim 2, further comprising an analog clamp circuit that performs analog clamping when switching from the low-power mode to another mode.

4. The imaging system according to claim 2, wherein the low-power mode is the default mode.

5. The imaging system according to claim 2, wherein the average value holding unit holds the average value for each driving mode of the image sensor.

6. The imaging system according to claim 2, wherein the low-power mode comprises a first low-power mode and a second low-power mode with different power consumption, and in the second low-power mode, only a portion of a predetermined number of OPB rows in which the OPB pixels are arranged is read out.

7. The imaging system according to claim 2, further comprising a sensor signal processing unit that determines whether or not a scene has changed based on a signal from a Time of Flight (ToF) sensor or an Event-based Vision Sensor (EVS), and supplies the determination result to the mode determination unit.

8. The imaging system according to claim 2, wherein the photosensitive pixel includes a phase difference pixel for detecting a phase difference, and the mode determination unit determines whether the scene has changed based on the amount of defocus obtained by converting the phase difference signal from the phase difference pixel, and if the scene has changed, it transitions from the low power mode to the normal mode.

9. The imaging system according to claim 1, further comprising: a conversion unit that converts each of the temperatures measured at multiple timings within the exposure period into dark current amounts and outputs them as temperature measurement point dark current amounts when the estimated value is greater than a predetermined second threshold; a dark current amount correction unit that corrects the pixel signals from the OPB pixels and the photosensitive pixels using the temperature measurement point dark current amounts; and an average value calculation unit that calculates the average value of the pixel signals of the OPB pixels.

10. The imaging system according to claim 9, further comprising a plurality of thermometers arranged directly below the photosensitive area of ​​the pixel array with the light-receiving side facing upward, wherein the conversion unit converts the temperature measured by each of the plurality of thermometers into the dark current amount.

11. The imaging system according to claim 1, further comprising: a plurality of thermometers arranged directly below the photosensitive area of ​​the pixel array with the light-receiving side facing upward; and a weighting processing unit that weights the pixel signals of the OPB pixels and the photosensitive pixels according to the weights corresponding to the temperatures measured by each of the plurality of thermometers.

12. The imaging system according to claim 1, further comprising: a signal processing unit that corrects the pixel signals from the photosensitive pixels based on the average value of the pixel signals from the OPB pixels; and a mode determination unit that transitions to one of a plurality of different modes of correction operation of the signal processing unit based on the estimated value, wherein the pixel array unit, the dark current estimation unit, and the signal processing unit are arranged on an image sensor, the mode determination unit is arranged on an application processor, and the image sensor outputs the dark current estimation result to the application processor.

13. The imaging system according to claim 12, wherein the application processor controls the image sensor and an external device of the image sensor based on the determination result of the mode.

14. The imaging system according to claim 12, wherein the application processor supplies a control signal to the image sensor for controlling the black level correction operation of the image sensor.

15. The imaging system according to claim 1, further comprising a white point correction unit that corrects the white point based on the pixel signals within a predetermined region that includes the pixel signal of interest from the pixel signals of the photosensitive pixels, wherein the size of the region is set to a larger value as the estimated value increases.

16. The imaging system according to claim 1, further comprising a white spot correction unit that detects and corrects white spots based on whether the difference between the average value of pixel signals within a predetermined region including the pixel signal of interest from the photosensitive pixels and the pixel signal of interest exceeds a predetermined detection threshold, wherein the detection threshold is set to a larger value as the estimated value increases.

17. An image sensor comprising a pixel array section having light-shielded OPB pixels and light-free photosensitive pixels arranged in a row, and a dark current estimation section that estimates the amount of dark current in the pixel array section based on the exposure time and the temperature measured at a timing synchronized with a predetermined vertical synchronization signal, and outputs the estimated value.

18. A control method for an imaging system comprising a dark current estimation procedure that estimates the dark current amount of a pixel array, which is arranged in a pixel array portion having light-shielded OPB pixels and light-free photosensitive pixels, based on the exposure time and the temperature measured at a timing synchronized with a predetermined vertical synchronization signal, and outputs the estimated value.

Images