Image sensor, photodetection device, and image sensor control method

Abstract

The present invention shortens the time before appropriate exposure is obtained in an image sensor having an Auto Exposure (AE) function.　A predetermined number of photometric pixels and a predetermined number of normal pixels are arranged in a pixel array unit. A comparator compares a first pixel signal from the photometric pixel with a predetermined reference signal and outputs a comparison result. An exposure determination unit performs photometry on the basis of the comparison result, and determines whether or not a photometric amount exceeds a predetermined threshold. A vertical drive unit starts exposure of the normal pixel and the photometric pixel, and also drives the photometric pixel to output the first pixel signal. When the photometric amount exceeds the threshold, the vertical drive unit ends the exposure and drives the normal pixel to output a second pixel signal.

Description

Image sensor, photodetection device, and method for controlling image sensor

[0001] The present technology relates to an image sensor, and more particularly to an image sensor for controlling exposure, a light detection device, and a method for controlling an image sensor.

[0002] Conventionally, imaging devices and the like have been equipped with an AE (Auto Exposure) function that controls exposure so that the brightness of an image is appropriate. For example, a device has been proposed that acquires brightness from a first preliminary captured frame, calculates an appropriate exposure from the brightness, and captures a second frame (see, for example, Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2014-216975

[0004] In the above-mentioned conventional technology, the AE function is realized by obtaining the brightness from a pre-captured frame and calculating the appropriate exposure period and aperture from that brightness. However, the above-mentioned conventional technology has a problem in that it requires a delay of one frame to obtain the appropriate exposure. This delay may cause the AE function to be unable to keep up when the brightness fluctuates significantly between two frames.

[0005] This technology was developed in light of these circumstances, and aims to shorten the time it takes to obtain the correct exposure in an image sensor with an AE function.

[0006] The present technology has been made to solve the above-mentioned problems, and a first aspect thereof is an image sensor and a control method thereof, comprising: a pixel array unit in which a predetermined number of photometric pixels and a predetermined number of normal pixels are arranged; a comparator that compares a first pixel signal from the photometric pixels with a predetermined reference signal and outputs the comparison result; an exposure determination unit that performs photometry based on the comparison result and determines whether the photometric amount exceeds a predetermined threshold; and a vertical drive unit that starts exposure of the normal pixels and the photometric pixels and drives the photometric pixels to output the first pixel signal, and if the photometric amount exceeds the threshold, ends the exposure and drives the normal pixels to output a second pixel signal, thereby shortening the time required to obtain an appropriate exposure.

[0007] In addition, in this first aspect, a predetermined number of the photometric pixels may be arranged for each column in the pixel array section, thereby providing an effect that photometric samples can be obtained for each column.

[0008] In the first aspect, the pixel array section may include rows in which the photometric pixels and the normal pixels are arranged, thereby providing an effect that the photometric pixels are arranged in a dispersed manner.

[0009] In the first aspect, the pixel array section may include a photometric row in which the photometric pixels are arranged and a plurality of normal rows in which the normal pixels are arranged, thereby improving image quality.

[0010] In addition, in this first aspect, the pixel array unit may include a plurality of photometric rows in which the photometric pixels are arranged and a plurality of normal rows in which the normal pixels are arranged, thereby providing an effect of increasing the number of photometric samples.

[0011] In addition, in this first aspect, the pixel array section may be wired with first drive lines connected to the photometric pixels and transmitting first drive signals, and second drive lines connected to the normal pixels and transmitting second drive signals, thereby providing the effect of independently controlling the normal pixels and the photometric pixels.

[0012] In addition, in this first aspect, the pixel circuit may further include a counter that counts a count value over a period until the comparison result is inverted and outputs a digital signal indicating the count value, the comparator compares the second pixel signal with the reference signal when the normal pixel is driven, and the counter and the comparator may be disposed within an analog-to-digital converter, thereby producing an effect of converting the pixel signal into a digital signal.

[0013] In addition, in this first aspect, the image sensor may further include an interpolation processing unit that interpolates a digital signal corresponding to the first pixel signal, the analog-to-digital converter converts the first pixel signal into the digital signal when the photometric amount exceeds the threshold, and the vertical drive unit starts exposure of the normal pixels in a predetermined row simultaneously with the photometric pixels, thereby eliminating the need to read out pixel signals from the photometric pixels.

[0014] In addition, in this first aspect, the analog-to-digital converter may convert the first and second pixel signals into the digital signals in sequence when the photometric amount exceeds the threshold, and the vertical drive unit may start exposing the normal pixels in a predetermined row when a predetermined time has elapsed since the start of exposure of the photometric pixels, thereby eliminating or reducing the need for interpolation of the photometric pixels.

[0015] In addition, in this first aspect, the level of the first pixel signal may include a first reset level when the floating diffusion layer is initialized and a first signal level when charge is transferred, and the level of the second pixel signal may include a second reset level when the floating diffusion layer is initialized and a second signal level when charge is transferred, and the analog-to-digital converter may convert into the digital signal in the order of the first signal level, the first reset level, the second reset level, and the second signal level. This brings about an effect that CDS processing is performed for each pixel.

[0016] According to a second aspect of the present technology, there is provided a photodetection device including: a pixel array unit in which a predetermined number of photometric pixels and a predetermined number of normal pixels are arranged; a comparator that compares a first pixel signal from the photometric pixels with a predetermined reference signal and outputs the comparison result; an exposure determination unit that performs photometry based on the comparison result and determines whether a photometric amount exceeds a predetermined threshold; a vertical drive unit that starts exposure of the normal pixels and the photometric pixels and drives the photometric pixels to output the first pixel signal, and, if the photometric amount exceeds the threshold, ends the exposure and drives the normal pixels to output a second pixel signal; and an image processing unit that processes image data generated from the second pixel signal. This provides the effect of shortening the time required to obtain an appropriate exposure in the photodetection device.

[0017] 1 is a block diagram showing an example configuration of an imaging device according to a first embodiment of the present technology. FIG. 2 is a block diagram showing an example configuration of an image sensor according to the first embodiment of the present technology. FIG. 3 is a circuit diagram showing an example configuration of a pixel according to the first embodiment of the present technology. FIG. 4 is a block diagram showing an example configuration of a column signal processing unit according to the first embodiment of the present technology. FIG. 5 is a block diagram showing an example configuration of an exposure determination unit according to the first embodiment of the present technology. FIG. 6 is a diagram for explaining operation of an image sensor according to the first embodiment of the present technology. FIG. 7 is an example histogram showing the luminance distribution of a general natural image. FIG. 8 is an example timing chart showing operation of an image sensor in a comparative example. FIG. 9 is a timing chart showing an example operation of an image sensor according to the first embodiment of the present technology. FIG. 10 is a timing chart showing an example operation during photometry of an image sensor according to a first embodiment of the present technology. FIG. 11 is a timing chart showing an example read operation of an image sensor according to the first embodiment of the present technology. FIG. 12 is a timing chart showing an example operation of a DAC (Digital to Analog Converter) and an ADC (Analog to Digital Converter) according to the first embodiment of the present technology. FIG. 13 is an example state transition diagram of an image sensor according to the first embodiment of the present technology. FIG. 14 is a timing chart showing an example operation during photometry of an image sensor according to a modified example of the first embodiment of the present technology. 10 is a timing chart showing an example of a readout operation of an image sensor in a modified example of the first embodiment of the present technology. FIG. 11 is a block diagram showing an example of a configuration of an image processing unit in a modified example of the first embodiment of the present technology. FIG. 12 is a diagram showing an example of a photometry row in a second embodiment of the present technology. FIG. 13 is a diagram showing another example of a photometry row in the second embodiment of the present technology. FIG. 14 is a timing chart showing an example of an operation during photometry of an image sensor in the second embodiment of the present technology. FIG. 15 is a timing chart showing an example of a readout operation of an image sensor in the second embodiment of the present technology. FIG. 16 is an example of a state transition diagram of an image sensor in the second embodiment of the present technology. FIG. 17 is a plan view showing an example of a configuration of a pixel array unit in a modified example of the second embodiment of the present technology. FIG. 18 is a diagram showing an example of a photometry row in a third embodiment of the present technology.10 is a timing chart showing an example of an operation during photometry of an image sensor according to a third embodiment of the present technology. FIG. 11 is a timing chart showing an example of a readout operation of an image sensor according to the third embodiment of the present technology. FIG. 12 is an example of a state transition diagram of an image sensor according to the third embodiment of the present technology. FIG. 13 is a block diagram showing an example of a schematic configuration of a vehicle control system. FIG. 14 is an explanatory diagram showing an example of installation positions of an outside-vehicle information detection unit and an imaging unit.

[0018] Hereinafter, modes for carrying out the present technology (hereinafter referred to as embodiments) will be described. The description will be given in the following order: 1. First embodiment (an example in which exposure of normal pixels and photometric pixels is started, and exposure is ended when the photometric amount exceeds a threshold) 2. Second embodiment (an example in which exposure of multiple normal rows and one photometric row is started, and exposure is ended when the photometric amount exceeds a threshold) 3. Third embodiment (an example in which exposure of multiple normal rows and multiple photometric rows is started, and exposure is ended when the photometric amount exceeds a threshold) 4. Example of application to a moving body

[0019] 1. First Embodiment [Configuration Example of Imaging Device] Fig. 1 is a block diagram showing a configuration example of an imaging device 100 according to a first embodiment of the present technology. The imaging device 100 is a device that captures image data, and includes an imaging lens 110, an image sensor 200, a SoC (System-on-a-Chip) 120, an imaging control unit 130, and a recording unit 140. The imaging device 100 is assumed to be a digital camera or an electronic device with an imaging function (such as a smartphone or a personal computer). Note that the imaging device 100 is an example of a light detection device as defined in the claims.

[0020] The image sensor 200 captures image data under the control of the imaging control unit 130. The image sensor 200 supplies the image data as frames to the SoC 120 via a signal line 209.

[0021] The imaging lens 110 collects light and guides it to the image sensor 200. The imaging control unit 130 controls the image sensor 200 to capture image data (i.e., frames). The imaging control unit 130 supplies imaging control signals including, for example, a vertical synchronization signal VSYNC to the image sensor 200 via a signal line 139.

[0022] The SoC 120 performs various processes on the frames, such as image recognition, and supplies the processed data to the recording unit 140.

[0023] Here, the vertical synchronization signal VSYNC is a signal that indicates the timing of imaging, and a periodic signal with a fixed frequency (such as 60 Hz) is used as the vertical synchronization signal VSYNC.

[0024] Note that the imaging device 100 may transmit the frames to an external device. In this case, an external interface for transmitting the frames is further provided. The imaging device 100 may also display the frames. In this case, a display unit is further provided.

[0025] 2 is a block diagram showing an example of the configuration of an image sensor 200 according to the first embodiment of the present technology. The image sensor 200 includes a vertical drive unit 210, a timing control unit 220, a DAC 230, a pixel array unit 240, a column signal processing unit 300, an image processing unit 270, and an output interface 280.

[0026] A plurality of pixels are arranged in a two-dimensional grid pattern in the pixel array unit 240. Some of these pixels are used for photometry and are referred to as photometric pixels 260. The remaining pixels are referred to as normal pixels 250. In the figure, the hatched areas indicate the photometric pixels 260, and the gray areas indicate the normal pixels 250.

[0027] The vertical driving section 210 drives the pixels in units of rows and outputs analog signals as pixel signals to the column signal processing section 300. The control of the vertical driving section 210 will be described in detail later.

[0028] The timing control section 220 controls the operation timing of the vertical drive section 210, the DAC 230, the column signal processing section 300, and the image processing section 270 in synchronization with the vertical synchronization signal VSYNC. The timing control section 220 also generates a horizontal synchronization signal HSYNC having a higher frequency than the vertical synchronization signal VSYNC and supplies it to the vertical drive section 210. Rows are selected in order in synchronization with this horizontal synchronization signal HSYNC.

[0029] The DAC 230 generates a predetermined reference signal by DA (Digital to Analog) conversion and supplies the reference signal to the column signal processing unit 300. Details of the reference signal will be described later.

[0030] The column signal processing unit 300 performs signal processing, including AD (Analog to Digital) conversion, on pixel signals of each column. The column signal processing unit 300 supplies the processed digital signals to the image processing unit 270. The column signal processing unit 300 also determines whether the exposure is appropriate and supplies the determination result to the timing control unit 220.

[0031] The image processing unit 270 performs various processes, including CDS (Correlated Double Sampling), on the digital signals from the column signal processing unit 300. The image processing unit 270 supplies frames in which the processed digital signals are arranged to the output interface 280. Note that at least a part of the processing by the image processing unit 270 can also be performed by a circuit external to the image sensor 200, such as the SoC 120.

[0032] The output interface 280 outputs the frame to an external device such as the SoC 120 .

[0033] 3 is a circuit diagram showing an example of a pixel configuration according to the first embodiment of the present technology. As described above, the pixel array unit 240 has a plurality of pixels arranged therein, including photometric pixels 260 and normal pixels 250. In addition to the photometric pixels 260 and normal pixels 250, pixels such as OB (optical black) pixels can also be arranged.

[0034] The normal pixel 250 includes a photoelectric conversion element 251, a transfer transistor 252, a reset transistor 253, a floating diffusion layer 254, an amplification transistor 255, and a selection transistor 256. The photometric pixel 260 includes a photoelectric conversion element 261, a transfer transistor 262, a reset transistor 263, a floating diffusion layer 264, an amplification transistor 265, and a selection transistor 266. For example, nMOS (n-channel Metal Oxide Semiconductor) transistors are used as the transistors in these pixels.

[0035] Furthermore, vertical signal lines VSL are wired along the column direction for each column in the pixel array unit 240. The number of columns is N (N is an integer), and the vertical signal line for the nth column (n is an integer from 1 to N) is denoted by VSLn.

[0036] The number of photometric pixels 260 is, for example, equal to or less than the number of vertical signal lines VSL (i.e., N). Because these photometric pixels 260 are used as samples in photometry, they are preferably arranged so that they are evenly distributed within the pixel array section 240. For example, one photometric pixel 260 is arranged per column, and two photometric pixels 260 are arranged in each even-numbered row. The remaining pixels in these even-numbered rows are normal pixels 250, and the number of these is N-2. On the other hand, N normal pixels 250 are arranged in odd-numbered rows.

[0037] Note that, as long as they are distributed evenly, the arrangement of the photometric pixels 260 is not limited to the example shown in the figure. For example, it is also possible to arrange one photometric pixel 260 per column and one photometric pixel 260 per row. It is also possible to arrange one photometric pixel 260 every two columns and one photometric pixel 260 per row.

[0038] Furthermore, when driving a row in which a photometric pixel 260 is arranged, the vertical drive unit 210 is capable of independently driving the photometric pixel 260 and the normal pixels 250 in that row. To achieve this, for example, in an even-numbered row, drive lines 241-ae, 242-ae, and 243-ae connected to the photometric pixels 260 in that row, and drive lines 241-n, 242-n, and 243-n connected to the normal pixels 250 in that row are wired in the horizontal direction. In addition, in odd-numbered rows, drive lines 241, 242, and 243 are wired in the horizontal direction.

[0039] Focus will be given on odd-numbered rows in which only normal pixels 250 are arranged. In these odd-numbered rows, photoelectric conversion elements 251 generate electric charges (electrons, etc.) through photoelectric conversion. Transfer transistors 252 transfer the electric charges from the photoelectric conversion elements 251 to the floating diffusion layers 254 in accordance with a drive signal TRG from the vertical drive unit 210. This drive signal TRG is transmitted via drive lines 242. Furthermore, let the number of rows be M (M is an integer), and let the drive signal TRG for the mth row (m is an integer from 1 to M) be TRGm.

[0040] The reset transistor 253 extracts charge from the floating diffusion layer 254 to initialize it in accordance with a drive signal RST from the vertical drive section 210. This drive signal RST is transmitted via a drive line 243. The drive signal RST for the m-th row is denoted as RSTm.

[0041] The floating diffusion layer 254 accumulates electric charges and generates a voltage corresponding to the amount of electric charges. The amplification transistor 255 forms a source follower circuit and outputs an analog signal having a level corresponding to the voltage of the floating diffusion layer 254 connected to its gate to the selection transistor 256 connected to its source. The drain of the amplification transistor 255 is connected to the power supply voltage VDD.

[0042] The selection transistor 256 outputs the analog signal from the amplification transistor 255 as a pixel signal to the vertical signal line VSL in accordance with the drive signal SEL from the vertical drive unit 210. This drive signal SEL is transmitted via the drive line 241. The drive signal SEL for the m-th row is denoted as SELm.

[0043] Next, attention is focused on the even-numbered rows. The drive signals for the normal pixels 250 in the even-numbered rows are assumed to be TRG_n, RST_n, and SEL_n. These signals are transmitted via drive lines 242_n, 243_n, and 241_n.

[0044] The circuit configuration of the photometric pixels 260 in the even-numbered rows is the same as that of the normal pixels 250. However, the drive signals to the photometric pixels 260 are TRG_ae, RST_ae, and SEL_ae. These are transmitted via drive lines 242_ae, 243_ae, and 241_ae.

[0045] Hereinafter, the level of the pixel signal when the floating diffusion layer is initialized in the normal pixel 250 or the photometric pixel 260 will be referred to as the "reset level" or "P-phase level." Also, the level of the pixel signal when the charge is transferred within the pixel will be referred to as the "signal level" or "D-phase level."

[0046] The drive signals TRG, RST, SEL, TRG_n, RST_n, and SEL_n sent to the normal pixels 250 are an example of a first drive signal as defined in the claims. The drive signals TRG_ae, RST_ae, and SEL_ae sent to the photometering pixels 260 are an example of a second drive signal as defined in the claims. The drive lines 241, 242, 243, 241_n, 242_n, and 243_n are an example of a first drive line as defined in the claims. The drive lines 241_ae, 242_ae, and 243_ae are an example of a second drive line as defined in the claims.

[0047] Furthermore, the circuit configuration of the normal pixel 250 and the photometric pixel 260 is not limited to the example shown in the figure, as long as it is capable of generating pixel signals.

[0048] 4 is a block diagram showing an example of the configuration of the column signal processing unit 300 according to the first embodiment of the present technology. The column signal processing unit 300 includes a plurality of ADCs 310, a plurality of memories 320, and an exposure determination unit 330. The ADCs 310 and the memories 320 are arranged for each column.

[0049] The ADC 310 converts analog pixel signals from the corresponding columns into digital signals. The ADC 310 is a single-slope ADC and includes a comparator 311 and a counter 312.

[0050] The comparator 311 compares the reference signal RMP from the DAC 230 with the pixel signal of the corresponding column. The inverting input terminal (-) of this comparator 311 is connected to the DAC 230, and the non-inverting input terminal (+) is connected to the vertical signal line VSL. The comparator 311 outputs the comparison result CMP to the counter 312 and the exposure determination unit 330. The comparison result CMP for the nth column is designated CMPn.

[0051] The counter 312 counts the count value over a period until the comparison result CMP is inverted, and outputs a digital signal indicating the count value CNT to the memory 320. The count value CNT of the n-th column is denoted as CNTn.

[0052] The memory 320 stores digital signals. If the data size of the digital signals is P bits (P is an integer), the memory 320 includes, for example, P flip-flops. The digital signals of each column are output in sequence to the image processing unit 270 by serial transfer.

[0053] The exposure determination unit 330 performs photometry based on the comparison result CMP for each column and determines whether the photometric amount exceeds a predetermined threshold value. The exposure determination unit 330 supplies the determination result to the timing control unit 220.

[0054] The exposure determination unit 330 has a simple configuration that can be realized by a logic circuit, which will be described later. In a typical image sensor 200, a comparator 311 and a counter 312 are already provided, and the first embodiment can be realized by adding the simple exposure determination unit 330 to these circuits. Therefore, the development costs for the image sensor 200 of the first embodiment are not significantly different from those of the conventional image sensor.

[0055] 5 is a block diagram showing an example of the configuration of the exposure determination unit 330 according to the first embodiment of the present technology. The exposure determination unit 330 includes a plurality of inverters 331, a bit counter 332, a register 333, and a determination circuit 334. An inverter 331 is provided for each column.

[0056] The inverter 331 inverts the comparison result CMP of the corresponding column and outputs the inverted signal to the bit counter 332. Since the number of columns is N, an N-bit bit string consisting of inverted signals of all columns is input to the bit counter 332.

[0057] The bit counter 332 counts the number of bits with a logical value of “1” in the input bit string. The bit counter 332 supplies a count value CNT_ae to the determination circuit 334.

[0058] The register 333 stores a threshold value Th that is compared with the count value CNT_ae. Ｇ It holds the following.

[0059] The determination circuit 334 determines whether the count value CNT_ae from the bit counter 332 is equal to or smaller than the threshold value Th held in the register 333. Ｇ The determination circuit 334 supplies the determination result to the timing control section 220.

[0060] The vertical driver 210 controls the DAC 230 to output a reference signal RMP of a constant value when measuring light. This value is a threshold value that is compared with the luminance value of the pixel. Ｌ When electrons are transferred as charges, the potential of the vertical signal line VSL (i.e., the level of the pixel signal) decreases as the exposure time increases, because the amount of charges increases. Then, when the potential becomes lower than the value of the reference signal RMP (i.e., the threshold value Th Ｌ ), the comparison result CMP falls from high level to low level.

[0061] The inverter 331 inverts the comparison result CMP. ＬThe more pixels that fall below the threshold Th, the more "1" there are in the N bits of the inverted signal. The count value CNT_ae of the bit counter 332 indicates the photometric amount of a pixel group consisting of N photometric pixels 260. Ｇ The determination circuit 334 determines whether the voltage Vcc exceeds the threshold voltage Vdc.

[0062] The configuration of the exposure determination unit 330 is not limited to the example shown in the figure. For example, the inverter 331 may not be provided, and the bit counter 332 may count the number of logical values ​​"0". Also, if the connection of the input terminal of the comparator 311 is reversed, the potential may be increased to the threshold value Th. Ｌ When the comparison result CMP becomes less than the low level, the comparison result CMP rises from the low level to the high level, so the inverter 331 becomes unnecessary.

[0063] The exposure determination unit 330 can also perform center-weighted metering, spot metering, multi-pattern metering, and other methods. To perform center-weighted metering, a circuit that weights the count value is added. In spot metering and multi-pattern metering, a frame is divided into multiple blocks, and the blocks to be metered are specified. In this case, the bit counter 332 only needs to count the bits of the blocks to be metered.

[0064] Furthermore, the exposure determination unit 330 can use machine learning or the like to obtain the appropriate exposure.

[0065] FIG. 6 is a diagram for explaining the operation of the image sensor 200 according to the first embodiment of the present technology.

[0066] As described above, a predetermined number of normal pixels 250 and a predetermined number of photometric pixels 260 are arranged in the pixel array section 240. For example, if the number of rows is M, the number of columns is N, and one photometric pixel 260 is arranged per column, the total number of pixels is M×N, of which the number of photometric pixels 260 is N.

[0067] The vertical driving unit 210 sequentially selects rows and starts exposure by using a rolling shutter method. The vertical driving unit 210 also supplies a constant value (threshold value Th Ｌ) reference signal RMP, all the photometry pixels 260 are caused to start exposure, and these pixels are driven to output pixel signals.

[0068] The comparator 311 of each column compares the pixel signal with the reference signal RMP and outputs the comparison result CMP to the exposure determination unit 330. The exposure determination unit 330 performs photometry based on the comparison result CMP, and determines the amount of photometry relative to a threshold Th Ｇ (in other words, whether or not proper exposure has been obtained), and supplies the determination result to the timing control section 220.

[0069] The timing control unit 220 determines whether the photometric amount is greater than the threshold value Th Ｇ When the reference pixel value exceeds the reference pixel value RMP, the vertical drive unit 210 controls the vertical drive unit 210 to end exposure of all photometric pixels 260 and the initial exposure of the first row. The vertical drive unit 210 outputs a sawtooth reference signal RMP to the DAC 230 while driving the first row to output pixel signals, and then ends exposure of the second row and subsequent rows in order to output pixel signals. An ADC 310, consisting of a comparator 311 and a counter 312, converts the pixel signals of each row into digital signals.

[0070] 7 is an example of a histogram showing the luminance distribution of a typical natural image, in which the horizontal axis represents the luminance value and the vertical axis represents the number of pixels of that luminance value as a frequency.

[0071] The high peak at the right end of this histogram indicates when the charge is saturated or when the brightness exceeds the upper limit of the digital value. As illustrated in the figure, natural images have a variety of brightness distributions, with some very bright areas among them. If the entire image is darkened by controlling the exposure time or the like to match these very bright pixels, the signal level drops, resulting in a poor image quality due to a worsening S / N (Signal-Noise) ratio. On the other hand, even if the very bright areas are saturated, if the area is limited, the S / N ratio of intermediate levels improves, thereby improving the image quality. Therefore, the exposure determination unit 330 sets the aforementioned threshold TH so that the proportion of pixels close to saturation remains constant. Ｌ and Th Ｇ is set.

[0072] Here, as a comparative example, a configuration will be considered in which brightness is acquired from a frame that has been preliminarily captured first, and an appropriate exposure is calculated from the brightness to capture a second frame.

[0073] 8 is an example of a timing chart showing the operation of the image sensor 200 in the comparative example. In the figure, the vertical axis represents the row address, and the horizontal axis represents time. The gray portions represent the exposure period of each row.

[0074] As illustrated in the figure, the vertical drive unit 210 of the comparative example performs shutter control to sequentially select rows and start exposure during the period from timing T1 to T3. Then, the vertical drive unit 210 sequentially selects rows over the period from timing T2 to T4 and ends exposure. Furthermore, the ADC 310 of each column performs AD conversion on the pixel signals of each row over the period from timing T2 to T4.

[0075] The SoC 120 of the comparative example then calculates the photometric amount of the pixel array unit 240 by integrating the digital signals read during the period from timing T2 to T4. Then, during the period from timing T4 to T5, the SoC 120 calculates the exposure time and aperture to achieve proper exposure based on the calculated photometric amount. In the example shown in the figure, the exposure time is calculated with priority. The shaded areas in the figure indicate the period during which the exposure time and other parameters are calculated. After timing T5, the vertical drive unit 210 sequentially selects rows and exposes them for the calculated exposure time.

[0076] The first frame is pre-photographed by driving up to timing T4. The second frame is photographed by driving from timing T5 onwards. If the imaging conditions do not change, proper exposure is obtained from the second frame onwards. As such, in the comparative example, a one-frame delay occurs before proper exposure is obtained. This delay may cause the AE function to be unable to keep up if the brightness suddenly changes between the two frames. In this case, proper exposure cannot be obtained, resulting in a deterioration in the image quality of the frame.

[0077] 9 is a timing chart showing an example of the operation of the image sensor 200 according to the first embodiment of the present technology. In the figure, the vertical axis represents the row address, and the horizontal axis represents time. The light gray portions represent the exposure periods of the normal pixels 250 in each row.

[0078] As in the comparative example, the vertical drive unit 210 performs shutter control to sequentially select rows and start exposure during the period from timing T1 to T3. Also, immediately after timing T1, the vertical drive unit 210 starts exposure of all photometry pixels 260 in the even-numbered rows, and causes the exposure determination unit 330 to start photometry. The dark gray areas in the figure indicate the exposure times of the photometry pixels 260 in the even-numbered rows.

[0079] The exposure determination unit 330 then measures the light based on the comparison result of the comparator 311, and immediately before timing T2, the amount of light measured exceeds the threshold value Th Ｇ At timing T2, the vertical drive unit 210 ends the exposure of the photometric pixels 260 and the first row, and selects rows in order over the period up to timing T4 to end the exposure of the normal pixels 250. This achieves the AE function.

[0080] The ADC 310 of each column performs AD conversion during the period from timing T2 to T4. The ADC 310 of each column performs AD conversion on the D-phase levels of all the photometry pixels 260 during the period from timing T2 to T21, and AD conversion on the P-phase levels of all the photometry pixels 260 during the period from timing T21 to T22.

[0081] The ADC 310 of each column then AD converts the P-phase level of the first row during the period from timing T22 to T23, and AD converts the D-phase level of that row during the period from timing T23 to T24. After timing T24, the ADC 310 of each column sequentially AD converts the P-phase levels and D-phase levels of the normal pixels 250 from the second row to the Mth row. The image processing unit 270 at the downstream stage performs CDS processing for each pixel to determine the difference between the AD-converted P-phase level and D-phase level. This results in the first frame being captured.

[0082] As described above, by also AD converting the pixel signals of the photometric pixels 260, a defect-free frame can be obtained. However, although the data from the photometric pixels 260 after CDS processing has a strong correlation with the data from the surrounding normal pixels 250, the data has slightly different properties due to the different exposure timing. For this reason, it is preferable that the image processing unit 270 and the SoC 120 perform correction processing on the data from the photometric pixels 260.

[0083] After timing T4, the image sensor 200 captures the second and subsequent frames under the same control as for the first frame.

[0084] As shown in the figure, the vertical drive unit 210 starts exposure of the normal pixels 250 and all the photometric pixels 260 in the first row at approximately the same time, and the amount of photometry is adjusted to a threshold value Th Ｇ , the exposure is terminated when the brightness exceeds the threshold value. This reduces the delay until the proper exposure is obtained. Therefore, unlike the comparative example in which a delay of one frame occurs, the image sensor 200 of the first embodiment can control the exposure to the proper level even when the brightness changes suddenly.

[0085] Although the image sensor 200 uses a rolling shutter system in which exposure is started and ended row by row, it can also use a global shutter system in which exposure is started and ended simultaneously for all rows. In this case, for example, an analog memory or a sample-and-hold circuit that holds an analog signal from the end of exposure to readout is provided for each pixel.

[0086] 10 is a timing chart showing an example of the operation during photometry of the image sensor 200 according to the first embodiment of the present technology. Immediately after timing T1, the vertical drive unit 210 sets the drive signals TRG2_ae and SEL2_ae for the second row to high level and supplies a high-level drive signal RST2_ae for the entire pulse period. Similar control is performed for all even-numbered rows from the fourth row to the Mth row. This initiates exposure of all photometry pixels 260, and pixel signals from those pixels are output. The dark gray portions in the figure indicate the exposure periods of the photometry pixels 260 for AE.

[0087] The timing T11 is the timing when the period Tp of the horizontal synchronization signal HSYNC has elapsed since timing T1. From timing T12, which is immediately after timing T11, the vertical drive unit 210 supplies high-level drive signals RST1 and TRG1 to the first row over a pulse period. This starts exposure of the first row. Note that the drive signal SEL1 remains low, and no pixel signals are output from the first row.

[0088] The timing T13 is the timing when a period Tp has elapsed since timing T11. From timing T14, which is immediately after timing T13, the vertical drive unit 210 supplies high-level drive signals RST2_n and TRG2_n to the second row over a pulse period. This starts exposure of the normal pixels 250 in the second row. Note that the drive signal SEL2_n remains at a low level, and pixel signals of the normal pixels 250 in the second row are not output. Using similar control, the third row and beyond are selected in order from timing T15 onwards, and exposure starts.

[0089] After timing T1, the exposure determination unit 330 calculates the photometric amount and sets the threshold value Th Ｇ For example, just before the timing T2, the photometric amount is compared with the threshold value Th Ｇ When the detected value exceeds the threshold, the vertical drive unit 210 sets the drive signal TRG2_ae to low level. Similar control is performed for all even-numbered rows from the fourth row to the Mth row. This completes the exposure of all photometry pixels 260. The exposure determination unit 330 also completes photometry.

[0090] As shown in the figure, the vertical drive unit 210 starts exposure of the first row when Tp has elapsed since the start of exposure of the photometric pixels 260. This is done to adjust the exposure time of the first row, since AD ​​conversion of the pixel signals of the photometric pixels 260 after exposure ends would increase that much.

[0091] It should be noted that the same operation as the conventional image sensor 200 can be performed by simultaneously controlling the drive signal (TRG2_ae, etc.) to the photometric pixel 260 and the drive signal (TRG2_n, etc.) to the normal pixel 250.

[0092] FIG. 11 is a timing chart showing an example of a readout operation of the image sensor 200 according to the first embodiment of the present technology.

[0093] The timing T22 is the time when the period Tp has elapsed since the timing T2 when the exposure of the photometry pixels 260 ends. The vertical drive unit 210 supplies the drive signal RST2_ae to the second row over a pulse period starting from timing T21 immediately after timing T2, and sets the drive signal SEL2_ae for the second row to low level at timing T22. Similar control is performed for all even-numbered rows from the fourth row to the Mth row. As a result, the pixel signals of all the photometry pixels 260 are AD converted.

[0094] Furthermore, because charge transfer within the photometering pixel 260 is completed at timing T2, the D-phase level (signal level) of the photometering pixel 260 is AD converted during the period from timing T2 to T21. Then, because the floating diffusion layer 264 is initialized at timing T21, the P-phase level (reset level) of the photometering pixel 260 is converted during the period from timing T21 to T22. The signal level and reset level of the photometering pixel 260 are an example of the first signal level and reset level set forth in the claims.

[0095] The timing T24 is the timing when a period Tp has elapsed since timing T22. The vertical drive unit 210 supplies a high-level drive signal RST1 to the first row for a pulse period from timing T22, and supplies a high-level drive signal TRG1 to the first row for a pulse period from timing T23 immediately after timing T22. The vertical drive unit 210 also supplies a high-level drive signal SEL1 to the first row for a period from timing T22 to T24. This completes exposure of the first row, and the pixel signals of that row are AD converted.

[0096] Furthermore, since the floating diffusion layer 254 is initialized at timing T22, the P-phase level (reset level) of the normal pixel 250 is converted during the period from timing T22 to T23. Then, since the charge is transferred at timing T23, the D-phase level (signal level) of the normal pixel 250 is AD-converted during the period from timing T23 to T24. Note that the signal level and reset level of the normal pixel 250 are an example of the second signal level and reset level set forth in the claims.

[0097] As described above, when reading out the photometric pixels 260, AD conversion is performed in the order of the signal level and the reset level, and when reading out the normal pixels 250, AD conversion is performed in the order of the reset level and the signal level.

[0098] The timing when a period Tp has elapsed since timing T24 is designated as T26. The vertical drive unit 210 supplies a high-level drive signal RST2_n to the second row over a pulse period from timing T24, and supplies a high-level drive signal TRG2_n to the second row over a pulse period from timing T25 immediately after timing T24. The vertical drive unit 210 also supplies a high-level drive signal SEL2_n to the second row over a period from timing T24 to T26. This completes exposure of the normal pixels 250 in the second row, and the pixel signals are AD converted.

[0099] Furthermore, the P-phase level of the normal pixel 250 is converted during the period from timing T24 to T25, and the D-phase level of the normal pixel 250 is AD-converted during the period from timing T25 to T26.

[0100] After timing T26, pixel signals of the normal pixels 250 in the third row to the Mth row are AD converted in sequence by the same control as in the first and second rows.

[0101] As shown in the figure, when exposure of all photometric pixels 260 is completed, their pixel signals are AD converted. Next, exposure of the normal pixels 250 from the first row to the Mth row is completed in order, and their pixel signals are AD converted. Furthermore, if exposure of the first to fifth rows has started by timing T2, exposure of the sixth and subsequent rows will start in order after timing T2 in parallel with readout. In the figure, shutter control after timing T2 is omitted for convenience of explanation.

[0102] FIG. 12 is a timing chart showing an example of the operation of the DAC 230 and the ADC 310 according to the first embodiment of the present technology.

[0103] After the timing T1 at which exposure of all the photometry pixels 260 starts, the timing control section 220 controls the DAC 230 to set the threshold value Th Ｌ Then, the supply of the reference signal RMP is started.

[0104] The potential of the vertical signal line VSL of each column (in other words, the level of the pixel signal of the photometry pixel 260) gradually decreases over time during the exposure period of the photometry pixel 260 from timing T1 onwards.

[0105] The comparator 311 of each column detects whether the potential of the vertical signal line VSL of the corresponding column is equal to or exceeds a threshold value Th Ｌ When the number of columns that have been inverted (in other words, the amount of exposure) is less than the threshold value Th Ｇ , the vertical drive unit 210 ends the exposure of the photometry pixel 260 at timing T2. The gray portions in the figure indicate periods during which the comparison result of the comparator 311 is not taken into consideration.

[0106] During the period from timing T1 to T2, the timing control section 220 controls the counter 312 for each column to stop counting. The gray areas in the figure indicate periods during which the counter 312 stops counting and its value is not taken into account.

[0107] Then, after timing T2, the timing control unit 220 controls the DAC 230 to start supplying the sawtooth reference signal RMP. In general, the amplitude of the reference signal RMP during P-phase level conversion is set to a smaller value than during D-phase level conversion.

[0108] The potential of the vertical signal line VSL of each column is at the D-phase level at timing T2.

[0109] At timing T2, the timing control section 220 sets the count value CNT of the counter 312 for each column to an initial value (for example, "0"). Then, in the period until timing T21, the counter 312 for each column counts up the count value CNT over the period until the comparison result CMP of the corresponding comparator 311 is inverted. This causes the D-phase levels of all the photometric pixels 260 to be AD converted.

[0110] At timing T21, the floating diffusion layer in the photometric pixel 260 is initialized, and the potential of the vertical signal line VSL of each column becomes the P-phase level.

[0111] Furthermore, the timing control unit 220 initializes the count value CNT of the counter 312 for each column at timing T21. The timing when a period Tp has elapsed since timing T2 is set to T22. During the period from timing T21 to T22, the counter 312 for each column counts up the count value CNT until the comparison result CMP of the corresponding comparator 311 is inverted. This causes the P-phase levels of all photometric pixels 260 to be AD converted.

[0112] The timing T24 is the timing when the period Tp has elapsed since timing T22. The vertical drive unit 210 selects the first row from timing T22 to T24 and transfers the charges at timing T23. As a result, the potential of the vertical signal line VSL of each column changes from the P-phase level to the D-phase level at timing T23.

[0113] The ADC 310 of each column performs AD conversion on the P-phase level of the first row during the period from timing T22 to T23, and AD conversion on the D-phase level of the first row during the period from timing T23 to T24.

[0114] After timing T24, the second and subsequent rows are selected in order, and the P-phase level and the D-phase level of the normal pixels 250 in each row are AD-converted in order.

[0115] The image processing unit 270 at the subsequent stage performs CDS processing for each pixel. Note that the ADC 310 can also perform CDS processing instead of the image processing unit 270. In this case, the counter 312 in the ADC 310 counts down during AD conversion of the P-phase level and counts up during AD conversion of the D-phase level.

[0116] 13 is an example of a state transition diagram of the image sensor 200 according to the first embodiment of the present technology. The black dots in the diagram indicate the time points at which an imaging operation is started due to the start of a predetermined application or the like.

[0117] When the image capturing operation is started, the image sensor 200 transitions to a photometry state 510. In this state, the vertical drive unit 210 drives all the photometry pixels 260, and the DAC 230 detects the threshold Th Ｌ The exposure determination unit 330 outputs the pixel signal of the photometric pixel 260 and the threshold value Th Ｌ Photometry is performed based on the comparison result CMP, and the photometric amount is compared with the threshold value Th. Ｇ (in other words, whether proper exposure has been obtained).

[0118] Photometric amount is threshold Th Ｇ If the difference exceeds , the image sensor 200 transitions to a photometric pixel readout state 521. In this state, the ADC 310 performs AD conversion on the D-phase level of the photometric pixel 260, and then AD conversion on the P-phase level. The image processing unit 270 calculates the difference between these as the net signal level.

[0119] After the period Tp has elapsed, the image sensor 200 transitions to a first-row readout state 522. In this state, the ADC 310 performs AD conversion on the P-phase level of the first row, and then AD conversion on the D-phase level thereof. The image processing unit 270 calculates the difference between these values.

[0120] After another period Tp has elapsed, the image sensor 200 transitions to a second row readout state 523. In this state, the ADC 310 AD converts the P-phase level of the normal pixels 250 in the second row, and then AD converts the D-phase level thereof. The image processing unit 270 calculates the difference between these.

[0121] Thereafter, every time the period Tp elapses, the third row to the Mth row are selected in order, and pixel signals are read out from the normal pixels 250. When the readout of the Mth row is completed, the image sensor 200 transitions to the photometry state 510.

[0122] Furthermore, when the period Tp has elapsed since the start of imaging, the image sensor 200 transitions to a first row shutter state 531. In this state, the vertical drive section 210 starts exposure of the first row.

[0123] When the period Tp further elapses, the image sensor 200 transitions to a second row shutter state 532. In this state, the vertical drive section 210 starts exposing the normal pixels 250 in the second row.

[0124] Thereafter, every time the period Tp elapses, exposure is started in order from the third row to the Mth row. The white circles in the figure indicate the time points at which the shutter control is completed.

[0125] As described above, according to the first embodiment of the present technology, the vertical drive unit 210 starts exposure of the normal pixels 250 and the photometric pixels 260, and the photometric amount reaches the threshold Th Ｇ Since the exposure is terminated when the exposure time exceeds the threshold, the time required to obtain the correct exposure can be shortened.

[0126] In the first embodiment described above, the vertical drive unit 210 adjusts the exposure time by starting exposure of the first row, in which only the normal pixels 250 are arranged, when Tp has elapsed since the start of exposure of the photometric pixels 260. However, the vertical drive unit 210 can also start exposure of the first row simultaneously with the start of exposure of the photometric pixels 260. The image sensor 200 in this modification of the first embodiment differs from the first embodiment in that exposure of the photometric pixels 260 and the normal pixels 250 starts simultaneously.

[0127] 14 is a timing chart showing an example of the operation during photometry of the image sensor 200 in the modification of the first embodiment of the present technology. The vertical drive unit 210 in the modification of the first embodiment differs from the first embodiment in that exposure of the first row starts simultaneously with the start of exposure of the photometry pixels 260.

[0128] 15 is a timing chart showing an example of a readout operation of the image sensor 200 according to the modification of the first embodiment of the present technology. The vertical drive unit 210 according to the modification of the first embodiment sets the drive signals TRG_ae and SEL_ae to the even-numbered rows to a low level immediately before timing T2, thereby ending exposure of all the photometry pixels 260.

[0129] The vertical drive unit 210 then drives the first row during the period from timing T2 to T22, and the ADC 310 of each column performs A / D conversion on the pixel signals of the first row. After timing T22, the second row and beyond are selected in order, and the pixel signals of the normal pixels 250 are A / D converted. Unlike the first embodiment, the pixel signals of the photometric pixels 260 are not A / D converted. As a result, the photometric pixels 260 become defective pixels in the frame.

[0130] 16 is a block diagram showing an example of a configuration of an image processing unit 270 according to the modification of the first embodiment of the present technology. The image processing unit 270 according to the modification of the first embodiment includes a CDS processing unit 271, a frame buffer 272, and an interpolation processing unit 273.

[0131] The CDS processing unit 271 performs CDS processing for each pixel on the digital signal from the column signal processing unit 300. The CDS processing unit 271 stores a frame in which the processed data is arranged in a frame buffer 272.

[0132] The interpolation processing unit 273 reads out data around the photometry pixel 260 from the frame buffer 272, and uses this data to interpolate the data of the photometry pixel 260. The interpolation processing unit 273 outputs the interpolated frame to the output interface 280.

[0133] For example, if the number of rows is 1000 and the number of columns is 2000, the total number of pixels is 1000 x 2000. If one photometry pixel 260 is arranged per column, the number of photometry pixels 260 is 2000. The ratio of the photometry pixels 260 to the total number of pixels is expressed by the following formula: 2000 / (1000 x 2000) = 1 / 1000 = 0.001

[0134] From the above formula, the proportion of the photometric pixels 260 is 0.1% in percentage terms, and even if these are interpolated, the effect on image quality is small.

[0135] In addition to the CDS processing and interpolation processing, the image processing unit 270 can also perform various other processes such as white balance correction and demosaic processing.

[0136] Thus, according to the modified example of the first embodiment of the present technology, since the ADC 310 does not perform AD conversion of the pixel signal of the photometry pixel 260, the vertical drive unit 210 can start exposure of the first row at the same time as the exposure of the photometry pixel 260 starts.

[0137] 2. Second Embodiment In the first embodiment described above, the normal pixels 250 and the photometric pixels 260 are arranged in even-numbered rows, but it is also possible to arrange only the photometric pixels 260 in any one row. The image sensor 200 in this second embodiment differs from the first embodiment in that only the photometric pixels 260 are arranged in any one row.

[0138] The image sensor 200 in the second embodiment can switch between a first mode in which the photometering pixels 260 are distributed across multiple even-numbered rows, and a second mode in which only the photometering pixels 260 are arranged in any one row. In the first mode, control is performed in the same manner as in the first embodiment. In the second mode, a row in which only the photometering pixels 260 are arranged is referred to as a "photometering row."

[0139] 17 is a diagram showing an example of a photometry row according to the second embodiment of the present technology. The circuit configuration of the pixel array unit 240 according to the second embodiment is the same as that of the first embodiment.

[0140] However, in the second mode, the timing control unit 220 in the image sensor 200 can select any row as the photometry row. For example, as illustrated in the figure, the timing control unit 220 can select one of the even-numbered rows (such as the second row) in which the normal pixels 250 and photometry pixels 260 were arranged in the first mode as the photometry row.

[0141] As shown in FIG. 18, the timing control section 220 can also select, as the photometry row, one of the odd-numbered rows (such as the first row) in which only normal pixels 250 are arranged in the first mode.

[0142] It is preferable to use a row near the top or bottom of the pixel array section 240, which has little effect on image quality, as the photometry row.

[0143] In the second mode, only the normal pixels 250 are arranged in rows other than the photometry rows, and no defects occur, so the image quality can be improved compared to the first embodiment.

[0144] The operation of the image sensor 200 in the second embodiment will be described with reference to Figures 19 to 21. In Figures 19 to 21, it is assumed that the second row is selected as the photometry row.

[0145] 19 is a timing chart showing an example of the operation during photometry of the image sensor according to the second embodiment of the present technology. Immediately after timing T1, the vertical drive unit 210 sets the drive signals TRG2_ae, SEL2_ae, TRG2_n, and SEL2_n for the second row to high level and supplies the drive signals RST2_ae and RST2_n at high level for the entire pulse period. This starts exposure of the photometry row.

[0146] As described above, the second row is the row in which the normal pixels 250 and photometric pixels 260 are arranged in the first mode, and therefore, in the second mode, in order to drive all the pixels in the second row, it is necessary to simultaneously control six drive signals from two systems.

[0147] Then, during the period from timing T11 to T13, the vertical drive unit 210 starts exposure of the first row. The timing when the cycle Tp has elapsed since timing T13 is designated as T15. Although shutter control is not performed during the period from timing T13 to T15, this period is inserted to prevent unnatural steps from appearing in the image. Then, during the period from timing T15 to T17, the vertical drive unit 210 starts exposure of the third row. From timing T17 onwards, the fourth row and beyond are selected in order and exposure is started. Note that the second row is a photometry row, and exposure is not started for this row in row-by-row shutter control.

[0148] In addition, when a row (such as the first row) in which only normal pixels 250 are arranged in the first mode is set as a photometry row in the second mode, one system of drive signals such as drive signals TRG1, SEL1, and RST1 is controlled.

[0149] FIG. 20 is a timing chart showing an example of a readout operation of the image sensor 200 according to the second embodiment of the present technology.

[0150] The vertical drive unit 210 supplies drive signals RST2_ae and RST2_n to the second row over a pulse period from timing T21 immediately after timing T2, and sets the drive signals SEL2_ae and SEL2_n for the second row to low level at timing T22, thereby performing AD conversion on the pixel signals of all the photometry pixels 260.

[0151] Then, during the period from timing T22 to T24, the ADC 310 performs AD conversion on the pixel signals of the first row. The vertical drive unit 210 supplies high-level drive signals RST2_ae and RST2_n to the fourth row over a pulse period starting from timing T24. The vertical drive unit 210 then supplies high-level drive signals TRG2_ae and TRG2_n to the fourth row over a pulse period starting from timing T25, immediately after timing T24. The vertical drive unit 210 also supplies high-level drive signals SEL2_ae and SEL2_n to the fourth row over a pulse period starting from timing T24 to T26. This results in AD conversion of the pixel signals of the second row. However, because the second row has already been read out during the period from timing T2 to T22, the data read out during the period from timing T24 to T26 is discarded. Then, during the period from timing T26 to T28, the ADC 310 performs AD conversion on the pixel signals of the third row.

[0152] The vertical drive unit 210 supplies high-level drive signals RST4_ae and RST4_n to the fourth row over a pulse period starting from timing T28. Then, the vertical drive unit 210 supplies high-level drive signals TRG4_ae and TRG4_n to the fourth row over a pulse period starting from timing T29, immediately after timing T28. The vertical drive unit 210 also supplies high-level drive signals SEL4_ae and SEL4_n to the fourth row over a period from timing T28 to T30. This completes exposure of the fourth row, and the pixel signals of that row are AD-converted. In this way, for even-numbered rows such as the fourth row, two systems of drive signals must be controlled simultaneously.

[0153] After timing T30, the fifth row and subsequent rows are selected in order, and the pixel signals are AD converted.

[0154] FIG. 21 is an example of a state transition diagram of the image sensor 200 according to the second embodiment of the present technology.

[0155] In a photometric pixel readout state 521, the ADC 310 sequentially AD converts the D-phase level and P-phase level of the second row, which is the photometric row. Then, in a first row readout state 522, the ADC 310 sequentially AD converts the P-phase level and D-phase level of the first row. Then, pixel signals from the second row onward are read out sequentially.

[0156] Furthermore, the vertical drive unit 210 starts exposure of the first row in a first row shutter state 531, and after Tp×2 has elapsed, starts exposure of the third row in a third row shutter state 532. Thereafter, exposure of the fourth row and onwards is started in order.

[0157] It should be noted that the modified example of the first embodiment in which exposure of the first row starts simultaneously with the start of exposure of the photometric pixels 260 can be applied to the second embodiment.

[0158] In this way, according to the second embodiment of the present technology, only the photometry pixels 260 are arranged in any row, so that it is possible to improve the image quality compared to the first embodiment.

[0159] [Modification] In the second embodiment described above, switching is performed between the first mode and the second mode, but in this configuration, it is necessary to wire two systems of drive lines to a row (such as the second row) in which normal pixels 250 and photometric pixels 260 are mixed in the first mode. The image sensor 200 in this modification of the second embodiment differs from the second embodiment in that it is configured not to switch to the first mode and the number of wires is reduced.

[0160] 22 is a plan view showing a configuration example of a pixel array unit 240 in a modified example of the second embodiment of the present technology. The pixel array unit 240 in this modified example of the second embodiment differs from the second embodiment in that three drive lines for one system are wired to all rows.

[0161] In this way, according to the modified example of the second embodiment of the present technology, since there is no need to switch to the first mode corresponding to the first embodiment, there is no need to wire two systems of drive lines in the even rows, and the number of wires can be reduced.

[0162] 3. Third Embodiment In the second embodiment described above, any one row was used as the photometry row, but the number of photometry samples may be insufficient with only one row. Image sensor 200 in this third embodiment differs from the second embodiment in that two or more rows are selected as photometry rows.

[0163] FIG. 23 is a diagram illustrating an example of photometry rows in the third embodiment of the present technology. In the third embodiment, the timing control unit 220 can select any two rows as photometry rows. For example, as illustrated in the same figure, the first row and the last M-th row are selected as photometry rows. Note that the number of photometry rows is not limited to two, and three or more rows can also be used as photometry rows. In addition, it is preferable to use rows that have little impact on image quality, such as those near the top or bottom of the pixel array unit 240, as photometry rows.

[0164] As shown in the figure, by providing two or more photometry rows, the number of photometry samples is increased compared to the second embodiment, and AE performance can be improved.

[0165] 24 is a timing chart showing an example of the operation during photometry of the image sensor according to the third embodiment of the present technology. Immediately after timing T1, the vertical drive unit 210 sets the drive signals TRG1, SEL1, TRGM_ae, SELM_ae, TRGM_n, and SELM_n to the first and Mth rows to high level. The vertical drive unit 210 also supplies high-level drive signals RST1, RSTM_ae, and RSTM_n over a pulse period immediately after timing T1. This starts exposure of the first and Mth rows, which are the photometry rows.

[0166] In the control shown in the figure, two pixels in the first and Mth rows output pixel signals to one vertical signal line VSL. In this case, a signal that has been added by a source follower is output, and its level is the average value of the two pixels.

[0167] The timing 2×Tp after timing T1 is defined as T13. From timing T14 immediately after timing T13, the vertical drive unit 210 supplies high-level drive signals RST2_ae, TRG2_ae, RST2_n, and TRG2_n to the second row over a pulse period. This starts exposure of the second row.

[0168] After timing T15, the third to (M-1)th rows are selected in order, and exposure of these rows is started. Note that the first and Mth rows are photometry rows, and exposure is not started for these rows in the row-by-row shutter control.

[0169] FIG. 25 is a timing chart showing an example of a readout operation of the image sensor 200 according to the third embodiment of the present technology.

[0170] The vertical drive unit 210 supplies the drive signal RST1 to the second row over a pulse period from timing T21 immediately after timing T2, and sets the drive signal SEL1 for the second row to low level at timing T22, thereby AD-converting the pixel signals of one of the two photometry rows.

[0171] Then, the vertical drive unit 210 supplies the drive signals RSTM_ae and RSTM_n to the Mth row over a pulse period from timing T23 immediately after timing T22, and sets the drive signals SELM_ae and SELM_n to low level at timing T24, thereby AD-converting the pixel signals of the other of the two photometry rows.

[0172] The vertical drive unit 210 then supplies high-level drive signals RST2_ae and RST2_n to the second row over a pulse period starting from timing T24. The vertical drive unit 210 then supplies high-level drive signals TRG2_ae and TRG2_n to the second row over a pulse period starting from timing T25. The vertical drive unit 210 also supplies high-level drive signals SEL2_ae and SEL2_n to the second row over a period from timing T24 to T26. This completes exposure of the second row, and the pixel signals of that row are AD converted.

[0173] The vertical drive unit 210 then supplies a high-level drive signal RST3 to the third row over a pulse period starting from timing T26, and a high-level drive signal TRG3 to the third row over a pulse period starting from timing T27. The vertical drive unit 210 also supplies a high-level drive signal SEL3 to the third row over a period from timing T26 to T28. This completes the exposure of the third row, and the pixel signals of that row are AD converted. From timing T28 onwards, the fourth to (M-1)th rows are selected in order, and their pixel signals are AD converted.

[0174] 26 is an example of a state transition diagram of the image sensor 200 according to the third embodiment of the present technology. In a photometry pixel readout state 521, the ADC 310 sequentially performs AD conversion on the D-phase level and the P-phase level of the first row, which is the photometry row. Next, the state transitions to a photometry pixel readout state 522, in which the ADC 310 sequentially performs AD conversion on the D-phase level and the P-phase level of the M-th row, which is the photometry row.

[0175] Then, the process shifts to a second row readout state 523, and the ADC 310 sequentially performs AD conversion on the P-phase level and D-phase level of the second row. Then, pixel signals from the third row to the (M-1)th row are sequentially read out.

[0176] Furthermore, when 2×Tp has elapsed since the start of imaging, the vertical drive unit 210 transitions to a second row shutter state 531 and starts exposure of the second row. Next, the vertical drive unit 210 transitions to a third row shutter state 532 and starts exposure of the third row. Thereafter, exposure of the fourth row to the (M−1)th row is started in order.

[0177] It should be noted that the modified example of the first embodiment in which exposure of the row of normal pixels 250 starts simultaneously with the start of exposure of the photometric pixels 260 can be applied to the third embodiment.

[0178] Furthermore, the modified example of the second embodiment that reduces the number of wirings can be applied to the third embodiment.

[0179] As described above, according to the third embodiment of the present technology, since there are two or more photometry rows, it is possible to improve the AE performance compared to the second embodiment.

[0180] 4. Application Examples to Mobile Bodies The technology according to the present disclosure (the present technology) can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of mobile body, such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, personal mobility, an airplane, a drone, a ship, or a robot.

[0181] FIG. 27 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile object control system to which the technology according to the present disclosure can be applied.

[0182] The vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001. In the example shown in Fig. 27, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside-vehicle information detection unit 12030, an inside-vehicle information detection unit 12040, and an integrated control unit 12050. Also shown as functional components of the integrated control unit 12050 are a microcomputer 12051, an audio / video output unit 12052, and an in-vehicle network I / F (Interface) 12053.

[0183] The drivetrain control unit 12010 controls the operation of devices related to the drivetrain of the vehicle in accordance with various programs. For example, the drivetrain control unit 12010 functions as a control device for a drive force generating device for generating a drive force of the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking device for generating a braking force of the vehicle.

[0184] The body system control unit 12020 controls the operation of various devices equipped in the vehicle body according to various programs. For example, the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as headlamps, backup lamps, brake lamps, turn signals, and fog lamps. In this case, radio waves transmitted from a portable device that serves as a key or signals from various switches can be input to the body system control unit 12020. The body system control unit 12020 receives these radio waves or signals and controls the vehicle's door lock device, power window device, lamps, etc.

[0185] The outside-vehicle information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, an imaging unit 12031 is connected to the outside-vehicle information detection unit 12030. The outside-vehicle information detection unit 12030 causes the imaging unit 12031 to capture images outside the vehicle and receives the captured images. The outside-vehicle information detection unit 12030 may perform object detection processing or distance detection processing for people, cars, obstacles, signs, characters on the road surface, etc. based on the received images.

[0186] The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal corresponding to the amount of light received. The imaging unit 12031 can output the electrical signal as an image or as distance measurement information. The light received by the imaging unit 12031 may be visible light or invisible light such as infrared light.

[0187] The in-vehicle information detection unit 12040 detects information inside the vehicle. For example, a driver state detection unit 12041 that detects the state of the driver is connected to the in-vehicle information detection unit 12040. The driver state detection unit 12041 includes, for example, a camera that captures an image of the driver, and the in-vehicle information detection unit 12040 may calculate the degree of fatigue or concentration of the driver based on the detection information input from the driver state detection unit 12041, or may determine whether the driver is dozing off.

[0188] The microcomputer 12051 can calculate control target values ​​for the driving force generating device, steering mechanism, or braking device based on the information inside and outside the vehicle acquired by the outside-vehicle information detection unit 12030 or the inside-vehicle information detection unit 12040, and output control commands to the drive system control unit 12010. For example, the microcomputer 12051 can perform cooperative control aimed at realizing the functions of an ADAS (Advanced Driver Assistance System), including vehicle collision avoidance or impact mitigation, following driving based on the distance between vehicles, maintaining vehicle speed, vehicle collision warning, vehicle lane departure warning, etc.

[0189] In addition, the microcomputer 12051 can perform cooperative control for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on driver operation, by controlling the driving force generating device, steering mechanism, braking device, etc. based on information about the surroundings of the vehicle obtained by the outside vehicle information detection unit 12030 or the inside vehicle information detection unit 12040.

[0190] Furthermore, the microcomputer 12051 can output a control command to the body system control unit 12020 based on the information outside the vehicle acquired by the outside information detection unit 12030. For example, the microcomputer 12051 can control the headlamps according to the position of a preceding vehicle or an oncoming vehicle detected by the outside information detection unit 12030, and perform cooperative control aimed at preventing glare, such as switching from high beams to low beams.

[0191] The audio / video output unit 12052 transmits at least one of audio and video output signals to an output device capable of visually or audibly notifying information to vehicle occupants or the outside of the vehicle. In the example of Fig. 27, the output devices are exemplified by an audio speaker 12061, a display unit 12062, and an instrument panel 12063. The display unit 12062 may include, for example, at least one of an on-board display and a head-up display.

[0192] FIG. 28 is a diagram showing an example of the installation position of the imaging unit 12031.

[0193] In FIG. 28, the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.

[0194] The imaging units 12101, 12102, 12103, 12104, and 12105 are provided, for example, at positions such as the front nose, side mirrors, rear bumper, back door, and the top of the windshield inside the vehicle cabin of the vehicle 12100. The imaging unit 12101 provided on the front nose and the imaging unit 12105 provided on the top of the windshield inside the vehicle cabin mainly acquire images of the front of the vehicle 12100. The imaging units 12102 and 12103 provided on the side mirrors mainly acquire images of the sides of the vehicle 12100. The imaging unit 12104 provided on the rear bumper or back door mainly acquires images of the rear of the vehicle 12100. The imaging unit 12105 provided on the top of the windshield inside the vehicle cabin is mainly used to detect preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, etc.

[0195] 28 shows an example of the imaging ranges of the imaging units 12101 to 12104. Imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose, imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided on the side mirrors, respectively, and imaging range 12114 indicates the imaging range of the imaging unit 12104 provided on the rear bumper or back door. For example, by overlaying the image data captured by the imaging units 12101 to 12104, an overhead image of the vehicle 12100 viewed from above can be obtained.

[0196] At least one of the image capturing units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the image capturing units 12101 to 12104 may be a stereo camera made up of multiple image capturing elements, or may be an image capturing element having pixels for phase difference detection.

[0197] For example, based on the distance information obtained from the imaging units 12101 to 12104, the microcomputer 12051 can calculate the distance to each three-dimensional object within the imaging ranges 12111 to 12114 and the change in this distance over time (relative speed with respect to the vehicle 12100), thereby extracting as a preceding vehicle, in particular, the three-dimensional object that is the closest three-dimensional object on the path of the vehicle 12100 and traveling in approximately the same direction as the vehicle 12100 at a predetermined speed (e.g., 0 km / h or higher). Furthermore, the microcomputer 12051 can set a vehicle-to-vehicle distance to be maintained in advance in front of the preceding vehicle, and perform automatic braking control (including follow-up stop control), automatic acceleration control (including follow-up start control), etc. In this way, cooperative control can be performed for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on driver operation.

[0198] For example, the microcomputer 12051 classifies and extracts three-dimensional object data regarding three-dimensional objects into two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects based on distance information obtained from the imaging units 12101 to 12104, and can use the data for automatic obstacle avoidance. For example, the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. The microcomputer 12051 then determines a collision risk that indicates the risk of collision with each obstacle, and when the collision risk is equal to or greater than a set value and a collision is possible, the microcomputer 12051 can provide driving assistance for collision avoidance by outputting an alarm to the driver via the audio speaker 12061 or the display unit 12062, or by performing forced deceleration or avoidance steering via the drive system control unit 12010.

[0199] At least one of the image capturing units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether a pedestrian is present in the images captured by the image capturing units 12101 to 12104. Such pedestrian recognition is performed, for example, by extracting feature points from the images captured by the image capturing units 12101 to 12104 as infrared cameras and performing pattern matching on a series of feature points that indicate the outline of an object to determine whether the object is a pedestrian. When the microcomputer 12051 determines that a pedestrian is present in the images captured by the image capturing units 12101 to 12104 and recognizes the pedestrian, the audio / image output unit 12052 controls the display unit 12062 to superimpose a rectangular outline on the recognized pedestrian for emphasis. The audio / image output unit 12052 may also control the display unit 12062 to display an icon or the like indicating the pedestrian at a desired position.

[0200] An example of a vehicle control system to which the technology according to the present disclosure can be applied has been described above. The technology according to the present disclosure can be applied to, for example, the image capturing unit 12031 of the above-described configuration. Specifically, the image capturing device 100 of FIG. 1 can be applied to the image capturing unit 12031. By applying the technology according to the present disclosure to the image capturing unit 12031, it is possible to obtain a captured image that is easier to see through proper exposure, thereby reducing driver fatigue.

[0201] Note that the above-described embodiment shows an example for realizing the present technology, and the matters in the embodiment and the matters specifying the invention in the claims correspond to each other. Similarly, the matters specifying the invention in the claims and the matters in the embodiment of the present technology having the same name correspond to each other. However, the present technology is not limited to the embodiment, and can be realized by applying various modifications to the embodiment within the scope of the gist thereof.

[0202] The effects described in this specification are merely examples and are not limiting, and other effects may also be obtained.

[0203] The present technology may also be configured as follows: (1) An image sensor comprising: a pixel array unit in which a predetermined number of photometric pixels and a predetermined number of normal pixels are arranged; a comparator that compares a first pixel signal from the photometric pixels with a predetermined reference signal and outputs the comparison result; an exposure determination unit that performs photometry based on the comparison result and determines whether a photometric amount exceeds a predetermined threshold; and a vertical drive unit that starts exposure of the normal pixels and the photometric pixels and drives the photometric pixels to output the first pixel signal, and, if the photometric amount exceeds the threshold, ends the exposure and drives the normal pixels to output a second pixel signal. (2) The image sensor according to (1), in which a predetermined number of the photometric pixels are arranged in each column in the pixel array unit. (3) The image sensor according to (2), in which the pixel array unit includes rows in which the photometric pixels and the normal pixels are arranged. (4) The image sensor according to (2) or (3), wherein the pixel array unit includes a photometric row in which the photometric pixels are arranged and a plurality of normal rows in which the normal pixels are arranged. (5) The image sensor according to (2) or (3), wherein the pixel array unit includes a plurality of photometric rows in which the photometric pixels are arranged and a plurality of normal rows in which the normal pixels are arranged. (6) The image sensor according to any of (1) to (5), wherein the pixel array unit is wired with a first drive line connected to the photometric pixels and transmitting a first drive signal, and a second drive line connected to the normal pixels and transmitting a second drive signal. (7) The image sensor according to any of (1) to (6), further comprising: a counter that counts a count value over a period until the comparison result is inverted and outputs a digital signal indicating the count value, wherein the comparator compares the second pixel signal with the reference signal when the normal pixel is driven, and wherein the counter and the comparator are disposed within an analog-to-digital converter. (8) The image sensor according to (7), further comprising an interpolation processing unit that interpolates a digital signal corresponding to the first pixel signal, wherein the analog-to-digital converter converts the first pixel signal into the digital signal when the photometric amount exceeds the threshold, and the vertical drive unit starts exposure of the normal pixels in a predetermined row simultaneously with the photometric pixels.(9) The image sensor according to (7), wherein the analog-to-digital converter converts the first and second pixel signals into the digital signals in order when the photometric amount exceeds the threshold, and the vertical drive unit starts exposure of the normal pixels in a predetermined row when a predetermined time has elapsed since the start of exposure of the photometric pixels. (10) The image sensor according to (9), wherein the level of the first pixel signal includes a first reset level when a floating diffusion layer is initialized and a first signal level when charge is transferred, and the level of the second pixel signal includes a second reset level when a floating diffusion layer is initialized and a second signal level when charge is transferred, and the analog-to-digital converter converts the digital signals in the order of the first signal level, the first reset level, the second reset level, and the second signal level. (11) A photodetection device comprising: a pixel array section in which a predetermined number of photometric pixels and a predetermined number of normal pixels are arranged; a comparator that compares a first pixel signal from the photometric pixels with a predetermined reference signal and outputs the comparison result; an exposure determination section that performs photometry based on the comparison result and determines whether the photometric amount exceeds a predetermined threshold; a vertical drive section that starts exposure of the normal pixels and the photometric pixels and drives the photometric pixels to output the first pixel signal, and if the photometric amount exceeds the threshold, ends the exposure and drives the normal pixels to output a second pixel signal; and an image processing section that processes image data generated from the second pixel signal. (12) A control method for an image sensor comprising: a comparison step of comparing a first pixel signal from a photometric pixel in a pixel array section in which a predetermined number of photometric pixels and a predetermined number of normal pixels are arranged with a predetermined reference signal and outputting the comparison result; an exposure determination step of performing photometry based on the comparison result and determining whether the photometric amount exceeds a predetermined threshold; and a vertical drive step of starting exposure of the normal pixels and the photometric pixels and driving the photometric pixels to output the first pixel signal, and if the photometric amount exceeds the threshold, ending the exposure and driving the normal pixels to output a second pixel signal.

[0204] 100 Imaging device 110 Imaging lens 120 SoC 130 Imaging control unit 140 Recording unit 200 Image sensor 210 Vertical drive unit 220 Timing control unit 230 DAC 240 Pixel array unit 250 Normal pixel 251, 261 Photoelectric conversion element 252, 262 Transfer transistor 253, 263 Reset transistor 254, 264 Floating diffusion layer 255, 265 Amplification transistor 256, 266 Selection transistor 260 Photometric pixel 270 Image processing unit 271 CDS processing unit 272 Frame buffer 273 Interpolation processing unit 280 Output interface 300 Column signal processing unit 310 ADC 311 Comparator 312 Counter 320 Memory 330 Exposure determination unit 331 Inverter 332 Bit counter 333 Register 334 Determination circuit 12031 Imaging unit

Claims

1. An image sensor comprising: a pixel array section in which a predetermined number of photometric pixels and a predetermined number of normal pixels are arranged; a comparator that compares a first pixel signal from the photometric pixels with a predetermined reference signal and outputs the comparison result; an exposure determination section that performs photometry based on the comparison result and determines whether the photometric amount exceeds a predetermined threshold; and a vertical drive section that starts exposure of the normal pixels and the photometric pixels and drives the photometric pixels to output the first pixel signal, and if the photometric amount exceeds the threshold, ends the exposure and drives the normal pixels to output a second pixel signal.

2. The image sensor according to claim 1, wherein a predetermined number of said photometric pixels are arranged for each column in said pixel array section.

3. The image sensor according to claim 2, wherein the pixel array section includes rows in which the photometric pixels and the normal pixels are arranged.

4. The image sensor according to claim 2, wherein the pixel array section includes a photometric row in which the photometric pixels are arranged and a plurality of normal rows in which the normal pixels are arranged.

5. The image sensor according to claim 2, wherein the pixel array section includes a plurality of photometric rows in which the photometric pixels are arranged and a plurality of normal rows in which the normal pixels are arranged.

6. The image sensor according to claim 1, wherein the pixel array section is wired with first drive lines connected to the photometric pixels and transmitting first drive signals, and second drive lines connected to the normal pixels and transmitting second drive signals.

7. The image sensor according to claim 1, further comprising a counter that counts a count value over a period until the comparison result is inverted and outputs a digital signal indicating the count value, wherein the comparator compares the second pixel signal with the reference signal when the normal pixel is driven, and the counter and the comparator are disposed within an analog-to-digital converter.

8. The image sensor according to claim 7, further comprising an interpolation processing unit that interpolates a digital signal corresponding to the first pixel signal, wherein the analog-to-digital converter converts the first pixel signal into the digital signal when the photometric amount exceeds the threshold, and the vertical drive unit starts exposure of the normal pixels in a predetermined row simultaneously with the photometric pixels.

9. The image sensor according to claim 7, wherein the analog-to-digital converter converts the first and second pixel signals into the digital signals in sequence when the photometric amount exceeds the threshold, and the vertical drive unit starts exposure of the normal pixels in a predetermined row when a predetermined time has elapsed since the start of exposure of the photometric pixels.

10. The image sensor of claim 9, wherein the level of the first pixel signal includes a first reset level when the floating diffusion layer is initialized and a first signal level when charge is transferred, and the level of the second pixel signal includes a second reset level when the floating diffusion layer is initialized and a second signal level when charge is transferred, and the analog-to-digital converter converts the digital signal in the order of the first signal level, the first reset level, the second reset level, and the second signal level.

11. A photodetector comprising: a pixel array section in which a predetermined number of photometric pixels and a predetermined number of normal pixels are arranged; a comparator that compares a first pixel signal from the photometric pixels with a predetermined reference signal and outputs the comparison result; an exposure determination section that performs photometry based on the comparison result and determines whether the photometric amount exceeds a predetermined threshold; a vertical drive section that starts exposure of the normal pixels and the photometric pixels and drives the photometric pixels to output the first pixel signal, and if the photometric amount exceeds the threshold, ends the exposure and drives the normal pixels to output a second pixel signal; and an image processing section that processes image data generated from the second pixel signal.

12. A control method for an image sensor comprising: a comparison step of comparing a first pixel signal from a photometric pixel in a pixel array section in which a predetermined number of photometric pixels and a predetermined number of normal pixels are arranged with a predetermined reference signal and outputting the comparison result; an exposure determination step of performing photometry based on the comparison result and determining whether the photometric amount exceeds a predetermined threshold; and a vertical drive step of starting exposure of the normal pixels and the photometric pixels and driving the photometric pixels to output the first pixel signal, and if the photometric amount exceeds the threshold, terminating the exposure and driving the normal pixels to output a second pixel signal.

Images