Image pickup element and image pickup device

Abstract

To provide an imaging device capable of substantially improving imaging performance designated for each screen region while maintaining an AD conversion rate of an imaging element, so as to output a video signal with improved effective image quality of the whole screen.SOLUTION: An imaging element comprises: a pixel array divided into pixel blocks and controlled; and a signal reading circuit consisting of column reading circuits. The column reading circuit comprises: an analog integrator that has function of performing integration while applying gains different from each other to pixel signals from respective pixels of the pixel block; and an ADC that converts an output of the analog integrator into a digital signal. An imaging device comprises a signal processing unit that, on the basis of a mode selection signal which designates a priority imaging performance for each the pixel block, performs calculation according to the mode selection signal to an output signal of the imaging element, and thereby, generates a video signal.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image sensor and an image pickup apparatus for acquiring a video signal. [Background technology]

[0002] In recent years, signal processing in imaging devices has been digitized, and image output from imaging elements is also a digital signal. In order to support higher resolution (multiple pixels), column-parallel ADCs, in which an ADC (analog-to-digital converter) is provided for each column of a pixel array, are widely used in high-speed pixel signal readout circuits of CMOS (Complementary Metal Oxide Semiconductor) imaging elements (for example, Patent Document 1).

[0003] Signal readout period T per pixel row of an image sensor using a column-parallel ADC H is the number of pixels V in the vertical direction (signal readout direction) of the image sensor and the frame rate f f Therefore, to realize a high-resolution, high-frame-rate image sensor, it is essential to increase the speed of the readout circuit. For example, the signal readout period T H Furthermore, when performing fixed pattern noise removal using digital CDS (correlated double sampling) (for example, Patent Document 2), it is necessary to perform AD conversion of both the reset level and the signal level of the pixel for one readout operation, so high-speed AD conversion of 0.5 μs or less is required.

[0004] Also, in order to substantially improve image quality with a limited AD conversion speed, it has been proposed to provide an image sensor with an operation mode specialized for a specific imaging performance such as frame rate and resolution (for example, Patent Document 3). By using such an image sensor, a mode suitable for the shooting scene is selected, such as a high frame rate / low resolution mode when shooting a fast-moving subject, and a low frame rate / high resolution mode when shooting a high-definition subject with little movement. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2020-188360 A [Patent Document 2] JP 2009-296423 A [Patent Document 3] JP 2020-141405 A Summary of the Invention [Problem to be solved by the invention]

[0006] Generally, the operating speed of an ADC is in a trade-off relationship with performance such as noise performance, bit depth, power consumption, ADC area, etc. As described above, in a multi-pixel high-speed image sensor, the time allowed for AD conversion becomes shorter, and issues arise such as deterioration of noise performance, insufficient bit depth, increased power consumption, and increased chip area due to the limitations of the ADC operating speed.

[0007] In addition, even if an image sensor is provided with an operation mode specialized for a specific imaging performance, when performing wide-field imaging such as 360° video, it is expected that multiple imaging scenes with different characteristics will be captured simultaneously on the same imaging screen. In this case, for example, if imaging in a high frame rate mode is selected to match a subject moving quickly on the screen, the definition of a stationary high-definition subject captured simultaneously on the same screen will decrease. Conversely, if imaging in a high resolution mode is selected to match a high-definition subject on the screen, the fast-moving subject will be blurred, and the movement will not be captured.

[0008] Furthermore, when an image is taken by selecting an operating mode of the image sensor, the output signal of the image sensor itself becomes a video signal of the selected imaging performance, which poses the problem that it is not possible to change the image quality after taking the image.

[0009] Therefore, in consideration of the above-mentioned problems, an object of the present invention is to provide an imaging element that can improve the imaging performance selected for each screen area consisting of multiple pixels by changing the processing method of the output signal while maintaining the AD conversion speed in the imaging element.

[0010] Furthermore, in consideration of the above-mentioned problems, an object of the present invention is to provide an imaging device that can substantially improve the imaging performance (resolution, frame rate, or noise performance) specified for each screen area while maintaining the AD conversion speed of the image sensor, and output a video signal with improved effective image quality over the entire screen. [Means for solving the problem]

[0011] In order to solve the above problems, the imaging element of the present invention is an imaging element comprising a pixel array which is controlled by blocking into pixel blocks each consisting of a plurality of adjacent pixels, and a signal readout circuit consisting of a column readout circuit corresponding to a pixel block column of the pixel block, wherein the column readout circuit comprises an analog integrator having a function of integrating pixel signals of each pixel in the pixel block by applying mutually different gains, and an ADC (analog-to-digital converter) which converts the output of the analog integrator into a digital signal.

[0012] It is also preferable that the column readout circuit of the imaging device further performs correlated double sampling, which subtracts a reset level output of a pixel from a signal level output of the pixel.

[0013] It is also preferable that the imaging device varies the combination of the pixel signal of each pixel and the gain to be applied for each frame, with the number of frames corresponding to the number of pixels in the pixel block being a period.

[0014] It is also preferable that in the imaging device, the column readout circuits are arranged above and below a column of pixel blocks in the pixel array, one controlling the pixel blocks in odd-numbered rows and the other controlling the pixel blocks in even-numbered rows.

[0015] In order to solve the above problems, the imaging device of the present invention is characterized in having an imaging element and a signal processing unit that performs a calculation on an output signal of the imaging element in accordance with a mode selection signal that specifies a priority imaging performance for each pixel block, and generates a video signal.

[0016] In the imaging device, it is preferable that the mode selection signal is any one of resolution priority, frame rate priority, and noise performance priority.

[0017] Furthermore, when the mode selection signal is resolution priority, it is desirable that the imaging device determines the video signal of the pixel block by arithmetic processing for each pixel, and each pixel outputs a constant signal value for a period of a number of frames corresponding to the number of pixels in the pixel block.

[0018] Furthermore, when the mode selection signal indicates high frame rate priority, it is preferable that the image pickup device obtains a video signal of the pixel block for each frame, and that all pixels of the pixel block output the same signal value for each frame.

[0019] Furthermore, when the mode selection signal is noise performance priority, it is desirable that the imaging device output, as the video signal of the pixel block, the same signal value for a period of a number of frames corresponding to the number of pixels in the pixel block for all pixels of the pixel block. Effect of the Invention

[0020] According to the imaging element of the present invention, the imaging performance selected for each screen area can be improved by changing the processing method of the output signal while maintaining the AD conversion speed.

[0021] Furthermore, according to the imaging device of the present invention, it is possible to substantially improve the imaging performance (resolution, frame rate, or noise performance) specified for each screen area while maintaining the AD conversion speed of the imaging element, and output a video signal with improved image quality across the entire screen. [Brief description of the drawings]

[0022] [Figure 1] 1 is a diagram illustrating an example of the configuration of an imaging device according to an embodiment of the present invention. [Diagram 2] 1 is an example of a block diagram of an image sensor according to an embodiment of the present invention; [Diagram 3] FIG. 2 is a diagram showing an example of a circuit configuration of one pixel. [Figure 4] FIG. 2 is a diagram illustrating an example of a circuit configuration of a pixel block and a column readout circuit of an image sensor. [Diagram 5] 5A and 5B are diagrams illustrating an example of a timing of a readout operation and an output signal of an image sensor. [Figure 6] 4 is an example of a timing diagram of a drive signal for an image sensor. [Figure 7] FIG. 2 is a diagram illustrating an example of signal processing in an imaging element. [Figure 8] 11 is an example of a block diagram of a column signal processing unit of the signal processing unit. [Figure 9] FIG. 11 is an example of a block diagram of an image sensor according to another embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0024] (Embodiment) 1 shows an example of the configuration of an image pickup device according to an embodiment of the present invention. The image pickup device is composed of a lens 1, an image sensor 2, and a signal processing unit 3. A mode selection signal F generated by a mode selection signal generating unit 4 is input to the image pickup device.

[0025] A lens 1 forms an image of a subject on a pixel array light receiving surface of an image sensor 2. The image sensor 2 converts the image of the subject into image sensor output signals D(1), D(2), ..., D(H) and transmits them to a signal processor 3. The signal processor 3 includes H column signal processors 50, each of which (1)-(H) receives image sensor output signals D(1), D(2), ..., D(H) and mode selection signals F(1), F(2), ..., F(H), and outputs output video signals X(1), X(2), ..., X(H), respectively.

[0026] The configuration and operation principle of the image sensor and the image pickup device will be described in detail below.

[0027] (1) Image sensor 2 shows an example of a block diagram of an image sensor 2 of the present invention. The image sensor 2 includes a pixel array 100, a row selection / pixel drive circuit 110, a signal readout circuit 200, and a drive signal generation circuit 210.

[0028] The pixel array 100 is configured and controlled by blocking adjacent k×l pixel regions (k pixels horizontally and l pixels vertically, where k and l are integers equal to or greater than 2) 10 as one block unit. Here, the pixel array 100 will be described as being configured of H horizontal blocks×V vertical blocks (2H horizontal pixels×2V vertical pixels) with 2×2 pixel regions as pixel blocks 10. The size (number of pixels) of the pixel blocks 10 can be freely set based on image resolution when a high frame rate is prioritized. Two (k lines, i.e., one line for each pixel column) signal readout lines are wired in the vertical direction for each pixel block column (V pixel blocks 10 arranged vertically), and are connected in parallel to H column readout circuits 201.

[0029] Each pixel (Pixel 1 to Pixel 4 in this embodiment) of the pixel block 10 is composed of, for example, a 4Tr (transistor) type pixel. An example of the circuit configuration of one pixel is shown in Fig. 3. The pixel includes a photodiode (PD) 11, a transfer gate 12, a floating diffusion (FD) 13, an output transistor (source follower circuit) 14, a selection transistor 15, and a reset transistor 16.

[0030] A photodiode (PD) 11 performs photoelectric conversion in response to incident light and generates a signal charge based on the amount of exposure.

[0031] The transfer gate 12 is controlled by a charge transfer signal TX. When the signal TX goes high, the transfer gate 12 becomes conductive and transfers the signal charge generated in the photodiode (PD) 11 to the floating diffusion (FD) 13 .

[0032] The floating diffusion (FD) 13 essentially functions as a capacitor and accumulates the transferred signal charge. The potential of the floating diffusion (FD) 13 based on the signal charge is applied to the gate electrode of the output transistor 14.

[0033] The output transistor 14 constitutes a source follower circuit, and outputs an output signal (output voltage) to the source side in response to a voltage applied to the gate electrode.

[0034] The selection transistor 15 is controlled by a row selection signal SL, and is turned on when the signal SL goes High, and outputs an output signal from the output transistor (source follower circuit) 14 to a signal read line 17.

[0035] The reset transistor 16 is controlled by an FD reset signal RT, and when the signal RT goes high, it becomes conductive, discharges the signal charge accumulated in the floating diffusion (FD) 13, and resets the output of the pixel.

[0036] In this way, output signals from the pixels (Pixel 1 to Pixel 4) of each pixel block 10 are output to the signal read line 17. Note that the pixel structure is not limited to the above-mentioned 4Tr type pixel, and any structure may be used as long as the pixel has a configuration for outputting an output signal based on incident light to the signal read line 17.

[0037] Returning to FIG. 2, the row selection and pixel drive circuit 110 selects a row of the pixel array 100 and drives the pixels of the selected row. Row selection selects and drives pixels for each pixel block row (H pixel blocks 10 arranged in the horizontal direction). The row selection and pixel drive circuit 110 simultaneously drives the pixel blocks 10 in the same pixel block row (m rows) with the same control signal, but supplies different pixel selection signals and drive signals to each pixel (Pixel 1 to Pixel 4) in the pixel block 10 to control each pixel. In this embodiment, the selection signals for each pixel (Pixel 1 to Pixel 4) of the pixel block 10 in the mth row are designated as SL1(m) to SL4(m), and the pixel drive signals are designated as TX1(m) to TX4(m) and RT1(m) to RT4(m), respectively.

[0038] The signal readout circuit 200 is made up of H column readout circuits 201 (column readout circuits (1) to (H)) corresponding to H horizontal blocks of the pixel array 100. The column readout circuit 201 is made up of a switched capacitor integrator (hereinafter, SC integrator) 20, an ADC 30, and a digital CDS (hereinafter, DCDS) 40 from the upstream side where the signal is input. Each column readout circuit (1) to (H) reads out the output signal of a selected and driven pixel block row, performs processing such as integration and AD conversion, and outputs the signal as image sensor output signals D(1), D(2), ..., D(H).

[0039] The drive signal generation circuit 210 supplies the same SC integrator control signal and ADC control signal to the column readout circuits (1) to (H), and further controls the DCDS. Each column readout circuit 201 operates simultaneously by the drive signal supplied from the drive signal generation circuit 210.

[0040] FIG. 4 shows an example of a circuit configuration of the pixel block 10 and the column readout circuit 201 of the image sensor 2. As shown in FIG.

[0041] The pixel block 10 is composed of, for example, 4Tr type pixels Pixel 1 to Pixel 4 described in FIG. 3. Here, the pixel block 10 in the nth column and mth row is illustrated. Signals SL1(m) to SL4(m) for selecting each pixel (Pixel 1 to Pixel 4) in the mth row of the pixel block 10, charge transfer signals TX1(m) to TX4(m) for pixel driving, and FD reset signals RT1(m) to RT4(m) are input from the row selection / pixel drive circuit 110 to the pixel block 10. The outputs of Pixels 1 and 3 and Pixels 2 and 4, which are aligned in the vertical direction, are each connected to the same signal readout line 17. The two signal readout lines 17 are each connected to a bias voltage V b is connected to the drain terminal of a common-source NMOS transistor 18 having a gate connected to a switch φ L ,φ R 1. The input terminal of the SC integrator 20 is connected to the input terminal of the SC integrator 20 via the

[0042] The SC integrator 20 includes a sampling capacitor 21 (capacity C 1a ),22(capacity C 1b ),23(capacity C 1c ), an integrating capacitor 24 (capacitor C2), an OP amplifier 25, and a switch φ SMP ,φ AMP ,φ RT ,φ 1b ,φ 1c It is composed of switch φ RT When the switch φ is turned ON (conducting), the charge accumulated in the integrating capacitor 24 is initialized. SMP is ON, φ AMP When the switch φ is turned off, the sampling capacitor is charged by the input voltage. SMP is OFF, φ AMP When the switch is turned ON, the charge stored in the sampling capacitor is transferred to the integrating capacitor 24. The SC integrator 20 performs the above sampling and charge transfer operation four times (k×l times) corresponding to the number of pixels in the pixel block 10 (k×l pixels: 4 in this embodiment), and performs an integrating operation of the input voltage. In other words, the SC integrator 20 functions as an analog integrator.

[0043] The sampling capacitance of the SC integrator 20 is the switch φ 1b ,φ 1c By operating, C 1a ,C 1a +C 1b ,C 1a +C 1c ,C 1a +C 1b +C 1c When each sampling capacitance is selected, the gains given to the input signal in the integration operation are G1, G2, G3, and G4, respectively. The gains G1 to G4 are given by the following equations (1) to (4), respectively.

[0044]

number

[0045]

number

[0046]

number

[0047]

number

[0048] Here, G1 to G4 are the inverse matrices G of the matrix G shown in the following formula (5). -1 must be set to exist.

[0049]

number

[0050] In this embodiment, the gains are G1=1, G2=2, G3=3, and G4=4 (C 1a =C2,C 1b =C2,C 1c= 2C2). Note that this value is just an example, and the values ​​of the gains G1 to G4 are different from one another. The inverse matrix G of the matrix G shown in the formula (5) is set as -1 If the pixel block has k×l pixels, kl types of gains G1 to G kl Using Equation (5), G1 to G kl It becomes a kl × kl matrix with these as elements.

[0051] The ADC 30 converts the output of the SC integrator 20 into a digital signal.

[0052] The DCDS 40 includes a memory 41 and an adder 42. The ADC output of the pixel reset level is stored in the memory 41, and the reset level ADC output is subtracted from the ADC output of the pixel signal level that is subsequently output, to obtain an image sensor output signal. The output signals are output in parallel from each column readout circuit 201 as signals D(n) (n=1, 2, ..., H) from the image sensor 2.

[0053] 5 shows an example of a timing diagram of the readout operation and output signal of the image sensor 2. The row selection / pixel drive circuit 110 reads out the signal from the image sensor 2 during one frame period T f =1 / f f (f f is the frame frequency), scanning is performed from the 1st row to the Vth row of the pixel block. In this embodiment, the number of frames corresponding to the number of pixels in the pixel block 10 is set as a period (k×l frame period, here 4 frame period) and the readout circuit performs a different readout operation for each frame. That is, readout operation (A) in the first frame, readout operation (B) in the second frame, readout operation (C) in the third frame, and readout operation (D) in the fourth frame are each repeated V times to obtain an output signal for each frame. The period of each readout operation per pixel block row is T H =T f / V=1 / f f / V.

[0054] FIG. 6 is a timing diagram showing an example of a drive signal for an image sensor. For each of the readout operations (A) to (D), a switch φ1b ,φ 1c Only the operation timing of (A) to (D) differs from one another, and the other operation timings are common to all frames (read operations (A) to (D)).

[0055] Pixel drive signals SL, RT, TX, and switch φ L ,φ R By this operation, pixel signals are input to the SC integrator 20 in the order of the reset level (Pixel 1→2→3→4) and the signal level (Pixel 1→2→3→4).

[0056] The SC integrator 20 is a switch φ RT After resetting the charge of the integrating capacitor 24 by 1b ,φ 1c The input signals are charged with different sampling capacitances according to the operation of the integrating capacitor 24, and the charged charges are transferred to the integrating capacitor 24 for integration. At this time, gains (G1 to G4) expressed by the above-mentioned formulas (1) to (4) according to the sampling capacitance are applied to each input signal. The output voltage of the integrator after integration is converted to a digital signal by the ADC 30. Then, the switch φ RT The charge of the integrating capacitor 24 is reset by this, and the same operation as above is performed on the subsequently input signals of signal levels Pixel 1 to 4 to obtain a digital signal of the ADC 30.

[0057] The DCDS 40 subtracts two digital signals to generate an image sensor output signal. The image sensor output of the i-th frame (i=1, 2, 3, 4) read from the pixel block in the m-th row and n-th column (m=1, 2, ..., V, n=1, 2, ..., H) is converted into a D mn,i If we write it as D mn,i is expressed by the following equation (6).

[0058]

number

[0059] Here, S mn,ijrepresents the pixel signal (difference between signal level and reset level) of Pixel j (j=1,2,3,4) in the ith frame (i=1,2,3,4) in the pixel block in the mth row and nth column. In equation (6), gains G1 to G4 are cyclically multiplied by the pixel signal, and the combination of the pixel signal of each pixel and the gain applied to the pixel signal is different for each frame. The signal group represented by equation (6) is output as the image sensor output signal D(n) in the output order of equation (7) below.

[0060]

number

[0061] FIG. 7 illustrates an example of signal processing in an image sensor. FIG. 7 shows signal calculations for the n-th column of the image sensor described above. From the left, pixel signals (S mn,ij : i-th frame, Pixel j). These pixel signals are subjected to calculations (above equation (6)) by readout operations (A) to (D) for each frame. mn,i (i is the frame) are arranged in order to produce an image sensor output signal D(n). This image sensor output signal D(n) is sent to the signal processing unit 3.

[0062] The image sensor of the present invention constantly outputs the image sensor output signal D(n) that has been subjected to the above-mentioned processing while maintaining the AD conversion speed. By changing the processing method of this image sensor output signal D(n) as described later, it is possible to improve the imaging performance selected for each screen region.

[0063] (2) Mode selection signal generator The mode selection signal generation unit 4 generates a mode selection signal F. The mode selection signal F(n) input to the column readout circuit (n) is expressed by the following equation (8).

[0064]

number

[0065] Each element F of the mode selection signal F(n) mn specifies the imaging performance to be prioritized in the pixel block in the nth column and mth row. In this embodiment, it is defined as 0 to prioritize resolution, 1 to prioritize frame rate, and 2 to prioritize noise performance. The method of generating the mode selection signal F(n) is not limited in the present invention, but it is preferable to use it as a feedback signal of the captured image. For example, a method can be considered in which the spatiotemporal frequency components and noise components of a video signal acquired by this imaging device or another imaging device are analyzed, and the mode selection signal F(n) is generated in real time based on the results.

[0066] Also, the element F of the mode selection signal F(n) mn (Element F mn (sometimes simply referred to as the mode selection signal) is set according to which of the conditional expressions (13) to (15) described later the pixel signal satisfies. mn It is preferable to select the mode selection signal F mn is preferably updated once for a group of read operations corresponding to the number of pixels in the pixel block (four frames in this embodiment). Alternatively, an appropriate update period may be selected according to the moving speed of the subject or the rate of change over time of the shooting scene.

[0067] In this embodiment, the mode selection signal generator 4 is disposed outside the imaging device, but may be provided inside the imaging device as necessary.

[0068] (3) Signal processing section The signal processing unit 3 is composed of a plurality (H units) of column signal processing units 50, and performs a calculation on the output signal of the image sensor 2 according to the mode selection signal. FIG. 8 shows an example of a block diagram of the column signal processing unit 50 of the signal processing unit 3. The column signal processing unit (n) receives the image sensor output signal D(n) of the image sensor and the mode selection signal F(n), and generates an output signal X(n). The column signal processing unit 50 includes a memory 51 and a calculation unit 52. The memory 51 has a memory capacity capable of holding 4V (k×l×V) signals according to the number of elements of the image sensor output signal D(n). The calculation unit 52 is composed of a calculation element such as a CPU (Central Processing Unit), for example.

[0069] The input image sensor output signal D(n) is temporarily stored in the memory 51. Then, D mn,1 ,D mn,2 ,D mn,3 ,D mn,4 are simultaneously transmitted to the calculation unit 52.

[0070] The calculation unit 52 receives the D mn,1 ,D mn,2 ,D mn,3 ,D mn,4 The mode selection signal F mn The following calculation (i) to (iii) is performed according to the value of

[0071]

number

[0072]

number

[0073]

number

[0074] where X mn,ij indicates the output signal (pixel-by-pixel video signal) of the ith frame (i=1,2,3,4) in the mth row, nth column pixel block, Pixel j (j=1,2,3,4). Therefore, according to equation (9), each pixel outputs a different signal value (has resolution information) due to different arithmetic processing, but outputs a constant signal value (frame direction information is lost) during a frame period (four frames) corresponding to the number of pixels. According to equation (10), the output signal of the pixel block is calculated for each frame (has frame direction information), but all pixels in the pixel block output the same signal value (resolution information is lost). Also, according to equation (11), all pixels in the pixel block output the same signal value (resolution information and frame direction information are lost) during a frame period (four frames) corresponding to the number of pixels, but noise is reduced.

[0075] By performing the above calculation for m=1, 2, . . . , V, the output video signal X(n) expressed by equation (12) is generated.

[0076]

number

[0077] In the case of calculating the equations (9) to (11) corresponding to the above conditions (i) to (iii), in particular, the pixel signal S mn,ij When the following conditions expressed by the formulas (13) to (15) are satisfied, the mode selection signal F mn The imaging performance specified by is improved, and the impact of the imaging performance sacrificed accordingly is limited.

[0078]

number

[0079]

number

[0080]

number

[0081] For example, equation (13) means that for four consecutive frames of signals output from adjacent 2 × 2 pixels, each of the four pixels outputs a different signal, but the signal hardly changes in the frame direction. For example, this applies to an area where a stationary, high-resolution subject is captured.

[0082] In addition, equation (14) shows that for four consecutive frames of signals output from adjacent 2 × 2 pixels, the four pixels output almost equal signals frame by frame, but the signals are different in the frame direction. For example, this applies to areas where there is a large motion blur in the imaging area of ​​a fast-moving subject, or areas where the defocused background is blurred when following the movement of a subject.

[0083] Moreover, equation (15) means that the output signals are almost equal for adjacent 2 × 2 pixels in the four consecutive frame directions. For example, this applies to areas where the camera is fixed and subjects with little movement and no fine patterns, such as the sky or a flat wall, are captured, or areas where a defocused stationary object is captured.

[0084] The output signal X of the imaging device obtained by the calculation of equations (9) to (11) mn,ij As long as the conditions shown in equations (13), (14), and (15) are satisfied,

[0085]

number

[0086] That is, by performing the calculations of (9), (10), and (11) under the conditions of (13), (14), and (15) for any m and n, the mode selection signal (F mn ) can be locally improved.

[0087] For example, if equation (9) is calculated under the condition that equation (13) is satisfied, imaging with a local resolution equivalent to 2H × 2V can be achieved. In this case, the frame rate is f f / 4, but the effect of the frame rate drop is limited due to the condition of equation (13). Also, by calculating equation (10) under the condition that equation (14) is satisfied, the frame rate f f At this time, the resolution will drop to H×V, but the effect is limited due to the condition of equation (14). When the calculation of equation (11) is performed under the condition that equation (15) is satisfied, the signal processing unit performs averaging of the image sensor output signal in addition to the integration operation in the image sensor, so that imaging with locally reduced random noise components can be achieved. f / 4, but the effect of these is limited due to the condition in equation (15).

[0088] According to the imaging device of the present invention, the output signal S of the pixel block mn,ij The formula ( 13 )~( 15 When any of the conditions shown in (1) and (2) are satisfied, the mode selection signal (F mn ) to prioritize the specified imaging performance in each area (resolution 2H x 2V pixels / frame rate f f It is possible to select either low-noise operation or low-noise operation, and the effect of the sacrifice of imaging performance on image quality can be limited accordingly.

[0089] In the conventional technology, it was not possible to change the imaging performance for each region (for example, the minimum conversion period T given by the operation speed constraints of the ADC). H (T H =1 / f f / V), the number of pixels H×V and the frame rate f fOn the other hand, according to the present invention, the image quality of the entire screen is improved compared to the conventional technology by prioritizing resolution in areas with high spatial frequency and low temporal frequency, frame rate in areas with low spatial frequency and high temporal frequency, and noise in areas where both are low. In addition, the specified imaging performance can be improved by changing the subsequent signal processing of the output signal from the imaging element after shooting.

[0090] (Other embodiments) FIG. 9 shows an example of a block diagram of an image sensor according to another embodiment of the present invention.

[0091] In the above embodiment, H column readout circuits 201 are provided below the pixel array 100, and the readout circuits are connected in parallel to the pixel block columns, but the number of column readout circuits in parallel to the pixel block columns is arbitrary. For example, as shown in FIG. 9, signal readout circuits (column readout circuits 201) may be provided above and below the pixel array 100 (of the pixel block columns), and the total number of column readout circuits 201 may be increased to 2H. In this case, by using a wiring structure such as connecting odd-numbered rows of pixel blocks to signal lines to the upper circuits and even-numbered rows to signal lines to the lower circuits as shown in FIG. 9, the number of column readout circuits 201 in parallel to the pixel block columns can be doubled while keeping the column pitch of the signal readout circuits 200 constant. For this reason, even if the conversion period T H Frame rate f f can be doubled.

[0092] In the above embodiment, the CDS operation is performed in the digital domain using the DCDS 40. R ,φ L An analog CDS circuit may be installed between the sampling capacitance of the SC integrator and the sampling capacitance of the SC integrator to perform CDS in the analog domain. In that case, the integration and AD conversion operations of both the signal level and the reset level shown in the embodiment are not necessary, and only the output signal of the analog CDS is integrated and AD converted before being output.

[0093] Mode selection signal (Fmn In addition to the methods of generating the above-mentioned embodiment, the following methods (a) to (c) are also possible.

[0094] (a) A value arbitrarily set by the user is input as a fixed value or a value that is updated at a certain update frequency.

[0095] (b) The area of ​​interest, focus area, and subject movement speed on the screen are determined based on the subject position and position information obtained by the distance sensor and lens parameters (focus position, focal length, f-number, etc.), and the image is generated in real time based on the results.

[0096] (c) Using image analysis techniques such as object recognition, the area is divided into areas for each subject, and an appropriate mode selection signal F is generated according to the characteristics of each subject (estimated spatial frequency components and moving speed). mn is automatically given.

[0097] In the above embodiment 1, the configuration and operation of the image sensor and the image pickup device are described, but the present invention is not limited to this, and may be configured as a signal processing method for processing an imaged signal. That is, the present invention may be configured as a signal processing method for processing a signal imaged by a pixel based on a mode selection signal in accordance with the processing of the data in each figure.

[0098] Although the above-mentioned embodiment has been described as a representative example, it is clear to those skilled in the art that many modifications and substitutions can be made within the spirit and scope of the present invention. Therefore, the present invention should not be interpreted as being limited by the above-mentioned embodiment, and various modifications or changes can be made without departing from the scope of the claims. For example, the functions included in each block, each step, etc. described in the embodiment can be rearranged so as not to be logically inconsistent, and multiple configuration blocks, steps, etc. can be combined into one or divided. [Explanation of symbols]

[0099] 1 Lens 2. Image sensor 3. Signal Processing Section 4. Mode selection signal generator 10 pixel blocks 11 Photodiode (PD) 12 Transfer Gate 13 Floating Diffusion (FD) 14 Output transistor 15 Select transistor 16 Reset transistor 17 Signal readout line 18 NMOS transistor 20 Switched Capacitor (SC) Multiplier 21-23 Sampling capacity 24 Accumulated Capacity 25 OP amps 30 ADC (Analog / Digital Converter) 40 Digital CDS (Correlated Double Sampling) 41 Memory 42 Adder 50 Column signal processing section 51 Memory 52 Arithmetic section 100 pixel array 110 Row selection and pixel driving circuit 200 Signal readout circuit 201 Column readout circuit 210 Drive signal generating circuit

Claims

1. a pixel array which is divided into pixel blocks each including a plurality of adjacent pixels and controlled; an image sensor comprising a signal readout circuit including a column readout circuit corresponding to a pixel block column of the pixel block, The column readout circuit includes: an analog integrator having a function of integrating pixel signals of each pixel of the pixel block by applying different gains to the pixel signals; an ADC (analog / digital converter) for converting the output of the analog integrator into a digital signal; An imaging element comprising:

2. 2. The imaging device according to claim 1, The image pickup device, wherein the column readout circuit further performs correlated double sampling for subtracting a reset level output of the pixel from a signal level output of the pixel.

3. 3. The imaging device according to claim 1, An imaging device, characterized in that a combination of a pixel signal of each pixel and an applied gain is changed for each frame, with the number of frames corresponding to the number of pixels in the pixel block being a period.

4. 4. The imaging device according to claim 1, the column readout circuits are arranged above and below a column of pixel blocks in the pixel array, one of which controls the pixel blocks in odd-numbered rows and the other of which controls the pixel blocks in even-numbered rows.

5. The imaging device according to claim 1 , a signal processing unit that performs a calculation corresponding to a mode selection signal specifying a priority imaging performance for each pixel block on an output signal of the imaging element to generate a video signal; An imaging device comprising:

6. 6. The imaging device according to claim 5, 13. An imaging apparatus, wherein the mode selection signal is any one of a resolution priority, a frame rate priority, and a noise performance priority.

7. 7. The imaging device according to claim 5, an imaging device characterized in that, when the mode selection signal is resolution priority, a video signal of the pixel block is calculated for each pixel by arithmetic processing, and each pixel outputs a constant signal value for a period of a number of frames corresponding to the number of pixels of the pixel block.

8. 7. The imaging device according to claim 5, an imaging device, characterized in that, when the mode selection signal indicates high frame rate priority, a video signal of the pixel block is obtained for each frame, and all pixels of the pixel block output the same signal value for each frame.

9. 7. The imaging device according to claim 5, an imaging device, characterized in that, when the mode selection signal is noise performance priority, all pixels of the pixel block output the same signal value for a period of a number of frames corresponding to the number of pixels of the pixel block.

Images