Imaging device and image processing method

Abstract

To provide an imaging apparatus capable of realizing appropriate correction while suppressing increase in a correction value, and its control method.SOLUTION: The imaging apparatus comprises: an image pickup device which includes a pixel part in which a plurality of pixels are arranged in matrix and a plurality of column signal lines provided at each of the columns of the pixels and is capable of reading a plurality of types of pixel signals from the pixels; a first correction circuit; and a second correction circuit. The first correction circuit and the second correction circuit share application of correction of offset caused by the difference of the column signal lines from which the pixel signal is read.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to an imaging device and an image processing method, and particularly to an image correction technique.

Background Art

[0002] An imaging device such as a digital (video) camera uses an image sensor. The image sensor has millions to tens of millions of fine pixels, and a voltage value corresponding to the amount of charge accumulated by photoelectric conversion in each pixel is acquired as an image signal.

[0003] Variations in signal values due to variations in the characteristics of the photodiodes and transistors of each pixel become noise components included in the image signal. Therefore, it is necessary to apply correction for removing or suppressing the noise components to the read image signal.

[0004] In addition, there are variations in the characteristics of the readout circuits (vertical signal lines and column circuits) provided for each pixel column for reading signals from the pixels. Therefore, it is also necessary to correct the influence of the variations in the characteristics of the readout circuits on the pixel signals.

[0005] Patent Document 1 discloses correcting variations in characteristics using correction values for each readout circuit in an imaging device having a configuration in which a plurality of readout circuits are provided for each pixel column.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0007] In order to correct the variations in the characteristics of the readout circuit with high precision, it is necessary to use different correction values for each readout circuit and for each type of signal to be read. For example, each pixel may have two sub-pixels A and B, and there are cases where the pixel signal is read out collectively from both sub-pixels A and B, and cases where the pixel signal can be read out from only one of sub-pixels A or B. In this case, it is necessary to use separate correction values for each case. Therefore, when two types of signals can be read out from each pixel and six vertical signal lines are provided for each pixel column, twelve types of correction values are required.

[0008] When the correction values to be stored increase, the required memory capacity and component cost increase. Also, since the time required to read the correction values from the memory at the start-up of the imaging device increases, it becomes a factor in the reduction of the responsiveness of the imaging device.

[0009] In addition, there are limitations on the amount of data and the data rate that can be communicated to the correction circuit for correcting the image signal within one frame. Therefore, an increase in the correction values used limits the amount of image data that can be transmitted to the correction circuit and becomes a factor in preventing an improvement in the shooting frame rate and the number of pixels per frame.

[0010] In one aspect of the present invention, there is provided an imaging device and a control method thereof capable of realizing appropriate correction while suppressing an increase in correction values.

Means for Solving the Problems

[0011] The above object is achieved by an imaging device having an image sensor in which a plurality of pixels are arranged in a matrix, a plurality of column signal lines provided for each column of pixels, and capable of reading an image signal and a focus detection signal from the pixel array, a first correction circuit provided in the image sensor and applying a predetermined correction process to the image signal, and provided outside the image sensor, Synthesize the signals for focus detection for multiple lines, and after synthesis a second correction circuit that applies a predetermined correction process to the focus detection signal.

Effects of the Invention

[0012] According to one aspect of the present invention, it is possible to provide an imaging device and a control method thereof that can realize appropriate correction while suppressing an increase in correction values.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Embodiments for Carrying Out the Invention

[0014] Hereinafter, the present invention will be described in detail based on its exemplary embodiments with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Also, although a plurality of features are described in the embodiments, not all of them are essential to the invention, and a plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant descriptions are omitted.

[0015] In the following embodiments, the case where the present invention is implemented in an imaging device that is a digital camera or a digital video camera will be described. However, the present invention can be implemented in any electronic device having an imaging function. Such electronic devices include computer devices (personal computers, tablet computers, media players, PDAs, etc.), mobile phones, smartphones, game machines, robots, drones, and drive recorders. Note that these are examples, and the present invention can also be implemented in other electronic devices.

[0016] ●(First Embodiment) FIG. 1 is a block diagram showing a configuration example of an imaging element according to an embodiment of the present invention. The imaging element 100 has a pixel unit 101 in which a plurality of pixels 102 are arranged in a matrix. The pixel unit 101 is also called a pixel array. A color filter in a primary color Bayer array is provided in the pixel unit 101. In FIG. 1, R, G, and B attached to the pixel 102 indicate the colors of the color filters provided in the pixel. Also, six vertical signal lines (column signal lines) are provided for each pixel column. Note that the number of vertical signal lines for each pixel column is an example, and can be any plurality, such as 4, 12, 20, etc. In FIG. 1, only 12 pixels in 6 rows and 2 columns are shown out of several million to tens of millions of pixels provided in the pixel unit 101.

[0017] The pixel 102A is connected to the vertical signal line 105A by a selection switch (not shown) and outputs the pixel signal to the column circuit 106A row by row. The pixels of the (6n + 1)-th row (n is an integer of 0 or more) are connected to the vertical signal line 105A every 6 rows from the first row. Similarly, the pixels of the (6n + 2)-th row are connected to the vertical signal line 105B, the pixels of the (6n + 3)-th row are connected to the vertical signal line 105C, the pixels of the (6n + 4)-th row are connected to the vertical signal line 105D, the pixels of the (6n + 5)-th row are connected to the vertical signal line 105E, and the pixels of the 6(n + 1)-th row are connected to the vertical signal line 105F. Also, corresponding column circuits 106A to 106F are connected to each of the vertical signal lines 105A to 105F.

[0018] The on and off of the selection switch are controlled row by row for the pixels by the signal line 103 connected to the vertical scanning circuit 104. In this embodiment, the vertical scanning circuit 104 controls the selection switch at the same timing for a plurality of pixel rows, so that signals can be read out at once from the pixels 102 for up to 6 pixel rows.

[0019] The timing generator (TG) 110 outputs a pulse signal for controlling the operations of the vertical scanning circuit 104 and the horizontal scanning circuit 112. The vertical scanning circuit 104 controls the operations of the transistors included in each pixel 102 of the pixel section 101 based on the pulse signal output from the TG 110. The TG 110 also outputs a reference signal used by the D / A converter 111. The D / A converter (DAC) 111 generates a reference signal (a slope signal or a ramp signal) whose voltage changes with time and supplies it to the comparator 107 included in the column circuit 106.

[0020] Next, the configuration of the column circuit 106 will be described. The column circuit 106 is provided for each vertical signal line 105, and in FIG. 1, column circuits 106A to 106F corresponding to the vertical signal lines 105A to 105F are formed.

[0021] The column circuit 106 is composed of a comparator 107, a counter 108, and a column memory 109. The reference signal generated by the DAC 111 is input to one input of the comparator 107, and the pixel signal from the vertical signal line 105 is input to the other input.

[0022] Comparator 107 compares the voltage value (potential V) of the pixel signal input from the vertical signal line 105 with the voltage value of the reference signal, and changes the level of the output signal when the magnitude relationship of the signals changes. The output signal of comparator 107 controls the operation of counter 108. Counter 108 measures the number of clock pulses input from a signal source (not shown). Counter 108 terminates the measurement of the number of pulses according to the output signal of comparator 107 and outputs the measured value at that time. The measured value of counter 108 is a digital value corresponding to the voltage value of the pixel signal. Note that time may be measured instead of the number of pulses. The measured value of counter 108 is held in column memory 109. Then, the measured value of counter 108 is reset.

[0023] In this way, each column circuit 106 functions as an A / D converter that converts the pixel signal input from the corresponding vertical signal line 105 into a digital signal having a value within a predetermined range (for example, 0 to 255). Column circuit 106A is connected to horizontal signal line 113A via a switch whose on / off is controlled by horizontal scanning circuit 112. Column circuits 106B to 106F are similarly connected to horizontal signal lines 113B to 113F.

[0024] Horizontal scanning circuit 112 controls the on / off of the switch and outputs the digital signal held in column memory 109 of column circuit 106 to horizontal signal lines 113A to 113F. The operation of horizontal scanning circuit 112 is controlled by a control signal supplied from TG 110. The digital signal output to horizontal signal lines 113A to 113F is supplied to first correction circuit 114. First correction circuit 114 applies offset correction to the digital signal according to settings by control circuit 406 described later, or outputs it to the outside of imaging device 100 without applying offset correction. Substrate memory 115 is a memory used when first correction circuit 114 applies offset correction, and is, for example, a DRAM. Details of first correction circuit 114 will be described later.

[0025] FIG. 2 is a circuit diagram showing a configuration example of pixel 102. Pixel 102 has a plurality of photodiodes (PDs) 201A and 201B that share one microlens (not shown). PDs 201A and 201B each function as a photoelectric conversion unit that generates charges according to the incident light amount. PDs 201A and 201B are also each called a sub-pixel.

[0026] Hereinafter, a signal based on the charges generated by PD 201A is called an A signal, and a signal based on the charges generated by PD 201B is called a B signal. Also, a signal obtained by combining the A signal and the B signal obtained from the same pixel is called an A + B signal. An image composed of the A signal and an image composed of the B signal form a parallax image pair. On the other hand, by using the A signal sequence and the B signal sequence obtained from a plurality of pixels 102, automatic focus detection (AF) of the phase difference detection method can be performed. Therefore, the A signal and the B signal are also called focus detection signals or AF signals.

[0027] Also, the A + B signal can be regarded as an image signal obtained by a general imaging device in which pixel 102 has one PD. Therefore, when the A signal and the B signal are called focus detection signals, the A + B signal may be called an image signal or a captured image signal. Note that the A signal may be obtained by subtracting the B signal from the A + B signal, and the B signal may be obtained by subtracting the A signal from the A + B signal. A pixel signal means a signal read out from a pixel, and can be any of the A + B signal, the A signal, the B signal, or the reset signal N described later.

[0028] PD 201A is connected to a common floating diffusion (hereinafter FD) 203 via transfer switch 202A, and PD 201B is connected to common floating diffusion 203 via transfer switch 202B. The on / off of transfer switch 202A is controlled by transfer pulse PTXA, and the on / off of transfer switch 202B is controlled by transfer pulse PTXB. Transfer pulses PTXA and PTXB are supplied from vertical scanning circuit 104.

[0029] FD203 temporarily stores the charge transferred from at least one of PD201A and 201B, and shows a voltage corresponding to the amount of the transferred charge. Therefore, FD203 has a function of converting charge into voltage.

[0030] The reset switch 204 supplies the reference potential VDD to FD203. The on / off of the reset switch 204 is controlled by the reset pulse PRES supplied by the vertical scanning circuit 104.

[0031] The pixel amplifier 205 is a source follower circuit composed of a MOS transistor and a constant current source. The selection switch 206 outputs the potential fluctuation of the pixel amplifier 205 to the vertical signal line 105. The on / off of the selection switch 206 is controlled by the selection pulse PSEL supplied by the vertical scanning circuit 104.

[0032] The transfer switches 202A and 202B, the reset switch 204, and the selection switch 206 are all MOS transistors.

[0033] FIG. 3 is a perspective view schematically showing an implementation example of the imaging device 100. The imaging device 100 can have a structure in which a first semiconductor substrate 301 on which a pixel portion 101 is mainly formed, a readout circuit such as a column circuit 106, a first correction circuit 114, and a second semiconductor substrate 302 on which a substrate memory 115 is formed are stacked. The first semiconductor substrate 301 and the second semiconductor substrate 302 are electrically connected by, for example, bumps, and the column circuit 106 mounted on the second semiconductor substrate 302 can receive the pixel signal read from the pixel 102 provided on the first semiconductor substrate 301.

[0034] FIG. 4 is a block diagram showing a configuration example of an imaging device 400 as an example of an electronic device using the imaging device 100. The photographing lens 401 forms an object optical image on the imaging surface of the imaging device 100. The photographing lens 401 may be detachable from the electronic device and thus exchangeable. The photographing lens 401 has movable lenses such as a focus lens and a diaphragm. The operations of the movable lenses and the diaphragm are controlled by the control circuit 406 through the lens drive circuit 402.

[0035] The shutter 403 is a mechanical shutter and controls the exposure amount of the imaging device 100. The operation of the shutter 403 is controlled by the control circuit 406 through the mechanical drive circuit 404. When the diaphragm also serves as a mechanical shutter, the mechanical drive circuit 404 may be configured to drive the diaphragm and the lens drive circuit 402 may be configured to drive the movable lenses.

[0036] The signal processing circuit 405 applies predetermined image processing to the signals obtained from the imaging device 100, corrects the signals, generates other signals, generates image data, and acquires and / or generates various information. The signal processing circuit 405 also performs the offset correction processing described later. The signal processing circuit 405 may be a dedicated hardware circuit such as an ASIC designed to realize a specific function, or may be configured such that a programmable processor such as a DSP realizes a specific function by executing software. Alternatively, the control circuit 406 may also serve as at least a part of the signal processing circuit 405.

[0037] signal The image processing applied by the processing circuit 405 includes preprocessing, color interpolation processing, correction processing, detection processing, data processing, evaluation value calculation processing, special effect processing, and the like. The preprocessing includes offset correction, signal amplification, reference level adjustment, defective pixel correction, and the like. The color interpolation processing is a process of interpolating the values of color components that cannot be obtained during photographing, and is also called demosaic processing or synchronization processing. The correction processing includes white balance adjustment, tone correction (gamma processing), processing for correcting the influence of optical aberration and peripheral light reduction of the photographing lens 401, color correction processing, and the like. The detection process includes detection of a feature region (for example, a face region or a human body region) and its movement, and person recognition processing, etc. The data processing includes synthesis processing, scaling processing, encoding and decoding processing, header information generation processing, etc. The evaluation value calculation process includes generation of signals and evaluation values used for autofocus detection (AF), and calculation processing of evaluation values used for automatic exposure control (AE), etc. The special effect processing includes addition of blurring, change of color tone, relighting processing, etc. Note that these are signal illustrative examples of image processing applicable to the processing circuit 405, signal and do not limit the image processing applied by the processing circuit 405. Also, it is not necessary for all of the illustrated image processing to be signal executable by the processing circuit 405.

[0038] The control circuit 406 has, for example, a CPU (also called an MPU or a microprocessor), a ROM, and a RAM. The control circuit 406 controls the operations of each part of the imaging device 400 and realizes the functions of the imaging device 400 by reading the program stored in the ROM into the RAM and executing it. When the photographing lens 401 is interchangeable, the control circuit 406 controls the operation of the photographing lens 401 while communicating with the control circuit included in the photographing lens 401.

[0039] The memory 407 is used, for example, to hold image data and offset correction values. The memory 407 is accessible from both the first correction circuit 114 included in the imaging element 100 and the second correction circuit 4051 included in the signal processing circuit 405 provided outside the imaging element 100. Note that the second correction circuit 4051 is not a part of the signal processing circuit 405, and the signal processing circuit 405 may function as the second correction circuit 4051.

[0040] The recording medium 408 is, for example, a removable memory card and is used to record data (for example, imaging image data) based on the signals acquired using the imaging element 100. The display circuit 409 has a display device such as an LCD, and displays information of the imaging device 400, the captured image and its information, the GUI image, and the like.

[0041] The operation member 410 is a general term for input devices provided for the user to give instructions to the imaging device 400. Typical input devices are buttons and switches. Also, when the display circuit 409 is a touch display, the touch panel of the touch display is included in the operation member 410. The input devices included in the operation member 410 are given names according to the assigned functions, such as a shutter button, a power button, a decision button, direction keys, a menu button, and the like. Note that the functions assigned to the same operation member may change according to the operation mode of the imaging device 400 and the like.

[0042] FIG. 5 shows an example of a timing chart when signals are read from the imaging element 100. In the present embodiment, the imaging element 100 has first to third readout modes. FIG. 5 is a timing chart of the operation in the first readout mode. In the first readout mode, it is assumed that a reset signal N, an image signal (A + B signal), and a focus detection signal (A signal) are read from the imaging element 100.

[0043] Before time t500, by setting the reset pulse PRES to Hi and the transfer pulse PTX to Hi, the pixel 102 is reset. At time t500, by setting the transfer pulse PTX to Lo, the reset of the pixel 102 is released, and the accumulation of charges is started. As described above, these control pulses are supplied from the vertical scanning circuit 104 to the pixel 102 based on the control signal supplied from the TG110.

[0044] At time t501, the selection pulse PSEL of the pixel row for reading the signal becomes Hi, and the pixel 102 included in the pixel row is connected to the vertical signal line 105.

[0045] At time t502, the reset pulse PRES changes from Hi to Lo. As a result, the potential V (reset signal N) of FD203 at the reset time is input to the comparator 107 via the vertical signal line 105. The reset signal N is used to remove the reset noise generated when resetting the pixel 102.

[0046] At time t503, the DAC 111 starts outputting a reference signal VL (ramp signal) whose voltage level linearly increases or decreases with time. In the example of FIG. 5, a linearly decreasing reference signal VL is used. When the DAC 111 outputs the reference signal VL, the counter 108 starts counting pulses.

[0047] When the magnitude relationship between the reset signal N and the reference signal VL is reversed at time t504, the output level of the comparator 107 changes from Hi to Lo. When the output level of the comparator 107 changes from Hi to Lo, the count value of the counter 108 is held in the column memory 109.

[0048] After that, after the reference signal VL transitions until it reaches a predetermined lower limit value at time t505, when the horizontal scanning circuit 112 outputs the count value corresponding to the reset signal N from the column circuit 106, the reading (N-reading) of the reset signal N of the pixel 102 is completed.

[0049] When the transfer pulse PTXA becomes Hi at time t506, the charge accumulated in the PD201A is transferred to the FD203. As a result, the potential V of the vertical signal line 105 becomes a potential corresponding to the charge accumulated in the PD201A. This potential corresponds to the value of the A signal. Also, at time t506, the comparator 107 is reset.

[0050] At time t507, the DAC 111 starts outputting the reference signal VL again. When the DAC 111 outputs the reference signal VL, the counter 108 starts counting pulses. When the magnitude relationship between the A signal and the reference signal VL reverses at time t508, the output level of the comparator 107 changes from Hi to Lo. When the output level of the comparator 107 changes from Hi to Lo, the count value of the counter 108 is held in the column memory 109.

[0051] After that, after the reference signal VL transitions until it reaches a predetermined lower limit value at time t509, when the horizontal scanning circuit 112 outputs the count value corresponding to the A signal from the column circuit 106, the reading of the A signal (A read) of the pixel 102 is completed.

[0052] When both of the transfer pulses PTXA and PTXB become Hi at time t510, the charges accumulated in PD201A and PD201B are transferred to FD203. As a result, the potential V of the vertical signal line 105 becomes a potential corresponding to the charges accumulated in PD201A and PD201B. This potential corresponds to the value of the A + B signal. Also, the comparator 107 is reset at time t510.

[0053] At time t511, the DAC 111 starts outputting the reference signal VL again. When the DAC 111 outputs the reference signal VL, the counter 108 starts counting pulses. When the magnitude relationship between the A + B signal and the reference signal reverses at time t512, the output level of the comparator 107 changes from Hi to Lo. When the output level of the comparator 107 changes from Hi to Lo, the count value of the counter 108 is held in the column memory 109.

[0054] After that, after the reference signal VL transitions until it reaches a predetermined lower limit value at time t513, when the horizontal scanning circuit 112 outputs the count value corresponding to the A + B signal from the column circuit 106, the reading of the A + B signal (A + B read) of the pixel 102 is completed.

[0055] Thereafter, predetermined signal processing is performed, such as subtracting the reset signal N from each of the A signal and the A + B signal. Also, the B signal is obtained by subtracting the A signal from the A + B signal. In this way, the image signal (A + B signal) and the focus detection signals (A signal and B signal) are obtained from the pixel 102.

[0056] FIG. 5 shows the read operation from one pixel 102. However, by performing the same operation in parallel for a plurality of pixels 102 connected to the vertical signal lines 105A to 105F and the corresponding column circuits 106A to 106F, signals can be read out in parallel from up to six pixel rows.

[0057] FIG. 6 is a timing chart of the operation in the second read mode of the imaging device 100. In the second read mode, the reset signal N and the image signal (A + B signal) are to be read.

[0058] Since the operation from time t500 to t505 in the first read mode described with reference to FIG. 5 is the same as the operation from time t600 to t605 until the end of the read of the reset signal N (N read), the description thereof is omitted.

[0059] When the transfer pulses PTXA and PTXB become Hi at time t606, the charges accumulated in PD201A and the charges accumulated in PD201B are transferred to FD203. As a result, the potential V of the vertical signal line 105 becomes a potential corresponding to the charges accumulated in PD201A and PD201B. This potential corresponds to the value of the image signal (A + B signal). Also, the comparator 107 is reset at time t606.

[0060] At time t607, the DAC 111 starts outputting the reference signal VL again. When the DAC 111 outputs the reference signal VL, the counter 108 starts counting the pulses. When the magnitude relationship between the A + B signal and the reference signal VL reverses at time t608, the output level of the comparator 107 changes from Hi to Lo. When the output level of the comparator 107 changes from Hi to Lo, the count value of the counter 108 is held in the column memory 109.

[0061] After that, after the reference signal VL transitions until it reaches a predetermined lower limit value at time t609, when the horizontal scanning circuit 112 outputs the count value corresponding to the A + B signal from the column circuit 106, the reading of the A + B signal of the pixel 102 (A + B read) is completed.

[0062] After that, predetermined signal processing such as subtracting the N signal from the A + B signal is performed. In this way, an image signal (A + B signal) is obtained from the pixel 102.

[0063] In this way, in the second read mode, the reset signal N and the image signal (A + B signal) are read, and the reading of the focus detection signal (A signal or B signal) performed in the first read mode is not performed.

[0064] Note that in the first read mode, by not transferring the A signal by the horizontal scanning circuit 112 at time t509, a reading substantially the same as the second read mode may be realized. The case where the focus detection signal is not transferred in the first read mode is called the third read mode.

[0065] FIG. 7 is a schematic diagram for explaining the column offset of the pixel 102 included in the pixel unit 101. The column offset is a variation in the signal level caused by the variation in the characteristics between the reading circuits provided in association with the pixel columns. The column offset is superimposed on the image signal as streak-like noise.

[0066] In FIG. 7, as an example, it shows the distribution of the column offset in a state where the pixel unit 101 is not exposed, based on the signals output from each of the six column circuits 106A to 106F provided in the same pixel column.

[0067] The mapping from mapping n to mapping n + 5 obtains the average value of the offsets for a plurality of pixels connected to the same vertical signal line 105 for each pixel column, and plots the average value for each pixel row. For example, mapping n is the horizontal distribution of the average value of the offsets obtained for a plurality of pixels 102 included in the 6n-th row (where n is an integer greater than or equal to 0) of the pixel section 101, which is connected to the vertical signal line 105A. The mappings from (n + 1) to (n + 5) are the distributions of the average values of the offsets similarly obtained for the pixels included in the rows from the (6n + 1)-th row to the (6n + 5)-th row, respectively.

[0068] As shown in FIG. 7, the average value of the offsets is not constant. Therefore, the deviation between the predetermined offset level indicated by the broken line and the actual offset level appears as pattern noise such as partial color unevenness, vertical streaks, and horizontal streaks in the image signal. Also, generally, the column offset varies in level for each vertical signal line due to voltage level fluctuations caused by power impedance, capacitive coupling, etc. For this reason, when a plurality of vertical signal lines 105A to 105F are provided for each pixel column as in the image sensor 100, periodic horizontal streak noise is likely to occur. Also, the column offset may vary depending on the type of signal read out from the pixel (whether it is an image signal or a focus detection signal). The difference in offset due to the type of signal read out is mainly caused by the difference in the time from when the pixel is reset until the signal is read out.

[0069] Offset correction is performed to suppress the above-described variations in the offset. FIG. 8 is a diagram schematically showing the offset correction. The image sensor 100 can apply offset correction individually to the signals output from each of the column circuits 106A to 106F by the first correction circuit 114.

[0070] The first correction circuit 114 applies offset correction that subtracts a previously stored identical value from the signals read out from the pixels 102 that are arranged in the same column and connected to the same vertical signal line 105.

[0071] The value of the signal read out from the pixel is not constant due to influences such as the brightness of the subject and random noise. However, subtracting the average value of the offset shown as a mapping in FIG. 7 corresponds to subtracting a constant value (offset correction value) corresponding to the column.

[0072] During the manufacture of the imaging device or the like, for example, based on the image signal or the focus detection signal read out with the pixel unit 101 shielded from light, the average value of the offset calculated as described with reference to FIG. 7 can be calculated as a correction value. Then, this correction value is stored, for example, in the non-volatile substrate memory 115 provided together with the first correction circuit 114. Then, the first correction circuit 114 can apply offset correction by subtracting the correction value from the signal output to the horizontal signal line 113 by the horizontal scanning circuit 112.

[0073] By offset correction, as shown in FIG. 8, the variation in the offset in the column direction is suppressed, and the offset in the column direction becomes a value near a predetermined level. Therefore, horizontal stripe noise and color unevenness can be suppressed.

[0074] Note that the second correction circuit 4051 included in the signal processing circuit 405 has the same correction function as the first correction circuit 114. The second correction circuit 4051 may acquire the correction value used in offset correction from the first correction circuit 114 or the substrate memory 115 of the imaging device 100, or may generate it based on the image signal or the focus detection signal read out from the imaging device 100 in the light-shielded state.

[0075] Note that the correction values used by the first correction circuit 114 and the second correction circuit 4051 do not necessarily have to be pre-stored correction values. For example, it may be configured to generate and use a correction value based on the signal obtained from the imaging device 100 immediately before shooting.

[0076] Also, the object of correction is not limited to the offset of the pixel value during light shielding. For example, it is also possible to store in advance the variation in the gain for each vertical signal line as a correction value and apply it as gain correction for each column.

[0077] Next, with reference to FIG. 9, the types of signals read out row by row from the imaging device 100 will be described. FIG. 9(a) schematically shows a case where an image signal (A + B signal) and a focus detection signal (A signal or B signal) are read out for all rows of the imaging device 100. Hereinafter in FIG. 9 and after, the image signal is denoted as A + B, and the focus detection signal is denoted as A. In the present embodiment, since six vertical signal lines 105A to 105F are provided for each column, the pixel rows of the pixel section 101 of the imaging device 100 are shown as the 6n-th row to the 6(n + 5)-th row (n is an integer of 0 or more).

[0078] As shown in FIG. 9(a), when reading out an image signal and a focus detection signal for all pixel rows, signals are read out in the first read mode from all pixel rows.

[0079] On the other hand, FIG. 9(b) shows a case where the image signal (A + B signal) is read out from all pixel rows, but the focus detection signal (A signal or B signal) is read out only from some pixel rows. Specifically, FIG. 9(b) shows a case where the focus detection signal (A signal) is read out from the 6n-th row, the 6(n + 1)-th row, the 6(n + 4)-th row, and the 6(n + 5)-th row, and the focus detection signal (A signal) is not read out from the 6(n + 2)-th row and the 6(n + 3)-th row.

[0080] In the case shown in FIG. 9(b), for the pixel rows where both the image signal and the focus detection signal are read out, signals are read out in the first read mode, and for the pixel rows where only the image signal is read out, signals are read out in the second or third read mode. By providing pixel rows from which the focus detection signal is not read out as in FIG. 9(b), the signal readout time for one frame can be reduced.

[0081] Hereinafter, when the focus detection signals are read out from all pixel rows, it is called full-row AF readout, and when the focus detection signals are read out from some pixel rows, it is called partial AF readout. The readout operations corresponding to the first to third readout modes can be realized by the control circuit 406 controlling the operation of TG110. Also, regarding which row to read the focus detection signal from in partial AF readout, it can be controlled by the control circuit 406 controlling the operation of TG110.

[0082] Next, how the first correction circuit 114 and the second correction circuit 4051 share offset correction will be described with reference to FIG. 10. FIG. 10 shows an example in the case of full-row AF readout.

[0083] As shown in FIG. 10, in the case of full-row AF readout, the first correction circuit 114 applies offset correction to the image signal (A + B signal) and does not apply offset correction to the focus detection signal (A signal). On the other hand, the second correction circuit 4051 applies offset correction to the focus detection signal (A signal) and does not apply offset correction to the image signal (A + B signal).

[0084] Specifically, the control circuit 406 sets the first correction circuit 114 to output the signal after applying offset correction to the image signal (A + B signal) and output the signal without performing offset correction on the focus detection signal (A signal). Also, the control circuit 406 sets the second correction circuit 4051 not to apply offset correction to the image signal (A + B signal) and to perform offset correction on the focus detection signal (A signal).

[0085] Therefore, in the operation of the first read mode shown in FIG. 5, the first correction circuit 114 writes the focus detection signal (A signal) output from time t509 to the memory 407 without applying offset correction. Then, the second correction circuit 4051 (signal processing circuit 405) reads the focus detection signal (A signal) from the memory 407 and starts applying offset correction. The second correction circuit 4051 (signal processing circuit 405) rewrites the focus detection signal (A signal) to which offset correction has been applied to the memory 407 again.

[0086] After that, the first correction circuit 114 applies offset correction to the image signal (A + B signal) output from time t513 and then writes it to the memory 407. On the other hand, the second correction circuit 4051 (signal processing circuit 405) does not apply offset correction to the image signal (A + B signal) written to the memory 407.

[0087] The signal processing circuit 405 generates a focus detection signal (B signal) from the offset-corrected image signal (A + B signal) and focus detection signal (A signal) written to the memory 407.

[0088] Thus, in this embodiment, when reading an image signal and a focus detection signal from the imaging device, the correction processing applied to each signal is applied by the correction circuit in the imaging device for one signal and by the correction circuit outside the imaging device for the other signal. Thereby, when using different correction values for each type of signal to be read, the amount of correction values held in the imaging device can be reduced. In particular, when a plurality of vertical signal lines are provided for each pixel column and correction values for each vertical signal line are used, the effect is significant.

[0089] Also, when reading a plurality of types of pixel signals corresponding to one exposure process, such as an image signal and a focus detection signal, the upper limit of the continuous shooting speed is limited by the time required for reading. In this embodiment, since the correction processing is shared by two correction circuits according to the type of signal, the time required for reading can be shortened, contributing to an improvement in the continuous shooting speed.

[0090] In the above description, offset correction is applied to the image signal and the focus detection signal. However, offset correction may also be applied to the reset signal N. In this case, the offset correction of the reset signal N may be applied by either the first correction circuit 114 or the second correction circuit 4051. The control circuit 406 is set to apply offset correction to the reset signal N to the correction circuit that stores the correction value for the reset signal N.

[0091] ●(Second Embodiment) Next, a second embodiment of the present invention will be described. Since this embodiment may be the same as the first embodiment except for the operation of the second correction circuit 4051, the operation of the second correction circuit 4051 in this embodiment will be mainly described below.

[0092] FIG. 11 is a diagram showing the readout and correction operations in the second embodiment in the same manner as FIG. 10. Here, the image signal (A + B signal) and the focus detection signal (A signal) are read out over the entire row, as in the first embodiment.

[0093] The offset correction process applied by the first correction circuit 114 to the image signal (A + B signal) is the same as that in the first embodiment. On the other hand, the second correction circuit 4051 that corrects the focus detection signal generates a focus detection signal to which correction is applied from the focus detection signals read from the pixels of 6n rows to (6n + 5) rows respectively connected to the vertical signal lines 105A to 105F.

[0094] The second correction circuit 4051 synthesizes pixel signals for a plurality of rows into one row. For example, the second correction circuit 4051 synthesizes the focus detection signals read from the pixels of 6n rows, (6n + 1) rows, and (6n + 2) rows into a focus detection signal for one row. Also, the second correction circuit 4051 synthesizes the focus detection signals read from the pixels of (6n + 3) rows, (6n + 4) rows, and (6n + 5) rows into a focus detection signal for one row. The synthesis may be, for example, to add and average the focus detection signals read from the same column of each row.

[0095] Note that regarding whether the second correction circuit 4051 performs synthesis and which row of pixel signals is to be synthesized, the control circuit 406 sets these for the second correction circuit 4051. Also, when the second correction circuit 4051 supports a plurality of synthesis methods, the control circuit 406 also sets the synthesis method to be used for the second correction circuit 4051.

[0096] Then, the second correction circuit 4051 applies offset correction to the generated focus detection signals for two rows. Therefore, the correction value used by the second correction circuit 4051 is the correction value corresponding to the focus detection signal after synthesis. The correction value can be generated, for example, by synthesizing the correction value used when synthesis is not performed (the correction value used by the second correction circuit 4051 in the first embodiment) in the same manner as the focus detection signal. Also, the correction value corresponding to the focus detection signal after synthesis may be acquired and held in advance.

[0097] In this way, by reducing the focus detection signals to which correction is applied, the amount of correction values used by the second correction circuit 4051 can be reduced. Although the accuracy of correction decreases due to synthesis, since the correction accuracy required for the focus detection signal is lower than that for the image signal, the effect of reducing the data amount of the correction value is greater. When a plurality of vertical signal lines are provided for each pixel column, if synthesis is not performed, a correction value is required for each vertical signal line, so the effect is particularly significant.

[0098] Note that in the configuration in FIG. 11 where six vertical signal lines are provided for each pixel column, a common correction value is used for the signals read from three adjacent vertical signal lines. However, the number and positional relationship of the vertical signal lines using the common correction value are merely examples.

[0099] For example, lines from 6n to (6n + 5) may be combined into one line, and a common correction value may be used for all focus detection signals. Further, when a color filter in a primary color Bayer array is provided in the pixel unit 101 as shown in FIG. 1, the focus detection signals read from a plurality of pixels provided with the same color color filter may be combined. Specifically, the signals read from the pixels of lines 6n, (6n + 2), and (6n + 4) can be combined into one line, and the signals read from the pixels of lines (6n + 1), (6n + 3), and (6n + 5) can be combined into one line.

[0100] Alternatively, instead of performing full-line AF reading, partial AF reading (FIG. 9(b)) may be performed. In this case, the time required for reading the A signal and the time required for synthesis in the second correction circuit 4051 can be reduced. The second correction circuit 4051 applies offset correction using a correction value according to the reading method and synthesis method of the focus detection signal.

[0101] In the present embodiment, in addition to the effects of the first embodiment, it is possible to reduce the storage capacity and processing time required for offset correction of the focus detection signal while maintaining the correction accuracy of the image signal.

[0102] ●(Third Embodiment) Next, a third embodiment of the present invention will be described. Since this embodiment may be the same as the first embodiment except for the operation of the second correction circuit 4051, the operation of the second correction circuit 4051 in this embodiment will be mainly described below.

[0103] FIG. 12 is a diagram showing the reading and correction operations in the third embodiment in the same manner as FIG. 10. The reading of the image signal (A + B signal) and the offset correction for the image signal (A + B signal) are the same as in the first embodiment.

[0104] On the other hand, for the focus detection signal (A signal), there are cases of full-line AF reading (Fig. 12(a)) and partial AF reading (Fig. 12(b)). The method of reading the focus detection signal is, for example, pre-associated with the shooting mode of the imaging device 400. Therefore, the control circuit 406 determines whether to perform full-line AF reading or partial AF reading for the method of reading the focus detection signal from the imaging element 100 according to the set shooting mode, and sets the reading method to the TG110. Also, when the second correction circuit 4051 synthesizes pixel signals, the control circuit 406 sets the rows to be synthesized and the synthesis method to the second correction circuit 4051.

[0105] For example, for a shooting mode in which a scene where AF accuracy is likely to decrease, such as a low-illumination scene, full-line AF reading is performed, and for a shooting mode for shooting other scenes, partial AF reading is performed. Alternatively, for a shooting mode that requires a high shooting frame rate, partial AF reading may be performed, and full-line AF reading may be performed in other shooting modes. Note that these are merely examples of the association between the reading method and the shooting mode, and other associations are also possible.

[0106] Also, not limited to the shooting mode, the reading method may be determined according to other conditions. For example, when it is necessary to reduce power consumption or when low-power operation is set, partial AF reading can be performed.

[0107] In the example shown in Fig. 12(b), when performing partial AF reading, the focus detection signal (A signal) is read from pixels other than the (6n + 2)-th row and the (6n + 5)-th row, but the number and arrangement of the rows from which the signal is not read are not limited to this.

[0108] In the case of full-line AF reading, the second correction circuit 4051, similar to the second embodiment, synthesizes three consecutive rows of the focus detection signal (A signal) for six rows into one row each to generate two rows of the focus detection signal.

[0109] On the other hand, in the case of partial AF reading, the second correction circuit 4051 synthesizes the signal read from the pixels in the 6n-th row and the signal read from the pixels in the (6n + 1)-th row into one row. The second correction circuit 405 1 further generates focus detection signals for two rows by synthesizing the signal read from the pixels in the (6n + 3)-th row and the signal read from the pixels in the (6n + 4)-th row into one row. The synthesis may be, for example, adding and averaging the pixel values for each column. By partial AF reading, the amount of data read for the focus detection signal can be reduced, and an improvement in the shooting frame rate and a reduction in power consumption can be achieved.

[0110] According to this embodiment, similarly to the second embodiment, the amount of data of the correction value used for offset correction applied to the focus detection signal can be reduced. Specifically, for each of the cases of full-frame AF reading and partial AF reading, it is sufficient to store correction values for two rows.

[0111] Note that in this embodiment, the number of rows of the focus detection signal after synthesis is the same for both full-frame AF reading and partial AF reading. Further, the rows included in the rows to be synthesized in the case of full-frame AF reading are synthesized in the case of partial AF reading. Therefore, if it is necessary to further reduce the amount of data of the correction value to be prepared, the correction value used in the case of full-frame AF reading may also be used in the case of partial AF reading. Thereby, the amount of data of the correction value can be further reduced.

[0112] The correction value can be generated, for example, by synthesizing the correction value when no synthesis is performed (the correction value used by the second correction circuit 4051 in the first embodiment) in the same manner as the row data synthesis method in the second correction circuit 4051.

[0113] The second correction circuit 4051 applies offset correction by subtracting the corresponding correction value from each of the focus detection signals for two rows after synthesis.

[0114] In the present embodiment, in addition to the effects of the second embodiment, by using partial AF reading in combination, the time required for reading and correcting the focus detection signal can be shortened. Further, if the correction value in the case of full-line AF reading is also used in the case of partial AF reading, the storage capacity for the correction value can be suppressed to the same level as in the second embodiment.

[0115] ●(Fourth Embodiment) Next, a fourth embodiment of the present invention will be described. The fourth embodiment relates to a correction operation according to the type of signal recorded by the imaging device 400. The imaging device 400 can change the type of signal to be recorded according to user settings. For example, it is assumed that the imaging device 400 can set whether to record a focus detection signal in addition to the image signal.

[0116] FIG. 13(a) schematically shows a reading operation and a correction operation when recording a focus detection signal. In this case, the image signal (A + B signal) and the focus detection signal (A signal) are read for the entire line. In this way, by recording the focus detection signal for all pixels, it becomes possible to perform adjustment of the focusing distance and ghost removal after shooting. Since the A signal can also be used as an image signal in these processes, it is necessary to perform offset correction with the same accuracy as the A + B signal.

[0117] Therefore, when recording the focus detection signal (A signal), as in the first embodiment, for the A + B signal, offset correction is applied by the first correction circuit 114, and for the A signal, offset correction is applied by the second correction circuit 4051.

[0118] When not recording the focus detection signal (A signal), it is a case where the focus detection signal is used only for the AF operation of the imaging device 400, similar to the focus detection signal (A signal) in the first to third embodiments. In this case, as shown in FIG. 13(b), partial AF reading is performed, and offset correction is applied to the focus detection signal (A signal) by the second correction circuit 4051. When not recording the focus detection signal (A signal), offset correction for the focus detection signal (A signal) may be applied in the same manner as in the second embodiment (FIG. 11(a)) or the third embodiment (FIG. 12(b)).

[0119] The control circuit 406 sets the TG110, the first correction circuit 114, and the second correction circuit 4051 so as to perform the reading and offset correction shown in FIGS. 13(a) and 13(b) according to the setting of the signal to be recorded.

[0120] According to the present embodiment, appropriate offset correction can be applied to the focus detection signal when recording the focus detection signal in addition to the image signal and when not recording the focus detection signal. Further, by sharing the offset correction process between the two correction circuits, the data amount of the correction values held in the imaging element can be reduced, and the time required for offset correction can be shortened.

[0121] When recording the focus detection signal (A signal), offset correction may be applied to not only the image signal (A + B signal) but also the focus detection signal (A signal) by the first correction circuit 114. When it is necessary to reduce the data amount of the correction values held in the imaging element, correction values may be input from outside the imaging element.

[0122] ●(Fifth Embodiment) Next, a fifth embodiment of the present invention will be described. This embodiment relates to offset correction when a plurality of vertical signal lines provided for each column are divided into two groups and the readout periods (readout rates) of the vertical signal lines in the first group and the vertical signal lines in the second group are different.

[0123] FIG. 14 is a diagram showing the reading and correction operations in the fifth embodiment in the same manner as FIG. 10. In this embodiment, the six vertical signal lines 105A to 105F provided for each column are divided into two groups, namely, vertical signal lines 105A to 105D and 105E to 105F, and different readout periods are set for each group.

[0124] Specifically, from the pixels of the vertical signal lines 105A to 105D (6n-th row to (6n + 3)-th row), an image signal (A + B signal) and a focus detection signal (A signal) are read out in the first period (FIG. 14(a)). The read pixel signals are used for generating image data and focus detection.

[0125] Also, from the pixels of the vertical signal lines 105E to 105F ((6n + 4)-th row to (6n + 5)-th row), an image signal (A + B signal) is read out in a second period shorter than the first period, and the focus detection signal is not read out (FIG. 14(b)). The read pixel signals are generally called multi-stream signals and are used for detecting flicker of ambient light and exposure control.

[0126] Note that the method of dividing the vertical signal lines 105 into groups and the types of signals to be read are merely examples. For example, only the image signal may be read out in the first period, and the focus detection signal may be read out in the second period and offset correction may be applied by the second correction circuit 4051. However, generally, the number of vertical signal lines read out in the second period is made smaller than the number of vertical signal lines read out in the first period in order to read out in a shorter period than the first period.

[0127] Such control of the readout period according to the vertical signal lines 105 can be realized by the control circuit 406 controlling the operation of the TG 110.

[0128] In this embodiment, offset correction is applied to pixel signals read out in a first period by the first correction circuit 114, and to pixel signals read out in a second period by the second correction circuit 4051. Note that the relationship between the correction circuit and the pixel signal to be corrected may be reversed. Basically, the second correction circuit 4051 is used when pixel signals are combined and offset correction is applied, but when pixel signals are not combined, either the first correction circuit 114 or the second correction circuit 4051 may apply offset correction.

[0129] 14, a total of eight types of correction values ​​are stored in the substrate memory 115 for each of the vertical signal lines 105A to 105D and for each type of pixel signal to be read out. In addition, the memory 407 stores correction values ​​for a total of two types of image signals for each of the vertical signal lines 105E to 105F. As in the previous embodiment, the correction values ​​stored in the substrate memory 115 may be stored in advance or may be obtained from the memory 407.

[0130] In addition, in the second correction circuit 4051, the pixel signals (A+B signals) read from the pixels in the (6n+4)th row and the (6n+5)th row may be combined into one row before applying offset correction. In this case, the amount of correction value used in the second correction circuit 4051 can be reduced by half.

[0131] The control circuit 406 controls the horizontal signal lines 113A to 113D corresponding to the vertical signal lines 105A to 105D. to D The first correction circuit 114 is set to apply offset correction only to pixel signals that are read out. The control circuit 406 also sets the second correction circuit 4051 to apply offset correction only to pixel signals that are read out to the horizontal signal lines 113E to 113F that correspond to the vertical signal lines 105E to 105F. The control circuit 406 can also set the second correction circuit 4051 as to whether or not to perform synthesis before correction.

[0132] According to this embodiment, when performing reading with different periods according to pixel rows, for the pixel signals read in the first period, offset correction is applied by the first correction circuit, and for the pixel signals read in the second period, offset correction is applied by the second correction circuit. Therefore, while reducing the correction values held in the imaging device, the load of offset correction can be dispersed.

[0133] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0134] The present invention is not limited to the content of the above-described embodiments, and various changes and modifications are possible without departing from the spirit and scope of the invention. Therefore, claims are attached to disclose the scope of the invention.

Explanation of reference numerals

[0135] 100... Imaging device, 114... First correction circuit, 304... Substrate memory, 400... Imaging apparatus, 401... Photographing lens, 402... Lens drive circuit, 403... Shutter, 404... Mechanical drive circuit, 405... Signal processing circuit, 406... Control circuit, 407... Memory

Claims

1. An imaging device having a pixel array in which a plurality of pixels are arranged in a matrix, and a plurality of column signal lines provided for each column of the pixels, capable of reading an image signal and a focus detection signal from the pixel array, a first correction circuit provided in the imaging device and applying a predetermined correction process to the image signal, a second correction circuit provided outside the imaging device, synthesizing the focus detection signals for a plurality of rows, and applying the predetermined correction process to the synthesized focus detection signal, characterized by comprising the above.

2. The correction value used by the first correction circuit for the predetermined correction process is held in a memory of the imaging device, and the correction value used by the second correction circuit for the predetermined correction process is held in a memory provided outside the imaging device. The imaging device according to Claim 1.

3. The correction value held in the memory of the imaging device is acquired from a memory provided outside the imaging device. The imaging device according to Claim 2.

4. The predetermined correction process includes an offset correction process for correcting variations in signal levels caused by circuit characteristics for each column of the pixels. The imaging device according to any one of Claims 1 to 3.

5. The second correction circuit synthesizes the focus detection signals for the plurality of rows such that the number of rows after synthesis of the focus detection signals is equal when the focus detection signal is read from pixels of all rows and when the focus detection signal is read from pixels of some rows. The imaging device according to Claim 1.

6. The second correction circuit applies the predetermined correction process to the synthesized focus detection signal using the same correction value when the focus detection signal is read from pixels of all rows and when the focus detection signal is read from pixels of some rows. The imaging device according to Claim 5.

7. Whether the focus detection signal is read from pixels of all rows or from pixels of some rows is changed according to the setting of the imaging device. The imaging device according to Claim 5 or Claim 6.

8. The imaging device according to Claim 7, characterized in that the setting is a shooting mode.

9. When the image signal and the focus detection signal are recorded, the image signal and the focus detection signal are read out from the pixels of all rows. When the image signal is recorded and the focus detection signal is not recorded, the image signal is read out from the pixels of all rows, and the focus detection signal is read out from the pixels of some rows. The imaging device according to any one of claims 1 to 7, characterized in that.

10. When the image signal is read out from the first group of the plurality of column signal lines in the first period, and the focus detection signal is read out from the second group of the plurality of column signal lines in a second period shorter than the first period. The first correction circuit applies the predetermined correction process to one of the image signals with different readout periods, and the second correction circuit applies the predetermined correction process to the other. The imaging device according to any one of claims 1 to 9, characterized in that.

11. Furthermore, when the focus detection signal is read out in the first period, the predetermined correction process is applied to the focus detection signal by a correction circuit that applies the predetermined correction process to the image signal read out in the first period. When the focus detection signal is read out in the second period, the predetermined correction process is applied to the focus detection signal by a correction circuit that applies the predetermined correction process to the image signal read out in the second period. The imaging device according to claim 10, characterized in that.

12. The imaging device according to claim 10 or 11, characterized in that the number of column signal lines belonging to the first group is larger than the number of column signal lines belonging to the second group.

13. The pixel has a plurality of photoelectric conversion units, the image signal is a pixel signal read out from the plurality of photoelectric conversion units, and the focus detection signal is a pixel signal read out from a part of the plurality of photoelectric conversion units. The imaging device according to any one of claims 1 to 12, characterized in that.

Images