Image sensor and image pickup apparatus

Abstract

The invention relates to an image sensor and an image pickup apparatus. The image sensor includes: a pixel portion in which unit pixels are arranged, each of the unit pixels including one microlens and a plurality of photoelectric conversion units; a readout section for reading out a signal from the pixel section, the readout section being capable of performing a first readout operation for reading out a mixed signal and a second readout operation for reading out a non-mixed signal; a classification means for classifying signals obtained by the first read-out operation and the second read-out operation into a captured image signal and a focus detection signal; a first output unit that outputs the captured image signal classified by the classification unit; and a second output unit that outputs the focus detection signal classified by the classification unit.

Description

Technical Field

[0001] The present invention relates to an image sensor and an imaging device. Background Art

[0002] In recent years, for example, in imaging devices, an increase in the number of pixels, an increase in the readout speed, and an increase in the frame rate have been advancing. In addition, using the signals obtained by the image sensor, not only generation of images such as still images and moving images is performed, but also control such as focus adjustment is performed.

[0003] For example, Japanese Patent Application Laid-Open No. 2001-124984 discloses a technique that enables focus detection based on a pupil division method using signals obtained from an image sensor. Specifically, since each pixel in the image sensor includes a microlens and two photodiodes, each photodiode receives light passing through different pupil regions of the photographic lens. Focus detection can be performed by comparing the output signals from these two photodiodes. In addition, a captured image can be generated by adding the output signals from the two photodiodes.

[0004] However, in the case where each unit pixel includes a plurality of photodiodes as in Japanese Patent Application Laid-Open No. 2001-124984, the amount of data read from the pixel increases, and the time period required to read the signal becomes longer. In addition to the increase in the number of pixels, the increase in the readout speed, and the increase in the frame rate, due to the generation of data for focus detection, the amount of data transmitted per unit time from the image sensor to the signal processing IC increases. The increase in the amount of data transmitted per unit time results in a decrease in the frame rate.

[0005] If an attempt is made to widen the data transmission band and improve the processing ability of the signal processing IC in response to the increase in the data transmission amount, the chip costs of the image sensor and the signal processing IC will increase. Summary of the Invention

[0006] The present invention has been made in view of the above problems, and provides an imaging device that can suppress a decrease in the frame rate while suppressing an increase in device cost even when the amount of data read from the image sensor increases.

[0007] According to a first aspect of the present invention, there is provided an image sensor, comprising: a pixel section in which unit pixels are arranged in a matrix, each of the unit pixels including a microlens and a plurality of photoelectric conversion units; a readout component for reading signals from the pixel section, the readout component being capable of performing a first readout operation of reading a mixed signal obtained by mixing the signals of the plurality of photoelectric conversion units of the unit pixel, and a second readout operation of reading an unmixed signal that is not a mixture of the signals of the plurality of photoelectric conversion units of the unit pixel; a classification component for classifying the signals obtained by the first readout operation and the second readout operation into a photographed image signal and a focus detection signal; a first output unit that outputs the photographed image signal classified by the classification component; and a second output unit that outputs the focus detection signal classified by the classification component.

[0008] According to a second aspect of the present invention, there is provided an imaging device, comprising: an image sensor, the image sensor including: a pixel section in which unit pixels are arranged in a matrix, each of the unit pixels including a microlens and a plurality of photoelectric conversion units; a readout component for reading signals from the pixel section, the readout component being capable of performing a first readout operation of reading a mixed signal obtained by mixing the signals of the plurality of photoelectric conversion units of the unit pixel, and a second readout operation of reading an unmixed signal that is not a mixture of the signals of the plurality of photoelectric conversion units of the unit pixel; a classification component for classifying the signals obtained by the first readout operation and the second readout operation into a photographed image signal and a focus detection signal; a first output unit that outputs the photographed image signal classified by the classification component; and a second output unit that outputs the focus detection signal classified by the classification component; a first processing component for processing the photographed image signal output from the first output unit; and a second processing component for processing the focus detection signal output from the second output unit.

[0009] Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the accompanying drawings. Description of the Drawings

[0010] Figure 1 is a diagram showing the structure of an imaging device according to a first embodiment of the present invention.

[0011] Figure 2 is a diagram schematically showing the configuration of pixels in an image sensor.

[0012] Figure 3 is a diagram schematically showing the relationship between the light beam emitted from the exit pupil of a photographic lens and the pixels.

[0013] Figure 4 It is a diagram showing the circuit structure of a unit pixel of an image sensor.

[0014] Figure 5 It is a diagram showing a focus detection area provided for a pixel array in an image sensor.

[0015] Figure 6A and Figure 6B It is a timing diagram showing the operation of reading out a row of unit pixels in an image sensor.

[0016] Figure 7A and Figure 7B It is a diagram illustrating the relationship between the focus state and the correlation between image signals. Detailed Description of the Invention

[0017] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments are not intended to limit the scope of the claimed invention. Multiple features are described in the embodiments, but the invention is not limited to an invention that requires all such features, and multiple such features can be appropriately combined. In addition, in the drawings, the same reference numerals are given to the same or similar structures, and their redundant descriptions are omitted.

[0018] (First Embodiment)

[0019] Figure 1 It is a diagram showing the structure of a photographing apparatus 100 according to the first embodiment of the present invention.

[0020] The photographing lens 101 is constituted by, for example, an interchangeable lens unit that can be attached to and detached from the photographing apparatus 100, guides light from a subject to the image sensor unit 102, and forms a subject image on the pixels of an image sensor 102a (see Figure 2 ) included in the image sensor unit 102.

[0021] The image sensor unit 102 includes an image sensor 102a such as a CMOS image sensor for outputting an image signal according to incident light, a printed circuit board on which the image sensor 102a is mounted, and a power supply for driving the image sensor 102a. More specifically, the image sensor unit 102 includes a photoelectric conversion unit 103, a readout unit 104, and a data classification unit 105 for classifying the read data into photographed image data and focus detection data. The image sensor unit 102 further includes a compression unit 106 for encoding the photographed image data, an output interface (I / F) unit (first output unit) 107 for transmitting the photographed image data, a resizing unit 108 for resizing the focus detection data, and a compression unit 109 for encoding the focus detection data. The image sensor unit 102 further includes a memory 110 and an output interface (I / F) unit (second output unit) 111 for transmitting the focus detection data.

[0022] In addition, the imaging device 100 includes, for example, an image processing unit 201 and an image processing unit 301 constituted by different image processing ICs. Further, the imaging device 100 includes a control unit 312 for controlling its overall operation. The control unit 312 controls the entire imaging device 100 by expanding the control program stored in the ROM 314 into the RAM 316 and executing the control program.

[0023] The photoelectric conversion unit 103 includes a photodiode or the like, receives incident light, and converts the incident light into an electric signal. Further, the readout unit 104 converts the analog signal output from the photoelectric conversion unit 103 into a digital signal. Details of these components are now described using Figure 2 and Figure 3 Description of these components is now provided.

[0024] Figure 2 is a diagram schematically showing the configuration of pixels in the image sensor 102a.

[0025] As Figure 2 shown, the pixel portion of the image sensor 102a includes unit pixels 400 arranged in a matrix, and a red (R) color filter, a green (G) color filter, and a blue (B) color filter are arranged in a Bayer format for the unit pixels 400.

[0026] In addition, inside each unit pixel 400, sub-pixels a and sub-pixels b are placed, and a photodiode 401a and a photodiode 401b are respectively placed in the sub-pixels a and sub-pixels b. Each signal (non-mixed signal) output from the sub-pixels a and sub-pixels b is used for focus detection, while the a-and-b mixed signal, which is a signal obtained by mixing the signals output from the sub-pixels a and sub-pixels b, is used for image generation.

[0027] Figure 3 is a diagram schematically showing the relationship between the light beam emitted from the exit pupil of the photographic lens 101 and the unit pixel 400. In Figure 3 the same reference numerals are used to denote components similar to those in Figure 2 .

[0028] As Figure 3 shown in, in the image sensor 102a, a color filter 501 and a microlens 502 are formed on each unit pixel 400.

[0029] The light passing through the exit pupil 503 of the photographic lens is incident on the unit pixel 400 with the optical axis 504 as the center of incidence. The light beam passing through the pupil region 505, which is a partial region of the exit pupil 503 of the photographic lens 101, passes through the microlens 502 and is received by the sub-pixel a. On the other hand, the light beam passing through the pupil region 506, which is another partial region of the exit pupil 503, passes through the microlens 502 and is received by the sub-pixel b.

[0030] Therefore, the sub-pixel a and the sub-pixel b receive light from different pupil regions 505 and 506 of the exit pupil 503 of the photographic lens 101, respectively. Therefore, focus detection based on the phase difference method can be performed by comparing the output signals from the sub-pixel a and the sub-pixel b. The signal of the sub-pixel a is obtained from a plurality of unit pixels arranged in the row direction and the column direction, and the subject image formed by the group of these output signals is regarded as the image signal A; the signal of the sub-pixel b is obtained from a plurality of unit pixels arranged in the row direction and the column direction, and the subject image formed by the group of these output signals is regarded as the image signal B. The image displacement amount (pupil division phase difference) is detected by performing a correlation calculation on the image signal A and the image signal B. In addition, the focus position corresponding to an arbitrary subject position within the screen can be calculated by multiplying the image displacement amount by the conversion coefficient determined according to the focus position of the photographic lens 101 and the optical system. The focus position of the photographic lens 101 can be controlled based on the information of the focus position calculated here to perform imaging plane phase detection autofocus (AF). Further, the signal obtained by adding the image signal A and the image signal B is regarded as the image signal AB, and this image signal AB can be used for normal captured images.

[0031] Figure 4 is a diagram showing the circuit structure of the unit pixel of the image sensor.

[0032] In Figure 4In this case, the light incident on the photodiode (photoelectric conversion unit) 401a of the above sub-pixel a is subjected to photoelectric conversion by the photodiode 401a, the light incident on the photodiode (photoelectric conversion unit) 401b of the above sub-pixel b is subjected to photoelectric conversion by the photodiode 401b, and charges corresponding to the exposure amount are accumulated in the photodiode 401a and the photodiode 401b.

[0033] By setting the control signal Txa applied to the gate of the transfer gate 402a and the control signal Txb applied to the gate of the transfer gate 402b to high levels respectively, the charges accumulated in the photodiode 401a and the photodiode 401b are transferred to the floating diffusion (FD) unit 403.

[0034] The FD unit 403 is connected to the gate of the floating diffusion amplifier 404 (hereinafter referred to as the FD amplifier), and the FD amplifier 404 converts the amount of charges transferred from the photodiode 401a and the photodiode 401b into a voltage value.

[0035] The reset switch 405 is a reset switch for resetting the FD unit 403 and the photodiode 401a and the photodiode 401b; setting the control signal Res applied to the gate of this reset switch 405 to a high level resets the FD unit 403. In addition, in order to reset the charges in the photodiode 401a and the photodiode 401b, the control signal Res and the control signal Txa and the control signal Txb are set to high levels simultaneously. Therefore, the transfer gate 402a and the transfer gate 402b and the reset switch 405 are all placed in the conducting state, and the photodiode 401a and the photodiode 401b are reset via the FD unit 403.

[0036] Setting the control signal Sel applied to the gate of the pixel selection switch 406 to a high level causes the pixel signal converted into a voltage value by the FD amplifier 404 to be output to the analog-to-digital converter (ADC) via the column output line 402.

[0037] The FD amplifier 404 operates as a source follower amplifier together with an unillustrated constant current source connected to the column output line 402.

[0038] The ADC block includes a comparator 407, an up / down counter (U / D CNT) 410, and a digital-to-analog converter (DAC) 409.

[0039] The above-described column output line 402 is connected to one of a pair of input terminals of the comparator 407, and the DAC 409 is connected to the other input terminal of the pair of input terminals. The DAC 409 outputs a ramp signal whose level changes like a ramp based on a reference signal input from the timing control circuit 408. Then, the comparator 407 compares the level of the ramp signal input from the DAC 409 with the level of the image signal input from the column output line 402. The timing control circuit 408 outputs the reference signal to the DAC 409 based on a command from the control unit 312 (see Figure 1 ).

[0040] For example, the comparator 407 outputs a high-level comparison signal when the level of the image signal is lower than the level of the ramp signal, and outputs a low-level comparison signal when the level of the image signal is higher than the level of the ramp signal.

[0041] The up / down counter 410 is connected to the comparator 407, and a clock for timing is input from the timing control circuit 408 to the up / down counter 410. For example, the up / down counter 410 counts the clock during a period when the comparison signal of the comparator 407 is at a high level or during a period when the comparison signal is at a low level. This counting process converts the output signal of each unit pixel 400 into a digital value. The output signal converted into a digital signal is stored in the line memory 411.

[0042] Next, the operation of reading out the image signal A and the operation of reading out the image signal AB which is a mixed signal of the image signal A and the image signal B will be described. The structure of the present embodiment allows selection of reading out only the image signal AB or reading out the image signal A and the image signal AB based on each row.

[0043] Figure 5 The pixel region (Region_i) in the pixel region of the above-described image sensor 102a that performs both focus detection processing and image generation, and the pixel region (Region_c) that does not perform focus detection processing but only performs image generation are shown.

[0044] The image signal A and the image signal AB are read out from the rows of unit pixels included in the region Region_i indicated by the shaded portion. The image signal AB is read out only from the rows of unit pixels included in the region Region_c which is a region other than the region Region_i; these image signals are not used in focus detection calculation but only in image generation. The entire region may also be set as the pixel region (Region_i) that performs both focus detection processing and image generation.

[0045] Next, Figure 6A andFigure 6B Describe the operation of reading the signals of the image sensor 102a. Assume that charges have been accumulated in the photodiodes 401a and 401b.

[0046] Figure 6A is for Figure 5 The timing diagram of the read operation for each row in the region Region_c.

[0047] The control signal Sel is set to high level, and the pixel selection switch 406 in the unit pixel is turned on. Then, the control signal Res is set to low level, and the reset switch 405 is turned off, thus completing the reset of the FD unit 403.

[0048] Next, based on the ramp signal of the DAC 409, before turning on the transfer gates 402a and 402b, the reference signal N as the reference level is subjected to AD conversion, and the reference signal N is stored in the row memory 411. The operation of reading the reference signal N will be referred to as N readout. In addition, the input unit of the ADC includes a sampling and holding circuit (not illustrated), and can hold the signal level at each timing of N signal sampling and S signal sampling in Figure 6A The N signal sampling and S signal sampling in.

[0049] Next, the control signals Txa and Txb are set to high level, so as to turn on the transfer gates 402a and 402b. As a result of this operation, the signal obtained by mixing the charge signal accumulated in the photodiode 401a of the sub-pixel a and the charge signal accumulated in the photodiode 401b of the sub-pixel b is output to the column output line 402 via the FD amplifier 404 and the pixel selection switch 406.

[0050] The signal of the column output line 402 is input to the comparator 407, and the signal of the column output line 402 is subjected to AD conversion based on the ramp signal of the DAC 409; the difference from the previously recorded reference signal N is stored in the row memory 411, and the AB mixed signal (image signal AB) corresponding to one row is output to the data classification unit 105. Note that the operation of reading the charges accumulated in the photodiode will be referred to as S readout.

[0051] The above is the operation of reading each row of the unit pixels in the region Region_c. As a result, the image signal AB is read out.

[0052] Next, Figure 6B will be used to Figure 6B Describe the operation of reading each row in the region Region_i.

[0053] is the timing diagram of the operation until the image signal A and the image signal AB in one row of the region Region_i are read out.The operations until the reference signal N is stored in the row memory 411 are similar to those Figure 6A illustrated in

[0054] When the storage of the reference signal N is completed, the control signal Txa is set to high level, so that the transfer gate 402a is turned on. This operation causes the signal accumulated in the photodiode 401a of the sub-pixel a to be output to the column output line 402 via the FD amplifier 404 and the pixel selection switch 406.

[0055] The image signal A output to the column output line 402 is input to the comparator 407, and an AD conversion is performed on the image signal A based on the ramp signal of the DAC 409; the difference from the earlier recorded reference signal N is stored in the row memory 411. Then, the signal (image signal A) of the sub-pixel a corresponding to one row is output to the data classification unit 105.

[0056] The reading of the image signal A is completed while the control signal res and the control signal sel are maintained at low level and high level respectively. In this way, the image signal A in the FD unit 403 is held without reset.

[0057] After the reading of the image signal A is completed, the operation of reading the image signal AB is then performed. The control signal Txa and the control signal Txb are set to high level, so that the transfer gate 402a and the transfer gate 402b are turned on. This operation will cause the signal accumulated in the photodiode 402b of the sub-pixel b to be mixed with the signal of the sub-pixel a held in the FD unit 403, and the mixed signal will be output to the column output line 402 via the FD amplifier 404 and the pixel selection switch 406. The subsequent operations are the same as those Figure 6A described for the region Region_c.

[0058] This ends the operations of reading each row in the region Region_i. As a result, the image signal A and the image signal AB are sequentially read. Note that the image signal B is obtained by subtracting the image signal A from the image signal AB.

[0059] Now return to Figure 1Description. The data classification unit 105 classifies the read data into captured image data and focus detection data. Specifically, the data classification unit 105 sends only the read image signal AB as the captured image data to the compression unit 106. In addition, the data classification unit 105 sends the image signal A and the image signal AB read from the pixel region (Region_i) where the focus detection process is performed as the focus detection signal to the resizing unit 108. In the case where focus detection is not required, the transmission of the focus detection data from the data classification unit 105 to the resizing unit 108 can also be omitted. This allows, for example, the focus detection data to be transmitted only once for every plurality of frames during continuous shooting.

[0060] The compression unit 106 performs compression encoding on the captured image data transmitted from the data classification unit 105 using a predetermined method. The output I / F unit 107 transmits the captured image data encoded by the compression unit 106 to the input interface (I / F) unit 202 of the image processing unit 201 via the transmission path 112.

[0061] The output I / F unit 107, the transmission path 112, and the input I / F unit 202 need to have an interface (I / F) with a relatively high speed (high communication speed) for transmitting the captured image data. In response to an increase in the number of pixels required by the current imaging device, an increase in the read speed, and an increase in the frame rate, etc., an increase in the transmission speed is required. To increase the signal transmission speed, it is sufficient to adopt a method of simply increasing the frequency and an I / F and a transmission path that support, for example, pulse amplitude modulation (PAM) 4 transmission that can increase the number of bits that can be transmitted at a 1 unit interval (UI). Increasing the signal transmission frequency and supporting PAM4 transmission result in an increase in the chip cost of the image sensor, the transmission path, and the signal processing unit. If an attempt is made to use the output I / F unit 107 that supports the above high-speed transmission to transmit both the captured image data and the focus detection data, the chip cost will increase; therefore, the output I / F unit 107 is used to transmit only the captured image data.

[0062] The resizing unit 108 resizes the focus detection data transmitted from the data classification unit 105 using a predetermined method. It is sufficient to determine the resizing method based on the data transmission speed of the output interface (I / F) unit 111 and the accuracy required for focus detection, etc. The compression unit 109 performs compression encoding on the resized focus detection data transmitted from the resizing unit 108 using a predetermined method. The memory 110 can temporarily store the compression-encoded focus detection data corresponding to a plurality of frames; this enables adjustment of the speed for the subsequent output I / F unit 111 (adjustment of the processing speed).

[0063] The output I / F unit 111 sends the focus detection data stored in the memory 110 to the input interface (I / F) unit 306 of the image processing unit 301 via the transmission path 113.

[0064] Since the output I / F unit 111, the transmission path 113, and the input I / F unit 306 send the focus detection data that has been reduced by reading with binning interval elimination or by the resizing unit, the output I / F unit 111, the transmission path 113, and the input I / F unit 306 can use an interface (I / F) with a relatively low speed (relatively low communication speed). For this reason, unlike the output I / F unit 107, the transmission path 112, and the input I / F unit 202, the output I / F unit 111, the transmission path 113, and the input I / F unit 306 can adopt a relatively inexpensive circuit.

[0065] The image processing unit 201 includes an input I / F unit 202, a decompression unit 203, a data correction unit 204, an image processing circuit group 205, and a data transmission unit 206.

[0066] The image processing unit 301 is an image processing unit separate from the image processing unit 201, and includes a data receiving unit 302 for receiving data from the image processing unit 201, a data correction unit 303, an image processing circuit group 304, a recording unit 305, an input I / F unit 306, a decompression unit 307, a phase difference detection unit 308, and a lens control unit 309.

[0067] For example, different image processing ICs are prepared for the image processing unit 201 and the image processing unit 301, respectively.

[0068] The input I / F unit 202 receives the captured image data transmitted from the output I / F unit 107 via the transmission path 112. The input I / F unit 202 transmits the obtained captured image data to the decompression unit 203. The decompression unit 203 decodes the captured image data compressed by the compression unit 106. The decoded data is transmitted to the data correction unit 204. The data correction unit 204 performs various types of correction processes on the captured image data obtained by the image sensor unit 102. For example, processes such as shadow correction, gain correction, and defect correction are performed. The data after correction by the data correction unit 204 is output to the image processing circuit group 205. The image processing circuit group 205 applies white balance adjustment, noise removal processing, etc. to the output data from the data correction unit 204, and transmits the processed captured image data to the data transmission unit 206. The image processing unit 201 includes a buffer memory (not shown), etc., and can appropriately adjust the speed of image processing (adjust the processing speed) using the buffer memory. The data transmission unit 206 transmits the captured image data to the data reception unit 302 of the image processing unit 301.

[0069] The data reception unit 302 receives the captured image data from the data transmission unit 206 and transmits the captured image data to the data correction unit 303. The data correction unit 303 performs various types of correction processes on the captured image data. For example, processes such as shadow correction, gain correction, and defect correction are performed. The captured image data after correction by the data correction unit 303 is output to the image processing circuit group 304. Since the image processing unit 201 also includes the data correction unit 204, the correction process can be performed in only one of the data correction unit 204 and the data correction unit 303, or the processing can be shared between these two data correction units. The image processing circuit group 304 applies white balance adjustment, noise removal processing, etc. to the captured image data output from the data correction unit 303, and transmits the captured image data to the recording unit 305. Since the image processing unit 201 also includes the image processing circuit group 205, the image processing can be performed in only one of the image processing circuit group 205 and the image processing circuit group 304, or the processing can be shared between these two image processing circuit groups. The recording unit 305 records the captured image data into the recording medium 310. The recording medium 310 includes, for example, memory cards such as SD cards and CF cards, stores image data, and can be attached to and detached from the imaging device 100.

[0070] The input I / F unit 306 receives the focus detection data transmitted from the output I / F unit 111 via the transmission path 113. The input I / F unit 306 transmits the obtained focus detection data to the decompression unit 307. The decompression unit 307 decodes the focus detection data compressed by the compression unit 109. The decoded data is transmitted to the phase difference detection unit 308. The phase difference detection unit 308 calculates the focus displacement amount (defocus amount).

[0071] Figure 7A and Figure 7B is a diagram showing the correlation between the image signal waveform 701 obtained from the sub-pixel a and the image signal waveform 702 obtained from the sub-pixel b for different focus states.

[0072] As Figure 7A shown, in the defocus state, the image signal waveform 701 and the image signal waveform 702 obtained from the sub-pixel a and the sub-pixel b respectively do not match and are significantly shifted from each other. When the focus state is substantially achieved, as Figure 7B shown, the displacement between the image signal waveform 701 and the image signal waveform 702 decreases; in the in-focus state, the image signal waveform 701 and the image signal waveform 702 overlap each other. In this way, the phase difference detection unit 308 can calculate the focus displacement amount (defocus amount) based on the displacement amount between the image signal waveform 701 obtained from the sub-pixel a and the image signal waveform 702 obtained from the sub-pixel b.

[0073] The lens control unit 309 calculates the drive information of the optical system based on the information on the focus displacement amount (defocus amount) calculated by the phase difference detection unit 308, and controls the photographic lens 101.

[0074] As described above, according to the first embodiment, it is possible to perform focus detection and the like in response to an increase in the amount of data from the image sensor while suppressing an increase in the chip cost of the image sensor and the signal processing IC.

[0075] In addition, in the present embodiment, although it is assumed that different image processing ICs are respectively prepared for the image processing unit 201 and the image processing unit 301, a structure in which two chips of the image processing unit 201 and the image processing unit 301 are in one package is also allowed.

[0076] (Second Embodiment)

[0077] In the second embodiment, only the image signal AB read out by the data classification unit 105 is transmitted as the captured image data to the compression unit 106, and only the image signal A read out from the pixel region (Region_i) where the focus detection process is performed is transmitted as the focus detection signal to the size adjustment unit 108.

[0078] The decompression unit 203 decompresses the captured image data composed of the image signal AB and sent to the image processing unit 201. Then, the decompressed captured image data is sent to the image processing unit 301 via the data correction unit 204 and the image processing circuit group 205 through the data transmission unit 206, and stored in a buffer memory (not shown). The subsequent data correction unit 303 and the image processing circuit group 304 process the captured image data composed of the image signal AB stored in the buffer memory, thereby generating a captured image.

[0079] In addition, in parallel with the foregoing processing, the decompression unit 307 decompresses the focus detection data composed of the image signal A and sent to the image processing unit 301, and obtains the image signal B by subtracting the focus detection data from the image signal AB stored in the foregoing buffer memory. The phase difference can be detected based on these image signals A and image signal B. In this way, the focus detection data can be further reduced.

[0080] Other embodiments

[0081] Embodiments of the present invention can also be implemented by the following method, that is, a software (program) that executes the functions of the above embodiments is provided to a system or device through a network or various storage media, and the computer or the central processing unit (CPU) or microprocessing unit (MPU) of the system or device reads and executes the program.

[0082] Although the present invention has been described with reference to exemplary embodiments, it should be understood that the present invention is not limited to the disclosed exemplary embodiments. The scope of the appended claims will be given the broadest interpretation so as to cover all such modifications and equivalent structures and functions.

Claims

1. An image sensor, comprising: a pixel portion in which unit pixels are arranged in a matrix, each of the unit pixels including a microlens and a plurality of photoelectric conversion units; a readout unit for reading out a signal from the pixel portion, the readout unit being capable of performing a first readout operation of reading out a mixed signal obtained by mixing the signals of the plurality of photoelectric conversion units of the unit pixel, and a second readout operation of reading out a non-mixed signal which is not a mixture of the signals of the plurality of photoelectric conversion units of the unit pixel; a classification component for classifying the signals obtained by the first readout operation and the second readout operation into a captured image signal and a focus detection signal; a first output unit, which outputs the captured image signal classified by the classification component; as well as A second output unit outputs the focus detection signal classified by the classification component.

2. The image sensor according to claim 1, wherein: For a unit pixel in a first area of ​​the pixel portion, the first readout operation is performed and the mixed signal is read out, and for a second area of ​​the pixel portion different from the first area, the first readout operation and the second readout operation are performed and the mixed signal and the non-mixed signal are read out.

3. The image sensor according to claim 2, wherein: The classification section distributes the mixed signal having been read out from the first area to the first output unit, and distributes the mixed signal and the non-mixed signal having been read out from the second area to the second output unit.

4. The image sensor according to claim 1, wherein: A communication speed at which a signal is output from the first output unit is higher than a communication speed at which a signal is output from the second output unit.

5. The image sensor according to claim 1, further comprising: A size adjustment component is used to reduce the focus detection signal before outputting the focus detection signal from the second output unit.

6. The image sensor according to claim 1, further comprising: A memory is used to adjust the processing speed according to the communication speed of the second output unit.

7. The image sensor according to claim 1, wherein: The classification means transmits the focus detection signal to the second output unit once for every plurality of frames.

8. A camera device, comprising: An image sensor, the image sensor comprising: a pixel portion in which unit pixels are arranged in a matrix, each of the unit pixels including a microlens and a plurality of photoelectric conversion units; a readout unit for reading out a signal from the pixel portion, the readout unit being capable of performing a first readout operation of reading out a mixed signal obtained by mixing the signals of the plurality of photoelectric conversion units of the unit pixel, and a second readout operation of reading out a non-mixed signal which is not a mixture of the signals of the plurality of photoelectric conversion units of the unit pixel; a classification component for classifying the signals obtained by the first readout operation and the second readout operation into a captured image signal and a focus detection signal; a first output unit that outputs the captured image signal classified by the classification component; and a second output unit that outputs the focus detection signal classified by the classification unit; a first processing unit that processes the captured image signal output from the first output unit; and The second processing component is used to process the focus detection signal output from the second output unit.

9. The imaging device according to claim 8, wherein: The first processing section transmits the captured image signal to the second processing section.

10. The imaging device according to claim 9, wherein: The second processing unit records the captured image signal in a recording medium.

11. The imaging device according to claim 8, wherein: The second processing section performs focus detection based on the focus detection signal.

Images