Imaging element, detection device, and imaging device

Abstract

To provide an imaging element capable of shortening latency.SOLUTION: The image sensor 22 includes a plurality of first pixels, each of which includes a first opto-electrical conversion part 222 that performs opto-electrical conversion on light that has passed through the optical system to generate an electrical charge and outputs a signal used for image generation on the basis of the electrical charge generated by the first opto-electrical conversion part 222, and a second opto-electrical conversion part 223 that performs opto-electrical conversion on light that has passed through the optical system to generate an electrical charge. The solid-state imaging device includes a plurality of second pixels that output signals used for detection of an imaging state based on charges generated by the second photoelectric conversion unit 223, a first output unit 225 that outputs first data based on signals output from the plurality of first pixels, and a second output unit 226 that outputs second data based on signals output from the plurality of second pixels, and the second output unit 226 outputs a plurality of pieces of second data while the first output unit 225 outputs the first data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an imaging element, a detection device, and an imaging device. [Background technology]

[0002] An image sensor that outputs main pixel signals read from normal pixels and focus pixel signals read from focus detection pixels is known (see, for example, Patent Document 1). There has long been a demand for faster focus detection. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-108176 Summary of the Invention

[0004] According to an aspect of the invention, an imaging element includes a first photoelectric conversion unit that photoelectrically converts light that has passed through an optical system to generate electric charges, and a plurality of first pixels that output signals used for image generation based on the electric charges generated by the first photoelectric conversion unit; a second photoelectric conversion unit that photoelectrically converts light that has passed through the optical system to generate electric charges, and a plurality of second pixels that output signals used for detecting a shooting state based on the electric charges generated by the second photoelectric conversion unit; a first output unit that outputs first data based on the signals output from the plurality of first pixels; and a second output unit that outputs second data based on the signals output from the plurality of second pixels, and the second output unit outputs a plurality of the second data while the first output unit outputs the first data. [Brief explanation of the drawings]

[0005] [Figure 1] FIG. 1 is a block diagram showing a camera according to this embodiment. [Figure 2] 2 is a block diagram of the camera control unit and the image sensor. [Figure 3]FIG. 3 is a front view showing a schematic arrangement of pixels included in an imaging element. [Figure 4] FIG. 4 is a diagram illustrating a schematic configuration of an imaging element. [Figure 5] FIG. 5 is a diagram illustrating the circuit configuration of a pixel. [Figure 6] FIG. 6 is a diagram showing an example of exposure of focus detection pixels and normal pixels, and two streams output from the image sensor. [Figure 7] FIG. 7 is a diagram showing an example of exposure of focus detection pixels and normal pixels, and two streams output from the image sensor. [Figure 8] FIG. 8 is a diagram showing an example of exposure of focus detection pixels and normal pixels, and one stream output from the image sensor. DETAILED DESCRIPTION OF THE INVENTION

[0006] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0007] 1 is a diagram showing the essential configuration of a digital camera 1 according to this embodiment of the present invention. The digital camera 1 according to this embodiment (hereinafter simply referred to as camera 1) comprises a camera body 2 and a lens barrel 3.

[0008] The lens barrel 3 is, for example, an interchangeable lens that can be attached to and detached from the camera body 2. As shown in Figure 1, the lens barrel 3 has a photographing optical system that includes a zoom lens 31, a focus lens 32, and an aperture .

[0009] The zoom lens 31 is provided so as to be movable along the optical axis L1 of the photographing optical system. The zoom operation is performed by moving the zoom lens 31 in the direction of the optical axis L1. The position of the zoom lens 31 is detected by an encoder and transmitted to the lens control unit 36.

[0010] The focus lens 32 is provided so as to be movable along the optical axis L1 of the photographing optical system. The focal length of the photographing optical system is adjusted by moving the focus lens 32 in the direction of the optical axis L1. The position of the focus lens 32 is detected by an encoder 35, and the position is adjusted by a drive motor 36.

[0011] Position information of focus lens 32 detected by encoder 35 is sent to camera control unit 21 (described later) via lens control unit 37. Camera control unit 21 calculates a new position of focus lens 32 based on the position information of focus lens 32. The calculated position information of focus lens 32 is sent from camera control unit 21 via lens control unit 37 to drive motor 36. Drive motor 36 adjusts the position of focus lens 32 based on the calculated position information of focus lens 32.

[0012] The aperture 34 is configured so that the diameter of the opening centered on the optical axis L1 can be adjusted, and limits the amount of light incident on the imaging element 22.

[0013] The camera body 2 includes a camera control unit 21, an image sensor 22, a recording unit 24, a display unit 25, and an operation unit 26. The camera control unit 21 is a controller that controls the entire camera 1. The camera control unit 21 includes a CPU and peripheral components such as memory. The camera control unit 21 controls the operation of the camera 1, such as controlling the image sensor 22, reading signals from the image sensor 22, adjusting the position of the photographic optical system included in the lens barrel 3, displaying images based on signals read from the image sensor 22 on the display unit 25, and recording image data in the recording unit 24. The camera control unit 21 also communicates with the lens control unit 37 to receive lens information and transmit camera information (such as defocus amount and aperture value). The camera control unit 21 also includes an image processing unit 217 and a detection unit 218, which will be described later. The image processing unit 217 generates image data, and the detection unit 218 performs focus detection and calculations for autofocus (AF). The camera control unit 21, image processing unit 217, and detection unit 218 may be provided separately.

[0014] The image sensor 22 has a plurality of pixels, each having a photoelectric conversion unit, arranged two-dimensionally, and is, for example, a CCD image sensor, a CMOS image sensor, etc. The image sensor 22 photoelectrically converts light that has passed through an optical system such as a lens 31 to generate electric charges, and outputs a signal based on the generated electric charges to the camera control unit 21.

[0015] The image sensor 22 is disposed on the optical axis L1 of the light beam from the subject in the camera body 2, at the intended focal plane of the photographic optical system including the zoom lens 31 and the focus lens 32.

[0016] The recording unit 24 has a card slot into which a storage medium such as a memory card can be inserted. The recording unit 24 stores the image data and various data generated by the image processing unit 217 in the storage medium inserted in the card slot. The recording unit 24 has an internal memory. In this case, the recording unit 24 can also record the image data and various data generated by the image processing unit 217 in the internal memory.

[0017] The display unit 25 displays an image based on the image data, an image showing the focus detection area, information related to shooting such as the shutter speed and F-number, a menu screen, etc. The display unit 25 is, for example, a liquid crystal monitor (also referred to as a rear monitor) mounted on the back of the camera body 2. The operation unit 26 includes various setting switches such as a shutter button, a power switch, and switches for switching between various modes, and outputs signals according to the respective operations to the camera control unit 21. By operating the operation unit 26, the user can select a focus detection area, a shooting mode, and take still images or videos.

[0018] Next, the configurations of the camera control unit 21 and the image sensor 22 will be described with reference to FIGS. 2, 3, 4, and 5. FIG. 2 is a block diagram of the camera control unit 21 and the image sensor 22. The image sensor 22 includes a pixel array 221 and a circuit unit 224. FIG. 3 is a front view schematically showing the arrangement of pixels included in the image sensor. The pixel array 221 is an arrangement of multiple pixels included in the image sensor 22, and includes normal pixels and detection pixels. The RGB pixels shown in FIG. 3 are normal pixels, and are pixels used for normal image capture (still image capture, video capture, and live view image capture). The RGB pixels are arranged in, for example, a Bayer array. The Pa and Pb pixels shown in FIG. 3 are detection pixels (focus detection pixels) used for focus detection using a phase difference detection method, and form a pair of pixels at the right and left openings. The focus detection pixels Pa and Pb are also arranged discretely in some rows of the normal pixels (RGB pixels) arranged in the Bayer array. Note that the arrangement of the focus detection pixels Pa and Pb is not limited to this. Furthermore, a signal output from a normal pixel is called a pixel signal, and a signal output from a detection pixel is called a detection signal.

[0019] The pixel signals read out from the normal pixels are used to generate still images and moving images, and are output from the image sensor 22 in response to the timing of shooting when the user fully presses the shutter button. In addition, the pixel signals read out from the normal pixels are used to generate an image (live view image (through image)) to be displayed on the display unit 25 at times other than the timing of shooting, and are output from the image sensor 22 at a frame rate according to a predetermined display rate.

[0020] The detection signals read out from the focus detection pixels are used to detect the shooting state. The detection signals output from the focus detection pixels are used to detect the amount of focus shift (defocus amount) relative to the subject. The detection signals are output from the image sensor 22 at intervals shorter than the output intervals of pixel signals read out from normal pixels. In this case, the exposure time of the detection pixels is shorter than that of the normal pixels, and the number of exposures is greater than that of the normal pixels.

[0021] The detection signal is not necessarily limited to a signal output from a focus detection pixel, but may be a signal output from a normal pixel. For example, when detecting flicker as a shooting state, the detection signal is a signal output from a normal pixel. When detecting flicker, the normal pixel also serves as a detection pixel. The camera control unit 21 calculates exposure conditions (exposure time, aperture value) and number of exposures suitable for flicker detection, and controls the image sensor 22 based on the calculated exposure conditions and number of exposures. The image sensor 22 performs exposure for flicker detection using detection pixels (flicker detection pixels) and outputs a signal used for flicker detection. The signal output from the focus detection pixel may also be used for flicker detection.

[0022] Returning to FIG. 3, some lines (rows) of the pixel array shown in FIG. 3, which will explain the reading of pixels for generating a live view image and the reading of pixels for flicker detection, are pixel lines L including pixels that output signals used to generate a live view image. A (first pixel row) and pixel line L including pixels that output signals used to detect flicker B (second pixel row) and pixel line L including pixels that output signals used for focus detection C (third pixel row). The image sensor 22 has a pixel line L A Image data for generating a live view image based on signals read from the pixel line L B The detection data used for flicker detection based on the signal read from the pixel line L A The detection data used for focus detection is output based on the signal read from the image sensor.

[0023] FIG. 4 is a diagram illustrating a schematic configuration of the imaging element 22 according to this embodiment. 4, the image sensor 22 includes a pixel array 221 including a plurality of pixels arranged two-dimensionally in M ​​rows and N columns (M and N are any natural numbers), a plurality of control lines including transfer control lines, reset control lines, and selection control lines, a plurality of vertical signal lines including first vertical signal lines and second vertical signal lines, and a circuit unit 224 including a drive control unit 228, a vertical drive unit 227, a horizontal drive unit 230, a signal processing unit 229, a first output unit 225, and a second output unit 226. As described above, the pixels are arranged in a Bayer array and include normal pixels and detection pixels.

[0024] As shown in Figure 4, the pixels are connected in the horizontal direction (row direction) to transfer control lines, reset control lines, and selection control lines. The pixels are also connected in the vertical direction (column direction) to vertical signal lines so that signals can be read out for each row. The pixels are configured to accumulate charge with different exposure times for each row and to read out signals. The exposure times and number of exposures for the normal pixels and detection pixels can be set independently. The signals for the normal pixels and detection pixels can also be read out independently.

[0025] FIG. 5 is a diagram illustrating the circuit configuration of a pixel according to this embodiment. As shown in Fig. 5, pixel (m, n) has a photoelectric conversion unit 240 such as a photodiode (PD) and a readout unit 231. The normal pixel has a first photoelectric conversion unit 222, and the focus detection pixels Pa and Pb each have a second photoelectric conversion unit 223. The photoelectric conversion unit 240 has a function of converting incident light into electric charges and accumulating the photoelectrically converted electric charges. The readout unit 231 has a transfer unit 2311, a discharge unit 2312, a floating diffusion (FD) 2313, an amplifier 2314, a first selection unit 2315, and a second selection unit 2316.

[0026] The transfer unit 2311 is controlled by the vertical driver 227 via a transfer control line, and transfers the charges photoelectrically converted by the photoelectric conversion unit 240 to the floating diffusion. That is, the transfer unit 2311 forms a charge transfer path between the photoelectric conversion unit 240 and the floating diffusion 2313. The floating diffusion 2313 holds (accumulates) the charges. The amplifier 2314 outputs a signal based on the charges held in the floating diffusion 2313. The amplifier 2314 is connected to a first vertical signal line 241 via a first selection unit 2315 and to a second vertical signal line 242 via a second selection unit 2316. The discharge unit (reset unit) 2312 is controlled by the vertical driver 227 via a reset control line, and discharges the charges from the floating diffusion 2313 and resets the potential of the floating diffusion 2313 to a reset potential (reference potential).

[0027] The first selection unit 2315 is controlled by the vertical drive unit 227 via a selection control line and outputs a signal from the amplifier unit 2314 to a first vertical signal line 241. The second selection unit 2316 is controlled by the vertical drive unit 227 via a selection control line and outputs a signal from the amplifier unit 2314 to a second vertical signal line 242. The transfer unit 2311, the discharge unit 2312, the amplifier unit 2314, the first selection unit 2315, and the second selection unit 2316 are each composed of, for example, transistors M1, M2, M3, M4, and M5. Note that although there is no difference between normal pixels and focus detection pixels in terms of their circuit diagrams, there is a structural difference in that focus detection pixels have a light-shielding unit that blocks part of the light that enters the photoelectric conversion unit 240.

[0028] Returning to Figure 4, the drive control unit generates vertical synchronization signals and control signals that serve as the basis for the operation of the vertical drive unit, signal processing unit, and horizontal drive unit 230, etc., based on a master clock input from outside and a control signal input from the camera control unit 21, and controls the vertical drive unit 227, signal processing unit 229, horizontal drive unit 230, etc.

[0029] The vertical drive unit 227 supplies control signals to N pixels 51 arranged in the mth row via control lines to control the operation of each pixel 51. The signal processing unit has an analog-to-digital conversion unit (ADC circuit), a correlated double sampling unit (CDS circuit), and a memory. The ADC circuit converts analog signals from the pixels input via the first and second vertical signal lines into digital signals. The CDS circuit performs correlated double sampling, removes noise from the digital signals converted by the ADC circuit, and outputs the noise-removed digital signals to memory. The memory stores the signals of each pixel (pixel signals) output from the CDS circuit. The horizontal drive unit 230 outputs the pixel signals stored in the signal processing unit (memory) 229 to the first output unit 225 or the second output unit 226 based on a control signal from the drive control unit.

[0030] For example, the vertical drive unit 227 reads out signals from normal pixels to the signal processing unit 229 via a first vertical signal line 241, and reads out signals from focus detection pixels to the signal processing unit 229 via a second vertical signal line 242. The first output unit 225 outputs the normal pixel signals read out to the signal processing unit 229, and the second output unit 226 outputs the focus detection pixel signals read out to the signal processing unit 229, to the camera control unit 21. Note that the first output unit 225 and the second output unit 226 may output the normal pixel signals to the camera control unit 21. The first output unit 225 outputs image data (first data) based on pixel signals output from the first output unit 225. The second output unit 226 outputs detection data (second data) based on detection signals output from the detection pixels. The image data is data for constituting image frames of consecutive images such as still images, videos, and live view images, and the detection data is data for constituting detection frames used for focus detection or flicker detection. The first output unit 225 is configured to form a signal flow (first stream) of image data. The second output unit 226 is configured to form a signal flow (second stream) of detection data. The first output unit 225 and the second output unit 226 can output the image data and the detection data to the camera control unit 21 in parallel.

[0031] The second output unit 226 may output image data other than the detection data. For example, the first output unit 225 thins out signals from the normal pixels and outputs image data for live view (live view image data) for displaying a live view image. Furthermore, the second output unit 226 outputs image data for still images (still image data: third data) based on signals output from the normal pixels in parallel with the image data for live view from the first output unit 225. The still image data is stored in memory. That is, the image sensor 22 can process the image data for live view and the detection data separately and simultaneously using the two streams from the first output unit 225 and the second output unit 226. Alternatively, the image sensor 22 can process the image data for live view and the still image data separately and simultaneously. In this way, the image sensor 22 outputs data in two streams.

[0032] The image processing unit 217 performs image processing on the signal output from the image sensor 22 to generate image data for live view and image data for still images and / or moving images to be recorded in the recording unit 24.

[0033] The detector 218 performs focus detection and / or flicker detection using a phase difference detection method based on the signal output from the image sensor 22. In the case of focus detection, the camera controller 21 calculates exposure conditions (exposure time, aperture value) and the number of exposures suitable for focus detection. The camera controller 21 controls the image sensor 22 using the calculated exposure conditions and the number of exposures. The image sensor 22 performs exposure for focus detection using detection pixels (focus detection pixels) described below. The detector 218 analyzes the phase difference between a pair of images based on detection data derived from the detection signals output from the detection pixels, and detects the amount of focus deviation (defocus amount). The camera controller 21 outputs the defocus amount detected by the detector 218 to the lens controller 37. The lens controller 37 adjusts the position of the focus lens 32 based on the defocus amount.

[0034] In the case of flicker detection, the camera control unit 21 calculates exposure conditions (exposure time, aperture value) and the number of exposures suitable for flicker detection. The camera control unit 21 controls the image sensor 22 using the calculated exposure conditions and the number of exposures. The image sensor 22 performs exposure for flicker detection using detection pixels (RGB pixels). The detection unit 218 performs flicker detection based on detection data derived from detection signals output from the detection pixels. The detection unit 218, for example, compares a flicker detection frame identified from the detection data with frames before and after the flicker detection frame to detect the flicker cycle (frequency) of the light source from changes in brightness. Based on the flicker detection result detected by the detection unit 218, the camera control unit 21 calculates exposure conditions (exposure start timing, exposure end timing, exposure time, aperture value) and the number of exposures so that the shooting timing matches the flicker cycle. The camera control unit 21 controls the image sensor 22 using the calculated exposure conditions and the number of exposures. For example, the camera control unit 21 controls the image sensor 22 to use exposure conditions and the number of exposures that match the peak of the flicker (when the light source is brightest / darkest) so as to reduce the effect of the flicker on the image.

[0035] Next, an example of a live view image processing and autofocus (AF) processing sequence will be described. FIG. 6 is a diagram showing an example of the exposure and signal readout of normal pixels and detection pixels, and two streams output from the image sensor 22. (a1) in FIG. 6 shows the exposure and signal readout of normal pixels, and (a2) shows the exposure and signal readout of focus detection pixels. (b1) in FIG. 6 shows the output of image data based on signals (pixel signals) read out from normal pixels, and illustrates a processing sequence using the first stream (output of image data by the first output unit 225). (b2) shows the output of detection data based on signals (detection signals) read out from focus detection pixels, and illustrates a processing sequence using the second stream (output of detection data by the second output unit 226). The processing sequences in (a1) and (b1) in FIG. 6 show the processing sequence of image data from normal pixels (image data used to generate a live view image), and the processing sequences in (a2) and (b2) show the processing sequence of detection data from focus detection pixels. The vertical synchronization signal represents a synchronization signal generated at a predetermined time interval by the drive control unit.

[0036] As shown in (a1) of FIG. 6, multiple exposure processes (V1, V2, V3) are performed by normal pixels. The exposure processes are performed in synchronization with a vertical synchronization signal. In the exposure process (V1), exposure of multiple normal pixels is performed at time t а1 Starting sequentially from time t а2 After the exposure process (time t а2 After that, as shown in FIG. 6(b1), the first stream is processed by the first output unit 225. The processing of the first stream is performed in synchronization with the vertical synchronization signal. In the processing of the first stream, the output of image data based on the exposure process (V1) is started at time t а2 The first output unit 225 starts outputting the signal at time t а3 The exposure processes (V1, V2, V3) and the first stream process are executed at a predetermined cycle (a cycle that is an integral multiple of the vertical synchronization signal).

[0037] As shown in (a2) of FIG. 6, the multiple exposure processes by the focus detection pixels are performed at time intervals shorter than the intervals between the exposure processes of the normal pixels. The exposure processes of the detection pixels are performed independently of the exposure processes of the normal pixels. In the example of FIG. 6, the multiple exposure processes by the detection pixels are performed seven times during the interval between the exposure processes of the normal pixels (between V1 and V2). In other words, the number of multiple exposure processes by the detection pixels per unit time (a time that is an integer multiple of the vertical synchronization signal) is greater than the number of exposure processes of the normal pixels. Furthermore, the exposure processes of the detection pixels are performed multiple times during one exposure process of the normal pixels. In the example of FIG. 5, the exposure processes of the detection pixels are performed three times during one exposure process of the normal pixels.

[0038] After the exposure process of the focus detection pixels, as shown in FIG. 6(b2), the second output unit 226 processes the second stream. The processing of the second stream is performed in synchronization with the vertical synchronization signal. In the processing of the second stream, detection data based on the exposure process is output. The exposure process of the focus detection pixels and the processing of the second stream are performed at a predetermined cycle (the cycle of the vertical synchronization signal).

[0039] Image data used to generate a live view image is output as a first stream, and detection data (focus detection data) used for focus detection is output as a second stream. Processing of the first stream and processing of the second stream are performed independently. Frames (image frames) made up of image data and detection frames (focus detection frames) made up of focus detection data are output independently and in parallel; in the example of FIG. 6, two detection frames are output in parallel with one image frame. Furthermore, the number of detection frames output from the second output unit 226 per unit time (a time that is an integer multiple of the vertical synchronization signal) is greater than the number of image frames output from the first output unit 225. Then, the focus detection data (focus detection frames) is input to the detection unit 218 and used to detect the defocus amount.

[0040] However, when capturing images under difficult conditions, such as when the subject moves at high speed, the subject has low contrast, or the image is backlit, focus detection becomes difficult, which can result in a deterioration in AF performance (speed and accuracy). Unlike the present embodiment, when one focus detection frame is output for one image frame, the following effects occur: For example, when the subject moves at high speed, focus detection cannot track the subject's movement, and AF speed decreases. Furthermore, when the subject has low contrast, for example, the focus detection data used to detect the defocus amount becomes obscured by noise, making it impossible to focus on the subject and resulting in a decrease in AF accuracy. Furthermore, when the scene has a large difference between light and dark within the image capture range, such as a backlit scene, there is little focus detection data (focus detection frames) used to detect the defocus amount, making it impossible to focus on the appropriate position and resulting in a decrease in AF accuracy.

[0041] In this embodiment, the number of exposures for focus detection pixels is greater than the number of exposures for normal pixels, and the number of focus detection frames per unit time is greater than the number of image frames. In other words, the image sensor 22 outputs multiple focus detection data for one image data. This increases the frame rate of the focus detection data, suppresses degradation of AF performance for fast-moving or low-contrast subjects, and maintains high AF performance.

[0042] This embodiment is suitable for shooting conditions where the subject is moving at high speed and has low contrast. In addition, because the frame rate of focus detection data is high, the frame difference is small, making it possible to track and detect focus even on fast-moving subjects.

[0043] In a scene where the subject is moving at high speed, the detection unit 218 performs a detection process for focus detection on each of the multiple focus detection data items output from the image sensor 22, and moves the optical system based on the results of the detection process. This allows the focus lens 32 to be moved to a focused position so as to track the movement of the subject. Furthermore, in a scene where the subject has low contrast, the detection unit 218 performs an average calculation on the multiple focus detection data items output from the image sensor 22 and then performs a detection process for focus detection. That is, the detection unit 218 performs an average calculation on the multiple focus detection data items before performing focus detection on the optical system. By taking the average, noise components contained in the focus detection data are also averaged, and therefore the evaluation value used for focus detection can be determined from the data after the averaging process.

[0044] In this embodiment, in a scene with a high contrast (e.g., a backlit scene), exposure signals from detection pixels are read out using the exposure time and number of exposures shown in FIG. 7. FIG. 7 is a diagram showing an example of the exposure of focus detection pixels and normal pixels, and two streams output from the image sensor 22. (a1) in FIG. 7 shows the exposure and signal readout of normal pixels, and (a2) shows the exposure and signal readout of detection pixels. (b1) in FIG. 7 shows the output of image data based on signals (pixel signals) read out from normal pixels, and illustrates a processing sequence for the first stream (output of image data by the first output unit 225). (b2) shows the output of detection data based on signals (detection signals) read out from focus detection pixels, and illustrates a processing sequence for the second stream (output of detection data by the second output unit 226). Note that the exposure of normal pixels shown in (a1) and the processing sequence for the first stream shown in (b1) are the same as the exposure of normal pixels and the processing sequence for the first stream described with reference to (a1) and (b1) in FIG. 6, respectively, and therefore will not be described again.

[0045] As shown in (a2) of FIG. 7, the exposure process for the focus detection pixels is performed with two types of exposure times: long exposure time and short exposure time. In the example of FIG. 7, the exposure process for the normal pixels is performed once, and the exposure process for the focus detection pixels is performed at least twice. Note that the number of times the focus detection pixels are exposed does not necessarily have to be two, as long as it is greater than the number of times the normal pixels are exposed. The long exposure time (first exposure time) is performed at time t b1 Starting from time t b2 The exposure for the short exposure time (second exposure time) ends at time t b3 (Time t b2 The time starts from t b4 It ends at time t b1 From time t b2 The period from time t b3 From time t b4 The period from the moment the subject is shot to the moment the subject is shot may be a short exposure time that is set to match the bright part of the image capture area. The camera control unit 21 sets the length of the exposure time depending on the scene to be captured. The exposure time may be set depending on the image capture mode selectable by the user.

[0046] After the exposure process of the focus detection pixels, as shown in FIG. 7(b2), the second output unit 226 processes the second stream. In the processing of the second stream, two sets of focus detection data (focus detection frames) based on the exposure process are output. Image data (image frames) used to generate a live view image are output in the first stream. The image sensor 22 outputs two sets of focus detection data for one piece of image data. The processing of the first stream and the processing of the second stream are each performed independently. Then, the detection unit 218 executes focus detection processing based on the focus detection data.

[0047] In this embodiment, exposures are performed with different exposure times. That is, the image sensor 22 outputs two focus detection frames for one image frame, and performs multiple exposures with different exposure times to obtain focus detection data. This increases the frame rate of the focus detection frames, and by performing focus detection processing based on two focus detection data sets with different exposure times, it is possible to suppress deterioration of AF performance even with subjects with high contrast, and to achieve high-precision focusing.

[0048] A modified example of this embodiment will be described below. In the following description, the same components as those in the above embodiment will be assigned the same reference numerals, and the description above will be used where appropriate. In this modified example, the image sensor 22 outputs image data and detection data in a single stream. In this modified example, the circuit unit 224 included in the image sensor 22 outputs image data based on pixel signals output from normal pixels (RGB pixels) and detection data based on detection signals output from detection pixels (focus detection pixels). The circuit unit 224 is not configured to generate separate signal flows for image data and detection data, but is configured to generate a single stream for both image data and detection data. The configuration of the camera control unit 21 and the configuration of the image sensor 22 other than the circuit unit 224 are the same as those in the above embodiment.

[0049] Fig. 8 is a diagram showing an example of exposure of focus detection pixels and normal pixels, and one stream output from the image sensor 22. (a1) of Fig. 8 shows exposure and readout of normal pixels, and (a2) shows exposure and readout of detection pixels. (b) of Fig. 8 shows output of image data by the circuit unit 224 based on signals read out from normal pixels (pixel signals), and output of detection data based on signals read out from focus detection pixels (detection signals).

[0050] As shown in (a1) of FIG. 8, multiple exposure processes (V1, V2, V3) are performed by normal pixels. The exposure processes are performed in synchronization with a vertical synchronization signal. In the exposure process (V1), exposure of multiple normal pixels is performed at time t а1 Starting sequentially from time tа2 After the exposure process (time t а2 After that, as shown in FIG. 8(b), the circuit unit 224 processes the stream. In the stream processing, the image data based on the exposure process (V1) is output at time t а2 The circuit unit 224 starts the process at time t а3 The exposure process (V1, V2, V3) and the stream process are executed at a predetermined cycle (a cycle that is an integral multiple of the vertical synchronization signal).

[0051] As shown in (a2) of FIG. 8, multiple exposure processes by focus detection pixels are performed at time intervals shorter than the intervals between exposure processes by normal pixels. The exposure processes by detection pixels are performed independently of the exposure processes by normal pixels. In the example of FIG. 9, multiple exposure processes by detection pixels are performed five times during the interval between exposure processes by normal pixels (between V1 and V2). In other words, the number of multiple exposure processes by detection pixels per unit time (a time that is an integer multiple of the vertical synchronization signal) is greater than the number of exposure processes by normal pixels. Furthermore, the exposure processes by detection pixels are performed multiple times during one exposure process of normal pixels. Furthermore, of the five exposures of the detection pixels, the exposure time of one exposure is longer than the other four exposures.

[0052] After the exposure process of the focus detection pixels, the circuit unit 224 processes the stream as shown in FIG. 8(b). In the stream processing, the detection data based on the exposure process is output. The detection data is output during the image data output period (t in FIG. 8). а2 From t а3 It is not performed during the period corresponding to the time until the image data is output, but during the period other than the image data output period. For example, а2 From time t а4 In the period up to t, the output of the detection data is started after the output of the image data based on the exposure process (V1) is completed (time t а3 (after the previous one) will be carried out sequentially.

[0053] The circuit unit 224 outputs one image frame and five detection frames per predetermined period. That is, since the circuit unit 224 is configured with one stream, it outputs detection frame data before and after outputting a normal image frame.

[0054] Even when data output from the image sensor 22 is configured as a single stream, as in the modified example, the number of exposures of the focus detection pixels is greater than the number of exposures of the normal pixels, and the number of focus detection frames per unit time can be greater than the number of image frames. In other words, the image sensor 22 can output multiple focus detection data for one image data. This increases the frame rate of the focus detection data, suppresses deterioration of AF performance for fast-moving or low-contrast subjects, and maintains high AF performance.

[0055] Furthermore, as in the modified example, the image sensor 22 performs multiple exposures with different exposure times to obtain focus detection data, which increases the frame rate of focus detection frames and performs focus detection processing based on two sets of focus detection data with different exposure times, thereby suppressing degradation of AF performance even when the subject has a high contrast ratio, enabling highly accurate focusing.

[0056] 6 to 8, the image data from the normal pixels is used to generate a live view image. However, the image data from the normal pixels may be used to generate a still image or a moving image. Furthermore, the detection data from the detection pixels is used to detect focus. However, the detection data may be used to detect flicker. This increases the frame rate of the detection data, thereby improving the accuracy of flicker detection. In particular, high detection performance can be achieved even in shooting scenes where flicker detection is difficult. [Explanation of symbols]

[0057] 1 camera 2 Camera body 3 Lens barrel 4 Mounting section 21 Camera control unit 22 Image sensor 24 Recording section 25 Display section 26 Control section 31, 32 Lenses 34 Aperture 35 Encoder 36 Focus lens drive motor 37 Lens control unit 217 Image Processing Unit 218 Detector 219 Image sensor control unit 221 pixel array 222 First photoelectric conversion unit 223 Second photoelectric conversion unit 224 Circuit section 225 First Output Section 226 Second output section

Claims

1. a plurality of first pixels each having a first photoelectric conversion unit that performs photoelectric conversion on light that has passed through an optical system to generate electric charges, and that output a signal used for image generation based on the electric charges generated by the first photoelectric conversion unit; a plurality of second pixels each having a second photoelectric conversion unit that performs photoelectric conversion on light that has passed through the optical system to generate electric charges, and that output a signal used to detect an imaging state based on the electric charges generated by the second photoelectric conversion unit; a first output unit that outputs first data based on signals output from the plurality of first pixels; a second output unit that outputs second data based on signals output from the plurality of second pixels, The second output unit outputs a plurality of pieces of the second data while the first output unit outputs the first data.

2. 2. The imaging device according to claim 1, The second pixel is an imaging element that outputs a plurality of signals while a signal is output from the first pixel.

3. 3. The imaging device according to claim 1, The second output section outputs the second data for a number of frames greater than the number of frames of the first data output from the first output section per unit time.

4. 4. The imaging device according to claim 3, The second output unit is an image sensor that outputs the second data used for focus detection of the optical system based on the signal output from the second pixel.

5. 4. The imaging device according to claim 3, The second output unit is an imaging element that outputs the second data used for detecting flicker of a light source based on the signal output from the second pixel.

6. 6. The imaging device according to claim 4, The image sensor outputs third data used to generate a still image based on a signal output from the first pixel while the second output unit is not outputting the second data.

7. 7. The imaging device according to claim 6, The imaging element outputs the third data while the second output unit does not output the second data.

8. 6. The imaging device according to claim 4, the second pixel outputs a first signal based on charges generated in the second photoelectric conversion unit for a first time period and a second signal based on charges generated in the second photoelectric conversion unit for a second time period longer than the first time period; The second output unit outputs the second data based on the first signal while the first output unit outputs the first data, and outputs the second data based on the second signal.

9. A detection device equipped with the imaging element according to claim 5, a detection unit that detects a cycle of flicker based on the second data; a control unit that controls timings for starting and ending exposure of the first pixel based on a period of flicker detected by the detection unit.

10. A detection device including the imaging element according to claim 9, The control unit controls the timing of starting and ending exposure of the second pixel so that the detection unit can detect a flicker cycle.

11. 11. The detection device according to claim 9 or 10, an operation unit that is operated to generate an image based on the first data; The detection unit is a detection device that detects a flicker cycle before the operation unit is operated.

12. 5. A detection device comprising the imaging element according to claim 4, a detection unit that performs focus detection of the optical system based on the second data; a control unit that controls a position of the optical system based on focus detection by the detection unit.

13. 13. The detection device according to claim 12, The detection unit is a detection device that performs an averaging operation on a plurality of the second data to perform focus detection of the optical system.

14. The imaging element according to any one of claims 1 to 5, a generation unit that generates an image based on the first data.

Images