Image capture device, image capture device driving method, and program

Abstract

To provide an imaging device, a driving method, and a program that enable appropriate determination of whether a pixel group including a phase difference pixel is to be subjected to short-time exposure or long-time exposure.SOLUTION: An imaging device includes a processor, and an imaging element in which a column signal line for reading out signals extends in a first direction, a first pixel group including a plurality of phase difference pixels and a plurality of imaging pixels arranged in a second direction intersecting the first direction, and a second pixel group including a plurality of imaging pixels arranged in the second direction are arranged in the first direction, and the processor is configured to execute an exposure time setting process for setting one of a first exposure time during which the first pixel group is exposed and a second exposure time during which the second pixel group is exposed to light shorter than the other, and is configured to determine which of the first exposure time and the second exposure time is to be set shorter than the other in the setting process on the basis of information on a subject imaged by the imaging element.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] The technology of the present disclosure relates to an imaging device, a driving method for the imaging device, and a program. [Background technology]

[0002] The image processing device described in Patent Document 1 includes a synthesis processing unit that synthesizes first image data based on first signal charges read out from unit pixels on a first line of a solid-state imaging element exposed during a first exposure period, and second image data based on second signal charges read out from unit pixels on a second line of the solid-state imaging element different from the first line, which is exposed during a second exposure period that is included in the first exposure period but is shorter than the first exposure period.

[0003] The image sensor described in Patent Document 2 uses the following method to read out signal charges accumulated in an area sensor having multiple image sensors arranged in a matrix. First, multiple exposure times with different time lengths are set, and these exposure times are assigned individually to each line of the area sensor. Next, the signal charges accumulated in the image sensors during the assigned exposure times are read out line by line of the area sensor. Then, the read-out signal charges are combined for each screen of the area sensor. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2013-121130 [Patent Document 2] International Publication No. 2006 / 049098 Summary of the Invention

[0005] One embodiment of the technology disclosed herein provides an imaging device, a driving method, and a program that enable appropriate determination of whether a pixel group including a phase difference pixel should be subjected to short-time exposure or long-time exposure. [Means for solving the problem]

[0006] In order to achieve the above object, the imaging device of the present disclosure includes a processor; and an imaging element in which a first pixel group including a plurality of phase difference pixels and a plurality of imaging pixels arranged in a second direction intersecting the first direction, and column signal lines for reading out signals extend in a first direction, and a second pixel group including the plurality of imaging pixels arranged in the second direction, are arranged in the first direction; the processor is configured to execute an exposure time setting process for setting one of a first exposure time for which the first pixel group is exposed and a second exposure time for which the second pixel group is exposed to be shorter than the other, and is configured to determine which of the first exposure time and the second exposure time is to be shorter than the other, based on information of a subject imaged by the imaging element, in the setting process.

[0007] The processor is preferably configured to detect, as the information, brightness of a region of interest within the imaging region using the imaging element, and to determine the first exposure time and the second exposure time based on the detected brightness.

[0008] Preferably, the processor is configured to make the first exposure time shorter than the second exposure time when the brightness is greater than or equal to a first threshold, and to make the second exposure time shorter than the first exposure time when the brightness is less than the first threshold.

[0009] The processor is preferably configured to detect the moving speed of the subject as the information, and to determine the first exposure time and the second exposure time based on the detected moving speed.

[0010] Preferably, the processor is configured to make the first exposure time shorter than the second exposure time when the movement speed is equal to or greater than a second threshold, and to make the second exposure time shorter than the first exposure time when the movement speed is less than the second threshold.

[0011] Preferably, the processor is configured to read out signals from one of the first pixel group and the second pixel group within a frame period, and then read out signals from the other of the first pixel group and the second pixel group.

[0012] In the imaging device disclosed herein, the processor detects the position of the subject in a first direction within an imaging area captured by the imaging element, and if the detected position is on one side in the first direction, reads out a signal from one of the first pixel group and the second pixel group from the other side in the first direction, and then reads out a signal from the other of the first pixel group and the second pixel group from the one side; and if the detected position is on the other side in the first direction, reads out a signal from one of the first pixel group and the second pixel group from the one side in the first direction, and then reads out a signal from the other of the first pixel group and the second pixel group from the other side in the first direction.

[0013] The processor is preferably configured to perform a combining process to generate a video signal by combining signals read out from the first group of pixels and signals read out from the second group of pixels.

[0014] The processor is preferably configured to perform a combining process after performing phase processing to reduce a phase shift between the signals read out from the first pixel group and the signals read out from the second pixel group.

[0015] Preferably, the processor is configured to perform phase processing on the signals read out from the second group of pixels.

[0016] The processor is preferably configured to perform electronic image stabilization processing on the video signal generated by the synthesis processing.

[0017] The imaging device of the present disclosure includes a processor; and an imaging element in which a first pixel group including a plurality of phase difference pixels and a plurality of imaging pixels arranged in a second direction intersecting the first direction, with column signal lines for reading out signals extending in a first direction, and a second pixel group including the plurality of imaging pixels arranged in the second direction, are arranged in the first direction, and the processor includes a first setting process that sets one of a first exposure time for which the first pixel group is exposed and a second exposure time for which the second pixel group is exposed to be shorter than the other, and a first combining process that generates a first video signal by combining a signal read out from the first pixel group and a signal read out from the second pixel group. The image capturing apparatus is configured to be able to selectively execute a first mode, and a second mode including a second setting process that makes one of the first exposure time and second exposure time in the first frame period and the first exposure time and second exposure time in the second frame period shorter than the other, and a second synthesis process that generates a second video signal by synthesizing signals read out from the first pixel group and the second pixel group in the first frame period and signals read out from the first pixel group and the second pixel group in the second frame period, and the upper limit setting value of the frame rate in the first mode is made higher than the upper limit setting value of the frame rate in the second mode.

[0018] The imaging device of the present disclosure includes a processor; and an imaging element in which column signal lines for reading out signals extend in a first direction, and a first pixel group and a second pixel group including a plurality of imaging pixels arranged in a second direction intersecting the first direction are arranged in the first direction, wherein the first pixel group includes a plurality of first phase difference pixels having a first aperture area, and the second pixel group includes a plurality of second phase difference pixels having a second aperture area smaller than the first aperture area, and the processor is configured to execute an exposure time setting process for making a first exposure time for which the first pixel group is exposed shorter than a second exposure time for which the second pixel group is exposed.

[0019] The processor is configured to perform a synthesis process to generate a video signal by synthesizing the signals read out from the first pixel group and the signals read out from the second pixel group.

[0020] The method for driving an imaging device disclosed herein is a method for driving an imaging device having an imaging element in which a first pixel group including a plurality of phase difference pixels and a plurality of imaging pixels arranged in a second direction intersecting the first direction, and column signal lines for reading out signals extend in a first direction, and a second pixel group including a plurality of imaging pixels arranged in the second direction, are arranged in a first direction, and includes an exposure time setting process for setting one of a first exposure time for which the first pixel group is exposed and a second exposure time for which the second pixel group is exposed to be shorter than the other, and in the setting process, it is determined which of the first exposure time and the second exposure time is to be shorter than the other, based on information of a subject imaged by the imaging element.

[0021] The program disclosed herein is a program for operating an imaging device having an imaging element in which a first pixel group including a plurality of phase difference pixels and a plurality of imaging pixels arranged in a second direction intersecting the first direction, and column signal lines for reading out signals extend in a first direction, and a second pixel group including a plurality of imaging pixels arranged in the second direction, are arranged in the first direction, and the program causes the imaging device to execute an exposure time setting process in which one of a first exposure time in which the first pixel group is exposed and a second exposure time in which the second pixel group is exposed is made shorter than the other, and in the setting process, determines which of the first exposure time and the second exposure time is made shorter than the other based on information about the subject imaged by the imaging element. [Brief explanation of the drawings]

[0022] [Figure 1] FIG. 2 is a schematic perspective view showing an example of the front side of an imaging device. [Figure 2] FIG. 2 is a schematic perspective view showing an example of the rear side of the imaging device. [Figure 3] FIG. 1 is a diagram illustrating an example of the internal configuration of an imaging device. [Figure 4] FIG. 2 is a block diagram illustrating an example of a functional configuration of a processor. [Figure 5] FIG. 2 is a diagram illustrating an example of the configuration of an image sensor. [Figure 6] FIG. 2 is a diagram illustrating an example of a configuration of an imaging pixel. [Figure 7] FIG. 2 is a diagram illustrating an example of a configuration of a phase difference pixel. [Figure 8] FIG. 2 is a diagram illustrating an example of a pixel array of an image sensor. [Figure 9] FIG. 2 is a diagram illustrating focus control and exposure control. [Figure 10] 10 is a diagram showing an example of image capturing timing when the brightness of the AF area is equal to or greater than a first threshold value. FIG. [Figure 11] 10 is a diagram showing an example of image capturing timing when the brightness of the AF area is less than a first threshold value. FIG. [Figure 12] 10 is a flowchart illustrating an example of a flow of exposure control. [Figure 13] FIG. 1 is a diagram illustrating HDR synthesis processing. [Figure 14] 10 is a graph showing an example of a relational expression between the coefficient α and the luminance of a pixel. [Figure 15] 10 is a diagram schematically showing the signal levels of a PG1 signal and a PG2 signal relative to the amount of incident light. FIG. [Figure 16] 10 is a flowchart illustrating an example of exposure control according to the second embodiment. [Figure 17] FIG. 11 is a diagram illustrating an example of imaging timing according to the third embodiment. [Figure 18] 10A and 10B are diagrams illustrating an example in which the signal readout direction of the first pixel group and the second pixel group is changed depending on the position of the subject. [Figure 19] 10A to 10C are diagrams illustrating HDR merging processing including phase processing according to the fourth embodiment. [Figure 20] FIG. 13 is a diagram showing an example of imaging timing in the second mode according to the fifth embodiment. [Figure 21] FIG. 13 is a diagram showing an example of a pixel array of an image sensor according to a sixth embodiment. [Figure 22] FIG. 2 is a diagram illustrating an example of a configuration of a first phase difference pixel. [Figure 23] FIG. 10 is a diagram illustrating an example of a configuration of a second phase difference pixel. DETAILED DESCRIPTION OF THE INVENTION

[0023] An example of an embodiment of the technology of the present disclosure will be described with reference to the accompanying drawings.

[0024] First, the terms used in the following description will be explained.

[0025] In the following explanation, "IC" is an abbreviation for "Integrated Circuit." "CPU" is an abbreviation for "Central Processing Unit." "ROM" is an abbreviation for "Read Only Memory." "RAM" is an abbreviation for "Random Access Memory." "CMOS" is an abbreviation for "Complementary Metal Oxide Semiconductor." "HDR" is an abbreviation for "High Dynamic Range." "AF" is an abbreviation for "Auto Focus." EEPROM is an abbreviation for "Electrically Erasable Programmable Read-Only Memory."

[0026] "FPGA" is an abbreviation for "Field-Programmable Gate Array." "PLD" is an abbreviation for "Programmable Logic Device." "ASIC" is an abbreviation for "Application Specific Integrated Circuit." "JPEG" is an abbreviation for "Joint Photographic Experts Group."

[0027] In the present disclosure, "equal" includes not only being completely equal, but also being substantially equal in the sense of including a generally accepted margin of error in the technical field to which the technology of the present disclosure belongs.

[0028] [First embodiment] The technology of the present disclosure will be described using an interchangeable lens digital camera as an example of a first embodiment of an imaging device. Note that the technology of the present disclosure is not limited to interchangeable lens digital cameras, and can also be applied to digital cameras with an integrated lens.

[0029] (Configuration of imaging device) FIG. 1 shows an example of the front side of an imaging device 10. As shown in FIG. 1, the imaging device 10 is a digital camera with interchangeable lenses. The imaging device 10 is composed of a main body 11 and an imaging lens 12 that is interchangeably attached to the main body 11. The imaging lens 12 is attached to the front surface 11C of the main body 11 via a camera-side mount 11A and a lens-side mount 12A (see FIG. 3). The imaging lens 12 is an example of a lens according to the technology of the present disclosure.

[0030] A dial 13 and a release button 14 are provided on the top surface of the main body 11. The dial 13 is operated when setting the operation mode, etc. The operation modes of the imaging device 10 include, for example, a still image capturing mode, a video capturing mode, and an image display mode. The release button 14 is operated by the user when starting to capture a still image or a video.

[0031] Fig. 2 shows an example of the rear side of the imaging device 10. As shown in Fig. 2, a display 15, command keys 16, and a finder eyepiece 18 for a finder (not shown) are provided on the rear surface 11D of the main body 11. The finder may be an optical viewfinder or an electronic viewfinder. The display 15 displays images based on video signals obtained by imaging, various menu screens, and the like. The command keys 16 accept various commands.

[0032] 3 shows an example of the internal configuration of the imaging device 10. The main body 11 and imaging lens 12 are electrically connected by electrical contacts 11B provided on the camera-side mount 11A coming into contact with electrical contacts 12B provided on the lens-side mount 12A.

[0033] The imaging lens 12 includes an objective lens 30, a focus lens 31, a rear end lens 32, and an aperture 33. The components are arranged along the optical axis LA of the imaging lens 12 in the following order from the objective side: objective lens 30, aperture 33, focus lens 31, and rear end lens 32. The objective lens 30, focus lens 31, and rear end lens 32 constitute an imaging optical system. The type, number, and arrangement order of the lenses that make up the imaging optical system are not limited to the example shown in FIG. 3 .

[0034] The imaging lens 12 also has a lens drive control unit 34. The lens drive control unit 34 is configured with, for example, a CPU, RAM, and ROM. The ROM also includes rewritable EEPROM, flash memory, etc. The lens drive control unit 34 is electrically connected to a processor 40 in the main body 11 via electrical contacts 12B and 11B.

[0035] The lens drive control unit 34 drives the focus lens 31 and the diaphragm 33 based on a control signal transmitted from the processor 40. In order to perform focus adjustment of the imaging lens 12, the lens drive control unit 34 controls the drive of the focus lens 31 based on a control signal for focus adjustment transmitted from the processor 40. The processor 40 performs focus adjustment using a phase difference method.

[0036] The main body 11 is provided with an image sensor 20, a processor 40, an operation unit 42, a memory 45, a display 15, and a shake detection sensor 47. The operations of the image sensor 20, the memory 45, and the display 15 are controlled by the processor 40. The processor 40 is configured with, for example, a CPU, RAM, and ROM. In this case, the processor 40 executes various processes based on a program 45A stored in the memory 45. The processor 40 may be configured with a collection of multiple IC chips. The image sensor 20 is, for example, a CMOS image sensor. The image sensor 20 is an example of an "imaging element" according to the technology of the present disclosure.

[0037] The display 15 displays images based on the video signal generated by the image processing unit 52 (see FIG. 4). The images include still images, videos, and live view images. The live view images are images that are displayed in real time on the display 15 by sequentially outputting image data generated by the image processing unit 52 to the display 15.

[0038] The video signal generated by the image processing unit 52 can be stored as image data in an internal memory (not shown) built into the main body 11 or in a storage medium (e.g., a memory card) that can be attached to or detached from the main body 11.

[0039] The operation unit 42 includes the aforementioned dial 13, release button 14, and command keys 16 (see FIGS. 1 and 2). The processor 40 controls each part in the main body 11 and the lens drive control unit 34 in the imaging lens 12 in response to the operation of the operation unit 42.

[0040] The shake detection sensor 47 detects the amount of shake applied to the imaging device 10. The shake detection sensor 47 is, for example, a five-axis shake detection sensor that detects shake in the roll direction, yaw direction, pitch direction, X direction, and Y direction. The X direction and Y direction are directions perpendicular to the optical axis LA.

[0041] The shake detection sensor 47 is composed of, for example, a gyro sensor (not shown) that detects rotational shake and angular shake, and an acceleration sensor (not shown) that detects translational shake. The shake detection sensor 47 outputs a detection signal of the detected shake to the processor 40. The shake detection sensor 47 may be provided inside the imaging lens 12. In this case, the processor 40 may obtain a detected value of the amount of shake from the shake detection sensor 47 inside the imaging lens 12 via the electrical contacts 12B and 11B.

[0042] (Processor configuration) Fig. 4 shows an example of the functional configuration of the processor 40. The processor 40 realizes various functional units by executing processing in accordance with a program 45A stored in the memory 45. As shown in Fig. 4, for example, the processor 40 realizes a main control unit 50, an imaging control unit 51, an image processing unit 52, a focus detection unit 53, a brightness detection unit 54, and a correction amount calculation unit 55.

[0043] The main control unit 50 performs overall control of the operation of the imaging device 10 based on instruction signals input from the operation unit 42. The imaging control unit 51 controls the imaging sensor 20 to execute imaging processing that causes the imaging sensor 20 to perform an imaging operation. The imaging control unit 51 drives the imaging sensor 20 in a still image imaging mode or a video imaging mode. The imaging control unit 51 also realizes an HDR (High Dynamic Range) function that generates HDR images with an expanded dynamic range in the still image imaging mode or the video imaging mode.

[0044] The user can select between a still image capture mode and a video capture mode using the operation unit 42. The user can also set exposure values ​​including the shutter speed and aperture value by operating the operation unit 42. Furthermore, the user can select whether to turn the HDR function on or off by operating the operation unit 42.

[0045] Furthermore, the user can select an automatic focus detection (hereinafter referred to as AF) mode by operating the operation unit 42. When the AF mode is selected, the main control unit 50 controls the position of the focus lens 31 based on the focus detection result detected by the focus detection unit 53.

[0046] The focus detection unit 53 performs focus detection based on signals output from the phase difference pixels ZL and ZR (see FIGS. 7 and 8) among the video signals output from the imaging sensor 20. Note that focus detection refers to detecting the position of the focus lens 31 where the subject is in focus.

[0047] The focus detection unit 53 also performs focus detection based on signals output from phase difference pixels ZL, ZR (see FIG. 7) included in an AF area 21A (see FIG. 9) set within the imaging region 21 of the image sensor 20. For example, the position and size of the AF area 21A within the imaging region 21 can be set by the user operating the operation unit 42. The position and size of the AF area 21A may also be set based on the results of subject detection (e.g., face detection) performed by the main control unit 50 using a video signal. The AF area 21A is an example of a "region of interest" according to the technology of the present disclosure. When capturing images with a fixed focus, the AF area 21A may be a different subject region determined by the user or the imaging device, different from the focal region.

[0048] The brightness detection unit detects the brightness of the AF area 21A (an example of an area of ​​interest) based on the video signal output from the imaging sensor 20. The main control unit 50 performs processing to set the exposure time based on the brightness of the AF area 21A detected by the brightness detection unit .

[0049] Correction amount calculation unit 55 calculates the correction amount for performing electronic shake correction (i.e., electronic image stabilization processing) based on the detected value of the amount of shake output from shake detection sensor 47. Correction amount calculation unit 55 outputs the calculated correction amount to image processing unit 52.

[0050] The image processing unit 52 performs various image processing on the video signal to generate image data in a predetermined file format (for example, JPEG format, etc.). The image data output from the image processing unit 52 is recorded, for example, in the memory 45. The video signal constituting the image data from the image processing unit 52 is output to the display 15, and the image is displayed on the display 15.

[0051] The image processing unit 52 also includes an HDR synthesis unit 56 and a shake correction unit 57. The HDR synthesis unit 56 operates when the HDR function is turned on. The shake correction unit 57 operates when the video shooting mode is selected.

[0052] When the HDR function is turned on, the HDR synthesis unit 56 synthesizes a pair of video signals output from the image sensor 20 with different exposure times to generate an HDR image.

[0053] In video capture mode, the shake correction unit 57 performs electronic stabilization processing on the video signal based on the correction amount input from the correction amount calculation unit 55. For example, the electronic stabilization processing is performed by changing the cropping area for each frame of the video signal based on the correction amount. When the HDR function is turned on, the shake correction unit 57 performs electronic stabilization processing on the video signal (i.e., the HDR image) generated by the HDR synthesis unit 56. This simplifies the electronic stabilization processing and enables the electronic stabilization processing in the processor 40 to be performed at a higher speed. Furthermore, it is preferable that the shake correction unit 57 performs electronic stabilization processing on an HDR image that has been subjected to demosaic processing.

[0054] (Image sensor configuration) Fig. 5 shows an example of the configuration of the image sensor 20. The image sensor 20 shown in Fig. 5 is a CMOS image sensor. The image sensor 20 has an imaging area 21, a vertical scanning circuit 22, a line memory 23, a horizontal scanning circuit 24, and an output amplifier 25.

[0055] A plurality of pixels 26 are arranged in a two-dimensional matrix along the X and Y directions in the imaging region 21. In addition, a plurality of row selection lines L1 and a plurality of row reset lines L2 are wired in the imaging region 21 along the X direction, and a plurality of column signal lines L3 are wired along the Y direction.

[0056] The column signal line L3 extends in the Y direction. The row select line L1 and the row reset line L2 extend in the X direction that intersects with the Y direction. The Y direction is an example of a "first direction" according to the technology of the present disclosure. The X direction is an example of a "second direction" according to the technology of the present disclosure.

[0057] The pixels 26 are connected to a row selection line L1, a row reset line L2, and a column signal line L3. Hereinafter, a plurality of pixels 26 arranged in the X direction may be simply referred to as a "row." As will be described in more detail later, some of the plurality of pixels 26 are phase difference pixels for performing focus adjustment.

[0058] Each pixel 26 includes a photodiode D1, an amplifier transistor M1, a pixel selection transistor M2, and a reset transistor M3. The photodiode D1 photoelectrically converts incident light to generate a signal charge corresponding to the amount of incident light and accumulates the generated signal charge. The amplifier transistor M1 generates a voltage (hereinafter referred to as a pixel signal S) corresponding to the amount of signal charge accumulated in the photodiode D1.

[0059] The pixel selection transistor M2 is controlled by the vertical scanning circuit 22 via the row selection line L1 and outputs the pixel signal S generated by the amplifier transistor M1 to the column signal line L3. The reset transistor M3 is controlled by the vertical scanning circuit 22 via the row reset line L2 and discards the signal charge accumulated in the photodiode D1 to the power supply line. Hereinafter, discarding the signal charge from the photodiode D1 is referred to as resetting the pixel 26.

[0060] The vertical scanning circuit 22 generates a row selection signal SEL and a reset signal RST based on a vertical synchronization signal input from the imaging control unit 51. During a signal readout operation, the vertical scanning circuit 22 applies the row selection signal SEL to a row selection line L1, thereby causing the pixel 26 connected to the row selection line L1 to output a pixel signal S to the column signal line L3.

[0061] Furthermore, during the reset operation, the vertical scanning circuit 22 applies a reset signal RST to the row reset line L2, thereby resetting the pixels 26 connected to the row reset line L2.

[0062] The line memory 23 stores pixel signals S output from one row of pixels 26. The line memory 23 is composed of capacitors and the like. The line memory 23 is connected to a horizontal output line 24A via a transistor 29 serving as a switch. An output amplifier 25 is connected to one end of the horizontal output line 24A. The horizontal scanning circuit 24 performs horizontal scanning to sequentially select the transistors 29, thereby sequentially outputting one row of pixel signals S stored in the line memory 23 to the horizontal output line 24A. The pixel signals S output to the horizontal output line 24A are output as video signals to an external image processing unit 52 via the output amplifier 25.

[0063] The operations of the vertical scanning circuit 22, the line memory 23, and the horizontal scanning circuit 24 are controlled by an imaging control unit 51 (see FIG. 4). The imaging control unit 51 controls the vertical scanning circuit 22 to select the row selection line L1 row by row and output the pixel signal S. The imaging control unit 51 also controls the vertical scanning circuit 22 to reset the pixels 26.

[0064] In the moving image capturing mode, the capturing control unit 51 drives the imaging sensor 20 at a constant frame period T (see FIGS. 10 and 11). The capturing control unit 51 causes the imaging sensor 20 to repeatedly read out and reset the video signal at the frame period T.

[0065] The configuration of the image sensor 20 is not limited to the configuration shown in Fig. 5. For example, the image sensor 20 may be provided with an A / D converter.

[0066] (Pixel configuration) The multiple pixels 26 arranged in the imaging region 21 include an imaging pixel N for imaging and phase difference pixels ZL and ZR. Fig. 6 shows an example of the configuration of the imaging pixel N. Fig. 7 shows an example of the configuration of the phase difference pixels ZL and ZR. Each of the phase difference pixels ZL and ZR receives one of the light beams split in the X direction around the principal ray.

[0067] 6, the imaging pixel N includes a photodiode D1 as a photoelectric conversion element, a color filter CF, and a microlens ML. The color filter CF is disposed between the photodiode D1 and the microlens ML.

[0068] The color filter CF is a filter that transmits light of any one of the colors R (red), G (green), and B (blue). The microlens ML focuses the light beam LF incident from the exit pupil EP of the imaging lens 12 onto approximately the center of the photodiode D1 via the color filter CF.

[0069] 7, each of the phase detection pixels ZL and ZR includes a photodiode D1, a light-shielding layer SF, and a microlens ML. Similar to the imaging pixel N, the microlens ML focuses the light beam LF incident from the exit pupil EP of the imaging lens 12 onto approximately the center of the photodiode D1.

[0070] The light-shielding layer SF is formed of a metal film or the like and is disposed between the photodiode D1 and the microlens ML. The light-shielding layer SF blocks a part of the light flux LF that is incident on the photodiode D1 via the microlens ML.

[0071] In the phase detection pixel ZL, the light-shielding layer SF shields the negative side (first side) in the X direction with respect to the center of the photodiode D1. That is, in the phase detection pixel ZL, the light-shielding layer SF allows the light flux LF from the exit pupil EP1 on the first side to be incident on the photodiode D1, and shields the light flux LF from the exit pupil EP2 on the positive side (second side) in the X direction.

[0072] In the phase detection pixel ZR, the light-shielding layer SF shields the positive side (second side) in the X direction with respect to the center of the photodiode D1. That is, in the phase detection pixel ZR, the light-shielding layer SF allows the light flux LF from the exit pupil EP2 on the second side to be incident on the photodiode D1, and shields the light flux LF from the exit pupil EP1 on the negative side (first side) in the X direction.

[0073] (pixel array) FIG. 8 shows an example of a pixel arrangement of the image sensor 20. The color arrangement of the color filters CF shown in FIG. 8 is a so-called Bayer arrangement. In a Bayer arrangement, of the four 2×2 pixels, G color filters CF are arranged in two diagonal pixels, and R and B color filters CF are arranged in the remaining two pixels. In FIG. 8, "R" represents an imaging pixel N provided with an R color filter CF. "G" represents an imaging pixel N provided with a G color filter CF. "B" represents an imaging pixel N provided with a B color filter CF. Note that the color arrangement of the color filters CF is not limited to the Bayer arrangement, and other color arrangements may be used.

[0074] The phase difference pixels ZL, ZR are arranged in the imaging region 21 by replacing some of the imaging pixels N in the Bayer array. In this embodiment, the phase difference pixels ZL, ZR are arranged so as to replace some of the imaging pixels N provided with B color filters CF. The phase difference pixels ZL, ZR are arranged in the X direction. In this embodiment, one of the phase difference pixels ZL, ZR is arranged every two pixels (i.e., every other pixel) in the X direction. That is, one imaging pixel N is arranged between the phase difference pixel ZL and the phase difference pixel ZR.

[0075] The phase difference pixels ZL, ZR are arranged every 16 pixels in the Y direction. A plurality of imaging pixels N (G and B) are arranged between the phase difference pixels ZL arranged in the Y direction. A plurality of imaging pixels N (G and B) are arranged between the phase difference pixels ZR arranged in the Y direction.

[0076] The arrangement pattern of the phase difference pixels ZL and ZR is not limited to the example shown in Fig. 8. For example, the phase difference pixels ZL and ZR may be arranged adjacent to each other in the X direction.

[0077] The plurality of pixels 26 (see FIG. 5) included in the imaging region 21 are divided into a first pixel group PG1 and a second pixel group PG2. The first pixel group PG1 includes a plurality of phase difference pixels ZL, ZR and a plurality of imaging pixels N arranged in the X direction. The second pixel group PG2 includes a plurality of imaging pixels N arranged in the X direction. The first pixel group PG1 and the second pixel group PG2 are arranged alternately in the Y direction.

[0078] In this embodiment, when the row address is n (here, n is a natural number including 0), the first pixel group PG1 includes a row with a row address of "4n" and a row with a row address of "4n+1." The second pixel group PG2 includes a row with a row address of "4n+2" and a row with a row address of "4n+3." The phase difference pixels ZL and ZR are present in the row with a row address of "16n." That is, the phase difference pixels ZL and ZR are included only in the first pixel group PG1.

[0079] 9 illustrates focus control and exposure control by the main controller 50. As shown in FIG. 9, the focus detector 53 generates focus information by performing focus detection based on signals (hereinafter referred to as phase difference pixel signals) output from phase difference pixels ZL and ZR included in the AF area 21A set in the imaging region 21. The focus information is information that indicates the position of the focus lens 31 at which the subject is in focus. Specifically, the focus detector 53 generates focus information based on the phase difference between the phase difference pixel signal output from the phase difference pixel ZL and the phase difference pixel signal output from the phase difference pixel ZR, and outputs the generated focus information to the main controller 50.

[0080] The brightness detection unit 54 generates brightness information representing the brightness of the AF area 21A based on signals output from the imaging pixels N included in the AF area 21A (hereinafter referred to as imaging pixel signals). For example, the brightness detection unit 54 calculates a brightness value for each pixel from the imaging pixel signals and calculates an average brightness value in the AF area 21A. The brightness detection unit 54 outputs the calculated average brightness value to the main control unit 50 as brightness information. Note that the brightness information is an example of "subject information" according to the technology of the present disclosure.

[0081] The main controller 50 controls the position of the focus lens 31 via the lens drive controller 34 based on focus information input from the focus detector 53. The main controller 50 also controls the exposure times of the first pixel group PG1 and the second pixel group PG2 via the imaging controller 51 based on brightness information input from the brightness detector 54. Specifically, the main controller 50 sets one of a first exposure time E1 during which the first pixel group PG1 is exposed and a second exposure time E2 during which the second pixel group PG2 is exposed to light shorter than the other based on the brightness information.

[0082] (imaging timing) 10 and 11 show an example of the imaging timing of the image sensor 20 in the video imaging mode when the HDR function is turned on.

[0083] The frame period T of moving image capture is defined by a vertical synchronization signal VD supplied from the imaging control unit 51 to the vertical scanning circuit 22. Based on the vertical synchronization signal VD, the vertical scanning circuit 22 supplies a row selection signal SEL and a reset signal RST to the first pixel group PG1 and the second pixel group PG2.

[0084] Specifically, the vertical scanning circuit 22 supplies a row selection signal SEL to the first pixel group PG1 and the second pixel group PG2 in synchronization with a vertical synchronization signal VD. In this embodiment, the vertical scanning circuit 22 selects a row selection line L1 (see FIG. 5) while changing the row address in order from 0 to 1, 2, and so on, and supplies the row selection signal SEL to the selected row selection line L1.

[0085] The vertical scanning circuit 22 supplies the reset signal RST to the first pixel group PG1 and the second pixel group PG2 according to their respective exposure times. The first exposure time E1 is the time (i.e., the charge accumulation time) from when the reset signal RST is input until the row selection signal SEL is input in the first pixel group PG1. The second exposure time E2 is the time from when the reset signal RST is input until the row selection signal SEL is input in the second pixel group PG2. The exposure period is shifted for each row. That is, the imaging sensor 20 performs exposure with an electronic shutter of the focal plane method.

[0086] Note that the periods hatched in FIGS. 10 and 11 are the light shielding periods. During the light shielding period, the reset signal RST may be periodically supplied to the first pixel group PG1 and the second pixel group PG2. Also, during the light shielding period, the reset signal RST may be constantly supplied to the first pixel group PG1 and the second pixel group PG2.

[0087] FIG. 10 shows an example of imaging timing when the brightness of the AF area 21A is equal to or greater than an arbitrary first threshold value. In this case, the main control unit 50 controls the reset timing so that E1 < E2. FIG. 11 shows an example of imaging timing when the brightness of the AF area 21A is less than the first threshold value. In this case, the main control unit 50 controls the reset timing so that E2 < E1.

[0088] (Exposure Control) Next, an example of the flow of exposure control in the video imaging mode and when the HDR function is turned on will be described while referring to the flowchart shown in FIG. 12. First, the main control unit 50 determines whether there is an instruction to start video imaging by an operation of the operation unit 42 (step S10).

[0089] When the main control unit 50 determines that an instruction to start video imaging has been given (step S10: YES), it controls the imaging control unit 51 to cause the imaging sensor 20 to perform an imaging operation at a fixed frame period T, and acquires the brightness information of the AF area 21A from the brightness detection unit 54 (step S11). Note that the main control unit 50 acquires the brightness information from the brightness detection unit 54 for each frame period T.

[0090] Based on the acquired brightness information, the main control unit 50 determines whether the brightness of the AF area 21A is equal to or greater than the first threshold value (step S12). When the main control unit 50 determines that the brightness of the AF area 21A is equal to or greater than the first threshold value (step S12: YES), it controls the reset timing so that E1 < E2 in the next frame period T (step S13). On the other hand, when the main control unit 50 determines that the brightness of the AF area 21A is less than the first threshold value (step S12: NO), it controls the reset timing so that E2 < E1 in the next frame period T (step S14).

[0091] Next, the main control unit 50 determines whether an instruction to end video imaging has been given by an operation on the operation unit 42 (step S15). When the main control unit 50 determines that there is no instruction to end video imaging (step S15: NO), it transfers the process to step S11 and acquires the brightness information again. The processes from step S11 to step S15 are repeatedly executed for each frame period T until it is determined in step S15 that an instruction to end video imaging has been given. Then, when the main control unit 50 determines that an instruction to end video imaging has been given (step S15: YES), it ends the process.

[0092] As described above, when the brightness of the AF area 21A is equal to or greater than the first threshold, the main controller 50 sets the first exposure time E1, during which the first pixel group PG1 including the phase difference pixels ZL and ZR is exposed, to be shorter than the second exposure time E2, during which the second pixel group PG2 not including the phase difference pixels ZL and ZR is exposed. In this way, when the brightness of the AF area 21A is equal to or greater than the first threshold, the main controller 50 sets the first exposure time E1 to be shorter than the second exposure time E2 so as to prevent overexposure due to saturation of the signals of the phase difference pixels ZL and ZR, thereby obtaining more accurate phase difference information.

[0093] On the other hand, when the brightness of the AF area 21A is less than the first threshold, the main controller 50 sets the second exposure time E2, during which the second pixel group PG2 that does not include the phase difference pixels ZL, ZR is exposed, to be shorter than the first exposure time E1, during which the first pixel group PG1 that includes the phase difference pixels ZL, ZR is exposed. In this way, when the brightness of the AF area 21A is less than the first threshold, the main controller 50 sets the first exposure time E1 to be longer than the second exposure time E2 so as to prevent crushed blacks caused by small signals from the phase difference pixels ZL, ZR, thereby obtaining more accurate phase difference information.

[0094] Therefore, when the first exposure time E1 and the second exposure time E2 are made different to expand the dynamic range, by performing the above-described exposure control, it is possible to obtain accurate phase difference information regardless of the brightness of the AF area 21 A. This makes it possible to perform accurate phase difference focus adjustment even when expanding the dynamic range.

[0095] (HDR compositing processing) 13 schematically shows the HDR synthesis process performed by the HDR synthesis unit 56. The HDR synthesis unit 56 generates an HDR image by adding together signals output from a first pixel group PG1 and a second pixel group PG2 that are adjacent in the Y direction.

[0096] Specifically, the HDR synthesis unit 56 adds a signal output from the first pixel group PG1 (hereinafter referred to as the PG1 signal) and a signal output from the second pixel group PG2 (hereinafter referred to as the PG2 signal) for each pair of corresponding pixels. Note that a pair of corresponding pixels refers to a pair of pixels located at the same address in the X direction and at addresses that differ by two pixels in the Y direction.

[0097] For example, in HDR compositing, only imaging pixel signals are used without using phase difference pixel signals. Therefore, for the phase difference pixels ZL and ZR, signals obtained by performing pixel interpolation using imaging pixel signals of imaging pixels N present around the phase difference pixels ZL and ZR in the first pixel group PG1 are used for HDR compositing.

[0098] Note that HDR combining may be performed using a phase difference pixel signal in addition to the imaging pixel signal. Because the phase difference pixels ZL and ZR receive a smaller amount of light than the imaging pixel N, the HDR combining unit 56 may perform gain correction on the phase difference pixel signal and use it for HDR combining.

[0099] The HDR synthesis unit 56 may synthesize the PG1 signal and the PG2 signal in the form of analog signals, or may synthesize them after converting them into digital signals by A / D conversion. Also, the HDR synthesis unit 56 may synthesize the PG1 signal and the PG2 signal in the form of RAW data, or may synthesize them after performing demosaic processing.

[0100] In addition, in this embodiment, the HDR synthesizing unit 56 adds the value obtained by multiplying the PG1 signal by a coefficient α to the value obtained by multiplying the PG2 signal by a coefficient 1-α, where the coefficient α is a value that satisfies 0≦α≦1.

[0101] Fig. 14 is a graph showing an example of the relationship between the coefficient α and pixel luminance. Based on the relationship shown in Fig. 14, the HDR synthesis unit 56 calculates the coefficient α for each pair of pixels to be added, and adds the PG1 signal and the PG2 signal.

[0102] When E1 < E2, the HDR synthesis unit 56 determines the coefficient α using the relational expression F1 in which the coefficient α increases as the luminance increases. On the other hand, when E2 < E1, the HDR synthesis unit 56 determines the coefficient α using the relational expression F2 in which the coefficient α decreases as the luminance increases. In either case, the higher the luminance, the higher the addition ratio of the signal on the short-time exposure side. The luminance of the pixel is based on, for example, the luminance information (R, G, B signal values) of the pixel with the shorter exposure time.

[0103] FIG. 15 schematically shows the signal levels of the PG1 signal and the PG2 signal with respect to the incident light amount. When E1 < E2, the PG1 signal exhibits the effect of enhancing the sensitivity for regions with a large incident light amount, and the PG2 signal exhibits the effect of enhancing the sensitivity for regions with a small incident light amount. On the other hand, when E2 < E1, the PG1 signal exhibits the effect of enhancing the sensitivity for regions with a small incident light amount, and the PG2 signal exhibits the effect of enhancing the sensitivity for regions with a large incident light amount. As a result, the dynamic range of the HDR image generated by adding the PG1 signal and the PG2 signal is expanded.

[0104] As described above, according to the technology of the present disclosure, it is possible to appropriately determine whether to use a pixel group including phase difference pixels (the first pixel group PG1 in the present embodiment) for short-time exposure or long-time exposure.

[0105] [Second Embodiment] Next, the second embodiment will be described. In the first embodiment, in the video imaging mode and when the HDR function is turned on, the main control unit 50 performs exposure control based on the "brightness of the AF area 21A" as information about the subject. In contrast, in the second embodiment, the main control unit 50 performs exposure control based on the "moving speed of the subject" as information about the subject.

[0106] In this embodiment, the main control unit 50 detects the moving speed of a subject by obtaining the movement vector of the subject between frames based on the video signal output from the imaging area 21 of the imaging sensor 20, for example, every frame period T. Note that the main control unit 50 may also detect the moving speed of the subject based on the video signal output from the AF area 21A.

[0107] FIG. 16 shows an example of exposure control according to the second embodiment. First, the main control unit 50 determines whether there is an instruction to start video imaging due to an operation of the operation unit 42 (step S20).

[0108] When the main control unit 50 determines that there is an instruction to start video imaging (step S20: YES), it controls the imaging control unit 51 to cause the imaging sensor 20 to perform an imaging operation at a constant frame period T, and detects the moving speed of the subject based on the video signal output from the imaging sensor 20 (step S21).

[0109] The main control unit 50 determines whether the moving speed is equal to or higher than an arbitrary second threshold value based on the detected moving speed of the subject (step S22). When the main control unit 50 determines that the moving speed is equal to or higher than the second threshold value (step S22: YES), it controls the reset timing so that E1 < E2 in the next frame period T (step S23). On the other hand, when the main control unit 50 determines that the moving speed is less than the second threshold value (step S22: NO), it controls the reset timing so that E2 < E1 in the next frame period T (step S24).

[0110] Next, the main control unit 50 determines whether an instruction to end video capture has been issued by operating the operation unit 42 (step S25). If the main control unit 50 determines that an instruction to end video capture has not been issued (step S25: NO), the process returns to step S21, and the moving speed of the subject is detected again. The processes from step S21 to step S25 are repeatedly executed for each frame period T until it is determined in step S25 that an instruction to end video capture has been issued. Then, if the main control unit 50 determines that an instruction to end video capture has been issued (step S25: YES), the process ends.

[0111] As described above, when the moving speed of the subject is equal to or greater than the second threshold, the main control unit 50 sets the first exposure time E1, during which the first pixel group PG1 including the phase difference pixels ZL, ZR is exposed, to be shorter than the second exposure time E2, during which the second pixel group PG2 not including the phase difference pixels ZL, ZR is exposed. On the other hand, when the moving speed of the subject is less than the second threshold, the main control unit 50 sets the second exposure time E2, during which the second pixel group PG2 not including the phase difference pixels ZL, ZR is exposed, to be shorter than the first exposure time E1, during which the first pixel group PG1 including the phase difference pixels ZL, ZR is exposed.

[0112] In this manner, in this embodiment, when the moving speed of the subject is fast, the exposure time is shortened to acquire phase difference information, whereas when the moving speed of the subject is slow, the exposure time is lengthened to acquire phase difference information in a sufficiently exposed state.

[0113] [Third embodiment] Next, a third embodiment will be described. In the first embodiment, as shown in Figures 10 and 11, when reading out video signals from the image sensor 20, the main control unit 50 sequentially selects row addresses 0, 1, 2, ... and reads out the signals. That is, in the first embodiment, the main control unit 50 reads out signals alternately from the first pixel group PG1 and the second pixel group PG2. In contrast, in the third embodiment, the main control unit 50 reads out signals from one of the first pixel group PG1 and the second pixel group PG2, and then reads out signals from one of the first pixel group PG1 and the second pixel group PG2.

[0114] 17 shows an example of imaging timing according to the third embodiment. As shown in FIG. 17, in this embodiment, signal readout is performed by first supplying a row selection signal SEL to the first pixel group PG1 having row addresses of "4n" and "4n+1." Thereafter, signal readout is performed by supplying a row selection signal SEL to the second pixel group PG2 having row addresses of "4n+2" and "4n+3." Note that signal readout from the first pixel group PG1 and signal readout from the second pixel group PG2 are performed consecutively within the same frame period T.

[0115] In this embodiment, as with signal readout, the resetting is performed by resetting the first pixel group PG1 and then resetting the second pixel group PG2.

[0116] 17, the main control unit 50 may read out signals from the second pixel group PG2 and then read out signals from the first pixel group PG1. However, because the phase difference pixels ZL and ZR are only included in the first pixel group PG1, it is preferable to read out signals from the first pixel group PG1 before reading out signals from the second pixel group PG2. By reading out signals from the first pixel group PG1 first, phase difference information can be acquired early after the start of the frame period.

[0117] 17, the main control unit 50 reads out signals from both the first pixel group PG1 and the second pixel group PG2 in the same direction in the Y direction. Specifically, if the direction in which the row address increases is defined as the positive direction and the direction in which the row address decreases is defined as the negative direction, in the example shown in FIG. 17, the main control unit 50 reads out signals from both the first pixel group PG1 and the second pixel group PG2 in the positive direction. The positive direction in the Y direction is an example of "one direction in the first direction" according to the technology of the present disclosure. The negative direction in the Y direction is an example of "another direction in the first direction" according to the technology of the present disclosure.

[0118] The signal readout direction may be reversed between the first pixel group PG1 and the second pixel group PG2. Also, the signal readout direction may be changed depending on the position of the subject within the imaging region 21.

[0119] 18 illustrates an example in which the signal readout direction of the first pixel group PG1 and the second pixel group PG2 is changed depending on the position of the subject. In this example, the main controller 50 performs subject detection (e.g., face detection) based on the video signal output from the imaging sensor 20, and detects the position of the subject within the imaging region 21. The main controller 50 then determines whether the subject is located on the positive or negative side in the Y direction relative to a virtual line C located at the center of the imaging region 21 in the Y direction.

[0120] When the subject is located on the positive side of the virtual line C, the main control unit 50 reads out signals from the first pixel group PG1 from the negative side in the Y direction to the positive side, and then reads out signals from the second pixel group PG2 from the positive side in the Y direction to the negative side. On the other hand, when the subject is located on the negative side of the virtual line C, the main control unit 50 reads out signals from the first pixel group PG1 from the positive side in the Y direction to the negative side, and then reads out signals from the second pixel group PG2 from the negative side in the Y direction to the positive side.

[0121] In this way, by changing the direction of signal readout from the first pixel group PG1 and the second pixel group PG2 depending on the position of the subject, the time lag between the PG1 signal and the PG2 signal readout from near the subject is reduced, thereby improving the image quality of the subject in the HDR image.

[0122] 18, signals from the first pixel group PG1 are read out before signals from the second pixel group PG2, but signals from the second pixel group PG2 may also be read out before signals from the first pixel group PG1. Furthermore, when the subject intersects with virtual line C, the position of the subject may be determined based on whether the center of gravity of the subject is located in an area on the positive side or the negative side of virtual line C. Furthermore, when the subject intersects with virtual line C, the position of the subject may be determined based on whether the area of ​​the subject is larger on the positive side or the negative side of virtual line C.

[0123] [Fourth embodiment] Next, a fourth embodiment will be described. In the first embodiment, the center of gravity positions of the first pixel group PG1 and the second pixel group PG2 are shifted by two pixels in the Y direction. That is, the PG1 signal read from the first pixel group PG1 and the PG2 signal read from the second pixel group PG2 are shifted in phase in the Y direction. In the fourth embodiment, the image processing unit 52 performs phase processing to reduce the phase shift between the PG1 signal and the PG2 signal, and then combines the PG1 signal and the PG2 signal.

[0124] 19 schematically illustrates HDR blending processing including phase processing. In this embodiment, the image processing unit 52 has a low-pass filter processing unit 58. The low-pass filter processing unit 58 performs low-pass filtering on an image generated by a PG1 signal (hereinafter referred to as a PG1 image) and an image generated by a PG2 signal (hereinafter referred to as a PG2 image). The PG1 image and the PG2 image are, for example, images that have been subjected to demosaic processing.

[0125] In this embodiment, the HDR synthesis unit 56 generates an HDR image by synthesizing the PG1 image and the PG2 image that has been subjected to low-pass filtering. The PG2 image is an image that has been blurred by the low-pass filtering, so that the phase shift with respect to the PG1 image is reduced.

[0126] In this embodiment, a low-pass filter process is performed on the PG2 image. This is because the PG2 image is not the "main image" regardless of whether the subject is bright or dark. For example, as shown in FIG. 12, when the brightness of the AF area 21A is equal to or greater than the first threshold value, E1 < E2, so the second pixel group PG2 is overexposed. Conversely, when the brightness of the AF area 21A is less than the first threshold value, E2 < E1, so the second pixel group PG2 is underexposed. Thus, the second pixel group PG2 is overexposed when the subject is bright and underexposed when the subject is dark. Therefore, in any case, the PG2 image read from the second pixel group PG2 is not the main image, and the PG1 image is the main image.

[0127] As described above, in this embodiment, the non-main image between the PG1 image and the PG2 image is subjected to a low-pass filter process and then a synthesis process is performed, so that it is possible to reduce the phase shift while suppressing the deterioration of the image quality.

[0128] Note that it is also possible to reduce the phase shift between the PG1 signal and the PG2 signal by performing a pixel compensation process on the PG2 image instead of the low-pass filter process. The pixel signal at the pixel position corresponding to the PG1 image may be calculated by performing a pixel compensation process on the PG2 image. Thus, the phase process is a concept including the low-pass filter process and the pixel compensation process.

[0129] [5th Embodiment] Next, the 5th embodiment will be described. As described above, in the 1st embodiment, the main control unit 50 executes the 1st mode including the 1st setting process of making one of the 1st exposure time E1 and the 2nd exposure time E2 shorter than the other, and the 1st synthesis process of generating the HDR image (the 1st video signal) by synthesizing the PG1 signal and the PG2 signal.

[0130] In the fifth embodiment, in addition to the first mode, the main control unit 50 can also execute a second mode in which the exposure time is changed for each frame period. The second mode includes a second setting process and a second combining process. In the second setting process, the main control unit 50 sets one of the first exposure time E1 and the second exposure time E2 in the first frame period and the first exposure time E1 and the second exposure time E2 in the second frame period following the first frame period to be shorter than the other. In the second combining process, the main control unit 50 generates an HDR image (second video signal) by combining signals read out from the first pixel group PG1 and the second pixel group PG2 in the first frame period with signals read out from the first pixel group PG1 and the second pixel group PG2 in the second frame period.

[0131] 20 shows an example of imaging timing in the second mode. In the second mode, the first exposure time E1 and the second exposure time E2 are equal in the first frame period and the second frame period, respectively. That is, in the second mode, the main control unit 50 does not perform processing to set the exposure time based on brightness information of the AF area 21A.

[0132] Furthermore, the first exposure time E1 and the second exposure time E2 are different between the first frame period and the second frame period. In this embodiment, the first exposure time E1 and the second exposure time E2 in the first frame period are shorter than the first exposure time E1 and the second exposure time E2 in the second frame period. The first frame period and the second frame period are alternately and repeatedly executed for each frame period T.

[0133] In the second mode, an HDR image is generated by combining two frames of the PG1 signal and the PG2 signal, resulting in a high resolution HDR image but a low frame rate. In contrast, in the first mode, an HDR image is generated by combining the PG1 signal and the PG2 signal for each frame, resulting in a high frame rate but a low resolution HDR image. Therefore, it is preferable that the main control unit 50 set the upper limit of the frame rate in the first mode higher than the upper limit of the frame rate in the second mode.

[0134] Furthermore, it is preferable that the main control unit 50 allows the user to select between the first mode and the second mode by operating the operation unit 42. This allows the user to select an appropriate mode depending on whether they place more importance on resolution or frame rate.

[0135] [Sixth embodiment] Next, a sixth embodiment will be described. In the first embodiment, as shown in Fig. 8, of the first pixel group PG1 and the second pixel group PG2, only the first pixel group PG1 includes phase difference pixels ZL and ZR. In the sixth embodiment, the first pixel group PG1 includes first phase difference pixels Z1L and Z1R, and the second pixel group PG2 includes second phase difference pixels Z2L and Z2R.

[0136] 21 shows an example of a pixel arrangement of the image sensor 20 according to the sixth embodiment. The first phase difference pixels Z1L and Z1R and the second phase difference pixels Z2L and Z2R are arranged alternately every 18 pixels in the Y direction. That is, the first phase difference pixels Z1L and Z1R are included in a first pixel group PG1, and the second phase difference pixels Z2L and Z2R are included in a second pixel group PG2.

[0137] 22 shows an example of the configuration of the first phase difference pixels Z1L and Z1R. The first phase difference pixels Z1L and Z1R have the same configuration as the phase difference pixels ZL and ZR of the first embodiment (see FIG. 7). Each of the first phase difference pixels Z1L and Z1R has an aperture area (hereinafter referred to as a first aperture area) that is approximately 50% of the light receiving area of ​​the photodiode D1 because approximately 50% of the light receiving area of ​​the photodiode D1 is shielded by a light-shielding layer SF.

[0138] 23 shows an example of the configuration of the second phase difference pixels Z2L and Z2R. In the second phase difference pixels Z2L and Z2R, for example, 50% or more of the area of ​​the light receiving surface of the photodiode D1 is shielded by a light-shielding layer SF. That is, the second phase difference pixels Z2L and Z2R have a second aperture area smaller than the first aperture area of ​​the first phase difference pixels Z1L and Z1R.

[0139] In this embodiment, the main controller 50 sets the first exposure time E1, during which the first pixel group PG1 is exposed, to be shorter than the second exposure time E2, during which the second pixel group PG2 is exposed, regardless of the brightness of the AF area 21A. That is, the main controller 50 sets the exposure time of the first phase difference pixels Z1L and Z1R to be shorter than the exposure time of the second phase difference pixels Z2L and Z2R.

[0140] As described above, in the present embodiment, in the second pixel group PG2 for long-time exposure, the phase difference pixel signals are acquired by the second phase difference pixels Z2L and Z2R having small aperture areas, so the phase difference pixel signals are less likely to suffer from whiteout. Also, in the first pixel group PG1 for short-time exposure, the phase difference pixel signals are acquired by the first phase difference pixels Z1L and Z1R having large aperture areas, so the phase difference pixel signals are less likely to suffer from blackout. Therefore, in the present embodiment, appropriate phase difference information can be obtained that does not suffer from whiteout or blackout.

[0141] Furthermore, in this embodiment, the phase difference pixel signal is unlikely to reach a high level that would cause whiteout or a low level that would cause blackout, so that image quality is improved when an HDR image is generated using the phase difference pixel signal in addition to the imaging pixel signal.

[0142] [Variations] In each of the above embodiments, HDR compositing processing for compositing the PG1 signal and the PG2 signal is performed inside the imaging device 10, but the PG1 signal and the PG2 signal may be configured to be output as RAW data directly to the outside of the imaging device 10. Also, the PG1 signal and PG2 signal output from the imaging device 10 may be imported into an external device such as a personal computer, and HDR compositing processing may be performed using software installed in the external device.

[0143] In addition, in each of the above embodiments, the image sensor 20 is configured with the imaging area 21, the vertical scanning circuit 22, the line memory 23, the horizontal scanning circuit 24, and the output amplifier 25, but the image sensor 20 may be configured with only the imaging area 21. The vertical scanning circuit 22, the line memory 23, the horizontal scanning circuit 24, and the output amplifier 25 may be provided within the processor 40.

[0144] Furthermore, the above-described embodiments can be combined with each other as long as no contradiction occurs.

[0145] In each of the above embodiments, the hardware structure of the control unit, with processor 40 being an example, can use the following various processors. The above various processors include a CPU, which is a general-purpose processor that functions by executing software (programs), as well as a processor such as an FPGA, whose circuit configuration can be changed after manufacture. FPGAs include dedicated electrical circuits, such as PLDs or ASICs, which are processors with a circuit configuration designed specifically to execute specific processes.

[0146] The control unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple control units may be configured with a single processor.

[0147] There are several possible examples of configuring multiple control units with a single processor. A first example is a form in which one processor is configured with a combination of one or more CPUs and software, as typified by computers such as client and server, and this processor functions as multiple control units. A second example is a form in which a processor is used to realize the functions of an entire system including multiple control units on a single IC chip, as typified by system-on-chip (SOC). In this way, the control unit can be configured as a hardware structure using one or more of the various processors described above.

[0148] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0149] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0150] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference. [Explanation of symbols]

[0151] 10. Imaging device 11 Main unit 11A Camera side mount 11B Electrical contacts 11C Front 11D back 12 Imaging lens 12A Lens side mount 12B Electrical Contact 13 Dial 14 Release button 15 Display 16 instruction keys 18 Finder eyepiece 20 Image sensor 21 Imaging area 21A AF Area 22 Vertical scanning circuit 23 Line Memory 24 Horizontal scanning circuit 24A horizontal output line 25 output amplifier 26 pixels 29 Transistor 30 objective lenses 31 Focus Lens 32 Rear lens 33 Aperture 34 Lens drive control unit 40 processors 42 Operation section 45 memory 45A Program 47 Shake detection sensor 50 Main control unit 51 Imaging control unit 52 Image processing section 53 Focus detection unit 54 Brightness detection unit 55 Correction amount calculation section 56 HDR synthesis section 57 Image stabilization unit 58 Low-pass filter processing section C Virtual line CF color filter D1 Photodiode E1 First exposure time E2 Second exposure time EP exit pupil L1 row select line L2 row reset line L3 column signal line LA optical axis LF Luminous Flux M1 Amplifier transistor M2 pixel selection transistor M3 Reset transistor ML Micro Lens N imaging pixels PG1 1st pixel group PG2 2nd pixel group RST Reset signal S pixel signal SEL Row selection signal SF light shielding layer T frame period VD Vertical sync signal ZL, ZR phase detection pixels Z1L, Z1R First phase detection pixel Z2L, Z2R Second phase difference pixel

Claims

1. a processor; an imaging element in which a first pixel group including a plurality of phase difference pixels and a plurality of imaging pixels and a second pixel group including a plurality of imaging pixels are arranged in a first direction, The processor: a first exposure time during which the first pixel group is exposed and a second exposure time during which the second pixel group is exposed are set in accordance with information about a subject imaged by the image sensor. An imaging device, The processor: determining which of the first exposure time and the second exposure time should be made shorter than the other, depending on the information about the subject; Imaging device.

2. A method for driving an imaging device including an imaging element in which a first pixel group including a plurality of phase difference pixels and a plurality of imaging pixels, and a second pixel group including a plurality of imaging pixels are arranged in a first direction, setting a first exposure time during which the first pixel group is exposed and a second exposure time during which the second pixel group is exposed in accordance with information about a subject imaged by the image sensor; determining which of the first exposure time and the second exposure time should be shorter than the other, depending on information about the subject; A driving method for an imaging device comprising:

3. A program for operating an imaging device including an imaging element in which a first pixel group including a plurality of phase difference pixels and a plurality of imaging pixels and a second pixel group including a plurality of imaging pixels are arranged in a first direction, setting a first exposure time during which the first pixel group is exposed and a second exposure time during which the second pixel group is exposed in accordance with information about a subject imaged by the image sensor; determining which of the first exposure time and the second exposure time should be shorter than the other, depending on information about the subject; and causing the imaging device to execute a process including program.

Images