Image capture device, image capture device control method, and program

Abstract

To provide an image capture device, an image capture device control method, and a program that can detect flicker regardless of the frequency of a light source while preventing any effect on an imaging condition such as an exposure time, for example, during imaging of a live view image.SOLUTION: An imaging device 190 includes an imaging element 100 and a flicker detection unit 102 that acquires flicker information. The imaging element 100 is operable in a first readout mode in which a first image is read out, a second readout mode in which a second image is read out while the first readout mode is operating, and a third readout mode in which a third image is read out at a different timing from the second readout mode while the first readout mode is operating. The flicker detection unit 102 is capable of acquiring flicker information on the basis of the second image and the third image.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The present invention relates to an imaging device, a control method for an imaging device, and a program. [Background technology]

[0002] Shooting may be performed under a light source such as a fluorescent lamp that periodically blinks on and off (hereinafter referred to as a "flicker light source"). In this case, periodic fluctuations in luminance values ​​may occur within an image due to, for example, the exposure timing or the time required to read signals from pixels. As a result, flickering may occur, causing stripes to appear within the image. Patent Documents 1 and 2 disclose means for detecting such periodic fluctuations in luminance values ​​in an image, i.e., flickering.

[0003] Patent Document 1 discloses a configuration for detecting flicker by accumulating two images with different accumulation times in parallel and integrating the difference between the two image signals. Patent Document 2 discloses a configuration for detecting flicker by periodically reading out a plurality of images for flicker detection between frames of a live view display image that is periodically read out, and detecting flicker based on evaluation values ​​of these images. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6525525 [Patent Document 2] Patent Publication No. 2021-190992 Summary of the Invention [Problem to be solved by the invention]

[0005] In Patent Document 1, pixel signals of all lines of the image sensor are read out, and when flicker detection is performed using an image for live view display, stripes due to the flicker light source may not appear on the image depending on the relationship between the accumulation time and the flicker frequency. Therefore, the accumulation time of the live view image that is also used for flicker detection is controlled so that stripes due to the flicker light source appear on the image. However, with such control, flickering or the like occurs on the live view screen between frames where flicker detection is performed and frames where flicker detection is not performed, which may cause a user viewing the live view image to feel uncomfortable.

[0006] In addition, in Patent Document 2, in the case of a flicker light source with a known frequency such as a fluorescent lamp, multiple images for flicker detection are captured and the flicker frequency is detected by evaluating the change in the photometric value of each image. However, for a flicker light source with a high frequency and a wide frequency range such as an LED, there is a concern that it may be difficult to detect the flicker frequency.

[0007] The present invention has been made in view of the above problems, and has an object to provide an imaging device, a control method for an imaging device, and a program that can detect flicker regardless of the frequency of a light source while preventing an effect on imaging conditions such as exposure time during imaging of a live view image, for example. [Means for solving the problem]

[0008] In order to achieve the above-mentioned object, an imaging device of the present invention is an imaging device capable of imaging when illuminated with light from a light source, and has a pixel area composed of a plurality of pixels arranged in a matrix and a plurality of output lines that read out signals from the pixel area for each column of the pixels, and is equipped with an imaging means for obtaining an image by imaging, and a flicker information acquisition means for acquiring flicker information regarding flicker of the image caused by the light source, wherein the imaging means is operable in a first readout mode in which the imaged image is a first image and the first image is read out for each column, a second readout mode in which a second image that is a part of the image is read out for each column while the first readout mode is operating, and a third readout mode in which a third image that is a part of the image is read out for each column at a timing different from that of the second readout mode while the first readout mode is operating, and the flicker information acquisition means is capable of acquiring the flicker information based on the second image and the third image. Effect of the Invention

[0009] According to the present invention, for example, during capturing of a live view image, flicker can be detected regardless of the frequency of the light source while preventing influence on capturing conditions such as exposure time. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram showing an example of a hardware configuration of an imaging apparatus according to a first embodiment of the present invention. [Diagram 2] 2 is a circuit diagram showing a configuration of a pixel included in the imaging element. [Diagram 3] FIG. 2 is a diagram illustrating an example of a circuit configuration of an image sensor. [Figure 4A] FIG. 2 is a diagram showing a readout state of an image sensor in a live view image (first image). [Figure 4B] 13 is a diagram showing a readout state of an image sensor in a flicker detection image 1 (second image). FIG. [Figure 4C] 13 is a diagram showing a readout state of an image sensor in a flicker detection image 2 (third image). FIG. [Figure 5A] FIG. 2 is a diagram showing the relationship between rows of pixels read out by a horizontal synchronization signal (HD) and readout time. [Figure 5B] 13 is a diagram showing the relationship between the row of pixels to be read out and the readout time when the readout row is changed. FIG. [Figure 6] 1 is a timing chart showing the operations of capturing a live view image, an image for flicker detection 1, and an image for flicker detection 2. [Figure 7A] 11 is a timing chart showing the relationship between an exposure possible period of a live view image, a readout time of the live view image, and a readout time of an image for flicker detection. [Figure 7B] 11 is a timing chart showing the relationship between an exposure possible period of a live view image, a readout time of the live view image, and a readout time of an image for flicker detection. [Figure 8A] 1A and 1B are diagrams for explaining a phenomenon in which flicker occurs on an image when the image is captured in an environment in which the light intensity of a flickering light source changes periodically. [Figure 8B] 10 is a diagram illustrating an average value of luminance of pixels included in each pixel row of a flicker detection image. FIG. [Figure 9] 13 is a flowchart showing a flicker information acquisition process. [Figure 10] FIG. 11 is a diagram showing an example of a circuit configuration of an image sensor according to a second embodiment. [Figure 11A] FIG. 2 is a diagram showing a readout state of an image sensor in a live view image (first image). [Figure 11B] 13 is a diagram showing a readout state of an image sensor in a flicker detection image 1 (second image). FIG. [Figure 11C] 13 is a diagram showing a readout state of an image sensor in a flicker detection image 2 (third image). FIG. [Figure 12] 1 is a timing chart showing the operations of capturing a live view image, an image for flicker detection 1, and an image for flicker detection 2. [Figure 13] FIG. 13 is a diagram showing an example of a circuit configuration of an image sensor according to a third embodiment. [Figure 14A] FIG. 2 is a diagram showing a readout state of an image sensor in a live view image (first image). [Figure 14B] 13 is a diagram showing a readout state of an image sensor in a flicker detection image 1 (second image). FIG. [Figure 14C] 13 is a diagram showing a readout state of an image sensor in a flicker detection image 2 (third image). FIG. [Figure 15] 1 is a timing chart showing the operations of capturing a live view image, an image for flicker detection 1, and an image for flicker detection 2. [Figure 16] 13 is a diagram showing differences in frequency detection bands due to differences in readout times of a live view image, an image 1 for flicker detection, and an image 2 for flicker detection. FIG. [Figure 17] 10 is a flowchart showing a process for switching which combination of images is to be subjected to flicker detection. [Figure 18] FIG. 13 is a diagram showing an example of a circuit configuration of an image sensor according to a fourth embodiment. [Figure 19A] FIG. 2 is a diagram showing a readout state of an image sensor in a live view image (first image). [Figure 19B] 13 is a diagram showing a readout state of an image sensor in a flicker detection image 1 (second image). FIG. [Figure 19C] 13 is a diagram showing a readout state of an image sensor in a flicker detection image 2 (third image). FIG. [Figure 20] 1 is a timing chart showing the operations of capturing a live view image, an image for flicker detection 1, and an image for flicker detection 2. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Each embodiment of the present invention will be described in detail below with reference to the drawings. However, the configurations described in each of the following embodiments are merely examples, and the scope of the present invention is not limited by the configurations described in each embodiment. For example, each part constituting the present invention can be replaced with any configuration that can exert a similar function. In addition, any configuration may be added. In addition, any two or more configurations (features) of each embodiment can be combined.

[0012] First Embodiment The first embodiment will be described below with reference to Figs. 1 to 9. Fig. 1 is a block diagram showing an example of a hardware configuration of an imaging device according to the first embodiment of the present invention. An imaging device 190 shown in Fig. 1 is, for example, a digital camera, and has a still image capturing function for capturing still images and a moving image capturing function for capturing moving images. This imaging device 190 has an imaging element (imaging means) 100, a control unit 101, a flicker detection unit (flicker information acquisition means) 102, an operation unit 103, a display unit 104, an image processing unit 105, and a recording unit 106.

[0013] The image sensor 100 receives light that has passed through an optical system, such as a lens, which forms an image of light reflected from a subject (not shown), and converts the light into an electrical signal. The image sensor 100 then converts the electrical signal into digital image data and outputs a captured image (imaging step). The image sensor 100 can also capture an image of a subject when light is irradiated from a light source. As this light source, for example, a light source that periodically blinks, such as a fluorescent lamp or an LED lamp (hereinafter sometimes referred to as a "flicker light source"), is used. In this case, flicker caused by the flicker light source may occur in the captured image. The flicker detection unit 102 detects and acquires flicker information related to the flicker based on a plurality of image signals obtained from the image sensor 100 (flicker information acquisition step).

[0014] The control unit 101 is communicably connected to the image sensor 100, the flicker detection unit 102, the operation unit 103, the display unit 104, the image processing unit 105, and the recording unit 106, and controls the operation of these units, that is, the entire image capturing device 190. Although not shown in the figure, the image capturing device 190 also has a storage unit that stores a program for causing the control unit 101, which is a computer, to execute each unit and each means (a control method for the image capturing device) of the image capturing device 190.

[0015] The operation unit 103 includes a main switch for starting the imaging device 190, a shooting switch for the user to command the imaging device 190 to shoot still images or moving images, and the like. Another switch included in the operation unit 103 is a switch for starting the flicker detection unit 102 to perform flicker detection. This switch is a switch for switching ON / OFF the flicker detection by the flicker detection unit 102, is normally in the OFF state, and is switched to the ON state when flicker detection (flicker elimination) is required. Note that the switch for switching ON / OFF the flicker detection is not particularly limited, and may be, for example, a switch configured with push button type hardware, or may be a switch configured with software that can be displayed on the display unit 104.

[0016] The operation unit 103 is also used when setting various imaging conditions, etc. The display unit 104 displays still images and moving images according to image data, as well as various menus, etc. The image processing unit 105 performs correction processing of still images and moving images, image compression processing, etc. The image processing unit 105 also performs predetermined arithmetic processing using image signals obtained from the imaging element 100, and performs AE (automatic exposure) processing based on the results of this arithmetic processing. The recording unit 106 records image data, etc.

[0017] Fig. 2 is a circuit diagram showing the configuration of one pixel included in the image sensor. The pixel 200 shown in Fig. 2 has a photodiode (PD) 201, a transfer gate 202, a floating diffusion (FD) section 203, an amplifying MOS transistor 204, a reset switch 205, a pixel selection switch 206, and an output terminal (vout) 207.

[0018] The photodiode 201 functions as a photoelectric conversion element that photoelectrically converts an incident optical signal and accumulates an electric charge according to the amount of exposure. By setting a control signal tx output from a vertical scanning circuit (not shown) to a high level, the transfer gate 202 is turned on (conductive). As a result, the electric charge accumulated in the photodiode 201 is transferred to the floating diffusion section 203. The floating diffusion section 203 is connected to the gate of an amplifying MOS transistor 204. The amplifying MOS transistor 204 outputs a voltage signal according to the amount of electric charge transferred from the photodiode 201 to the floating diffusion section 203.

[0019] The reset switch 205 is a switch for resetting the charge of the floating diffusion portion 203 and the photodiode 201. The reset switch 205 is turned on (conductive state) by setting the control signal res output from the vertical scanning circuit to a high level. This resets the floating diffusion portion 203. When resetting the charge of the photodiode 201, the control signal tx and the control signal res are simultaneously set to a high level to turn on the transfer gate 202 and the reset switch 205. This resets the photodiode 201 via the floating diffusion portion 203.

[0020] The pixel selection switch 206 is turned on (conductive) by setting the control signal sel output from the vertical scanning circuit to a high level. This connects the amplification MOS transistor 204 to the output terminal 207. In this connected state, the pixel signal converted to a voltage by the amplification MOS transistor 204 is output to the output terminal 207. At this time, the pixel selection switch 206 is turned on and off by the control signal sel to adjust the row of pixels from which the pixel signal is read out, thereby changing the thinning rate of the pixel row and outputting the image signal.

[0021] Fig. 3 is a diagram showing an example of a circuit configuration of an image sensor. As shown in Fig. 3, the image sensor 100 has a pixel region 400 composed of a plurality of pixels 200 arranged in a matrix. In this embodiment, for ease of understanding, 3 columns x 12 rows of pixels 200 are representatively shown, but in reality, hundreds of thousands to tens of millions of pixels 200 are usually arranged in a matrix. "R" attached to the pixel 200 means that it is provided with a red color filter, "G" means that it is provided with a green color filter, and "B" means that it is provided with a blue color filter.

[0022] The image sensor 100 also has vertical output lines 300a-300c, an AD converter (ADC) 301, a current source 302, a vertical scanning circuit 303, a horizontal scanning circuit 304, and a data output unit 305. The vertical scanning circuit 303 supplies control signals res, tx, sel, etc. to each pixel 200. A current source 302 is connected to the vertical output lines 300a-300c. This allows the vertical output lines 300a-300c to read out signals output from predetermined pixels 200 in the pixel region 400 for each row of the pixels 200 (for each vertical arrangement in FIG. 3).

[0023] The vertical output lines 300a to 300c are connected to the inputs of an AD converter 301. The AD converter 301 performs analog-to-digital conversion on the pixel signals output from the respective pixels 200. In addition, the pixels 200 connected to the current source 302 and the vertical output lines 300a to 300c, and the amplification MOS transistor 204 form a source follower circuit.

[0024] The horizontal scanning circuit 304 performs horizontal scanning. The horizontal scanning circuit 304 sequentially repeats this row-by-row readout. Then, the AD converter 301 outputs the digitally converted pixel signals from the data output unit 305 as a first image signal, a second image signal, and a third image signal. The vertical output line 300a can read out the first image. The vertical output line 300b can read out the second image. The vertical output line 300c can read out the third image. In this way, the image sensor 100 is configured not to share the vertical output lines used in the first readout mode, the second readout mode, and the third readout mode.

[0025] Furthermore, the pixel region from which the vertical output line 300c reads out signals is shared with the pixel region from which the vertical output line 300a reads out signals. On the other hand, the pixel region from which the vertical output line 300b reads out signals is different from the pixel regions from which the vertical output lines 300a and 300c read out signals. As a result, the accumulation of the second image and the readout of the image signals are performed independently of the accumulation and readout of the first and third images.

[0026] In addition, in this embodiment, the number of vertical pixels in the pixel regions of the second image and the third image is the same. Meanwhile, the number of vertical output lines 300b for reading out the second image is two, whereas the number of vertical output lines 300c for reading out the third image is one. That is, the total number of vertical output lines 300b used in the second readout mode is different from the total number of vertical output lines 300c used in the third readout mode.

[0027] Next, the readout of the first image, the second image, and the third image and the readout time thereof will be described with reference to Figs. 4A to 4C, 5A, and 5B. Fig. 4A is a diagram showing the readout state of the imaging element in a live view image (first image). Fig. 4B is a diagram showing the readout state of the imaging element in an image 1 for flicker detection (second image). Fig. 4C is a diagram showing the readout state of the imaging element in an image 2 for flicker detection (third image). In Figs. 4A to 4C, the white areas indicate readout pixels (readout rows), and the shaded areas indicate non-readout pixels (readout rows) that are not readout. The white areas and shaded areas are the same in Figs. 11A to 11C, 14A to 14C, and 19A to 19C.

[0028] Fig. 5A is a diagram showing the relationship between the row of pixels from which signals are read out by a horizontal synchronization signal (HD) and the signal readout time. Fig. 5B is a diagram showing the relationship between the row of pixels from which signals are read out when the readout row is changed and the signal readout time. A "live view image (image for live view display)" is a captured image displayed in real time on the liquid crystal screen or electronic viewfinder included in the display unit 104, and is usually a moving image. In this embodiment, the live view image is the first image.

[0029] The image sensor 100 is configured by the control unit 101 to be operable in a first readout mode, a second readout mode, and a third readout mode. The first readout mode is a mode in which a live view image (first image) is read out. The second readout mode is a mode in which a second image that is a part of the live view image is read out while the first readout mode is operating. The third readout mode is a mode in which a third image that is a part of the live view image is read out at a timing different from that of the second readout mode while the first readout mode is operating.

[0030] The second image and the third image are images used for flicker detection, and the second image is sometimes called "flicker detection image 1 (first flicker detection image)" and the third image is sometimes called "flicker detection image 2 (second flicker detection image)." Signals are read out from the pixels sequentially by a vertical output line (any of the vertical output lines 300a to 300c) assigned within the pixel region 400 every time a horizontal synchronization signal (HD) is issued.

[0031] As shown in FIGS. 4A to 4C, pixel region 400 has 16 pixels 200 arranged side by side in each row, for a total of 12 rows.

[0032] In the first readout mode, four rows out of six rows are read out in the RG row including the pixels 200 having the color filters R and G and the GB row including the pixels 200 having the color filters G and B. In the example shown in FIG. 4A, the pixel rows that are read out are the first to fourth rows and the seventh to tenth rows.

[0033] In the second readout mode, two rows are read out in a six row cycle. In the example shown in Fig. 4B, the pixel rows read out are the third and fourth rows, and the ninth and tenth rows, which are part of the pixel rows read out in the live view image. In the third readout mode, two rows are read out in a six row cycle. In the example shown in Fig. 4C, the pixel rows read out are the fifth and sixth rows, and the eleventh and twelfth rows, which are different from the pixel rows read out in the live view image.

[0034] Fig. 5A(a) shows pixel rows of a live view image from which signals are read out by a horizontal synchronization signal (HD) and the relationship required for signal readout. Fig. 5A(b) shows pixel rows of a flicker detection image 1 from which signals are read out by a horizontal synchronization signal (HD) and the relationship required for signal readout. Fig. 5A(c) shows pixel rows of a flicker detection image 2 from which signals are read out by a horizontal synchronization signal (HD) and the relationship required for signal readout.

[0035] A period T_HD indicates one HD period, during which each vertical output line reads out signals from the assigned pixel row. Then, in the next HD period, the vertical output line reads out signals from the next pixel row. By repeating this readout, signals from all assigned pixel rows can be read out in each readout mode.

[0036] Time T_RO1 is the time required for the vertical output line 300a to read out signals from all pixel rows assigned to the live view image. Time T_RO2 is the time required for the vertical output line 300b to read out signals from all pixel rows assigned to the flicker detection image 1. Time T_RO3 is the time required for the vertical output line 300c to read out signals from all pixel rows assigned to the flicker detection image 2.

[0037] As described above, since there are two vertical output lines 300b (see FIG. 3), signals from two rows of pixels are read out in one HD period. Therefore, the signal readout time can be expressed by the following formula (1).

[0038] Signal readout time [ms] = 1HD period [ms] × (number of vertical pixels ÷ number of vertical output lines) (1) The "number of vertical pixels" refers to the number of rows of pixels in a pixel area assigned to each readout mode.

[0039] As described above, in both the second readout mode and the third readout mode, signals of two rows are read out in a six-row cycle. Therefore, the number of vertical pixels in the flicker detection image 1 is the same as the number of vertical pixels in the flicker detection image 2. Therefore, the difference in readout time between the two flicker detection images is determined by the number of vertical output lines of each flicker detection image. For example, in this embodiment, the ratio of the number of vertical pixels between the two flicker detection images is 1:1, and the ratio of the number of vertical output lines (the number of vertical output lines 300b:the number of vertical output lines 300c) is 2:1. In this case, the ratio of the readout times between the two flicker detection images is given by the following formula (2).

[0040] T_RO2:T_RO3=1×4÷2:1×4÷1=1:2···(2)

[0041] As described later, the two flicker detection images used for flicker detection need to have different readout timings. In this embodiment, the second readout mode and the third readout mode can be operated at different timings by adjusting the number of vertical output lines 300 used in the second readout mode and the number of vertical output lines 300 used in the third readout mode. This makes it easy to differentiate the readout timings of the two flicker detection images.

[0042] In this embodiment, the two flicker detection images are read out from two different rows in a six-row cycle, but this is not limited thereto. For example, the number of rows of pixels read out may be different between the two flicker detection images, or signals from the same pixel row may be read out. In addition, when reading out the two flicker detection images, different vertical output lines are used, and the number of lines is two and one, respectively, but this is not limited thereto. For example, pixel signals may be read out from the same vertical output line between the two flicker detection images. In any case, the imaging device 190 may make the readout timings of the two flicker detection images different by adjusting the number of vertical pixels to be read out or the number of vertical output lines.

[0043] In the imaging device 190, the readout timing of the two flicker detection images may be made different by changing the 1 HD period. In FIG. 5B, a period T_HD1 indicates the 1 HD period of a live view image. A period T_HD2 indicates the 1 HD period of an image 1 for flicker detection. A period T_HD3 indicates the 1 HD period of an image 2 for flicker detection. The ratio of the 1 HD periods of the images for flicker detection is adjusted to 3:1. In this manner, in this embodiment, the cycle in which the second readout mode reads out the image 1 for flicker detection and the cycle in which the third readout mode reads out the image 2 for flicker detection are different from each other. As described above, the readout time can be expressed by the above formula (1). Therefore, the ratio of the readout times of the two images for flicker detection is given by the following formula (3).

[0044] T_RO2:T_RO3=3×4÷2:1×4÷1=3:2···(3)

[0045] In this way, by changing the 1HD period as a software change, the readout timings of the two flicker detection images can be made different, which makes it possible to adjust the readout timings without making any changes to the hardware configuration (e.g., the number of vertical output lines) of the image sensor 100.

[0046] Fig. 6 is a timing chart showing the operation of capturing a live view image, a flicker detection image 1, and a flicker detection image 2. For example, if the image sensor 100 is a rolling shutter type image sensor in which pixel reset and readout are performed sequentially row by row, exposure for each pixel is performed sequentially in the line direction every time a horizontal synchronization signal (HD) is issued. The timing chart shown in Fig. 6 shows the operation of resetting pixels, reading out an image signal of a live view image synchronized with a vertical synchronization signal (VD), and reading out an image signal of a flicker detection image not synchronized with the vertical synchronization signal, sequentially performed row by row.

[0047] 6, readout scans r11 and r12 indicate readout scans of a live view image in the image sensor 100. When a vertical synchronization signal (VD) is asserted to the image sensor 100 at times t600 and t605, the image sensor 100 starts reading out a live view image, i.e., the first readout mode, as shown in readout scans r11 and r12. Then, the respective readouts are completed at times t601 and t606. This live view image is read out on the vertical output line 300a.

[0048] Moreover, readout scans r21 and r22 indicate readout scans of the flicker detection image 1 in the image sensor 100. As shown in readout scans r21 and r22, the image sensor 100 starts reading out the flicker detection image 1, i.e., the second readout mode, at times t602 and t607. Then, the respective readouts are completed at times t603 and t608. This flicker detection image 1 is read out on the vertical output line 300b.

[0049] Moreover, readout scans r31 and r32 indicate readout scans of the flicker detection image 2 in the image sensor 100. As shown by readout scans r31 and r32, the image sensor 100 starts reading out the flicker detection image 2, i.e., the third readout mode, at times t602 and t607. Then, the respective readouts are completed at times t604 and t609. This flicker detection image 2 is read out on the vertical output line 300c.

[0050] At this time, the readout times of the readout scans r31 and r32 are longer than those of the readout scans r21 and r22. This is because the signal readout time is changed by controlling the number of vertical pixels from which signals are read out in the image sensor 100 and the number of vertical output lines that read out the signals of these vertical pixels.

[0051] In order to unify the exposure time of each pixel row, resetting is also performed sequentially in the line direction, similar to readout scanning. Reset scanning s11, reset scanning s12, reset scanning s21, reset scanning s22, reset scanning s31, and reset scanning s32 indicate reset scanning of pixels 200 included in the image sensor 100. In this manner, in the image sensor 100, a live view image and each flicker detection image are read out while readout scanning and reset scanning are controlled.

[0052] At this time, the reset scan s11 of the live view image is controlled so that the accumulation time of the live view image becomes accumulation time T1. This accumulation time T1 is determined by the AE processing performed by the image processing unit 105. Also, the reset scan s21 of the flicker detection image 1 and the reset scan s31 of the flicker detection image 2 are controlled so that the accumulation times of the flicker detection image 1 and the flicker detection image 2 become accumulation time T2, respectively. At this time, the exposure possible period in the live view image of the next frame becomes exposure possible period T3.

[0053] In the frame following the frame in which the flicker detection image 1 is captured at the accumulation time T2, the reset scan s22 of the flicker detection image 1 is controlled so that the accumulation time of the flicker detection image 1 becomes the accumulation time T4. Similarly, in the frame following the frame in which the flicker detection image 2 is captured at the accumulation time T2, the reset scan s32 of the flicker detection image 2 is controlled so that the accumulation time of the flicker detection image 2 becomes the accumulation time T4.

[0054] Here, the accumulation time T4 will be explained. When the flicker detection image 1 is captured with the accumulation time T2, flicker, i.e., stripes, do not necessarily occur on the flicker detection image 1. For example, consider a case where the period of the flicker light source is 4000 Hz and the accumulation time T2 is set to N / 4000 seconds (N is a natural number). In this case, the exposure amount for every pixel row is N periods of the flicker light source. As a result, the ratio of high and low luminance of the flicker light source during the exposure time of each pixel row becomes equal, so that the influence of the flicker light source, i.e., flicker, does not appear on the image.

[0055] As described later, flicker information is acquired based on the change in luminance value due to flicker on the image. Therefore, in order to cause a change in luminance value due to flicker on the image, it is necessary to control the accumulation time T4 so that the accumulation time T4 is not an integer multiple of the period of the flicker (stripes). Therefore, the accumulation time T4 is controlled so that the accumulation time T2 and the accumulation time T4 are not in an integer multiple relationship. As a result, even if no flicker occurs on the image during the accumulation time T2, for example, a flicker can be generated on the image during the accumulation time T4.

[0056] Fig. 7A is a timing chart showing the relationship between the exposure period of a live view image, the readout time of the live view image, and the readout time of an image for flicker detection. Fig. 7A is a timing chart when the readout time of image 1 for flicker detection is shorter than the readout time of the live view image. Fig. 7B is a timing chart showing the relationship between the exposure period of a live view image, the readout time of the live view image, and the readout time of an image for flicker detection. Fig. 7B is a timing chart when the readout time of image 1 for flicker detection is longer than the readout time of the live view image.

[0057] In the case of FIG. 7A, the readout time T_RO1_A of the flicker detection image 1 is shorter than the readout time T_RO2_A of the live view image. At this time, the section between the reset scan s21_A and the readout scan r21_A is the accumulation time T2 of the flicker detection image 1. Also, the section between the reset scan s12_A and the readout scan r12_A is the exposure possible period T3_A during which exposure is possible with the live view image. As described above, the live view image shares some pixel rows (pixel areas) with the flicker detection image 1. Note that, in this embodiment, these shared pixel rows are the 3rd, 4th, 9th, and 10th rows (see FIG. 4A and FIG. 4B). As a result, exposure of the live view image starts after exposure of the pixel row that starts exposure first among the pixel rows shared by the flicker detection image 1 and the live view image is completed.

[0058] In the case of FIG. 7B, the readout time T_RO1_B of the flicker detection image 1 is longer than the readout time T_RO2_B of the live view image. At this time, the section between the reset scan s21_B and the readout scan r21_B is the accumulation time T2 of the flicker detection image 1. Also, the section between the reset scan s12_B and the readout scan r12_B is the exposure possible period T3_B during which exposure is possible with the live view image. The live view image shares some pixel rows with the flicker detection image 1. As a result, the exposure start of the live view image is before the readout time T_RO2_B, which is the reset scan time from the time exposure of the pixel row that lastly starts exposure among the pixel rows shared by the flicker detection image 1 and the live view image ends. In this way, the period during which accumulation of the live view image in the next frame for flicker detection is possible is determined by the presence or absence of a shared pixel row with the flicker detection image and the relationship with the readout time.

[0059] FIG. 8A is a diagram for explaining a phenomenon in which flicker occurs on an image when imaging is performed in an environment in which the light intensity of a flicker light source changes periodically. As shown in FIG. 8A, the light source intensity 801 of the flicker light source changes periodically over time. In addition, the section between the shutter scan 802 and the readout scan 803 is the accumulation time. The accumulation timing T804 of the pixel row 804 included in the image sensor 100 is included more frequently in a time period in which the luminance of the flicker light source is high than in a time period in which the luminance of the flicker light source is low. On the other hand, the accumulation timing T805 of the pixel row 805 included in the image sensor 100 is included more frequently in a time period in which the luminance of the flicker light source is low than in a time period in which the luminance of the flicker light source is high. In imaging in such an environment in which the light source intensity 801 changes periodically over time, the exposure amount in each pixel row changes. As a result, a periodic fluctuation in the signal level, that is, flicker occurs in the image 806.

[0060] FIG. 8B is a diagram showing a schematic diagram of the average value of the luminance of pixels included in each pixel row of the flicker detection image. As shown in FIG. 8B, the image 807 shows a state in which the luminance value of the flicker detection image having a short signal read time from the image sensor 100 has changed. The image 808 shows a state in which the luminance value of the flicker detection image having a long signal read time from the image sensor 100 has changed. Note that, although the number of vertical pixels of the images read out may differ in practice, the images 807 and 808 are described as having the same size in order to compare the period (flicker) of stripes caused by the flicker light source at the same angle of view. As described above, the exposure amount in each pixel row changes. As a result, a periodic luminance value fluctuation occurs in the image signal depending on the light emission period of the flicker light source. Therefore, the average value of the luminance of each pixel row changes as shown in the images 807 and 808. The imaging device 190 acquires flicker information using two flicker detection images having different luminance fluctuations in this way.

[0061] Next, the flicker information acquisition process and its action will be described. Fig. 9 is a flowchart showing the flicker information acquisition process. As shown in Fig. 9, in step S901, the control unit 101 controls the image sensor 100 to capture a live view image as a first image. The control unit 101 also controls the image sensor 100 to capture a flicker detection image 1 as a second image and a flicker detection image 2 as a third image.

[0062] In step S902, the control unit 101 controls the flicker detection unit 120 to divide the flicker detection image 1 (second image) and the flicker detection image 2 (third image) into regions in the vertical direction, and calculates an average value of luminance information for each divided region. In this manner, the imaging device 190 obtains the luminance of the flicker detection image 1 and the luminance of the flicker detection image 2. Note that if the two flicker detection images have different numbers of vertical pixels, the control unit 101 controls the image processing unit 105 to perform a resizing process. As a result, the live view image and each flicker detection image are adjusted to the same size. Thereafter, the flicker detection unit 120 calculates an average value of luminance information for each region.

[0063] In step S903, the control unit 101 controls the flicker detection unit 120 to calculate the difference in luminance between the two flicker detection images by subtraction processing using the average value of the luminance information for each vertical region of the two flicker detection images calculated in step S902. The difference by this subtraction processing becomes flicker information. Then, from this difference, image information of a component that does not depend on the flicker light source, i.e., noise, can be removed from the image signal, and thus periodic fluctuations in luminance values ​​that depend on the light emission cycle of the flicker light source can be extracted and quantified in each divided region. Note that, in this embodiment, the extraction of the fluctuations in luminance values ​​due to the flicker light source is performed by subtraction processing, but is not limited to this and may be performed, for example, by other comparison calculation methods.

[0064] In step S904, the control unit 101 controls the flicker detection unit 120 to determine whether or not a flicker has been detected. For example, in step S904, the flicker detection unit 120 adds up the absolute difference values ​​of the luminance values ​​in each divided area, which have been quantified for each divided area in step S903. The flicker detection unit 120 then averages the sum by the number of divided areas to calculate the level of luminance change for the entire image. Thereafter, the control unit 101 compares the level with a detection threshold to determine whether or not a flicker has been detected. Thus, in this embodiment, the control unit 101 also functions as a determination unit that determines the presence or absence of a flicker based on the difference obtained by the flicker detection unit 120.

[0065] Note that a part functioning as a determination means may be provided separately from the control unit 101. Also, as an example here, when the level of the luminance change for the entire screen is greater than the detection threshold, the control unit 101 determines that there is flicker, and the process proceeds to step S905. On the other hand, when the level of the luminance change for the entire screen is less than the detection threshold, the control unit 101 determines that there is no flicker, and the process proceeds to step S906.

[0066] In step S905, the control unit 101 calculates the frequency of the flicker based on the frequency analysis result of the fluctuation in luminance value on the image due to the flicker light source and the readout times of the two flicker detection images. In this embodiment, the control unit 101 performs frequency analysis using a Fourier transform and calculates the frequency (number) of stripes due to the flicker on the two flicker detection images. Specifically, the control unit 101 calculates the spectrum of the stripe pattern on the image in the frequency domain by performing a Fourier transform on the comparison calculation result in step S903.

[0067] This spectrum is obtained as information on the number of stripes in the vertical direction of the image. Two peaks exist on the obtained spectrum, one of which is the frequency (number) of stripes caused by flicker on the flicker detection image 1, and the other is the frequency (number) of stripes caused by flicker on the flicker detection image 2. As described above, the period of stripes caused by flicker on the screen depends on the readout time. Therefore, it is possible to predict the light emission period of the flicker light source by using the information on the number of stripes obtained in step S905 and the information on the readout time. For example, by analyzing the number of stripes in step S905, it is possible to identify that the frequencies (number) of stripes caused by flicker on the two flicker detection images are 4 and 16.

[0068] Assume that the readout time of the flicker detection image 1 is 1 ms, and the readout time of the flicker detection image 2 is 4 ms. In this case, since the number of stripes on the image is proportional to the readout time, it can be seen that 4 stripes due to flicker appear on the flicker detection image 1 and 16 stripes due to flicker appear on the flicker detection image 2. In addition, the period [s] of the flicker light source can be calculated by the readout time [s] divided by the number of stripes [lines]. Considering the flicker detection image 1, the period of the flicker light source is 0.001 [s] divided by 4 [lines] = 1 / 4000 [s], and the frequency of the flicker light source can be calculated to be 4000 Hz. Note that, here, the frequency of the flicker light source is calculated from the readout time of the flicker detection image 1 and the number of stripes, but is not limited thereto. For example, the frequency of the flicker light source may be calculated from the readout time of the flicker detection image 2 and the number of stripes.

[0069] In step S906 after step S904, the control unit 101 determines whether or not a flicker exists based on the number of times the series of processes from step S901 to step S904 have been repeated since the start of flicker information calculation. In this embodiment, after reading out two flicker detection images in accumulation time T2, the accumulation time is changed and a live view image and a flicker detection image are also read out in accumulation time T4 (see FIG. 5). In this manner, in this embodiment, the accumulation time in the second readout mode and the accumulation time in the third readout mode are changed each time the second readout mode and the third readout mode are performed multiple times.

[0070] The control unit 101 also functions as a time changing means for changing this accumulation time. If the accumulation time has been changed once since the control unit 101 started calculating flicker information, the process proceeds to step S908, where it is determined that there is no flicker, and the process ends. If the accumulation time has been changed less than once, the process proceeds to step S907, where the accumulation time is changed from T2 to T4, and the process returns to step S901, and the subsequent steps are executed in order. Note that a part that functions as a time changing means may be provided separately from the control unit 101.

[0071] As described above, the imaging device 190 is configured to acquire the second and third images while changing the accumulation time. In this embodiment, by setting the accumulation time to be shorter than the shortest period of the detection target, stripes due to flicker can be generated on the image. In this case, since flicker can be detected in one frame and there is no need to change the accumulation time, step S906 can be omitted and the process can be terminated.

[0072] Furthermore, in this embodiment, if it is determined in step S904 that there is no flicker and the number of times the accumulation time has been changed in step S906 is less than one, the accumulation time is changed. In contrast, regardless of the presence or absence of flicker, the accumulation time may be changed immediately after the start of flicker information calculation to obtain flicker detection image 1 and flicker detection image 2 multiple times. This allows the imaging device 190 to capture images in which fluctuations in luminance value due to flicker appear on the image under conditions where there is flicker of various frequencies, without waiting until the presence or absence of flicker is determined.

[0073] Furthermore, the control unit 101 can control the accumulation time to an integer multiple of the period of the flicker light source based on the frequency of the flicker light source calculated by the flicker information acquisition process (flowchart shown in FIG. 9). This makes it possible to capture a still image with reduced influence of the flicker light source.

[0074] As described above, the imaging device 190 reads out a live view image in the first readout mode. Moreover, while operating in the first readout mode, the imaging device 190 reads out the image 1 for flicker detection in the second readout mode, and reads out the image 2 for flicker detection in the third readout mode at a timing different from that in the second readout mode. Then, the flicker detection unit 102 can acquire flicker information based on the image 1 for flicker detection and the image 2 for flicker detection. This makes it possible to detect flicker of a wider range of frequencies, for example, during imaging of a live view image, without affecting the exposure conditions for imaging the live view image, that is, while preventing an effect on imaging conditions such as exposure time. This makes it possible to perform imaging with reduced effects from a flicker light source.

[0075] <Second embodiment> Hereinafter, the second embodiment will be described with reference to Figs. 10 to 12. The differences from the above-described embodiment will be mainly described, and a description of the similar points will be omitted.

[0076] Fig. 10 is a diagram showing an example of a circuit configuration of an image sensor according to the second embodiment. As shown in Fig. 10, the pixel region (pixels 200) read out by the vertical output line 300b and the pixel region (pixels 200) read out by the vertical output line 300a are shared. Also, the pixel region (pixels 200) read out by the vertical output line 300c and the pixel region read out by the vertical output line 300a are shared. Also, the pixel region read out by the vertical output line 300b and the pixel region read out by the vertical output line 300c are different. Due to such a shared / non-shared state, the accumulation of the second image and the readout of the image signal can be performed independently of the accumulation and readout of the third image.

[0077] Furthermore, in this embodiment, there are three vertical output lines 300a for reading out a live view image, whereas there is one vertical output line 300b for reading out image 1 for flicker detection, and one vertical output line 300c for reading out image 2 for flicker detection. That is, the number of vertical output lines for reading out a live view image is different from the number of vertical output lines for reading out image 1 for flicker detection, and the number of vertical output lines for reading out image 2 for flicker detection. Furthermore, the number of vertical output lines for reading out image 1 for flicker detection is the same as the number of vertical output lines for reading out image 2 for flicker detection.

[0078] Next, the readout of the first image, the second image, and the third image in this embodiment will be described with reference to Figs. 11A to 11C. Fig. 11A is a diagram showing the readout state of the image sensor for a live view image (first image). Fig. 11B is a diagram showing the readout state of the image sensor for an image 1 for flicker detection (second image). Fig. 11C is a diagram showing the readout state of the image sensor for an image 2 for flicker detection (third image).

[0079] In the first readout mode, signals are read out from all rows in a six-row cycle in row RG including pixels 200 having color filters R and G and row GB including pixels 200 in which color filters G and B are arranged. In the example (live view image) shown in FIG 11A, the pixel rows from which signals are read out are rows 1 to 12.

[0080] In the second readout mode, signals from four rows are read out every six rows. In the example shown in Fig. 11B (flicker detection image 1), the pixel rows from which signals are read out are rows 1 to 4 and rows 7 to 12. In the third readout mode, signals from two rows are read out every six rows. In the example shown in Fig. 11C (flicker detection image 2), the pixel rows from which signals are read out are rows 5 and 6 and rows 11 and 12, which are different from the pixel rows from which signals are read out in the live view image.

[0081] As described above, in this embodiment, the total number of pixel rows from which signals are read out in the second readout mode is greater than the total number of pixel rows from which signals are read out in the third readout mode. That is, the total number of pixel rows from which signals are read out in the second readout mode and the total number of pixel rows from which signals are read out in the third readout mode are different from each other.

[0082] As described above, in this embodiment, the second readout mode and the third readout mode share pixel rows (arrays) with the first readout mode. That is, the second readout mode and the first readout mode share the 1st to 4th row and the 7th to 12th row arrays. In addition, the third readout mode and the first readout mode share the 5th, 6th, 11th, and 12th rows.

[0083] In this embodiment, both the second readout mode and the third readout mode share the array with the first readout mode, but this is not limited thereto, and at least one of the readout modes may share the array with the first readout mode. The pixel rows read out in the second readout mode are the first to fourth rows and the seventh to twelfth rows, and the pixel rows read out in the third readout mode are the fifth and sixth rows and the eleventh and twelfth rows. As a result, the second readout mode and the third readout mode do not share pixel rows with each other. Due to such a shared / non-shared state, all of the pixels 200 used for imaging among the pixels 200 of the image sensor 100 can be assigned to a live view image. This makes it possible to obtain an image while preventing a decrease in resolution when performing flicker detection.

[0084] As described above, the readout time can be expressed by the above formula (1). In this embodiment, the ratio of the readout times of the two flicker detection images is given by the following formula (4).

[0085] T_RO2:T_RO3=1×4÷1:1×2÷1=2:1···(4)

[0086] In this embodiment, the two flicker detection images are read out from different pixel rows, 4 rows and 2 rows, out of a 6 row cycle, but this is not limited thereto, and for example, the number of pixel rows may be changed or the same pixel rows may be used. Also, in this embodiment, different vertical output lines are used when reading out the two flicker detection images, and the number of lines is one for each, but this is not limited thereto, and for example, the images may be read out from the same vertical output line.

[0087] 12 is a timing chart showing the shooting operation of a live view image, flicker detection image 1, and flicker detection image 2. As described above, the two flicker detection images share the pixel area (pixel rows) with the live view image. In addition, the readout time of flicker detection image 1 is longer than the readout time of the live view image. Here, consider the time at which exposure can start for the pixel area of ​​the live view image in the next frame in which flicker detection is performed. As described in the first embodiment, this time is the time before the time required for reset scanning of the live view image from time t604 when exposure of the pixel row that last starts exposure ends in the pixel area shared by flicker detection image 1 and the live view image.

[0088] As shown in FIG. 12, the readout time of the flicker detection image 2 is shorter than the readout time of the flicker detection image 2. In addition, the two flicker detection images start reset scanning and read scanning at the same time. As a result, the time when the pixel area of ​​the live view image in the next frame where flicker detection is performed can start exposure is determined by the time t604 when the exposure of the pixel row that last starts exposure is completed among the pixel areas shared by the flicker detection image 1 and the live view image. Note that, in this embodiment, the reset scanning and read scanning are started at the same time for the two flicker detection images, but this is not limited thereto. For example, the center of gravity of the exposure of the live view image and each flicker detection image may be aligned to increase the simultaneity. This improves the performance of flicker detection when the subject is a moving object.

[0089] <Third embodiment> Hereinafter, the third embodiment will be described with reference to Figs. 13 to 17. The differences from the above-described embodiments will be mainly described, and descriptions of similar points will be omitted.

[0090] Fig. 13 is a diagram showing an example of a circuit configuration of an image sensor according to the third embodiment. As shown in Fig. 13, pixel regions (pixels 200) read out by the vertical output line 300a, the vertical output line 300b, and the vertical output line 300c are separate regions and are not shared with each other. This allows the accumulation of the first image and the readout of the image signal, the accumulation of the second image and the readout of the image signal, and the accumulation of the third image and the readout of the image signal to be performed independently.

[0091] Next, the readout times of the first image, the second image, and the third image in this embodiment will be described with reference to Figs. 14A to 14C. Fig. 14A is a diagram showing the readout state of the image sensor for a live view image (first image). Fig. 14B is a diagram showing the readout state of the image sensor for an image 1 for flicker detection (second image). Fig. 14C is a diagram showing the readout state of the image sensor for an image 2 for flicker detection (third image).

[0092] In the first readout mode, 6 rows out of 12 rows are read out in row RG including pixels 200 having color filters R and G and row GB including pixels 200 having color filters G and B. In the example (live view image) shown in Fig. 14A, the pixel rows from which signals are read out are rows 1 to 6.

[0093] In the second readout mode, four rows are read out in a 12-row cycle. In the example shown in Fig. 14B (flicker detection image 1), the pixel rows from which signals are read out are the seventh to tenth rows. In the third readout mode, two rows are read out in a 12-row cycle. In the example shown in Fig. 14C (flicker detection image 2), the pixel rows from which signals are read out are the eleventh and twelfth rows.

[0094] As described above, in this embodiment, the second readout mode and the third readout mode do not share pixel rows (arrays) with the first readout mode. In addition, the second readout mode and the third readout mode also do not share pixel rows with each other.

[0095] As described above, the readout time can be expressed by the above formula (1). In this embodiment, the ratio of the readout times of the two flicker detection images is given by the following formula (5).

[0096] T_RO2:T_RO3=1×4÷1:1×2÷1=2:1···(5)

[0097] In this embodiment, the two flicker detection images are read out from different pixel rows, 4 rows and 2 rows, out of a 12 row cycle, but this is not limited to this, and for example, the number of pixel rows may be changed or the same pixel rows may be used. Also, in this embodiment, different vertical output lines are used when reading out the two flicker detection images, and the number of lines is one for each, but the number is not limited to this.

[0098] Fig. 15 is a timing chart showing the shooting operation of a live view image, flicker detection image 1, and flicker detection image 2. As shown in Fig. 15, readout scans r11 and r12, which are readout scans of the live view image, are performed periodically in synchronization with a vertical synchronization signal (VD). The live view image, flicker detection image 1, and flicker detection image 2 have image areas (pixel rows) independent of each other, and are output from independent vertical line outputs. This makes it possible to set independent accumulation times and readout timings for each image.

[0099] It is also possible to set the timing of the readout start r11, readout scan r21, and readout scan r31 to be simultaneous. This makes it possible to set the readout timing of the flicker detection image 1 and the flicker detection image 2 simultaneously, which makes it easy to detect flickers even when capturing an image of a moving subject.

[0100] FIG. 16 is a diagram showing the difference in frequency detection band due to the difference in readout time of the live view image, the flicker detection image 1, and the flicker detection image 2. As described above, the detected frequency band differs due to the difference in readout time (see FIG. 8A and FIG. 8B). Therefore, as shown in FIG. 16, the frequency detection band can be made wider by detecting flicker from the live view image and the flicker detection image 1 than by detecting the frequency from the flicker detection image 1 and the flicker detection image 2. However, when the live view image is used as an image for flicker detection, it is necessary to change the exposure time of the live view image to the exposure time for flicker detection. Therefore, there is a risk that flickering or the like occurs on the live view screen, causing a sense of discomfort to the user.

[0101] Therefore, a configuration for switching which combination of images is to be subjected to flicker detection will be described with reference to Fig. 17. Fig. 17 is a flowchart showing a process for switching which combination of images is to be subjected to flicker detection.

[0102] 17, in step S1701, the control unit 101 determines whether or not a live view is being displayed. "Live view is being displayed" refers to a state in which a live view image is being displayed on the display unit 104. If it is determined in step S1701 that a live view is not being displayed, the process proceeds to step S1702. On the other hand, if it is determined in step S1701 that a live view is being displayed, the process proceeds to step S1705.

[0103] In step S1702, the control unit 101 controls the flicker detection unit 102 to perform flicker detection based on the live view image and the image 1 for flicker detection.

[0104] In step S1703, the control unit 101 controls the flicker detection unit 120 to determine whether or not a flicker has been detected. If it is determined in step S1703 that there is no flicker, the process proceeds to step S1704. On the other hand, if it is determined in step S1703 that there is flicker, the process ends.

[0105] In step S1704, the control unit 101 controls the flicker detection unit 102 to perform flicker detection based on the live view image and the flicker detection image 2, and then the process ends.

[0106] In step S1705 after step S1701 is executed, the control unit 101 determines whether or not the flicker detection is one designated in advance by the user. "Flicker detection designated in advance by the user" refers to flicker detection arbitrarily designated by the user using the operation unit 103, and this information is stored in the storage unit. If it is determined in step S1705 that the flicker detection is one designated by the user, the process proceeds to step S1702, and the subsequent steps are executed in order. On the other hand, if it is determined in step S1705 that the flicker detection is not one designated by the user, the process proceeds to step S1706.

[0107] In step S1706, the control unit 101 controls the flicker detection unit 102 to perform flicker detection based on the flicker detection use image 1 and the flicker detection use image 2, and then the process ends.

[0108] As described above, in this embodiment, it is possible to detect flicker (acquire flicker information) based on a live view image and flicker detection image 1, and to detect flicker based on a live view image and flicker detection image 2. Of course, the flicker detection unit 102 is also capable of detecting flicker based on flicker detection image 1 and flicker detection image 2. In the imaging device 190 configured in this way, it is possible to simultaneously set the readout timing of flicker detection image 1 and the readout timing of flicker detection image 2. This makes it easy to detect flicker even when imaging a moving subject.

[0109] <Fourth embodiment> Hereinafter, the fourth embodiment will be described with reference to Figs. 18 to 20. The differences from the previously described embodiments will be mainly described, and descriptions of similar points will be omitted.

[0110] Fig. 18 is a diagram showing an example of a circuit configuration of an image sensor according to the fourth embodiment. In the configuration shown in Fig. 18, a first image is read out through a vertical output line 300a, and a second image and a third image are read out through a vertical output line 300b.

[0111] Next, the readout times of the first image, the second image, and the third image in this embodiment will be described with reference to Figs. 19A to 19C. Fig. 19A is a diagram showing the readout state of the image sensor for a live view image (first image). Fig. 19B is a diagram showing the readout state of the image sensor for an image 1 for flicker detection (second image). Fig. 19C is a diagram showing the readout state of the image sensor for an image 2 for flicker detection (third image).

[0112] In the first readout mode, 8 rows out of 12 rows are read out in row RG including pixels 200 having color filters R and G and row GB including pixels 200 having color filters G and B. In the example (live view image) shown in FIG 19A, the pixel rows from which signals are read out are rows 1 to 8.

[0113] In the second readout mode, four rows are read out in a 12-row cycle. In the example shown in Fig. 19B (flicker detection image 1), the pixel rows from which signals are read out are the 9th to 12th rows. In the third readout mode, two rows are read out in a 12-row cycle. In the example shown in Fig. 19C (flicker detection image 2), the pixel rows from which signals are read out are the 11th and 12th rows.

[0114] As described above, in this embodiment, the second readout mode and the third readout mode do not share pixel rows (arrays) with the first readout mode, and the second readout mode and the third readout mode share pixel rows with each other.

[0115] Fig. 20 is a timing chart showing the shooting operation of a live view image, flicker detection image 1, and flicker detection image 2. As shown in Fig. 20, readout scans r11 and r12, which are readout scans of the live view image, are performed periodically in synchronization with a vertical synchronization signal (VD). Flicker detection image 1 and flicker detection image 2 do not share an image area (pixel row) and vertical output lines with the live view image. This makes it possible to set the accumulation time and readout timing of flicker detection image 1 and flicker detection image 2 independently of the live view image.

[0116] Note that since the flicker detection image 1 and the flicker detection image 2 share a vertical output line, reading them out at the same time is restricted. Also, the flicker detection image 1 and the flicker detection image 2 share a pixel area. Therefore, it is preferable to perform reset scanning s21 and reset scanning s21 after reading out the flicker detection image 1.

[0117] The disclosure of each embodiment includes the following configurations, methods, and programs. (Configuration 1) An imaging device capable of capturing an image while being irradiated with light from a light source, an imaging means having a pixel area composed of a plurality of pixels arranged in a matrix and a plurality of output lines for reading out signals from the pixel area for each column of the pixels, the imaging means obtaining an image by imaging; a flicker information acquisition unit that acquires flicker information regarding flicker in the captured image caused by the light source, a first readout mode in which the captured image is a first image and the first image is read out for each column; a second readout mode in which a second image that is a part of the captured image is read out for each column while the first readout mode is operating; During operation in the first readout mode, the image sensor can operate in a third readout mode in which a third image, which is a part of the captured image, is read out for each column at a timing different from that in the second readout mode, The imaging device, wherein the flicker information acquisition means is capable of acquiring the flicker information based on the second image and the third image. (Configuration 2) The imaging device according to configuration 1, wherein the second readout mode and the third readout mode do not share the columns with the first readout mode. (Configuration 3) The imaging device according to configuration 1 or 2, wherein the second readout mode and the third readout mode do not share the columns with each other. (Configuration 4) The imaging device according to configuration 1, wherein at least one of the second readout mode and the third readout mode shares the columns with the first readout mode. (Configuration 5) The imaging device according to configuration 4, wherein both the second readout mode and the third readout mode share the columns with the first readout mode. (Configuration 6) The imaging device according to any one of configurations 1 to 5, wherein the second readout mode and the third readout mode do not share the output lines with the first readout mode. (Configuration 7) The imaging device according to any one of configurations 1 to 6, wherein the second readout mode and the third readout mode do not share the output lines with each other. (Configuration 8) The imaging device according to any one of configurations 1 to 7, characterized in that the total number of columns read out in the second readout mode and the total number of columns read out in the third readout mode are different from each other. (Configuration 9) The imaging device according to any one of configurations 1 to 8, characterized in that the total number of the output lines used in the second readout mode and the total number of the output lines used in the third readout mode are different from each other. (Configuration 10) The imaging device according to any one of configurations 1 to 9, wherein a period in which the second readout mode reads out the second image and a period in which the third readout mode reads out the third image are different from each other. (Configuration 11) An imaging device described in any one of configurations 1 to 10, characterized in that the flicker information acquisition means is capable of acquiring the flicker information based on the first image and the second image, and is capable of acquiring the flicker information based on the first image and the third image. (Configuration 12) An imaging device described in any one of configurations 1 to 11, characterized in that it comprises a time changing means for changing the accumulation time of the second readout mode and the accumulation time of the third readout mode each time the second readout mode and the third readout mode are performed multiple times. (Configuration 13) The imaging device according to any one of configurations 1 to 12, wherein the second image and the third image are acquired multiple times each. (Configuration 14) An imaging device described in any one of configurations 1 to 13, characterized in that the second readout mode and the third readout mode can operate at different timings by adjusting the number of output lines used in the second readout mode and the number of output lines used in the third readout mode. (Configuration 15) When the second image is read out in the second readout mode, a luminance of the second image is obtained; When the third image is read out in the third readout mode, a luminance of the third image is obtained; 15. The imaging device according to any one of configurations 1 to 14, wherein the flicker information acquisition means acquires, as the flicker information, a difference between a luminance of the second image and a luminance of the third image. (Configuration 16) The imaging device according to configuration 15, further comprising a determination means for determining the presence or absence of flicker based on the difference. (Configuration 17) The imaging device according to any one of configurations 1 to 16, wherein the captured image is a live view image. (Configuration 18) The imaging device according to any one of configurations 1 to 17, wherein a light source that periodically blinks is used for imaging as the light source. (Method 1) A method for controlling an imaging device capable of capturing an image while being irradiated with light from a light source, comprising the steps of: an imaging step of obtaining an image by imaging the image having a pixel area constituted by a plurality of pixels arranged in a matrix and a plurality of output lines for reading out signals from the pixel area for each column of the pixels; and acquiring flicker information regarding flicker in the captured image caused by the light source, the imaging step being a first readout mode in which the captured image is a first image and the first image is read out for each column; a second readout mode in which a second image that is a part of the captured image is read out for each column while the first readout mode is being executed; During execution of the first readout mode, a third readout mode can be executed at a timing different from that of the second readout mode in which a third image that is a part of the captured image is read out for each column, The method for controlling an imaging apparatus, wherein the flicker information acquiring step is capable of acquiring the flicker information based on the second image and the third image. (Program 1) A program for causing a computer to execute each means of the imaging device according to configuration 1.

[0118] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to the above-mentioned embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. The present invention can also be realized by supplying a program that realizes one or more functions of the above-mentioned embodiments to a system or device via a network or storage medium, and having one or more processors of a computer in the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]

[0119] 100 Imaging element (imaging means) 102 Flicker detection unit (flicker information acquisition means) 190 Imaging Device 200 pixels 300a~300c vertical output line 400 pixel area

Claims

1. An imaging device capable of capturing an image in a state where light is irradiated from a light source, an imaging means having a pixel area made up of a plurality of pixels arranged in a matrix and a plurality of output lines for reading out signals from the pixel area for each row of the pixels, and for obtaining a captured image by the imaging; a flicker information acquisition unit that acquires flicker information regarding flicker in the captured image caused by the light source, the imaging means operates in a first readout mode in which the captured image is a first image and the first image is read out row by row; a second readout mode in which a second image that is a part of the captured image is read out row by row while the first readout mode is operating; During operation of the first read mode, at a timing different from that of the second read mode, a third readout mode in which a third image that is a part of the captured image is read out row by row; The imaging device, wherein the flicker information acquisition means is capable of acquiring the flicker information based on the second image and the third image.

2. The imaging device according to claim 1 , wherein the second readout mode and the third readout mode do not share the same rows as the first readout mode.

3. The imaging device according to claim 1 , wherein the second readout mode and the third readout mode do not share the rows with each other.

4. 2. The imaging device according to claim 1, wherein at least one of the second readout mode and the third readout mode shares the row with the first readout mode.

5. 5. The imaging device according to claim 4, wherein both the second readout mode and the third readout mode share the same rows with the first readout mode.

6. 2. The imaging device according to claim 1, wherein the second readout mode and the third readout mode do not share the output line with the first readout mode.

7. 2. The imaging device according to claim 1, wherein the second readout mode and the third readout mode do not share the output lines with each other.

8. 2. The imaging device according to claim 1, wherein the total number of rows read out in the second readout mode and the total number of rows read out in the third readout mode are different from each other.

9. 2. The imaging device according to claim 1, wherein the total number of the output lines used in the second readout mode and the total number of the output lines used in the third readout mode are different from each other.

10. 2. The imaging device according to claim 1, wherein a cycle in which the second readout mode reads out the second image and a cycle in which the third readout mode reads out the third image are different from each other.

11. The imaging device according to claim 1, characterized in that the flicker information acquisition means is capable of acquiring the flicker information based on the first image and the second image, and is capable of acquiring the flicker information based on the first image and the third image.

12. 2. The imaging device according to claim 1, further comprising a time change unit that changes the accumulation time of the second readout mode and the accumulation time of the third readout mode each time the second readout mode and the third readout mode are performed multiple times.

13. The imaging device according to claim 1 , wherein a plurality of the second images and a plurality of the third images are acquired.

14. 2. The imaging device according to claim 1, wherein the second readout mode and the third readout mode can be operated at different timings by adjusting the number of output lines used in the second readout mode and the number of output lines used in the third readout mode.

15. When the second image is read out in the second readout mode, a luminance of the second image is obtained; When the third image is read out in the third readout mode, a luminance of the third image is obtained; 2. The imaging device according to claim 1, wherein the flicker information acquisition means acquires, as the flicker information, a difference between the luminance of the second image and the luminance of the third image.

16. 16. The imaging apparatus according to claim 15, further comprising a determination unit that determines whether or not flicker occurs based on the difference.

17. 2. The imaging device according to claim 1, wherein the captured image is a live view image.

18. 2. The imaging device according to claim 1, wherein the light source used for imaging is a light source that periodically blinks.

19. A method for controlling an imaging device capable of capturing an image in a state where light is irradiated from a light source, comprising: an imaging step of obtaining a captured image by imaging the image sensor, the image sensor having a pixel area composed of a plurality of pixels arranged in a matrix and a plurality of output lines for reading out signals from the pixel area for each row of the pixels; a flicker information acquiring step of acquiring flicker information regarding flicker in the captured image caused by the light source, the imaging step is a first readout mode in which the captured image is a first image and the first image is read out row by row; a second readout mode in which a second image that is a part of the captured image is read out row by row while the first readout mode is being executed; During execution of the first read mode, at a timing different from that of the second read mode, a third readout mode in which a third image that is a part of the captured image is read out row by row, The method for controlling an imaging device, wherein the flicker information acquisition step is capable of acquiring the flicker information based on the second image and the third image.

20. A program for causing a computer to execute each means of the imaging apparatus according to claim 1.

Images