Imaging device, control method for imaging device, and storage medium

Abstract

The present invention provides an imaging device, a control method for the imaging device, and a storage medium. The imaging device includes: an image sensor configured to capture a subject image formed via a lens and output an image; a detection component configured to detect flicker from the image output from the image sensor; and a control component configured to adjust driving of the lens during a shooting period in which the image sensor captures an image of a subject for flicker detection, when the lens is driven according to a driving operation for changing an amount of light received by the image sensor during the shooting period.

Description

Technical Field

[0001] The present invention relates to a technique for detecting flicker in an imaging device. Background Art

[0002] With the recent increase in the ISO value (increased sensitivity) in digital cameras, it has become possible to take images with a high-speed shutter even under conditions of a flickering artificial light source. The high-speed shutter has the advantage of enabling the taking of blur-free photos such as photos of indoor sports, etc., but when used to take images under conditions of a flickering light source, uneven exposure or color may occur within each frame or even within a single frame.

[0003] Known methods for preventing this problem include reducing the influence of flicker by detecting flicker and performing exposure at the flicker peak position where the change in light amount is minimal.

[0004] In order to use such a method for reducing the influence of flicker, it is necessary to detect flicker and its frequency. Known techniques related to detecting flicker and its frequency include, for example, Japanese Unexamined Patent Application Publication No. 2017-11352. In Japanese Unexamined Patent Application Publication No. 2017-11352, by using a first evaluation value obtained from photometric values obtained by performing photometry multiple times at fixed time intervals, from photometric values obtained at a first interval close to the in-phase of flicker, and a second evaluation value obtained from photometric values obtained at a second interval close to the anti-phase of flicker, it is determined whether there is flicker and the frequency of flicker.

[0005] In addition, in Japanese Unexamined Patent Application Publication No. 2020-80512, an imaging device is described that enables a flicker detection operation to be performed during live view display without stopping the live view display.

[0006] However, the known technique described in Japanese Unexamined Patent Application Publication No. 2017-11352 is based on the assumption that when performing image exposure for flicker detection, only the brightness of the subject brought about by the flickering light source will change, and other exposure conditions such as the aperture will not change. When this assumption condition is not satisfied, the flicker detection performance deteriorates.

[0007] Using the technique described in Japanese Unexamined Patent Application Publication No. 2020-80512, during live view display, it is necessary to continuously control exposure and focus to be in a preferred state for the user. Therefore, when performing a flicker detection operation during live view display, the flicker detection operation is affected by the changes in exposure and focus during live view display, and the image exposure conditions for flicker detection may change. Summary of the Invention

[0008] The present invention is made in view of the above problems, and aims to provide an imaging device capable of suppressing a decrease in detection accuracy when detecting flicker during live view display.

[0009] According to a first aspect of the present invention, there is provided an imaging device including: an image sensor configured to capture a subject image formed via a lens and output an image; a detection component configured to detect flicker from the image output from the image sensor; and a control component configured to adjust driving of the lens during a shooting period in which the image sensor captures an image of a subject for flicker detection, when driving the lens according to a driving operation for changing an amount of light received by the image sensor.

[0010] According to a second aspect of the present invention, there is provided an imaging device including: an image sensor configured to capture a subject image formed via a lens and output an image; a detection component configured to detect flicker from the image output from the image sensor; and a control component configured to change a method for detecting flicker via the detection component, when performing a driving operation of the lens for changing an amount of light received by the image sensor during a shooting period in which the image sensor captures an image of a subject for flicker detection.

[0011] According to a third aspect of the present invention, there is provided a control method for an imaging device provided with an image sensor and a detection component, the image sensor being configured to capture a subject image formed via a lens and output an image, the detection component being configured to detect flicker from the image output from the image sensor, the control method including: controlling adjustment of driving of the lens during a shooting period in which the image sensor captures an image of a subject for flicker detection, when driving the lens according to a driving operation for changing an amount of light received by the image sensor.

[0012] According to a fourth aspect of the present invention, there is provided a control method for an imaging device provided with an image sensor and a detection component, the image sensor being configured to capture a subject image formed via a lens and output an image, the detection component being configured to detect flicker from the image output from the image sensor, the control method including: controlling change of a method for detecting flicker via the detection component, when performing a driving operation of the lens for changing an amount of light received by the image sensor during a shooting period in which the image sensor captures an image of a subject for flicker detection.

[0013] According to a fifth aspect of the present invention, there is provided a computer-readable storage medium storing a program for causing a computer to execute the above method.

[0014] Other features of the present invention will become apparent from the following description of exemplary embodiments with reference to the accompanying drawings. Description of the Drawings

[0015] Figure 1 is a diagram showing an example structure of a digital single-lens mirrorless camera as an imaging device according to a first embodiment of the present invention.

[0016] Figure 2 is a diagram for describing how to drive an image sensor during live view.

[0017] Figure 3 is a flowchart showing a flicker detection operation during live view in the first embodiment.

[0018] Figure 4A and Figure 4B is a diagram showing the relationship between the acquisition timing of an image for flicker detection and the photometric output.

[0019] Figures 5A to 5C is a diagram showing a method for determining whether there is flicker.

[0020] Figure 6 is a diagram showing the photometric output when driving the aperture during accumulation of an image for flicker detection.

[0021] Figure 7 is a flowchart showing a flicker detection operation during live view in the second embodiment.

[0022] Figure 8 is a flowchart showing a flicker detection operation during live view in the third embodiment.

[0023] Figure 9 is a flowchart showing a flicker detection operation during live view in the fourth embodiment.

[0024] Figure 10 is a flowchart showing a flicker detection operation during live view in the fifth embodiment. Detailed Description of the Invention

[0025] Hereinafter, embodiments will be described in detail with reference to the drawings. Note that the following embodiments do not limit the scope of the claimed invention. Multiple features are described in the embodiments, but the invention does not require all of these features, and multiple of these features can be appropriately combined. In addition, in the drawings, the same reference numerals are assigned to the same or similar structures, and redundant descriptions thereof are omitted.

[0026] The present invention is made in view of the following problem: An imaging control operation (e.g., driving an aperture in a lens, driving a focusing lens, driving a zoom lens) for changing the amount of light incident from a lens (the amount of light received by an image sensor) causes disturbance in flicker detection. Hereinafter, an example of an imaging control operation (a driving operation of a lens unit) represented by driving an aperture of a lens is used to describe an embodiment. However, this is only for facilitating understanding of the description. The imaging control operation is not limited to driving an aperture of a lens, and a similar method can be used for other imaging control operations.

[0027] In addition, in the embodiment, a digital single-lens mirrorless camera is used as an example of an imaging device. However, the present invention is not limited thereto, and any of various electronic devices having a camera function can be used. For example, the imaging device according to the present invention can be a mobile communication terminal having a camera function (such as a mobile phone or a smart phone, etc.), a portable computer having a camera function, or a portable game machine having a camera function, etc.

[0028] First Embodiment

[0029] Figure 1 FIG. is a diagram showing an example structure of a digital single-lens mirrorless camera (hereinafter referred to as a camera) as an imaging device according to a first embodiment of the present invention.

[0030] As Figure 1 shown, the camera mainly includes a camera body 100 and an imaging lens 200, which are mechanically and electrically connected via a lens mounting mechanism 110. Note that in the present embodiment, the imaging lens 200 for forming a subject image is configured to be detachable from the camera body 100, but may be integrally formed with the camera body 100 in other embodiments.

[0031] First, the camera body 100 will be described. The image sensor 101 is a CMOS or similar type of image sensor including a photoelectric conversion element and an infrared cut-off filter or a low-pass filter, etc. The optical image of the subject is photoelectrically converted by the image sensor 101, and an image signal is output. When not taking an image, the shutter 104 is closed to shield the image sensor 101, and when in live view (LV) or taking an image, the shutter 104 is opened to guide the incident light passing through the imaging lens 200 to the image sensor 101.

[0032] The system control unit 102 is an arithmetic processing device of a unit that controls the camera body 100. The memory 103 includes a ROM and a RAM. The ROM stores programs and variables, etc. executed by the system control unit 102, and the RAM includes a working area for the system control unit 102 to load programs and / or a storage area for temporarily storing image data, etc.

[0033] In addition, the system control unit 102 is connected to the lens control unit 201 via the lens mounting mechanism 110. The lens control unit 201 controls the position of the focusing lens 202 via the lens driving unit 203. The lens control unit 201 also controls the aperture 204 via the aperture driving unit 205. Specifically, control is performed by performing AE / AF-related calculations at the system control unit 102 based on the image acquired at the image sensor 101, and communicating the position of the focusing lens and / or the f-number according to the calculation result to the lens control unit 201.

[0034] In addition, the shutter 104 is also connected to the system control unit 102 via the shutter control unit 105, and exposes the image sensor 101 for an exposure time corresponding to the calculation result obtained at the system control unit 102.

[0035] Next, the operation of displaying the image acquired at the image sensor 101 will be described. The operations described here include continuously capturing images at the image sensor 101 and real-time displaying the captured images on the display unit, allowing the user to observe the subject in a so-called live view mode.

[0036] There are two types of destinations for displaying the image captured at the image sensor 101 (the rear monitor 106 and the viewfinder display unit 107), and the image is displayed while switching between the two. This switching is performed based on the detection result from the eye proximity detection unit 109. Specifically, when the user's eye is detected to be closely approaching, the image is displayed on the viewfinder display unit 107. Then, the user can confirm the subject displayed on the viewfinder display unit 107 through the eyepiece lens 108 by peeping through the viewfinder. When the eye proximity detection unit 109 detects that the user's eye is not closely approaching, the image is displayed on the rear monitor 106.

[0037] Next, Figure 2 will be used to describe how to drive the image sensor 101 in the live view mode. In Figure 2 , the horizontal direction represents time, the vertical direction represents the position of the image sensor 101 in the vertical direction, and the slashes in the figure represent the readout timing of the image sensor 101 in the live view mode. As Figure 2 shown, first, a single image for live view display is read out, and then, a plurality of images for flicker detection (the imaging period of the images for flicker detection) are read out. Obtaining the image for live view and the plurality of images for flicker detection is defined as a set, and the operation of one set is repeated.

[0038] Images for flicker detection are captured at intervals of 1.66 ms. One set is repeated every T (ms) period. Therefore, this period is set to the frame rate of the live view display, that is, 1 / T (fps). The number of images for flicker detection that can be acquired within one set depends on the period T (ms) of one set. If T (ms) is long, the number of images for flicker detection that can be acquired increases. In this embodiment, the period of one set (the frame rate of the live view display) is 33.33 ms (30 fps), and the number of images for flicker detection that can be acquired within one set is 18.

[0039] Next, the flicker detection operation during the live view of the camera according to this embodiment will be described using Figure 3 . The operations in the flowchart are implemented by loading the program stored in the ROM of the memory 103 to the RAM by the system control unit 102 and executing the program. Figure 3 Note that the operations in the flowchart described later are similarly implemented by the system control unit 102 executing the program in the memory 103. Figures 7 to 10

[0040] When a live view operation start instruction is received in step S101, the system control unit 102 first sets the maximum speed V_max of the aperture 204 drive speed during accumulation for flicker detection to v0 in step S102. Although described in more detail later, generally, when exposing an image for flicker detection, if the amount of light entering from the lens changes due to driving the aperture, the flicker detection performance deteriorates. In addition, the greater the aperture drive speed, the greater the performance degradation. Therefore, in step S102, in the preparation for the first flicker detection operation after starting the live view, the aperture 204 drive speed is set to the maximum value V_max.

[0041] Generally, the state of flicker in the shooting environment is determined by the illumination light in the environment. Therefore, it is unlikely that the state of flicker (that is, whether there is flicker and the frequency) changes rapidly. Therefore, when the purpose of the first flicker detection is to reliably detect flicker, it is preferable to set V_max to a relatively slow speed. The maximum value of the aperture drive speed during the first flicker detection is set to v0. v0 is a predetermined value. However, for example, when v0 = 0 holds, the aperture 204 is stopped from being driven during flicker detection, thereby allowing high-precision flicker detection to be achieved. After setting V_max, the system control unit 102 moves the process to step S103.

[0042] Step S103 is a step of waiting for the time to perform flicker detection. As described above, since the flicker state is unlikely to change rapidly, it is excessive to perform flicker detection for each frame considering the computational load and the like. Instead, flicker detection should be performed every time a predetermined period of time has elapsed. Therefore, in the present embodiment, after activating the live view, the system control unit 102 performs flicker detection, and then repeats the flicker detection every T seconds. For example, when T is set to 1, flicker detection can be performed every 1 second. In step S103, the system control unit 102 determines whether the flicker detection timing has arrived. Then, only when the flicker detection timing has arrived, the process moves to step S104 of performing the flicker detection operation. Otherwise, the operation enters the standby state and continues the normal live view display.

[0043] Steps S104 to S107 are steps of actually performing flicker detection. First, in step S104, in order to accumulate images for flicker detection, the system control unit 102 limits (adjusts) the driving speed of the aperture 204 to a maximum speed V_max or less (maximum value or less). After limiting the aperture driving speed, in step S105, the system control unit 102 causes the image sensor 101 to perform accumulation for flicker detection.

[0044] Figure 4A is a diagram showing the change in signal value over time when there is 100 Hz flicker during the accumulation of images for flicker detection. As used above Figure 2 described, in the present embodiment, the image sensor 101 is driven such that after reading out the image for live view display, the images for flicker detection are read out 18 times at intervals of 1.66 ms to form a set. For flicker detection, among the 18 images taken at intervals of 1.66 ms, the first 12 images are used. For each of these 12 images, accumulation and readout are performed, and as shown in the figure, the nth accumulation performed is represented by "accumulation n", the signal readout of accumulation n is represented by "readout n", and the photometric value (signal value) obtained from the result of readout n is represented by "AE_(n)". Here, the first 12 accumulations performed will be focused on, and photometric values AE_1 to AE_12 are obtained. In addition, since the accumulation is performed within a limited amount of time, the central value of the accumulation period (the position of the centroid of the parallelogram in the figure) is used to represent the acquisition time of each photometric value, and is represented by t_1 to t_12. In step S105, the system control unit 102 causes the image sensor 101 to acquire images for flicker detection in this way at intervals of 1.66 ms.

[0045] After obtaining the image for flicker detection in step S105, in step S106, the system control unit 102 removes the limitation on the driving speed of the aperture 204 in step S104. Then, in step S107, the system control unit 102 performs flicker detection calculation using AE_1 to AE_12.

[0046] In the flicker detection calculation, first, the system control unit 102 calculates an evaluation value from AE_1 to AE_12 for judging the frequency of flicker. In this embodiment, the evaluation value for judging the frequency of flicker is defined by the following formula.

[0047]

[0048] SAD is the abbreviation of sum of absolute difference, and is an index representing similarity in fields such as pattern matching. m is the following value, which indicates how many subsequent photometric values are used to calculate the similarity with the photometric result AE_(n) in the nth of the 12 photometric measurements performed. Therefore, SAD(m) is a formula for calculating the similarity with the photometric value after (1.667×m) ms has passed. From the formula, it can be seen that the higher the similarity, the lower the value of SAD(m).

[0049] For example, in an environment with 100 Hz flicker, since the flicker period is approximately 10 ms, and the relationship between the photometric period of 1.66 ms is 10÷1.66≈6. Therefore, as Figure 4A shown, regardless of the accumulation timing, the same photometric value is obtained in 6 cycles. In other words, the relationship AE_(n)≈AE_(n + 6) is satisfied. According to this characteristic, in an environment with 100 Hz flicker, calculating SAD_6 results in SAD_6≈0 being established. In addition, to detect the presence of 100 Hz flicker, SAD_3 is also calculated. SAD_3 is a value obtained by calculating the similarity with the photometric value after 1.667×3 = 5 ms has passed. In an environment with 100 Hz flicker, since the photometric values at the 5 ms offset timing have an anti-correlation relationship, SAD_3 is a value much larger than SAD_6. In other words, when SAD_3 is a large value and SAD_6 is a small value, it can be considered that there is 100 Hz flicker.

[0050] Using a similar inference, in an environment with 120 Hz flicker, SAD_5 and SAD_3 should be calculated. In an environment with 120 Hz flicker, the light source lighting period is 8.333 ms. Therefore, as Figure 4BAs shown, AE_(n)≈AE_(n+5) and SAD_5≈0 hold. Additionally, in the case of 120Hz flicker, a completely anti-correlated relationship is obtained at 4.16ms. Therefore, ideally, the similarity with the waveform after 4.16ms is judged. However, 4.16ms is not an integer multiple of the 1.667ms frame period. Thus, as a relatively close value, the value of SAD_3 indicating the similarity with the waveform after 5ms is alternatively used. Even in an environment with 120Hz flicker, since SAD_3 represents a similarity close to anti-phase, SAD_3 is a value much larger than SAD_5.

[0051] As described above, the values of SAD_6, SAD_5, and SAD_3 are calculated, and they are used to perform the final flicker frequency judgment. As described above, in an environment with 100Hz flicker, SAD_3 is a value much larger than SAD_6. Therefore, looking at Figure 5A the plane with SAD_3 on the horizontal axis and SAD_6 on the vertical axis as shown, in an environment with 100Hz flicker, a plot in the relatively lower right region of the plane is obtained. In other words, by dividing the Figure 5A region as shown and setting a region for judging that the flicker is 100Hz and a region for judging that the flicker is not 100Hz, it is possible to accurately judge whether there is 100Hz flicker from the plot position.

[0052] In a similar manner, by dividing the Figure 5B plane with SAD_3 on the horizontal axis and SAD_5 on the vertical axis as shown, it is possible to judge whether there is 120Hz flicker.

[0053] Note that Figure 5A and Figure 5B the region division lines shown are only examples, and the inclination and inflection points are not limited to this.

[0054] It is possible to judge whether there is 100Hz flicker from the plot position of the plane shown in Figure 5A , and it is possible to judge whether there is 120Hz flicker from the plot position of the plane shown in Figure 5B . Then, finally, a process is executed to merge the judgment results.

[0055] Based on the Figure 5C table shown, the merging process is executed. The table will be described below. In the case of 100Hz flicker, Figure 5A the judgment result is "100Hz" and Figure 5B the judgment result is "not 120Hz". Therefore, Figure 5C the lower left box of Figure 5CThe upper right box is "120Hz".

[0056] For example, in the absence of flicker and when the subject is in constant DC-generated light, since the photometric value does not change over time, AE_1≈AE_2≈AE_3≈...≈AE_12 holds. And thus, SAD_6≈SAD_5≈SAD_3≈0 holds. In other words, in a DC environment, a plot close to the origin of the plane of both is obtained, and thus Figure 5A and Figure 5B the judgment result of Figure 5A is "not 100Hz", and Figure 5B the judgment result of Figure 5C the lower right box in the table of

[0057] In addition, Figure 5C the upper left box in the table of

[0058] represents the cases where the results are "100Hz" and "120Hz". Generally, it can be considered unlikely to obtain such judgment results. However, since the subject being photographed is different across AE_1 to AE_12 due to subject movement or panning operation, etc., such results may be obtained. Therefore, in this case, a judgment of DC indicating a flicker detection error is obtained. In this way, in step S107, the system control unit 102 determines whether there is flicker and the frequency of the flicker. Figure 6 is a diagram showing an example of photometric output in the case of performing 12 accumulations for flicker detection in an environment where there is 100Hz flicker and the aperture is being opened.

[0059] In the photometric output as shown in Figure 6 AE_(n)≤AE_(n + 6) holds. In an environment with 100Hz flicker, the expected SAD_6≈0 does not hold, and SAD_6>0 holds. The higher the driving speed of the aperture 204, the larger the value for SAD_6. In other words, as shown in Figure 5A the point that should be in the area for judging as 100Hz becomes a point in the area formed by SAD_6 being greater than 0 due to the driving speed of the aperture, and thus it can be judged as not 100Hz. In an environment with 120Hz flicker, this phenomenon also occurs for SAD_5.

[0060] However, a larger amplitude of the flicker to be detected results in a larger value for SAD_3. As shown in Figures 5A to 5CAs can be seen, the boundary of the flicker determination area extends diagonally upward to the right. Therefore, even if the drive aperture 204 is driven and SAD_6 > 0 holds, the greater the amplitude of the flicker, the smaller the possibility of a determination error occurring. In other words, it can be seen that in the case of a flicker with a large amplitude, even if the lens aperture is driven at a relatively high speed, there is less likelihood of an incorrect determination.

[0061] After the flicker detection calculation in step S107 is completed, the system control unit 102 moves the process to step S109 and changes the live view display operation according to the detected flicker. In an environment where there is flicker, by setting the accumulation time for acquiring an image to an integer multiple of the flicker period, line flicker in the captured image can be prevented. Therefore, in step S109, the system control unit 102 controls the accumulation time of the image for the live view display according to the flicker detection result.

[0062] In steps S110 to S111, the system control unit 102 calculates the amplitude of the flicker and determines the maximum value V_max of the drive speed of the aperture 204 for the next accumulation for flicker detection according to the calculation result.

[0063] As described above, in the case of a flicker with a large amplitude, even if the lens aperture is driven at a relatively high speed, there is less likelihood of an incorrect determination. Therefore, from Figure 4A and Figure 4B the difference between the maximum value and the minimum value of AE_1 to AE_12 is used to obtain the amplitude of the flicker, and V_max is determined according to the obtained amplitude. Then, the process returns to step S103, and the system control unit 102 repeats the live view display. Therefore, in an environment where there is a flicker with a large amplitude, the maximum value V_max of the drive speed of the aperture 204 can be set to a relatively large value. Therefore, unnecessary restrictions on the drive speed of the aperture are avoided, and detection errors can be prevented even in an environment where there is a flicker with a small amplitude.

[0064] Second Embodiment

[0065] Next, the flicker detection operation during the live view of the camera according to the second embodiment will be described using Figure 7 Note that the structure of the camera body 100 and the driving method of the image sensor 101 in the live view mode are similar to those described in the first embodiment using Figure 1 and Figure 2 and thus their description is omitted.

[0066] In the Figure 3In the flowchart, the amplitude of the flicker is detected, and the maximum value V_max of the driving speed of the aperture is updated according to the amplitude. In this embodiment, more simply, a fixed predetermined value is set as V_max regardless of the amplitude of the flicker. In other words, in steps S201 to S209 showing this embodiment, Figure 7 steps S102, S110, and S111 are omitted, and V_max is a fixed value. Thus, a good balance between the flicker detection accuracy and the exposure control of the live view display can be achieved via a very simple control. Figure 3

[0067] Third Embodiment

[0068] Next, the flicker detection operation during the live view of the camera according to the third embodiment will be described using Figure 8 . Note that the structure of the camera body 100 and the driving method of the image sensor 101 in the live view mode are similar to those described using Figure 1 and Figure 2 in the first embodiment, and thus their descriptions are omitted.

[0069] During the live view, it is rare for the aperture 204 to continuously move at a high speed. Therefore, it is possible to discard only the flicker detection result when the aperture is moving at a high speed (since it has low reliability), and instead, the period T of the flicker detection can be shortened without generally causing major problems. In the third embodiment, based on this inference, the flicker detection operation as shown in Figure 8 is performed.

[0070] Steps S301 and S303 are similar to steps S101 and S103. In step S304, the system control unit 102 starts monitoring the driving speed V of the aperture. In steps S305 and S307, the cumulative sum calculation for flicker detection is performed as in steps S105 and S107. Then, in step S308, the system control unit 102 determines the monitored driving speed of the aperture. In other words, during the cumulative time period for flicker detection in step S305, it is determined whether the monitored driving speed V of the aperture is a speed higher than a predetermined fixed threshold V_th. Then, when the driving speed V is a speed higher than the threshold V_th, it is determined that the flicker detection result in step S307 is unreliable, and the process returns to step S303. On the other hand, when V is equal to or less than the threshold V_th (equal to or less than the threshold), the system control unit 102 uses (selects) the flicker detection result in step S307, and controls the live view display in step S309 according to this result. Then, the process returns to step S303.

[0071] ​Accordingly, by evaluating the reliability of the flicker detection result based on the aperture driving speed during the cumulative time period for flicker detection in step S305, flicker detection errors can be suppressed.

[0072] Fourth Embodiment

[0073] Next, Figure 9 will be used to describe the flicker detection operation during live view of the camera according to the fourth embodiment. Note that the structure of the camera body 100 and the driving method of the image sensor 101 in the live view mode are similar to those described using Figure 1 and Figure 2 in the first embodiment, and thus their descriptions are omitted.

[0074] Steps S401 and S403 are similar to steps S101 and S103. In step S404, the system control unit 102 starts monitoring the aperture position of the aperture 204, and in step S405, an image for flicker detection is acquired. For example, in step S405, while driving the aperture 204 in the opening direction, 12 times of accumulation and readout are performed to obtain photometric values AE_1 to AE_12 as shown in Figure 6 . However, since the monitoring of the aperture position starts in step S404, the aperture positions at time points t_1 to t_12 corresponding to the time of the 12 times of accumulation can be identified. Additionally, in step S406, based on this, the photometric values AE_1 to AE_12 can be corrected. For example, by using the aperture position at t_1 as a reference and adding the difference in the amount of light calculated from the relative positions of the aperture at time points t_2 to t_12 as it is to AE_2 to AE_12, the values of AE_1 to AE_12 from which the influence of the aperture driving is removed can be obtained. As a result, even in the case of values such as those shown in Figure 6 , the actually obtained AE_1 to AE_12 can be corrected to photometric values such as those of AE_1 to AE_12 shown in Figure 4A through correction based on the aperture position.

[0075] Steps S407 and S409 are similar to steps S107 and S109, and thus their descriptions are omitted.

[0076] According to the above operation, in this embodiment, even when the aperture is driven during the accumulation of the image for flicker detection, a good balance can be achieved between the flicker detection performance and the exposure control of the live view display.

[0077] Fifth Embodiment

[0078] In the first to fourth embodiments, SAD_6, SAD_5, and SAD_3 are calculated from 12 images for flicker detection obtained at intervals of 1.66 ms, and flicker detection is performed using their evaluation values. For ease of explanation, this is referred to as the first flicker detection method.

[0079] In this embodiment, in addition to the first flicker detection method, a second flicker detection method that is relatively resistant to interference during the accumulation period for flicker detection is also used, and during the accumulation period for flicker detection, the flicker detection method to be used is switched according to the driving speed of the aperture 204.

[0080] The second flicker detection method will now be described. In the second flicker detection method, the images for live view display are used as they are for flicker detection. In the images captured by a rolling shutter type image sensor, the timings of the start and end of exposure are different for each horizontal line. In other words, in an environment where there is flicker, images with different brightnesses for each horizontal line are obtained, and a stripe pattern is observed. In order to extract the flicker component from the stripe pattern, a signal component that is stable in the time direction is extracted via the following formula.

[0081] mem = ave × k + mout × (1 - k)

[0082] Here, mem is the value stored in the memory as the output of the formula, ave is the signal value of the color component of each row in the input image, k is the filter coefficient of the cyclic low-pass filter, and mout is the calculation result of the formula when calculating the signal value of the image in the previous 1 frame.

[0083] The signal component that is stable in the time direction can be extracted by performing the above calculation for each horizontal line of the input image. The flicker component (the level change component of the input image signal) is calculated by dividing the signal value of the horizontal line of the new image by the extracted stable signal component. A flicker model of the change characteristic of the signal level in the vertical direction is generated from the calculated flicker component. The flicker model can be approximated as a periodic function having a specific amplitude w, frequency f, and phase θ in the vertical direction. Since the voltage change in the AC power supply driving the light source has the property of a trigonometric function, a sine wave (or cosine wave) is usually used as the periodic function for modeling, but other periodic ideal functions can also be used. The frequency f is determined using the frame rate and the power frequency of the light source. In addition, regarding the phase θ, the phase of each row can be calculated by setting the row in the detected change component where the change rate is 1 and the change amount of the change rate in the vertical direction is 1 such that θ = 0. Furthermore, the amplitude w is calculated from the change ratios of π / 2 and 3π / 2 of the calculated phase.

[0084] Compared with the first flicker detection method, this method emphasizes frequency components more and has the characteristic of being able to detect in a stable manner that is more adaptable to the brightness change caused by driving the aperture 204. However, since the information of the stripes caused by using the rolling shutter is used, it is difficult to find flicker only in a part of the screen. In addition, in order to obtain stable detection results, an image with multiple frames is required, which means that the characteristic of the method is that it takes a long time to perform one detection.

[0085] However, the first flicker detection method uses the photometric value of the reduced target area. Therefore, flicker detection can only be performed in one area of the screen. In addition, the amount of time spent on detection is only the amount of time required to capture 12 images at an interval of 1.66 ms, that is, a very short time of about 20 ms. In the fifth embodiment, the first flicker detection method and the second flicker detection method with different characteristics are used according to the situation.

[0086] Figure 10 It is a flowchart showing the operation of this embodiment. Steps S501 and S509 are similar to steps S301 and S309 of the third embodiment. However, in step S308, when the aperture driving speed V during the accumulation period for flicker detection is a speed higher than the predetermined fixed threshold V_th, the detection result is discarded in the third embodiment. Instead, in this embodiment, even when the aperture driving speed V is a speed higher than the threshold V_th, flicker detection unaffected by the aperture driving can be achieved by performing the second flicker detection in step S510.

[0087] Other embodiments

[0088] Embodiments of the present invention can also be implemented by the following method, that is, by providing software (program) that executes the functions of the above embodiments to a system or device through a network or various storage media, and the computer or central processing unit (CPU), microprocessing unit (MPU) of the system or device reads and executes the program.

[0089] Although the present invention has been described with reference to exemplary embodiments, it should be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the appended claims will be given the broadest interpretation to cover all such modifications and equivalent structures and functions.

Claims

1. An imaging device, comprising: an image sensor configured to capture a subject image formed via a lens and output an image; a detection component configured to detect flicker from the image output from the image sensor; and a control component configured to, when the image sensor captures an image of a subject during an imaging period for flicker detection and the aperture of the lens is driven, limit a driving speed of the aperture to not exceed a predetermined maximum value during the imaging period for the flicker detection, and cancel the limitation after completion of the imaging for the flicker detection.

2. The imaging device according to claim 1, wherein the detection component detects the presence or absence of flicker and a frequency of the flicker.

3. The imaging device according to claim 1, wherein the control component changes the predetermined maximum value according to a flicker amplitude.

4. The imaging device according to claim 1, wherein the control component sets the predetermined maximum value to a fixed value.

5. An imaging device, comprising: an image sensor configured to capture a subject image formed via a lens and output an image; a detection component configured to detect flicker from the image output from the image sensor; and a control component configured to change a method for detecting flicker via the detection component during an imaging period in which the image sensor captures an image of a subject for flicker detection and a driving operation of the lens for changing an amount of light received by the image sensor is performed, wherein the control component switches between a first method for detecting flicker and a second method for detecting flicker, in the first method, multiple images for flicker detection are used to detect flicker, and in the second method, an image for live view is used to detect flicker.

6. The imaging device according to claim 5, wherein the detection component detects the presence or absence of flicker and a frequency of the flicker.

7. The imaging device according to claim 5, wherein the driving operation causes at least one of driving of an aperture in the lens, driving of a focusing lens, and driving of a zoom lens to be performed.

8. The imaging device according to claim 7, wherein the control component determines a reliability of a flicker detection result from the detection component based on a driving speed of the aperture of the lens, and does not use a detection result with low reliability.

9. The imaging device according to claim 8, wherein when the driving speed of the aperture of the lens is higher than a predetermined threshold, the control component determines the reliability to be low and does not use a flicker detection result using an image captured during a period in which the driving speed of the aperture of the lens is higher than the predetermined threshold.

10. The imaging device according to claim 7, wherein the control component monitors a position of the aperture of the lens and corrects a flicker detection result from the detection component based on the position of the aperture.

11. The imaging device according to claim 5, wherein The control component selects the first method when the driving speed of the aperture of the lens is equal to or less than a predetermined threshold, and selects the second method when the driving speed is higher than the predetermined threshold.

12. A control method for an imaging device, the imaging device being provided with an image sensor and a detection component, the image sensor being configured to capture a subject image formed via a lens and output an image, the detection component being configured to detect flicker from the image output from the image sensor, the control method comprises: When driving the aperture of the lens during an imaging period in which the image sensor captures an image of a subject for flicker detection, limiting the driving speed of the aperture to not exceed a predetermined maximum value during the imaging period for the flicker detection, and canceling the limitation after completion of the imaging for the flicker detection.

13. A control method for an imaging device, the imaging device being provided with an image sensor and a detection component, the image sensor being configured to capture a subject image formed via a lens and output an image, the detection component being configured to detect flicker from the image output from the image sensor, the control method comprises: When performing a driving operation of the lens for changing the amount of light received by the image sensor during an imaging period in which the image sensor captures an image of a subject for flicker detection, controlling to change a method for detecting flicker via the detection component, wherein, in the control, the method for detecting flicker is switched between a first method for detecting flicker and a second method for detecting flicker. In the first method, a plurality of images for flicker detection are used to detect flicker, and in the second method, an image for live view is used to detect flicker.

14. A computer-readable storage medium storing a program for causing a computer to execute the control method according to claim 12 or 13.

15. A computer program product comprising a program for causing a computer to execute the control method according to claim 12 or 13.

Images