Imaging device, control method thereof, program, and storage medium

Abstract

To provide an imaging device capable of suppressing deterioration in detection accuracy when flicker is detected during live view display.SOLUTION: An imaging device includes an imaging element that captures a subject image formed by a lens and outputs an image, a detection unit that detects flicker from the image output from the imaging element, and a control unit that adjusts driving of the lens during an imaging period in which the subject is imaged in order to detect flicker with the imaging element when the lens is driven in accordance with a driving operation that causes a change in the amount of light received by the imaging element.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a technique for detecting flicker in an imaging device. [Background technology]

[0002] In recent years, digital cameras have become higher ISO (higher sensitivity) and can now shoot with a high shutter speed even under artificial light sources that cause flicker. While a high shutter speed has the advantage of being able to take blur-free photos when shooting indoor sports, etc., when shooting under a flickering light source, the flicker can cause uneven exposure and color in the image, either from frame to frame or even within a single frame.

[0003] To address this problem, a method is known in which the effects of flicker are reduced by detecting flicker and performing exposure at the peak position of the flicker, where the change in brightness is the smallest.

[0004] In order to use a method for reducing the effect of such flicker, it is necessary to detect the flicker and its frequency. Regarding the detection of the flicker and its frequency, for example, a technique disclosed in Patent Document 1 is known. Patent Document 1 shows an example in which photometry is performed multiple times at a constant cycle, and the presence or absence of flicker and its frequency are determined from a first evaluation value obtained from photometric values ​​acquired at a first interval close to the same phase as the flicker and a second evaluation value obtained from photometric values ​​acquired at a second interval close to the opposite phase of the flicker among the multiple photometric values ​​obtained.

[0005] Moreover, Patent Document 2 discloses an imaging device that makes it possible to carry out a flicker detection operation during so-called live view display, without stopping the live view display. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2017-11352 A [Patent Document 2] JP 2020-80512 A Summary of the Invention [Problem to be solved by the invention]

[0007] However, the conventional technology disclosed in the above-mentioned Patent Document 1 is premised on the fact that the exposure conditions such as the aperture do not change when exposing an image for flicker detection, and only the brightness of the subject due to the flicker light source changes. If this precondition is not met, the performance of flicker detection will be degraded.

[0008] On the other hand, in the technology disclosed in Patent Document 2, it is necessary to constantly control the exposure and focus to a state that is preferable for the user during live view display. Therefore, if an attempt is made to perform a flicker detection operation during live view display, there is a problem that the exposure conditions of the image for flicker detection change due to the influence of changes in exposure and focus during live view display.

[0009] The present invention has been made in consideration of the above-mentioned problems, and has as its object to provide an imaging device that can suppress a decrease in detection accuracy when detecting flicker during live view display. [Means for solving the problem]

[0010] The imaging device according to the present invention includes an imaging element that captures an image of a subject formed by a lens and outputs the image, a detection unit that detects flicker from the image output from the imaging element, and during an imaging period in which the imaging element captures an image of the subject to detect flicker, Aperture and a control means for limiting the drive speed of the diaphragm to a predetermined value or less during an image capturing period for detecting flicker when the diaphragm is driven, and for releasing the limit after the image capturing for detecting flicker is completed. Effect of the Invention

[0011] According to the present invention, it is possible to provide an imaging device that can suppress a decrease in detection accuracy when detecting flicker during live view display. [Brief description of the drawings]

[0012] [Figure 1] 1 is a diagram showing a schematic configuration of a digital mirrorless camera that is a first embodiment of an imaging apparatus of the present invention. [Diagram 2] 4A and 4B are diagrams for explaining driving of an image sensor during live view. [Diagram 3] 5 is a flowchart showing a flicker detection operation during live view in the first embodiment. [Figure 4] 6 is a diagram showing the relationship between the timing of acquiring an image for flicker detection and a photometric output. [Diagram 5] 1 is a diagram showing a method for determining the presence of flicker. [Figure 6] 6A and 6B are diagrams showing photometric outputs when the aperture is driven during accumulation of an image for flicker detection. [Figure 7] 10 is a flowchart showing a flicker detection operation during live view in the second embodiment. [Figure 8] 13 is a flowchart showing a flicker detection operation during live view in the third embodiment. [Figure 9] 5 is a flowchart showing a flicker detection operation during live view in the first embodiment. [Figure 10] 5 is a flowchart showing a flicker detection operation during live view in the first embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0014] The present invention addresses the problem that imaging control operations (e.g., lens aperture drive, focusing lens drive, zoom lens drive) that change the amount of light taken in from the lens (amount of light received by the imaging element) are disturbances to flicker detection. In the following explanation, in all of the embodiments, an example is described in which lens aperture drive affects flicker detection as a representative of the above imaging control operations (driving operations of lens components). However, this is for the sake of clarity, and in reality, the present invention is not limited to lens aperture drive, and a similar technique can be used for other imaging control operations.

[0015] In addition, in each embodiment, a so-called digital mirrorless camera is taken as an example of an imaging device to be described. However, the present invention is not limited to this, and may be various electronic devices equipped with a camera function. For example, the imaging device according to the present invention may be a mobile communication terminal with a camera function such as a mobile phone or a smartphone, a mobile computer with a camera function, or a mobile game machine with a camera function.

[0016] (First embodiment) FIG. 1 is a diagram showing a schematic configuration of a digital mirrorless camera (hereinafter, simply referred to as a camera) that is a first embodiment of an imaging device of the present invention.

[0017] 1, the camera is broadly composed of a camera body 100 and a photographing lens 200, which are mechanically and electrically connected via a lens mount mechanism 110. In this embodiment, the photographing lens 200 that forms a subject image is configured to be detachable from the camera body 100, but may also be integrated with the camera body 100.

[0018] First, the camera body 100 will be described. The imaging element 101 is an image sensor including an infrared cut filter, a low-pass filter, etc., and equipped with a photoelectric conversion element such as a CMOS type. The optical image of the subject is photoelectrically converted by the imaging element 101 and output as an image signal. The shutter 104 closes to block light from the imaging element 101 when not shooting, and opens during live view (LV) or shooting to guide incident light that has passed through the shooting lens 200 to the imaging element 101.

[0019] The system control unit 102 is an arithmetic processing unit that controls each unit of the camera body 100. The memory 103 includes a ROM that stores programs and variables executed by the system control unit 102, and a RAM that has a working area in which the system control unit 102 expands programs and a storage area for temporary image data, etc.

[0020] The system control unit 102 is also connected to a lens control unit 201 through a lens mount mechanism 110. The lens control unit 201 controls the position of a focusing lens 202 through a lens driving unit 203. It also controls an aperture 204 through an aperture control unit 205. More specifically, the system control unit 102 performs calculations related to AE / AF from an image acquired by the image sensor 101, and communicates the position of the focusing lens and the aperture value according to the calculation results to the lens control unit 201 for control.

[0021] The shutter 104 is also connected to the system control unit 102 via a shutter control unit 105 , and exposes the image sensor 101 for an exposure time according to the results of calculations in the system control unit 102 .

[0022] Next, we will explain the operation of displaying images acquired by the image sensor 101. Here, we will explain the operation in a so-called live view mode in which the image sensor 101 continuously captures images and displays the captured images in real time on the display unit, allowing the user to observe a subject.

[0023] There are two display destinations for the image captured by the image sensor 101: rear monitor 106 and viewfinder display unit 107, and the image is displayed by switching between these. The switching is performed based on the detection result of eyepiece detection unit 109. Specifically, if it is detected that the user has their eye in close contact, the image is displayed on the viewfinder display unit 107. The user looks into the viewfinder and checks the subject displayed on the viewfinder display unit 107 through eyepiece lens 108. On the other hand, if eyepiece detection unit 109 detects that the user has not placed their eye in close contact, the image is displayed on the rear monitor 106.

[0024] Next, the driving of the image sensor 101 in live view mode will be described with reference to Fig. 2. In Fig. 2, the horizontal direction indicates time, the vertical direction indicates the vertical position of the image sensor 101, and the diagonal lines in the figure indicate the readout timing of the image sensor 101 in live view mode. As shown in Fig. 2, first one image for live view display is read out, and then multiple images for flicker detection are read out (imaging period of images for flicker detection). The acquisition of this image for live view and multiple images for flicker detection constitutes one set, and this set of operations is repeated.

[0025] A plurality of images for flicker detection are captured at intervals of 1.66 [ms]. One set is repeated at a cycle of T [ms], which is the frame rate of the live view display, and is 1 / T [fps]. How many images for flicker detection can be acquired in one set depends on the cycle T [ms] of one set, and the longer T [ms] is, the more images for flicker detection can be acquired. In this embodiment, the cycle of one set (frame rate of the live view display) is set to 33.33 ms (30 fps), and the number of images for flicker detection acquired in one set is set to 18.

[0026] Next, a flicker detection operation during live view in the camera of this embodiment will be described with reference to Fig. 3. The operation of the flowchart in Fig. 3 is realized by the system control unit 102 expanding a program stored in the ROM of the memory 103 into the RAM and executing it. Note that the operations of the flowcharts in Figs. 7 to 10, which will be described later, are also realized by the system control unit 102 executing a program in the memory 103 in a similar manner.

[0027] Upon receiving an instruction to start a live view operation in step S101, the system control unit 102 first sets the upper limit speed V_max of the drive speed of the aperture 204 to v0 when performing accumulation for flicker detection in step S102. Although details will be described later, in general, when exposing an image for flicker detection, if the amount of light taken in from the lens changes due to aperture drive, the flicker detection performance decreases. The degree of performance degradation increases as the aperture drive speed increases. Therefore, in step S102, the upper limit value V_max of the drive speed of the aperture 204 is set for the first flicker detection operation after the start of live view.

[0028] Usually, the state of flicker in a shooting environment is determined by the illumination light in that environment, so it is unlikely that the state of flicker, such as the presence or absence of flicker and its frequency, will change rapidly. Therefore, it is desirable to set V_max to a relatively slow speed in order to perform reliable detection for the first flicker detection. The upper limit of the aperture drive speed at the time of this first flicker detection is set to v0. v0 is a predetermined value, but if v0=0, for example, the drive of the aperture 204 will stop when flicker is detected, and highly accurate flicker detection can be realized. After setting V_max, the system control unit 102 proceeds to step S103.

[0029] Step S103 is a step of waiting for the timing to perform flicker detection. As already described, since it is unlikely that the flicker state will change rapidly, performing the flicker detection operation for every frame is excessive in terms of the computation load, etc., and it is considered that the flicker detection operation should be performed every time a predetermined period has passed. Therefore, in this embodiment, the system control unit 102 performs flicker detection after live view is started, and then repeats flicker detection every time T seconds have passed. For example, it is conceivable to set T=1 and perform flicker detection every second. In step S103, the system control unit 102 determines whether or not this flicker detection timing has come. Then, the system control unit 102 proceeds to step S104 where the flicker detection operation is performed only when the flicker detection timing has come, and otherwise waits while continuing the normal live view display.

[0030] Steps S104 to S107 are steps for actually performing the flicker detection operation. First, in step S104, the system control unit 102 limits (adjusts) the drive speed of the diaphragm 204 to an upper limit speed V_max or less (upper limit value or less) when storing an image for flicker detection. After the diaphragm drive speed is limited, in step S105, the system control unit 102 causes the image sensor 101 to perform storage for flicker detection.

[0031] FIG. 4(a) is a diagram showing a change in signal value over time when accumulation for flicker detection is performed and a 100 Hz flicker is present. As already shown in FIG. 2, in this embodiment, after reading out an image for live view display, the image sensor 101 is driven by reading out an image for flicker detection 18 times at 1.66 ms intervals as one set. For flicker detection, the first 12 images out of the 18 images captured at 1.66 ms intervals are used. Accumulation and readout are performed for each of the 12 images, and as shown in the figure, the nth accumulation is described as "accumulation n", the signal readout of accumulation n is described as "readout n", and the photometric value (signal value) obtained from the result of readout n is described as "AE(n)". Here, the first 12 accumulations are focused on, so photometric values ​​AE(1) to AE(12) are obtained. Furthermore, because accumulation is performed over a finite period of time, the acquisition times of the photometric values ​​are represented by the median values ​​during the accumulation period (the center of gravity of the parallelogram in the drawing), and are set to t(1) to t(12), respectively. In step S105, the system control unit 102 causes the image sensor 101 to acquire images for flicker detection at intervals of 1.66 ms in this manner.

[0032] After acquiring the image for flicker detection in step S105, in step S106, the system control unit 102 releases the restriction set in step S104 on the drive speed of the diaphragm 204. Then, in step S107, the system control unit 102 performs flicker detection calculation using AE(1) to AE(12).

[0033] In the flicker detection calculation, the system control unit 102 first calculates an evaluation value used in determining the frequency of the flicker from AE(1) to AE(12). In this embodiment, the evaluation value used in determining the flicker frequency is defined by the following equation.

[0034]

number

[0035] SAD is an abbreviation for Sum of Absolute Difference, and is an index of similarity used in fields such as pattern matching. m is a value indicating how many photometric values ​​from the nth photometric result AE(n) out of 12 photometric measurements are to be used to calculate the similarity with the photometric value after (1.667×m) ms have elapsed, so SAD(m) is a formula for calculating the similarity with the photometric value after (1.667×m) ms have elapsed. As can be seen from the formula, the higher the similarity, the smaller the SAD(m) value.

[0036] For example, in an environment where a 100 Hz flicker exists, the flicker period is about 10 ms, and the relationship with the photometric period of 1.66 ms is 10÷1.66≒6, so as shown in FIG. 4(a), the same photometric value is obtained in six periods regardless of the accumulation timing. In other words, the relationship is AE(n)≒AE(n+6). Due to this property, when SAD(6) is calculated in an environment where a 100 Hz flicker exists, SAD(6)≒0. Furthermore, in order to detect the presence of a 100 Hz flicker, SAD(3) is also calculated. SAD(3) is a value calculated as the similarity with the photometric value after 1.667×3=5 ms has elapsed. In an environment where a 100 Hz flicker exists, the photometric value at a timing shifted by 5 ms has an inverse phase relationship, so SAD(3) is a very large value compared to SAD(6). In other words, when SAD(3) is large and SAD(6) is small, it is believed that 100 Hz flicker is present.

[0037] Using the same concept, in an environment where 120 Hz flicker exists, SAD(5) and SAD(3) can be calculated. In an environment where 120 Hz flicker exists, the light source lighting period is 8.333 ms, so AE(n) ≒ AE(n+5), and SAD(5) ≒ 0, as shown in Figure 4(b). In addition, in a 120 Hz flicker, the completely opposite phase relationship occurs after 4.16 ms has elapsed, so it is ideal to determine the similarity with the waveform after 4.16 ms has elapsed. However, since 4.16 ms is not an integer multiple of the frame period of 1.667 ms, the value of SAD(3), which indicates the similarity with the waveform after 5 ms has elapsed, is used instead as a relatively close value. Even in an environment where 120 Hz flicker exists, SAD(3) indicates a similarity close to the opposite phase, so SAD(3) has a very large value compared to SAD(5).

[0038] From the above, SAD(6), SAD(5), and SAD(3) are calculated, and the final flicker frequency determination is performed using these. As already explained, in an environment where a 100 Hz flicker exists, SAD(3) will be a much larger value than SAD(6). Therefore, if we consider a plane as shown in FIG. 5(a) with SAD(3) on the horizontal axis and SAD(6) on the vertical axis, in an environment where a 100 Hz flicker exists, a plot will be obtained in the relatively lower right area of ​​this plane. In other words, if an area determined to be 100 Hz and an area determined not to be 100 Hz are set using the area division as shown in FIG. 5(a), it is possible to accurately determine whether or not the flicker is 100 Hz based on the position of the plot.

[0039] Similarly, by dividing the plane shown in FIG. 5(b) with SAD(3) on the horizontal axis and SAD(5) on the vertical axis, it is possible to determine whether or not there is a 120 Hz flicker.

[0040] It should be noted that the area dividing lines shown in FIGS. 5(a) and 5(b) are merely examples, and the inclinations and bending points of the lines are not limited to these.

[0041] The presence or absence of 100 Hz flicker can be determined from the plot positions on the plane shown in FIG. 5(a), and the presence or absence of 120 Hz flicker can be determined from the plot positions on the plane shown in FIG. 5(b). Finally, a process is performed to combine these determination results.

[0042] The integration process is performed based on the table shown in Figure 5(c). This table will be explained below. If 100 Hz flicker is present, the judgment result in Figure 5(a) will be "100 Hz," and the judgment result in Figure 5(b) will be "not 120 Hz," so the box in the lower left of Figure 5(c) will be "100 Hz." Using the same concept, the box in the upper right of Figure 5(c) will be "120 Hz."

[0043] If there is no flicker and the subject is constant DC light, the photometric value does not change over time. AE(1)≒AE(2)≒AE(3)≒ … ≒AE(12) This results in SAD(6) ≒ SAD(5) ≒ SAD(3) ≒ 0 In other words, in a DC environment, plots are obtained near the origin of both the planes in Figure 5(a) and Figure 5(b), and the judgment result in Figure 5(a) is "not 100Hz," and the judgment result in Figure 5(b) is "not 120Hz." Therefore, the box in the bottom right of the table in Figure 5(c) is a DC judgment with no change in luminance.

[0044] Moreover, the box in the upper left of the table in FIG. 5(c) is a case where it is both "100 Hz" and "120 Hz." Normally, it is unlikely that such a judgment result would be obtained, but if the subject being acquired from AE(1) to AE(12) is not the same due to subject movement or panning, etc., such a result may occur. Therefore, in such a case, a DC judgment is made, which means a flicker detection error. In this way, in step S107, system control unit 102 judges the presence or absence of flicker and its frequency.

[0045] The above describes the flicker detection calculation when an ideal flicker is present, but next consider the case where the diaphragm 204 is driven and opens while an image for flicker detection is being accumulated. Fig. 6 is a diagram showing an example of the photometric output when accumulation for flicker detection is performed 12 times in an environment where the diaphragm is opening and 100 Hz flicker is present.

[0046] In the photometric output shown in Fig. 6, AE(n) ≦ AE(n + 6), and SAD(6) is not ≈ 0 as expected in an environment where 100 Hz flicker exists, but SAD(6) > 0, and the faster the speed at which the aperture 204 is driven, the larger the SAD(6) value becomes. In other words, a point that should be in the area determined to be 100 Hz in Fig. 5(a) may be in the area where SAD(6) > 0 depending on the aperture drive speed, and may be determined not to be 100 Hz. This phenomenon also occurs with respect to SAD(5) in an environment where 120 Hz flicker exists.

[0047] On the other hand, the larger the amplitude of the flicker to be detected, the larger the value of SAD(3). In Fig. 5, the boundary of the flicker determination region slopes upward to the right, and it can be seen that the larger the amplitude of the flicker, the smaller the probability of erroneous determination even if the aperture 204 is driven to make SAD(6)>0. In other words, it can be seen that with large amplitude flicker, erroneous determination is unlikely to occur even if the lens aperture is driven relatively quickly.

[0048] After completing the flicker detection calculation in step S107, the system control unit 102 advances the process to step S109, where it changes the display operation of the live view in accordance with the detected flicker. In a flicker environment, it is possible to prevent line flicker from occurring in the captured image by setting the accumulation time of the acquired image to an integer multiple of the flicker period. Therefore, in step S109, the system control unit 102 controls the accumulation time of the image for displaying the live view in accordance with the flicker detection result.

[0049] In steps S110 to S111, the system control unit 102 calculates the amplitude of the flicker, and determines the upper limit V_max of the drive speed of the diaphragm 204 in the next accumulation for flicker detection according to the calculation result.

[0050] As already explained, if the flicker has a large amplitude, erroneous detection is unlikely to occur even if the drive speed of the aperture 204 is made relatively fast. Therefore, the amplitude of the flicker is calculated from the difference between the MAX value and the MIN value of AE(1) to AE(12) in FIG. 4, and V_max is determined according to the calculated amplitude. Then, the process returns to step S103, and the system control unit 102 repeats the live view display. This makes it possible to set the maximum value V_max of the drive speed of the aperture 204 to a relatively large value in an environment with large amplitude flicker. Therefore, it is possible to avoid restricting the drive speed of the aperture more than necessary, and also to prevent erroneous detection in an environment with small amplitude flicker.

[0051] Second embodiment Next, the flicker detection operation during live view of the camera in the second embodiment will be described with reference to Fig. 7. Note that the configuration of the camera 100 and the driving method of the image sensor 101 in live view mode are the same as those in Fig. 1 and Fig. 2 in the first embodiment, and therefore the description will be omitted.

[0052] In the flowchart of FIG. 3 in the first embodiment, the amplitude of the flicker is detected, and the upper limit V_max of the aperture drive speed is updated according to the amplitude. In this embodiment, more simply, a fixed, predetermined value is set to V_max regardless of the amplitude of the flicker. That is, in steps S201 to S209 in FIG. 7 showing this embodiment, steps S102, S110, and S111 in FIG. 3 are omitted, and V_max is set to a fixed value. This makes it possible to achieve both high flicker detection accuracy and exposure control of live view display with very simple control.

[0053] (Third embodiment) The flicker detection operation during live view of the camera in the third embodiment will be described with reference to Fig. 8. Note that the configuration of the camera 100 and the driving method of the image sensor 101 in live view mode are similar to those in Fig. 1 and Fig. 2 in the first embodiment, and therefore the description will be omitted.

[0054] During live view, it is rare for the aperture 204 to always move at high speed. Therefore, if only the flicker detection results when the aperture is moving at high speed are deemed to be of low reliability and discarded, and instead the flicker detection cycle T is set to a short value, it can be considered that there is no major problem overall. Based on this concept, in the third embodiment, a flicker detection operation is performed as shown in FIG.

[0055] Steps S301 and S303 are the same as steps S101 and S103. In step S304, the system control unit 102 starts monitoring the drive speed V of the aperture. In steps S305 and S307, accumulation and calculation for flicker detection are performed in the same manner as in steps S105 and S107. After that, in step S308, the system control unit 102 judges the drive speed of the aperture that has been monitored. That is, during the accumulation period for flicker detection in step S305, it is judged whether the drive speed V of the aperture that has been monitored is faster than a predetermined fixed threshold value V_th. If the drive speed V is faster than the threshold value V_th, the flicker detection result in step S307 is deemed unreliable, and the process returns to step S303. On the other hand, if V is equal to or less than the threshold value V_th (threshold value or less), the system control unit 102 adopts (selects) the flicker detection result in step S307, and controls the live view display accordingly in step S309. Then, the process returns to step S303.

[0056] In this way, by evaluating the reliability of the flicker detection result according to the aperture drive speed during the accumulation period for flicker detection in step S305, erroneous detection of flicker can be suppressed.

[0057] (Fourth embodiment) The flicker detection operation during live view of the camera in the fourth embodiment will be described with reference to Fig. 9. Note that the configuration of the camera 100 and the driving method of the image sensor 101 in live view mode are similar to those in Fig. 1 and Fig. 2 in the first embodiment, and therefore the description will be omitted.

[0058] Steps S401 and S403 are the same as steps S101 and S103. In step S404, the system control unit 102 starts monitoring the aperture position of the aperture 204, and in step S405, an image for flicker detection is acquired. In step S405, for example, accumulation and readout are performed 12 times while driving the aperture 204 in the opening direction, and photometric values ​​AE(1) to AE(12) as shown in FIG. 6 are obtained. However, since the aperture position monitoring is started in step S404, it is possible to specify the aperture position at each of t(1) to t(12), which are the times when the 12 accumulations were performed. Then, based on this, it is possible to correct the photometric values ​​AE(1) to AE(12). For example, if the aperture position at t(1) is used as a reference and the light amount difference calculated from the relative positions of the aperture at t(2) to t(12) is directly added to AE(2) to AE(12), it is possible to obtain values ​​of AE(1) to AE(12) that exclude the influence of the aperture drive. As a result, even if the actually obtained values ​​AE(1) to AE(12) are as shown in FIG. 6, they can be corrected to the photometric values ​​AE(1) to AE(12) in FIG. 4(a) by performing correction based on the aperture position.

[0059] Steps S407 and S409 are similar to steps S107 and S109, and therefore the description thereof will be omitted.

[0060] By the above operation, in this embodiment, even if the diaphragm is driven while storing the flicker detection image, it is possible to achieve both flicker detection performance and exposure control for live view display.

[0061] Fifth embodiment In the first to fourth embodiments, SAD(6), SAD(5), and SAD(3) were calculated from 12 flicker detection images acquired at 1.66 ms intervals, and flicker detection was performed using these evaluation values. For convenience of explanation, this will be called the first flicker detection method.

[0062] In this embodiment, in addition to the first flicker detection method, a second flicker detection method that is relatively resistant to disturbances during accumulation for flicker detection is also used, and the flicker detection method to be used is switched depending on the drive speed of the aperture 204 during accumulation for flicker detection.

[0063] Here, the second flicker detection method will be described. In the second flicker detection method, the image for live view display itself is used to detect flicker. In an image captured by a rolling shutter type image sensor, the timing of exposure start and end differs for each horizontal line. In other words, in a flicker environment, an image with different brightness for each horizontal line is obtained, which is observed as a striped pattern. In order to extract the flicker component from this striped pattern, the stationary signal component in the time direction is extracted using the following formula.

[0064] mem = ave × k + mout × (1-k) Here, mem is the value stored in memory as the output of the above equation, ave is the signal value of the color component of each row in the input image, k is the filter coefficient of the recursive low-pass filter, and mout is the result of the above equation calculated when the signal value of the image one frame previous is input.

[0065] By performing the above calculation for each horizontal line of the input image, a stationary signal component in the time direction can be extracted. A flicker component (a level fluctuation component of the input image signal) is calculated by dividing the signal value of a horizontal line of a newly input image by the extracted stationary signal component. A flicker model, which is a fluctuation characteristic of the signal level in the vertical direction, is generated from the calculated flicker component. The flicker model can be approximated as a periodic function having a specific amplitude w, frequency f, and phase θ in the vertical direction, for example. Since the voltage fluctuation of the AC power supply that drives the light source has the characteristics of a trigonometric function, a sine wave (or cosine wave) is generally used as the periodic function to be modeled, but other periodic ideal functions may also be used. The frequency f is determined by the frame rate and the power supply frequency of the light source. In addition, for the phase θ, the phase of each row can be calculated by setting the phase θ=0 for the row in which the fluctuation ratio of the detected fluctuation component is 1 and the change amount of the fluctuation ratio is 1 in the vertical direction. Furthermore, the amplitude w is calculated from the fluctuation ratio at π / 2 and 3π / 2 of the calculated phase.

[0066] This method focuses more on frequency components, and has the characteristic of being more resistant to changes in brightness caused by driving the aperture 204 than the first flicker detection method, and being able to detect flicker stably. However, since it uses information on stripes generated by the rolling shutter format, it is difficult to find flicker that exists only in a part of the screen. Also, since multiple frames of images are required to obtain stable detection results, it has the characteristic that it takes a long time to perform one detection.

[0067] On the other hand, the first flicker detection method can detect flicker that exists only in a part of the screen by using a photometric value limited to the area of ​​interest. Also, the time required for detection is very short, about 20 ms, since it is sufficient to take 12 images at 1.66 ms intervals. In this way, the first and second flicker detection methods, which have different characteristics, are used selectively in the fifth embodiment.

[0068] 10 is a flowchart showing the operation of this embodiment. Steps S501 to S509 are similar to steps S301 to S309 in the third embodiment. However, in step S308, if the aperture drive speed V during accumulation for flicker detection is faster than a predetermined fixed threshold value V_th, the detection result is discarded in the third embodiment. In contrast, in this embodiment, even if the aperture drive speed V is faster than the threshold value V_th, second flicker detection is performed in step S510, thereby making it possible to realize flicker detection that is not affected by aperture drive.

[0069] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0070] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0071] 101: image sensor, 102: system control unit, 103: memory, 201: lens control unit, 203: lens driving unit, 204: aperture, 205: aperture driving unit

Claims

1. an image sensor that captures an image of a subject formed by a lens and outputs the image; A detection means for detecting flicker from an image output from the imaging device; a control means for limiting a drive speed of the diaphragm to a predetermined value or less during an imaging period in which an image of a subject is captured by the image sensor in order to detect flicker, when the diaphragm is driven during the imaging period for detecting flicker, and for releasing the limit after the imaging for detecting flicker is completed; An imaging device comprising:

2. 2. The imaging apparatus according to claim 1, wherein said detection means detects the presence or absence of flicker and the frequency of the flicker.

3. 3. The imaging apparatus according to claim 1, wherein the control means changes the upper limit of the predetermined value based on the amplitude of flicker.

4. 3. The imaging apparatus according to claim 1, wherein the control means sets the upper limit of the predetermined value to a fixed value.

5. A method for controlling an imaging device including an imaging element that captures an object image formed by a lens and outputs the image, and a detection unit that detects flicker from the image output from the imaging element, comprising: A control method for an imaging device, comprising a control step of, when an aperture is driven during an imaging period in which an image of a subject is imaged by the image sensor to detect flicker, limiting a drive speed of the aperture to a predetermined value or less during the imaging period for detecting flicker, and releasing the limit after imaging for detecting flicker is completed.

6. A program for causing a computer to execute the control method according to claim 5.

7. A computer-readable storage medium storing a program for causing a computer to execute the control method according to claim 5.

Images