Imaging device, flicker detection method and program

Abstract

To effectively detect flicker irrespective of a light source, and effectively notify a user of information on the detected flicker.SOLUTION: An apparatus includes: first flicker detection means for detecting a frequency of changes in quantity of light of flicker, which is the changes in quantity of light of a subject; determination means for determining whether the frequency of changes in quantity of light of the flicker detected by the first flicker detection means exceeds a detection target range; and control means for performing control to notify of information on the flicker on the basis of the result detected by the first flicker detection means and the result determined by the determination means. The control means performs control to notify of first information that a frequency of changes in quantity light of a currently-occurring flicker exceeds the detection target range when the determination means determines that flicker whose frequency of changes in quantity of light exceeds the detection target range has been detected.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an imaging device, a flicker detection method, and a program, and more particularly to a technique for calculating characteristics related to periodic changes in the amount of light (called flicker) of a subject. [Background technology]

[0002] In recent years, the sensitivity of image sensors in imaging devices such as digital cameras and mobile phones has been increasing. As a result, it has become possible to capture bright images with reduced subject blur by setting a high shutter speed (short exposure time) to capture a subject in a relatively dark environment, such as indoors, compared to outdoor scenes during the day.

[0003] Furthermore, fluorescent lights, which are widely used as indoor light sources, are known to cause flicker, a phenomenon in which the amount of light in a subject image changes periodically due to the influence of commercial power supply frequencies. If a subject is captured using a high shutter speed under a light source that causes such flicker, uneven exposure and color may occur within a single image (frame), or variations in exposure and color temperature may occur among multiple images captured in succession.

[0004] Patent Document 1 proposes a technology for detecting flicker based on multiple images acquired continuously at a rate that is the least common multiple of the flicker frequencies (100 Hz and 120 Hz) caused by two commercial power frequencies, 50 Hz and 60 Hz. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-220763 Summary of the Invention [Problem to be solved by the invention]

[0006] In recent years, the use of light-emitting diodes (LEDs) as light sources has been increasing. LEDs use a different method of supplying current than fluorescent lamps, controlling the drive current with a rectifier circuit. As a result, the amount of light changes with a different cycle and waveform than the commercial power frequency. Therefore, flicker occurs under an LED light source, just like under a fluorescent lamp, but the frequency of the change in light amount of the flicker differs from that under light sources such as fluorescent lamps.

[0007] Patent document 1 discloses a method for detecting flicker at frequencies (100 Hz and 120 Hz) caused by two commercial power frequencies, 50 Hz and 60 Hz, such as under light sources such as fluorescent lamps, but does not mention how to deal with flicker that occurs under LED light sources.

[0008] Like flicker caused by commercial power frequency, it is difficult to estimate the light intensity change frequency of flicker that occurs under a light source such as an LED in advance. Therefore, it is difficult for a user to manually adjust the shutter speed (exposure time) in advance to reduce the effects of flicker. Furthermore, if the currently occurring flicker exceeds the detectable range of the imaging device, the flicker may not be detected correctly. In other words, the technology described in Patent Document 1 has limitations on the information about flicker that a user can obtain in advance.

[0009] An object of the present invention is to effectively detect flicker regardless of the light source and to effectively notify the user of information about the detected flicker. [Means for solving the problem]

[0010] In order to achieve the above object, an imaging device includes a detection means for detecting a flicker frequency of a light source by sampling an output signal of an imaging element using a first sampling process, a determination means for determining whether or not the flicker frequency detected by the detection means is within a detection target range by sampling the output signal of the imaging element using a second sampling process different from the first sampling process, and a control means for controlling the device to notify a user when the determination means determines that the flicker frequency detected by the detection means is outside the detection target range of the first sampling process. In the first sampling process, sampling is performed at a first sampling frequency, and in the second sampling process, sampling is performed at a second sampling frequency that is higher than the first sampling frequency, and the second sampling frequency is a value based on a Nyquist frequency corresponding to the first sampling frequency. It is characterized by: [Effects of the Invention]

[0011] According to the present invention, flicker can be effectively detected regardless of the light source, and information about the detected flicker can be effectively notified to the user. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram illustrating the configuration of a camera body 100, a lens unit 200, and a light emitting device 300, which are embodiments of an imaging device embodying the present invention. [Figure 2] FIG. 2 is a diagram illustrating an example of a shutter speed setting (index) table according to the present invention. [Figure 3] 4 is a flowchart showing a flicker reduction process according to the first embodiment of the present invention. [Figure 4] 3 is a flowchart illustrating a flicker detection process according to the first embodiment of the present invention. [Figure 5] 3A to 3C are diagrams illustrating an example of a method for selecting a plurality of imaging periods when detecting flicker according to the first embodiment of the present invention. [Figure 6] 10A and 10B are diagrams illustrating an example of a modification of the method of selecting a plurality of imaging periods when detecting flicker according to the first embodiment of the present invention. [Figure 7]10 is a diagram (graph) illustrating an example of the relationship between a method for determining each imaging period for flicker detection and the number of imaging periods according to the present embodiment. [Figure 8] 10A and 10B are diagrams illustrating an example of luminance changes based on images continuously obtained by a global shutter method. [Figure 9] 10A and 10B are diagrams illustrating an example of luminance changes based on images continuously obtained by a rolling shutter method. [Figure 10] FIG. 4 is a diagram illustrating setting values ​​of exposure time (shutter speed) in a first pattern of a plurality of imaging periods for flicker detection according to the first embodiment of the present invention. [Figure 11] FIG. 10 is a diagram illustrating setting values ​​of exposure time (shutter speed) in a second pattern of a plurality of imaging periods for flicker detection according to the first embodiment of the present invention. [Figure 12] 10A and 10B are diagrams illustrating an example of a difference in signal readout time depending on the number of readout lines of an image sensor. [Figure 13] 10A and 10B are diagrams illustrating an example of a difference in sampling frequency depending on the number of samples of an image signal in the vertical direction. [Figure 14] 4 is a flowchart illustrating a process for determining a flicker reduction exposure time according to the first embodiment of the present invention. [Figure 15] 10A and 10B are diagrams illustrating an example of a method for setting an ideal flicker-reducing exposure time when flicker that changes at a predetermined light intensity change frequency occurs according to the present invention. [Figure 16] 4 is a flowchart illustrating a shutter speed selection process according to the first embodiment of the present invention. [Figure 17] 3A and 3B are diagrams illustrating an example of the relative relationship between a shutter speed selected by the shutter speed selection process according to the first embodiment of the present invention and an ideal shutter speed for reducing the influence of flicker. [Figure 18] 3A to 3C are diagrams illustrating exemplary notification images displayed on the display unit 102 by the display processing according to the first embodiment of the present invention. [Figure 19] 5A to 5C are diagrams illustrating exemplary notification images when no flicker is detected by the display process according to the first embodiment of the present invention. [Figure 20] 5A to 5C are diagrams illustrating exemplary notification images when flicker exceeds a frequency range to be detected by the display processing according to the first embodiment of the present invention. [Figure 21] FIG. 10 is a diagram illustrating an example of a notification image for guiding a user to a method for manually setting the shutter speed when flicker exceeds the frequency range to be detected, using the display processing according to the first embodiment of the present invention. [Figure 22] 10A and 10B are diagrams illustrating exemplary notification images displayed on the display unit 102 by the display processing according to the second embodiment of the present invention. [Figure 23] FIG. 11 is a diagram illustrating an example of a transition screen for flicker reduction processing during live view display according to the third embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0013] (First embodiment) (Basic configuration of imaging device) Preferred embodiments of the present invention will be described below with reference to the accompanying drawings. FIG. 1 is a block diagram illustrating the configuration of a camera body 100, a lens unit 200, and a light emitting device 300, which are embodiments of an imaging device embodying the present invention. Note that one or more of the functional blocks shown in FIG. 1 may be realized by hardware such as an ASIC or a programmable logic array (PLA). Alternatively, they may be realized by a programmable processor (microprocessor, microcomputer) such as a CPU or MPU executing software. Alternatively, they may be realized by a combination of software and hardware.

[0014] Therefore, even when different functional blocks are described as the main operators in the following description, the same hardware may be used to implement the functions. Also, although an image capture device will be described as an example of this embodiment, the present invention is not limited to an image capture device and may be implemented as a flicker detection device or program as long as it is a suitable embodiment that can embody the flicker detection method.

[0015] First, the components that make up the camera body 100 will be described. The camera body 100 is equipped with a frame memory (not shown), which functions as a storage unit that temporarily stores signals (video signals) and can read them out when needed. Generally, frame memory is also called RAM, and in recent years, DDR3-SDRAM (DUAL DATA RATE 3-SYNCHRONOUS DYNAMIC RAM) and the like are often used. Using this frame memory enables a variety of processes.

[0016] The image sensor 101 is an imaging unit using a charge-storage solid-state image sensor such as a CMOS or CCD, which receives a light beam from a subject guided into the camera body 100 via the lens unit 200 and converts it into an electrical image signal. Images (signals) obtained by the image sensor 101 under drive control by a CPU 103 (described later) are handled as various image signals, such as for live view display, flicker detection, and captured images for recording. Since the electrical signals obtained by the image sensor 101 are analog values, the image sensor 101 also has a function for converting them into digital values. Based on the image signal output from the image sensor 101, an evaluation value (photometric value) of the subject's brightness can be detected. The exposure time of the image sensor 101 can be controlled according to a shutter speed that can be set as an exposure control value for the image sensor 101.

[0017] The mechanical shutter 104 is a light-oblique device that can travel in a direction parallel to the signal scanning direction of the image sensor 101. The exposure time of the image sensor 101 can be controlled by adjusting the exposure opening formed by the multiple shutter blades of the mechanical shutter in accordance with the shutter speed described above. The exposure time adjustment according to the present invention can be achieved by utilizing or combining the mechanical shutter 104 with a so-called electronic shutter that adjusts the signal reset and readout timing of the image sensor 101.

[0018] The display unit 102 is a display device that can be visually confirmed by the user, and enables the user to check the operating status of the camera body 100. For example, the display unit 102 displays images that have been subjected to image processing based on image signals of a subject, setting menus, and the like. However, an LCD (Liquid Crystal Display) or an organic EL (Organic Electroluminescence) may also be used as the display unit 102. By displaying images acquired by the image sensor 101 and setting conditions such as exposure control values ​​in real time on the display unit 102 while capturing an image of a subject, so-called live view display is possible. Note that the display unit 102 of this embodiment is equipped with a resistive or capacitive thin-film element known as a touch panel, and also serves as an operation unit that can be touched by the user.

[0019] The CPU 103 is a control means capable of comprehensively controlling each part of the camera body 100 and the accessories attached to the camera body 100. A ROM (Read Only Memory) and a RAM (Random Access Memory) are connected to the CPU 103. The ROM (not shown) is a non-volatile storage element that stores programs for operating the CPU 103, various adjustment parameters, and the like. Programs read from the ROM are expanded into volatile RAM (not shown) and executed. Generally, RAM is a slower, lower-capacity element than frame memory (not shown).

[0020] Next, details of the lens unit 200 will be described. The lens unit 200 is an accessory that can be attached to and detached from the camera body 100, and is a so-called interchangeable lens that includes a group of lenses 201 such as a focus lens, a zoom lens, and a shift lens. For example, the focusing lens included in the group of lenses 201 can adjust the focus on a subject by adjusting the lens position in the direction of the optical axis of the lens.

[0021] The diaphragm 202 is a light amount adjusting member for adjusting the amount of light related to the light flux of the subject that is guided into the camera body 100 via the lens unit 200. In this embodiment, the light amount can be adjusted by adjusting the aperture diameter of the diaphragm 202, which is realized by changing the aperture value as an exposure control value related to the aperture diameter of the diaphragm.

[0022] The LPU 203 is a control means that controls each part of the lens unit 200, and can control, for example, the driving of the lens group 201 and the aperture 202. The LPU 203 is connected to the CPU 103 of the camera body 100 via a terminal group (not shown), and can drive each part of the lens unit 200 in response to control instructions from the CPU 103.

[0023] Next, details of the light emitting device 300 will be described. The light emitting device 300 is an external light emitting device that can be attached and detached via a connector (not shown) provided on the camera body 100. The SPU 301 is a control means that controls each part of the light emitting device 300, and is mainly capable of controlling light emission and communication with the camera body 100. The SPU 301 is connected to the CPU 103 of the camera body 100 via a group of contacts (not shown), and can drive each part of the light emitting device 300 in response to control instructions from the CPU 103.

[0024] Although the components of the imaging device according to the first embodiment of the present invention have been described above, the present invention is not limited to the above-described configuration. For example, the camera body 100 may have built-in devices equivalent to the lens unit 200 and the light emitting device 300.

[0025] (How to set the shutter speed) Next, a method for setting the shutter speed, which is an exposure control value for controlling the exposure time of the image sensor 101 according to this embodiment, will be specifically described with reference to Fig. 2. Fig. 2 is a diagram for explaining an example of a shutter speed setting (index) table according to the present invention.

[0026] While it is generally known that shutter speeds can be changed in 1 / 2 or 1 / 3 increments of the light intensity, this embodiment allows for finer shutter speed adjustment to address flickering that occurs under LED light sources that periodically blink at various frequencies. Specifically, in this embodiment, shutter speeds can be adjusted in 1 / 4 increments from 1 / 8192.0 to 1 / 4871.0, and in 1 / 8 increments from 1 / 4096.0 to 1 / 2233.4. Furthermore, shutter speeds can be adjusted in 1 / 16 increments from 1 / 2048.0 to 1 / 1069.3, and in 1 / 32 increments from 1 / 1024.0 to 1 / 523.2. Furthermore, shutter speeds can be adjusted in 1 / 64 increments from 1 / 512.0 to 1 / 258.8, in 1 / 128 increments from 1 / 256.0 to 1 / 128.7, and in 1 / 256 increments from 1 / 128.0 to 1 / 50.0.

[0027] In consideration of visibility, some shutter speeds are omitted from the table shown in Fig. 2. The index values ​​in the table shown in Fig. 2 are used in the shutter speed selection process for reducing flicker, which will be described later.

[0028] Furthermore, the camera body 100 according to this embodiment prioritizes the use of an electronic shutter in order to allow for the free setting of a shutter speed from a high shutter speed shorter than 1 / 8000 second as described above to a slower shutter speed longer than 1 / 50 second (not shown). The shutter method (use of the electronic shutter and / or the mechanical shutter 104) can be changed at any time by the user manually via a menu screen displayed on the display unit 102, for example.

[0029] (Flicker reduction processing) Next, the flicker reduction process according to this embodiment will be described with reference to the flowchart shown in Fig. 3. Fig. 3 is a flowchart showing the flicker reduction process according to the first embodiment of the present invention.

[0030] First, flicker reduction processing is started in response to a predetermined operation, such as a manual operation by the user based on a menu displayed on the display unit 102 or the like. Note that the flicker reduction processing according to this embodiment is a process of controlling so that unevenness due to flicker does not occur in moving images such as live view displays by setting a shutter speed (i.e., exposure time) that reduces the effects of detected flicker. The flicker reduction processing according to the present invention is not limited to this, and for example, a method of reducing flicker may be configured to apply a gain to the image that reduces unevenness, in addition to adjusting the shutter speed.

[0031] When the flicker reduction process starts, first in step S301, CPU 103 repeats the process of S301 until flicker detection process (detection process) starts. If it is determined in step S301 that detection has started, CPU 103 executes flicker detection process in step S302. The details of the flicker detection process will be described later.

[0032] Next, in step S303, CPU 103 determines whether flicker has occurred based on the processing result of step S302. If it is determined in step S303 that flicker has been detected, the process proceeds to step S304, and if it is determined that flicker has not been detected, the process proceeds to step S307. Note that, as a result of flicker detection, flicker is deemed to have been detected if flicker of a predetermined level or more has occurred. A method for calculating the flicker level will be described later.

[0033] In step S304, CPU 103 determines whether the blinking cycle of the detected flicker is a light intensity change frequency of flicker within the detection target range. If it is determined that the light intensity change frequency of the flicker is within the detection target range, the process proceeds to step S305, and if it is not within the detection target range (exceeds the detection target range), the process proceeds to step S308. The method for determining whether it is within the detection target frequency range will be described in detail later.

[0034] In step S305, CPU 103 determines an exposure time (shutter speed) that reduces the effect of the previously detected flicker (flicker reduction exposure time determination process). Details of the flicker reduction exposure time determination process will be described later.

[0035] Next, in step S306, CPU 103 executes shutter speed selection processing to select an arbitrary shutter speed capable of reducing the influence of flicker, based on the information on the exposure time suitable for reducing flicker determined in step S305. The shutter speed selection processing will be described in detail later.

[0036] In step S307, CPU 103 executes a display process to display, as the processing results of steps S305 and S306, the flicker detection result (presence or absence of detection), a selectable value as a shutter speed that can reduce the influence of flicker, etc. Details of this display process will be described later.

[0037] In step S308, based on the detection result of step S302, CPU 103 notifies the user that the light intensity change frequency of the currently occurring flicker is outside the range of frequencies guaranteed as detection targets by camera body 100. Details of this notification will be described later. By performing the flicker reduction process described above, it becomes possible to obtain an image in which the effects of flicker have been reduced, regardless of the flicker frequency, and to display or record the image based on this image.

[0038] (Flicker detection processing) Next, the flicker detection process (flicker detection process) according to this embodiment will be described with reference to Fig. 4. As mentioned above, unlike light sources such as fluorescent lamps, LED light sources and the like use a rectifier circuit to control the drive current, which causes changes in the light intensity (blinking), i.e., flicker, to occur at a cycle different from the power supply frequency when driving the light source. Therefore, when detecting flicker caused by light sources such as LEDs, it is not possible to narrow down the frequency to be detected to a specific value, as is the case with the drive power supply frequency, and it is therefore necessary to analyze whether or not flicker occurs over a wide range of frequencies.

[0039] On the other hand, when the light intensity change frequency of the flicker (the blinking period of the light source) and the imaging period when continuously capturing images of a subject match or are an integer multiple (hereinafter, this state will be referred to as synchronization), changes in light intensity (blinking) between continuously captured images are suppressed. In this case, for example, in a live view display in which images are continuously displayed, image quality degradation such as unevenness caused by flicker does not occur. However, still images captured by capturing images at an arbitrary shutter speed may have exposure unevenness caused by flicker. Note that even if the imaging frame rate for live view display images matches the light intensity change frequency of the flicker, if a video for recording is captured at a different frame rate, the video may have exposure unevenness or brightness fluctuation caused by flicker.

[0040] A known method for identifying the frequency of change in light intensity of flicker is to detect and compare the difference in light intensity (brightness and darkness) between images obtained by continuous imaging. Therefore, when using this method to identify the frequency of change in light intensity of flicker, it is necessary to adjust the frequency of change in light intensity of flicker so that it is not synchronized with the imaging cycle (frame rate).

[0041] Therefore, in this embodiment, the occurrence of flicker is detected by analyzing the frequency of change in light intensity of flicker at multiple imaging cycles. According to this method, by analyzing the frequency of change in light intensity of flicker at multiple frequencies, it is possible to avoid synchronization between the frequency of change in light intensity of flicker and the imaging cycle, and it is possible to effectively detect flicker over a wide range of frequencies.

[0042] FIG. 4 is a flowchart of flicker detection processing according to the first embodiment of the present invention. As shown in FIG. 4, in step S401, CPU 103 performs photometric calculation of the subject (subject photometry) to determine the exposure for capturing an image of the subject in the flicker detection processing. Any method may be used for the photometric calculation. For example, in this embodiment, an evaluation value is obtained based on the average value of image signals obtained by accumulating charges for photometric calculation using image sensor 101. CPU 103 then obtains the representative luminance (photometric value) of the subject as the photometric result based on the obtained evaluation value. The photometric value is calculated by dividing the angle of view corresponding to the image signal into multiple blocks, calculating the average value of signals output from corresponding pixels for each block, and averaging the average values ​​calculated for each block to calculate the photometric value (representative luminance). The photometric value is calculated in units of 1 BV in the so-called APEX (ADDITIVE SYSTEM OF PHOTOGRAPHIC EXPOSURE) system, where 1 BV corresponds to one step of luminance value, but other units may also be used.

[0043] Next, in step S402, the CPU 103 adjusts the imaging period to an imaging period (non-frame rate) for flicker detection. The method for adjusting the imaging period for flicker detection will be described in detail later.

[0044] Next, in step S403, CPU 103 determines an exposure control value (changes exposure) based on the photometric value determined earlier. The exposure control values ​​according to this embodiment are shutter speed (i.e., accumulation time), aperture value, and shooting sensitivity (ISO sensitivity), and are parameters that can adjust the brightness of an image obtained by capturing an image of a subject. The determined exposure control value is stored in the RAM described above, and the exposure of camera body 100 is changed, and acquisition of an image for flicker detection begins.

[0045] Next, in step S404, CPU 103 determines whether or not there is a luminance change in the acquired image (i.e., whether or not flicker is occurring). As described above, if the blinking cycle of the light source and the imaging cycle of the subject are synchronized, flicker cannot be detected correctly, so the presence or absence of a luminance change is determined based on the acquired image. If it is determined that there is no luminance change in the acquired image, it is determined that the imaging cycle and the light intensity change frequency of the flicker related to the subject are synchronized, or that flicker is not occurring, and the detection operation at the current frame rate (imaging cycle) is skipped.

[0046] If it is determined that a luminance change has been detected in the acquired image (YES in step S404), then in step S405, CPU 103 analyzes (detects) whether or not flicker has occurred at a plurality of different frequencies. The method of detecting flicker at a plurality of frequencies in step S405 will be described in detail later.

[0047] Next, in step S406, the CPU 103 determines whether detection has been completed for a predetermined number (n) of imaging cycles. If it is determined that detection has not been completed for the predetermined number of imaging cycles (NO in step S406), the process returns to step S402, the imaging cycle (frame rate) is changed, and the processes from step S403 onward are repeated.

[0048] If it is determined that detection has been completed for a predetermined number of imaging cycles (YES in step S406), in step S407 CPU 103 identifies the frequency of flicker of the subject based on the detection results up to step S405. In the processing of step S407, the presence or absence of flicker of multiple different frequencies has been detected for multiple imaging cycles (frame rates).

[0049] Therefore, the flicker levels detected for each frequency are compared, and the flicker with the highest level is the final detection result as the currently occurring flicker of the subject. In this embodiment, the flicker level is compared based on the magnitude of the change in light quantity (the magnitude of the amplitude in the curve showing the regular change in light quantity), but this is not limiting. For example, a configuration may be adopted in which the stability of the change in light quantity, etc., is compared in addition to the flicker level.

[0050] Here, the imaging cycle (frame rate) for flicker detection described above will be described in detail. As described above, the camera body 100 according to this embodiment performs flicker detection processing at multiple imaging cycles. For example, consider a case where the imaging cycle is switched between 100 fps and 120 fps to detect the frequency of change in the light intensity of flicker. In this case, for flicker whose light intensity changes at a cycle of k (k is a natural number) × 100 Hz, such as 100 Hz, 200 Hz, or 300 Hz, which are integer multiples of the 100 fps imaging cycle, the imaging cycle and the frequency of change in the light intensity of the flicker are synchronized, making it impossible to detect the flicker. On the other hand, consider a case where the imaging cycle is used to detect the frequency of change in the light intensity of flicker. In this case, for flicker whose light intensity changes at a cycle of m (m is a natural number) × 120 Hz, such as 120 Hz, 240 Hz, or 360 Hz, which are integer multiples of the 120 fps imaging cycle, the imaging cycle and the frequency of change in the light intensity of the flicker are synchronized, making it impossible to detect the flicker. Note that 600Hz and 1200Hz, which are frequencies that satisfy both the conditions k × 100Hz (k is a natural number) and m × 120Hz (m is a natural number), are the least common multiples of 100Hz and 120Hz. If flicker occurs, where the light intensity changes at such frequencies, the frequency of the flicker's light intensity change will be synchronized with both the 100fps and 120fps imaging cycles, and the flicker cannot be detected correctly using images obtained at either imaging cycle.

[0051] For example, in light sources with rectifier circuits, such as LED light sources, the adjusted power supply frequency generally falls within the range of 50 Hz to 1000 Hz. Therefore, even if flicker with the aforementioned 600 Hz light intensity change frequency occurs, it may not be possible to detect the flicker correctly depending on the imaging cycle. In other words, even if flicker detection is performed using images acquired at two imaging cycles, there are frequencies at which the flicker cannot be detected correctly among the wide range of frequencies that are expected to occur in LED light sources.

[0052] In the above example, we have described flicker that changes at a frequency that is exactly the same as an integer multiple of the imaging cycle (frame rate), but there is a risk that the accuracy of flicker detection will decrease even if the frequency does not match an integer multiple of the imaging cycle.For example, with flicker that changes at a frequency that is close to an integer multiple of the imaging cycle when acquiring an image for flicker detection, the impact of exposure unevenness that occurs in the image is small, and it may take a long time to detect the flicker, or the flicker may not be detected correctly.

[0053] Therefore, in this embodiment, in order to effectively detect flicker over a wide range of frequencies that can occur under an LED light source, the number n of multiple imaging periods (frame rates) used during flicker detection is adjusted to satisfy the condition "n≧3 (n) is a natural number." In other words, flicker detection is performed using n or more imaging periods, where n is a natural number greater than or equal to 3.

[0054] Note that the higher the frequency of the change in light intensity of the flicker to be detected, the more accurately the frequency of the change in light intensity of the flicker can be detected by increasing the number n of imaging cycles used for detection. However, increasing the number of imaging cycles used for flicker detection may extend the period required for flicker detection, and it is also necessary to consider the possibility of a release time lag or a decrease in the display frame rate of the live view image. Therefore, in this embodiment, the number of imaging cycles used for flicker detection is set to n=3, which is the sampling number that can effectively detect flicker that may occur in light sources that are generally expected to be used in many cases, such as LED light sources.

[0055] Next, a method for selecting specific values ​​for each of the n imaging periods will be described.

[0056] In this embodiment, a reference imaging cycle is first set. For example, assume that the reference imaging cycle is 100 fps. The light intensity change frequency of flicker synchronized with an imaging cycle of 100 fps is an integer multiple of 100 Hz, and if flicker occurs at this light intensity change frequency, the flicker cannot be detected correctly.

[0057] Note that even when sampling at an imaging cycle of 200 fps, which is twice the standard imaging cycle of 100 fps, the same problem occurs as when sampling at the standard imaging cycle of 100 fps. That is, if an integer multiple of the imaging cycle for obtaining images for flicker detection matches an integer multiple of the frequency of change in light intensity of the flicker, the imaging cycle and the frequency of change in light intensity will be synchronized, making it impossible to correctly detect flicker based on the images obtained for sampling.

[0058] Therefore, in this embodiment, as n imaging periods (n=3 in this embodiment), the remaining n-1 imaging periods (2 in this embodiment) are set between the reference imaging period and the next imaging period that is an integer multiple of the reference imaging period. For example, if there are three imaging periods and the reference imaging period is 100 fps, the remaining imaging periods between 100 fps and 200 fps are set as multiple imaging periods for detecting flicker, in addition to 100 fps. Note that in this embodiment, each imaging period (frequency) is set so that the least common multiple of the n imaging periods is equal to or greater than a predetermined frequency. For example, since the blinking frequency of an LED light source is generally 10,000 Hz or less, the frequency of each imaging period is determined so that the least common multiple of the n imaging periods (frame rates) is equal to or greater than 10,000. Furthermore, to reduce the effects of flicker in camera body 100, each imaging period (frequency) is set so that the least common multiple of the n imaging periods is greater than the reciprocal of the upper limit on the high-speed side of the shutter speed that can be set by camera body 100. This configuration makes it possible to effectively detect flicker that occurs in light sources such as LED light sources whose light intensity changes at high frequencies, while also reducing the effects of the detected flicker by adjusting the shutter speed.

[0059] 5 is a diagram illustrating an example of a method for selecting multiple imaging periods when detecting flicker according to the first embodiment of the present invention. In order to accurately detect the light intensity change frequency of flicker, the imaging periods are spaced as far apart as possible, thereby ensuring a difference between any of the multiple imaging periods and the light intensity change frequency of the flicker to be detected (the blinking period of the light source) to the extent that flicker can be detected satisfactorily. Therefore, in this embodiment, as shown in FIG. 5(a), the range of imaging periods to be detected (100 fps to 200 fps) is divided at predetermined intervals, and imaging periods are set that are spaced apart by 1 / 3 of the imaging period 2 to detect flicker.

[0060] Specifically, in this embodiment, as shown in FIG. 5(a), three imaging periods are defined as a reference imaging period of 100 fps, 100 fps × 2^(1 / 3) = 125.99 fps ≈ 126 fps, and 100 fps × 2^(2 / 3) = 158.74 fps ≈ 159 fps. These three imaging periods differ by 2^(1 / 3) = 1.2599 ≈ 1.26 times, resulting in a difference of approximately 26% between the imaging periods. With this configuration, even when flicker detection is performed by dividing a wide frequency range from 50 to over 1000 Hz into multiple ranges, there is no significant deviation between the frequencies targeted for detection within each range. Furthermore, it is possible to ensure that one of the imaging periods is sufficiently different from the light intensity change frequency of the flicker to be detected. That is, when n imaging periods are set and flicker detection is performed at each imaging period, the deterioration of detection accuracy for each frequency to be detected can be suppressed by setting each imaging period to a multiple of 2(1 / n).

[0061] Fig. 5(b) is a diagram illustrating an example of the correspondence relationship between n imaging cycles and the light intensity change frequency of the flicker to be detected. In this embodiment, flicker is detected based on an image obtained by an imaging cycle that is the furthest from the light intensity change frequency of the flicker to be detected among the n imaging cycles. Specifically, in this embodiment, as shown in Fig. 5(b), flicker detection is performed based on a data table in which the light intensity change frequencies of flicker from 50 Hz to 1008 Hz are divided into ranges (A) to (P) for the three imaging cycles shown in Fig. 5(a).

[0062] In this embodiment, the effects of flicker are reduced by capturing an image of a subject at a shutter speed that is the reciprocal of the flicker's light intensity change frequency, thereby setting an imaging period synchronized with the flicker's light intensity change frequency. Therefore, when there is a discrepancy between the ideal shutter speed synchronized with the flicker's light intensity change frequency and the actual shutter speed, a slower shutter speed will cause a greater impact from flicker (such as uneven exposure) on the image than a faster shutter speed. For example, assume that shutter speeds of 1 / 101 second and 1 / 1001 second are set for flicker with a light intensity change frequency of 100 Hz and 1000 Hz, respectively, which are 1 Hz off from the ideal shutter speed for reducing the effects of flicker. In both cases, there is a 1 Hz discrepancy between the shutter speed capable of reducing the effects of flicker and the actual shutter speed. However, the discrepancy is 1% at a shutter speed of 1 / 100 second, while it is 0.1% at a shutter speed of 1 / 1000 second. In other words, the faster shutter speed causes less impact from flicker on the image for a 1 Hz change in shutter speed. However, as the shutter speed becomes longer, the period for capturing the change in light quantity caused by flicker becomes longer, and it is therefore more likely that an image in which the change in light quantity has been smoothed will be obtained. Therefore, when detecting flicker with a light quantity change frequency that is reduced at a shutter speed equal to or greater than a predetermined value (for example, 1 / 25 second or longer), the detection range in the low-frequency region of the flicker may be appropriately adjusted and widened.

[0063] Therefore, in this embodiment, as shown in Fig. 5(b), the range of light intensity change frequency of the flicker to be detected is divided into multiple ranges, and the detection ranges are set so that the frequencies of each successive range differ by 2^(1 / 3) = 1.26 times. For example, while range (N) shown in Fig. 5(b) is 159 to 200 Hz, the next range (C) is set to detect flicker from 200 to 252 Hz, which is about 1.26 times the range (N).

[0064] As shown in FIG. 5(b), the range of flicker light intensity change frequencies that can be detected with the same imaging cycle varies by approximately two times between successive ranges. For example, the detection frequencies for ranges (A), (B), and (c) shown in FIG. 5(b), which correspond to an imaging cycle of 159 fps, range from 50 Hz, 100 Hz, and 200 Hz to 63 Hz, 126 Hz, and 252 Hz, respectively. This is because the light intensity change due to flicker is the same at integer multiples of each frequency. With the above-described configuration, the imaging device according to this embodiment can detect flicker over a wide range of frequencies with stable accuracy.

[0065] In this embodiment, the imaging periods for detecting flicker differ by a factor of m (1 / n) (m and n are natural numbers). While the above description assumes m=2, this is not limiting. For example, the imaging period may be set to m=3. In this case, the difference between the imaging periods becomes greater, and the detection accuracy for the light intensity change frequency of the flicker to be detected may be lower than when m=2. However, when the same frequency range is to be detected, m=3 can shorten the detection time compared to m=2, and is therefore suitable for detecting a wider range of light intensity change frequencies of flicker.

[0066] Here, a method (variant) of selecting n imaging periods that is different from the above-described method will be described with reference to Fig. 6. Fig. 6 is a diagram illustrating an example of a variant of the method of selecting multiple imaging periods when detecting flicker according to the first embodiment of the present invention. The difference between this variant and the example described above with reference to Fig. 5 is the method of setting n imaging periods for the range of imaging periods to be detected.

[0067] In this modification, as shown in Fig. 6(a), multiple imaging periods are set by equally dividing the range of the imaging period to be detected. That is, if the range of the imaging period for flicker detection (100 fps to 200 fps) is set as 100% of the range, n imaging periods are set to imaging periods that differ by 33% and 66%, respectively, from the reference imaging period of 100 fps. Specifically, the three imaging periods are set to the reference imaging period of 100 fps, 100 fps x 1.333 = 133.333... fps ≒ 133 fps, and 100 fps x 1.666 = 166.666 fps ≒ 167 fps.

[0068] The differences between the three imaging periods are 133.333 / 100 = 1.33333, 166.666 / 133.33 = 1.25, and 200 / 166.666 = 1.2, which means that the imaging periods differ by 20% or more.

[0069] Fig. 6(b) is a diagram illustrating an example of the correspondence relationship between the n imaging cycles shown in Fig. 6(a) and the light intensity change frequency of the flicker to be detected. As shown in Fig. 6(b), in this modification, as in Fig. 5(b) described above, flicker is detected based on an image obtained by an imaging cycle that is the furthest away from the light intensity change frequency of the flicker to be detected among the n imaging cycles.

[0070] Here, the differences between the multiple imaging periods for flicker detection will be explained. As mentioned above, as the number of multiple imaging periods for flicker detection increases, the difference between each imaging period decreases, but the time required for sampling increases. Therefore, in order to detect flicker accurately in a short period of time, it is preferable that the difference between each imaging period is as large as possible and the number of imaging periods for sampling is as small as possible, within a range that allows detection of a wide range of flicker light intensity change frequencies.

[0071] 5, the period from the reference imaging period to a period twice the reference imaging period is set to 100%, and the period is divided into increments of 1 / n of 2. In this case, the multiple imaging periods for flicker detection differ at intervals shown in the following formula (1). {2^(1 / n)-1}×100[%] (Formula 1)

[0072] Also, consider the case where the interval between the reference imaging cycle and a cycle twice the reference imaging cycle is set to 100%, and the interval therebetween is changed by 100 / n% as explained with reference to FIG.

[0073] As calculated with n=3, the difference is smallest between an imaging cycle that is 100% × (n-1) / n away from the reference imaging cycle and an imaging cycle that is twice the reference imaging cycle, and the difference is [200 / {100+{(100×(n-1) / n}-1]×100[%]={200n / (200n-100)-1}[%]={2n / (2n-1)-1}×100[%]={1 / (2n-1)}×100[%] (Formula 2) is calculated as follows. In other words, when each imaging period differs by 100 / n [%], the multiple imaging periods (frame rates) used for flicker detection differ from each other by at least a ratio of [{2n / (2n-1)-1}] × 100% or more. Note that in camera body 100 according to the first embodiment of the present invention, the multiple imaging periods (frame rates) used for flicker detection differ from each other by at least a ratio of [{2n / (2n-1)}-1] × 100% or more. This also includes the case where the period from the reference imaging period to a period twice the reference imaging period is set to 100%, and the interval therebetween is divided by 1 / n power of 2.

[0074] Here, based on the above-described formulas 1 and 2, the relationship between the method of determining each imaging period and the difference in the number of imaging periods can be expressed in a graph as shown in FIG. 7. FIG. 7 is a drawing (graph) illustrating an example of the relationship between the method of determining each imaging period for flicker detection and the number of imaging periods according to this embodiment. As shown in FIG. 7, formula 2, indicated by a solid line, produces a smaller difference between imaging periods corresponding to differences in the number n of imaging periods than formula 1, indicated by a dashed line. Note that this condition also applies to a larger number n of imaging periods not shown in FIG. 7. That is, in the above-described example, two different methods of determining the imaging period were described, but it can be seen that for both of these methods, the difference between imaging periods is ensured to be equal to or greater than the value calculated by formula 2.

[0075] Although two examples of imaging periods for flicker detection have been described above, the imaging period for flicker detection is not limited to these. As long as the imaging device according to the present invention can accurately detect flicker, the imaging periods (frame rates) may be three or more different natural numbers n, as long as the least common multiple of the n imaging periods does not include any of the n imaging periods. For example, even if the imaging periods for flicker detection are set to 50 Hz, 150 Hz, 300 Hz, etc., the luminance changes in the images acquired at each imaging period do not change over the same period, making it impossible to accurately detect flicker. In the imaging device of this embodiment, it is preferable that each imaging period for flicker detection be a high rate of 100 fps or more, and that the least common multiple of each imaging period be adjusted so as not to be less than 10,000 Hz, which may be used as the flicker frequency of a light source such as an LED.

[0076] Next, the details of the process of analyzing (detecting) the occurrence of flicker at a plurality of different frequencies in step S405 described above will be explained. In the imaging device according to this embodiment, changes in luminance over time are extracted based on the luminance of continuously acquired images, and the periodicity of the luminance changes is analyzed to detect the frequency of changes in light intensity of flicker. Note that the luminance changes occurring in an image differ depending on the image acquisition method used for detection. For example, the luminance changes in an image differ when an object is captured using a so-called global shutter method such as a CCD and when an object is captured using a so-called rolling shutter method such as a CMOS. Below, we will explain how luminance changes when an image is captured using each of the above methods.

[0077] First, referring to Fig. 8, the change in luminance of an image obtained using the global shutter method will be described. Fig. 8 is a diagram illustrating an example of the change in luminance based on images continuously obtained using the global shutter method. When an image of a subject subjected to flickering of a light source caused by flicker is captured, a captured image is obtained that is affected by the intensity of the flickering of the light source. When the luminance of the entire captured image is measured, a photometric value that is affected by the intensity of the flickering of the light source is obtained.

[0078] Note that the luminance in this description may refer to a luminance signal calculated by multiplying the R, G1, G2, and B signals in a Bayer array RAW image by a certain coefficient, or may refer to the color signals of the R, G1, G2, and B signals themselves. It may also refer to color and luminance signals obtained from a sensor array other than the Bayer array.

[0079] Then, for the captured images obtained by the above-mentioned method, the difference or ratio in brightness (photometric value) of multiple consecutive images is calculated. Alternatively, an average image of multiple images is used as a reference image and the difference / ratio in brightness of each image relative to the reference image is calculated. By plotting the brightness change for each image obtained by this method, it is possible to detect the transition of brightness change in the image as shown in Figure 8.

[0080] Next, the luminance change of an image obtained by the rolling shutter method will be described with reference to Fig. 9. Fig. 9 is a diagram illustrating an example of the luminance change based on images obtained continuously by the rolling shutter method. When the sensor is driven by the rolling shutter method, the exposure and readout timing differs for each row (line) of the sensor, so the effect of blinking of the light source caused by flicker differs for each row (line), and the luminance change that occurs in the vertical direction of the image differs.

[0081] Therefore, when the sensor (image sensor 101 in this embodiment) is driven using the rolling shutter method, the luminance change due to the flickering of the light source can be extracted by acquiring an integral value for each row (line) of the captured image. Specifically, as shown in FIG. 9 , the luminance change of the same line in consecutive frames (N-1) and (N) of the image obtained by continuously capturing images of the subject is extracted. In this case, an integral value is calculated for each row of the captured image corresponding to the Nth and (N-1)th frames. As described above with respect to the global shutter method, this integral value may be a luminance signal obtained by multiplying a color signal by a certain ratio, or may be the integral of the color signal itself. By comparing the integral values ​​for the Nth and (N-1)th frames for each row and calculating the difference / ratio, the luminance change in the vertical direction (i.e., the scanning direction of the sensor) of the captured image can be detected, as shown in FIG. 9 .

[0082] The frames to be compared do not have to be two consecutive frames. For example, the signal values ​​of multiple captured images may be averaged to obtain an average image, and the average image may be used as a reference image, and the integral value for each line may be compared with the integral value for each line of the Nth frame to calculate the change in luminance in the vertical direction of the image.

[0083] By analyzing the captured image obtained using the rolling shutter method using the method described above, it is possible to detect the transition of vertical brightness changes in the captured image as described above, and this brightness change represents the flickering of the light source (i.e., the change in light intensity due to flicker).

[0084] Next, we will explain a method for analyzing the frequency of luminance changes from the transition of image luminance changes. A common method for converting a signal that changes in the time direction into frequency components is the Fourier transform. In this case, the signal f(t) that changes in the time direction is converted into a frequency function F(ω).

[0085]

number

[0086] If we focus on the exponential function in Equation 3, it is generally known that the exponential function can be expanded into trigonometric functions with real and imaginary parts due to the relationship between the Maclaurin expansion and the nth derivative of a trigonometric function (shown in Equation 4 below).

[0087]

number

[0088] Furthermore, since the integral can be calculated by defining the transition of the change in the image signal as f(t) and the sampling interval of the transition of the change as dt, Equation 4 can be expressed as Equation 5 below. F(ω)=A(ω)+j×B(ω) (Equation 5)

[0089] Since this is a complex function of frequency ω, its magnitude is calculated as |F(ω)|. If the transition of luminance change in the image contains a luminance change component due to frequency ω, |F(ω)| will be a large value, and if the transition of luminance change in the image does not contain a luminance change component due to frequency ω, |F(ω)| will be a small value. In other words, |F(ω)| can be regarded as the flicker level for each frequency. Therefore, by calculating each frequency component using Equation 5 above for a wide range of frequencies to be detected, it is possible to detect the presence or absence of luminance changes due to flickering of the light source over a wide frequency range (i.e., the frequency of change in light intensity of flicker).

[0090] Furthermore, if the transition of brightness change does not include at least one cycle of the flickering of the light source (one cycle of the change in the amount of light from the flicker), the target frequency cannot be detected properly and may be mistaken for another frequency. Therefore, it is preferable to continue capturing images of the subject for at least one cycle of the frequency to be detected, and to detect each of the above-mentioned frequencies (i.e., the frequency of change in the amount of light from the flicker) based on the images obtained by the capture.

[0091] Next, the exposure operation during flicker detection in the aforementioned step S403 will be specifically described. As mentioned above, if the imaging cycle when detecting flicker and the blinking frequency of the light source (the frequency of change in light intensity of the flicker) are synchronized, it is difficult to effectively detect flicker based on the sampled image. Furthermore, in addition to the imaging cycle, if the exposure time (i.e., shutter speed) when capturing an image of a subject is synchronized with the blinking frequency of the light source, no effective brightness change occurs in the image obtained in this state, making it difficult to detect flicker.

[0092] Therefore, in this embodiment, in each imaging cycle when performing flicker detection operation, an exposure time (shutter speed) is set to synchronize with each imaging cycle so as not to synchronize with frequencies other than that of the imaging cycle. In other words, when detecting flicker, it is preferable to capture an image of a subject with an exposure time (shutter speed) that is 1 / N (N is an integer) of the imaging cycle (frame rate) for detection.

[0093] 10 is a diagram illustrating an example of the setting values ​​of the exposure time (shutter speed) in a first pattern of multiple imaging cycles for flicker detection according to the first embodiment of the present invention. For example, as described above, when the multiple imaging cycles for flicker detection are 100 fps, 126 fps, and 159 fps, an exposure time as shown in FIG. 10 is set to capture an image of a subject.

[0094] 11 is a diagram illustrating an example of the setting values ​​of the exposure time (shutter speed) in the second pattern of the multiple imaging cycles for flicker detection according to the first embodiment of the present invention. For example, as described above, when the multiple imaging cycles for flicker detection are 100 fps, 133 fps, and 167 fps, the exposure time shown in FIG. 11 is set to capture an image of the subject.

[0095] As shown in Figures 10 and 11, by acquiring images for flicker detection with an exposure time that is 1 / N (N is an integer) of the imaging period (frame rate) for flicker detection, it is possible to prevent synchronization between the exposure time and the frequency of change in light intensity of the flicker.

[0096] Furthermore, if the exposure conditions differ at each frequency of the flicker light intensity change, differences will occur in the detected flicker level, reducing the detection accuracy. Therefore, in this embodiment, by performing exposure operations for the above-mentioned multiple imaging cycles based on the photometry results of step S401, it is possible to suppress differences in exposure amount between imaging cycles and enable stable flicker level detection.

[0097] Next, the processing of S304 mentioned above will be described in detail. The range of light intensity change frequencies of flicker that can be detected by the camera body 100 according to this embodiment differs based on the sampling period of the image used for detection. Generally, according to the sampling theorem, if the maximum frequency contained in the original signal is f [Hz], it is known that the original signal can be restored by sampling at a frequency higher than 2f [Hz]. If the original signal is a change in light intensity due to flicker (blinking of the subject), sampling at 2f [Hz] or higher is required to be able to detect up to a frequency of f [Hz]; in other words, sampling at intervals of 1 / 2f [sec] or less is required.

[0098] Furthermore, when sampling at 2f [Hz], if a signal contains a high-frequency signal exceeding the frequency f [Hz], the sampling theorem makes it impossible to accurately restore the original signal. In this case, aliasing occurs, resulting in frequencies exceeding the limit frequency (the so-called Nyquist frequency) being detected as frequencies that are different from the actual frequency. For example, when sampling at 2f [Hz] for a subject flickering at a frequency of f+K [Hz], the detected frequency for K [Hz], which exceeds f [Hz], becomes fK [Hz] due to the aliasing phenomenon described above. In this case, it is difficult to distinguish whether the detected flicker light intensity change frequency fK [Hz] is a frequency detected due to aliasing caused by exceeding the Nyquist frequency or the actual frequency.

[0099] Therefore, in step S304, the camera body 100 according to this embodiment performs sampling at a sampling frequency even higher than the sampling frequency of step S302, separate from the flicker detection process performed in step S302. Then, flicker is detected based on the acquired detection signals. The number of signals (data) acquired by sampling may be fewer than the number of data acquired in step S302. This is because the purpose of the process in step S304 is to determine whether the light intensity change frequency of the flicker detected in step S302 is the actual flickering frequency of the subject or is due to aliasing, and a large amount of data is not required. This configuration minimizes the time required for sampling while preventing erroneous detection of the light intensity change frequency of flicker, which changes in light intensity at a high frequency exceeding the Nyquist frequency.

[0100] The following describes a method for setting the sampling frequency in step S304, using a specific example. For example, the blinking frequency of a light source such as an LED is typically between 50 Hz and 2000 Hz. In this case, a sampling frequency of 4000 Hz is sufficient to accurately detect the maximum flicker frequency of 2000 Hz to be detected. However, because there are light sources and electronic billboards (digital signage) that blink at frequencies above 2000 Hz, a sampling frequency of 4000 Hz may not accurately detect flicker. For example, for a light source that blinks at 2500 Hz, a sampling frequency of 4000 Hz results in a Nyquist frequency of 2000 Hz. Therefore, for flicker that changes at 2500 Hz, the detection limit is exceeded by 500 Hz. In this case, due to frequency aliasing beyond the Nyquist frequency, the flicker frequency may be observed as 1500 Hz (2000-500).

[0101] Therefore, in order to accurately detect the light intensity change frequency of flicker exceeding the Nyquist frequency, sampling is performed at a higher sampling frequency of 8000 Hz in addition to the first sampling frequency of 4000 Hz. That is, in addition to sampling at the first sampling frequency of 4000 Hz in step S302, sampling is performed at a second sampling frequency of 8000 Hz in step S304. For example, if a 1500 Hz flicker is detected in the first sampling and a 2500 Hz flicker is detected in the second sampling, it can be determined that the light intensity change frequency of the currently occurring flicker is outside the detection range of the first sampling. In this way, the accuracy of the light intensity change frequency of the flicker detected in the first sampling can be determined based on the result of the second sampling, so that the light intensity change frequency of flicker that flickers exceeding the Nyquist frequency can be accurately detected.

[0102] As described above, sampling at the second sampling frequency is performed to determine whether the frequency of change in light intensity of flicker is outside the detection range for the first sampling frequency. Therefore, a method for analyzing the signal obtained by sampling at the second sampling frequency may be configured to employ a method that is simpler than the method for analyzing the signal obtained at the first sampling frequency. For example, a configuration may be adopted in which the frequency of change in light intensity of flicker is detected using the so-called zero-crossing method that uses sampling data obtained at the second sampling frequency. Details of the method for detecting the frequency of change in light intensity of flicker according to this embodiment using the zero-crossing method will be described later.

[0103] Next, with reference to FIGS. 12 and 13, a drive control method for an image sensor during the first and second sampling periods will be described. FIG. 12 is a diagram illustrating, by way of example, the difference in signal readout time depending on the number of readout lines of the image sensor. FIG. 13 is a diagram illustrating, by way of example, the difference in sampling frequency depending on the number of vertical sampled image signals. Both FIGS. 12 and 13 assume that the image sensor is driven using the rolling shutter method. FIG. 12(a) shows the readout time when all lines of the image sensor are read out, while FIG. 12(b) shows the readout time when the number of readout lines of the image sensor is reduced by half compared to FIG. 12(a). As shown in FIG. 12(b), by reducing (thinning out) the number of readout lines compared to FIG. 12(a), the time required to read the entire image is shortened. When flicker occurs, driving the image sensor using the rolling shutter method causes the effect of changes in light intensity due to flicker to vary from line to line, resulting in striped brightness variations in the image. In the case where the readout time is fast as shown in FIG. 12(b), it is possible to analyze flicker at a higher sampling frequency than in the case shown in FIG. 12(a).

[0104] Next, Figures 13(a) and 13(b) show cases where the number of samples in the vertical direction within an image is different, while Figure 13(c) shows a case where the number of samples is the same as in Figure 13(a) but is sampled in a specific region. The more samples in the vertical direction of the image, the higher the sampling frequency, enabling more accurate flicker detection. As shown in Figure 13(c), the number of samples can be kept the same, but the sampling frequency can be set locally by sampling only the center of the image. As described above, the sampling frequency of the second sampling can be increased by appropriately using the methods described with reference to Figures 12 and 13. When the image sensor is driven using a global shutter method to read out image signals, the second sampling image can be acquired at a faster cycle than the first sampling.

[0105] The above description assumes that the first and second sampling frequencies are within a typical flicker frequency range that takes into account light sources such as LEDs, but the method for setting the first and second sampling frequencies can be adjusted as appropriate. For example, if a signal is obtained near the upper limit of the frequency range in which flicker can be detected, the accuracy of detecting the frequency of change in light intensity of the flicker may not be good. Therefore, a configuration may be adopted in which the detectable frequency range is set higher than the frequency of the flicker to be detected.

[0106] Next, we will specifically explain how to detect the light intensity change frequency of flicker using the aforementioned zero-crossing method, using the second detection signal acquired by the second sampling. For each region shown in Figure 13(b) or 13(c), signal values ​​for two consecutive frames are acquired. Here, the signal value may be a luminance (Y) signal generated by weighted averaging RGB at a fixed ratio, or the RGB signal may be used directly. By taking the difference between the signal values ​​of each region acquired over two frames, the subject components in each region are removed, allowing the change in signal value corresponding to the flickering to be extracted. Strictly speaking, the signal value and the subject portion may be misaligned due to subject movement between the two frames. However, when acquiring detection images at a frame rate exceeding 100 fps as described above, this effect is limited and will be ignored here.

[0107] Even if we take the difference between two frames containing a subject flickering at a specific frequency, the amplitude will be modulated but the frequency will not be, considering the composition of trigonometric functions. Therefore, if we vertically analyze the difference in signal values ​​between the two frames, ideally, the difference in signal values ​​will pass through zero twice per flicker cycle. However, because the signal values ​​are acquired discretely, it is not always possible to detect a signal difference that is exactly zero. In such cases, the sign of the difference between the signal values ​​before and after passing through zero will be different. In other words, the number of times the sign changes vertically (from positive to negative, or from negative to positive) can be counted to count the number of times it reaches zero. As a result, the period of flickering in an image caused by flicker can be easily determined, and the approximate frequency can be calculated from the number of waves contained in the second detection signal.

[0108] In step S304, the light intensity change frequency of the flicker detected by the method described above is compared with the light intensity change frequency of the flicker previously detected in step S302. As a result, it is possible to determine whether the flicker detected in step S302 is within the frequency range to be detected, and if the flicker detected in S302 is outside the detection range, processing is executed in step S308 to notify the user that the flicker is outside the detection range.

[0109] By employing the configuration described above, the imaging device according to this embodiment can stably and effectively detect flicker over a wide range of frequencies that are considered to be the light intensity change frequencies of flicker.

[0110] Next, the details of the flicker reduction exposure time determination process executed in the above-mentioned step S304 will be described with reference to Fig. 14. Fig. 14 is a flowchart of the flicker reduction exposure time determination process according to the first embodiment of the present invention. First, in step S1201, CPU 103 reads from memory the light intensity change frequency of the flicker detected by the flicker detection process executed in the above-mentioned step S302.

[0111] Next, in step S1202, CPU 103 calculates an ideal exposure time (IdealFlkExpTime) for reducing the influence of the detected flicker based on the reciprocal of the light intensity change frequency of the flicker read out in step S1201. For example, if the light intensity change frequency of the detected flicker is 540.0 Hz, then IdealFlkExpTime=1 / 540.0.

[0112] Next, in step S1203, CPU 103 acquires the currently set shutter speed (CurTv). The current shutter speed CurTv may be, for example, a shutter speed manually set by the user. In this embodiment, it is assumed that the shooting mode of camera body 100 is set to manual mode in advance, and that all of the multiple exposure control values ​​(parameters) have been manually set by the user.

[0113] Next, in step S1204, CPU 103 executes initialization processing to integrally multiply the ideal flicker reduction exposure time IdealFlkExpTime. Specifically, in step S1204, the integer N is set to 1, and information about the ideal flicker reduction exposure time IdealFlkExpTime before integral multiplication is stored as PreIdealFlkExpTime.

[0114] Next, in step S1205, CPU 103 compares the currently set shutter speed CurTv acquired in step S1203 with the ideal flicker reduction exposure time IdealFlkExpTime. If the value of CurTv is equal to or less than IdealFlkExpTime (i.e., the exposure time is short), the process proceeds to step S1207 (YES determination in step S1205). On the other hand, if CurTv is greater than IdealFlkExpTime (the exposure time is long), the process proceeds to step S1206 (NO determination in step S1205).

[0115] In step S1206, CPU 103 holds the current ideal flicker reduction exposure time as the previous PreIdealFlkExpTime, increments the integer N by one, and then multiplies the ideal flicker reduction exposure time by an integer N. Specifically, in step S1206, IdealFlkExpTime is substituted for PreIdealFlkExpTime, N is incremented to N+1, and IdealFlkExpTime is multiplied by an integer N. The process of step S1206 is repeated until the shutter speed currently set in step S1205 becomes equal to or less than the ideal flicker reduction exposure time (CurTv≦IdealFlkExpTime). That is, the process of step S1206 is a process for bringing the ideal flicker reduction exposure time IdealFlkExpTime as close as possible to the currently set shutter speed. According to this process, since CurTv is located between IdealFlkExpTime and PreIdealFlkExpTime, it is possible to narrow down the flicker reduction exposure time to an exposure time close to the shutter speed set by the user, for example.

[0116] Next, in step S1207, CPU 103 compares the absolute value of the difference between IdealFlkExpTime and PreIdealFlkExpTime and CurTv. If the determination in step S1207 is NO, the current flicker reduction exposure time determination process ends. This is because it can be determined that the currently set ideal flicker reduction exposure time, IdealFlkExpTime, is closer to the current shutter speed than PreIdealFlkExpTime.

[0117] On the other hand, if the determination in step S1207 is YES, it can be determined that the previously set ideal flicker reduction exposure time PreIdealFlkExpTime is closer to the current shutter speed than the currently set ideal flicker reduction exposure time. Therefore, in this case, the process proceeds to step S1208, where CPU 103 replaces the previously set PreIdealFlkExpTime with the ideal flicker reduction exposure time IdealFlkExpTime, and ends the current flicker reduction exposure time determination process.

[0118] According to the flicker reduction exposure time determination process of this embodiment described above, for example, it is possible to determine an exposure time (shutter speed) for reducing flicker at a value close to the shutter speed set by the user. With this configuration, for example, it is possible to obtain an image in which the effects of flicker are reduced while suppressing any deviation from the intended photographic effect caused by the user adjusting the shutter speed.

[0119] 15A and 15B are diagrams illustrating an example of a method for setting an ideal flicker reduction exposure time when flicker that changes at a predetermined light intensity change frequency occurs according to the present invention. Fig. 15A shows a case where, for example, the shutter speed is set to 1 / 5792.6 by the user (CurTv=1 / 5792.6). Fig. 15B shows a case where, for example, the shutter speed is set to 1 / 250.5 by the user (CurTv=1 / 250.5).

[0120] For example, if the light intensity change frequency of the detected flicker is 540.0 Hz, then in the example shown in FIG. 15(a), the ideal flicker reduction exposure time IdealFlkExpTime is 1 / 540.0. Furthermore, for the same flicker light intensity change frequency, in the example shown in FIG. 15(b), the ideal flicker reduction exposure time IdealFlkExpTime is 1 / 270.0. The flicker light intensity change is the same for integer multiples of the frequency. Therefore, even if a subject is captured at a shutter speed slower than the reciprocal of the flicker light intensity change frequency, which is the reciprocal of an integer multiple of the flicker frequency, the effects of flicker can be reduced. Therefore, if the shutter speed set by the user is equal to or lower than the reciprocal of the light intensity change frequency of the detected flicker, then the ideal flicker reduction exposure time can be set to the reciprocal of the integer multiple of the flicker frequency that is the smallest difference from the shutter speed set by the user.

[0121] Next, details of the shutter speed selection process executed in step S305 described above will be described with reference to FIG. 16. FIG. 16 is a flowchart related to the shutter speed selection process according to the first embodiment of the present invention. First, in step S1401, CPU 103 executes initialization processing for selecting an arbitrary shutter speed from the shutter speed setting (index) table described above with reference to FIG. 2. Specifically, in step S1401, CPU 103 sets a settable flicker reduction shutter speed (SetPosFlkTv) from the shutter speed setting table, with index i=1 of the shutter speed setting table. Note that in this embodiment, as shown in FIG. 2, when index i=1, SetPosFlkTv=1 / 8192.0.

[0122] Next, in step S1402, CPU 103 increments index i of the shutter speed setting table by 1. Next, in step S1403, CPU 103 compares the absolute value of the difference between SetPosFlkTv and the above-mentioned ideal flicker reduction exposure time IdealFlkExpTime for each shutter speed [i] in the shutter speed setting table. If the difference between SetPosFlkTv and IdealFlkExpTime is less than or equal to the difference between shutter speed [i] and IdealFlkExpTime (determined NO in step S1403), the process proceeds to step S1405.

[0123] On the other hand, if it is determined that the difference between SetPosFlkTv and IdealFlkExpTime is greater than the difference between shutter speed[i] and IdealFlkExpTime (YES in step S1403), the process proceeds to step S1404. Then, in step S1404, CPU 103 selects a settable flicker reduction shutter speed based on the determination result of step S1403. Specifically, in step S1404, CPU 103 sets the settable flicker reduction shutter speed SetPosFlkTv to the shutter speed[i] corresponding to index i in the current shutter speed setting table, and then proceeds to step S1405.

[0124] Next, in step S1405, CPU 103 determines whether index i in the shutter speed setting table is equal to or greater than the maximum index. If the current index [i] is smaller than the maximum index (determined NO in step S1405), the process returns to step S1402, and steps S1402 to S1405 are repeated. Note that the maximum index in this embodiment is 600, as shown in FIG. 2. If it is determined in step 1405 that the current index [i] has reached the maximum index (YES in step S1405), the current SetPosFlkTv is selected as the settable flicker reduction shutter speed, and the shutter speed selection process ends.

[0125] In the above example, the shutter speed selection process is performed for all indexes that can be referenced in the shutter speed setting table. However, this is not limiting. For example, if the currently set shutter speed CurTv is acquired during the flicker reduction exposure time determination process, the available flicker reduction shutter speeds may be determined by narrowing down the search to the vicinity of CurTv. Specifically, if a specific value is recorded as the currently set shutter speed CurTv, the index corresponding to the shutter speed closest to CurTv is identified. Then, the difference between the ideal flicker reduction exposure time and the shutter speeds corresponding to that index and other indexes adjacent to that index is calculated, and the shutter speed with the smallest difference is determined as the available flicker reduction shutter speed. This configuration is particularly effective when a specific shutter speed is set by the user. By adopting this configuration, the deviation from the user's intended shutter speed is minimized, and the number of indexes to be compared is significantly reduced, thereby reducing the processing time and processing load associated with the shutter speed selection process.

[0126] By performing the shutter speed selection process described above, it is possible to select a shutter speed that can effectively reduce the effects of pre-detected flicker from among the shutter speeds that can be set by camera body 100. In other words, camera body 100 of this embodiment can select (set) the shutter speed that is closest to IdealFlkExpTime, the ideal shutter speed for reducing the effects of detected flicker, from among the shutter speeds that can be set.

[0127] 17A and 17B are diagrams illustrating an example of the relative relationship between the shutter speed selected by the shutter speed selection process according to the first embodiment of the present invention and the ideal shutter speed for reducing the effects of flicker. In FIG. 17A, it is assumed that the light intensity change frequency of flicker is 540.0 Hz and the ideal flicker reduction exposure time IdealFlkExpTime is 1 / 540.0. FIG. 17A shows a case where the shutter speed (CurTv) currently set by the user is 1 / 5792.6, and FIG. 17B shows a case where the shutter speed (CurTv) currently set by the user is 1 / 250.5.

[0128] 17(a), the difference between Tv=1 / 546.4 indicated by index 58 in the shutter speed setting table and IdealFlkExpTime Tv=1 / 540.0 is shown as Δ58. Also, in FIG. 17(a), the difference between Tv=1 / 534.7 indicated by index 59 in the shutter speed setting table and IdealFlkExpTime Tv=1 / 540.0 is shown as Δ59. In the case shown in FIG. 17(a), Δ59<Δ58, so Tv=1 / 534.7 is selected as SetPosFlkTv by the shutter speed selection process described above.

[0129] 17(b), the difference between Tv=1 / 273.2 indicated by index 119 in the shutter speed setting table and IdealFlkExpTime Tv=1 / 270.0 is shown as Δ119. Also, in FIG. 17(b), the difference between Tv=1 / 270.2 indicated by index 120 in the shutter speed setting table and IdealFlkExpTime Tv=1 / 270.0 is shown as Δ120. In the case shown in FIG. 17(b), Δ120<Δ119, so Tv=1 / 270.2 is selected as SetPosFlkTv by the shutter speed selection process described above.

[0130] As described above, the camera body 100 of this embodiment can effectively detect the light intensity change frequency of flicker occurring in the current shooting environment and the ideal shutter speed (exposure time) that reduces the effects of the detected flicker in the shortest possible time.

[0131] Furthermore, camera body 100 of this embodiment can set a shutter speed that takes into account the shutter speed currently set by the user, etc., as the ideal shutter speed for reducing the effects of flicker. Therefore, camera body 100 of this embodiment can detect a shutter speed that can reduce the effects of flicker while minimizing differences in the exposure conditions and shooting effects intended by the user.

[0132] Furthermore, camera body 100 of this embodiment can automatically select (set) a shutter speed that is closest to the ideal shutter speed that can reduce the effects of flicker among the shutter speeds that can be set by camera body 100. Therefore, camera body 100 of this embodiment can automatically select (set) a shutter speed that can reduce the effects of flicker without the need for the user to manually adjust the shutter speed, etc.

[0133] Next, the display processing of step S306 according to the first embodiment of the present invention will be described in detail with reference to Figures 18 and 17. Figure 18 is a diagram illustrating an example of a notification image displayed on the display unit 102 by the display processing according to the first embodiment of the present invention.

[0134] Of these, Fig. 18(a) shows a case where a 540.0 Hz flicker is detected, CurTv is 1 / 5792.6, and SetPosFlkTv is 1 / 534.7. Fig. 18(b) shows a case where a 540.0 Hz flicker is detected, CurTv is 1 / 250.5, and SetPosFlkTv is 1 / 270.2. Fig. 19 is a diagram illustrating an example of a notification image when no flicker is detected by the display processing according to the first embodiment of the present invention.

[0135] In the detected flicker area 1601, information indicating the frequency of change in the amount of light of the flicker detected based on the method described above is displayed (540.0 Hz in the illustrated example).

[0136] The selectable shutter speed area 1602 displays the settable flicker reduction shutter speed SetPosFlkTv calculated based on the method described above (1 / 534.7 is shown in FIG. 18(a), and 1 / 250.5 is shown in FIG. 18(b)).

[0137] The current shutter speed area 1603 displays the shutter speed currently set on the camera body 100, for example, as manually set by the user (1 / 5792.6 in Figure 18(a) and 1 / 270.2 in Figure 18(b)).

[0138] The first user selection icon 1604 displays an option for when the user does not agree to the change to the configurable flicker reduction shutter speed SetPosFlkTv displayed on the notification screen, while the second user selection icon 1605 displays an option for when the user agrees to the change to the configurable flicker reduction shutter speed SetPosFlkTv displayed on the notification screen.

[0139] Furthermore, if the flicker detection process does not detect flicker at a predetermined level or above, as shown in FIG. 19, a description 1701 indicating that flicker has not been detected and an icon 1702 allowing the user to input whether or not to confirm are displayed on the display unit 102.

[0140] As described above, when flicker at a predetermined light intensity change frequency is detected by the flicker detection process, various icons and text such as those shown in Figures 18(a) and (b) are displayed on the display unit 102 to prompt the user to change the shutter speed. This configuration, for example, reduces the need for the user to manually adjust the shutter speed to reduce the effects of flicker, and makes it possible to easily set a shutter speed that can reduce the effects of flicker. Therefore, the camera body 100 according to the present invention can capture images that reduce the effects of flicker at a wide range of light intensity change frequencies, regardless of the light source, without requiring complex operations, and can reduce unevenness in the image caused by flicker.

[0141] Furthermore, as described above, if the result of the processing in step S304 indicates that the number of changes in light intensity of the currently occurring flicker exceeds the detection target range, the user is notified in S308 that flicker outside the detection target range is occurring. Fig. 20 is a diagram illustrating an example of a notification image displayed when flicker exceeds the detection target frequency range by the display processing according to the first embodiment of the present invention. As shown in Fig. 20, if the light intensity change frequency of the currently occurring flicker is outside the frequency range that can be detected with high accuracy, the user can easily know this, thereby reducing the number of failed images captured due to the influence of flicker.

[0142] Furthermore, the camera body 100 of this embodiment can manually change the shutter speed even if the light intensity change frequency of flicker cannot be accurately detected through automatic detection. Therefore, as shown in FIG. 21 , a configuration may be adopted in which the user is prompted to switch to an operation screen for manually changing the shutter speed when the currently occurring flicker exceeds the detection range. FIG. 21 illustrates an example of a notification image used by the display processing according to the first embodiment of the present invention to guide the user to a method for manually setting the shutter speed when the flicker exceeds the frequency range to be detected. This configuration reduces the number of user operations required to transition from the automatic flicker detection state to the manual flicker detection state. Note that the processing in step S308 may be configured to display the screen shown in FIG. 19.

[0143] Note that the method of informing the user of the frequency of change in light intensity of flicker, the shutter speed that can reduce the effects of flicker, and the method of changing the shutter speed are not limited to those described above. For example, in the above example, a notification image is displayed on the display unit 102, but the notification image may be displayed on another display device or an external device connected to the camera body 100. Furthermore, the notification method does not have to be limited to image display. Various notification means such as audio guidance or changing the lighting state or color of a lamp (not shown) provided on the camera body 100 can be used instead.

[0144] Furthermore, while the camera body 100 according to the present embodiment employs a method of prompting the user to confirm whether or not to change to a settable flicker reduction shutter speed, the present invention is not limited to this. For example, the shutter speed may be automatically changed to a settable flicker reduction shutter speed without the user's consent, or the camera may be configured to determine whether or not to prompt the user about whether or not to change to a settable flicker reduction shutter speed depending on the shooting mode.

[0145] In this case, if the shooting mode is an auto mode in which the exposure control parameters are automatically determined by camera body 100, it is preferable for camera body 100 to automatically set the settable flicker reduction shutter speed. On the other hand, if the shooting mode is a manual mode in which the exposure control parameters (exposure control values) are manually set by the user, it is preferable to employ a method of asking the user whether or not to change the shutter speed, as in the example described above.

[0146] Furthermore, while the camera body 100 according to this embodiment has been described as preferentially using the electronic shutter, this is not limiting. For example, the camera body 100 may be configured to use the mechanical shutter 104 to adjust the exposure time of the image sensor 101 in accordance with an arbitrary shutter speed.

[0147] When capturing an image of a subject using mechanical shutter 104 with a high shutter speed set, the timing at which mechanical shutter 104 operates relative to the ideal exposure time may deviate depending on changes in the physical characteristics of mechanical shutter 104 and differences in the environment. In other words, if the shutter speed set as the settable flicker reduction shutter speed SetPosFlkTv is high, it may not be possible to capture an image of the subject with an exposure time that can properly reduce the effects of flicker.

[0148] Therefore, when adjusting the exposure time using the mechanical shutter 104, a configuration may be adopted in which the settable flicker reduction shutter speed SetPosFlkTv is limited so that the shutter speed is equal to or greater than a predetermined speed. The predetermined speed (shutter speed) may be any value that ensures that the deviation (i.e., error) between the ideal exposure time and the timing of exposure and shading of the image sensor 101 caused by driving the mechanical shutter 104 falls within a predetermined range. In this embodiment, the predetermined shutter speed is illustratively set to 1 / 4000 seconds. In this case, the aforementioned shutter speed setting table may be used within a range excluding indexes corresponding to shutter speeds equal to or less than 1 / 4000 seconds, or new table data may be used to determine the settable flicker reduction shutter speed.

[0149] Note that camera body 100 of this embodiment may be configured to dynamically adjust whether to use the electronic shutter or the mechanical shutter 104 depending on the value of settable flicker reduction shutter speed SetPosFlkTv. For example, when the shutter speed is faster than 1 / 4000 seconds, only the electronic shutter may be usable, and at other shutter speeds, both the electronic shutter and the mechanical shutter 104 may be usable.

[0150] (Second embodiment) In the first embodiment described above, a configuration was described in which only one arbitrarily settable flicker reduction shutter speed was notified to the user. In contrast, in this embodiment, a configuration in which a plurality of options are notified to the user as settable flicker reduction shutter speeds will be described with reference to FIG. 22. Note that the configurations and basic driving methods of the camera body 100, lens unit 200, and light emitting device 300, which are the imaging device according to this embodiment, are substantially the same as in the first embodiment described above, and therefore the same reference numerals are used for each component, and description thereof will be omitted. This embodiment differs from the first embodiment described above in the display processing of step S306.

[0151] 22A and 22B are diagrams illustrating exemplary notification images displayed on the display unit 102 by the display process according to the second embodiment of the present invention. Fig. 22A shows a case where a 540.0 Hz flicker is detected, CurTv is 1 / 5792.6, and SetPosFlkTv is 1 / 534.7. Fig. 22B shows a case where a 540.0 Hz flicker is detected, CurTv is 1 / 250.5, and SetPosFlkTv is 1 / 270.2.

[0152] Information indicating the frequency of change in light intensity of detected flicker is displayed in the detected flicker area 1801. The current shutter speed area 1802 displays the shutter speed currently set in the camera body 100, such as by manual setting by the user (1 / 5792.6 is shown in FIG. 22(a) and 1 / 270.2 is shown in FIG. 22(b)).

[0153] The settable flicker reduction shutter speed SetPosFlkTv obtained based on the method described in the first embodiment is displayed as the first candidate shutter speed selectable by the user in the selectable shutter speed first candidate area 1803. Note that the selectable shutter speed first candidate area 1803 shows 1 / 534.7 in Fig. 22(a) and 1 / 270.2 in Fig. 22(b).

[0154] The shutter speed corresponding to the index whose difference from IdealFlkExpTime is the next smallest after SetPosFlkTv is displayed as the second candidate for the shutter speed selectable by the user in the second candidate selectable shutter speed area 1804. Note that the second candidate selectable shutter speed area 1804 shows 1 / 546.4 in Figure 22(a) and 1 / 273.2 in Figure 22(b).

[0155] In the selectable shutter speed candidate area 1805, if there is a shutter speed that is more effective at reducing the effects of flicker, regardless of the difference from CurTv, the corresponding shutter speed is displayed as another candidate shutter speed that the user can select. For example, in FIG. 22(a), the selectable shutter speed candidate area 1805 displays 1 / 270.2, which is close to Tv=1 / 270.0, which is twice the IdealFlkExpTime Tv=1 / 540.0. In a situation where 540 Hz flicker is detected, Tv=1 / 270.2 has a larger difference from CurTv, but is more effective at reducing the effects of flicker than SetPosFlkTv (1 / 534.7).

[0156] The shutter speed selection icon 1806 displays an icon that allows the user to select from selectable shutter speed candidates. Among these icons, a white arrow indicates that no candidate shutter speeds exist, and a black arrow indicates that a candidate shutter speed exists. In FIG. 22(a), since there are no other SetPosFlkTv candidates in the first selectable shutter speed candidate area 1803, a white arrow icon is displayed next to the first selectable shutter speed candidate area 1803n. This also applies to the example shown in FIG. 22(b). Furthermore, in FIG. 22(a), since another shutter speed (1 / 180.0) that is highly effective in reducing the effects of flicker exists in the selectable shutter speed candidate area 1805, a black arrow icon is displayed next to the selectable shutter speed candidate area 1805. In Figure 22(b), since there is another shutter speed (1 / 135.0) that is more effective at reducing the effects of flicker than the candidate area 1805 for selectable shutter speeds, a black arrow icon is displayed next to the candidate area 1805 for selectable shutter speeds.

[0157] As described above, camera body 100 of this embodiment can notify the user of multiple candidates, in addition to the arbitrary SetPosFlkTv, as shutter speeds that can reduce the effects of flicker. This configuration, for example, reduces the user's manual operation to adjust the shutter speed to reduce the effects of flicker, and allows the user to easily set the shutter speed they desire from multiple candidates that can reduce the effects of flicker. Therefore, camera body 100 according to the present invention can capture images that reduce the effects of flicker over a wide range of light intensity change frequencies, regardless of the light source, without requiring complex operations, and can reduce unevenness in the image caused by flicker.

[0158] (Third embodiment) In the first embodiment described above, an example was described in which a specific notification screen was displayed on the display unit 102. In contrast to this, in this embodiment, a configuration for performing flicker detection processing during live view display in which captured images are displayed sequentially will be described with reference to Fig. 23. Note that the configurations and basic driving methods of the camera body 100, lens unit 200, and light emitting device 300, which are the image capture device according to this embodiment, are substantially the same as those in the first embodiment described above, and therefore the same reference numerals are used for the various components, and their description will be omitted.

[0159] 23 is a diagram illustrating an example of a transition screen to flicker reduction processing during live view display according to the third embodiment of the present invention. Note that, although this embodiment describes a configuration in which live view display is performed on the display unit 102, a configuration in which live view display is performed on an electronic viewfinder (not shown) may also be used. Note that, during live view display, sampling (charge accumulation) for flicker detection is performed in the image sensor 101 at a timing different from the timing of charge accumulation for obtaining a captured image used for live view display.

[0160] 23, flicker detection icon 1901 is an icon display for indicating that flicker has been detected when flicker is detected by the flicker detection process described above in the first embodiment. Note that if a flicker detection process different from the flicker detection process described above can be executed, icon 1901 may be configured to display the same as icon 1901, or an icon different from icon 1901 may be used. Here, as another flicker detection process, a process for detecting specific flicker (100 Hz, 120 Hz) that occurs due to periodic changes in commercial power may be envisioned.

[0161] Also, icon 1901 may be configured to be displayed only when flicker is detected, or may be configured to be constantly displayed as an icon and change (update) the display content depending on whether flicker is detected. Furthermore, a configuration may be adopted in which CPU 103 performs control so that flicker detection processing is executed when the user presses flicker detection icon 1901.

[0162] The flicker reduction menu icon 1902 is an icon for transitioning the display content of the display unit 102 to the notification screen described in the first and second embodiments when the icon is pressed (including touched) by the user. In other words, the color body 100 according to this embodiment allows the user to transition directly to the notification screen during live view display without going through another user interface such as a menu screen.

[0163] As described above, the camera body 100 of this embodiment can detect flicker that changes over a wide range of frequencies even during subject shooting conditions, such as during live view display, and transition to image capture with reduced effects of flicker through simple user operations. This configuration reduces the number of manual operations required by the user for flicker detection, while allowing the user to easily set the desired shutter speed from among multiple options that can reduce the effects of flicker. Therefore, the camera body 100 of the present invention can perform image capture with reduced effects of flicker over a wide range of light intensity change frequencies, regardless of the light source, without requiring complex operations, and can reduce unevenness in the image caused by flicker.

[0164] (Fourth embodiment) In the first embodiment described above, the flicker reduction exposure time determination process was described for the case where the current shutter speed CurTv was set in advance. In contrast, in this embodiment, the flicker reduction exposure time determination process will be described for the case where a specific shutter speed (CurTv) has not been set, for example, by manual operation by the user. Note that the configurations and basic driving methods of the camera body 100, lens unit 200, and light emitting device 300, which are the image capture device according to this embodiment, are substantially the same as those in the first embodiment described above, and therefore the same reference numerals are used for each component, and description thereof will be omitted.

[0165] In addition to the auto mode and manual mode described above, the shooting modes that can be set on camera body 100 include a priority mode in which the user manually sets an exposure control value and other exposure control values ​​are automatically set. As this priority mode, camera body 100 according to the present embodiment can set, for example, a shutter speed priority mode in which the user can manually set the shutter speed.

[0166] For example, in an automatic exposure control state where the shooting mode of camera body 100 is set to auto mode or the like, the user cannot set an arbitrary shutter speed. Therefore, in the flicker reduction exposure time determination process in the first embodiment described above, there is little need to determine the ideal flicker reduction exposure time IdealFlkExpTime taking into account the current shutter speed CurTv.

[0167] Therefore, in this embodiment, the ideal flicker reduction exposure time IdealFlkExpTime is determined based on the result of a determination as to whether the current shutter speed CurTv is the shutter speed CurUserTv manually set by the user. More specifically, in the camera body 100 of this embodiment, the CPU 103 determines whether CurTv≠CurUserTv. If it is determined that CurTv≠CurUserTv, the CPU 103 sets the shutter speed in the shutter speed setting table that has the smallest difference from the ideal flicker reduction exposure time as the settable flicker reduction shutter speed.

[0168] If the above-described configuration is applied to the flicker reduction exposure time determination process, the processes from step S1203 to step S1205 onward are unnecessary. In this case, the ideal flicker reduction exposure time IdealFlkExpTime is set to an exposure time that is the reciprocal of the light intensity change frequency of the detected flicker, but this is not limiting. For example, as described in the second embodiment, a settable flicker reduction shutter speed may be set so that the difference between the ideal flicker reduction exposure time multiplied by an integer N is minimized, so as to maximize the effect of reducing the effects of flicker. In this case, a shutter speed that can be set according to the shutter speed setting table is repeatedly compared with an integer multiple of the ideal flicker reduction exposure time IdealFlkExpTime. The shutter speed with the smallest difference is then selected as the settable flicker reduction shutter speed SetPosFlkTv.

[0169] For example, in the first and second embodiments described above, the value of the settable flicker reduction shutter speed SetPosFlkTv was determined assuming that CurTv had been set and taking into account the difference from CurTv, but this is not limited to this. For example, the camera body 100 may compare the difference between the shutter speed corresponding to each index and the frequency of change in light intensity of flicker and the reciprocal of its integer multiple, and determine the value with the smallest difference as the settable flicker reduction shutter speed SetPosFlkTv. In this case, a range of the frequency of change in light intensity of flicker that can be reduced by shutter speeds that can be set on the camera body 100 may be determined, and only the reciprocals of frequencies that fall within this range may be compared.

[0170] In this embodiment, the determination as to whether or not CurTv≠CurUserTv may be made based on the shooting mode currently set in the camera body 100.

[0171] As described above, camera body 100 of this embodiment can calculate an optimal shutter speed that can effectively reduce the effects of flicker that changes over a wide range of frequencies, even if the user has not set an arbitrary shutter speed. This configuration makes it possible to easily set a shutter speed that can most effectively reduce the effects of flicker, regardless of the shooting conditions of camera body 100, and without requiring complicated operations by the user.

[0172] Therefore, the camera body 100 according to the present invention can capture images that reduce the effects of flicker over a wide range of light intensity change frequencies, regardless of the light source, without requiring complex operations, and can reduce unevenness in the image caused by flicker.

[0173] (Fifth embodiment) In the first embodiment described above, flicker reduction processing related to imaging a subject when capturing a still image was explained. In contrast, in this embodiment, flicker reduction processing related to imaging a subject when capturing a moving image will be explained. Note that the configurations and basic driving methods of the camera body 100, lens unit 200, and light emitting device 300, which are the imaging device according to this embodiment, are substantially the same as those of the first embodiment described above, so the same reference numerals are used for each component and their explanations will be omitted.

[0174] When capturing moving images, the shutter speed that can be set is limited by the update period of each frame that forms the moving image. That is, there are shutter speeds that cannot be set depending on the recording frame rate of the moving image.

[0175] Furthermore, even among the available shutter speeds, there are some that are not desirable for capturing moving images. For example, if the shutter speed is too short, the exposure time for each frame will be too short, and the time difference between each frame that forms the moving image will be too large, making the movement of the subject in the moving image appear less smooth.

[0176] Therefore, in this embodiment, for flicker reduction processing when capturing a moving image, the ideal flicker reduction exposure time is set to the longest exposure time that can be set at the set frame rate of the moving image. Note that the flicker reduction exposure time and the settable flicker reduction shutter speed may not match. Therefore, if the settable flicker reduction shutter speed selected based on the newly determined ideal flicker reduction exposure time is a value that cannot be set at the current frame rate of the moving image, the settable flicker reduction shutter speed is adjusted. Specifically, the settable flicker reduction shutter speed is set to the shutter speed that is closest to the newly determined ideal flicker reduction exposure time among the shutter speeds that are not limited by the frame rate of the moving image.

[0177] In this embodiment, the process of comparing with CurTv in the process of determining the flicker reduction exposure time described above can be omitted. However, a configuration may be adopted in which the longest exposure time among (an integer multiple of) the ideal flicker reduction exposure time, whose difference from the current shutter speed CurTv falls within a predetermined range, is set as the final ideal flicker reduction exposure time.

[0178] As described above, the camera body 100 of this embodiment is capable of detecting flicker that changes over a wide range of frequencies and capturing images with reduced effects of flicker, even when capturing a subject to acquire a moving image, while preventing degradation of the moving image quality. With this configuration, the camera body 100 of this embodiment allows the user to easily set a shutter speed that reduces the effects of flicker when capturing both still images and moving images, without requiring any additional operations by the user. Therefore, the camera body 100 of the present invention can capture images with reduced effects of flicker over a wide range of light intensity change frequencies, regardless of the light source, without requiring complex operations, and can reduce unevenness in the image caused by flicker.

[0179] (Sixth embodiment) In the first embodiment described above, a configuration was described in which an ideal flicker reduction exposure time was set so that the difference with the current shutter speed CurTv was small. In contrast, in this embodiment, a method for setting an ideal flicker reduction exposure time that can reduce the effects of camera shake and subject blur will be described. Note that the configurations and basic driving methods of the camera body 100, lens unit 200, and light emitting device 300, which are the image capture device according to this embodiment, are substantially the same as in the first embodiment described above, so the same reference numerals are used for the various components and their descriptions will be omitted.

[0180] Generally, as the shutter speed (exposure time) increases, the probability of capturing an image with blurred subject portions increases due to the effects of camera shake and subject movement (so-called subject blur) during image capture. In other words, to reduce blurring in an image, it is desirable to shorten the shutter speed as much as possible.

[0181] In the flicker reduction exposure time determination process according to the first embodiment, the camera body 100 according to this embodiment determines an ideal flicker reduction exposure time that is shorter than the predetermined exposure time. The predetermined exposure time may be any value that can reduce the effects of subject blur in an image, but in this embodiment, the predetermined exposure time is illustratively set to 1 / 125 seconds.

[0182] In this embodiment, the process of comparing with CurTv in the flicker reduction exposure time determination process described above can be omitted. However, a configuration may be adopted in which the ideal flicker reduction exposure time is determined to be an exposure time (an integer multiple of the ideal flicker reduction exposure time) whose difference from the current shutter speed CurTv falls within a predetermined range and which is shorter than the predetermined exposure time.

[0183] Furthermore, when a condition for reducing blur (for example, a specific shooting scene (such as a sports scene)) is set as a shooting condition for camera body 100, the camera may be configured to set an ideal flicker reduction exposure time that reduces the effects of blur on the subject.

[0184] As described above, the camera body 100 of this embodiment is capable of detecting flicker that changes over a wide range of frequencies and capturing images with reduced flicker effects while suppressing the effects of subject blur in the image. With this configuration, the camera body 100 of this embodiment can easily set a shutter speed that reduces the effects of flicker, without requiring any additional user operations, even when specific shooting conditions are set that aim to reduce blur. Therefore, the camera body 100 of the present invention can capture images with reduced effects of flicker over a wide range of light intensity change frequencies, regardless of the light source, without requiring complex operations, and can reduce unevenness in the image caused by flicker.

[0185] (Seventh embodiment) In this embodiment, a description will be given of flicker reduction processing that occurs during light-emitting imaging using a light-emitting device 300. Note that the configurations and basic driving methods of the camera body 100, which is the imaging device according to this embodiment, the lens unit 200, and the light-emitting device 300 are substantially the same as those of the first embodiment described above, and therefore the same reference numerals are used for the various components, and their description will be omitted.

[0186] When photographing with flash using the light emitting device 300, the settable flicker reduction shutter speed is limited by a synchronization speed determined based on the timing at which the image sensor 101 is exposed and the timing at which the light is emitted by the light emitting device 300. That is, the camera body 100 according to this embodiment sets the settable flicker reduction shutter speed from candidate shutter speeds that are slower than the synchronization speed of the light emitting device 300. Specifically, the CPU 103 determines whether or not to perform photographing with flash using the light emitting device 300. If it is determined that photographing with flash will be performed, the CPU 103 limits the selectable shutter speeds in the shutter speed setting table to a range that is slower than the synchronization speed of the light emitting device 300.

[0187] In this embodiment, the process of comparing with CurTv in the process of determining the flicker reduction exposure time described above can be omitted. However, a configuration may be adopted in which the final ideal flicker reduction exposure time is the tuning speed of the light emitting device 300 that has the smallest difference from the current shutter speed CurTv among the ideal flicker reduction exposure times (integral multiples of the ideal flicker reduction exposure time).

[0188] As described above, the camera body 100 of this embodiment is capable of detecting flicker that changes over a wide range of frequencies and capturing images with reduced effects of flicker while maintaining an appropriately illuminated subject, even when capturing images with light using a light-emitting device. This configuration allows the camera body 100 of this embodiment to easily set a shutter speed that reduces the effects of flicker during flash capture, without requiring any additional user operations. Therefore, the camera body 100 of the present invention can capture images with reduced effects of flicker over a wide range of light intensity change frequencies, regardless of the light source, without requiring complex operations, and can reduce unevenness in the image caused by flicker.

[0189] While preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments and various modifications and variations are possible within the scope of the present invention. For example, in the above-described embodiments, a digital camera was assumed as an example of an imaging device for implementing the present invention, but the present invention is not limited to this. For example, imaging devices other than digital cameras, such as portable devices such as digital video cameras and smartphones, wearable devices, in-vehicle cameras, and security cameras, may also be used.

[0190] Furthermore, in the above-described embodiment, a configuration capable of detecting and reducing flicker that changes over a wide range of frequencies without specifying a light source has been described, but the present invention is not limited to this. For example, a configuration may be adopted in which a specific light source is designated in advance and flicker is detected according to the frequency range in which it is likely to occur. In this case, for example, a configuration may be adopted in which table data is prepared for each light source (or a group of similar light sources) as in the shutter speed setting table illustrated in FIG. 2, and shutter speeds that are likely to be set for each table data are limited based on the light intensity change period of the light source. With this configuration, a shutter speed that can reduce the effects of flicker can be efficiently set according to the flicker that is likely to occur with each light source, thereby making it possible to reduce the amount of table data as much as possible while effectively reducing the effects of flicker.

[0191] In the above-described embodiment, the various components constituting the imaging system cooperate with each other, centering on the CPU 103, to control the operation of the entire device. However, this is not limiting. For example, a (computer) program following the flow illustrated in each of the above-described figures may be stored in advance in a ROM or the like of the camera body 100. Then, the program may be executed by a microprocessor such as the CPU 103 to control the operation of the entire imaging system. Furthermore, as long as the program functions, any form of program may be used, such as object code, a program executed by an interpreter, or script data supplied to an OS. Furthermore, the recording medium for supplying the program may be, for example, a magnetic recording medium such as a hard disk or magnetic tape, or an optical / magneto-optical recording medium.

[0192] In the above-described embodiment, a digital camera has been described as an example of an imaging device for implementing the present invention, but the present invention is not limited to this. For example, various imaging devices may be used, such as a portable device such as a digital video camera or a smartphone, a wearable device, or a security camera.

[0193] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]

[0194] 100 camera body 101 Image sensor 102 Display Unit 103 CPU 200 shooting lenses 300 Light-emitting device

Claims

1. a detecting means for detecting a flicker frequency of the light source by sampling an output signal of the image sensor through a first sampling process; a determination means for determining whether or not the flicker frequency detected by the detection means is within a detection target range by sampling the output signal of the image sensor using a second sampling process different from the first sampling process; a control means for controlling the device to notify a user when the determination means determines that the flicker frequency detected by the detection means is outside a detection range for the first sampling process, In the first sampling process, sampling is performed at a first sampling frequency, and in the second sampling process, sampling is performed at a second sampling frequency that is higher than the first sampling frequency; The imaging device, wherein the second sampling frequency is a value based on a Nyquist frequency corresponding to the first sampling frequency.

2. 2. The imaging device according to claim 1, wherein the second sampling process is simpler than the first sampling process.

3. 3. The image pickup apparatus according to claim 2, wherein the first sampling process and the second sampling process have different sampling numbers in the vertical direction of the image pickup element.

4. 4. The imaging device according to claim 3, wherein the number of samples taken in the vertical direction of the imaging element in the second sampling process is smaller than that in the first sampling process.

5. When the determination means determines that the flicker frequency detected by the detection means is within a detection target range, the control means 5. The imaging device according to claim 1, wherein the information that flicker has been detected is notified to a user.

6. 2. The imaging device according to claim 1, wherein, when the determination means determines that the flicker frequency detected by the detection means is outside a detection target range, the control means prompts the user to manually set the shutter speed.

7. 2. The imaging device according to claim 1, wherein, when the determination means determines that the flicker frequency detected by the detection means is outside the detection target range, the control means notifies the user that the flicker frequency is outside the detection target range.

8. a detecting step of detecting a flicker frequency of a subject light source by sampling an output signal of the image sensor in a first sampling process; a determination step of determining whether or not the flicker frequency detected in the detection step is within a detection target range by sampling the output signal of the image sensor using a second sampling process different from the first sampling process; a control step of controlling to notify a user when it is determined in the determination step that the flicker frequency detected in the detection step is outside a detection target range for the first sampling process, In the first sampling process, sampling is performed at a first sampling frequency, and in the second sampling process, sampling is performed at a second sampling frequency that is higher than the first sampling frequency; The flicker detection method according to claim 1, wherein the second sampling frequency is a value based on a Nyquist frequency corresponding to the first sampling frequency.

9. A computer-readable program for causing a computer to execute the flicker detection method according to claim 8.

Images