Electronic device, control method thereof and storage medium

Abstract

The present invention provides an electronic device, a control method thereof, and a storage medium. The electronic device includes: an image sensor, which includes a first pixel group and a second pixel group, and is capable of driving the pixel groups using the same or different shooting conditions; a controller, which switches between a first drive control that drives the first pixel group and the second pixel group using the same shooting conditions and a second drive control that drives the first pixel group and the second pixel group using different shooting conditions; a detector that detects flicker based on a signal read out from the image sensor; and a judgment unit that judges whether the detector is capable of detecting flicker based on a signal read out from the image sensor through the second drive control. If the judgment unit judges as positive, the controller controls to drive the second pixel group using the shooting conditions for flicker detection in the second drive control.

Description

Technical Field

[0001] The present invention relates to an electronic device capable of photographing images, a control method thereof and a storage medium, and in particular to a technology for detecting flicker. Background Art

[0002] When capturing images using a digital camera, mobile phone, camcorder, or the like, brightness may vary within one image or between consecutively captured images due to the influence of the light source in the shooting environment. If the light source is a lighting device using commercial power, flicker, a phenomenon in which illumination light fluctuates periodically, may occur due to the influence of the frequency of the commercial power source. When capturing images under a light source that generates such flicker (hereinafter referred to as a "flicker light source"), uneven exposure and / or uneven color may occur in one image, or exposure and color temperature may vary between a plurality of consecutively captured images.

[0003] For flicker that occurs when the power frequency is 50Hz, the flicker light source blinks at 100Hz, so the influence of flicker can be avoided by controlling the charge accumulation period to N / 100 seconds (N is a natural number). In addition, for flicker that occurs when the power frequency is 60Hz, the flicker light source blinks at 120Hz, so the influence of flicker can be avoided by controlling the charge accumulation period to N / 120 seconds. Therefore, if it is possible to specify whether the power frequency is 50Hz or 60Hz, the influence of flicker on image quality can be reduced.

[0004] Furthermore, conventionally, a technique is known in which a change in brightness of continuously captured images is analyzed and the frequency of flicker that is occurring is specified. Japanese Patent Publication No. 6225000 discloses a technique in which flicker is detected by switching shooting conditions such as a drive cycle and exposure conditions of an image pickup device from a normal shooting condition to a shooting condition suitable for flicker detection, so that flicker can be detected satisfactorily. However, since detection is performed by switching from a normal shooting condition to a shooting condition suitable for flicker detection, flicker cannot be detected during shooting of a moving image or while a live view image is displayed.

[0005] Japanese Patent Publication No. 2020-80512 discloses a method for detecting the frequency of flickering occurring in a subject while displaying a live view image or capturing a moving image. In this method, a second pixel row is read out at a different timing and cycle from a first pixel row used for live view display or moving image capture, and an image signal of the second pixel row is used to analyze the flickering of a light source. For example, an image signal can be obtained by setting the frame rate and exposure period of the second pixel row to shooting conditions that facilitate flicker detection.

[0006] For example, in the case where the flicker frequency is 120 Hz, if the frame rate of the moving image is set to 60 fps, the frame rate is synchronized with the flicker of 120 Hz, and the influence of the flicker does not change between frame images, so the flicker cannot be detected from the image signal obtained from the first pixel row. On the other hand, since the second pixel row can be set to a shooting condition suitable for detecting flicker, by reading the image signal at, for example, 90 fps (which is not synchronized with the 100 Hz flicker or the 120 Hz flicker), flicker of either frequency can be satisfactorily detected.

[0007] However, in recent years, the performance of image sensors has improved, and there are image sensors that can read out image signals at very high speeds. For example, it is possible to perform slow motion recording at a frame rate of 120fps or more in motion image shooting, and to shoot and display smooth images with almost no delay at 120fps or 240fps in live view display.

[0008] With such an image sensor, when capturing a moving image at a frame rate faster than a flicker frequency of 100 Hz or 120 Hz, it is not possible to set the charge accumulation period of each frame image of the moving image to N / 100 seconds or N / 120 seconds. Therefore, it is not possible to reduce the effect of flicker caused by the flickering of the light source by adjusting the charge accumulation period. As a result, as shown in Japanese Patent Gazette No. 2020-80512, although it is possible to detect whether flicker occurs and the frequency of flicker by analyzing the changes in the image signal obtained from the second pixel row, it is not possible to reduce the flicker that appears in the image of the image signal obtained from the first pixel row.

[0009] Furthermore, the method for detecting flicker in Japanese Patent Publication No. 2020-80512 has a problem of high power consumption compared to the method in Japanese Patent Publication No. 6225000 and the conventionally performed method. Summary of the invention

[0010] The present invention has been made in consideration of the above circumstances, and performs flicker detection based on this purpose.

[0011] According to the present invention, there is provided an electronic device comprising: an image sensor having a plurality of pixels divided into a plurality of pixel groups including a first pixel group and a second pixel group, and capable of driving each pixel group using shooting conditions independently set for each pixel group; a controller switching between a first drive control and a second drive control, wherein the first drive control drives the first pixel group and the second pixel group using the same shooting conditions, and the second drive control drives the first pixel group and the second pixel group using shooting conditions different from each other; a detector detecting flicker based on a signal read out from the image sensor; and a judgment unit judging whether the detector is capable of detecting flicker based on a signal read out from the image sensor by the second drive control, wherein if the judgment unit judges that flicker is capable of detecting flicker based on a signal read out from the image sensor by the second drive control, the controller controls to drive the second pixel group using the shooting conditions for flicker detection in the second drive control, and the detector detects flicker based on the detection signal read out from the second pixel group.

[0012] In addition, according to the present invention, there is provided a control method for an electronic device, wherein the electronic device is equipped with an image sensor, the image sensor having a plurality of pixels divided into a plurality of pixel groups including a first pixel group and a second pixel group, and each pixel group can be driven using a shooting condition independently set for each pixel group, the control method comprising: controlling to switch between a first drive control and a second drive control, wherein the first drive control drives the first pixel group and the second pixel group using the same shooting condition, and wherein the second drive control drives the first pixel group and the second pixel group using shooting conditions different from each other; detecting flicker based on a signal read out from the image sensor; and determining whether flicker can be detected based on the signal read out from the image sensor by the second drive control, wherein if it is determined that flicker can be detected based on the signal read out from the image sensor by the second drive control, controlling so that the second pixel group is driven using the shooting condition for flicker detection in the second drive control, and flicker is detected based on the detection signal read out from the second pixel group.

[0013] Furthermore, according to the present invention, there is provided a non-transitory computer-readable storage medium storing a program executable by a computer, wherein the program includes a program code for causing the computer to function as an electronic device, the electronic device being equipped with an image sensor having a plurality of pixels divided into a plurality of pixel groups including a first pixel group and a second pixel group, and being capable of driving each pixel group using a shooting condition independently set for each pixel group, and the electronic device including: a controller switching between a first drive control in which the first pixel group and the second pixel group are driven using the same shooting condition and a second drive control in which the first drive control drives the first pixel group and the second pixel group. In a second drive control, the first pixel group and the second pixel group are driven using shooting conditions different from each other; a detector detects flicker based on a signal read out from the image sensor; and a judgment unit judges whether the detector is capable of detecting flicker based on the signal read out from the image sensor through the second drive control, wherein if the judgment unit judges that flicker can be detected based on the signal read out from the image sensor through the second drive control, the controller controls to drive the second pixel group using the shooting conditions for flicker detection in the second drive control, and the detector detects flicker based on the detection signal read out from the second pixel group.

[0014] Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

[0016] Figure 1 is a diagram showing a schematic configuration of a digital mirrorless camera according to an embodiment of the present invention.

[0017] Figure 2 is a diagram showing an example of pixel arrangement in an image sensor according to an embodiment.

[0018] Figure 3A and Figure 3B is a diagram for explaining generation of a moving image and a flicker detection image in terms of pixel arrangement of an image sensor according to an embodiment.

[0019] Figure 4 is a timing chart of the motion image scan and the flicker detection scan in the second drive control according to the embodiment.

[0020] Figure 52 is a diagram illustrating features of flicker detection using a moving image and flicker detection using a flicker detection image according to an embodiment.

[0021] Figure 6 is a conceptual diagram illustrating a method of reducing the influence of flicker in a case where an image sensor is driven by a rolling shutter method according to an embodiment.

[0022] Figure 7 is a conceptual diagram illustrating a method of reducing the influence of flicker in the case where an image sensor is driven by a global shutter method according to an embodiment.

[0023] Figure 8 is a flowchart showing the flicker detection control and flicker coping method according to the first embodiment.

[0024] Fig. 9 is a flowchart showing flicker detection control according to the second embodiment.

[0025] Fig.10 It is a summary Fig. 9 The processed diagram is shown.

[0026] Fig.11A and Fig. 11B is a flowchart illustrating flicker detection control according to a modification example. DETAILED DESCRIPTION

[0027] Embodiments will be described in detail below with reference to the accompanying drawings. Note that the following embodiments are not intended to limit the scope of the present invention, and are not limited to the invention of the combination of all features required to be described in the embodiments. Two or more features of the multiple features described in the embodiments may be appropriately combined. In addition, the same reference numerals are given to the same or similar configurations, and redundant descriptions thereof are omitted.

[0028] In this embodiment, a digital mirrorless camera will be described as an example. However, the present invention can be applied to various electronic devices as long as they have a live view image display function and a moving image shooting function, and can be applied to, for example, smartphones, camcorders, game consoles with cameras, etc.

[0029] Figure 11 is a diagram showing a schematic configuration of a digital mirrorless camera according to the present embodiment, and mainly includes a camera body 100 and an imaging lens 200. In the camera body 100, reference numeral 101 denotes an image sensor such as a CCD or CMOS sensor including an infrared cut filter, a low-pass filter, etc., and an optical image of a subject is formed on a light receiving surface of the image sensor 101 through the imaging lens 200. The image sensor 101 performs photoelectric conversion on the optical image of the subject, and an image based on the obtained electrical signal is displayed as a live view image or recorded as a captured image. The electrical signal obtained from the image sensor 101 is also used to detect flicker. A method of detecting flicker using the electrical signal will be described in detail later. In the image sensor 101, the charge accumulation period can be controlled according to the shutter speed set as the shooting setting information.

[0030] The display unit 102 is composed of, for example, a TFT liquid crystal panel or the like. When framing, an image acquired by the image sensor 101 and various shooting setting information are displayed in real time to realize live framing. In addition, the captured image can be displayed according to the user operation and confirmed by the user. In addition, a touch panel can be provided on the display unit 102 and used as an operating member, and the touch panel detects the user's touch operation on the icon or the like displayed on the display unit 102, so that a function associated with the display content on the display unit 102 at the touch position can be executed.

[0031] The CPU 103 controls various parts of the camera body 100. The CPU 103 also controls the image sensor 101 and the display unit 102.

[0032] Next, the imaging lens 200 will be described.

[0033] Reference numeral 201 denotes a lens group including a focus lens, and the focus state can be adjusted by moving the focus lens. Figure 1 200 is represented by one lens, but it is actually composed of a plurality of lenses. Reference numeral 202 denotes an aperture, which adjusts the amount of light to be transmitted into the camera. Reference numeral 203 denotes a CPU that controls various parts of the imaging lens 200. In addition to controlling the lens group 201 and the aperture 202, the CPU 203 communicates with the CPU 103 in the camera body 100 to provide the camera body 100 with information such as the position of the focus lens and the aperture.

[0034] Next, a driving method of the image sensor 101 in the present embodiment will be described.

[0035] Figure 2is a diagram showing the pixel arrangement of the image sensor 101, and as an example, the pixel arrangement of the Bayer arrangement is shown. In addition, the image sensor 101 of the present embodiment has a structure that can be independently controlled in units of rows. In addition, the image sensor 101 can be selectively driven by a first drive control or a second drive control, in which all pixels of the image sensor 101 are driven under the same shooting conditions, and in the second drive control, different shooting conditions are used to drive the first pixel row (first area) and the second pixel row (second area). In the first drive control, the image sensor 101 can be driven by a conventionally commonly used reading method, and the image sensor 101 can be controlled so that signals are read out from all pixels, or signals are read out while performing thinning and / or addition. Specifically, the reading method can be changed according to the intended use of the read signal, the resolution and shooting mode set by the user, the destination standard of the read signal, the power state of the camera body 100, etc.

[0036] Next, we will refer to Figure 3A and Figure 3B The second drive control is described.

[0037] Figure 3A Shows that from Figure 2 The concept of a method for reading out a signal from a first pixel group of pixels arranged as shown and generating a moving image for recording or display. In addition, Figure 3B Shows that from Figure 2 Concept of a method for reading out signals from the second pixel group of pixels arranged as shown and generating an image to be used for detecting flicker (hereinafter referred to as a "flicker detection image" (detection signal)). In the present embodiment, the same readout control is repeated with three pixel rows as one control unit. First, in the first three rows, signals are read out from the first and third rows as the first pixel group, and signals are read out from the second row as the second pixel group. Then, in the next three rows, signals are read out from the fourth and sixth rows as the first pixel group, and signals are read out from the fifth row as the second pixel group.

[0038] When such control is repeated, signals are read out from the first pixel group by 2 / 3 thinning. Figure 3A In the pixel arrangement shown in the center of FIG, the color arrangement of the readout signals of two consecutive rows is the same. In the following description, this readout scanning is referred to as "moving image scanning". Figure 3AAs shown in the pixel arrangement on the right side of , if the signals of two consecutive rows read by the moving image scanning are added in the vertical direction, an R' signal is generated from the R signals of two consecutive pixels in the vertical direction, a G' signal is generated from the G signals of two consecutive pixels, and a B' signal is generated from the B signals of two consecutive pixels. Since the moving image composed of the R', G', and B' signals generated in this way has a Bayer arrangement, the same image processing for a normal image can be applied to the generated moving image.

[0039] In addition, since the signal is read out from the second pixel group by 1 / 3 thinning, it is possible to obtain a signal having the following characteristics: Figure 3B Flicker detection image of the Bayer arrangement shown on the right. In the following description, this readout scan is referred to as a "flicker detection scan."

[0040] Although only the pixel arrangement corresponding to the XY addressing method is described here, the present invention is not limited thereto. As long as the image sensor can be independently controlled in units of rows, the second drive control can be performed in a global shutter method generally applicable to CCD sensors or a rolling shutter method generally applicable to CMOS sensors.

[0041] Figure 4 1 is a diagram showing an example of the timing of performing motion image scanning and flicker detection scanning, wherein the horizontal axis shows time and the vertical axis shows the vertical position (row) of the image sensor 101. In addition, the oblique solid line indicates the timing of performing motion image scanning, and the oblique dotted line indicates the timing of performing flicker detection scanning. Figure 4 In the example shown, during one frame period of a moving image, a moving image scan is first performed once, and then a plurality of flicker detection scans are performed. The flicker detection scan does not necessarily have to be performed in each frame, and may be performed, for example, every a plurality of frames. The number of times a flicker detection scan may be performed in each frame depends on shooting conditions, the performance of the image sensor 101, etc., but a moving image scan is performed once in each frame period. In addition, the moving image scan does not necessarily have to be performed first, and may be performed, for example, after the flicker detection scan is performed.

[0042] Next, we will refer to Figure 5 The table describes the characteristics of the flicker detection method using the flicker detection image obtained by flicker detection scanning and the characteristics of the conventional flicker detection method using a motion image.

[0043] First, regarding "Do shooting conditions affect image quality?", flicker detection scanning can be performed using shooting conditions such as a different exposure period from that of motion image scanning, which helps flicker detection. Motion image scanning must be performed according to the standards of the display device and recording method, camera settings, user settings, etc. in order to obtain a motion image. In contrast, flicker detection images are neither displayed nor recorded, so flicker detection scanning can be performed under free shooting conditions.

[0044] Next, regarding "Is it possible to specify 50Hz or 60Hz?", it is described as follows. For moving images, there are two main video standards, one is the NTSC (National Television System Committee) video standard, and the other is the PAL (Phase Alternation Line) standard. The transmission rate standard of NTSC is 29.97 frames per second (≈30fps), and the transmission rate standard of PAL is 25 frames per second, which are synchronized with flickers of 120Hz and 100Hz, respectively. As a result, in the case of flicker detection using moving images, flickers of 100Hz or 120Hz cannot be detected.

[0045] On the other hand, in the flicker detection scan, the shooting conditions can be freely determined as described above. Therefore, by driving the image sensor 101 at 90fps, 180fps, etc. that is not synchronized with the flicker of 120Hz or 100Hz, the obtained flicker detection image can be used to detect both 100Hz flicker and 120Hz flicker.

[0046] Next, the flicker reduction method will be described. As described above, it is known that by setting the charge accumulation period to an integer multiple of the flicker cycle, an image in which the influence of flicker is suppressed can be captured. That is, whether a moving image or a flicker detection image is used to detect flicker, the influence of flicker can be reduced by setting the charge accumulation period of the moving image to an integer multiple of the detected flicker cycle according to the detected flicker frequency.

[0047] However, if the frame rate is high, it may not be possible to set the charge accumulation period to an integer multiple of the flicker period. For example, when a flicker of 100 Hz is detected when shooting a moving image at 120 fps, 1 / 100 second is longer than the period of 120 fps, so the charge accumulation period of each frame cannot be controlled to an integer multiple of the flicker period that can reduce the influence of the flicker. This also applies to high-speed photography such as 200 fps and 240 fps. In this case, instead of controlling the exposure period, the influence of the flicker can be corrected by applying image correction (gain) to the moving image to eliminate the influence of the flicker. The image correction method applied to images captured using a rolling shutter is different from the image correction method applied to images captured using a global shutter.

[0048] Figure 6601 is a conceptual diagram showing a flicker reduction method by performing image processing on an image captured using a rolling shutter. With a rolling shutter, since the charge accumulation timing differs from row to row, the amount of light received differs from row to row due to the flickering of the light source. As a result, the brightness fluctuates in the vertical direction of the image, as shown in 601. By analyzing this brightness variation (602) and performing image processing that multiplies each row by a digital gain (603), which is the inverse of the brightness variation in the vertical direction, flicker (604) can be corrected.

[0049] Figure 7 A conceptual diagram of a method for correcting a moving image captured by using a global shutter is shown. In the global shutter method, charge accumulation starts and ends at the same timing in all rows of each frame image, so that there is no brightness change due to flicker in each frame image. However, when analyzing the brightness of multiple images captured continuously in the time direction, since the amount of light received differs between frame images according to the flicker of the light source (701), the brightness between frame images differs (702). Therefore, the brightness change between frame images is analyzed (703), and a correction gain (704) that eliminates the influence of flicker is applied to each frame image to make the brightness between frame images uniform (705). By doing so, flicker can be corrected when viewing frame images continuously.

[0050] Here, in the case of detecting flicker by using a flicker detection image, the influence of flicker cannot be reduced by using gain. This is because the flicker detection image is shot under shooting conditions different from those of the motion image, and the flicker of the light source has a different influence on the flicker detection image than on the motion image. In the case of using a rolling shutter, the number and density of stripes are different between the flicker detection image and the motion image, and in the case of using a global shutter, the brightness change period and the amplitude appearing in the image are different between the flicker detection image and the motion image. Therefore, even if it is possible to detect whether a flicker of 100Hz or 120Hz occurs from the flicker detection image, the gain (image processing coefficient) applied to the motion image cannot be calculated.

[0051] Finally, the power consumption is as follows. That is, when comparing the power consumption with and without flicker detection scanning, the power consumption of the image sensor 101 increases when flicker detection scanning is performed compared to the case where flicker detection scanning is not performed. In addition, when only the first drive control or motion image scanning is performed, the power consumption does not change compared to the conventional case. Therefore, when flicker detection scanning is performed, the battery consumption rate increases and the number of images that can be captured decreases compared to the case where flicker detection scanning is not performed.

[0052] In consideration of the features of flicker detection using a moving image and flicker detection using a flicker detection image as described above, in the first embodiment, flicker reduction is performed based on the ability to reduce flicker. Figure 8 Controls shown. Figure 8 is a flowchart showing flicker detection and flicker handling in a still image live view mode or a moving image mode. When the still image live view mode or the moving image mode is indicated, the Figure 8 Controls shown.

[0053] First, in step S100, it is determined whether the second drive control can be performed. Figures 2 to 4 As in the second drive control described above, the second pixel group that is not used to generate the motion image is used to generate the flicker detection image. However, since the second pixel group cannot be ensured, for example, in a mode in which all pixels are read out (such as in a 4K motion image mode requiring high resolution), the second drive control cannot be performed. In this case, the first drive control is started in step S107, the obtained motion image is used to detect flicker in step S108, and it is determined in step S109 whether flicker occurs. Here, the absence of flicker includes a case where flicker does not actually exist and a case where flicker cannot be detected even if flicker occurs as described above.

[0054] If flicker is detected, the process proceeds to step S110 to reduce the influence of flicker in the moving image by changing the charge accumulation period or by applying gain. On the other hand, if flicker is not detected, the process proceeds to step S112 to determine whether to end shooting. If shooting is not ended, the process returns to step S108, the above process is repeated for the next input image, and if shooting is ended, the process ends.

[0055] On the other hand, if it is determined in step S100 that the second drive control can be performed, the second drive control is started in step S101. Then, flicker detection is performed using the flicker detection image acquired by the second drive control, and it is determined in step S102 whether flicker occurs. If flicker is not detected, the process proceeds to step S111.

[0056] If flicker is detected in step S102, the process proceeds to step S103, and it is determined whether the influence of flicker can be reduced by controlling the charge accumulation period. Based on the frame rate of the motion image scan and the frequency of the detected flicker, it is determined whether the influence of flicker can be reduced by controlling the charge accumulation period. For example, if the frame rate of the motion image scan is 120fps and a flicker of 120Hz is detected, the influence of flicker can be reduced by setting the charge accumulation period to 1 / 120 seconds. On the other hand, if a 100Hz flicker is detected when the frame rate of the motion image scan is 120fps, since the charge accumulation period cannot be set to 1 / 100 seconds, it is determined that the influence of flicker cannot be reduced. That is, the period of flicker and the frame rate of the motion image scan are compared, and if the period of flicker is equal to or less than the frame rate, it is determined that the influence of flicker can be reduced by controlling the charge accumulation period. If it is "yes" in step S103, in step S104, the charge accumulation period of the motion image scan is set to a charge accumulation period (an integer multiple of the flicker period) that can reduce the influence of flicker, and the process proceeds to step S111.

[0057] On the other hand, if it is determined in step S103 that the influence of the flicker cannot be reduced by controlling the charge accumulation period, flicker detection is performed by analyzing the brightness of the moving image in step S105, and a gain value applied to the moving image in image processing is calculated. Then, in step S106, the influence of the flicker on the moving image is corrected by image processing using the calculated gain value, and the process proceeds to step S111.

[0058] In step S111, it is determined whether the shooting is finished. If the shooting is not finished, the process returns to step S102, and flicker detection is performed using the next input flicker detection image. If the shooting is finished, the process ends.

[0059] As described above, according to the first embodiment, when the image sensor has a structure capable of acquiring a flicker detection image from a part of the pixel area of ​​the image sensor, the flicker detection method is changed according to the flicker reduction method. This makes it possible to obtain a moving image less affected by flicker under various shooting conditions.

[0060] <Second Embodiment>

[0061] Next, a second embodiment of the present invention will be described. Since the configuration of the device in the second embodiment is the same as that in the first embodiment, the description thereof will be omitted.

[0062] In the second embodiment, reference will be made to Fig. 9 The flowchart description is from reference Figure 5 The power consumption angle switching drive control method is described.

[0063] First, in step S200, it is determined whether the second drive control can be performed. For example, if a setting related to the readout operation of a moving image is set (such as a 4K moving image mode), the second drive control including the flicker detection scan cannot be performed. In addition, since the second drive control accompanied by the flicker detection scan consumes a large amount of power, in the second embodiment, the second drive control is not performed when a camera setting that prioritizes power saving (such as a power saving priority setting) is set.

[0064] If the second drive control cannot be performed, the process proceeds to step S211 to start the first drive control. As described above, in the first drive control, all pixels of the image sensor 101 are driven under the same shooting conditions, but signals do not need to be read out from all pixels, and thinning reading and addition reading can be performed according to the required resolution. In the next step S212, flicker detection is performed using the moving image obtained by the first drive control, and the process proceeds to step S213.

[0065] On the other hand, in the case where the second drive control can be performed, the power state of the camera is checked in steps S202 and S203. If power is supplied from the outside via a power cord, a USB cord, or the like ("Yes" in step S202), the power consumption problem due to the second drive control does not occur, so the process proceeds to step S205 to start the second drive control. Here, for example, flicker detection scanning is performed at intervals of 0.2 seconds. Then, the process proceeds to step S208.

[0066] On the other hand, if power is not supplied from the outside and the remaining battery level is equal to or less than a predetermined threshold value Tb1 (e.g., 10%) ("Yes" in step S203), it is considered that the remaining power is not much, and the process proceeds to step S211 to start the first drive control. Then, in step S212, flicker detection is performed using the motion image obtained by the first drive control, and the process proceeds to step S213.

[0067] In step S213, it is determined whether the shooting is finished. If the shooting is not finished, the process returns to step S212 to detect flicker for the next input image, and if the shooting is finished, the process ends.

[0068] Furthermore, in step S203, even if power is not supplied from the outside, if the remaining battery level is higher than a predetermined threshold value Tb1 (e.g., 10%) ("No" in step S203), it does not mean that the remaining power is low, and the processing proceeds to step S204 to further judge the remaining battery level. For example, if the remaining battery level is greater than a predetermined threshold value Tb2 (e.g., 50%) and the power state is sufficient (within a range between 50% and 100%), the processing proceeds to step S206, and the second drive control is started. Here, for example, flicker detection scanning is performed at intervals of 1 second, and the processing proceeds to step S208.

[0069] In step S208 , flicker is detected using only the flicker detection image, and the process proceeds to step S210 .

[0070] On the other hand, if the remaining battery level is equal to or less than a predetermined threshold value Tb2 (a range of 10% to 50%) ("Yes" in step S204), the power state is not low, but it is desired to suppress power consumption, so the processing proceeds to step S207, and the second drive control is started. Here, for example, a flicker detection scan is performed at intervals of 10 seconds. Then, the processing proceeds to step S209. In step S209, flicker detection is performed using a flicker detection image at a timing when a flicker detection image exists, and using a motion image at a timing when a flicker detection image does not exist, and the processing proceeds to step S210.

[0071] In step S210, it is determined whether the shooting is finished. If the shooting is not finished, the process returns to step S202 and the above process is repeated. If the shooting is finished, the process ends.

[0072] Fig.10 is the above Fig. 9 An overview of the processing. When power is being supplied from the outside, there is no need to worry about the remaining battery, so the second drive control is performed and a flicker detection scan is performed at a high-frequency first interval (for example, an interval of 0.2 seconds). Then, flicker detection is performed using only the flicker detection image. On the other hand, even if power is not supplied from the outside, when the remaining battery is sufficient (for example, 50% to 100%), the second drive control is performed and a flicker detection scan is performed at a second interval (for example, an interval of 1 second) at a frequency lower than the frequency of the state in which power is supplied from the outside. Then, flicker detection is performed using only the flicker detection image.

[0073] In addition, if power is not supplied from the outside and the remaining battery power is low (10% to 50%), power consumption is suppressed preferentially, and in the second drive control, the interval of the flicker detection scan is further extended to a third interval (e.g., an interval of 10 seconds). Then, when a flicker detection image is generated, flicker detection is performed using the flicker detection image, and when a flicker detection image is not generated, flicker detection is performed using a moving image.

[0074] As mentioned above Figure 5 As described, only 50 Hz or 60 Hz flicker can be detected using a moving image. Therefore, in a case where a flicker detection image cannot be acquired, there is always flicker with an undetectable frequency. In view of the above, although the intervals are long, flicker detection scans are periodically performed, and between flicker detection scans, flicker is continuously detected using a moving image, so that flicker detection with a good balance in which power consumption is suppressed and detection accuracy is not significantly reduced can be achieved.

[0075] In addition, if there is no external power supply and the remaining battery power is extremely low (10% or less), or when the second drive control cannot be performed, the highest priority is given to reducing power consumption, the second drive control is not performed, and flicker detection is performed using moving images obtained by normal shooting.

[0076] In the second embodiment, an example of periodically performing flicker detection scanning at regular time intervals is shown, but it is not always necessary to perform flicker detection scanning periodically. For example, control may be performed so that changes in a subject are monitored in a moving image, and if a change in the subject occurs, flicker detection scanning is restarted, or changes in flicker levels are monitored based on flicker detection results using a moving image, and if a change in flicker level is detected, flicker detection scanning is restarted. By continuously or conditionally performing flicker detection scanning according to conditions while flicker detection is performed using a moving image, substantially good flicker detection performance can be achieved while suppressing power consumption.

[0077] In addition, in this embodiment, as an example of the threshold value for switching flicker detection control, the remaining battery level of 10%, 50%, etc. is given, but other percentages can be used. In addition, the percentage can be changed according to the camera settings such as the moving image mode and the still image mode.

[0078] As described above, according to the second embodiment, the flicker detection method is changed according to the power supply state. This makes it possible to perform good flicker detection according to the power supply state.

[0079] It should be noted that, according to the detected flickering cycle, the Figure 8 The influence of flicker is reduced as described in step S103 and subsequent steps.

[0080] <Modifications>

[0081] In the first and second embodiments, the process of performing flicker detection scanning during motion image shooting has been described, however, flicker detection may be performed when the camera is blacked out. For example, flicker detection may be performed during a black screen period just after the power is turned on, or during a black screen period during still image shooting in response to pressing a release button. Since in flicker detection during a black screen period, a captured image is not displayed on the display unit 102, flicker detection may be performed using a motion image or a flicker detection image, or flicker detection using other methods may be performed.

[0082] In this modification, including this case, reference will be made to Fig.11A and Fig. 11B The flowchart shown describes the flow of the entire flicker detection control process from power-on to camera state transition.

[0083] First, in step S300, flicker detection is performed during the black screen period when the power is turned on. Subsequently, in step S301, it is determined whether the mode when the power is turned on is a still image mode or a moving image mode. In the case of the still image mode, the frame rate of the live view display is determined in step S302, and if the frame rate is above 120fps, a second drive control is performed in step S305 to perform the above-mentioned flicker detection scanning at a third interval, and flicker detection is performed using both the moving image and the flicker detection image, and it is determined whether flicker occurs.

[0084] If the frame rate of the live view display is less than 120fps in step S302, the power state is determined in step S303. If power is supplied from the outside or if the remaining battery is sufficient, the process proceeds to step S304, in which a second drive control is performed to perform the above-mentioned flicker detection scan at the first interval or the second interval, and flicker detection is performed using only the flicker detection image. In addition, if power is not supplied from the outside but the remaining battery is sufficient, a second drive control is performed to perform the above-mentioned flicker detection scan at a third interval in step S305, and flicker detection is performed using both the flicker detection image and the motion image. If power is not supplied from the outside and the remaining battery is not much, the first drive control is performed in step S306, and flicker detection is performed using only the motion image.

[0085] In the case where a still image shooting instruction is received during live view display in step S307, flicker detection is performed at the timing of switching from live view display to still image shooting in step S308, and then still image shooting is performed in step S309. After the still image shooting is completed, the process returns to step S301 again to determine the still image mode or the moving image mode.

[0086] Next, the case where it is determined in step S301 that it is a moving image mode will be described. In the moving image mode, in a drive mode such as a 4K moving image in which thinning readout cannot be performed, flicker detection scanning cannot be performed. Therefore, it is determined in step S310 whether flicker detection scanning can be performed. If flicker detection scanning cannot be performed, the first drive control is performed in step S315, and flicker detection is performed using only moving images.

[0087] If it is determined in step S310 that flicker detection scanning can be performed, then in step S311, it is determined whether the frame rate of the live view display is less than 120fps. If it is determined that the frame rate is greater than 120fps, then in step S314, the second drive control of the flicker detection scanning is started at a third interval, and flicker detection is performed using both the flicker detection image and the moving image.

[0088] If the frame rate is less than 120fps, the power status is determined in step S312. As in the case of live view display of still images, if power is supplied from an external source or sufficient battery is remaining, the process proceeds to step S313, and a second drive control is performed to perform the above-mentioned flicker detection scan at a first interval or a second interval, and flicker detection is performed using only the flicker detection image. In addition, if power is not supplied from an external source but a certain amount of battery is remaining, a second drive control to perform the above-mentioned flicker detection scan at a third interval is started in step S314, and flicker detection is performed using both the flicker detection image and the moving image. If power is not supplied from an external source and the remaining battery is not much, the first drive control is performed in step S315, and flicker detection is performed using only the moving image.

[0089] While the flicker detection processing is being performed, the process returns to step S301 again to check the still image mode or the moving image mode, and the flicker detection processing is continued.

[0090] <Other embodiments>

[0091] The present invention can be applied to a system composed of a plurality of devices or an apparatus composed of one device.

[0092] The embodiments of the present invention may also be implemented by providing software (program) for performing the functions of the above-described embodiments to a system or device via a network or various storage media, and a computer or a central processing unit (CPU) or a microprocessing unit (MPU) of the system or device reads and executes the program.

[0093] While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments.The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

Claims

1. An electronic device, comprising: an image sensor having a plurality of pixels divided into a plurality of pixel groups including a first pixel group and a second pixel group, and capable of driving each pixel group with a shooting condition independently set for each pixel group; a controller that switches between a first drive control in which the first pixel group and the second pixel group are driven using the same shooting condition and a second drive control in which the first pixel group and the second pixel group are driven using shooting conditions different from each other; a detector that detects flicker based on a signal read out from the image sensor; as well as a determination unit that determines whether the detector is capable of detecting flicker based on a signal read out from the image sensor by the second drive control, wherein, if the determination unit determines that flicker can be detected based on the signal read out from the image sensor by the second drive control, the controller controls to drive the second pixel group using the shooting conditions for flicker detection in the second drive control, and the detector detects flicker based on the detection signal read out from the second pixel group, wherein if the determination unit determines that flicker cannot be detected based on the signal read out from the image sensor by the second drive control, the controller controls to perform the first drive control, and When the first drive control is performed, the detector detects flicker based on an image read out from the image sensor.

2. The electronic device according to claim 1, wherein: The determination unit determines that flicker cannot be detected based on a signal read out from the image sensor by the second drive control when a resolution of the image read out from the image sensor is higher than a predetermined threshold value.

3. The electronic device according to claim 1, wherein: When flicker is detected and the period of the flicker is equal to or less than the frame rate of the image read out from the image sensor, the controller sets the shooting conditions of the first pixel group so that the charge accumulation period of the image sensor is an integer multiple of the period of the flicker.

4. The electronic device according to claim 1, wherein: The electronic device further includes a corrector that corrects the effect of flicker in the image read out from the image sensor, wherein, when flicker is detected and the period of the flicker is longer than the frame rate of the image read out from the image sensor, the detector detects a brightness change within an image or a brightness change between images based on the image read out from the image sensor by the first drive control or the signal read out from the first pixel group by the second drive control, and Therein, the corrector applies a gain set to correct brightness variations to the image.

5. The electronic device according to claim 1, wherein: The controller controls the image sensor through the first drive control or the second drive control during a period from when the electronic device is turned on to when it is ready for image capturing.

6. The electronic device according to claim 1, wherein: In a case where still image shooting is instructed, the controller controls the image sensor through the first drive control or the second drive control before starting the still image shooting.

7. An electronic device, comprising: an image sensor having a plurality of pixels divided into a plurality of pixel groups including a first pixel group and a second pixel group, and capable of driving each pixel group with a shooting condition independently set for each pixel group; a controller that switches between a first drive control in which the first pixel group and the second pixel group are driven using the same shooting condition and a second drive control in which the first pixel group and the second pixel group are driven using shooting conditions different from each other; a detector that detects flicker based on a signal read out from the image sensor; as well as a determination unit that determines whether the detector is capable of detecting flicker based on a signal read out from the image sensor by the second drive control, wherein, if the determination unit determines that flicker can be detected based on the signal read out from the image sensor by the second drive control, the controller controls to drive the second pixel group using the shooting conditions for flicker detection in the second drive control, and the detector detects flicker based on the detection signal read out from the second pixel group, The controller further switches between the first drive control and the second drive control according to the power status.

8. The electronic device according to claim 7, wherein: The determination unit determines that flicker cannot be detected based on a signal read out from the image sensor by the second drive control when a resolution of the image read out from the image sensor is higher than a predetermined threshold value.

9. The electronic device according to claim 7, wherein: The determination unit determines that flicker cannot be detected based on a signal read out from the image sensor by the second drive control when a power saving priority setting that prioritizes reduction of power consumption is set.

10. The electronic device according to claim 7, wherein: When the second drive control is to be performed, if the power state is in a first state, the controller drives the second pixel group at a higher frequency than when the power state is in a second state lower than the first state.

11. The electronic device according to claim 10, wherein: The controller controls to: In a first power state where power is supplied from the outside, driving the second pixel group at a first frequency in the second drive control; In the case of a second power state where power is not supplied from the outside and the power state is greater than a predetermined first threshold, in the second drive control, driving the second pixel group at a second frequency lower than the first frequency; In a case where power is not supplied from the outside and the power state is equal to or less than the first threshold and greater than a second threshold smaller than the first threshold, driving the second pixel group at a third frequency lower than the second frequency in the second drive control; as well as The first drive control is performed in a fourth power state in which power is not supplied from the outside and the power state is equal to or less than the second threshold.

12. The electronic device according to claim 11, wherein: The detector: detecting flicker based on the detection signal in the case of the first power state and the second power state; in the case of the third power state, detecting flicker based on the detection signal and a signal read out from the first pixel group; and In the case of the fourth power state, flicker is detected based on a signal obtained by the first drive control.

13. A control method of an electronic device, the electronic device being equipped with an image sensor, the image sensor having a plurality of pixels divided into a plurality of pixel groups including a first pixel group and a second pixel group, and capable of driving each pixel group using a shooting condition independently set for each pixel group, the control method comprising: performing control to switch between a first drive control in which the first pixel group and the second pixel group are driven using the same shooting condition and a second drive control in which the first pixel group and the second pixel group are driven using shooting conditions different from each other; detecting flicker based on a signal read out from the image sensor; as well as determining whether flicker can be detected based on a signal read out from the image sensor by the second drive control, wherein, if it is determined that flicker can be detected based on the signal read out from the image sensor by the second drive control, control is performed so that the second pixel group is driven using a shooting condition for flicker detection in the second drive control, and flicker is detected based on the detection signal read out from the second pixel group, wherein, if it is determined that flicker cannot be detected based on the signal read out from the image sensor by the second drive control, control is performed to perform the first drive control, and When the first drive control is performed, flicker is detected based on the image read out from the image sensor.

14. A control method of an electronic device, the electronic device being equipped with an image sensor, the image sensor having a plurality of pixels divided into a plurality of pixel groups including a first pixel group and a second pixel group, and capable of driving each pixel group using a shooting condition independently set for each pixel group, the control method comprising: performing control to switch between a first drive control in which the first pixel group and the second pixel group are driven using the same shooting condition and a second drive control in which the first pixel group and the second pixel group are driven using shooting conditions different from each other; detecting flicker based on a signal read out from the image sensor; as well as determining whether flicker can be detected based on a signal read out from the image sensor by the second drive control, wherein, if it is determined that flicker can be detected based on the signal read out from the image sensor by the second drive control, control is performed so that the second pixel group is driven using a shooting condition for flicker detection in the second drive control, and flicker is detected based on the detection signal read out from the second pixel group, The first drive control and the second drive control are switched according to the power status. 15 . A non-transitory computer-readable storage medium storing a program for causing a computer to execute the control method according to claim 13 or 14 .

Images