Flickering light source detection method and electronic device

Abstract

The embodiment of the present application provides a flickering light source detection method and an electronic device, and the electronic device implementing the method has two or more rolling shutter cameras, including a first camera for image preview display and a second camera for detecting the flickering light source. In the method, after starting the first camera to provide a preview image, the electronic device can start the second camera again, and set its exposure time to be shorter than the light energy cycle, such as 9 milliseconds, and perform image recognition on the image captured by the second camera to achieve the purpose of detecting the flickering light source. The image captured by the second camera is not sent for display, but is only used to detect the flickering light source. In this way, the flickering light source can be detected without the need to introduce additional dedicated detection devices, and the user can also be prevented from seeing the generation of water ripples.

Description

Technical Field

[0001] The present application relates to the field of electronic technology, and in particular to a flashing light source detection method and an electronic device. Background Art

[0002] With the development of electronic technology, mobile phones, tablet computers and other electronic devices are generally equipped with cameras. Most cameras use rolling shutters. When taking pictures under AC light sources (such as fluorescent lights, TVs, computer screens or household light sources), this type of camera will cause water ripples in the image, seriously affecting the image quality. Summary of the invention

[0003] In a first aspect, the present application provides a method for detecting a flickering light source, which may include: an electronic device starts a first camera and displays an image captured by the first camera on a display screen; the electronic device also starts a second camera and sets the exposure time of the second camera to a second duration, where the second duration is shorter than a light energy cycle. The electronic device may perform image recognition on the image captured by the second camera to determine whether the image content contains water ripple features. If so, it is considered that there is a flickering light source in the shooting environment, and the exposure time of the first camera is controlled to be an integer multiple of the light energy cycle. The image captured by the second camera will not be displayed, but is only used for flickering light source detection.

[0004] Through the method of the first aspect, the electronic device can control the exposure of the first camera according to the flickering light source detection result, and control the exposure time of the first camera to be an integer multiple of the light energy cycle to avoid water ripples in the preview image provided by the first camera.

[0005] For example, when the shooting mode is switched from the normal shooting mode to the motion capture mode, when reducing the exposure time to alleviate the motion ghosting problem, the electronic device can specifically reduce the exposure time of the first camera to a small integer multiple such as one or two times the light energy period, that is, the reduced exposure time is controlled to be an integer multiple of the light energy period, so as to avoid the problem of water ripples in the imaging picture under the flickering light source.

[0006] In combination with the first aspect, in some embodiments, if the image captured by the second camera includes water ripple features, when reducing the exposure time of the first camera, the electronic device can specifically control the reduced exposure time of the first camera to one light energy period.

[0007] In combination with the first aspect, in some embodiments, when starting the first camera, the electronic device can set the exposure time of the first camera to a first duration, and the first duration is greater than the light energy period and is an integer multiple of the light energy period, for example 50ms. In this way, the user can avoid seeing the generation of water ripples at the beginning of the camera startup.

[0008] In combination with the first aspect, in some embodiments, before the electronic device starts the first camera, it also includes: detecting a user operation of the user opening a camera application. That is, the start of the first camera may be triggered by the event that the user opens the camera application. The first camera may be a main camera, and the second camera is not a main camera.

[0009] In combination with the first aspect, in some embodiments, the first camera may be multiple cameras. In this case, the preview image displayed by the camera application on the display screen is composed of images simultaneously captured by the multiple cameras, so as to provide the user with the function of taking or recording photos with multiple cameras.

[0010] In combination with the first aspect, in some embodiments, the second duration can be further set to meet the exposure time of detecting the flickering light source under multiple power frequencies, that is, less than the minimum light energy cycle among the light energy cycles of the multiple power frequencies. Taking 50 Hz and 60 Hz as examples, the second duration can be set to 7 ms, which is shorter than the light energy cycle of a 60 Hz light source (about 8.3 milliseconds).

[0011] In combination with the first aspect, in some embodiments, before the electronic device starts the second camera, it also includes: the electronic device determines that the ambient brightness is neither too bright nor too dark, wherein the brightness of the overbright environment is higher than a first high brightness, such as 10,000 lux, and the brightness of the overdark environment is lower than a first low brightness, such as 50 Lux.

[0012] In combination with the first aspect, in some embodiments, the exposure duration of the first camera may be set by the automatic exposure control AEC; the electronic device determines that the ambient brightness is neither too bright nor too dark, which may specifically include: determining that the ambient brightness is not too bright based on the exposure duration of the first camera being greater than the third duration (e.g., 20ms), determining that the ambient brightness is not too dark based on the exposure duration of the first camera being less than the fourth duration (200ms), and the third duration being less than the fourth duration. The third duration and the fourth duration are recorded in the exposure table of the first camera, for example, the third duration belongs to several shortest exposure times corresponding to high EV in the exposure table, and the fourth duration belongs to several longest exposure times corresponding to low EV in the exposure table.

[0013] In conjunction with the first aspect, in some embodiments, which camera is the first camera or the second camera is not fixed. The following uses the first camera as the main camera as an example of various switching situations of the first camera.

[0014] The electronic device may also include a telephoto camera. At this time, after the electronic device displays the preview image on the display screen, the method may further include: the electronic device detects a user operation of increasing the zoom ratio to the zoom range of the telephoto camera, and switches the first camera from the main camera to the telephoto camera.

[0015] The electronic device may also include a wide-angle camera. At this time, after the electronic device displays the preview image on the display screen, the method may further include: the electronic device detects a user operation of reducing the zoom ratio to the zoom range of the wide-angle camera, and switches the first camera from the main camera to the wide-angle camera.

[0016] The main camera may specifically include a rear main camera and a front main camera. At this time, after the electronic device displays the preview image on the display screen, the method may further include: the electronic device detects a user operation of flipping the camera, and switches the first camera from the rear main camera to the front main camera, or switches the first camera from the front main camera to the rear main camera.

[0017] After the electronic device displays the preview image on the display screen, the method further includes: the electronic device detects a user operation of switching the shooting mode to multi-channel photo taking or video recording, starts one or more cameras other than the main camera, and also adds them as the first camera.

[0018] In combination with the first aspect, in some embodiments, when the first camera is switched, the second camera can be selected from the remaining cameras. The second camera can be selected to switch to the original first camera, especially for the case where the electronic device has only two cameras, there is no need to start a new camera, and the response speed is faster. Of course, in practical applications, it is also possible to consider switching the second camera to a camera with lower power consumption based on issues such as power consumption, and the embodiments of the present application are not limited to this. The second camera can also be fixed, such as a depth of field camera. The image data collected by this type of camera does not participate in the preview image display, but is only used as auxiliary data (such as depth of field data) for other cameras, such as depth data of the face, and therefore will not be occupied by the first camera due to changes in the shooting mode.

[0019] In a second aspect, the present application provides an electronic device, comprising one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, and the computer program code includes computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the method described in the first aspect and any possible implementation method of the first aspect.

[0020] In a third aspect, an embodiment of the present application provides a chip system, which is applied to an electronic device, and the chip system includes one or more processors, which are used to call computer instructions so that the electronic device executes the method described in the first aspect and any possible implementation method of the first aspect.

[0021] In a fourth aspect, the present application provides a computer-readable storage medium comprising instructions, which, when executed on an electronic device, causes the electronic device to execute the method described in the first aspect and any possible implementation of the first aspect.

[0022] In a fifth aspect, the present application provides a computer program product comprising instructions, which, when executed on an electronic device, enables the electronic device to execute the method described in the first aspect and any possible implementation of the first aspect.

[0023] It can be understood that the electronic device provided in the second aspect, the chip system provided in the third aspect, the computer storage medium provided in the fourth aspect, and the computer program product provided in the fifth aspect are all used to execute the method provided in the present application. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 The working principle of the rolling shutter is shown by way of example;

[0025] Figure 2 This example shows that the preview image provided by the rolling shutter camera has a water ripple problem;

[0026] Figure 3 An example is shown in which the exposure time of a rolling shutter camera is set to be twice as long as the light energy period to solve the water ripple problem;

[0027] Figure 4 The overall process of the flashing light source method provided by the embodiment of the present application is shown;

[0028] Figure 5 The hardware structure of the electronic device of the embodiment of the present application is exemplarily shown;

[0029] Figure 6 The software architecture of the electronic device of the embodiment of the present application is exemplarily shown;

[0030] Figure 7 The detailed implementation process of the flashing light source method provided in the embodiment of the present application based on the internal architecture of the electronic device is shown. DETAILED DESCRIPTION

[0031] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application.

[0032] The shutter is an important part of the camera, which is used to control the exposure time of the photosensitive film. The shutter can be divided into a global shutter and a rolling shutter, and the exposure methods of the two are different. Among them, the exposure method of the global shutter is: all pixels on the photosensitive film start to be exposed to light at the same time, and after experiencing the same exposure time, they end at the same time; the exposure method of the rolling shutter is: each row of pixels on the photosensitive film is exposed row by row in a rolling manner, and the exposure time of each row is the same until all pixels are exposed. The exposure start time and end time of different rows are different.

[0033] Taking pictures with a rolling shutter camera under certain light sources may cause "banding" to appear in the image. These light sources are often driven by alternating current. The energy transmitted by alternating current is not constant, but changes with a fixed frequency (such as 50Hz or 60Hz), which causes periodic fluctuations in the light intensity of the light source. The period followed is the light energy cycle corresponding to the fixed frequency (also called the industrial frequency). For example, the light intensity fluctuation frequency of a 50Hz AC light source is 100Hz, and the light energy cycle is 10 milliseconds, that is, the light source flickers 100 times in 1 second. In this article, this light source is also called a flickering light source.

[0034] For rolling shutter cameras, when the exposure time is less than the light energy cycle of the light source, since the exposure start and end time of each row of pixels on the photosensitive film are different, the accumulated light energy received by each row of pixels within the same exposure time is different, and light and dark stripes will appear on the image. Figure 1 As shown in FIG. 1 , when imaging the first frame of the image, the light energy received by the first row of pixels on the photosensitive film is the light energy accumulated from time 0 to time t2, while the light energy received by the nth row of pixels on the photosensitive film is the light energy accumulated from time t1 to 5 milliseconds. The light energy received by the nth row is less than the light energy received by the first row. For another example, Figure 1 As shown in the figure, when imaging the N+1th frame, the light energy received by the pixels in the first row on the photosensitive film is the light energy accumulated from time t3 to time t5, while the light energy received by the pixels in the nth row on the photosensitive film is the light energy accumulated from time t4 to time t6. The light energy received by the first row is less than the light energy received by the nth row. In a frame of imaging, the rows with large light energy appear as bright stripes, while the rows with small light energy appear as dark stripes, which can be shown as Figure 2 As shown in the figure, the light and dark stripes are water ripples, which is also called flicker. Moreover, when shooting preview or recording, users can also observe the rolling water ripples.

[0035] In shooting scenes with fast shutter speeds, such as motion capture, high dynamic range (HDR) and other shooting modes, water ripples are more likely to appear in the image. After detecting that the light source in the shooting environment is a flickering light source, one way to avoid this problem is to set the exposure time to an integer multiple of the light energy cycle. Figure 3 As shown, the light energy cycle is 10 milliseconds, and the exposure time can also be set to 10 milliseconds, that is, 1 times the light energy cycle.

[0036] Detection of flickering light sources is the key to solving the water ripple problem. One detection method is to use an image recognition algorithm to identify the water ripple features in the image content, confirm that there are water ripples in the image, and then adjust the exposure time to avoid water ripples. However, before the water ripples are eliminated, users will still see the water ripples, which is a poor user experience. Another detection method is to use a dedicated flicker detector such as a high-sampling rate ambient light sensor, a multi-spectral flicker detector, etc. to detect whether there is a flickering light source in the shooting environment, and cooperate with the camera to solve the water ripple problem. However, such devices require additional costs and design space.

[0037] The embodiment of the present application provides a flickering light source detection method, which can detect the flickering light source without introducing additional dedicated detection devices.

[0038] The electronic device implementing the method has two or more rolling shutter cameras. When one or some cameras are used for image preview display, a camera used to detect flickering light sources is selected from the remaining cameras. The camera used for preview display is called the first camera, and the camera used for flickering light source detection (also known as flicker detection) is called the second camera. The second camera is not specially introduced for flickering light source detection, and it is usually used as a depth of field camera, a wide-angle camera, or a macro camera. The shutters of the first camera and the second camera are both rolling shutters.

[0039] Figure 4 The overall process of the flickering light source detection method provided by the embodiment of the present application is shown below.

[0040] S401, the electronic device starts a first camera.

[0041] The activation of the first camera may be triggered by the event that the user opens the camera application. At this time, the shooting mode of the camera application may be a normal shooting mode, the zoom ratio is 1x, the first camera may be a rear main camera, and the second camera is selected from the remaining cameras. The rear main camera may be the one with the highest pixel on the electronic device, and its focal length is usually between the focal length of the wide-angle camera and the focal length of the telephoto camera.

[0042] Specifically, when starting the first camera, the exposure time of the first camera can be set to a first duration, and the first duration is greater than the light energy cycle and is an integer multiple of the light energy cycle, such as 50ms, so that the user can avoid seeing the generation of water ripples at the beginning of the camera startup. Unlike the motion capture mode and HDR photo mode, which need to reduce the exposure time, the exposure time of the first camera in the normal shooting mode can be relatively long, such as 50ms, 100ms and other high multiples of the light energy cycle. The first duration of this exposure time can be set by the ISP for the first camera through the AEC. The light energy cycle may not be limited to the light energy cycle under the 50Hz operating frequency, but may also be the light energy cycle under the 60Hz operating frequency, or the light energy cycle under other operating frequencies.

[0043] The scenario of starting the first camera is not limited to the scenario where the user opens the camera application. The scenario of starting the first camera can also be other scenarios that require starting the camera to capture images, such as video call scenarios, scan code payment, identity verification and other scenarios. For example, in the scan code payment scenario, when it is detected that the user has turned on the scan code payment, the electronic device can start the rear main camera to scan the payment QR code. At this time, the first camera is the rear main camera. For another example, in the identity verification scenario, the electronic device can start the front main camera and the front depth of field camera to scan and obtain facial feature data for user identity verification. At this time, the first camera can be the front main camera. For another example, in the video call scenario, the electronic device can start the front main camera to capture the portrait of the user in front of the screen.

[0044] The image captured by the first camera will be sent to the display so as to be used as the preview image displayed on the display screen by the camera application.

[0045] S402: The electronic device displays the image captured by the first camera on a display screen.

[0046] The first camera can be a single camera, in which case the preview image displayed on the display screen by the camera application only comes from this camera. The first camera can also be multiple cameras, in which case the preview image displayed on the display screen by the camera application is composed of images simultaneously captured by the multiple cameras, so as to provide the user with the function of taking photos or recording videos with multiple cameras.

[0047] As time passes, S402 may continue to be executed, and the preview stream from the first camera may continue to be displayed, so as to continuously provide a preview display to the user.

[0048] S403, the electronic device starts the second camera and sets the exposure time of the second camera to a second duration, which is shorter than the light energy cycle, for example 9ms, so that the image collected by the second camera under a 50Hz or 60Hz flickering light source will inevitably produce water ripples, and the flickering light source can be detected.

[0049] The second duration can be further set to meet the exposure time of detecting flickering light sources under various power frequencies, that is, less than the minimum light energy cycle among the light energy cycles of the various power frequencies. Taking 50Hz and 60Hz as examples, the second duration can be set to 7ms, which is shorter than the light energy cycle of a 60Hz light source (about 8.3 milliseconds).

[0050] Before executing S403, the electronic device can also determine that the ambient brightness is appropriate, such as the indoor lighting environment, based on the exposure time set by the AEC for the first camera, and then determine that flicker detection is required. This is because the exposure time of the photosensitive sensor set by the AEC can reflect the brightness of the shooting environment to a certain extent.

[0051] Flicker detection is generally not required in a shooting environment that is too bright or too dark, because in a too bright environment, the main consideration is the overexposure problem, and the exposure time needs to be significantly reduced, and it is difficult to take into account the water ripple problem; in a too dark environment, there is generally no need to reduce the exposure time to less than the light energy cycle, otherwise the problem of underexposure may easily occur. Here, too bright or too dark can be defined with reference to the exposure table on the electronic device 100. For example, the shortest exposure times corresponding to the high EV in the exposure table are determined as the exposure time of the too bright environment. Once the AEC exposure time belongs to these shortest exposure times, it is considered that the environment is too bright. For another example, the longest exposure times corresponding to the low EV in the exposure table are determined as the exposure time of the too dark environment. Once the AEC exposure time belongs to these longest exposure times, it is considered that the environment is too dark. That is, the reference exposure value for too bright or too dark can be derived from the exposure table. Then, if the AEC exposure time belongs to the remaining exposure time in the exposure table, it can reflect that the ambient brightness is neither too bright nor too dark. The electronic device is not limited to analyzing the ambient brightness by the exposure time set by the AEC for the first camera. The electronic device can also collect ambient light through an ambient light brightness sensor to determine whether the shooting environment is too bright or too dark. This application does not impose any restrictions on its technical implementation.

[0052] S404, the electronic device performs image recognition on the image captured by the second camera (not displayed), and determines whether the image content contains water ripple features. If so, the exposure time of the first camera is controlled to be an integer multiple of the light energy cycle. The image captured by the second camera will not be displayed, but only used for flicker detection, so the user will not perceive it. The flicker detection result may include: a flickering light source is detected, and a flickering light source is not detected. If the image content of the image captured by the second camera contains water ripple features, it is considered that there is a flickering light source in the shooting environment. Thereafter, the electronic device can control the exposure of the first camera according to the flicker detection result, and control the exposure time of the first camera to be an integer multiple of the light energy cycle to avoid water ripples in the preview image provided by the first camera.

[0053] For example, when the shooting mode is switched from the normal shooting mode to the motion capture mode, when reducing the exposure time to alleviate the motion ghosting problem, the electronic device can specifically reduce the exposure time of the first camera to a small integer multiple such as one or two times the light energy period, that is, the reduced exposure time is controlled to be an integer multiple of the light energy period, so as to avoid the problem of water ripples in the imaging picture under the flickering light source.

[0054] Which camera is the first camera or the second camera is not fixed.

[0055] When the user first opens the camera application, the electronic device can default to the normal photo mode with a zoom ratio of 1. At this time, the first camera can be the main camera, and the second camera can be selected from the remaining cameras.

[0056] However, when the shooting mode changes or the zoom ratio changes, the first camera can be switched from the main camera to other cameras.

[0057] For example, during photo preview, video preview, or video recording, if it is detected that the user increases the zoom ratio to the zoom range of the telephoto camera, the electronic device may switch the first camera from the main camera to the telephoto camera. For another example, during photo preview, video preview, or video recording, if it is detected that the user decreases the zoom ratio to the zoom range of the wide-angle camera, the electronic device may switch the first camera from the main camera to the wide-angle camera. For another example, if it is detected that the photo mode is switched from the normal mode to the macro shooting mode, the electronic device may switch the first camera from the main camera to the macro camera.

[0058] The second camera can be switched to the original first camera, especially for the case where the electronic device has only two cameras, there is no need to start a new camera, and the response speed is faster. Of course, in practical applications, it is also possible to consider switching the second camera to a camera with lower power consumption based on issues such as power consumption, and the embodiments of the present application are not limited to this. The second camera can also be fixed, such as a depth of field camera. The image data collected by this type of camera does not participate in the preview image display, but is only used as auxiliary data (such as depth of field data) for other cameras, such as depth data of the face, and therefore will not be occupied by the first camera due to changes in the shooting mode.

[0059] Next, the software and hardware architecture of the electronic device provided in the embodiment of the present application is introduced.

[0060] Electronic equipment can be equipped Or portable terminal devices with other operating systems, such as mobile phones, tablet computers, desktop computers, laptop computers, handheld computers, notebook computers, ultra-mobile personal computers (UMPC), netbooks, as well as cellular phones, personal digital assistants (PDA), augmented reality (AR) devices, virtual reality (VR) devices, artificial intelligence (AI) devices, wearable devices, vehicle-mounted devices, smart home devices and / or smart city devices, etc.

[0061] Figure 5 The electronic device 100 provided by the embodiment of the present application is exemplarily shown. The electronic device 100 can detect whether there is a flickering light source in the shooting environment, and does not require an additional dedicated flicker detection device, and can also prevent the user from seeing the generation of water ripples.

[0062] like Figure 5 As shown, the electronic device 100 may include: a processor 110 , a memory 120 , a camera 130 , and a display screen 140 .

[0063] in:

[0064] The processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices or integrated into one or more processors. The processor 110 may include one or more interfaces, such as an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc. These interfaces are used for data exchange between the processor 110 and peripherals.

[0065] The memory 120 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM). The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store data of users and applications. The non-volatile memory can also store executable programs and store data of users and applications, etc., and can be loaded into the random access memory in advance for direct reading and writing by the processor 110. A storage unit can also be set in the processor 110, and the storage unit can be a high-speed cache storage unit, which can be used to save instructions or data just used or cyclically used by the processor 110. The implementation code of the flickering light source detection method provided in the embodiment of the present application can be stored in the NVM. When the camera application is started, the code can be loaded into the RAM. In this way, the processor 110 can directly read the program code from the RAM to implement the flickering light source detection method provided in the embodiment of the present application. In addition, image files such as photos and videos obtained by the user using the camera application can be written into the NVM for storage for the user to browse.

[0066] The camera 130 may include: a lens, a photosensitive sensor and a flexible printed circuit board (FPCB). The FPCB is responsible for connecting other components of the camera 130 to the processor 110, for example, transmitting the raw data output by the photosensitive sensor to the processor 110. When taking a photo, the shutter of the camera 130 is opened, and light enters and shines on the photosensitive sensor. The photosensitive sensor converts the light signal into an electrical signal, and then further converts the electrical signal into a digital signal through analog-digital conversion (ADC) to pass it to the ISP for processing. The ISP can perform the following processing on the output data of the photosensitive sensor: automatic exposure control (AEC), automatic gain control (AGC), automatic white balance (AWB), color correction, removal of bad pixels, etc. The ISP can also be integrated into the camera 130.

[0067] The display screen 140 can be used to display the image captured by the camera 130. The image processed by the ISP will be sent to the display screen 140 to show the user a preview of the image captured by the camera. Here, sending to the display screen means pushing the image captured by the camera to the frame buffer (FB) for storage. The frame buffer is a storage space that can be located in the video memory or the internal memory, and is used to store the rendering data processed by the graphics card chip or to be extracted. The content of the frame buffer corresponds to the interface display on the display screen 140, and it can be simply understood as the cache corresponding to the content displayed on the display screen 140. Modifying the content in the frame buffer is to modify the content on the display screen 140.

[0068] The electronic device 100 has multiple cameras 130, for example, a rear main camera, a front main camera, a telephoto camera, a wide-angle camera, a depth of field camera, a macro camera, etc. In terms of usage, these multiple cameras 130 may at least include: a camera 130 for preview display (referred to as the first camera) and a camera 130 for flicker detection (referred to as the second camera). Among them, the image captured by the first camera will be displayed, so as to be used as a preview image displayed by the camera application on the display screen 140; the image captured by the second camera will not be displayed, but only used for flicker detection, so the user will not perceive it. The shutters of the first camera and the second camera are both rolling shutters.

[0069] When the first camera is turned on, for example, the user opens the camera application to trigger the opening, the electronic device 100 can then turn on the second camera to perform flicker detection. When performing flicker detection, the exposure time of the second camera is set to be shorter than the light energy cycle to detect the flicker light source. Because, as long as the light source is a flickering light source, the image collected by the second camera with an exposure time shorter than the light energy cycle will contain water ripple features, which can be detected by image recognition technology. Based on the flicker detection result, the electronic device 100 can effectively avoid the problem of water ripples in the imaging picture of the first camera. For example, after switching from the normal camera mode to the motion camera mode, the exposure time of the first camera needs to be reduced to reduce the problem of motion smear. At the same time, in order to take into account the problem of water ripples in the imaging picture under the flickering light source, the exposure time of the first camera can be set to be as long as the light energy cycle, such as 10ms.

[0070] The first camera may be a single camera, in which case the preview image displayed by the camera application on the display screen 140 is only from the single camera. The first camera may also be multiple cameras, in which case the preview image displayed by the camera application on the display screen 140 is composed of images simultaneously captured by the multiple cameras, so as to provide the user with the function of taking photos or recording videos with multiple cameras.

[0071] The second camera is not specially introduced for flicker detection. It can usually be a depth of field camera, a wide-angle camera, or a macro camera, etc. It has a lower resolution and lower power consumption, and is a non-primary camera. The embodiment of the present application gives it a new function. The second camera can also be one or more cameras.

[0072] In addition to the flicker detection result provided by the second camera, the exposure time of the first camera can also be controlled by the automatic exposure control (AEC) function. In short, AEC automatically adjusts the exposure time according to the light intensity. In different environments, the intensity of light varies greatly, and the exposure time of the photosensor in the camera 130 also needs to be adapted accordingly to ensure normal exposure of the image. The automatic exposure control (AEC) provided by the ISP provides this adaptive capability. Therefore, the exposure time of the photosensor can reflect the brightness of the shooting environment to a certain extent.

[0073] In addition, the electronic device 100 may also store an exposure table to be used by the AEC. The exposure table of a camera sets the exposure parameters (such as exposure time, exposure gain, aperture size) to be used by the camera at different exposure values ​​(EV), which may be debugged and recorded by engineers.

[0074] like Figure 5 As shown, the electronic device 100 may further include: an audio module 150, a speaker 150A, a receiver 150B, a microphone 150C, and an earphone interface 150D.

[0075] Among them, the electronic device can realize audio functions through the audio module 150, the speaker 150A, the receiver 150B, the microphone 150C, the headphone interface 150D, and the application processor (AP). For example, music playing, recording, etc. The audio module 150 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 150 can also be used to encode and decode audio signals. In some embodiments, the audio module 150 can be arranged in the processor 110, or some functional modules of the audio module 150 are arranged in the processor 110. The speaker 150A, also known as the "speaker", is used to convert audio electrical signals into sound signals. The electronic device can listen to music or listen to hands-free calls through the speaker 150A. The receiver 150B, also known as the "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device answers a call or voice message, the voice can be answered by placing the receiver 150B close to the human ear. Microphone 150C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can make a sound by putting his mouth close to the microphone 150C to input the sound signal into the microphone 150C. The electronic device can be provided with at least one microphone 150C. In other embodiments, the electronic device can be provided with two microphones 150C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device can also be provided with three, four or more microphones 150C to realize the collection of sound signals, noise reduction, identification of sound sources, realization of directional recording functions, etc. The headphone jack 150D is used to connect wired headphones. The headphone jack 150D can be a USB interface, or a 3.5mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0076] like Figure 5 As shown, the electronic device 100 may further include a sensor module 160, which may specifically include a pressure sensor 160A, a distance sensor 160F, a proximity light sensor 160G, a touch sensor 160K, an ambient light sensor 160L, and the like.

[0077] Figure 5 The illustrated structure does not constitute a specific limitation on the electronic device, and the electronic device may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0078] Figure 6 The main software architecture involved in the flickering light source detection method provided by the embodiment of the present application on the electronic device 100 is shown.

[0079] like Figure 6 As shown in the figure, the software architecture may include the following functions or modules: camera application (camera apk), camera service (camera service), camera hardware abstraction layer (camera HAL), flicker detection (CameraFlickerDetector), camera vendor native development kit (camera VNDK) and Kernel part. Among them,

[0080] The camera application (camera apk) can be located in the application layer. In addition to the camera application, the application layer can include a series of other applications, such as the gallery application. The camera application can be mainly responsible for human-computer interaction, such as detecting and responding to user operations such as selecting a shooting mode, adjusting the zoom ratio, and selecting a focus.

[0081] The camera service can be located in the application framework layer. The camera service can include interfaces such as camera management and camera devices, which can interact with the camera application through the application program interface (API) and with the camera HAL through the HAL interface definition language (HIDL). In addition, the camera service can also interact with the vendor native development kit (VNDK).

[0082] The camera hardware abstraction layer (camera HAL) can be the part of the virtual hardware function about the camera in the hardware abstraction layer (HAL). The hardware abstraction layer is located in the interface layer between the application framework layer and the driver layer, providing a virtual hardware platform for the operating system, and the upper layer does not need to care about the underlying hardware implementation. The camera HAL can communicate with the camera service through the HIDL interface, and can issue control operations for the camera through the standard hardware abstraction layer interface. The camera HAL can include a camera startup (CamEntry) module and an automatic exposure control processor (AECProcessor) module, wherein the CamEntry module can be used as the entrance of the HAL, receiving instructions such as open, close, stream configuration (configurestream), and request from the camera service, and is a general camera HAL function entrance. The AECProcessor module can provide ISP's AEC-related functions, such as providing the AEC parameters (such as exposure time) of the first camera to the CameraFlickerDetector, so that the CameraFlickerDetector can determine whether the current shooting environment requires flicker detection.

[0083] The flicker detection (CameraFlickerDetector) module can be located in the application framework layer (framework) or the application layer. The CameraFlickerDetector module can obtain the AEC parameters (such as exposure time) of the first camera from the camera HAL through the HIDL interface, so as to decide whether to start the second camera for flicker detection. When it is determined that flicker detection is to be performed, the CameraFlickerDetector module can issue control operations to start the second camera and set the exposure time through the interface of the camera VNDK. The CameraFlickerDetector module may include a CameraHelper module and an ImageReader module, wherein the CameraHelper module can be used to implement the call of functions such as opening (open), closing (close), stream configuration (configurestream), and request (request) of the camera service of the second camera through the call of the VNDK interface, and the ImageReader module is mainly used for receiving and further processing the images sent after the second camera is started.

[0084] The Vendor Native Development Kit (VNDK) is a collection of libraries for vendors to develop their own HALs (vendors). The camera VNDK is the "own camera HAL" developed by the vendor of the second camera. The camera VNDK can be used to receive the control operation of starting the second camera issued by CameraFlickerDetector and forward the control operation to the cameraservice. The camera VNDK may include the ACameraManager module, the ACameraDevice module, the ACameraReader module, the ACaptureSession module, and the ACaptureRequest module. Among them, the ACameraManager module provides the function of accessing the camera service from the vendor layer, the ACameraDevice module provides the function of accessing the camera device from the vendor layer, the ACameraReader module is responsible for reading image data, such as the data of the second camera replenishment, the ACaptureSession module provides the function of allowing the vendor to manage the frames captured by the camera, and the ACaptureRequest module is responsible for the settings and output targets required for the camera to capture a single image.

[0085] The kernel part may include various hardware driver control modules, such as ISP, photosensor, DSP, AP and other driver controls.

[0086] Figure 6 Only the main software architecture involved in the flickering light source detection method provided in the embodiment of the present application is shown, which is essentially only a part of the software architecture of the electronic device 100.

[0087] Figure 6 The implementation process of the flashing light source detection method is also shown, including:

[0088] 1. First camera startup process

[0089] The camera application calls the camera service, which authenticates the camera application. Authentication is used to confirm whether the camera application has the authority to access the hardware camera, such as turning the camera on and off, to prevent illegal applications from turning the camera on and off without user authorization, thereby effectively preventing user privacy from being violated. It can also prevent the camera application from conflicting with other applications that are accessing the camera (such as video call applications) in the use of camera resources. After authentication, the camera service calls the camera HAL to start the first camera through the camera HAL, and control the photosensor and ISP to output images. The image frame will be sent to the display to implement image preview, photo taking and other functions.

[0090] On this basis, in order to perform flicker detection without letting the user know, the electronic device 100 can start the second camera to perform detection. The second camera is not specially introduced for flicker detection. It can usually be a depth of field camera, a wide-angle camera, or a macro camera, etc. It has a low resolution and low power consumption. It is a non-primary camera. The embodiment of the present application gives it a new function.

[0091] 2. Flicker detection process

[0092] First, when the preview of the first camera is turned on, the AECProcessor module in the camera HAL can pass the exposure time and other parameters of the first camera to the CameraFlickerDetector module through the HIDL interface. The exposure time set by the AEC can reflect the brightness of the shooting environment. Therefore, the CameraFlickerDetector module can determine whether to perform flicker detection based on the exposure time. The environment for flicker detection is usually an indoor scene. Flicker detection is generally not required in a shooting environment that is too bright or too dark, because in a too bright environment, the main consideration is the overexposure problem and the exposure time needs to be significantly reduced, which makes it difficult to take into account the problem of avoiding water ripples; in a too dark environment, there is generally no need to reduce the exposure time to less than the light energy cycle, otherwise it is easy to cause underexposure. Here, too bright or too dark can be defined with reference to the exposure table on the electronic device 100. For example, the shortest exposure times corresponding to the high EV in the exposure table are determined as the exposure time of the too bright environment. Once the actual exposure time belongs to these shortest exposure times, it is considered that the environment is too bright. For another example, the longest exposure times corresponding to the low EV in the exposure table are determined as the exposure time of the too dark environment. Once the actual exposure time falls within these longest exposure times, it is considered that the environment is too dark.

[0093] Then, after determining that flicker detection is to be performed, the CameraFlickerDetector module transmits a control command to start the second camera to cameraVNDK.

[0094] The camera VNDK then passes the CameraFlickerDetector module to the camera service to start the control operation of the second camera.

[0095] After authenticating the CameraFlickerDetector module, the camera service calls the camera HAL again to start the second camera through the camera HAL, so that the second camera can be started from the camera HAL side without involving the application layer, so the user does not perceive it. At the same time, the CameraFlickerDetector module controls the exposure of the second camera so that the exposure time of the second camera is less than the light energy cycle. For example, if the exposure time is set to 9ms, the image collected by the second camera under a 50Hz or 60Hz flickering light source will inevitably produce water ripples, and the flickering light source can be detected. The CameraFlickerDetector module can also call the camera HAL to read the output image of the second camera, detect whether the image content has water ripple features through image recognition technology, and pass the detection results to the first camera. The output image of the second camera is not displayed and is only used for flicker detection.

[0096] After the first camera learns the flicker detection result, it can perform an appropriate exposure operation. For example, if motion capture or HDR shooting mode requires the acquisition of images with a short exposure time, in this case, the exposure time of the first camera can be reduced to twice the light energy cycle, such as 10ms, to solve the problem of water ripples in the image under the flickering light source.

[0097] The above software architecture newly introduces the CameraFlickerDetector module, and implements the interaction between the CameraFlickerDetector module and the camera HAL and camera service through VNDK and HIDL. It does not involve modifications to the camera application and various layers of the architecture, and has good portability, scalability, and maintainability.

[0098] Combine the following Figure 7 The internal structure of the electronic device 100 is further described to explain the flickering light source detection method of the embodiment of the present application. Figure 7The method is described by taking the photo or video preview provided by the camera application as an example, so that those skilled in the art can have a deeper understanding of the embodiments of the present application, and the details thereof shall not limit the protection scope of the present application.

[0099] Normal photo mode

[0100] S11. When a user operation of opening a camera application is detected, an application processor (AP) may start a rear main camera.

[0101] At this time, the rear main camera is used as the first camera. The photo mode can be a normal photo mode, and the exposure time of the rear main camera is longer than the light energy cycle, for example, 50ms, which is a multiple of the light energy cycle. The exposure time of the rear main camera can be set by the AEC of the ISP.

[0102] Prior to S11, the display screen may display a system desktop, including a desktop icon of a camera application. The user operation of opening the camera application may refer to the user clicking the desktop icon of the camera application.

[0103] S12. The rear main camera transmits the captured raw image frame (RAW) to the ISP.

[0104] The ISP can perform the following processing on the RAW image output by the rear main camera: automatic exposure control (AEC), automatic gain control (AGC), automatic white balance (AWB), color correction, removal of bad pixels, etc. The image processed by the ISP can be a color image, such as a YUV image.

[0105] S13.ISP can transmit the output YUV image frame of the rear main camera to the display screen.

[0106] Here, S13 only illustrates the flow path of image data, and does not mean that the ISP directly sends data to the display screen. Its implementation includes the process of sending and displaying. The YUV image frame of the rear main camera can be cached in the frame buffer first. When rendering the camera preview image, the GPU reads the YUV image frame of the rear main camera from the frame buffer and displays it on the display screen.

[0107] S14. The display screen displays the YUV image frame of the rear main camera.

[0108] From then on, the rear main camera began to provide preview images to realize the preview function in normal photo mode.

[0109] In normal photo mode, even if there is a flickering light source, since the exposure time of the rear main camera is longer than an integer multiple of the light energy cycle, no water ripples will appear in the preview image. However, early detection of flickering light sources is very necessary because users may switch to motion capture mode or HDR shooting mode at any time, and these shooting modes require reduced exposure time. How to reduce the exposure time of the first camera in these shooting modes without causing water ripples is a problem that the embodiments of the present application focus on solving.

[0110] S15.AP can obtain the exposure time set by AEC for the rear main camera from ISP. The exposure time set by AEC can reflect the brightness of the shooting environment.

[0111] S16.AP can determine whether flicker detection is required based on the exposure time set by AEC for the rear main camera.

[0112] The environment for flicker detection is usually an indoor scene. Flicker detection is generally not required in a shooting environment that is too bright or too dark. For specific instructions, please refer to the previous article and will not be repeated here.

[0113] Flicker detection

[0114] S17. After determining to perform flicker detection, the application processor (AP) can start the depth of field camera as a second camera, and set the exposure time of the depth of field camera to be less than the light energy cycle, such as 9 milliseconds.

[0115] Since the exposure time of the second camera is less than the light energy cycle, the image captured by the second camera under the flickering light source will inevitably produce water ripples, and the flickering light source can be detected.

[0116] After starting the first camera, the AP may also directly start the second camera without determining whether to perform flicker detection, that is, S15-S16 may not be executed.

[0117] S18. After being activated, the depth of field camera can transmit the captured raw image frame (RAW) to the ISP.

[0118] The ISP can perform a series of processing on the RAW image and output a YUV image frame. As the second camera, the YUV image frame of the depth of field camera is not sent for display, that is, it will not be transmitted to the display screen for display.

[0119] S19.AP can obtain the YUV image frame of the depth of field camera from the ISP.

[0120] S20.AP can perform image recognition on the YUV image frame of the depth of field camera to detect whether the image content contains water ripple features to obtain flicker detection results. If the image content contains water ripple features, it is considered that there is a flickering light source in the shooting environment, otherwise, it is considered that there is no flickering light source in the shooting environment.

[0121] Among them, AP can run Figure 6 The software architecture shown, especially the CameraFlickerDetector module, is used to implement flicker detection. Flicker detection can continue to be performed over time, especially during the first camera is turned on.

[0122] Thereafter, such as S21-S22, the photo taking mode may be switched to the motion capture mode or the HDR mode, and the electronic device may set the exposure time of the first camera based on the flicker detection result to avoid the water ripple problem.

[0123] S21. The AP learns that the camera mode is switched to the motion capture mode or the HDR mode.

[0124] S22.AP can reduce the exposure time of the rear main camera to meet the needs of motion capture mode or HDR mode. Specifically, it can reduce the exposure time to 1 times the light energy cycle to take into account the water ripple avoidance problem.

[0125] The exposure time is not limited to 1 times of the light energy cycle. The exposure time can also be reduced to 2 times, 3 times, or other low multiples of the light energy cycle, depending on the exposure reduction requirements of the motion capture mode or HDR mode.

[0126] The first camera is not static. The following steps illustrate the switching of the first camera in several typical scenarios (zooming, flipping the camera, multi-channel photo taking or video recording).

[0127] Zoom

[0128] In the zoom scenario, increasing the zoom ratio to the zoom range of the telephoto camera, or decreasing the zoom ratio to the zoom range of the wide-angle camera, can change the first camera that provides the preview image.

[0129] S23. The application processor (AP) detects that the zoom ratio is increased to the zoom range of the telephoto camera.

[0130] S24. The application processor (AP) can turn off the rear main camera.

[0131] S25. The application processor (AP) may activate the telephoto camera and use the telephoto camera as the first camera.

[0132] At this time, if you are in a shooting mode that requires a low exposure time, such as motion capture or HDR, when you start the telephoto camera, you can set its exposure time to 1 times the light energy cycle, refer to S26.

[0133] S27. The telephoto camera transmits the acquired raw image frame (RAW) to the ISP.

[0134] The ISP can perform a series of processing on the RAW image and output a YUV image frame.

[0135] S28.ISP can transmit the YUV image frame of the telephoto camera to the display.

[0136] Here, S27 only illustrates the flow path of image data, and does not mean that the ISP directly sends data to the display screen. Its implementation includes the process of sending and displaying.

[0137] S29. The display screen displays the YUV image frame of the telephoto camera.

[0138] From then on, the telephoto camera starts to provide a preview image, and the first camera used for preview display is switched from the rear main camera to the telephoto camera. The implementation process of reducing the zoom ratio to the zoom range of the wide-angle camera is similar to that of changing the first camera. Please refer to the adaptive adjustment of S23-S28, which will not be expanded here.

[0139] Setting the exposure time of the telephoto camera based on the flicker detection results can also avoid the appearance of water ripples in the imaging picture.

[0140] Flip camera

[0141] In a scenario where the user flips the camera, the first camera can be switched from the rear main camera to the front main camera.

[0142] S30. The application processor (AP) detects that the camera is flipped.

[0143] S31. The application processor (AP) can turn off the rear main camera.

[0144] S32. The application processor (AP) may start the front main camera and use the front main camera as the first camera.

[0145] At this time, if the front camera is in a shooting mode that requires a low exposure time, such as motion capture or HDR, by default, the exposure time can be set to 1 times the light energy cycle when the front main camera is started. Please refer to S33.

[0146] S34. The front main camera transmits the captured raw image frame (RAW) to the ISP.

[0147] The ISP can perform a series of processing on the RAW image and output a YUV image frame.

[0148] S35.ISP can transmit the YUV image frame of the front main camera to the display.

[0149] Here, S35 only illustrates the flow path of image data, and does not mean that the ISP directly sends the YUV image frame of the front main camera to the display screen. Its implementation includes the process of sending and displaying.

[0150] S36. The display screen displays the YUV image frame of the front main camera.

[0151] From then on, the front main camera starts to provide photo preview images, and setting its exposure time based on the flicker detection results can also avoid water ripples in its image. In the default front-facing shooting scene such as video call scene, flipping the camera can also trigger the first camera to switch from the front main camera to the rear main camera. The implementation process is similar and will not be expanded here.

[0152] Multi-channel photo or video recording

[0153] In a scenario where the shooting mode is switched to a multi-channel photo or video recording mode, one or more cameras other than the main camera are started and also added as the first camera.

[0154] S37. The application processor (AP) detects that the camera mode is switched to the front and rear dual-channel camera mode.

[0155] The front and rear dual-channel photo mode is only one of the multi-channel photo modes. The multi-channel photo mode can also include: the rear main camera, rear telephoto camera and rear wide-angle camera and other rear cameras taking photos at the same time. Multi-channel video recording is similar.

[0156] S38. The application processor (AP) may activate the front main camera and add the front main camera as the first camera.

[0157] At this time, the first camera includes two cameras: a rear main camera and a front main camera. The application processor (AP) does not turn off the rear main camera, and the YUV image frame of the rear main camera is not sent to the display independently, but needs to be spliced ​​with the YUV image frame of the front main camera and sent to the display together.

[0158] S39. Like the rear main camera, the front main camera can also transmit the captured raw image frames (RAW) to the ISP.

[0159] The ISP can also perform a series of processing on the RAW image and output the YUV image frame of the front main camera. The YUV image frame of the front main camera needs to be combined with the YUV image frame of the rear main camera to form a spliced ​​preview image to achieve the function of simultaneous front and rear camera photo preview.

[0160] The S40.ISP can transmit the stitched image frames from the front and rear main cameras to the display.

[0161] Here, S40 only illustrates the flow path of image data, and does not mean that the ISP directly sends the spliced ​​image frame to the display screen, and its implementation includes the process of sending and displaying.

[0162] S41. The display screen displays the spliced ​​image frame.

[0163] If the first camera needs to reduce the exposure time, the exposure time of the first camera can be set based on the flicker detection result to avoid water ripples in the imaging picture.

[0164] The above descriptions of "zoom", "flip camera", and "multi-channel photo or video recording" illustrate that the first camera can be changed. Not limited to these scenarios, the electronic device can also switch the first camera in other scenarios. For example, when the photo mode is switched to "macro mode", the first camera can be switched to a macro camera.

[0165] Figure 7 In the embodiment, when the first camera is switched, the second camera always uses the depth of field camera. Not limited to this, the second camera can be switched, and the embodiment of the present application does not limit its switching strategy. For example, for an electronic device with only two cameras (a front camera and a rear camera), when a flip camera is detected, the first camera switches to the front camera, and the second camera can be switched to the rear camera.

[0166] Figure 7 In the embodiment, each camera is coupled to one ISP. Not limited thereto, the electronic device may be configured with multiple ISPs, and multiple cameras may be coupled to different ISPs.

[0167] The term "user interface (UI)" in the specification, claims and drawings of this application refers to the medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface of an application is source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on the terminal device, and finally presented as content that the user can recognize, such as pictures, text, buttons and other controls. Controls, also known as widgets, are the basic elements of the user interface. Typical controls include toolbars, menu bars, text boxes, buttons, scroll bars, pictures and text. The properties and contents of controls in the interface are defined by tags or nodes, such as XML through <textview> 、 <imgview> 、

[0168] <videoview>The controls contained in the interface are specified by nodes such as <head> and <body>. A node corresponds to a control or attribute in the interface. After being parsed and rendered, the node is presented as user-visible content. In addition, many applications, such as hybrid applications, usually also contain web pages in their interfaces. A web page, also known as a page, can be understood as a special control embedded in the application interface. A web page is source code written in a specific computer language, such as hypertext markup language (HTML), cascading style sheets (CSS), JavaScript (JS), etc. The web page source code can be loaded and displayed as user-recognizable content by a browser or a web page display component with similar functions to a browser. The specific content contained in a web page is also defined by tags or nodes in the web page source code. For example, HTML is defined by <body>. 、 、 <video> 、 <canvas>To define the elements and attributes of a web page.

[0169] The most common form of user interface is graphical user interface (GUI), which refers to a user interface related to computer operation that is displayed in a graphical manner. It can be an icon, window, control or other interface element displayed on the display screen of an electronic device, where a control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets and other visual interface elements.

[0170] Each step in the above method embodiment provided by the present application can be completed by an integrated logic circuit of hardware in a processor or by instructions in the form of software. The method steps disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

[0171] The present application also provides an electronic device, which may include: a memory and a processor, wherein the memory may be used to store a computer program; and the processor may be used to call the computer program in the memory so that the electronic device executes the method in any one of the above embodiments.

[0172] The present application also provides a chip system, which includes at least one processor for implementing the functions involved in the method executed by the electronic device in any of the above embodiments.

[0173] In one possible design, the chip system also includes a memory, which is used to store program instructions and data, and the memory is located inside or outside the processor.

[0174] The chip system may be composed of the chip, or may include the chip and other discrete devices.

[0175] Optionally, the processor in the chip system may be one or more. The processor may be implemented by hardware or by software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented by software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

[0176] Optionally, the memory in the chip system may also be one or more. The memory may be integrated with the processor or may be separately arranged with the processor, which is not limited in the embodiments of the present application. Exemplarily, the memory may be a non-transient processor, such as a read-only memory ROM, which may be integrated with the processor on the same chip or may be arranged on different chips respectively. The embodiments of the present application do not specifically limit the type of memory and the arrangement of the memory and the processor.

[0177] Exemplarily, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chips.

[0178] The present application also provides a computer program product, which includes: a computer program (also referred to as code, or instruction), which enables a computer to execute the method executed by the electronic device in any of the above embodiments when the computer program is executed.

[0179] The present application also provides a computer-readable storage medium, which stores a computer program (also referred to as code or instruction). When the computer program is executed, the computer executes the method executed by the electronic device in any of the above embodiments.

[0180] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that the computer can access or a data storage device such as a server or a data center that includes one or more available media integrations. Available media can be magnetic media, (e.g., floppy disk, hard disk, tape), optical media (e.g., DVD), or semiconductor media (e.g., solid-state drive Solid State Disk), etc.

[0181] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

[0182] In short, the above are only embodiments of the technical solution of the present invention, and are not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made according to the disclosure of the present invention shall be included in the protection scope of the present invention.< / canvas> < / video> < / videoview> < / imgview> < / textview>

Claims

1. A method for detecting a flickering light source, characterized in that: include: The electronic device starts a first camera, where the first camera is a main camera; The electronic device displays a preview image on a display screen, where the preview image comes from an image captured by the main camera; The electronic device determines that the ambient brightness is neither higher than a first high brightness nor lower than a first low brightness; The electronic device starts the second camera and sets the exposure time of the second camera to a second duration, where the second duration is less than the light energy period; The electronic device performs image recognition on the image captured by the second camera to determine whether the image content contains water ripple features, and if so, controls the exposure time of the first camera to be an integer multiple of the light energy period; When the shooting mode changes or the zoom ratio changes, the electronic device controls the first camera to switch from the main camera to the target camera, and the target camera includes one or more of the following: a telephoto camera, a wide-angle camera, and a macro camera; The controlling the first camera to switch from the main camera to the target camera comprises: starting the target camera and displaying a preview image captured by the target camera on the display screen; The electronic device performs image recognition on the image captured by the second camera to determine whether the image content contains water ripple features, and if so, controls the exposure time of the first camera to be an integer multiple of the light energy period; Unlike the first camera, the second camera does not participate in imaging, but only provides imaging auxiliary data for the first camera, wherein the imaging auxiliary data includes depth of field data; the shutters of the first camera and the second camera are both rolling shutters.

2. The method according to claim 1, characterized in that Also includes: When the first camera is started, the exposure time of the first camera is a first duration, and the first duration is greater than the light energy period and is an integer multiple of the light energy period.

3. The method according to claim 2, characterized in that Before the electronic device starts the first camera, the method further includes: detecting a user operation of opening a camera application.

4. The method according to claim 1, characterized in that The first camera is a plurality of cameras, and the preview image is formed by splicing images captured by the plurality of cameras.

5. The method according to any one of claims 1 to 4, characterized in that The second duration is set to be shorter than the minimum light energy cycle among the light energy cycles of multiple power frequencies.

6. The method according to claim 5, characterized in that The exposure duration of the first camera is set by the automatic exposure control AEC; the electronic device determines that the ambient brightness is neither higher than a first high brightness nor lower than a first low brightness, specifically including: determining that the ambient brightness is not an overly bright environment based on the exposure duration of the first camera being greater than a third duration, determining that the ambient brightness is not an overly dark environment based on the exposure duration of the first camera being less than a fourth duration, and the third duration being less than the fourth duration.

7. The method according to claim 6, characterized in that The third duration and the fourth duration are recorded in an exposure table of the first camera.

8. The method according to claim 7, characterized in that If included, controlling the exposure time of the first camera to be an integer multiple of the light energy period specifically includes: If included, when reducing the exposure time of the first camera, the exposure time of the first camera is specifically reduced to one light energy period.

9. An electronic device, characterized in that: The method comprises one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program codes, wherein the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the method according to any one of claims 1 to 8 is executed.

10. A chip system, the chip system is applied to electronic equipment, the chip system comprises one or more processors, characterized in that: The processor is configured to call computer instructions so as to execute the method according to any one of claims 1 to 8.

11. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the method according to any one of claims 1 to 8 is executed.

Images