Method and electronic device for detecting flickering light source

Abstract

The present application provides a method and electronic device for detecting flickering light sources, in which the electronic device sets the exposure time of the first camera to an integer multiple of the first light energy period, sets the exposure time of the second camera to an integer multiple of the second light energy period, and performs flicker detection on the images captured by the two cameras. If the first image contains water ripple features, there is a flickering light source of the second light energy period in the environment, and the light energy periods corresponding to the two cameras are swapped. If the second image contains water ripples, there is a flickering light source of the first light energy period in the environment, and the current light energy period setting is maintained. If neither image contains water ripples, there is no flickering light source in the environment. In this way, it is possible to detect at one time whether there is a flickering light source of the first light energy period, a flickering light source of the second light energy period, or no flickering light source in the environment; and then adjust the exposure time according to the detection result to avoid the water ripple phenomenon.

Description

Technical Field

[0001] The present application relates to the field of terminals, and in particular to a method and electronic device for detecting a flickering light source. Background Art

[0002] With the development of mobile devices and the maturity of image processing technology, people's requirements for mobile photography are gradually increasing. In practical applications, using a rolling shutter camera under AC light sources (such as fluorescent lights, TVs, computer screens, or household lighting) can cause "water ripples" in the image, seriously affecting image quality.

[0003] To improve image quality, devices often incorporate dedicated flicker detectors to detect flickering light sources in the shooting environment. These sensors, in conjunction with the camera, set the exposure time to an integer multiple of the corresponding light energy cycle, thus addressing the water ripple problem. However, such devices incur additional costs and design space. Summary of the Invention

[0004] The embodiments of the present application provide a method and electronic device for detecting flickering light sources. Without introducing additional dedicated detection devices, it is possible to detect at one time whether there is a flickering light source in the environment and the specific type of the flickering light source. The exposure time of the camera can then be adjusted according to the specific type of the flickering light source, thereby preventing users from seeing water ripples caused by various types of flickering light sources in the images sent by the camera, thereby effectively improving the user experience.

[0005] In a first aspect, the present application provides a method for detecting flickering light sources, which is applied to an electronic device including a first camera and a second camera, the method comprising: setting an exposure time of the first camera to an integer multiple of a first light energy period, and the first camera capturing a first image; the electronic device displaying a preview image on a display screen, the preview image being the image captured by the first camera; setting the exposure time of the second camera to an integer multiple of a second light energy period, and the second camera capturing a second image; wherein the shutters of the first camera and the second camera are both rolling shutters; the electronic device performing water ripple detection based on the first image and the second image; when the first image contains water ripple features, there is a flickering light source corresponding to the second light energy period in the environment, and the electronic device sets the exposure time of the first camera to an integer multiple of the second light energy period, and sets the exposure time of the second camera to an integer multiple of the first light energy period; when the first image does not contain water ripple features and the second image contains water ripple features, there is a flickering light source corresponding to the first light energy period in the environment, and the exposure time of the first camera is maintained at an integer multiple of the first light energy period, and the exposure time of the second camera is maintained at an integer multiple of the second light energy period; when neither the first image nor the second image contains water ripple features, there is no flickering light source in the environment.

[0006] In implementing the method provided in the first aspect, the electronic device can respectively set the exposure time of the first camera to an integer multiple of the first light energy period and the exposure time of the second camera to an integer multiple of the second light energy period. Based on the first image captured by the first camera and the second image captured by the second camera, the electronic device can obtain information about the flickering light source in the current shooting environment, and then control the exposure of the first camera and the second camera. If the first image contains water ripple features, the electronic device can determine that there is a flickering light source corresponding to the second light energy period in the environment, and the electronic device can set the exposure time of the first camera to an integer multiple of the second light energy period, thereby avoiding the appearance of water ripples in the preview image provided by the first camera. In order to continue to detect the flickering light source in the shooting environment, the exposure time of the second camera can also be changed to an integer multiple of the first light energy period. If the first image does not contain water ripple features and the second image contains water ripple features, the electronic device can determine that there is a flickering light source corresponding to the first light energy period in the environment, and the electronic device can maintain the current settings of the light energy periods corresponding to the first camera and the second camera. If the images captured by both cameras do not contain water ripple features, the electronic device can determine that there is no flickering light source in the environment.

[0007] Compared to existing solutions that rely on dedicated flicker detectors to detect flickering light sources in the shooting environment, this solution leverages the electronic device's two existing cameras to detect flickering light sources in the shooting environment, effectively reducing solution costs and design space. The coordinated detection of two cameras also helps the electronic device quickly obtain flickering light source information in the current shooting environment in a single test, enabling faster water ripple removal and a better user experience.

[0008] In combination with the first aspect, in some embodiments, when the first image contains water ripple features, the method also includes: the electronic device adjusts the exposure time of the first camera, and the minimum adjustment unit and minimum value of the exposure time of the first camera are both the second light energy period currently corresponding to the first camera.

[0009] By implementing the method provided in the above embodiment, when the first image contains water ripples, the electronic device can determine the presence of a flickering light source corresponding to the second light energy cycle in the environment. If a flickering light source corresponding to the second light energy cycle is present in the shooting environment, the electronic device can further fine-tune the exposure time of the first camera using the first camera's current second light energy cycle as the adjustment precision (i.e., the minimum adjustment unit) when the first image currently captured by the first camera for display no longer contains water ripples (i.e., after swapping the light energy cycles corresponding to the first and second cameras by setting the exposure times). This can further improve image clarity and exposure, thereby enhancing imaging quality.

[0010] In combination with the first aspect, in some embodiments, when the first image does not contain water ripple features and the second image contains water ripple features, the method also includes: the electronic device adjusts the exposure time of the first camera, and the minimum adjustment unit and minimum value of the exposure time of the first camera are both the first light energy period currently corresponding to the first camera.

[0011] When implementing the method provided in the above embodiment, when the first image does not contain water ripple features and the second image does, the electronic device can determine the presence of a flickering light source corresponding to the first light energy period in the environment. If a flickering light source corresponding to the first light energy period is present in the shooting environment, the electronic device can further fine-tune the exposure time of the first camera after flicker detection, using the first camera's current first light energy period as the adjustment precision (i.e., the minimum adjustment unit), thereby further improving image clarity and exposure, and enhancing imaging quality.

[0012] In combination with the first aspect, in some embodiments, when neither the first image nor the second image contains water ripple features, the method further includes: the electronic device adjusts the exposure time of the first camera, and the minimum adjustment unit and minimum value of the exposure time of the first camera are both smaller than the first light energy period and the second light energy period.

[0013] By implementing the method provided in the above embodiment, when there is no flickering light source in the shooting environment, the electronic device is not limited to using the light energy period as the adjustment accuracy of the exposure time of the first camera (i.e., the minimum adjustment unit), so that the electronic device can adjust the exposure time according to other factors that actually affect the imaging quality, such as the actual lighting conditions of the current scene or the actual movement speed of the photographed object, thereby obtaining better shooting effects.

[0014] In combination with the first aspect, in some embodiments, the electronic device adjusts the exposure time of the first camera, specifically including: the electronic device performs image recognition on the first image to determine whether the content of the first image contains motion blur features; when the first image contains motion blur features, the electronic device reduces the exposure time of the first camera.

[0015] By implementing the method provided in the above embodiment, the electronic device can perform image recognition on the first image captured by the first camera, even if the image does not contain ripples, or after the ripples have been eliminated. If the first image contains motion blur, it indicates that the current exposure time of the first camera is too long. In this case, the electronic device can reduce the exposure time of the first camera, thereby reducing the motion blur in the first image caused by the long exposure time set during flicker detection, and improving image clarity.

[0016] In combination with the first aspect, in some embodiments, the electronic device adjusts the exposure time of the first camera, specifically including: the electronic device reduces the exposure time of the first camera through automatic exposure control AEC.

[0017] By implementing the method provided in the above embodiment, the electronic device can use AEC to further reduce the exposure time of the first camera when there is no water ripple in the image captured by the first camera, or after the water ripple is eliminated, so that the first camera for display can set the accurate exposure time in real time in the changing ambient light, thereby improving the imaging quality.

[0018] In combination with the first aspect, in some embodiments, the exposure time of the first camera is set to an integer multiple of the first light energy period, specifically including: when the electronic device starts the first camera, the exposure time of the first camera is set to an integer multiple of the first light energy period; the method also includes: when the electronic device starts the first camera, the second camera is started; the exposure time of the second camera is set to an integer multiple of the second light energy period, specifically including: when the second camera is started, the exposure time of the second camera is set to an integer multiple of the second light energy period.

[0019] By implementing the method provided in the above embodiment, the electronic device can set an initial exposure time when activating the first camera. Specifically, the exposure time of the first camera can be set to an integer multiple of the first light energy period. When the first camera is activated, the second camera can also be activated, and its exposure time can be set to an integer multiple of the second light energy period. Since the activation of the second camera is triggered by the activation of the first camera, the user does not perform any operation. Therefore, the user is unaware of the activation of the second camera, simplifying user operation.

[0020] In combination with the first aspect, in some embodiments, before the electronic device starts the first camera, the method further includes: the electronic device detects a user operation of opening a camera application.

[0021] In combination with the first aspect, in some embodiments, the exposure time of the first camera is set to an integer multiple of the first light energy cycle, specifically including: the electronic device determines the automatic exposure time of the first camera through the automatic exposure control AEC; the electronic device determines the value of the integer multiple based on the automatic exposure time and the first light energy cycle; the electronic device sets the exposure time of the first camera to an integer multiple of the first light energy cycle.

[0022] In combination with the first aspect, in some embodiments, after the electronic device displays a preview image on the display screen, the method also includes: the electronic device detects a first zoom operation; the preview image displayed by the electronic device is switched to an image captured by the second camera; the first zoom operation is used to adjust the zoom ratio from the zoom range of the first camera to the zoom range of the second camera.

[0023] By implementing the method provided in the above embodiment, the electronic device can switch the camera for display to the second camera when the zoom magnification is adjusted from the zoom range of the first camera to the zoom range of the second camera.

[0024] Adjusting the zoom ratio from the zoom range of the first camera to the zoom range of the second camera includes lowering the zoom ratio to the zoom range of the second camera and also includes increasing the zoom ratio to the zoom range of the second camera.

[0025] In combination with the first aspect, in some embodiments, in response to the first zoom operation, before switching the preview image to the image captured by the second camera, the method further includes: if the exposure time of the first camera is currently constrained to be an integer multiple of the first light energy period, setting the exposure time of the second camera to an integer multiple of the first light energy period, and the second camera captures a fourth image; setting the exposure time of the first camera to an integer multiple of the second light energy period, and the second camera captures a third image; if the exposure time of the first camera is currently constrained to be an integer multiple of the second light energy period, setting the exposure time of the second camera to an integer multiple of the second light energy period, and the second camera captures a fourth image; setting the exposure time of the first camera to an integer multiple of the first light energy period An integer multiple of the first light energy period or the second light energy period is set, and the second camera captures the third image; if the exposure time of the first camera is not currently constrained to be an integer multiple of the first light energy period or the second light energy period, the exposure time of the second camera is set to be an integer multiple of the second light energy period, and the second camera captures the fourth image; the exposure time of the first camera is set to be an integer multiple of the first light energy period, and the first camera captures the third image; or, when the exposure time of the first camera is not currently constrained to be an integer multiple of the first light energy period or the second light energy period, the exposure time of the second camera is set to be an integer multiple of the first light energy period, and the second camera captures the fourth image; the exposure time of the first camera is set to be an integer multiple of the second light energy period, and the first camera captures the third image.

[0026] When implementing the method provided in the above embodiment, if the exposure time of the first camera is constrained to an integer multiple of the first light energy period or the second light energy period, then when the electronic device's zoom ratio is adjusted to the zoom ratio of the second camera, the electronic device can reset the exposure time of the two cameras, thereby swapping the light energy periods corresponding to the two cameras. In this way, after performing water ripple removal, the electronic device can perform a zoom operation so that when the display camera switches from the first camera to the second camera, the first frame of the fourth image displayed by the second camera will not have water ripples, thereby achieving smooth zooming.

[0027] If the exposure time of the first camera is not limited to an integer multiple of the first light energy period or the second light energy period at this time, then when the zoom magnification of the electronic device is adjusted to the zoom magnification of the second camera, the electronic device can first set the first camera and the second camera to integer multiples of the light energy period, and then detect and eliminate water ripples for the third image and the fourth image captured after the set exposure time, thereby ensuring that there are no water ripples in the first frame of the fourth image sent by the second camera, and achieving smooth zoom.

[0028] In combination with the first aspect, in some embodiments, in response to the first zoom operation, before switching the preview image to the image captured by the second camera, the method also includes: the electronic device performs water ripple detection based on the third image and the fourth image; when the fourth image contains water ripple features, the electronic device sets the exposure time of the second camera to the first exposure time, and the first exposure time is an integer multiple of the light energy period different from the light energy period currently corresponding to the second camera; when the fourth image does not contain water ripple features and the third image contains water ripple features, the exposure time of the second camera is maintained at an integer multiple of the light energy period currently corresponding to the second camera; when neither the fourth image nor the third image contains water ripple features, there is no flickering light source in the environment.

[0029] When implementing the method provided in the above embodiment, a zoom operation can occur after any step of the above-described flickering light source detection method. Therefore, the electronic device cannot guarantee that it has completed at least one round of water ripple detection and elimination when zoom operation 1 is performed. If the electronic device has not completed at least one round of water ripple detection and elimination, that is, the light energy cycle set by the first camera is different from the flickering light source in the current shooting environment, no water ripples will be present in the image captured by the second camera. In this case, because the electronic device swaps the light energy cycles corresponding to the first and second cameras after performing zoom operation 1, the image captured by the second camera after the swapping will contain water ripple features. In this case, after performing zoom operation 1 and before displaying the fourth image captured by the second camera, the electronic device performs water ripple detection and elimination on the third image captured by the first camera and the fourth image captured by the second camera. This ensures that the fourth image displayed does not contain water ripple features even if the electronic device has not yet performed water ripple detection and elimination when the zoom operation occurs.

[0030] In combination with the first aspect, in some embodiments, before the electronic device detects the first zoom operation, the method also includes: when the zoom ratio is adjusted to the first zoom range, the exposure time of the second camera is constrained to the third light energy cycle, and the electronic device collects the fifth image of the first camera and the sixth image of the second camera respectively; the third light energy cycle is the first light energy cycle or the second light energy cycle; wherein, when the first zoom range is a zoom range between the first focal length threshold and the zoom range of the second camera, the first focal length threshold belongs to the zoom range of the first camera, and the minimum difference between the first focal length threshold and the focal length within the zoom range of the second camera is a preset value.

[0031] By implementing the method provided in the above embodiment, before switching the main camera for display to the second camera, when the zoom ratio is within the first zoom range (i.e., the transition zoom range), the exposure time of the second camera is constrained to the third light energy cycle (i.e., light energy cycle 3). In this way, the exposure time of the second camera is matched with the flickering light source in the environment in advance, so that the first image displayed by the second camera after switching the camera will not have water ripple features, thereby ensuring a smooth zoom user experience.

[0032] In combination with the first aspect, in some embodiments, before the electronic device detects the first zoom operation, the method also includes: the electronic device performs water ripple detection based on the fifth image and the sixth image; when the sixth image contains water ripple features, setting the exposure time of the second camera and the current exposure time of the first camera to be integer multiples of the fourth light energy cycle, and the fourth light energy cycle is a light energy cycle different from the third light energy cycle; when the sixth image does not contain water ripple features and the fifth image contains water ripple features, setting the current exposure time of the first camera to an integer multiple of the third light energy cycle; when neither the fifth image nor the sixth image contains water ripple features, there is no flickering light source in the environment.

[0033] When implementing the method provided in the above embodiment, a zoom operation can occur after any step of the above-described flickering light source detection method. Therefore, the electronic device cannot guarantee that it has completed at least one round of water ripple detection and elimination when performing zoom operation 1. In this case, the electronic device can perform water ripple detection and elimination based on the fifth and sixth images. This ensures that the fourth image displayed does not contain water ripple characteristics even if the electronic device has not yet performed water ripple detection and elimination when the zoom operation occurs.

[0034] In combination with the first aspect, in some embodiments, the method also includes: when the sixth image includes water ripple features, in response to the first zoom operation, setting the exposure time of the first camera to an integer multiple of the third light energy period; when the sixth image does not include water ripple features and the fifth image includes water ripple features, in response to the first zoom operation, setting the current exposure time of the first camera to an integer multiple of the fourth light energy period.

[0035] By implementing the method provided in the above embodiment, the electronic device can adaptively set the exposure time of the first camera after sending and displaying the fourth image, so that the light energy period corresponding to the first camera is different from the light energy period corresponding to the second camera, so that the electronic device can continue to detect the flickering light source in the shooting environment through the first camera and the second camera after performing a zoom operation and switching the sending and display camera from the first camera to the second camera.

[0036] In combination with the first aspect, in some embodiments, the first camera is a wide-angle camera, and the second camera is an ultra-wide-angle camera; or the first camera is an ultra-wide-angle camera, and the second camera is a wide-angle camera.

[0037] In a second aspect, the present application provides an electronic device comprising one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, and the computer program code comprises computer instructions. When the one or more processors execute the computer instructions, the electronic device executes the method described in the first aspect and any possible implementation of the first aspect.

[0038] In a third aspect, an embodiment of the present application provides a chip system, which is applied to an electronic device, and the chip system includes one or more processors, which are used to call computer instructions to enable the electronic device to execute the method described in the first aspect and any possible implementation method of the first aspect.

[0039] In a fourth aspect, the present application provides a computer-readable storage medium comprising instructions, which, when executed on an electronic device, enables the electronic device to execute the method described in the first aspect and any possible implementation of the first aspect.

[0040] It is understandable that the electronic device provided in the second aspect, the chip system provided in the third aspect, and the computer storage medium provided in the fourth aspect are all used to perform the method provided in this application. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1The working principle of the rolling shutter is shown as an example;

[0042] Figure 2 This example shows that the preview image provided by the rolling shutter camera has a water ripple problem;

[0043] Figure 3 The example shows that the exposure time of the rolling shutter camera is set to twice the light energy period to solve the water ripple problem;

[0044] Figure 4 The overall process of a flickering light source detection method provided by an embodiment of the present application is shown;

[0045] Figures 5A-5C Schematic diagram showing changes in exposure time in a group of different shooting environments provided by an embodiment of the present application;

[0046] Figure 6 The following is a flow chart illustrating a method for detecting a flickering light source in a smooth zoom scenario provided by an embodiment of the present application.

[0047] Figure 7A A schematic diagram exemplarily illustrates a flow chart of another method for detecting a flickering light source in a smooth zoom scenario provided by an embodiment of the present application;

[0048] Figure 7B-7C A schematic diagram of a set of zoom ranges provided in an embodiment of the present application is shown;

[0049] Figure 8 The hardware structure of the electronic device according to the embodiment of the present application is exemplified;

[0050] Figure 9 The software architecture of the electronic device according to the embodiment of the present application is exemplarily shown;

[0051] Figure 10 The detailed implementation process of a flashing light source method provided by an embodiment of the present application based on the internal architecture of an electronic device is shown;

[0052] Figure 11 The detailed implementation process of a flashing light source method in a smooth zoom scenario provided by an embodiment of the present application is shown based on the internal architecture of an electronic device. DETAILED DESCRIPTION

[0053] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application.

[0054] The shutter is a key component of a camera, controlling the exposure time of the photosensitive film. Shutters can be categorized as global shutters or rolling shutters, each with different exposure methods. With a global shutter, all pixels on the photosensitive film begin to be exposed to light simultaneously, and after the same exposure time, all pixels end their exposure at the same time. With a rolling shutter, each row of pixels on the photosensitive film is exposed row by row, with each row exposed for the same duration until all pixels are fully exposed. The exposure start and end times for different rows vary.

[0055] Taking pictures with a rolling shutter camera under certain light sources may cause "banding" to appear in the image. These light sources are often driven by alternating current. The energy transmitted by alternating current is not constant, but varies with a fixed frequency, causing periodic fluctuations in the light intensity of the light source. Currently, some countries have a fixed frequency of 50Hz for electrical appliances, while others have a fixed frequency of 60Hz. The period followed is the light energy period corresponding to this fixed frequency (also known as the industrial frequency). For example, the light intensity fluctuation frequency of a 50Hz AC light source is 100Hz, and the light energy period is 10 milliseconds, meaning that the light source flickers 100 times per second. In this article, this type of light source is also referred to as a flickering light source.

[0056] For rolling shutter cameras, when the exposure time is less than the light energy cycle of the light source, since the exposure start and end times of each row of pixels on the photosensitive film are different, the accumulated light energy received by each row of pixels within the same exposure time is different, and light and dark stripes will appear on the image. Figure 1 As shown in , when imaging the first frame of image, the light energy received by the first row of pixels on the photosensitive film is the light energy accumulated from time 0 to time t2, while the light energy received by the nth row of pixels on the photosensitive film is the light energy accumulated from time t1 to 5 milliseconds. The light energy received by the nth row is less than the light energy received by the first row. For another example, Figure 1 As shown in the figure, when imaging the N+1 frame, the light energy received by the pixels in the first row on the photosensitive film is the light energy accumulated from time t3 to time t5, while the light energy received by the pixels in the nth row on the photosensitive film is the light energy accumulated from time t4 to time t6. The light energy received by the first row is less than the light energy received by the nth row. In a frame of imaging, the rows with large light energy appear as bright stripes, while the rows with small light energy appear as dark stripes, which can be seen as follows: Figure 2 As shown in the figure, the alternating light and dark stripes are water ripples, also known as flicker. Furthermore, users can observe the water ripples rolling when shooting, previewing, or recording videos.

[0057] In shooting scenes with fast shutter speeds, such as motion capture and high dynamic range (HDR) shooting modes, water ripples are more likely to appear in the image. After detecting that the light source in the shooting environment is a flickering light source, one way to avoid this problem is to set the exposure time to an integer multiple of the light energy cycle. Figure 3 As shown, the light energy cycle is 10 milliseconds, and the exposure time can also be set to 10 milliseconds, that is, 1 times the light energy cycle.

[0058] Detecting flickering light sources is key to solving the water ripple problem. Existing detection methods use specialized flicker detectors, such as high-sampling ambient light sensors and multispectral flicker detectors, to detect the presence of flickering light sources in the shooting environment. These sensors, combined with cameras, can address the water ripple problem. However, these devices incur additional costs and design space.

[0059] In an embodiment of the present application, a detection method is provided. For an electronic device with dual cameras, one of the cameras is used as a main camera for preview display only, and the other camera is used as a secondary camera for detecting the presence of a flickering light source with a specific power frequency in the shooting environment, such as a 50Hz flickering light source. The method then eliminates the water ripples of the specific power frequency in the image captured by the main camera based on the detection results of the secondary camera. However, with globalization, electrical products from various countries are circulated among each other, and the user's environment may be subject to a flickering light source with a 50Hz or a 60Hz flickering light source. The above method can only detect flickering light sources with a specific power frequency and cannot detect at one time whether a flickering light source with a 50Hz, a 60Hz, or no flickering light source exists in the shooting environment. Consequently, the method cannot make subsequent adjustments based on the specific conditions of the flickering light source in the environment.

[0060] An embodiment of the present application also provides a flickering light source detection method, which can quickly detect the flickering light source information in the environment at one time without introducing additional dedicated detection devices, that is, the presence of a 50Hz flickering light source, or the presence of a 60Hz flickering light source, or the absence of a flickering light source.

[0061] The electronic device 100 implementing this method has at least two rolling shutter cameras, designated as a first camera and a second camera. The first camera and the second camera each detect flickering light sources of different frequencies in the shooting environment. The first camera or the second camera can also be used for preview display. The first and second cameras are not specifically designed for flickering light source detection; they can be existing cameras on the electronic device, such as an ultra-wide-angle camera or a macro camera.

[0062] Figure 4The overall process of a flickering light source detection method provided by an embodiment of the present application is shown and will be expanded below.

[0063] S101. Set the exposure time of the first camera to N1 times the first light energy period, capture a first image and send it for display.

[0064] In some embodiments, the operation of setting the exposure time of the first camera may be triggered by an event in which the user opens a camera application and starts the first camera.

[0065] Specifically, when the first camera is activated, the exposure time of the first camera can be set to N1 times the first light energy period. For example, when N1 = 1, the exposure time of the first camera is 10 milliseconds. In this way, if the flashing light source in the current shooting environment has a power frequency of 50Hz, the user can avoid the generation of water ripples when the camera is initially activated. It should be noted that the integer multiples N1, M1, N2, and M2 mentioned in this document are all positive integers.

[0066] The value of N1 may be determined by the ISP for the first camera according to an output result of an automatic exposure control (AEC) algorithm.

[0067] Specifically, the photosensitive element (such as an image sensor) in the first camera can collect light information in the environment. The above light information may include light intensity, brightness distribution, etc. The AEC algorithm can generate an appropriate exposure time based on the above light information collected by the first camera, which is called automatic exposure duration. The electronic device 100 can obtain the automatic exposure duration generated by the AEC, and then determine the integer multiple of the first light energy period closest to the above automatic exposure duration, and then determine the value of N1. Optionally, N1 times the above first light energy period should be greater than or equal to the automatic exposure duration. Exemplarily, the automatic exposure duration of the first camera generated by the ISP according to the AEC is 1 / 60 second (about 16.6 milliseconds). At this time, N1 can be set to 2, so that 2×1 / 100=1 / 50 second (i.e. 20 milliseconds) is the minimum integer multiple of the first light energy period greater than 1 / 60.

[0068] The scenario of starting the first camera is not limited to the scenario where the user opens the camera application. The scenario of starting the first camera can also be other scenarios that require starting the camera to capture images, such as video call scenarios, scan code payment, identity verification and other scenarios. For example, in the scan code payment scenario, when it is detected that the user has turned on the scan code payment, the electronic device can start the rear main camera to scan the payment QR code. At this time, the first camera is the rear main camera. For another example, in the identity verification scenario, the electronic device can start the front main camera and the front depth of field camera to scan and obtain facial feature data for user identity verification. At this time, the first camera can be the front main camera. For another example, in the video call scenario, the electronic device can start the front main camera to capture the portrait of the user in front of the screen. The main camera mentioned in this article is the camera that sends the display.

[0069] After the exposure time is set, the image captured by the first camera is recorded as the first image. This first image can be used for preview display, i.e., sent for display, thereby serving as the preview image displayed on the display screen by the camera application. It will be appreciated that as time passes, the sending for display can be continued, continuously displaying the preview stream of the first camera, thereby continuously providing a preview display to the user.

[0070] S102 : Setting the exposure time of the second camera to M1 times the second light energy period to capture a second image.

[0071] In some embodiments, when the user opens the camera application and activates the first camera to capture preview images and detect flickering light sources, the second camera is also activated to detect flickering light sources; the operation of setting the exposure time of the second camera can also be triggered by the above-mentioned event of activating the second camera.

[0072] In some embodiments, the process for setting the M1 value in the exposure time of the second camera is the same as the process for setting the N1 value in the exposure time of the first camera described above. In some embodiments, M1 = N1. In some embodiments, since the second camera is only used for detection, M1 can be a preset fixed value, such as 1.

[0073] It should be noted that the second light energy period corresponding to the exposure time of the second camera should be set to a light energy period different from the first light energy period corresponding to the exposure time of the first camera, so that as long as there is a flickering light source with an industrial frequency of 50Hz or 60Hz in the shooting environment, it can be detected by the first camera and the second camera. The first light energy period and the second light energy period are set according to the flickering light sources that may exist in the known environment; in the subsequent embodiments, "the first light energy period is the light energy period of the flickering light source of 50Hz, and the second light energy period is the light energy period of the flickering light source of 60Hz" is used as an example for illustrative description. It can be understood that in another implementation method, the first light energy period can be the light energy period of the flickering light source of 60Hz, and the second light energy period can be the light energy period of the flickering light source of 50Hz. It is not limited to flickering light sources of 50Hz and 60Hz, and flickering light sources of other industrial frequencies can also exist in the environment, which is not specifically limited here.

[0074] After the exposure time is set, the image captured by the second camera is recorded as the second image. The second image is not used for display and is only used for flicker detection, so the user does not perceive it.

[0075] S103 : Perform flicker detection on the first image and the second image respectively.

[0076] The result of the flicker detection may include: detecting a flickering light source and determining flickering light source information, and not detecting a flickering light source.

[0077] It can be understood that performing flicker detection on an image is essentially to identify whether there are water ripple features in the image, so the above flicker detection is also called water ripple detection.

[0078] Taking flicker detection on the first image as an example, optionally, the electronic device 100 may perform flicker detection by performing a difference calculation on two consecutive frame images.

[0079] Specifically, the electronic device 100 can capture a continuous sequence of image frames through a first camera. Then, two consecutive first images are selected from the image sequence captured by the first camera to perform pixel-level difference calculation. The pixel-level difference calculation refers to performing a subtraction operation on the corresponding pixels between the two consecutive first images. The result of the above-mentioned difference calculation is a difference image, which reflects the brightness difference between the two images. If there are alternating light and dark strips in the above-mentioned difference image, it is determined that the above-mentioned first image has water ripple features, which further indicates that there is a flickering light source in the shooting environment of the first camera. The flicker detection of the second image is performed in the same way.

[0080] Not limited to the above method, the electronic device may also implement flicker detection for the first image and the second image through other algorithms such as neural networks. The embodiment of the present application does not impose any particular limitation on the flicker detection algorithm adopted by the electronic device 100.

[0081] S104: Determine whether water ripples exist in the first image.

[0082] It is understood that when a flickering light source (e.g., a 50 Hz flickering light source) is present in the environment, if the exposure time of the first camera is not an integer multiple of the light energy cycle corresponding to the flickering light source, the image captured by the camera will contain water ripples. Therefore, if the first image contains water ripple characteristics, the electronic device 100 can determine that a flickering light source is present in the current shooting environment, and the light energy cycle corresponding to the flickering light source is different from the first light energy cycle set by the first camera.

[0083] In S101, the first camera is set to an integer multiple of the first light energy period (i.e., the light energy period corresponding to 50 Hz). Therefore, if the first image has water ripple characteristics, the electronic device can exclude the possibility that the flickering light source in the current shooting environment has an operating frequency of 50 Hz. Since the operating frequency of AC light sources used domestically and abroad is 50 Hz or 60 Hz, the electronic device can further determine whether there is a flickering light source with a light energy period of the second light energy period (a flickering light source with an operating frequency of 60 Hz) in the current shooting environment.

[0084] If the first image does not have the above-mentioned water ripple feature, the electronic device 100 may determine that there is no flickering light source in the current shooting environment, or that the flickering light source in the current shooting environment has an operating frequency of 50 Hz.

[0085] S105: Determine whether water ripples exist in the second image.

[0086] Correspondingly, if the second image has water ripple features, the electronic device 100 can determine that there is a 50Hz flickering light source in the current shooting environment. If the second image does not have water ripple features, the electronic device 100 can determine that there is no flickering light source in the current shooting environment, or that the flickering light source in the current shooting environment has a power frequency of 60Hz. The judgment logic is the same as that in step S104 and will not be repeated here.

[0087] In combination with steps S104 and S105 , the electronic device 100 can determine the flickering light source information of the current shooting environment: that is, whether there is a 50 Hz flickering light source, or a 60 Hz flickering light source, or no flickering light source.

[0088] S106. When the first image has water ripple features and the second image does not have water ripple features, set the exposure time of the first camera to an integer multiple of the second light energy period, and set the exposure time of the second camera to an integer multiple of the first light energy period.

[0089] In some embodiments, when the presence of water ripple features in the first image and the absence of water ripple features in the second image is detected, it can be determined that the flickering light source in the environment matches the second light energy cycle currently corresponding to the second camera, that is, the light energy cycle corresponding to the flickering light source in the environment is the same as the second light energy cycle currently corresponding to the second camera, that is, there is a 60Hz flickering light source. The electronic device 100 sets the exposure time of the first camera to an integer multiple of the second light energy cycle and the exposure time of the second camera to an integer multiple of the first light energy cycle. In this way, the light energy cycle corresponding to the first camera matches the flickering light source in the environment, eliminating water ripples in the image displayed by the first camera.

[0090] In some embodiments, before step S106, the electronic device 100 can obtain the light energy cycle currently corresponding to the first camera (for example, the first light energy cycle) and the light energy cycle currently corresponding to the second camera (for example, the second light energy cycle), and then reset the exposure time of each camera.

[0091] Taking the first camera as an example, in one implementation, the electronic device 100 obtains the current exposure time 1 of the first camera, and then determines the light energy period currently corresponding to the first camera based on the exposure time 1. In another implementation, the electronic device 100 stores the light energy period currently corresponding to the first camera in field 1, and the electronic device 100 can directly obtain the light energy period currently corresponding to the first camera from field 1.

[0092] It is understood that after obtaining the corresponding light energy cycles of the first and second cameras in step S106, the electronic device 100 sets the exposure times of the two cameras to swap their corresponding light energy cycles, thereby swapping the flicker light sources detectable by the two cameras. This eliminates the moire feature in the image sent by the first camera while enabling simultaneous detection of flicker light source information by both cameras. Therefore, the specific setting of the exposure times of the first and second cameras in step S106 is not fixed but rather depends on the light energy cycles of the first and second cameras when performing flicker detection.

[0093] The following describes how to determine the value of the integer multiple corresponding to the light energy period when setting the exposure time of the first camera. In one implementation, the exposure time of the second camera before step S106 is M1 times the second light energy period, and then in step S106, the electronic device 100 sets the exposure time of the first camera to M1 times the second light energy period. In one implementation, determine the integer multiple of the second light energy period that is closest to the exposure time 1 of the first camera, that is, the value of M1'; and update the exposure time of the first camera to M1' times the second light energy period. In one implementation, the AEC algorithm is used to determine the value of the integer multiple of the second light energy period corresponding to the exposure time of the first camera. In an embodiment of the present application, the value of the integer multiple can also be determined in combination with one or more of the above-mentioned implementation methods, and the embodiment of the present application does not specifically limit this.

[0094] The following describes how to determine the value corresponding to the integer multiple of the light energy cycle when setting the exposure time of the second camera. In step S106, the exposure time of the second camera can be set to N1 times the first light energy cycle. Alternatively, since the second camera is only used for detection, the value corresponding to the integer multiple of the first light energy cycle of the second camera's exposure time can be set to a preset fixed value, such as 1. Alternatively, the setting method for the first camera can be referred to and the value corresponding to the integer multiple of the first light energy cycle of the second camera's exposure time can be set similarly.

[0095] S107: When the first image does not have the water ripple feature and the second image has the water ripple feature, determine to maintain the light energy cycle corresponding to the current camera.

[0096] Specifically, when the first image does not have water ripple features but the second image does, it can be determined that the flickering light source in the environment matches the light energy cycle currently corresponding to the first camera, that is, there is a 50Hz flickering light source. The exposure time of the first camera can continue to be an integer multiple of the first light energy cycle, and the exposure time of the second camera can continue to be an integer multiple of the second light energy cycle. In one implementation, the exposure time of the first camera is maintained at N1 times the first light energy cycle, and the exposure time of the second camera is maintained at M1 times the second light energy cycle.

[0097] S108: When neither the first image nor the second image has water ripple features, adjust the exposure time of the first camera according to the second exposure strategy.

[0098] The second exposure strategy described above means that the adjustment precision of the exposure time of the first camera is less than the first light energy period and the second light energy period, and the minimum exposure time is less than the first light energy period and the second light energy period, for example, 1 millisecond. This adjustment precision is also referred to as the minimum adjustment unit. It will be understood that when adjusting the exposure time of the first camera according to the second exposure strategy described above, the exposure time of the first camera can be a non-integer multiple of the first light energy period or the second light energy period, or an integer multiple of the first light energy period or the second light energy period.

[0099] When neither the first image nor the second image has water ripple features, it means that there is no flickering light source in the shooting environment of the electronic device 100. Since the exposure time set for the first camera in S101 and S102 is relatively long, it is easy to cause blurred shooting images for some fast-moving subjects. In this case, reducing the exposure time of the camera will not only prevent the appearance of water ripple features, but also obtain a clearer image. Therefore, in some embodiments, under the premise of complying with the adjustment accuracy and minimum value of the second exposure strategy, the electronic device 100 can adjust the exposure time of the first camera based on motion detection to avoid motion blur in the shot.

[0100] Specifically, the electronic device 100 can obtain two consecutive frames of the first image from the first image captured by the first camera. The electronic device 100 can then estimate the direction and speed of pixel movement by analyzing the pixel displacement between the two consecutive frames of the first image, thereby calculating the speed of the moving object. Optionally, if the electronic device 100 determines that the first image contains motion blur characteristics through the above-mentioned motion detection, the electronic device 100 can further determine the speed of the moving object based on the motion blur characteristics, and determine the exposure time of the first camera based on the motion speed reference mapping table. The above-mentioned mapping table shows the correspondence between the object's motion speed range and the exposure time; the greater the speed indicated by the motion speed range, the shorter the exposure time corresponding to the motion speed range. The mapping table can be pre-established and built into the electronic device. It will be understood that when the subject is in motion and the speed of movement increases, the exposure time of the first camera can be reduced with the adjustment accuracy of the second exposure strategy.

[0101] For example, the current exposure time of the first camera is 10 milliseconds. If the electronic device 100 determines through motion detection that there is a motion blur feature in the first image, and further concludes that the moving speed of the moving object in the first image is 60 kilometers per hour, the exposure time of the first camera can be reduced to 1 millisecond according to the mapping table.

[0102] If the electronic device 100 determines through the above motion detection that there is no motion blur feature in the first image, the electronic device can maintain the current exposure time.

[0103] Not limited to motion detection, under the premise of following the second exposure strategy, the exposure time can also be adjusted using one or more other methods such as AEC, and the embodiments of the present application do not specifically limit this.

[0104] In the embodiment of the present application, S109 may be further executed after steps S106 and S107.

[0105] S109: Adjust the exposure time of the first camera according to the first exposure strategy.

[0106] The above-mentioned first exposure strategy means: adjusting the exposure time of the first camera by adjusting the value of an integer multiple (such as N1 or M1), that is, the adjustment accuracy of the above-mentioned first exposure strategy is equal to the current light energy cycle of the first camera, and the minimum value of the exposure time is equal to the current light energy cycle.

[0107] In some embodiments, before executing the first exposure strategy, the electronic device 100 should first obtain the light energy cycle currently corresponding to the first camera, so as to determine the adjustment accuracy of the subsequent exposure time.

[0108] Optionally, the exposure time of the first camera can be adjusted using an AEC algorithm, provided that the adjustment accuracy and minimum value of the first exposure strategy are followed. It is understood that during the execution of this method by the electronic device 100, the ambient light may have changed to some extent. The electronic device can reset the exposure time of the first camera based on the AEC algorithm, i.e., readjust the value of the integer multiple N1 or M1. For details, please refer to the relevant description in step S101, which will not be repeated here.

[0109] Optionally, under the premise of following the first exposure strategy, the exposure time of the first camera can also be adjusted by using motion detection. For details, please refer to the relevant description in step S108, which will not be repeated here.

[0110] Not limited to the AEC algorithm or motion detection, under the premise of following the first exposure strategy, the exposure time can also be adjusted in combination with one or more other algorithms. The embodiments of the present application do not make specific limitations on this.

[0111] In the embodiments of the present application, when adjusting the exposure time of the first camera according to the first exposure strategy (or the second exposure strategy), the exposure time of the second camera can be maintained without adjustment, or the exposure time of the second camera can be adjusted simultaneously according to the first exposure strategy (or the second exposure strategy). Preferably, the exposure time of the second camera can be maintained without adjustment, that is, maintained at an integer multiple of the light energy period.

[0112] In the above method, S108 and S109 are optional, and in some embodiments, only S101-S107 may be executed. In addition, in the above method, the execution order of S101-S102 and S104-S105 is not particularly limited. The electronic device 100 may first execute S102 to set the exposure time of the second camera, and then execute S101 to set the exposure time of the first camera. Similarly, S105 may also be executed before S104. However, preferably, the electronic device 100 executes S104 first and then executes S105. In step S103, flicker detection may be performed on the first image first, and when it is determined that water ripples exist in the first image, flicker light source detection may not be performed on the second image. It is understandable that Figure 4 The flowchart shown only shows a possible schematic flow of the electronic device 100 executing the above method. In the above method, for steps S101 and S102, it is sufficient as long as the light energy periods corresponding to the first camera and the second camera are different. That is to say, when the exposure time of the first camera is initially set, it can also be set to the exposure time corresponding to the second light energy period. At this time, the exposure time of the second camera can be set to the exposure time corresponding to the first light energy period. If based on the initial setting of the camera exposure time, the subsequent step S106 should also be adaptively modified to "when the first image has water ripple features and the second image does not have water ripple features, set the exposure time of the first camera to N1 times the first light energy period; set the exposure time of the second camera to M1 times the second light energy period".

[0113] In some embodiments, the user can carry the electronic device 100 and move around. During the user's movement, the shooting environment in which the electronic device 100 is located may be constantly changing. For example, when the user carries the electronic device 100 and moves from room A to room B, the power frequency of the flashing light source in the shooting environment may change from 50 Hz in room A to 60 Hz in room B. Therefore, after starting the first camera to shoot and triggering the electronic device 100 to execute the above method, the electronic device 100 can also execute the above method provided in the embodiment of the present application at a certain frequency, thereby avoiding the reappearance of water ripple features in the image captured by the camera due to changes in the shooting environment. The above frequency is determined by the developer based on experience.

[0114] In some embodiments, if the electronic device 100 can confirm the presence of a flickering light source in the shooting environment through the flicker detection, the electronic device 100 can jump to step S103 after executing steps S106, S107, or S109 to begin looping the above method. If the electronic device 100 can confirm the absence of a flickering light source in the shooting environment through the flicker detection, the electronic device can loop the above method starting from S101 after executing S108.

[0115] In some embodiments, the electronic device periodically cycles Figure 4 In the method flow shown (including steps S101 and S102 ), the second camera is only used for flicker detection, and the exposure time of the second camera does not need to be adjusted after step S102 . For example, the exposure time of the second camera does not need to be set in step S106 .

[0116] Figures 5A-5C A schematic diagram showing changes in exposure time in a group of different shooting environments provided by an embodiment of the present application is shown.

[0117] For example, if there is a 50Hz flickering light source in the current ambient light source, according to Figure 4 In the method flow shown, the first image does not have water ripple features, but the second image does have water ripple features.

[0118] When electronic device 100 executes this method, the initial exposure time of the first camera is set to N1 / 100 seconds, and the initial exposure time of the second camera is set to M1 / 100 seconds. After this method is completed, both cameras maintain the current light energy cycle, and the exposure time of the first camera can also be N1 / 100 seconds, and the exposure time of the second camera can also be M1 / 120 seconds.

[0119] If the user moves the electronic device from a 50Hz flickering light environment to a 60Hz flickering light environment, the first image will have water ripple features, while the second image will not have water ripple features. Since the electronic device executes this method at a certain frequency, the exposure time of the first camera and the second camera will change from Figure 5A Switch to Figure 5B .like Figure 5B As shown, the exposure time of the first camera can be switched to M1 / 120 seconds, and the exposure time of the second camera can be switched to N1 / 100 seconds.

[0120] If the user carries the electronic device from a 50Hz flickering light environment to an environment without flickering light, the exposure time of the first camera and the second camera will change from Figure 5A Switch to Figure 5C , at this time, there is no water ripple feature in the first image and the second image. Figure 5C As shown, the electronic device 100 can reduce the exposure time of the first camera to 1 millisecond by further executing the second exposure strategy (step S108), and the exposure time of the second camera can be maintained at M1 / 120 seconds.

[0121] In some embodiments, the above method can also be applied to zoom scenarios.

[0122] During the zooming process, the electronic device 100 may switch the camera to be displayed due to the change of focal length. Figure 4 and Figures 5A-5C If there is a flashing light source in the environment, Figure 4 After the process shown, the exposure time of the first camera used for display can match the flickering light source in the environment, and the displayed image does not have water ripple characteristics. The exposure time of the second camera does not match the flickering light source in the environment. After switching the camera used for display to the second camera, the displayed image will again show water ripple characteristics. In this case, the above method needs to further address the water ripple characteristics that may exist when switching cameras, so that the image does not abruptly produce water ripples during zooming, thereby achieving a smooth zoom effect and improving the user experience.

[0123] It is understandable that the above zoom operation can occur in Figure 4 Any step after S101 and S102 in the process shown. For example, the above zoom operation can occur Figure 4 After S106 and S107 of the method shown, the electronic device may first complete a round of flicker detection and water ripple elimination processing, and then perform a zoom operation, and then trigger subsequent steps S201-S210.

[0124] Figure 6 FIG. 1 is a flow chart showing a method for detecting a flickering light source in a zooming scene provided by an embodiment of the present application. Figure 6 As shown, the method may occur after the zoom ratio of the electronic device 100 is adjusted from the zoom range of the first camera to the zoom range of the second camera.

[0125] S201. When zoom operation 1 is detected, if the current exposure time of the first camera is constrained to be an integer multiple of the light energy period, execute S202A; if the current exposure time of the first camera is not constrained to be an integer multiple of the light energy period, execute S202B; zoom operation 1 is used to adjust the zoom ratio from the zoom range of the first camera to the zoom range of the second camera.

[0126] The zoom operation 1 is also referred to as a first zoom operation.

[0127] If the current exposure time of the first camera is constrained to be an integer multiple of the light energy period, execute S202A to swap the light energy periods corresponding to the two cameras. If the current exposure time of the first camera is not constrained to be an integer multiple of the light energy period, execute S202B to reset the exposure times corresponding to the two cameras and constrain them to be integer multiples of different light energy periods.

[0128] It is understood that if the electronic device 100 executes the second exposure strategy in step S108 before step S201, the exposure time of the first camera is not constrained to an integer multiple of the light energy period. If the electronic device 100 executes steps S106, S107, or S109 (i.e., executes the first exposure strategy) before step S201, the exposure time of the first camera can be constrained to an integer multiple of the light energy period.

[0129] In some embodiments, whether the exposure time of the first camera is constrained to be an integer multiple of the light energy period can be determined based on the current exposure time of the first camera. Alternatively, the electronic device 100 stores Field 2, which indicates whether the exposure time of the first camera is constrained to be an integer multiple of the light energy period. Field 2 and Field 1 can be the same field or different fields. This embodiment of the application does not specifically limit the implementation method of "determining whether the exposure time of the camera is constrained to be an integer multiple of the light energy period."

[0130] For ease of description, when the zoom ratio is adjusted to the zoom range of the second camera, before step S202A or S202B, the exposure time of the first camera may also be referred to as the first exposure time, and the exposure time of the second camera may also be referred to as the second exposure time.

[0131] S202A. Determine the light energy cycle 1 currently corresponding to the first camera and the light energy cycle 2 currently corresponding to the second camera; set the exposure time of the first camera to an integer multiple of the light energy cycle 2, and capture the third image; set the exposure time of the second camera to an integer multiple of the light energy cycle 1, and capture the fourth image.

[0132] After detecting zoom operation 1, electronic device 100 can switch the main camera used for display from the first camera to the second camera. In this embodiment of the application, after switching the cameras, the image captured by the first camera is called the third image; the image captured by the second camera is called the fourth image.

[0133] It is understood that in step S202A, the light energy cycles currently corresponding to the first and second cameras are swapped. Regarding how to determine the light energy cycles currently corresponding to the first and second cameras, and how to determine the integer multiples of the light energy cycles of each camera when swapping the light energy cycles, please refer to the relevant description of step S106 and will not be repeated here.

[0134] In some embodiments, the exposure time of the second camera is set to the first exposure time in step S202A. In this way, after the primary camera is switched to the second camera, the difference between the exposure time of the second camera and the first exposure time of the first camera is reduced, thereby avoiding a large difference in brightness between the fourth image displayed by the second camera and the first image displayed by the first camera, which would result in a poor user experience.

[0135] In one usage scenario, there is a flickering light source in the shooting environment. When the electronic device detects that the user performs a zoom operation 1, the electronic device 100 has performed Figure 4 The detection method shown in the figure, the current exposure time of the first camera matches the flickering light source in the environment, so that the image sent by the first camera will not produce water ripple characteristics. At this time, after the user switches the camera sent to the second camera through the zoom operation 1, the image collected by the second camera will contain water ripple characteristics. Therefore, when executing Figure 6 In the process shown, the light energy cycles corresponding to the first camera and the second camera are swapped first, which can ensure with a greater probability that the first frame fourth image sent by the second camera does not have water ripple features.

[0136] S202B, setting the exposure time of the first camera to an integer multiple of the first light energy period to capture the third image; setting the exposure time of the second camera to an integer multiple of the second light energy period to capture the fourth image.

[0137] When zoom operation 1 is detected, if the exposure time of the first camera is not constrained to be an integer multiple of the light energy period, the exposure time of the first camera can be reset to an integer multiple of the first light energy period, and the exposure time of the second camera can be reset to an integer multiple of the second light energy period.

[0138] In some embodiments, the specific configuration of step S202B can be referred to the description of steps S101 and S102. In some embodiments, a value that is an integer multiple of the first light energy period closest to the second exposure time of the second camera is determined, such as N2, and then the exposure time of the first camera is set to N2 times the first light energy period. In some embodiments, a value that is an integer multiple of the second light energy period closest to the first exposure time of the first camera is determined, such as M2, and then the exposure time of the first camera is set to M2 times the second light energy period.

[0139] It can be understood that in step S202B, the exposure time of the first camera can also be set to an integer multiple of the second light energy period, and the exposure time of the second camera can also be set to an integer multiple of the first light energy period. This embodiment of the present application does not specifically limit this.

[0140] After executing S202A or S202B, the electronic device 100 continues to execute steps S203 to S209 of the above method (i.e., performing flicker detection and elimination on the third and fourth images captured by the two cameras). The specific implementation of steps S203 to S209 can be referenced with the description of steps S103 to S109 (i.e., performing flicker detection and elimination on the first and second images captured by the two cameras), and will not be further described.

[0141] S203 : Perform flicker detection on the third image and the fourth image respectively.

[0142] S204: Whether there are water ripples in the fourth image.

[0143] S205: Whether there are water ripples in the third image.

[0144] S206: When the fourth image has water ripple features, set the exposure time of the first camera and the second camera to swap the light energy cycles currently corresponding to the two cameras.

[0145] If S202A is executed after step S201, the current light energy cycle of the second camera in step S206 is light energy cycle 1, and step S206 includes: "setting the exposure time of the second camera to an integer multiple of light energy cycle 1; setting the exposure time of the first camera to an integer multiple of light energy cycle 2".

[0146] If S202B is executed after step S201, the current light energy cycle of the second camera in step S206 is the second light energy cycle, and step S206 includes: "setting the exposure time of the first camera to an integer multiple of the second light energy cycle, and setting the exposure time of the second camera to an integer multiple of the first light energy cycle."

[0147] S207: When the fourth image does not have the water ripple feature and the third image has the water ripple feature, maintain the current light energy cycle.

[0148] S208 : When neither the third image nor the fourth image has water ripple features, adjust the exposure time of the first camera according to the second exposure strategy.

[0149] S209 , after resetting the camera exposure time in step S206 , or when maintaining the current light energy cycle in step S207 , the electronic device 100 may further adjust the exposure time of the first camera according to the first exposure strategy.

[0150] S210: Send and display the fourth image.

[0151] As will be appreciated, in the above method, after the zoom magnification is adjusted from the zoom range of the first camera to the zoom range of the second camera, the image captured by the second camera is displayed and used as a preview image displayed on the display screen by the camera application. In some embodiments, for smooth zooming, the fourth image captured by the second camera is not displayed until step S210. That is, after detecting the user's zoom operation 1, the first image captured by the first camera continues to be displayed until the electronic device 100 completes steps S201-S209 (i.e., completing the initial water ripple detection and water ripple elimination after zoom operation 1). This ensures that the image displayed by the electronic device 100 does not exhibit water ripple characteristics before and after switching the displayed camera during zooming, thereby achieving smooth zooming. In some embodiments, after detecting the user's zoom operation 1, the display immediately switches to the fourth image captured by the second camera; the execution speed of the electronic device 100 is fast enough to complete water ripple detection and elimination before displaying the fourth image captured by the second camera, thereby achieving smooth zooming.

[0152] Similarly, in the above method, S208 and S209 are optional. In some embodiments, step S210 can be executed directly after executing step S205, S207, or S206. In addition, the execution order of S205-S206 is not particularly limited. Preferably, the electronic device 100 executes S205 first and then executes S206. In step S203, flicker detection can be performed on the fourth image first. If it is determined that water ripples exist in the fourth image, flicker light source detection can be omitted for the third image. Figure 6 The flowchart shown only illustrates a possible process of the electronic device 100 executing the above method.

[0153] In some usage scenarios, after detecting the zoom operation 1, the main camera used for display is switched from the first camera to the second camera. Due to insufficient computing speed of the electronic device 100 or hardware and software jams, the display may not be completed within the gap between adjacent frames (i.e., from the time the first camera displays the last frame of the first image to the time the second camera displays the first frame of the fourth image). Figure 6 In view of this, the embodiment of the present application also proposes another flicker detection method in a smooth zoom scenario, which can enable the electronic device 100 to still complete flicker detection and water ripple elimination in the above situation.

[0154] Figure 7A A schematic flow chart of another method for detecting a flickering light source in a zoom scenario provided by an embodiment of the present application is shown.

[0155] S301: When a change in zoom ratio is detected, adjust the exposure time of the first camera.

[0156] In some embodiments, when the electronic device detects the above-mentioned change in zoom ratio, the electronic device 100 may adjust the exposure time of the first camera according to the zoom ratio.

[0157] In one implementation, if the current exposure time of the first camera is constrained to be an integer multiple of the light energy period, the electronic device may use a first exposure adjustment strategy and an AEC algorithm to adjust the exposure time of the first camera, i.e., adjust the value of the integer multiple of the light energy period. If the current exposure time of the first camera is not constrained to be an integer multiple of the light energy period, the electronic device may use a second exposure adjustment strategy and an AEC algorithm to adjust the exposure time of the first camera.

[0158] S302: When a change in zoom ratio is detected, adjust the exposure time of the second camera.

[0159] In some embodiments, the adjustment scheme for the exposure time of the second camera is the same as S301 and will not be repeated here.

[0160] Steps S301 and / or S302 are optional. In some embodiments, before the zoom ratio is adjusted to the transition zoom range, the exposure times of the first and second cameras do not need to be adjusted. The transition zoom range is also referred to as the first zoom range. In some embodiments, before the zoom ratio is adjusted to the transition zoom range, when S301 is executed to adjust the exposure time of the first camera, the exposure time of the second camera does not need to be adjusted.

[0161] S303: Detect whether the focal length reaches the transition zoom range of the first camera; if so, execute S304.

[0162] Exemplary, reference Figure 7B and Figure 7C As shown, the zoom range of the first camera includes a transition zoom range, which is adjacent to the zoom range of the second camera. The transition zoom range is the zoom range between a first focal length threshold and a second focal length threshold, where the second focal length threshold is either the minimum or maximum value within the zoom range of the second camera. The difference between the first focal length threshold and the second focal length threshold is a smaller preset value. This preset value is determined by the developer based on experience.

[0163] refer to Figure 7B , the focal length within the zoom range of the second camera is greater than the focal length within the zoom range of the first camera, the current zoom ratio belongs to the zoom range of the first camera, and the first camera is used for display; at this time, the user can increase the zoom ratio so that the focal length reaches the transition zoom range of the first camera; in step S303, it can continue to detect whether the focal length increases to the first focal length threshold. Figure 7C , the focal length within the zoom range of the second camera is smaller than the focal length within the zoom range of the first camera, the current zoom ratio belongs to the zoom range of the first camera, and the first camera is used for display; at this time, the user can reduce the zoom ratio so that the focal length reaches the transition zoom range of the first camera; in step S303, it can be continuously detected whether the focal length is reduced to the first focal length threshold.

[0164] The electronic device 100 continuously detects whether the zoom ratio reaches the transition zoom range of the first camera. When the zoom ratio reaches the transition zoom range, the zoom ratio is close to the zoom range of the second camera. At this time, the electronic device 100 performs flicker detection in advance and executes steps S304 to S307, so that the light energy cycle corresponding to the second camera matches the flickering light source in the environment in advance.

[0165] S304: constrain the exposure time of the second camera to be an integral multiple of the light energy period 3, and perform flicker detection on the fifth image captured by the first camera and the sixth image captured by the second camera.

[0166] In some embodiments, when the electronic device 100 detects that the zoom ratio has been adjusted from the non-transition zoom range of the first camera to the transition zoom range, if the exposure time of the first camera is constrained to be an integer multiple of the light energy cycle, the electronic device 100 may obtain the light energy cycle 3 currently corresponding to the first camera and set the exposure time of the second camera to an integer multiple of the light energy cycle 3. If the exposure time of the first camera is not constrained to be an integer multiple of the light energy cycle, the electronic device 100 may directly set the exposure time of the second camera to an integer multiple of the light energy cycle 3. The light energy cycle 3 may be any one of the first light energy cycle and the second light energy cycle. For how to determine the light energy cycle currently corresponding to the first camera and how to determine the value of the integer multiple when setting the exposure time of the second camera to an integer multiple of the light energy cycle 3, please refer to the relevant descriptions in step S106 and step S101, which will not be repeated here. The light energy cycle 3 is also referred to as the third light energy cycle.

[0167] For the convenience of description, when the zoom ratio is within the transition zoom range, the image captured by the first camera is referred to as the fifth image, and the image captured by the second camera is referred to as the sixth image.

[0168] S305 , determining whether the sixth image captured by the second camera has water ripples; if so, executing S306 ; otherwise, executing S311 .

[0169] In the embodiment of the present application, when water ripples exist in the sixth image, the electronic device 100 may execute steps S306 and S307.

[0170] S306. When water ripples exist in the sixth image, the electronic device 100 sets the exposure time of the second camera to an integer multiple of the light energy period 4; constrains the exposure time of the first camera to an integer multiple of the light energy period 4; and the light energy period 4 is different from the light energy period 3.

[0171] The above-mentioned light energy cycle 4 is also called the fourth light energy cycle.

[0172] Among them, when light energy period 3 is the first light energy period, light energy period 4 is the second light energy period; or, when light energy period 4 is the second light energy period, light energy period 3 is the first light energy period.

[0173] When water ripples appear in the sixth image, there is a flickering light source in the environment, which does not match the light energy cycle 3 currently corresponding to the second camera, that is, it matches the light energy cycle 4; the exposure time of the second camera is set to an integer multiple of the light energy cycle 4, so that when the zoom magnification is within the transition zoom range, the light energy cycle corresponding to the second camera matches the flickering light source in the environment in advance.

[0174] It should be noted that if it is determined in step S304 that the exposure time of the first camera is currently constrained to an integer multiple of the light energy cycle 4, step S306 continues to maintain the exposure time of the first camera as an integer multiple of the light energy cycle 4; if it is determined in step S304 that the exposure time of the first camera is currently not constrained to an integer multiple of the light energy cycle, step S306 sets the exposure time of the first camera to an integer multiple of the light energy cycle 4, so that the exposure time of the first camera currently displayed matches the flickering light source in the environment, thereby eliminating water ripples in the image currently displayed by the first camera.

[0175] S307: Determine whether the zoom ratio is adjusted to the zoom range of the second camera; if so, execute S308 to S310.

[0176] S308: Switch the main camera for display to the second camera, and display the fourth image captured by the second camera.

[0177] S309: Set the exposure time of the first camera to an integer multiple of the light energy period 3.

[0178] With reference to the relevant description of step S202A, for the convenience of description, after the cameras are switched, the image captured by the first camera is referred to as the third image; and the image captured by the second camera is referred to as the fourth image.

[0179] From step S306, it can be seen that there is a flickering light source in the current environment, and the light energy cycle corresponding to the flickering light source is light energy cycle 4. When the zoom ratio is adjusted to the zoom range of the second camera, the main camera for display is switched from the first camera to the second camera; at this time, the second camera is used for display and flicker detection, while the first camera is only used for flicker detection. In some embodiments, the exposure time of the second camera has been constrained in advance in step S306 to be an integer multiple of light energy cycle 4. In order to continue to regularly complete flicker detection (i.e., perform flicker detection), the exposure time of the second camera is set to 4. Figure 4 The flickering light source detection method shown in FIG2 needs to constrain the exposure time of the first camera to be an integer multiple of another light energy cycle (ie, light energy cycle 3).

[0180] In an embodiment of the present application, before switching the main camera for display to the second camera, when the zoom ratio is within the transition zoom range, the exposure time of the second camera is matched in advance with the flickering light source in the environment, so that the first image displayed by the second camera after switching the camera will not have water ripple features, thereby ensuring a smooth zoom user experience.

[0181] Step S309 is optional. In some embodiments, it is not necessary to execute S309, and the steps are executed periodically. Figure 4 In the flickering light source detection method shown, in step S101 , the exposure time of the first camera is reset.

[0182] The embodiment of the present application does not specifically limit the execution order of steps S308 and S309.

[0183] S310: Adjust the exposure time of the second camera currently sending the display according to the first exposure strategy.

[0184] For details, please refer to the relevant description of step S109, which will not be repeated here.

[0185] S311. When there are no water ripples in the sixth image, the electronic device 100 determines whether there are water ripples in the fifth image captured by the first camera; if so, execute S312; otherwise, execute S317.

[0186] S312: Set the exposure time of the first camera to an integer multiple of the light energy period 3.

[0187] Referring to the relevant contents of step S304, it can be seen that when adjusting the zoom magnification to the transition zoom range of the first camera, the first camera may or may not be constrained to an integer multiple of the light energy period 3. In step S311, the exposure time of the second camera is an integer multiple of the light energy period 3. If the sixth image captured by the second camera does not include water ripples, and the fifth image captured by the first camera does include water ripples, it indicates that there is a flickering light source in the environment, and the light energy period corresponding to the flickering light source is light energy period 3. In step S312, the exposure time of the first camera is set to an integer multiple of the light energy period 3 to ensure that the exposure time of the first camera currently being displayed matches the flickering light source in the environment, thereby eliminating water ripples in the displayed image.

[0188] S313: Determine whether the zoom ratio is adjusted to the zoom range of the second camera; if so, execute S314 to S316.

[0189] S314: Switch the main camera for display to the second camera, and display the fourth image captured by the second camera.

[0190] S315: Set the exposure time of the first camera to an integer multiple of the light energy period 4.

[0191] As can be seen from step S312, there is a flickering light source in the current environment, and the light energy cycle corresponding to the flickering light source is light energy cycle 3. When the zoom ratio is adjusted to the zoom range of the second camera, the main camera for display is switched from the first camera to the second camera; at this time, the second camera is used for display and flicker detection, while the first camera is only used for flicker detection; the exposure time of the second camera has been constrained in advance in step S304 to be an integer multiple of light energy cycle 3, in order to continue to regularly complete flicker detection and water ripple elimination (i.e., perform Figure 4 The flickering light source detection method shown in FIG2 needs to constrain the exposure time of the first camera to be an integer multiple of another light energy cycle (ie, light energy cycle 4).

[0192] Step S315 is optional. In some embodiments, it is not necessary to execute S315, and the steps are executed periodically. Figure 4 In the flickering light source detection method shown, in step S101 , the exposure time of the first camera is reset.

[0193] The embodiment of the present application does not specifically limit the execution order of steps S314 and S316.

[0194] S316: Adjust the exposure time of the second camera currently sending the display according to the first exposure strategy.

[0195] For details, please refer to the relevant description of step S109, which will not be repeated here.

[0196] S317: When there are no water ripples in the fifth image, the electronic device 100 determines whether the zoom ratio is adjusted to the zoom range of the second camera; if so, execute S318 to S319.

[0197] In step S317, the exposure time of the second camera is an integer multiple of the light energy period 3, and the first camera is constrained to be an integer multiple of the light energy period 3, or is not constrained to be an integer multiple of the light energy period; at this time, if the sixth image captured by the second camera does not include water ripples, the fifth image captured by the first camera also does not include water ripples, indicating that there is no flickering light source in the environment.

[0198] S318: Switch the main camera for display to the second camera, and display the fourth image captured by the second camera.

[0199] Furthermore, the electronic device may also fine-tune the exposure time of the second camera. The method further includes:

[0200] S319: The electronic device 100 may adjust the exposure time of the second camera according to the second exposure strategy.

[0201] For details, please refer to the relevant description of step S108, which will not be repeated here.

[0202] In some embodiments, in the above method, S310, S316, and S319 are optional, and the electronic device 100 not performing the above steps does not affect the implementation of the smooth zoom solution. Figure 7A The flowchart shown only illustrates a possible process of the electronic device 100 executing the above method.

[0203] In the above method, the first camera and the second camera are not fixed.

[0204] When a user first opens the camera app, the electronic device may default to normal photo mode with a zoom ratio of 1x. In this case, the first camera may be the wide-angle camera, used for display and flicker detection; the second camera can be selected from the remaining cameras and used only for flicker detection. However, when the shooting mode or zoom ratio changes, the main camera used for display can switch from the wide-angle camera to another camera.

[0205] For example, during photo preview, video preview or video recording, if it is detected that the user increases the zoom ratio to the zoom range of the telephoto camera, the electronic device may switch the main camera from the wide-angle camera to the telephoto camera, and the telephoto camera is used for display and flicker detection. For another example, during photo preview, video preview or video recording, if it is detected that the user reduces the zoom ratio to the zoom range of the ultra-wide-angle camera, the electronic device may switch the main camera from the wide camera to the ultra-wide-angle camera, and the ultra-wide-angle camera is used for display and flicker detection. For another example, if it is detected that the photo mode is switched from normal mode to macro shooting mode, the electronic device may switch the main camera from the main camera to the macro camera, and the macro camera is used for display and flicker detection. For another example, the electronic device detects the user operation of flipping the camera, switching the first camera from the rear main camera to the front main camera, or switching the main camera from the front main camera to the rear main camera, and the rear main camera is used for display and flicker detection.

[0206] The original first camera can be switched to a secondary camera used only for flicker detection, especially when the electronic device only has two cameras. This eliminates the need to activate a new camera, resulting in faster response. Of course, in actual applications, other cameras with lower power consumption can also be considered for switching to the secondary camera based on power consumption and other issues. This embodiment of the present application does not limit this.

[0207] Preferably, the first camera is a wide-angle camera, and the second camera is an ultra-wide-angle camera.

[0208] Next, the software and hardware architecture of the electronic device provided in the embodiments of the present application is introduced.

[0209] Electronic equipment can be equipped or portable terminal devices with other operating systems, such as mobile phones, tablet computers, desktop computers, laptop computers, handheld computers, notebook computers, ultra-mobile personal computers (UMPCs), netbooks, as well as cellular phones, personal digital assistants (PDAs), augmented reality (AR) devices, virtual reality (VR) devices, artificial intelligence (AI) devices, wearable devices, in-vehicle devices, smart home devices and / or smart city devices, etc.

[0210] Figure 8The electronic device 100 provided by the embodiment of the present application is exemplarily shown. The electronic device 100 can detect whether there is a flickering light source in the shooting environment, does not require an additional dedicated flicker detection device, and can also prevent the user from seeing the generation of water ripples.

[0211] like Figure 8 As shown, the electronic device 100 may include: a processor 110 , a memory 120 , a camera 130 , and a display screen 140 .

[0212] in:

[0213] The processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors. The processor 110 may include one or more interfaces, such as an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface. These interfaces are used for data exchange between the processor 110 and peripheral devices.

[0214] The memory 120 may include one or more random access memories (RAMs) and one or more non-volatile memories (NVMs). The RAM can be directly read and written by the processor 110 and can be used to store executable programs (e.g., machine instructions) for the operating system or other running programs, as well as user and application data. The NVM can also store executable programs and user and application data, and can be pre-loaded into the RAM for direct reading and writing by the processor 110. The processor 110 may also be provided with a storage unit, which may be a cache memory unit, for storing instructions or data that has just been used or is recycled by the processor 110. The implementation code of the flickering light source detection method provided in the embodiments of the present application may be stored in the NVM. When the camera application is launched, this code may be loaded into the RAM. In this way, the processor 110 can directly read the program code from the RAM to implement the flickering light source detection method provided in the embodiments of the present application. In addition, image files such as photos and videos captured by the user using the camera application can be written to the NVM for storage and viewing by the user.

[0215] The camera 130 may include a lens, a photosensor, and a flexible printed circuit board (FPCB). The FPCB is responsible for connecting the other components of the camera 130 to the processor 110, for example, transmitting the raw data output by the photosensor to the processor 110. When taking a photo, the shutter of the camera 130 is opened, and light enters and shines on the photosensor. The photosensor converts the light signal into an electrical signal, which is then further converted into a digital signal through analog-to-digital conversion (ADC) and passed to the ISP for processing. The ISP can perform the following processing on the output data of the photosensor: automatic exposure control (AEC), automatic gain control (AGC), automatic white balance (AWB), color correction, bad pixel removal, etc. The ISP can also be integrated into the camera 130.

[0216] The display screen 140 can be used to display the image captured by the camera 130. The image processed by the ISP will be sent to the display screen 140 to show the user a preview of the image captured by the camera. Here, sending to the display screen means pushing the image captured by the camera to the frame buffer (FB) for storage. The frame buffer is a storage space that can be located in the video memory or the internal memory, and is used to store the rendering data processed by the graphics card chip or to be extracted. The content of the frame buffer corresponds to the interface display on the display screen 140. It can be simply understood as a cache corresponding to the content displayed on the display screen 140. Modifying the content in the frame buffer is to modify the content on the display screen 140.

[0217] The electronic device 100 includes multiple cameras 130, such as a rear main camera, a front main camera, a telephoto camera, a wide-angle camera, a depth-of-field camera, a macro camera, and the like. These multiple cameras 130 may include at least two cameras 130 (e.g., a first camera and a second camera) that can be used for preview display and flicker detection. When the flicker light source detection method provided in the embodiments of the present application begins executing, the image captured by the first camera is displayed and used as a preview image displayed by the camera application on the display screen 140. The image captured by the second camera is not displayed and is used only for flicker detection, thus being invisible to the user. During zooming, when the electronic device 100 detects that the zoom ratio reaches the zoom range of the second camera, the image captured by the second camera is displayed and used as a preview image displayed by the camera application on the display screen 140. The image captured by the second camera is not displayed and is used only for flicker detection. During the execution of the flicker light source detection method, the image captured by the camera that is displayed is also used for flicker detection. The shutters of both the first and second cameras are rolling shutters.

[0218] When the electronic device needs to perform flicker detection, for example, when the user opens the camera application to trigger the first camera to turn on, or when the user performs a zoom operation, the first camera can be set to an integer multiple of a certain light energy cycle, and the second camera can be set to an integer multiple of another light energy cycle, so that as long as there is a flickering light source in the shooting environment, the image captured by the first camera or the image captured by the second camera will inevitably contain water ripple features, and the water ripple features can be detected by image recognition technology. Based on the flicker detection results, the electronic device 100 can effectively avoid the problem of water ripples in the image of the first camera by setting the exposure time of the first camera for preview display. When avoiding the problem of water ripples in the image of the first camera, the electronic device 100 can adaptively adjust the exposure time of the second camera so that the light energy cycles corresponding to the first camera and the second camera are different. After adjusting the exposure time, the first camera and the second camera can detect at one time whether there is a 50Hz flickering light source, a 60Hz flickering light source, or no flickering light source in the shooting environment.

[0219] The first camera and the second camera are not specially introduced for flicker detection. Usually, they can be ultra-wide-angle cameras, wide-angle cameras or macro cameras, etc. The embodiments of the present application give them new functions.

[0220] Taking the first camera for preview display as an example, in addition to the flicker detection results provided by the first and second cameras, the exposure time of the main camera can also be controlled by the automatic exposure control (AEC) function. In short, AEC automatically adjusts the exposure time according to the light intensity. In different environments, the intensity of light varies greatly, and the exposure time of the photosensor in the camera 130 also needs to adapt accordingly to ensure normal exposure of the image. The automatic exposure control (AEC) provided by the ISP provides this adaptive capability. Therefore, the exposure time of the photosensor can reflect the brightness of the shooting environment to a certain extent.

[0221] In addition, the electronic device 100 may also store an exposure table to be used by the AEC. A camera's exposure table specifies the exposure parameters (such as exposure time, exposure gain, and aperture size) to be used by the camera at different exposure values (EV) and can be debugged and recorded by engineers.

[0222] In addition, the exposure time of the main camera can also be controlled by the motion detection algorithm. Taking the main camera as the first camera as an example, the motion detection algorithm can identify whether there is a motion blur feature in the first image captured by the first camera. If so, the movement speed of the moving object can be further determined based on the above-mentioned motion blur feature, and then the exposure time of the first camera can be adjusted. For objects with different movement speeds, the corresponding clear imaging time is also different, and the exposure time of the photosensitive sensor in the camera 130 also needs to be adapted accordingly to ensure clear image imaging. The motion detection algorithm can be provided by the processor 110. Therefore, if the final image is clear, the exposure time of the photosensitive sensor can reflect the movement speed of the object in the image to a certain extent.

[0223] The electronic device 100 may also store a correspondence between the speed intervals of the moving object and the camera exposure time required for the motion detection algorithm. This correspondence may be obtained by simulation or actual measurement by engineers.

[0224] like Figure 8 As shown, the electronic device 100 may further include: an audio module 150, a speaker 150A, a receiver 150B, a microphone 150C, and an earphone interface 150D.

[0225] Among them, the electronic device can implement audio functions through the audio module 150, speaker 150A, receiver 150B, microphone 150C, headphone jack 150D, and application processor (AP). For example, music playback, recording, etc. The audio module 150 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 150 can also be used to encode and decode audio signals. In some embodiments, the audio module 150 can be set in the processor 110, or some functional modules of the audio module 150 can be set in the processor 110. The speaker 150A, also known as the "speaker", is used to convert audio electrical signals into sound signals. The electronic device can listen to music or listen to hands-free calls through the speaker 150A. The receiver 150B, also known as the "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device answers a call or voice message, the voice can be heard by placing the receiver 150B close to the human ear. Microphone 150C, also known as "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 150C to input the sound signal into the microphone 150C. The electronic device can be provided with at least one microphone 150C. In other embodiments, the electronic device can be provided with two microphones 150C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device can also be provided with three, four or more microphones 150C to realize sound signal collection, noise reduction, sound source identification, directional recording function, etc. The headphone jack 150D is used to connect wired headphones. The headphone jack 150D can be a USB interface, or a 3.5mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0226] like Figure 8 As shown, the electronic device 100 may further include a sensor module 160 , which may specifically include a pressure sensor 160A, a distance sensor 160F, a proximity light sensor 160G, a touch sensor 160K, an ambient light sensor 160L, and the like.

[0227] Figure 8 The illustrated structure does not constitute a specific limitation on the electronic device. The electronic device may include more or fewer components than shown, or some components may be combined or separated, or arranged differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0228] Figure 9 The main software architecture involved in the flickering light source detection method provided by the embodiment of the present application on the electronic device 100 is shown.

[0229] like Figure 9 As shown in the figure, the software architecture may include the following functions or modules: camera application (camera apk), camera service (camera service), camera hardware abstraction layer (camera HAL), flicker detection (CameraFlickerDetector), camera vendor native development kit (camera VNDK) and kernel part.

[0230] The camera app (camera APK) can be located in the application layer. Besides the camera app, the application layer can include a variety of other apps, such as the gallery app. The camera app is primarily responsible for user interaction, such as listening for and responding to user actions like selecting a shooting mode, adjusting the zoom factor, and selecting a focus point.

[0231] The camera service can reside in the application framework layer. It includes interfaces for camera management and camera devices, serving as a link between the upper and lower layers. It can interact with camera applications through the application programming interface (API) and with the camera HAL through the HAL interface definition language (HIDL). Furthermore, the camera service can interact with the vendor native development kit (VNDK).

[0232] The camera hardware abstraction layer (HAL) represents the virtualized hardware functionality related to the camera within the hardware abstraction layer (HAL). The HAL, located as an interface between the application framework and driver layers, provides a virtual hardware platform for the operating system, freeing upper layers from the underlying hardware implementation. The camera HAL communicates with the camera service via a HIDL interface and issues camera control operations via standard HAL interfaces. The camera HAL includes a camera entry (CamEntry) module and an automatic exposure control processor (AECProcessor) module. The CamEntry module serves as the entry point to the HAL, receiving commands such as open, close, stream configuration, and request from the camera service. It is a universal entry point for camera HAL functionality. The AECProcessor module provides ISP-related AEC functions, such as providing AEC parameters (e.g., exposure time) for the first and second cameras to the CameraFlickerDetector, enabling it to determine whether flicker detection is required in the current shooting environment.

[0233] The flicker detection (CameraFlickerDetector) module can be located in the application framework layer (framework) or the application layer. The CameraFlickerDetector module can obtain the AEC parameters (such as exposure time) of the first camera and the second camera from the camera HAL through the HIDL interface, so as to decide whether to start the first camera and the second camera for flicker detection. When it is determined that flicker detection is to be performed, the CameraFlickerDetector module can issue the control operation of starting the second camera and setting the exposure time of the first camera and the second camera through the interface of cameraVNDK. The CameraFlickerDetector module may include a CameraHelper module and an ImageReader module, wherein the CameraHelper module can be used to implement the call of functions such as opening (open), closing (close), stream configuration (configurestream), and request of the camera service of the second camera through the call of the VNDK interface, and the ImageReader module is mainly used to receive and further process the images sent by the first camera and the second camera when executing the flicker detection method.

[0234] The Vendor Native Development Kit (VNDK) is a collection of libraries that allows vendors to develop their own HALs (vendors). The camera VNDK is the "own camera HAL" developed by the vendor of the second camera. The camera VNDK can be used to receive the control operation issued by the CameraFlickerDetector to start the second camera and forward the control operation to the cameraservice. The camera VNDK can include the ACameraManager module, the ACameraDevice module, the ACameraReader module, the ACaptureSession module, and the ACaptureRequest module. Among them, the ACameraManager module provides the function of accessing the camera service from the vendor layer, the ACameraDevice module provides the function of accessing the camera device from the vendor layer, the ACameraReader module is responsible for reading image data, such as data captured by the first camera or the second camera, the ACaptureSession module provides the function of allowing the vendor to manage the frames captured by the camera, and the ACaptureRequest module is responsible for the settings and output targets required for the camera to capture a single image.

[0235] The kernel part may include various hardware driver control modules, such as ISP, photosensor, DSP, AP and other driver control.

[0236] Figure 9 Only the main software architecture involved in the flickering light source detection method provided in the embodiment of the present application is shown, which is essentially only a part of the software architecture of the electronic device 100.

[0237] Figure 9 The implementation process of the flickering light source detection method is also shown, including:

[0238] 1. First camera startup process

[0239] The camera app calls the camera service, which authenticates the camera app. Authentication confirms that the camera app has permission to access the hardware camera, such as turning the camera on and off. This prevents unauthorized apps from turning the camera on and off without user authorization, effectively protecting user privacy and preventing conflicts between the camera app and other apps accessing the camera (such as video call apps). After authentication, the camera service calls the camera HAL to activate the primary camera, control the photosensor, and image output from the ISP. Image frames are displayed for functions like image preview and photo capture.

[0240] On this basis, to perform flicker detection without the user noticing, the electronic device 100 can activate the second camera after activating the first camera to perform detection together. After the second camera is activated, the image frame is not sent for display. Neither the first camera nor the second camera is specifically introduced for flicker detection. The first camera can be a wide-angle camera, and the second camera can be an ultra-wide-angle camera. The embodiments of the present application give them new functions.

[0241] 2. Flicker detection process

[0242] First, when preview is enabled for the first camera, the AECProcessor module in the camera HAL transmits parameters such as the first camera's exposure time to the CameraFlickerDetector module via a HIDL interface. The exposure time set by the AEC reflects the brightness of the shooting environment. Therefore, the CameraFlickerDetector module can determine whether to perform flicker detection based on the exposure time. Flicker detection is typically performed indoors. Flicker detection is generally not necessary in shooting environments that are too bright or too dark. In bright environments, overexposure is the primary concern, requiring a significant reduction in exposure time, making it difficult to minimize water ripples. In dark environments, there's generally no need to reduce the exposure time to below the light energy cycle, as this can easily lead to underexposure. Here, "too bright" or "too dark" can be defined with reference to the exposure table on electronic device 100. For example, the shortest exposure times corresponding to high EV in the exposure table can be determined as exposure times for too bright environments. If the actual exposure time falls within these shortest exposure times, the environment is considered too bright. For another example, the longest exposure times corresponding to low EV in the exposure table can be determined as exposure times for too dark environments. Once the actual exposure time falls within these longest exposure times, it is considered that the environment is too dark.

[0243] Then, after determining that flicker detection is to be performed, the CameraFlickerDetector module transmits a control command to start the second camera to cameraVNDK.

[0244] The camera VNDK then passes the CameraFlickerDetector module to the camera service to start controlling the second camera.

[0245] After authenticating the CameraFlickerDetector module, the camera service calls the camera HAL to activate the second camera through the camera HAL. This allows the second camera to be activated from the camera HAL side without involving the application layer and, therefore, not perceived by the user. Simultaneously, the CameraFlickerDetector module controls the exposure of the first and second cameras, ensuring that the exposure time of the first camera is an integer multiple of the first light energy period, and the exposure time of the second camera is an integer multiple of the second light energy period. For example, the exposure time of the first camera is 20 milliseconds, and the exposure time of the second camera is 8.3 milliseconds (i.e., 1 / 120 second). As a result, images captured by the first camera under a 60Hz flickering light source will inevitably produce water ripples, and images captured by the second camera under a 50Hz flickering light source will inevitably produce water ripples. Thus, any flickering light source in the shooting environment can be detected. The CameraFlickerDetector module also calls the camera HAL to read the output images of the first and second cameras, detecting whether the image content has water ripple features using image recognition technology, and transmitting the detection results to the first and second cameras. The output image of the second camera is not displayed and is only used for flicker detection.

[0246] After receiving the flicker detection results, the first and second cameras can perform appropriate exposure operations. If the CameraFlickerDetector module detects the presence of moire features in the first image, the first camera's exposure time can be set to an integer multiple of the second light energy period, and the second camera's exposure time to an integer multiple of the first light energy period, thereby resolving the moire issue in images captured under flickering light sources.

[0247] Furthermore, or after the CameraFlickerDetector module detects the presence of moire features in the image content of the first image or the second image, the first camera can also adjust the exposure time of the first camera based on motion detection, while complying with the adjustment accuracy and minimum value of the first exposure strategy. For example, the exposure time of the first camera can be reduced to 10ms to avoid motion blur in the captured image. If the CameraFlickerDetector module detects that the image content of both the first image and the second image does not contain moire features, the exposure time of the first camera can be further reduced to less than the current corresponding light energy period, for example, 1ms, to address the issue of moire in the image under flickering light sources.

[0248] The above software architecture newly introduces the CameraFlickerDetector module and implements interaction between the CameraFlickerDetector module, the camera HAL, and the camera service through VNDK and HIDL. This does not involve modifications to the camera application or the architecture layers, and has good portability, scalability, and maintainability.

[0249] The following combination Figure 10 The flickering light source detection method of the embodiment of the present application is described in detail with reference to the internal structure of the electronic device 100 . Figure 10 The method is described only by taking the preview of a photo or video provided by a camera application as an example, so that those skilled in the art can have a deeper understanding of the embodiments of the present application, and its details should not constitute a limitation on the scope of protection of the present application. The following is an example of an electronic device 100 having a wide-angle camera and an ultra-wide-angle camera.

[0250] Normal photo mode

[0251] S11. Upon detecting a user operation of opening a camera application, the application processor (AP) may activate the wide-angle camera.

[0252] For example, taking a wide-angle camera as the first camera, the photographing mode may be a normal photographing mode, and the exposure time of the wide-angle camera may be set by the AEC of the ISP.

[0253] Prior to the S11, the display screen may display a system desktop, which includes a desktop icon for a camera application. The user operation of opening the camera application may refer to the user clicking the desktop icon for the camera application.

[0254] S12. The wide-angle camera transmits the captured raw image frame (RAW) to the ISP.

[0255] The ISP can perform the following processing on the RAW image output by the wide-angle camera: automatic exposure control (AEC), automatic gain control (AGC), automatic white balance (AWB), color correction, bad pixel removal, etc. The image processed by the ISP can be a color image, such as a YUV image.

[0256] S13, ISP can transmit the YUV image frame output by the wide-angle camera to the display screen.

[0257] Here, S13 only illustrates the flow path of image data and does not mean that the ISP directly sends data to the display. Its implementation includes the display transmission process. The YUV image frames of the wide-angle camera can be first cached in the frame buffer. When the GPU renders the preview image, it reads the YUV image frames of the wide-angle camera from the frame buffer and displays them on the display. The understanding of all the following YUV format image transmission to the display is the same.

[0258] S14. The display screen displays the YUV image frame of the wide-angle camera.

[0259] From then on, the wide-angle camera began to provide preview images to enable the preview function in normal photo mode.

[0260] In normal shooting mode, if flickering light is present in the shooting environment, ripples may appear in the preview image, as the wide-angle camera's exposure time may not be an integer multiple of the corresponding light energy cycle of the flickering light. Therefore, it is essential to detect and process flickering light sources as quickly as possible. The embodiments of this application focus on addressing the problem of how to quickly detect flickering light information in these shooting modes and adjust the exposure time in a timely manner to avoid the appearance of ripples.

[0261] S15. The AP can obtain the exposure time set by the AEC for the wide-angle camera from the ISP. The exposure time set by the AEC can reflect the brightness of the shooting environment.

[0262] S16. The AP may determine whether flicker detection is required based on the exposure time set by the AEC for the wide-angle camera.

[0263] Flicker detection is typically performed indoors. Flicker detection is generally not necessary in shooting environments that are too bright or too dark. This is because, in overly bright environments, the primary concern is overexposure, requiring a significant reduction in exposure time, making it difficult to address moiré. In overly dark environments, there's generally no need to reduce exposure time to below the light energy period, as this can easily lead to underexposure. Here, "too bright" or "too dark" can be defined with reference to the exposure table on electronic device 100. For example, the shortest exposure times corresponding to high EV in the exposure table can be used as exposure times for overly bright environments. If the AEC exposure time falls within these shortest exposure times, the environment is considered too bright. Another example is the longest exposure times corresponding to low EV in the exposure table can be used as exposure times for overly dark environments. If the AEC exposure time falls within these longest exposure times, the environment is considered too dark. In other words, the reference exposure values for overly bright or overly dark can be derived from the exposure table. Therefore, if the AEC exposure time falls within the remaining exposure times in the exposure table, it indicates that the environment is neither too bright nor too dark. The electronic device is not limited to analyzing the ambient brightness by the exposure time set by the AEC for the first camera. The electronic device can also collect ambient light through the ambient light brightness sensor to determine whether the shooting environment is too bright or too dark. This application does not impose any restrictions on its technical implementation.

[0264] Flicker detection and water ripple elimination

[0265] S17. After determining to perform flicker detection, the application processor (AP) may set the exposure time of the wide-angle camera to N1 times the first light energy period, such as 10 milliseconds.

[0266] Since the exposure time of the first camera is an integer multiple of the first light energy period, the image captured by the first camera under a flickering light source with an operating frequency other than 50 Hz will inevitably produce water ripples. In other words, in this case, a flickering light source with a frequency of 60 Hz can be detected.

[0267] After starting the first camera, the AP may also directly set the exposure time of the wide-angle camera without determining whether to perform flicker detection, that is, S15-S16 may not be executed.

[0268] S18. After being activated, the wide-angle camera may transmit the captured first image in a raw image frame (RAW) format to the ISP.

[0269] The ISP may perform a series of processing on the RAW image of the first image and output a YUV image frame.

[0270] S19. The AP may obtain a first image in a YUV format from the ISP.

[0271] S20. The AP may perform image recognition on the first image in YUV format to detect whether the image content contains water ripple features to obtain a flicker detection result. If the image content contains water ripple features, it is determined that a 60 Hz flickering light source is present in the shooting environment. Otherwise, it is determined that no flickering light source is present in the shooting environment, or that a 50 Hz flickering light source is present.

[0272] Among them, AP can run Figure 9 The software architecture shown, in particular the CameraFlickerDetector module, is used to implement flicker detection for the first image. Flicker detection can continue to be performed over time, in particular while the first camera is turned on.

[0273] S21. After determining to perform flicker detection, the application processor (AP) can start the ultra-wide-angle camera as the second camera and set the exposure time of the ultra-wide-angle camera to M1 times the second light energy period, such as 8.3 milliseconds (1 / 120 second).

[0274] Since the exposure time of the second camera is an integer multiple of the second light energy period, the image captured by the second camera under a flickering light source other than 60Hz will inevitably produce water ripples. In other words, in this case, a 50Hz flickering light source can be detected.

[0275] After starting the first camera, the AP may also directly start the ultra-wide-angle camera without determining whether to perform flicker detection, that is, S15-S16 may not be executed.

[0276] S22. After the ultra-wide-angle camera is started, the captured second image in the raw image frame (RAW) format may be transmitted to the ISP.

[0277] The ISP can perform a series of processing on the RAW image of the second image and output a YUV image frame. Among them, the YUV image frame of the second camera is not sent to the display, that is, the second image in YUV format is not transmitted to the display screen for display.

[0278] S23. The AP may obtain a second image in a YUV format from the ISP.

[0279] S24. The AP may perform image recognition on the second image in YUV format to detect whether the image content contains water ripple features, thereby obtaining a flicker detection result. If the image content contains water ripple features, it is determined that a 50 Hz flickering light source is present in the shooting environment. Otherwise, it is determined that no flickering light source is present in the shooting environment, or that a 60 Hz flickering light source is present.

[0280] Among them, AP can run Figure 9The software architecture shown, in particular the CameraFlickerDetector module, is used to implement flicker detection for the second image. Flicker detection can continue to be performed over time, especially while the first camera is turned on.

[0281] Among them, S17-S20 and S21-S24 can be executed in sequence, or S21-S24 can be executed first and then S17-S20. This application does not impose any special restrictions on the execution order here.

[0282] S25. The AP determines whether the first image has water ripple features.

[0283] If the AP determines that the first image has water ripple features, S26 is skipped and S27 to S31 are executed. If the AP determines that the first image does not have water ripple features, S26 is executed to further determine the flickering light source information in the shooting environment.

[0284] S26. The AP determines whether the second image has water ripple features.

[0285] If the AP determines that the first image does not have water ripple features but the second image has water ripple features, the current light energy cycle setting can be maintained and the subsequent steps of adjusting the exposure time of the first camera (S32-S37) can be continued.

[0286] S27. Set the exposure time of the wide-angle camera to M1 times the second light energy period.

[0287] Since the AP determines that the first image has water ripple characteristics, it means that the 60Hz flickering light source is present in the current shooting environment. Therefore, adjusting the exposure time of the first camera to an integer multiple of the second light energy period corresponding to 60Hz can eliminate the water ripple characteristics in the first image.

[0288] S28. After resetting the exposure time, the wide-angle camera may transmit the captured first image in a raw image frame (RAW) format to the ISP.

[0289] S29. The ISP may transmit the first image in a YUV format output by the wide-angle camera to the display screen.

[0290] S30. The display screen displays the first image in YUV format from the wide-angle camera.

[0291] From then on, the wide-angle camera began to provide preview images after removing water ripple features.

[0292] S31. Set the exposure time of the ultra-wide-angle camera to N1 times the first light energy period.

[0293] In step S27, the AP sets the exposure time of the wide-angle camera to M1 times the second light energy cycle, which not only eliminates the water ripple features in the first image captured by the wide-angle camera, but also enables the wide-angle camera to continue to detect whether there is a flickering light source other than 60Hz in the environment when the shooting environment changes.

[0294] In order to continuously detect the shooting environment of the electronic device 100, the exposure time of the ultra-wide-angle camera can be set to an integer multiple of another light energy cycle (i.e., the first light energy cycle), so that once there is a flickering light source in the environment, it will be detected by the first camera or the second camera.

[0295] S31 is not limited to being executed sequentially after S30, and may also occur after any step of S27-S29. This application does not impose any special limitation on the execution order of S31.

[0296] The exposure time of the first camera is not fixed after the flicker detection and water ripple elimination steps. The subsequent steps illustrate a possible solution for adjusting the exposure time of the first camera based on motion detection.

[0297] Motion detection

[0298] S32. The AP may obtain a first image in a YUV format from the ISP.

[0299] S33: The AP may perform image recognition on the first image in YUV format to detect whether the image content contains motion blur characteristics to obtain a motion detection result. If the image content contains motion blur characteristics, the exposure time is considered to be set too long, and an exposure strategy is executed to reduce the camera exposure time. Otherwise, the exposure time is considered to be set correctly and the current exposure time setting is maintained.

[0300] Thereafter, if the content of the first image contains motion blur features, the electronic device 100 can determine whether there is a flickering light source in the current shooting environment based on the aforementioned S25 and S26, and then execute the following steps S34A or S34B to reduce the exposure time of the camera.

[0301] S34A: When the content of the first image includes motion blur features, if the AP determines according to the aforementioned step that there is a flickering light source in the current shooting environment, the AP reduces the exposure time of the wide-angle camera according to the first exposure strategy.

[0302] The AP determines that there is a flickering light source in the current shooting environment, including: determining that there is a 60 Hz flickering light source in step S25; or determining that there is a 50 Hz flickering light source in step S26.

[0303] When the AP adjusts the exposure time of the wide-angle camera according to the first exposure strategy, the light energy period corresponding to the wide-angle camera at that time is used as the adjustment accuracy.

[0304] S34B: When the content of the first image contains motion blur features, if the AP determines according to the aforementioned step that there is no flickering light source in the current shooting environment, the AP reduces the exposure time of the wide-angle camera according to the second exposure strategy.

[0305] The AP determines that there is no flickering light source in the current shooting environment, specifically including: the AP does not detect the flickering light source in step S25 and step S26. Optionally, the AP can further analyze the motion blur characteristics in the first image to obtain motion blur information such as motion speed to determine the reduced exposure time.

[0306] S35 . After reducing the exposure time, the wide-angle camera may transmit the collected first image in a raw image frame (RAW) format to the ISP.

[0307] S36. The ISP may transmit the first image in the YUV format output by the wide-angle camera to the display screen.

[0308] S37. The display screen displays the first image in YUV format from the wide-angle camera.

[0309] From now on, the wide-angle camera will provide a preview image with motion blur removed.

[0310] During the execution of the above method, the camera used for display is always a wide-angle camera, but the camera used for display is not fixed. Figure 11 Shown Figure 7A The switching of the camera in the zoom scene is shown, as well as the detection of flickers and the elimination of water ripple features in the process.

[0311] Smooth zoom

[0312] In zoom scenarios, increasing the zoom ratio to the zoom range of the rear main camera or telephoto camera, or decreasing the zoom ratio to the zoom range of the ultra-wide-angle camera, changes the camera that provides the preview image. The following describes decreasing the zoom ratio to the zoom range of the ultra-wide-angle camera.

[0313] S38. The AP detects that the zoom ratio is reduced.

[0314] S39. The AP determines whether the zoom ratio is reduced to a first focal length threshold of a transition zoom range of the wide-angle camera.

[0315] S40: If the AP determines that the zoom ratio is reduced to the first focal length threshold, the exposure time of the ultra-wide-angle camera is set to an integer multiple of the light energy period 3.

[0316] S41. After setting the exposure time of the ultra-wide-angle camera to an integral multiple of the light energy period 3, the ultra-wide-angle camera may transmit the collected sixth image in the raw image frame (RAW) format to the ISP.

[0317] S42 . The AP may obtain a sixth image in a YUV format from the ISP.

[0318] S43 . The AP may perform image recognition on the sixth image in the YUV format to detect whether the image content contains water ripple features, so as to obtain a flicker detection result.

[0319] S44. After setting the exposure time of the ultra-wide-angle camera to an integral multiple of the light energy period 3, the wide-angle camera may transmit the fifth image in the raw image frame (RAW) format acquired to the ISP.

[0320] S45 . The AP may obtain a fifth image in a YUV format from the ISP.

[0321] S46 . The AP may perform image recognition on the fifth image in the YUV format to detect whether the image content contains water ripple features, so as to obtain a flicker detection result.

[0322] S47 , the AP determines whether the sixth image has water ripple features.

[0323] If the AP determines that the sixth image contains water ripple features, it indicates that the light energy cycle of the flickering light source in the current shooting environment does not match light energy cycle 3, and the AP then executes subsequent steps S48-S49. If the AP determines that the sixth image does not contain water ripples, the AP then jumps to step S50 to further determine the flickering light source information in the current shooting environment based on the image recognition results of the fifth image.

[0324] S48. If the AP determines that the sixth image has water ripple features, the exposure time of the ultra-wide-angle camera is set to an integer multiple of the light energy period 4.

[0325] For example, in the above example, if the AP determines that the sixth image has water ripple characteristics, it means that the light energy cycle of the flickering light source in the current shooting environment is not light energy cycle 3, and the light energy cycle of the flickering light source is light energy cycle 4. In this case, the AP can set the exposure time of the ultra-wide-angle camera to an integer multiple of light energy cycle 4 to avoid the generation of water ripple characteristics in the images captured by the ultra-wide-angle camera.

[0326] S49. If the AP determines that the sixth image has water ripple features, and if the exposure time of the wide-angle camera is not constrained to be an integer multiple of the light energy period, set the exposure time of the wide-angle camera to an integer multiple of the light energy period 4.

[0327] After executing S49, the electronic device can jump to execute S52. It is understandable that if the AP determines that the sixth image has water ripple features, the exposure time of the wide-angle camera has been constrained to an integer multiple of the light energy period, then after executing S48, it can directly jump to execute S52.

[0328] S50 , the AP determines whether the fifth image has water ripple features; if yes, execute S51 , otherwise execute S52 .

[0329] It can be understood that if the AP determines that there is a water ripple feature in the fifth image, then there is a flickering light source with a light energy period corresponding to light energy period 3 in the current shooting environment; if the AP determines that there is no water ripple feature in the fifth image, then combined with step S47, it can be determined that there is no flickering light source in the current shooting environment.

[0330] S51. If the AP determines that the fifth image has water ripple features, the exposure time of the wide-angle camera is set to an integer multiple of the light energy period 3.

[0331] S52. The AP determines whether the zoom ratio reaches the zoom range of the ultra-wide-angle camera.

[0332] S53: When the zoom range of the ultra-wide-angle camera is reached, the ultra-wide-angle camera may transmit the fourth image in the raw image frame (RAW) format captured to the ISP.

[0333] S54. The AP notifies the ISP to switch the display camera.

[0334] S55. The ISP may transmit the fourth image in the YUV format output by the ultra-wide-angle camera to the display screen.

[0335] S56. The display screen displays the fourth image in YUV format from the ultra-wide-angle camera.

[0336] Understandably, during step S48, due to the different light energy cycles set for the wide-angle and ultra-wide-angle cameras, and the presence of water ripples in the sixth image, the fifth image lacks these features. After step S54, the camera used for display is switched from the original wide-angle camera to the ultra-wide-angle camera, and the fourth image displayed by the ultra-wide-angle camera has had its water ripples removed after step S48. Therefore, even after the display camera is switched, the user does not experience any noticeable water ripples, achieving smooth zooming.

[0337] After switching the display camera, the exposure time of the wide-angle camera can be modified accordingly. If the AP determines that the sixth image has water ripple features before, S57A is executed; if the AP determines that the fifth image has water ripple features before, S57B is executed.

[0338] S57A: After the fourth image is sent for display, the AP sets the exposure time of the wide-angle camera to an integer multiple of the light energy period 3.

[0339] S57B: After the fourth image is sent for display, the AP sets the exposure time of the wide-angle camera to an integer multiple of the light energy cycle 4.

[0340] Furthermore, in the smooth zoom solution, after the water ripples are detected and eliminated, the exposure time of the display camera can be further adjusted, that is, the exposure time of the ultra-wide-angle camera can be further adjusted. The above method also includes: the AP can adjust the exposure time of the ultra-wide-angle camera according to the first exposure strategy or the second exposure strategy.

[0341] The above "smooth zoom" illustrates that the camera displayed can be changed according to the specific application scenario. Without limitation, the embodiments of the present application do not limit the application scenarios of the camera switching. For example, for an electronic device with only two cameras (front camera and rear camera), when the camera is flipped, the first camera switches to the front camera, and the second camera can be switched to the rear camera.

[0342] Figure 10 and Figure 11 In the embodiment, each camera is coupled to one ISP. However, the electronic device may be configured with multiple ISPs, and multiple cameras may be coupled to different ISPs.

[0343] The term "user interface (UI)" in the specification, claims and drawings of this application refers to the media interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface of an application is a source code written in a specific computer language such as Java and Extensible Markup Language (XML). The interface source code is parsed and rendered on the terminal device, and finally presented as content that the user can recognize, such as pictures, text, buttons and other controls. Controls, also known as widgets, are the basic elements of the user interface. Typical controls include toolbars, menu bars, text boxes, buttons, scroll bars, pictures and text. The properties and contents of controls in the interface are defined by tags or nodes, such as XML through <textview> 、 <imgview> 、 <videoview>The controls contained in the interface are specified by nodes such as <head> and <body>. A node corresponds to a control or attribute in the interface, and the node is presented as user-visible content after parsing and rendering. In addition, many applications, such as hybrid applications, usually also contain web pages in their interfaces. A web page, also known as a page, can be understood as a special control embedded in the application interface. A web page is a source code written in a specific computer language, such as hypertext markup language (HTML), cascading style sheets (CSS), JavaScript (JS), etc. The web page source code can be loaded and displayed as user-recognizable content by a browser or a web page display component with similar functions to a browser. The specific content contained in a web page is also defined by tags or nodes in the web page source code, such as HTML through <body>. 、 、 <video> 、 <canvas>To define the elements and attributes of a web page.

[0344] A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operations that uses graphics. It can be an icon, window, control, or other interface element displayed on the display of an electronic device. Controls can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, and other visual interface elements.

[0345] Each step in the above method embodiments provided in this application can be completed by hardware integrated logic circuits in a processor or by software instructions. The method steps disclosed in the embodiments of this application can be directly implemented as hardware processor execution, or can be implemented by a combination of hardware and software modules in the processor.

[0346] The present application also provides an electronic device, which may include a memory and a processor, wherein the memory may be used to store a computer program, and the processor may be used to call the computer program in the memory so that the electronic device executes the method in any one of the above embodiments.

[0347] The present application also provides a chip system, which includes at least one processor for implementing the functions involved in the method executed by the electronic device in any of the above embodiments.

[0348] In one possible design, the chip system also includes a memory, which is used to store program instructions and data. The memory is located inside or outside the processor.

[0349] The chip system can be composed of chips, or can include chips and other discrete devices.

[0350] Optionally, there may be one or more processors in the chip system. The processor may be implemented in hardware or software. When implemented in hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented in software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

[0351] Optionally, the memory in the chip system may be one or more. The memory may be integrated with the processor or may be provided separately from the processor, which is not limited in the embodiments of the present application. For example, the memory may be a non-transient processor, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or provided on different chips. The embodiments of the present application do not specifically limit the type of memory or the configuration of the memory and the processor.

[0352] Exemplarily, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chips.

[0353] The present application also provides a computer program product, which includes: a computer program (also referred to as code, or instruction), which enables a computer to execute the method executed by the electronic device in any of the above embodiments when the computer program is executed.

[0354] The present application also provides a computer-readable storage medium storing a computer program (also referred to as code or instruction). When the computer program is executed, the computer executes the method executed by the electronic device in any of the above embodiments.

[0355] A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. Available media can be magnetic media (e.g., floppy disk, hard disk, tape), optical media (e.g., DVD), or semiconductor media (e.g., solid-state drive Solid State Disk), etc.

[0356] Those skilled in the art will appreciate that all or part of the process steps in the above-described method embodiments can be implemented by a computer program instructing the relevant hardware. The program can be stored in a computer-readable storage medium, and when executed, the program can include the process steps in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0357] In summary, the above are only embodiments of the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made based on the disclosure of the present invention should be included in the scope of protection of the present invention.< / canvas> < / video> < / videoview> < / imgview> < / textview>

Claims

1. A method for detecting a flickering light source, applied to an electronic device comprising a first camera and a second camera, characterized in that: The method comprises: Setting the exposure time of the first camera to an integer multiple of the first light energy period, and the first camera capturing a first image; The electronic device displays a preview image on a display screen, where the preview image is an image captured by the first camera; Setting the exposure time of the second camera to an integer multiple of the second light energy period, and the second camera capturing a second image; Wherein, the shutters of the first camera and the second camera are both rolling shutters; The electronic device performs water ripple detection according to the first image and the second image; When the first image contains water ripple features, and a flickering light source corresponding to the second light energy period exists in the environment, the electronic device sets the exposure time of the first camera to an integer multiple of the second light energy period, and sets the exposure time of the second camera to an integer multiple of the first light energy period; When the first image does not contain water ripple features and the second image contains water ripple features, there is a flickering light source corresponding to the first light energy period in the environment, and the exposure time of the first camera is maintained at an integer multiple of the first light energy period, and the exposure time of the second camera is maintained at an integer multiple of the second light energy period; When neither the first image nor the second image includes water ripple features, there is no flickering light source in the environment.

2. The method according to claim 1, characterized in that When the first image includes a water ripple feature, the method further includes: The electronic device adjusts the exposure time of the first camera, and the minimum adjustment unit and the minimum value of the exposure time of the first camera are both the second light energy period currently corresponding to the first camera.

3. The method according to claim 1, characterized in that When the first image does not include a water ripple feature and the second image includes a water ripple feature, the method further includes: The electronic device adjusts the exposure time of the first camera, and the minimum adjustment unit and the minimum value of the exposure time of the first camera are both the first light energy period currently corresponding to the first camera.

4. The method according to claim 1, wherein When neither the first image nor the second image contains water ripple features, the method further includes: The electronic device adjusts the exposure time of the first camera, and a minimum adjustment unit and a minimum value of the exposure time of the first camera are both smaller than the first light energy period and the second light energy period.

5. The method according to any one of claims 2 to 4, characterized in that: The electronic device adjusting the exposure time of the first camera specifically includes: The electronic device performs image recognition on the first image to determine whether the first image content contains motion blur features; When the first image contains the motion blur feature, the electronic device reduces the exposure time of the first camera.

6. The method according to any one of claims 2 to 4, characterized in that: The electronic device adjusting the exposure time of the first camera specifically includes: The electronic device reduces the exposure time of the first camera through automatic exposure control AEC.

7. The method according to any one of claims 1 to 4, characterized in that The step of setting the exposure time of the first camera to an integer multiple of the first light energy period specifically includes: When the electronic device starts the first camera, the exposure time of the first camera is set to an integer multiple of the first light energy period; The method further includes: when the electronic device starts the first camera, starting the second camera; The step of setting the exposure time of the second camera to an integer multiple of the second light energy period specifically includes: When the second camera is started, the exposure time of the second camera is set to an integer multiple of the second light energy period.

8. The method according to claim 7, characterized in that Before the electronic device starts the first camera, the method further includes: the electronic device detecting a user operation of opening a camera application.

9. The method according to any one of claims 1 to 4, characterized in that The step of setting the exposure time of the first camera to an integer multiple of the first light energy period specifically includes: The electronic device determines the automatic exposure duration of the first camera through an automatic exposure control AEC; The electronic device determines the value of the integer multiple according to the automatic exposure duration and the first light energy cycle; The electronic device sets the exposure time of the first camera to the integer multiple of the first light energy period.

10. The method according to any one of claims 1 to 4, characterized in that After the electronic device displays the preview image on the display screen, the method further includes: The electronic device detects a first zoom operation; The preview image displayed by the electronic device is switched to the image captured by the second camera; the first zoom operation is used to adjust the zoom ratio from the zoom range of the first camera to the zoom range of the second camera.

11. The method according to claim 10, characterized in that In response to the first zoom operation, before switching the preview image to the image captured by the second camera, the method further includes: If the exposure time of the first camera is currently constrained to be an integer multiple of the first light energy period, then the exposure time of the second camera is set to an integer multiple of the first light energy period, and the second camera captures the fourth image; the exposure time of the first camera is set to an integer multiple of the second light energy period, and the second camera captures the third image; If the exposure time of the first camera is currently constrained to be an integer multiple of the second light energy period, setting the exposure time of the second camera to an integer multiple of the second light energy period, and the second camera captures the fourth image; setting the exposure time of the first camera to an integer multiple of the first light energy period, and the second camera captures the third image; If the exposure time of the first camera is not currently constrained to be an integer multiple of the first light energy period or the second light energy period, setting the exposure time of the second camera to an integer multiple of the second light energy period, and the second camera captures the fourth image; setting the exposure time of the first camera to an integer multiple of the first light energy period, and the first camera captures the third image; Alternatively, when the exposure time of the first camera is not currently constrained to be an integer multiple of the first light energy period or the second light energy period, the exposure time of the second camera is set to an integer multiple of the first light energy period, and the second camera captures the fourth image; the exposure time of the first camera is set to an integer multiple of the second light energy period, and the first camera captures the third image.

12. The method according to claim 11, characterized in that In response to the first zoom operation, before switching the preview image to the image captured by the second camera, the method further includes: The electronic device performs water ripple detection according to the third image and the fourth image; When the fourth image includes a water ripple feature, the electronic device sets the exposure time of the second camera to a first exposure time, where the first exposure time is an integer multiple of a light energy period that is different from a light energy period currently corresponding to the second camera; When the fourth image does not include the water ripple feature and the third image includes the water ripple feature, maintaining the exposure time of the second camera to be an integer multiple of the light energy period currently corresponding to the second camera; When neither the fourth image nor the third image includes water ripple features, there is no flickering light source in the environment.

13. The method according to claim 10, characterized in that Before the electronic device detects the first zoom operation, the method further includes: When the zoom ratio is adjusted to the first zoom range, the exposure time of the second camera is constrained to a third light energy period, and the electronic device captures a fifth image from the first camera and a sixth image from the second camera; the third light energy period is the first light energy period or the second light energy period; When the first zoom range is a zoom range between a first focal length threshold and a zoom range of the second camera, the first focal length threshold belongs to the zoom range of the first camera, and the minimum difference between the first focal length threshold and the focal length within the zoom range of the second camera is a preset value.

14. The method according to claim 13, characterized in that Before the electronic device detects the first zoom operation, the method further includes: The electronic device performs water ripple detection according to the fifth image and the sixth image; When the sixth image includes water ripple features, setting the exposure time of the second camera and the current exposure time of the first camera to be an integer multiple of a fourth light energy cycle, where the fourth light energy cycle is a light energy cycle different from the third light energy cycle; When the sixth image does not include the water ripple feature and the fifth image includes the water ripple feature, setting the current exposure time of the first camera to an integer multiple of the third light energy period; When neither the fifth image nor the sixth image includes water ripple features, there is no flickering light source in the environment.

15. The method according to claim 14, characterized in that The method further comprises: When the sixth image includes a water ripple feature, in response to the first zoom operation, setting the exposure time of the first camera to an integer multiple of the third light energy period; When the sixth image does not include a water ripple feature and the fifth image includes a water ripple feature, in response to the first zoom operation, a current exposure time of the first camera is set to an integer multiple of the fourth light energy period.

16. The method according to any one of claims 1 to 4, characterized in that The first camera is a wide-angle camera, and the second camera is an ultra-wide-angle camera; Alternatively, the first camera is an ultra-wide-angle camera, and the second camera is a wide-angle camera.

17. An electronic device, characterized in that: The method comprises one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, wherein the computer program code comprises computer instructions, and when the one or more processors execute the computer instructions, the method according to any one of claims 1 to 16 is executed.

18. A chip system, applied to electronic equipment, comprising one or more processors, characterized in that: The processor is configured to call computer instructions so as to execute the method according to any one of claims 1 to 16.

19. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the method according to any one of claims 1 to 16 is executed.

Images