A photographing method, an electronic device, and a storage medium

Abstract

This application provides a photographing method, electronic device, and storage medium, relating to the field of terminal devices, which can solve the problem of banding of varying brightness in captured images and improve the image quality of captured images. The method includes: launching a camera application in an electronic device; activating the capture function and detecting banding in response to a user's input of a capture mode setting in the camera application; acquiring images based on a first sensor mode and generating a first image queue and a second image queue in the presence of banding; determining a highlight moment and selecting a target image frame based on the highlight moment; and, in response to a photographing command, fusing a third image from the target image frame as a reference frame with a fourth image from the target image frame to generate a captured image.

Description

Technical Field

[0001] The application relates to the field of terminal technology, and in particular to a photographing method, electronic device, and storage medium. Background Technology

[0002] With the continuous development of electronic device technology, more and more electronic devices support multiple shooting modes. When users need to capture exciting moments with their electronic devices, they can set the camera application to work in capture mode to capture the subject's best moments. However, in capture mode, if the brightness of the image sensor in the electronic device changes during image acquisition, it will cause different rows in the image frame to have different exposures, resulting in stripes of varying brightness on the captured image, affecting the user's shooting experience. Summary of the Invention

[0003] This application provides a photographing method, electronic device, and storage medium that can solve the problem of stripes with varying brightness in captured images and improve the image quality of captured images.

[0004] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0005] In a first aspect, embodiments of this application provide a method for taking a picture, applied to an electronic device. The method includes: launching a camera application in the electronic device; responding to a user's input of a capture mode setting operation in the camera application, enabling a capture function and detecting banding; in the presence of banding, acquiring images based on a first sensor mode and generating a first image queue and a second image queue; at least one frame of the first image queue is an image acquired by the electronic device under a first exposure duration, and at least one frame of the second image queue is an image acquired by the electronic device under a second exposure duration, wherein the first exposure duration is longer than the second exposure duration; determining a highlight moment and selecting a target image frame based on the highlight moment; responding to a capture command, using a third image in the target image frame as a reference frame and fusing it with a fourth image in the target image frame to generate a captured image; wherein the exposure duration of the third image is the second exposure duration, and the fourth image includes images in the target image frame other than the third image.

[0006] Based on this solution, in the snapshot scenario, the presence of banding is detected. If banding is present, a double-exposure mode is used to acquire the image. When the snapshot command is triggered, the short frame is used as a reference frame and fused with the long frame to generate the snapshot image. Because the short frame has a shorter exposure time, its brightness and darkness are better, and banding is almost non-existent. Therefore, fusing the short frame as a reference frame with the long frame can produce a snapshot image without banding, and the quality of the snapshot image is high.

[0007] In one possible implementation of the first aspect, the above-mentioned detection of banding includes: acquiring the frequency of the light source in which the electronic device is located; and determining whether banding exists based on the frequency of the light source.

[0008] Based on this solution, the presence of banding can be accurately determined during the preview stage by obtaining the frequency of the light source in the environment where the electronic device is located and determining whether banding exists based on the light source frequency.

[0009] In one possible implementation of the first aspect, determining whether banding exists based on the frequency of the light source includes: determining that banding exists when the frequency of the light source is greater than or equal to a frequency threshold; and determining that banding does not exist when the frequency of the light source is less than the frequency threshold.

[0010] Based on this scheme, when the light source frequency is high, the duration of one flicker is short. Therefore, when acquiring images using single-exposure mode, the brightness changes during exposure due to light source flicker, resulting in banding in the acquired image. When the light source frequency is low, the duration of one flicker is long. Therefore, when acquiring images using single-exposure mode, light source flicker may not cause brightness changes during exposure, and thus banding does not exist. In other words, by comparing the light source frequency with a frequency threshold, the presence of banding can be determined more accurately. For example, this frequency threshold can be determined based on the exposure time corresponding to the single-exposure mode.

[0011] In one possible implementation of the first aspect, the method further includes: generating a preview image based on the first image in the presence of banding; and displaying the preview image on a preview interface.

[0012] Based on this solution, this application ensures good image quality for the user's preview by displaying long frames but not short frames during the preview stage of the capture scene, thus improving the preview effect. Furthermore, because the screen refresh rate is relatively fast during the preview stage, the user will not see banding caused by changes in light source brightness when long frames are displayed.

[0013] In one possible implementation of the first aspect, the above-mentioned selection of target image frames based on highlight moments includes: determining the sensor mode corresponding to the highlight moment based on the highlight moment; the sensor mode includes a first sensor mode and a second sensor mode; when the sensor mode is the first sensor mode, selecting the target image frame from the first image queue and the second image queue.

[0014] Based on this solution, after identifying the highlight moment, it can be determined whether the highlight moment uses double exposure mode or single exposure mode. If the highlight moment uses double exposure mode, then frames can be selected from the long frame queue and the short frame queue to ensure that the selected frames can generate the capture image that the user expects to take.

[0015] In one possible implementation of the first aspect, the above-mentioned response to the user's input of a capture mode setting operation in the camera application to enable the capture function includes: enabling the automatic capture function in response to the user's input of a first operation on the automatic capture control in the preview interface; or, enabling the manual capture function in response to the user's input of a second operation on the manual capture control in the preview interface; wherein the capture mode setting operation includes the first operation and the second operation.

[0016] Based on this solution, manual capture or automatic capture can be enabled in the preview interface. In both manual and automatic capture scenarios, short frames can be used as reference frames and fused with long frames to ensure that the generated capture images are free of banding and have good quality.

[0017] In one possible implementation of the first aspect, when the manual capture function is enabled, the capture command is the capture command entered by the user in the preview interface; when the automatic capture function is enabled, the capture command is the command generated by the electronic device when the exciting moment is detected.

[0018] Based on this solution, in manual capture scenarios, users need to input a photo-taking operation in the preview interface to trigger the photo-taking command and generate a captured image; in automatic capture scenarios, users do not need to input a photo-taking operation in the preview interface, and the sensing module in the electronic device can automatically trigger the photo-taking command when it detects a wonderful moment.

[0019] In one possible implementation of the first aspect, the method further includes: enabling the blur function in response to a user's input of a blur enable operation on the preview interface; and generating a blurred capture image based on the highlight moment and the captured image when the blur function is enabled.

[0020] Based on this solution, in a snapshot scenario, the user can enable the blur function. When the user enables the blur function, after the snapshot image is generated, it is necessary to further blur the snapshot image to obtain a blurred snapshot image.

[0021] In one possible implementation of the first aspect, generating a blurred capture image based on the highlight moment and the captured image includes: determining a fifth image corresponding to the highlight moment in the auxiliary path image queue; wherein the images in the first image queue and the second image queue are images captured by the first camera of the electronic device, and the images in the auxiliary path image queue are images captured by the second camera of the electronic device, and the first camera and the second camera are different; determining the parallax based on the fifth image and the captured image; and blurring the captured image based on the parallax to obtain a blurred capture image.

[0022] Based on this scheme, the captured image can be used as the main road image, and then the auxiliary road image (i.e., the fifth image) can be obtained from the auxiliary road queue (also known as the auxiliary road image queue) based on the highlight moment. Since the main road image and the auxiliary road image are images captured by different cameras, there is a parallax between the main road image and the auxiliary road image. Based on this parallax, the captured image can be blurred.

[0023] In one possible implementation of the first aspect, the method further includes: acquiring images based on a second sensor mode in the absence of banding, and generating a third image queue; at least one frame in the third image queue is an image acquired by the electronic device during the first exposure duration.

[0024] Based on this scheme, when banding is absent, images are acquired using single-exposure mode, resulting in an image frame queue. Since banding is absent when acquiring images using single-exposure mode, this mode can be continued without switching sensor modes. If banding is detected during the preview stage while acquiring images using single-exposure mode, the sensor mode is switched to double-exposure mode to ensure that the generated captured images are banding-free.

[0025] Secondly, embodiments of this application also provide a photographing device, which can be applied to an electronic device. The function of this device can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-described functions to implement any of the photographing methods described above.

[0026] Thirdly, this application provides an electronic device including a display screen, a memory, and one or more processors; the display screen, the memory, and the processors are coupled; wherein the memory stores computer program code, the computer program code including computer instructions, which, when executed by the processor, cause the electronic device to perform the photographing method provided by the first aspect and any possible design thereof.

[0027] Fourthly, this application provides a computer-readable storage medium including computer instructions that, when executed on an electronic device, cause the electronic device to perform the photographing method provided by the first aspect and any possible design thereof.

[0028] Fifthly, this application provides a computer program product that, when run on an electronic device, causes the electronic device to perform the photographing method provided by the first aspect and any possible design thereof.

[0029] Understandably, the beneficial effects that the technical solutions provided in the second to fifth aspects described above can be achieved can be referred to the beneficial effects in the first aspect and any of its possible design methods, which will not be repeated here. Attached Figure Description

[0030] Figure 1 A schematic diagram of a captured image exhibiting banding phenomenon, provided as an embodiment of this application;

[0031] Figure 2 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application;

[0032] Figure 3 A schematic diagram of the software architecture of an electronic device provided in an embodiment of this application;

[0033] Figure 4 A schematic diagram of the software architecture of another electronic device provided in an embodiment of this application;

[0034] Figure 5 A flowchart illustrating a photographing method provided in an embodiment of this application;

[0035] Figure 6 This is a schematic diagram of a camera startup procedure provided in an embodiment of this application;

[0036] Figure 7 A schematic diagram illustrating the setting of a snapshot mode according to an embodiment of this application;

[0037] Figure 8 A schematic diagram illustrating the activation of automatic or manual snapshot function as provided in this application embodiment;

[0038] Figure 9 This is a schematic diagram illustrating another way to enable the automatic snapshot function according to an embodiment of this application;

[0039] Figure 10 This is a schematic diagram illustrating the enabling of the blurring function in an embodiment of this application;

[0040] Figure 11 A schematic diagram illustrating frame selection based on highlights, provided as an embodiment of this application;

[0041] Figure 12 This is a schematic diagram illustrating how a photo-taking command is triggered in a preview interface, as provided in an embodiment of this application.

[0042] Figure 13 This is a schematic diagram illustrating the blurring process of a captured image, as provided in an embodiment of this application.

[0043] Figure 14 This is a flowchart illustrating another photographing method provided in an embodiment of this application. Detailed Implementation

[0044] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that “ / ” means “or,” for example, A / B can mean A or B; “and / or” in the text is merely a description of the relationship between related objects, indicating that three relationships can exist, for example, A and / or B can mean: A alone, A and B simultaneously, and B alone.

[0045] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0046] The terms "first" and "second" in the following embodiments of this application are for descriptive purposes only and should not be construed as implying relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0047] First, the terms used in the embodiments of this application are explained as follows:

[0048] Banding phenomenon: When an image sensor captures an image using line-by-line exposure, if pixels in different rows receive different amounts of energy, it will cause bright and dark stripes to appear in the image generated by the electronic device, which is called banding. For example, if the brightness changes during exposure due to factors such as light source flicker, different rows in a frame will be exposed differently, resulting in stripes of varying brightness in the generated image.

[0049] Single exposure mode: This means that the image sensor only exposes the subject once when acquiring an image. In other words, when the image sensor is working in single exposure mode, it only acquires one frame of the image.

[0050] Double exposure mode: This refers to an image sensor that can expose the subject twice during image acquisition. In other words, when the image sensor is operating in double exposure mode, it can capture two frames of images at once, with different exposure durations for each frame. For example, in double exposure mode, the subject is exposed twice based on a first exposure duration and a second exposure duration. The two frames captured by the image sensor are a long frame and a short frame, with the longer frame (e.g., the first exposure duration) having a longer exposure duration than the shorter frame (e.g., the second exposure duration).

[0051] A "moment of brilliance" refers to the optimal moment within a given period of time when the subject's state and / or movement are at their best. In some examples, the subject can be stationary or in motion. When the subject is stationary, a moment of brilliance is the moment when the image is sharp and without blur. For example, in a close-up portrait, a moment of brilliance is when the person is smiling with their eyes open and the image is sharp. When the subject is in motion, a moment of brilliance is the instant the subject moves. For example, in a shot of a person jumping, a moment of brilliance is when the person is in the air after taking off and the image is sharp.

[0052] Motion capture algorithms refer to techniques that capture the motion information of a subject and convert it into a computer-readable data format for processing and analysis. When processing images using Motion Capture, one or more frames can be captured first as reference frames. These reference frames provide information such as the initial position and posture of the subject for subsequent frames. Based on the information from the reference frames, subsequent frames are then corrected and adjusted to more accurately capture the subject's motion trajectory.

[0053] In this embodiment, when the image sensor uses a double-exposure mode to acquire images, the MotionCapture algorithm can fuse short frames as reference frames with long frames. Because short frames have shorter exposure times and better brightness / darkness effects with almost no banding, while long frames have longer exposure times and better image quality, fusing short frames as reference frames with high-quality long frames not only ensures that the generated captured image is free of banding but also ensures good image quality.

[0054] Typically, in snapshot scenarios, the image sensor in an electronic device can use a single-exposure mode to capture the subject in one exposure to obtain the snapshot image. When the image sensor uses single-exposure mode to acquire images, if the exposure time is longer than the flicker time of the light source, the flickering light source will cause different lines in the captured image frame to have different exposures. This results in stripes with varying brightness in the captured image, affecting the user's shooting experience.

[0055] Figure 1 This is a schematic diagram of a captured image exhibiting banding, provided as an embodiment of this application. Figure 1 As shown, when capturing a subject's jumping motion, the brightness changes during exposure, resulting in stripes of varying brightness in the captured image.

[0056] To address the aforementioned issues, this application provides a photographing method. In a snapshot scenario, this method detects the presence of banding. If banding is present, it drives the image sensor to acquire an image using a double-exposure mode. Upon triggering a photographing command, it fuses a short frame as a reference frame with a long frame to generate a snapshot image. Because the short frame has a shorter exposure time, its brightness and darkness are better, and banding is almost nonexistent. Therefore, fusing the short frame as a reference frame with the long frame yields a snapshot image free of banding, and the snapshot image quality is high.

[0057] Furthermore, when using double-exposure mode to capture images, to improve the preview effect, long frames are displayed while short frames are not during the preview stage, ensuring that the preview image quality seen by the user is relatively good. Also, because the screen refresh rate is relatively fast during the preview stage, the user will not see banding when long frames are displayed during the preview stage.

[0058] The technical solutions provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0059] The technical solutions provided in this application can be applied to electronic devices with snapshot capabilities. In some embodiments, the electronic device may be a mobile phone, tablet computer, handheld computer, personal computer (PC), ultra-mobile personal computer (UMPC), netbook, as well as cellular phone, personal digital assistant (PDA), augmented reality (AR) device, virtual reality (VR) device, artificial intelligence (AI) device, wearable device, in-vehicle device, smart home device, and / or smart city device, etc. The embodiments of this application do not impose any special limitations on the specific type of the electronic device.

[0060] For example, taking a mobile phone as an electronic device, Figure 2 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown.

[0061] Reference Figure 2 As shown, the electronic device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, antenna 1, antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a display screen 193, a subscriber identification module (SIM) card interface 194, and a camera 195, etc. The sensor module 180 may include pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, bone conduction sensors, etc.

[0062] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0063] A controller can be the nerve center and command center of an electronic device. Based on the instruction opcode and timing signals, the controller generates operation control signals to control the fetching and execution of instructions.

[0064] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0065] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0066] The charging management module 140 is used to receive charging input from a power supply device (such as a charger, laptop power supply, etc.). The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input via the wireless charging coil of the electronic device.

[0067] While charging the battery 142, the charging management module 140 can also supply power to the electronic device through the power management module 141. Specifically, the battery 142 can be composed of multiple batteries connected in series. The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110.

[0068] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, display screen 193, camera 195, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery voltage, current, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110.

[0069] The external memory interface 120 can be used to connect to external non-volatile memory, thereby expanding the storage capacity of the electronic device. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to perform data storage functions. For example, music, video, and other files can be stored in the external non-volatile memory.

[0070] Internal memory 121 may include one or more random access memory (RAM) and one or more non-volatile memory (NVM). The RAM can be directly read and written by the processor 110 and can be used to store executable programs (e.g., machine instructions) of the operating system or other running programs, as well as user and application data. The NVM can also store executable programs and user and application data, and can be pre-loaded into the RAM for direct read and write operations by the processor 110.

[0071] A touch sensor, also known as a "touch device," can be located on the display screen 193. The touch sensor and the display screen 193 together form a touchscreen, also called a "touchscreen." The touch sensor detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 193. In other embodiments, the touch sensor may also be located on the surface of the electronic device, in a different position than the display screen 193.

[0072] An ambient light sensor is used to sense the brightness of ambient light. For example, the ambient light sensor can measure the light intensity of four channels of ambient light. The ambient light sensor outputs the measured light intensity of the four channels of ambient light to the processor 110. The processor 110 can process the light intensity of the four channels of ambient light output by the ambient light sensor to obtain the ambient light intensity. In the screen-on state, the electronic device can adaptively adjust the display brightness according to the obtained ambient light intensity.

[0073] A pressure sensor is used to sense pressure signals and convert them into electrical signals. In some embodiments, the pressure sensor may be located on the display screen 193. There are many types of pressure sensors, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. When a touch operation is applied to the display screen 193, the electronic device monitors the intensity of the touch operation based on the pressure sensor. The electronic device can also calculate the touch location based on the monitoring signal from the pressure sensor. In some embodiments, touch operations applied to the same touch location but with different intensities can correspond to different operation commands. For example, when a touch operation with an intensity less than a first pressure threshold is applied to the SMS application icon, a command to view an SMS message is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the SMS application icon, a command to create a new SMS message is executed.

[0074] In some embodiments, an electronic device may include one or N cameras 195, where N is a positive integer greater than 1. In this application embodiment, the type of camera 195 can be distinguished based on hardware configuration and physical location. For example, a camera located on the side of the electronic device's display screen 193 can be called a front-facing camera, and a camera located on the side of the electronic device's back cover can be called a rear-facing camera; another example is that a camera with a short focal length and a wide field of view can be called a wide-angle camera, while a camera with a long focal length and a narrow field of view can be called a regular camera. Here, focal length and field of view are relative concepts and are not specifically limited by parameters. Therefore, wide-angle cameras and regular cameras are also relative concepts, and can be specifically distinguished based on physical parameters such as focal length and field of view.

[0075] The electronic device implements display functions through a GPU, a display screen 193, and an application processor. The GPU is a microprocessor for image editing, connected to the display screen 193 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0076] Electronic devices can achieve shooting functions through ISP, camera 195, video codec, GPU, display 193, and application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information. In this embodiment, the GPU's functions are used during the frame rendering process of each image frame to achieve better display effects and performance in the final displayed image.

[0077] The Information Service Provider (ISP) is used to process data fed back from the camera 195. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization on image noise and brightness. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be integrated into the camera 195. The camera 195 is used to capture still images or videos.

[0078] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when an electronic device is selecting a frequency, a DSP can perform a Fourier transform on the frequency energy.

[0079] Display screen 193 is used to display images, videos, etc. Display screen 193 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device may include one or N displays 193, where N is a positive integer greater than 1.

[0080] In this embodiment of the application, the display screen 193 can be used to display pages required by the electronic device (e.g., wizard pages (including highlight recommendation pages and external module access pages), etc.), and display images captured by any one or more cameras 195 in the interface.

[0081] The wireless communication function of electronic devices can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem, and baseband processor.

[0082] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in an electronic device can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization.

[0083] The mobile communication module 150 can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for use in electronic devices. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 can be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 can be housed in the same device.

[0084] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through audio devices (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 193. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0085] The wireless communication module 160 can provide solutions for wireless communication applications in electronic devices, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0086] The SIM card interface 194 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 194 to make contact with and detach from the electronic device. The electronic device can support one or more SIM card interfaces. The SIM card interface 194 supports Nano SIM cards, Micro SIM cards, and other SIM cards. Multiple cards can be inserted into the same SIM card interface 194 simultaneously. The SIM card interface 194 is also compatible with external memory cards. The electronic device interacts with the network through the SIM card to achieve functions such as calls and data communication. One SIM card corresponds to one user number.

[0087] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a limitation on the structure of the electronic device. In other embodiments of this application, the electronic device may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0088] Of course, it is understandable that the above... Figure 2 The illustration shown is merely an example when the electronic device is in the form of a mobile phone. If the electronic device is in the form of a tablet, handheld computer, PC, PDA, wearable device (such as a smartwatch, smart bracelet), or other similar device, the structure of the electronic device may include more advanced features. Figure 2 The fewer structures shown can also include more than Figure 2 The structures shown are not limited here.

[0089] It is understandable that, generally speaking, the implementation of electronic device functions requires not only hardware support but also software cooperation. The software system of electronic devices can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application's embodiment uses a layered architecture... Taking the system as an example, the software structure of the electronic device is illustrated.

[0090] Figure 3 This is a schematic diagram of the layered architecture of the software system of the electronic device provided in the embodiments of this application. The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces (e.g., APIs).

[0091] In some examples, refer to Figure 3 As shown in the embodiments of this application, the software of the electronic device is divided into five layers, from top to bottom.

[0092] From top to bottom are the application layer, application framework layer, system library, and... runtime ( The runtime, hardware abstraction layer (HAL), and driver layer (or kernel layer).

[0093] The application layer can include a series of applications. For example... Figure 3 As shown, the application layer can include applications such as camera, gallery, calendar, map, WLAN, Bluetooth, music, video, SMS, and call.

[0094] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer includes predefined functions or services. For example, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc., but this application embodiment does not impose any limitations on these.

[0095] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0096] Content providers store and retrieve data, making that data accessible to applications. This data can include videos, images, audio, phone calls made and received, browsing history and bookmarks, phone books, etc.

[0097] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon can include views for displaying text and views for displaying images. In some embodiments, the view system may also include or initiate a rendering thread to perform operations such as drawing framebuffers.

[0098] A phone manager is used to provide communication functionality for electronic devices. For example, a phone manager can manage the call status of a calling application (including initiation, connection, and termination).

[0099] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0100] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0101] The system library can include multiple functional modules. For example: a surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), and 2D graphics modules (e.g., SGL). The surface manager manages the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The media libraries support playback and recording of various common audio and video formats, as well as still image files. The media libraries support various audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG. OpenGL ES is used for 3D graphics drawing, image rendering, compositing, and layer processing. SGL is a 2D graphics drawing module.

[0102] The Android runtime consists of the core libraries and the virtual machine. The Android runtime is responsible for scheduling and managing the Android system. The core libraries comprise two parts: one part contains the functionalities that Java calls, and the other part consists of the Android core libraries. The application layer and application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0103] The Hardware Abstraction Layer (HAL) is the interface layer between the operating system kernel and the hardware circuitry, designed to abstract the hardware. It hides the platform-specific hardware interface details, providing the operating system with a virtual hardware platform that is hardware-independent and portable across multiple platforms. The HAL provides a standard interface that exposes device hardware functionality to the higher-level Java API framework (i.e., the framework layer). The HAL contains multiple library modules, each implementing an interface for a specific type of hardware component, such as: audio HAL, Bluetooth HAL, camera HAL (also known as camera HAL or camera hardware abstraction module), and sensors HAL (or i-sensor service).

[0104] The kernel layer is the layer between hardware and software. The kernel layer includes at least display drivers, camera drivers, audio drivers, sensor drivers, battery drivers, etc., but this application does not limit this. Specifically, the sensor driver can include the driver for each sensor included in the electronic device, such as an ambient light sensor driver. For example, the ambient light sensor driver can, in response to an indication or instruction from the sensor module to acquire detection data, promptly send the detection data from the ambient light sensor to the sensing module.

[0105] In the embodiments of this application, such as Figure 4 As shown, the application layer includes the camera application. The application framework layer may include the camera access interface, which provides the application programming interface and programming framework for the camera application. The HAL layer is located between the application framework layer and the driver layer. Figure 4 The interface layer between (not shown in the image) can provide a virtual hardware platform for the operating system.

[0106] like Figure 4As shown in this embodiment, after detecting a user's startup operation of the camera application, the camera application calls the camera access interface of the framework layer to start the camera application, and then calls the camera module in the HAL layer to drive the hardware device (such as an image sensor or detection device) to start. For example, the camera module in the HAL layer can send a startup command to the camera device driver in the driver layer. The camera device driver can drive the image sensor corresponding to the camera device to acquire an image, and then transmit the acquired image back to the HAL layer. After the camera of the electronic device is started, the detection device can detect the frequency of the light source where the electronic device is located, and transmit the detected light source frequency back to the 3A module of the HAL layer. The 3A module detects whether banding exists based on the light source frequency.

[0107] If the 3A module detects banding, it can send an instruction to the camera module, instructing it to drive the corresponding image sensor to acquire images in double-exposure mode. When the image sensor acquires images in double-exposure mode, the image front end (IFE) module can either merge the long and short frames before displaying them, or display the long frame but not the short frame. Since the long frame has a longer exposure time than the short frame, its image quality is higher. To ensure good image quality during the preview stage, if banding is present, the AE module can output a flag to the IFE module, instructing it not to merge the long and short frames and to directly display the long frame but not the short frame, thus ensuring good preview image quality. If banding is not present, the image sensor still acquires images in single-exposure mode.

[0108] During the preview phase, the perception module of the HAL layer can detect highlight images in real time. When the perception module detects a highlight image, it can determine the time corresponding to that highlight image as the highlight moment. The perception module can send the timestamp corresponding to the highlight moment to the decision module, so that the decision module can determine whether the highlight moment uses a double exposure mode or a single exposure mode based on the timestamp. When the decision module determines that the sensor mode corresponding to the highlight moment is a double exposure mode, the frame selection module can select the target image frame from the long frame queue (also known as the first image queue) and the short frame queue (also known as the second image queue) based on the highlight moment. When the decision module determines that the sensor mode corresponding to the highlight moment is a single exposure mode, the frame selection module can select the target image frame from one image frame queue based on the highlight moment. When the electronic device automatically triggers the shooting command or the user manually triggers the shooting command, the frame selection module sends the selected target image frame to the shooting algorithm module. When the sensor mode corresponding to the highlight moment is a double exposure mode, the target image frame includes a long frame and a short frame. The shooting algorithm module can use this target image frame as a reference frame to fuse the short frame with the long frame to ensure that the generated captured image does not have banding problems.

[0109] In some examples, users can enable the bokeh effect in snapshot scenarios. For example... Figure 4 As shown, the HAL layer can also include a blurring module. When the user enables the blurring function (such as background blurring), the blurring module can perform blurring processing on the captured image generated by the photo-taking algorithm module to obtain a blurred image.

[0110] The technical solutions provided in the embodiments of this application can all be implemented in electronic devices with the above-described hardware or software architecture.

[0111] Based on the above Figure 3 and Figure 4 The software architecture shown below, combined with Figure 5 As shown, the photographing method provided in the embodiments of this application will be described. Figure 5 This is a flowchart illustrating a photographing method provided in an embodiment of this application. Figure 5 As shown, the photographing method may include the following steps S501-S515.

[0112] S501, In response to a user's input of a start command for the camera application of the electronic device, the camera application is launched.

[0113] For example, the startup command can be a startup operation or a voice startup command. This application embodiment does not limit the method of starting the camera application. The following embodiments use a startup command as an example for illustrative explanation.

[0114] For example, taking the startup command as the startup operation, such as Figure 6As shown in (a), the electronic device displays a desktop that includes an icon for a camera application. The user can click the camera application icon on the desktop. The electronic device receives and responds to the user's click on the camera application icon (i.e., the launch operation), and launches the camera application, displaying... Figure 6 The preview interface 601 shown in (b) is an example. That is, the aforementioned startup command can also be considered as a command to start the image preview function of the camera application. It should be noted that the photo-taking method in this embodiment can be applied to any application that supports snapshot functionality. For example, the aforementioned camera application can be a system camera application or a third-party application with snapshot functionality.

[0115] For example, after a user launches the camera application, the camera application can send a preview request to the camera access interface in the framework layer. The camera access interface then sends the preview request to the camera module in the HAL layer. The camera module, in turn, sends the preview request to the camera device driver in the driver layer to start the camera. After the camera starts, the image sensor can acquire image light signals and transmit these signals to the image signal processor for preprocessing to obtain a raw image. The raw image is then transmitted back to the HAL layer via the camera device driver. The raw image is the image obtained after the photosensitive element in the camera converts the captured light source signal into an electrical signal; it can also be called the original image.

[0116] In some examples, after the camera application starts, it can continuously generate multiple preview requests and send each preview request to the image-related functional modules in the electronic device in sequence. The image-related functional modules in the electronic device respond to each preview request, acquire raw images, and generate preview images corresponding to each preview request based on the raw images.

[0117] S502, In response to the user's input of the snapshot mode setting operation, display the preview interface corresponding to the snapshot mode.

[0118] For example, the camera application includes various shooting modes, such as portrait mode, night scene mode, and snapshot mode. Images taken in portrait mode clearly show the facial features of the subject, images taken in snapshot mode can capture the instantaneous movement of the subject, and images taken in night scene mode have high clarity. The shooting method provided in this application embodiment can be applied to snapshot mode. The following embodiments use snapshot mode as an example to illustrate the shooting method provided in this application embodiment.

[0119] In some embodiments, a user can input one operation to activate the snapshot function in the electronic device, or input multiple operations to activate the snapshot function. The one or multiple operations can be referred to as the snapshot mode setting operation.

[0120] For example, such as Figure 7As shown in (a), the menu bar below the preview interface 701 includes a capture mode control 71. The user can click the capture mode control 71 in the preview interface 701 (this click operation is the capture mode setting operation). In response to the user's click operation on the capture mode control 71, the capture function is enabled and displayed. Figure 7 The preview interface 702 corresponding to the capture mode shown in (b) is shown in the image. In some examples, the user can also... Figure 7 In the preview interface 701 shown in (a), a sliding operation (i.e., a capture mode setting operation) is entered to slide the focus position to the capture mode control 71, thereby activating the capture function and displaying the image. Figure 7 The preview interface 702 corresponding to the capture mode shown in (b) is shown in the image.

[0121] The capture mode includes automatic and manual capture modes. In automatic capture mode (when the automatic capture function is enabled), the electronic device can automatically identify exciting moments and capture images without requiring user intervention, saving user time and preventing missed opportunities. In manual capture mode (when the manual capture function is enabled), the electronic device can identify exciting moments and select frames during the preview stage. When the user triggers a shooting command on the preview interface (such as clicking the camera control), the image algorithm module can merge the selected image frames to generate the captured image. In other words, automatic capture mode generates a captured image without requiring user intervention, while manual capture mode requires user intervention.

[0122] To avoid excessive image generation in automatic capture mode, which consumes too much memory and impacts the usability of electronic devices, and to prevent redundancy from too many images of the same scene, thus affecting user experience, a preset capture quantity threshold can be set in automatic capture mode. After recognizing a capture scene, the electronic device will generate capture images for each scene that do not exceed the threshold. For example, if the number of capture images for the current scene reaches the threshold, the electronic device will no longer automatically capture images for that scene.

[0123] The following is combined Figure 8 and Figure 9 The following provides an example illustrating how users can set up automatic and manual snapshot functions.

[0124] For example, such as Figure 8 As shown in (a), the preview interface 801 includes a manual capture control 81. The user can click the manual capture control 81 in the preview interface 801 (this click operation is the capture mode setting operation, also known as the second operation). In response to the user's click operation on the manual capture control 81, the manual capture function is enabled and displayed. Figure 8 The preview interface 802 is shown in (b) above. It should be noted that the color of the manual capture control 81 can be different when the manual capture function is on and off, so that users can see whether the manual capture function is on in the preview interface. For example, when the manual capture function is off, the manual capture control 81 is not filled with color, and when the manual capture function is on, the manual capture control 81 is filled with black or yellow. When the user needs to turn off the manual capture function, they can click the manual capture control 81 again in the preview interface 802 to turn off the manual capture function.

[0125] For example, such as Figure 8 As shown in (c), the preview interface 803 may also include an automatic capture control 82. The user can click the automatic capture control 82 in the preview interface 803 (this click operation is the capture mode setting operation, which can also be called the first operation). In response to the user's click operation on the automatic capture control 82, the automatic capture function is turned on.

[0126] This application embodiment does not limit the setting position of the manual capture control 81 and the automatic capture control 82 in the preview interface. The automatic capture control 82 can be set in... Figure 8 The preview interface 803 shown in (c) can also be set at the lower right corner. Figure 8 The preview interface shown in (d) is located at the top of 804. Figure 8 As shown in (d), the user can click the automatic capture control 82 in the preview interface 804 to enable the automatic capture function. When the user needs to disable the automatic capture function, they can click the automatic capture control 82 again in the preview interface 804 to disable the automatic capture function.

[0127] In some embodiments, the preview interface of the electronic device may not include the automatic capture control. In this case, the automatic capture function can be enabled through the settings control in the preview interface.

[0128] For example, such as Figure 9 As shown in (a), in order to enter the snapshot mode, the user can click the settings control 91 on the preview interface 901. In response to the user's click on the settings control 91, the following will be displayed: Figure 9 The settings interface shown in (b) is as follows. Figure 9 As shown in (b), the settings interface includes multiple controls related to taking photos (such as photo ratio control, smart photo control, and filter photo control) and multiple controls related to video (such as video resolution control, video frame rate control, high-efficiency video format control, and one-record-multiple-views control). Users can... Figure 9 In the settings interface shown in (b), a click operation is triggered on the smart camera control 92. In response to the user clicking the smart camera control 92, the following is displayed: Figure 9 The intelligent photo-taking settings interface shown in (c) includes voice-activated photo-taking controls, gesture-activated photo-taking controls, smile capture controls, and automatic capture controls. The electronic device has automatic capture functionality disabled by default, as shown in [image / description]. Figure 9 In the automatic snapshot control 93 shown in (c), the slider is on the left. The user can... Figure 9 In the intelligent photo-taking settings interface shown in (c), a click or swipe operation is triggered on the automatic capture control 93. This responds to the user clicking or swiping the automatic capture control 93, such as... Figure 9 As shown in (d), the electronic device can move the slider in the automatic capture control 93 to the right and activate the automatic capture function. Figure 9 As shown in (d), when the automatic capture function is enabled, the slider in the automatic capture control 93 moves to the right, and the left area of ​​the automatic capture control 93 is a predetermined color (e.g., blue).

[0129] In some embodiments, in video recording mode (also known as video shooting mode), the user can also enable automatic or manual snapshot functions. When the automatic snapshot function is enabled in video recording mode, in response to the user starting to record video, the electronic device can detect exciting moments in real time and select frames based on the exciting moments to generate snapshot images. During the video recording process of the electronic device, the generated snapshot images can be updated in real time in the playback control, and thumbnails of the snapshot images can be displayed in the playback control, thereby facilitating the user's understanding of the captured snapshot images. When the automatic snapshot function is enabled in video recording mode, the processing flow of steps S503-S514 below can be performed during video recording (i.e., the recording stage); when the automatic or manual snapshot function is enabled in photo shooting mode, steps S503-S510 in this application embodiment are executed in the preview stage, and steps S511-S514 are executed in the photo shooting stage. The following embodiments use automatic or manual snapshot in photo shooting mode as an example to illustrate the photo shooting method provided by this application embodiment.

[0130] In some examples, in order to highlight the subject, the user can enable the background blur function in the snapshot mode. This application embodiment does not limit the specific way for the user to enable the background blur function. The user can enable the blur function through the blur control or through other methods.

[0131] For example, taking the user enabling the blur function through the blur control as an example, such as Figure 10 As shown, the user can enable the automatic capture function or the manual capture function. Figure 10(Taking the user enabling the manual snapshot function as an example for illustration) After the user enables the manual snapshot function, the preview interface 1001 can include a blur control 101. The user can click the blur control 101 on the preview interface 1001 (also known as the blur enabling operation). In response to the user's click on the blur control 101, the blur function is enabled. In some examples, the blur control 101 has different colors when the blur function is enabled and disabled.

[0132] It should be noted that in both automatic and manual capture scenarios, when the user enables the bokeh effect, after the capture algorithm module generates the captured image, the bokeh module needs to further blur the captured image to obtain a blurred image.

[0133] S503. During the preview stage, the detection device detects the light source information in real time and sends the light source information to the 3A module.

[0134] For example, during previewing in automatic or manual capture scenarios, the detection device (such as a filter) in the electronic device can detect information about the light source (such as the light source frequency) in real time. For instance, after the camera of the electronic device is activated, the detection device in the electronic device can detect the light source frequency in real time, so that the 3A module can determine whether banding exists based on the light source frequency.

[0135] For example, 3A is a collective term for auto exposure (AE), auto focus (AF), and auto white balance (AWB). A 3A module may include an AE module, an AF module, and an AWB module.

[0136] The AE module automatically adjusts exposure parameters based on the ambient brightness of the electronic device to obtain a clear image. By monitoring the scene's brightness in real time, the AE module automatically adjusts the exposure time and aperture, enabling the camera to achieve ideal exposure results.

[0137] The AF module is used to automatically adjust the lens's focusing distance to ensure sharp images. AF modules typically use a motor-driven mechanism, employing algorithms to determine the focus point and automatically adjust the lens position to achieve autofocus.

[0138] The AWB module is used to automatically adjust the color balance of an image, making the colors in the captured image more realistic. By detecting the color temperature of the scene, the AWB module automatically adjusts color parameters, eliminating color differences and color casts, resulting in more natural and realistic colors in the captured image.

[0139] The S504 and 3A modules filter the light source information detected by the detection device and detect the presence of banding.

[0140] For example, after the detection device acquires the frequency of the light source, the 3A module can filter the light source and compare the frequency of the filtered light source with a frequency threshold to determine whether banding exists. If the frequency of the filtered light source is greater than or equal to the frequency threshold, banding is determined to exist; if the frequency of the filtered light source is less than the frequency threshold, banding is determined not to exist.

[0141] The higher the frequency of the light source, the more flashes occur per unit time (e.g., per second). Therefore, the brightness is more likely to change during a single exposure in single-exposure mode, resulting in banding in the captured image. Thus, banding is confirmed when the light source frequency is greater than or equal to a frequency threshold. Conversely, the lower the light source frequency, the fewer flashes occur per unit time. Therefore, the brightness may not change during a single exposure in single-exposure mode, and banding is not confirmed when the light source frequency is less than the frequency threshold.

[0142] For example, the frequency threshold can be determined based on the exposure time corresponding to the single-exposure mode. For instance, if the frequency of the light source is greater than the frequency corresponding to the exposure time of the single-exposure mode, it means that the duration of one flicker of the light source is less than the exposure time corresponding to the single-exposure mode. Therefore, in images acquired using the single-exposure mode, the brightness changes during exposure due to the flicker of the light source, resulting in different energy received by pixels in different rows, thus causing banding. Conversely, if the frequency of the light source is less than the frequency corresponding to the exposure time of the single-exposure mode, it means that the duration of one flicker of the light source is greater than the exposure time corresponding to the single-exposure mode. Therefore, in images acquired using the single-exposure mode, the flicker of the light source may not cause changes in brightness during exposure, and thus banding will not occur.

[0143] In some embodiments, the exposure time corresponding to the single-exposure mode of the image sensor can be the same as the first exposure time (i.e. the exposure time corresponding to the long frame) when the image sensor adopts the double-exposure mode, and the above-mentioned frequency threshold can also be determined based on the exposure time corresponding to the long frame.

[0144] In this embodiment, if banding is detected, the image sensor can be driven to acquire images using a double-exposure mode; if banding is not detected, the image sensor can be driven to acquire images using a single-exposure mode. That is, the sensor mode when banding is present is different from the sensor mode when banding is absent. To avoid frequent switching of the sensor mode in a short period affecting subsequent frame selection and image fusion in the image-taking algorithm module, the 3A module can increase the frequency threshold or set a detection period when detecting the presence of banding to ensure that the sensor mode used by the image sensor does not change within a short time.

[0145] For example, when determining the frequency threshold based on the exposure time corresponding to the single exposure mode, a preset value can be added to the frequency corresponding to the exposure time of the single exposure mode to increase the frequency threshold and ensure that the image sensor does not frequently switch sensor modes.

[0146] For example, when 3A detects the presence of banding, it can perform the detection based on a detection cycle. That is, it checks for banding every preset detection cycle, so the sensor mode remains unchanged within each detection cycle, thereby ensuring that the sensor mode does not switch frequently in a short period of time.

[0147] S505. In the presence of banding, the AE module sends an instruction message to the camera module and outputs a flag bit to the IFE module.

[0148] This instruction information is used to instruct the camera module to drive the image sensor to acquire images in a double-exposure mode (such as the first sensor mode). That is, in the presence of banding, the image sensor can acquire images in a double-exposure mode. For example, in the presence of banding, the image sensor can expose the subject twice to obtain a long frame (such as the first image) corresponding to the first exposure duration and a short frame (such as the second image) corresponding to the second exposure duration, where the first exposure duration is longer than the second exposure duration.

[0149] For example, when the image sensor acquires images using a double-exposure mode, the image front-end IFE module can either merge the acquired long and short frames (mergeraw) or not. Since the exposure times of the long and short frames acquired in double-exposure mode differ significantly, using the long frame as a reference frame for merging with the short frame during the preview stage will result in a poor-quality merged image (e.g., mergeaw). Because the long frame has a longer exposure time than the short frame, its image quality is better. Therefore, in the preview stage of the snapshot scene, this application can display the long frame but not the short frame to improve the preview effect, ensuring that the preview image seen by the user is of good quality. It should be noted that because the screen refresh rate is relatively fast during the preview stage, the user will not see banding caused by changes in light source brightness when displaying the long frame during the preview stage.

[0150] In some embodiments, to improve the preview effect, when banding is present, the AE module can output a flag to the IFE module to instruct the IFE module not to merge long frames and short frames, but to send long frames for display and not short frames for display.

[0151] In some embodiments, when banding is present, the image sensor can also use a triple-exposure mode to acquire images. That is, the image sensor can expose the subject three times to obtain a set of long frames, medium frames, and short frames (i.e., the image sensor outputs three frames at a time). The exposure time of the long frame is longer than that of the medium frame, and the exposure time of the medium frame is longer than that of the short frame. When the image sensor uses a triple-exposure mode to acquire images, the preview stream includes three image frame queues: a long frame queue, a medium frame queue, and a short frame queue. The frame selection module supports selecting multiple sets of image frames from these three queues. This application embodiment does not limit the sensor mode used by the image sensor when banding is present. For example, when banding is present, the image sensor can use a double-exposure mode, a triple-exposure mode, or more exposure modes to acquire images. Regardless of the sensor mode used, a long frame can be sent for display while the short frame is not sent for display during the preview stage. This application embodiment uses the image sensor using a double-exposure mode to acquire images when banding is present as an example for illustrative explanation.

[0152] S506: The camera module drives the image sensor to acquire long and short frames using a double-exposure mode based on the indication information.

[0153] For example, after receiving the instruction information sent by the AE module, the camera module determines that banding exists and sends a double exposure notification to the camera driver in the driver layer according to the instruction information. The camera driver then drives the image sensor to acquire images in double exposure mode based on the double exposure notification. When the image sensor acquires images in double exposure mode, the subject can be exposed twice at the same location. For example, the image sensor can expose the subject twice at the same location based on a first exposure duration and a second exposure duration, respectively. The two exposures can produce a first image with less noise but poor brightness and contrast (e.g., a long frame) and a second image with more noise but better brightness and contrast (e.g., a short frame). It is understood that when banding is not detected, the image sensor acquires images in single exposure mode (e.g., the second sensor mode). When banding is detected, the image sensor can be driven to switch sensor modes to switch from single exposure mode to double exposure mode. The exposure duration corresponding to the single exposure mode can be the same as the first exposure duration, or it can be different from the first exposure duration but longer than the second exposure duration.

[0154] When acquiring images using a double-exposure mode, the electronic device can alternate between a first exposure duration and a second exposure duration. The electronic device generates two image frame queues: a long frame queue (also known as the first image queue) and a short frame queue (also known as the second image queue). The long frame queue includes at least one first image (e.g., a long frame) acquired by the electronic device during the first exposure duration, and the short frame queue includes at least one second image (e.g., a short frame) acquired by the electronic device during the second exposure duration.

[0155] The S507 and IFE modules output long frames based on flag bits.

[0156] In the preview scene, if banding exists, the AE module can output a flag to the IFE module. The IFE module, based on this flag, will not fuse long and short frames and will output the long frame to the camera service in the frame layer for display in the camera application's preview interface. In other words, the preview image displayed in the electronic device's preview interface during the preview stage is a long frame.

[0157] S508: The perception module detects exciting moments in real time, determines the timestamp corresponding to the exciting moment, and sends the timestamp corresponding to the exciting moment to the decision module.

[0158] After enabling the automatic or manual capture function, the electronic device can identify image frames captured by the image sensor in real time during the preview stage and determine the highlight image frames. The moment corresponding to the highlight image frame is the highlight moment. For example, when the image sensor uses a double-exposure mode to capture images, the sensing module can determine the highlight image frames based on the long frames captured by the image sensor.

[0159] In both automatic and manual capture scenarios, the electronic device's sensing module can detect key moments in real time during the preview stage. For example, the sensing engine can detect parameters such as the state and / or movement of the subject, image exposure, and sharpness in each frame during the preview stage to determine whether the frame is a key image frame.

[0160] In some embodiments, the perception module may have a pre-set algorithm to identify highlight moments. For example, the perception module may use a pre-set image content extraction algorithm to obtain the subject and its motion state in each frame. When the subject is a person, the perception module may use the image content extraction algorithm to obtain the face and facial attributes (e.g., whether the face is smiling, has closed eyes, etc.). The perception module may also have a pre-set sharpness algorithm, which allows it to obtain the sharpness of each frame. In some examples, when an image includes a person, the person is in the optimal state of a preset action, and the image is clear, the perception module may determine that the frame is a highlight frame, and the moment corresponding to that frame is a highlight moment. In other examples, when an image includes a person, the person is smiling with open eyes, and the image is clear, the perception module may determine that the frame is a highlight frame.

[0161] In some examples, the perception module can acquire relevant data indicating image content and quality from the image frame, such as: the number, size, and position of the subjects, the degree of completion of actions, the number, size, and position of faces, the degree of facial smiles, the degree of eye opening, sharpness, exposure, and brightness. Then, based on this data, the image frame's "highlightability" (also known as a score) is determined. This score determines whether the image frame is "highlightable" or not; a higher score indicates a more "highlightable" image frame. The acquisition time corresponding to a highlightable image is called the "highlight moment."

[0162] For example, the pre-built algorithms in the perception module may include a subject detection algorithm, an action evaluation algorithm, a face detection algorithm, a face attribute algorithm, and a sharpness algorithm. Among these, the subject detection algorithm, action evaluation algorithm, face detection algorithm, and face attribute algorithm can be referred to as image content extraction algorithms, which can be used to obtain image content. The sharpness algorithm can be referred to as an image quality detection algorithm, which is used to obtain image sharpness.

[0163] Subject detection algorithms are used to detect objects in images, such as people, cat faces, and dogs. These algorithms can also extract local features of the object, such as facial features. When a subject detection algorithm detects an object in an image, it scores the object based on its size and position within the image; the more centrally located the object, the higher the score.

[0164] Action evaluation algorithms are used to detect the actions of the subject being photographed, such as jumping, looking back, throwing objects, playing table tennis, playing badminton, running, splashing water, playing baseball, playing rugby, and throwing frisbees. Based on the best completion point of each action, the algorithm determines the degree of completion of the action in any frame of the image, and then obtains the action score of the image. The higher the degree of action completion, the higher the action score of the image.

[0165] Understandably, determining the optimal completion point of an action requires identifying the trajectory of the subject across multiple frames of images. Therefore, an action evaluation algorithm can determine the optimal completion point of the subject within these frames after receiving them, and then evaluate the motion state of these frames based on this optimal completion point.

[0166] For example, the perception module can identify whether the subject is in motion by analyzing the motion changes of the subject in the preview image. When motion is detected, the module can determine the moment corresponding to the instant the motion is captured as the highlight moment. For instance, when the perception module detects a subject jumping, it can determine the highlight moment as the moment when the subject is in the air and the image is clear, based on the subject's posture after jumping.

[0167] Face detection algorithms are used to detect faces in images. Once a face is detected, the algorithm evaluates the faces in the image based on their number, size, and location. For example, the larger and more centrally located a face is, the higher its score.

[0168] Facial attribute algorithms are used to detect facial information in images, such as lip curvature and the distance between the upper and lower eyelids. Based on these parameters, it can be determined whether the face is open or smiling. Furthermore, the perception module can determine the degree of a smile based on lip curvature and the degree of eye openness based on the distance between the upper and lower eyelids. Combining the degree of smile and / or eye openness, the facial performance is evaluated, and a facial performance score is output. For example, a larger lip curvature corresponds to a higher degree of smile and a higher facial performance score; a larger distance between the upper and lower eyelids corresponds to a higher degree of eye openness and a higher facial performance score. If both lip curvature and eyelid distance are considered in a single image frame, the weights of eye openness and smile on the facial performance score can be set; these weights can be the same or different.

[0169] In some examples, face detection can be performed after the subject detection algorithm identifies a person, while the face attribute algorithm can be performed after the face detection algorithm identifies a face. The face detection algorithm can use the image containing the person identified by the subject detection algorithm to obtain the face, and the face attribute algorithm can use the face image output by the face detection algorithm to obtain the face information of each face, thereby saving computational costs and improving algorithm efficiency.

[0170] Sharpness algorithms are used to determine image sharpness. Images with no jitter and no noise receive higher sharpness scores.

[0171] In some examples, the perception module can determine an image quality score for each image frame by analyzing the attributes and features of the shooting scene and image, such as color, brightness, contrast, sharpness, and exposure. Understandably, the perception module can also be pre-configured with more detection algorithms to obtain more information indicating image content and image quality.

[0172] In some embodiments, when determining a highlight moment, the perception module can determine whether a frame is highlighting based on various data indicating the image content and image quality in a single frame, or it can obtain various data from one or more frames in a set of image frames to determine whether it is highlighting. For example, after determining a comprehensive score reflecting whether an image frame is highlighting and its degree of highlighting for each image frame, the perception module can determine the image frame with the highest comprehensive score as the highlighting image frame, or it can directly determine whether a frame is a highlighting image frame based on the comprehensive score of each frame.

[0173] In some embodiments, to improve the accuracy of image capture and user satisfaction, the perception module can continuously optimize and improve its algorithms and models through user feedback and learning mechanisms. By collecting user feedback on the capture results, the perception module can gradually learn user preferences and habits, thereby more accurately predicting and capturing memorable moments that satisfy users. This application does not limit the specific method by which the perception module determines memorable moments; the above embodiments are merely illustrative.

[0174] Once the perception module identifies a highlight image frame, it can define the time corresponding to that highlight image frame as the highlight moment and output the timestamp corresponding to the highlight moment to the decision module, so that the decision module can determine the sensor mode corresponding to the highlight moment based on the timestamp.

[0175] S509. The decision module determines the sensor mode corresponding to the highlight moment based on the timestamp of the highlight moment, and sends the sensor mode corresponding to the highlight moment to the frame selection module.

[0176] For example, when step S504 determines that there is no banding, the electronic device acquires images using a single-exposure mode. In this case, the image frame queue includes only one image frame queue (e.g., a third image queue), which contains at least one image frame acquired using the exposure duration corresponding to the single-exposure mode (e.g., a first exposure duration). When step S504 determines that banding exists, the electronic device acquires images using a double-exposure mode. In this case, the image frame queue includes a long frame queue and a short frame queue. Therefore, after the perception module determines the timestamp corresponding to the highlight moment, the decision module can determine whether the highlight moment uses a double-exposure mode or a single-exposure mode based on the timestamp.

[0177] For example, when the image corresponding to the highlight moment has only one frame, the sensor mode corresponding to the highlight moment is determined to be single exposure mode; when the image corresponding to the highlight moment has two frames, the sensor mode corresponding to the highlight moment is determined to be double exposure mode.

[0178] It should be noted that, to ensure that the image sensor's sensor mode does not switch frequently, when detecting banding in step S504, the frequency threshold can be increased or the detection period can be set to ensure that the sensor mode used by the image sensor does not change within a short period of time. Therefore, after determining the sensor mode corresponding to the highlight moment in step S509, when the frame selection module selects frames based on the highlight moment, it can select image frames acquired at the highlight moment and other moments adjacent to the highlight moment, where the sensor mode corresponding to the other moments is the same as the sensor mode corresponding to the highlight moment.

[0179] S510, the frame selection module selects frames from the image frame queue according to the sensor mode corresponding to the highlight moment.

[0180] When the sensor mode is in double exposure mode, the image frame queue includes a long frame queue and a short frame queue. Based on the timestamp corresponding to the highlight moment, the long and short frames corresponding to that highlight moment, as well as at least one set of long and short frames corresponding to other moments near the highlight moment, can be selected from these queues as the target image frame. This target image frame is the image frame required to generate the captured image.

[0181] For example, the frame selection module can select a set of long and short frames corresponding to the exciting moment, and a set of long and short frames corresponding to the next moment of the exciting moment, for a total of two sets of long and short frames as the target image frames for generating the captured image.

[0182] For example, such as Figure 11 As shown, taking the "highlight image frame" as the 4th preview image (i.e., the preview image corresponding to preview request 4), generating a captured image requires two sets of image frames. Since the "highlight image frame" is the 4th preview image, the "highlight moment" is the acquisition time of the 4th preview image. Figure 11 As shown in (a), when the highlight moment corresponds to the 4th frame preview image, the frame selection module can select long frame 1 corresponding to the highlight moment and long frame 2 corresponding to the next highlight moment (i.e., the 5th frame preview image) from the long frame queue; then, based on the positions of long frame 1 and long frame 2 in the long frame queue, short frame 1 and short frame 2 are selected from the short frame queue respectively. Specifically, the position of short frame 1 in the short frame queue is the same as the position of long frame 1 in the long frame queue, and the position of short frame 2 in the short frame queue is the same as the position of long frame 2 in the long frame queue. Figure 11 As shown in (a), the target image frames selected by the frame selection module include a set of long frames and short frames corresponding to preview request 4 and a set of long frames and short frames corresponding to preview request 5.

[0183] In some examples, when the target image frames selected by the frame selection module include both long and short frames, the multiple sets of long and short frames selected by the frame selection module can be stored in a long-short-long-short sequence. For example, the first long frame and the first short frame can be the long and short frames corresponding to the highlights.

[0184] For example, such as Figure 11 As shown in (a), the target image frames selected by the frame selection module include two sets of long and short frames, namely long frame 1, long frame 2, short frame 1 and short frame 2. These two sets of long and short frames can be stored in the cache in the sequence of long frame 1-short frame 1-long frame 2-short frame 2.

[0185] When the sensor mode is single exposure mode, the image frame queue consists of only one queue. At this time, based on the timestamp corresponding to the highlight moment, the image frame corresponding to the highlight moment and other image frames corresponding to moments near the highlight moment can be selected as the target image frame from this image frame queue.

[0186] For example, the frame selection module can select the image frame corresponding to the exciting moment and the image frame corresponding to the next moment after the exciting moment, for a total of two image frames as the target image frames for generating the captured image.

[0187] For example, such as Figure 11 As shown in (b) above, when acquiring images using single-exposure mode, if the highlight moment corresponds to the moment in the 4th frame preview image, the frame selection module can select image frame 1 corresponding to the highlight moment and image frame 2 corresponding to the next highlight moment (i.e., the moment in the 5th frame preview image) from the image frame queue. Figure 11 As shown in (b), the target image frames selected by the frame selection module include image frame 1 corresponding to preview request 4 and image frame 2 corresponding to preview request 5.

[0188] For example, the target image frame selected by the frame selection module can be stored in a cache. When the shooting command is triggered, the frame selection module can send the target image frame stored in the cache to the shooting algorithm module.

[0189] In some embodiments, when banding is present, the image sensor can also acquire images using a three-exposure mode. If the image sensor acquires images using a three-exposure mode, there are three image frame queues: a long frame queue, a medium frame queue, and a short frame queue. In this case, the frame selection module can select a set of long and short frames corresponding to the exciting moment, and another set of long and short frames corresponding to the next exciting moment, for a total of two sets of long and short frames as the target image frames for generating the captured image; alternatively, it can select a set of medium and short frames corresponding to the exciting moment, and another set of medium and short frames corresponding to the next exciting moment, for a total of two sets of frames as the target image frames for generating the captured image; or it can select a set of long, medium, and short frames corresponding to the exciting moment, and another set of long, medium, and short frames corresponding to the next exciting moment, for a total of two sets of frames as the target image frames for generating the captured image. This application embodiment does not limit the frame selection method of the frame selection module; it is merely an illustrative example.

[0190] It should be noted that after enabling manual or automatic capture, the electronic device's sensing module can detect exciting moments in real time during the preview stage. After the sensing module determines the timestamp corresponding to the exciting moment, the decision module will determine the sensor mode used by the electronic device at the exciting moment based on the timestamp. The frame selection module will then select frames based on the sensor mode and the timestamp corresponding to the exciting moment.

[0191] In manual snapshot scenarios, the user can use the electronic device's preview function to check whether the subject's actions, expressions, angles, and composition are optimal. When the user needs to capture a perfect moment, they can click the shutter button on the preview screen. Since the perfect moment usually occurs before the user clicks the shutter button, the electronic device can select frames during the preview stage. That is, the electronic device determines the perfect moment during the preview stage and selects the target image frame needed to synthesize the captured image based on that moment. In other words, steps S502-S511 are all performed during the preview stage after the camera application is launched.

[0192] For example, the decision-making module and frame selection module described above can be integrated into the camera module. In an automatic capture scenario, when the perception module detects a key moment, it sends a capture command to the camera module. In response to this capture command, the frame selection module in the camera module can send the selected target image frame to the camera algorithm module. It is understood that, unlike manual capture scenarios, in automatic capture scenarios, the capture command is triggered by the perception module in the electronic device when it detects a key moment.

[0193] S511, In response to the photo capture command, the frame selection module sends the selected frame to the photo capture algorithm module.

[0194] For example, the photo-taking command can be triggered by the user or automatically by the electronic device. In some examples, in manual capture scenarios, the user can trigger the photo-taking command by inputting a photo-taking operation in the preview interface, and the frame selection module sends the selected target image frame to the photo-taking algorithm module based on the user-triggered photo-taking command. In other examples, in automatic capture scenarios, the perception module in the electronic device can send a photo-taking command to the photo-taking module when it detects a highlight, and the frame selection module in the photo-taking module sends the selected target image frame to the photo-taking algorithm module based on the photo-taking command.

[0195] For example, in manual snapshot scenarios, such as Figure 12 As shown, the user can see whether the jumping action of the subject is in the optimal state in the preview interface 1201. When the user needs to capture the best jumping action of the subject, he / she can click the photo control 1202 in the preview interface 1201 (the user clicks the photo control 1202 in the preview interface to trigger the photo capture command). In response to the user clicking the photo control 1202, the frame selection module in the electronic device can send the target image frame selected for generating the captured image to the photo capture algorithm module.

[0196] For example, in an automatic capture scenario, once the perception module determines a key moment, it can send a capture command to the capture module. The frame selection module in the capture module then sends the selected target image frame to the capture algorithm module based on this capture command.

[0197] In some embodiments, in response to a photo-taking instruction, an electronic device may generate a photo-taking request and send the photo-taking request to an image-related functional module in the electronic device. The image-related functional module in the electronic device may generate a captured image in response to the photo-taking request.

[0198] For example, in an electronic device, the frame selection module responds to a photo capture request by sending a target image frame to the photo capture algorithm module, which then performs image processing based on the target image frame to generate a captured image.

[0199] S512, the image capture algorithm module uses an image capture algorithm to fuse target image frames to obtain a captured image.

[0200] For example, the image capture algorithm module can determine the corresponding image capture algorithm (such as the Motion Capture algorithm) based on the target image frame selected by the frame selection module, and use the corresponding image capture algorithm to perform fusion processing on the target image frame to generate a captured image.

[0201] In some examples, when the target image frame includes both long and short frames, the image capture algorithm module can use the short frame as a reference frame and fuse it with the long frame to obtain a captured image. For instance, when the target image frame includes multiple sets of long and short frames, the image capture algorithm module can use the first short frame (such as the third image) as a reference frame and fuse the other image frames in the target image frame (excluding the first short frame, such as the fourth image) with this reference frame to generate a captured image.

[0202] For example, such as Figure 11 As shown in (a), the target image frames selected by the frame selection module are long frame 1, long frame 2, short frame 1 and short frame 2. Since these two sets of long and short frames are stored in the cache in the sequence of long frame 1-short frame 1-long frame 2-short frame 2, after the image capture algorithm module obtains these 4 frames, it can use the first frame, short frame 1, as the reference frame and then fuse it with long frame 1, long frame 2 and short frame 2.

[0203] It is understandable that, since short frames have shorter exposure times and long frames have longer exposure times, short frames have better brightness and darkness effects and almost no banding, while long frames have better image quality. Therefore, when short frames are used as reference frames and fused with long frames to generate a captured image, it can ensure that the generated captured image does not have banding and that the generated captured image has good quality.

[0204] In some embodiments, in double-exposure mode, the target image frame includes multiple sets of long frames and short frames. Since the exposure duration of each set of long and short frames is different, the exposure amount is also different. When fusing long and short frames with different exposure amounts, it will have a certain impact on the quality of the fused image. In order to further improve the image quality of the captured image, when the image sensor uses double-exposure mode to acquire images, the exposure amount of the short frames can be increased to ensure that the exposure amount of the short frames and long frames is the same.

[0205] For example, when an image sensor acquires images using a dual-exposure mode, to ensure that the exposure levels of short and long frames are the same, the image quality module (PQ) can send exposure configuration information (such as a gain threshold) to the image sensor according to a configuration table. The image sensor then performs exposure according to this configuration information to improve the gain value of the short frame. This ensures that although the exposure durations of the long and short frames acquired by the image sensor are different, the exposure levels are the same.

[0206] In some embodiments, in a snapshot scenario, if the user enables a blurring function (such as background blurring), steps S513-S514 are included after step S512.

[0207] S513, The image capture algorithm module sends the captured image to the bokeh module.

[0208] S514, the blurring module generates a blurred capture image based on the captured image.

[0209] For example, when the blurring module generates a blurred capture image based on the captured image, it can use the captured image as the main image and then obtain auxiliary images (such as the fifth image) from the auxiliary image queue (also called the auxiliary image queue). The main image and the auxiliary images can be images captured by different cameras in the electronic device. For example, the long frame and short frame used to generate the capture image are image frames captured by one camera (such as the main camera) in the electronic device, while the images in the auxiliary image queue are image frames captured by another camera (such as the auxiliary camera) in the electronic device.

[0210] In some examples, the images corresponding to the highlights in the auxiliary road queue can be identified as auxiliary road images. Since the main road image and the auxiliary road image are images captured by different cameras, there is a parallax between the main road image and the auxiliary road image (parallax refers to the difference in position of the subject in the two images when two cameras take pictures of the same subject). Based on this parallax, the captured images can be blurred.

[0211] For example, generating a blurred capture image based on a main road image and an auxiliary road image includes: determining the parallax based on the main road image and the auxiliary road image, and blurring the capture image based on the parallax to obtain a blurred capture image.

[0212] For example, such as Figure 13As shown, the images in the long frame queue and short frame queue can be captured by the main camera in the electronic device, while the images in the auxiliary path queue can be captured by another camera (such as an auxiliary camera) in the electronic device. In step S512, the image capture algorithm module uses the first short frame 1 as a reference frame, and then fuses it with long frame 1, long frame 2, and short frame 2 to obtain the captured image. The blurring module uses this captured image as the main path image, and uses the image frame 3 corresponding to the highlight moment in the auxiliary path queue as the auxiliary path image. It calculates the disparity between the two frames of images, the main path image and the auxiliary path image, and then blurs the captured image based on the disparity to obtain a blurred captured image.

[0213] The image capture method provided in this application detects banding in the snapshot scene and uses a double-exposure mode to capture images when banding is present. During the preview stage, the longer frame is displayed while the shorter frame is not, ensuring good quality of the preview image seen by the user and improving the preview effect. Furthermore, because the screen refresh rate is relatively fast during the preview stage, the user will not see banding when the longer frame is displayed. During the snapshot stage, the shorter frame is used as a reference frame and fused with the longer frame to generate the snapshot image. Because the shorter exposure time of the shorter frame results in better brightness and darkness, with almost no banding, fusing the shorter frame as a reference frame with the longer frame yields a snapshot image without banding and of high quality.

[0214] Figure 14 A flowchart illustrating another photographing method provided in this application embodiment is shown below. Figure 14 As shown, the method includes the following steps S1401-S1405.

[0215] S1401. Launch the camera application on the electronic device.

[0216] For example, the camera application can be launched in response to a user's input of a launch command to the camera application of the electronic device. This application embodiment will not describe the launch method of the camera application in detail; please refer to the aforementioned step S501 for details.

[0217] S1402. In response to the user's input of the capture mode setting operation in the camera application, the capture function is enabled and banding is detected.

[0218] For example, in response to a user's input of a capture mode setting operation in the camera application, enabling the capture function includes: enabling the automatic capture function in response to a first operation input by the user to the automatic capture control in the preview interface; or, enabling the manual capture function in response to a second operation input by the user to the manual capture control in the preview interface; wherein the capture mode setting operation includes the first operation and the second operation. It is understood that the specific enabling methods of the automatic and manual capture functions are not described in detail in this application embodiment; please refer to the aforementioned step S502 for details.

[0219] For example, detecting banding includes: acquiring the frequency of the light source where the electronic device is located; and determining whether banding exists based on the frequency of the light source. If the frequency of the light source is greater than or equal to a frequency threshold, banding is determined to exist; if the frequency of the light source is less than the frequency threshold, banding is determined not to exist. It is understood that the specific activation methods of the automatic and manual capture functions are not described in detail in this application embodiment; please refer to steps S503-S504 above for details.

[0220] S1403. In the presence of banding, images are acquired based on the first sensor mode, and a first image queue and a second image queue are generated.

[0221] At least one first image (e.g., a long frame) in the first image queue (e.g., a long frame) is an image captured by the electronic device at a first exposure duration, and at least one second image (e.g., a short frame) in the second image queue (e.g., a short frame) is an image captured by the electronic device at a second exposure duration, wherein the first exposure duration is longer than the second exposure duration.

[0222] S1404. Determine the highlight moment and select the target image frame based on the highlight moment.

[0223] For example, selecting a target image frame based on a highlight moment includes: determining the sensor mode corresponding to the highlight moment based on the highlight moment; the sensor mode includes a first sensor mode (such as a double exposure mode) and a second sensor mode (such as a single exposure mode); when the sensor mode is the first sensor mode, selecting the target image frame in the first image queue and the second image queue.

[0224] In some embodiments, the double-exposure mode can also be called the double-exposure high dynamic range (HDR) mode. HDR refers to a technique used to achieve a greater dynamic range of exposure (i.e., greater difference between light and dark areas) than ordinary digital imaging techniques. In double-exposure HDR mode, the image sensor can acquire multiple frames based on different exposure durations to synthesize an image with high dynamic range. Since the captured image in this embodiment is generated by fusing long and short frames with different exposure durations, the image quality of the captured image is high.

[0225] It is understood that the methods for determining exciting moments and selecting frames in this application embodiment will not be described in detail. For details, please refer to the aforementioned steps S505-S510.

[0226] S1405 In response to the photo capture command, the third image in the target image frame is used as a reference frame and fused with the fourth image in the target image frame to generate a captured image.

[0227] The third image has the same exposure time as the second image, and the fourth image includes at least one frame from the target image frame, excluding the third image. The fourth image may include long frames and / or short frames. That is, when generating the captured image, short frames can be used as reference frames and fused with long frames to obtain the captured image.

[0228] When the manual capture function is enabled, the capture command is the one entered by the user in the preview interface; when the automatic capture function is enabled, the capture command is the command generated by the electronic device when it detects a wonderful moment.

[0229] It is understood that the method of generating the captured image will not be described in detail in the embodiments of this application. For details, please refer to the aforementioned steps S511-S512.

[0230] In some embodiments, if banding is determined to exist during the preview stage, this application can generate a preview image based on the first image and display the preview image on the preview interface. That is, if banding is determined to exist during the preview stage, this application will send long frames for display and not send short frames for display, in order to ensure good image quality during the preview stage and improve the preview effect.

[0231] For example, in the presence of banding, the AE module outputs a flag to the IFE module to instruct the IFE module not to fuse long frames and short frames, but to directly output long frames. For details, please refer to the aforementioned steps S505 and S507.

[0232] In some embodiments, the user can also enable the bokeh function. The method further includes: in a snapshot scenario, in response to the user's input of a bokeh enable operation in the preview interface, enabling the bokeh function; when the bokeh function is enabled, generating a bokeh snapshot image based on the exciting moment and the snapshot image.

[0233] For example, generating a blurred captured image based on a highlight moment and a captured image includes: determining a fifth image corresponding to the highlight moment in an auxiliary path image queue; wherein the images in the first image queue and the second image queue are images captured by the first camera of the electronic device, and the images in the auxiliary path image queue are images captured by the second camera of the electronic device, and the first camera and the second camera are different; determining the parallax based on the fifth image and the captured image; and blurring the captured image based on the parallax to obtain a blurred captured image.

[0234] It is understood that the method of blurring the captured image will not be described in detail in the embodiments of this application. For details, please refer to the aforementioned steps S513-S514.

[0235] In some embodiments, the method further includes: acquiring images based on a second sensor mode in the absence of banding, and generating a third image queue; at least one frame in the third image queue is an image acquired by the electronic device during the first exposure duration.

[0236] For example, the second sensor mode can also be called single-exposure mode. The method of generating preview images and captured images in single-exposure mode is the same as in the prior art, and this application embodiment does not limit this. It should be noted that since there is no banding when acquiring images using single-exposure mode, it is sufficient to continue using single-exposure mode to acquire images without switching sensor modes. If banding is detected during the preview stage while acquiring images using single-exposure mode, the sensor mode is switched to double-exposure mode to ensure that the generated captured image is free of banding.

[0237] It is understood that, in order to achieve the aforementioned functions, the electronic device includes corresponding hardware structures and / or software modules for performing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, the embodiments of the present invention can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in a hardware-driven or software-driven manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of the embodiments of this application.

[0238] This application embodiment can divide the above-described electronic device into functional modules based on the method example described above. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware or as software functional modules. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division; in actual implementation, there may be other division methods.

[0239] This application also provides a photographing device that can be applied to electronic devices. The functions of this device can be implemented through hardware or through hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, such as the photographing method provided in the foregoing embodiments.

[0240] This application also provides an electronic device, which includes: a display screen, a memory, and one or more processors; the display screen, the memory, and the processors are coupled; wherein, the memory stores computer program code, which includes computer instructions, and when the computer instructions are executed by the processor, the electronic device performs the photographing method provided in the foregoing embodiments. The specific structure of this electronic device can be referred to... Figure 4 The structure of the electronic device shown is illustrated.

[0241] This application also provides a computer-readable storage medium that includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the photographing method provided in the foregoing embodiments.

[0242] This application also provides a computer program product containing executable instructions that, when run on an electronic device, cause the electronic device to perform the photographing method provided in the foregoing embodiments.

[0243] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0244] In the several embodiments provided in this application, it should be understood that the disclosed apparatus / device and method can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0245] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0246] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0247] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0248] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A photographing method, characterized by, The method is applied to an electronic device, and the method comprises: starting a camera application in the electronic device; in response to a snapshot mode setting operation input by a user in the camera application, starting a snapshot function and detecting banding; in the presence of banding, acquiring images based on a first sensor mode and generating a first image queue and a second image queue; at least one frame of first image in the first image queue is an image acquired by the electronic device under a first exposure time, and at least one frame of second image in the second image queue is an image acquired by the electronic device under a second exposure time, the first exposure time being greater than the second exposure time; determining a highlight moment and selecting a target image frame from the first image queue and the second image queue based on the highlight moment; in response to a photographing instruction, fusing a third image in the target image frame as a reference frame with a fourth image in the target image frame to generate a snapshot image; wherein the exposure time of the third image is the second exposure time, and the fourth image comprises images other than the third image in the target image frame.

2. The method of claim 1, wherein, The detection of banding comprises: acquiring a frequency of a light source in which the electronic device is located; determining whether banding exists according to the frequency of the light source.

3. The method of claim 2, wherein, The determination of whether banding exists according to the frequency of the light source comprises: in the case that the frequency of the light source is greater than or equal to a frequency threshold, it is determined that banding exists; in the case that the frequency of the light source is less than the frequency threshold, it is determined that banding does not exist.

4. The method of claim 1, wherein, The method further comprises: in the presence of banding, generating a preview image based on the first image; displaying the preview image on a preview interface.

5. The method according to any one of claims 1-4, characterized in that, The selection of the target image frame from the first image queue and the second image queue based on the highlight moment comprises: determining a sensor mode corresponding to the highlight moment based on the highlight moment; the sensor mode comprises the first sensor mode and a second sensor mode; in the case that the sensor mode is the first sensor mode, selecting the target image frame from the first image queue and the second image queue.

6. The method according to any one of claims 1-4, characterized in that, The starting of the snapshot function in response to the snapshot mode setting operation input by the user in the camera application comprises: in response to a first operation input by the user on an automatic snapshot control in a preview interface, starting an automatic snapshot function; or in response to a second operation input by the user on a manual snapshot control in a preview interface, starting a manual snapshot function; wherein the snapshot mode setting operation comprises the first operation and the second operation.

7. The method of claim 6, wherein, In the case that the manual snapshot function is started, the photographing instruction is a photographing instruction input by the user in the preview interface; in the case that the automatic snapshot function is started, the photographing instruction is an instruction generated by the electronic device when the highlight moment is detected.

8. The method according to any one of claims 1-4, characterized in that, The method further comprises: in response to a blur starting operation input by the user in the preview interface, starting a blur function; In the case that the blurring function is turned on, a blurred snapshot image is generated based on the highlight moment and the snapshot image.

9. The method of claim 8, wherein, The generating of the blurred snapshot image based on the highlight moment and the snapshot image comprises: determining a fifth image corresponding to the highlight moment in a secondary image queue based on the highlight moment; wherein the images in the first image queue and the second image queue are images captured by a first camera of the electronic device, and the images in the secondary image queue are images captured by a second camera of the electronic device, the first camera being different from the second camera; determining a parallax based on the fifth image and the snapshot image; performing blurring processing on the snapshot image based on the parallax to obtain the blurred snapshot image.

10. The method of any one of claims 1-4, wherein, The method further comprises: in the absence of banding, capturing images based on a second sensor mode and generating a third image queue; at least one frame of image in the third image queue is an image captured by the electronic device under the first exposure time.

11. An electronic device, comprising: comprises: a display screen, a memory and one or more processors; the display screen, the memory and the processor are coupled; wherein the memory stores computer program code, the computer program code comprises computer instructions, when the computer instructions are executed by the processor, the electronic device executes the photographing method as claimed in any one of claims 1-10.

12. A computer-readable storage medium, characterized in that, comprises computer instructions, when the computer instructions run on the electronic device, the electronic device executes the photographing method as claimed in any one of claims 1-10.

Images