A frame preview method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a frame preview method and device, electronic equipment and storage medium. In response to the output mode of the camera sensor being adjusted from a first mode to a second mode, the camera sensor is set by using sensor exposure parameters, wherein the sensor exposure parameters include normal exposure parameters and variable exposure parameters; a normal exposure frame collected by the camera sensor under the normal exposure parameters and a variable exposure frame collected under the variable exposure parameters are obtained; the variable exposure frame is sent to a photographing channel; and the normal exposure frame is previewed and displayed. In the case of variable exposure photographing, the image frame can also be previewed, and since the frame previewed is the normal exposure frame, the user's preview experience is also improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a frame preview method, apparatus, electronic device, and storage medium. Background Technology

[0002] Backlighting and high-contrast environments are considered high dynamic range (HMR) scenes. In HMR scenes, variable exposure photography is often used to ensure the dynamic range and quality of the final image. Variable exposure photography adjusts the exposure time of the camera sensor through the automatic exposure (AE) module, thereby generating image frames with different exposures. However, when previewing image frames under variable exposure conditions, the different exposures can cause significant brightness changes in some frames, resulting in noticeable brightness jumps. To ensure a good preview experience, these variable exposure frames are discarded and not displayed in the preview. This causes a brief preview stutter in the preview area on the screen after the user clicks the shutter button.

[0003] For LivePhoto, it consists of a short preview clip and a captured photo. To ensure the smoothness of the preview clip, variable exposure strategies are not allowed when shooting LivePhoto. If a variable exposure strategy is used, the corresponding variable exposure frames will be discarded when sending the preview, thus affecting the smoothness of the preview clip. Summary of the Invention

[0004] The purpose of this application is to provide a frame preview method, apparatus, electronic device, and storage medium to solve the problem of poor preview effect when taking pictures with variable exposure. The specific technical solution is as follows:

[0005] Firstly, in order to achieve the above objectives, embodiments of this application provide a frame preview method, the method comprising:

[0006] In response to the camera sensor's output mode being adjusted from the first mode to the second mode, the camera sensor is set using sensor exposure parameters, wherein the sensor exposure parameters include normal exposure parameters and variable exposure parameters;

[0007] Acquire normal exposure frames captured by the camera sensor under the normal exposure parameters, and variable exposure frames captured by the camera sensor under the variable exposure parameters;

[0008] The variable exposure frame is sent into the imaging path;

[0009] The preview displays the normal exposure frame.

[0010] In this embodiment, in response to the camera sensor's output mode changing from a first mode to a second mode, the camera sensor is set using sensor exposure parameters, which include normal exposure parameters and variable exposure parameters. Normal exposure frames captured by the camera sensor under normal exposure parameters and variable exposure frames captured under variable exposure parameters are acquired. The variable exposure frames are sent to the image capture path. Normal exposure frames are then previewed and displayed. This allows for previewing image frames even when taking photos with variable exposure, and because the previewed frames are normal exposure frames, it also improves the user's preview experience.

[0011] In one possible implementation, the method further includes:

[0012] When the first mode is the merge mode, the normal exposure parameters are obtained by acquiring the first exposure parameters of the camera sensor in the merge mode.

[0013] In this embodiment of the application, for the case where the first mode is the merge mode, the first exposure parameters of the camera sensor in the merge mode are obtained. In this way, when the camera sensor mode is switched to the second mode, the exposure parameters of the camera sensor can be set using the first exposure parameters and the variable exposure parameters.

[0014] In one possible implementation, the method further includes:

[0015] When the first mode is a dual-gain conversion mode, short-exposure frames captured by the camera sensor in low-gain conversion mode and long-exposure frames captured in high-gain conversion mode are acquired; wherein, the dual-gain conversion mode includes low-gain conversion mode and high-gain conversion mode.

[0016] The short exposure frame and the long exposure frame are fused together;

[0017] The normal exposure parameters are obtained by determining the second exposure parameters of the fused exposure frame.

[0018] In this embodiment, for the case where the first mode is the dual-gain conversion mode, short exposure frames acquired by the camera sensor in low-gain conversion mode and long exposure frames acquired in high-gain conversion mode are obtained. The short exposure frames and long exposure frames are fused to determine the exposure parameters of the fused exposure frame, which is the second exposure parameter of the camera sensor in dual-gain conversion mode, and the normal exposure parameters are obtained. In this way, when the camera sensor mode is switched to the second mode, the exposure parameters of the camera sensor can be set using the second exposure parameter and the variable exposure parameter.

[0019] In one possible implementation, the method further includes:

[0020] When the first mode is a dual-gain conversion mode, the third exposure parameter of the camera sensor in the low-gain conversion mode and the fourth exposure parameter in the high-gain conversion mode are obtained; wherein, the dual-gain conversion mode includes a low-gain conversion mode and a high-gain conversion mode, and the normal exposure parameters include the third exposure parameter and the fourth exposure parameter.

[0021] In this embodiment of the application, when the first mode is the dual-gain conversion mode, the image output mode of the camera sensor is to output two frames simultaneously, namely the short exposure frame corresponding to the low-gain conversion mode and the long exposure frame corresponding to the high-gain conversion mode. At this time, it is necessary to obtain the third exposure parameter of the camera sensor in the low-gain conversion mode and the fourth exposure parameter in the high-gain conversion mode. In this way, when the camera sensor mode is switched to the second mode, the exposure parameters of the camera sensor can be set using the third exposure parameter, the fourth exposure parameter and the variable exposure parameter.

[0022] In one possible implementation, acquiring the normal exposure frame captured by the camera sensor under the normal exposure parameters includes:

[0023] Acquire short exposure frames captured by the camera sensor under the third exposure parameter and long exposure frames captured under the fourth exposure parameter;

[0024] The preview displays the normal exposure frame, including:

[0025] The short exposure frame and the long exposure frame are fused together;

[0026] The preview shows the merged frames.

[0027] In this embodiment of the application, for the case where the first mode is a dual-gain conversion mode, a short exposure frame is acquired using the third exposure parameter in the low-gain conversion mode, and a long exposure frame is acquired using the fourth exposure parameter in the high-gain conversion mode. The consistency of the preview effect can be ensured by using the short exposure frame and the long exposure frame.

[0028] In one possible implementation, the method further includes:

[0029] After acquiring a preset number of variable exposure frames, the camera sensor's output mode is switched from the second mode to the first mode.

[0030] Restore the camera sensor's exposure parameters to the corresponding exposure parameters in the first mode;

[0031] The camera sensor is driven to acquire image frames with the exposure parameters corresponding to the first mode.

[0032] In this embodiment of the application, after the variable exposure photography is completed, the output mode of the camera sensor is adjusted from the second mode to the first mode, so that the camera sensor still acquires image frames with the corresponding exposure parameters in the first mode.

[0033] In one possible implementation, the second mode is an interleaved high dynamic range imaging mode.

[0034] In one possible implementation, the method further includes:

[0035] After receiving a shooting request for variable exposure photography, the camera sensor's output mode is switched from the first mode to the second mode.

[0036] Secondly, embodiments of this application provide a frame preview device, the device comprising:

[0037] The exposure parameter setting module is used to set the camera sensor using sensor exposure parameters in response to the camera sensor's output mode being adjusted from a first mode to a second mode. The sensor exposure parameters include normal exposure parameters and variable exposure parameters.

[0038] The acquisition module is used to acquire normal exposure frames collected by the camera sensor under the normal exposure parameters, and variable exposure frames collected by the camera sensor under the variable exposure parameters.

[0039] The camera module is used to send the variable exposure frame into the camera path;

[0040] The preview module is used to preview and display the normal exposure frame.

[0041] In one possible implementation, the device further includes:

[0042] The first exposure parameter acquisition module is used to obtain the first exposure parameters of the camera sensor in the merge mode to obtain the normal exposure parameters when the first mode is the merge mode.

[0043] In one possible implementation, the device further includes:

[0044] The second exposure parameter acquisition module is used to acquire short exposure frames collected by the camera sensor in low gain conversion mode and long exposure frames collected in high gain conversion mode when the first mode is a dual gain conversion mode; wherein, the dual gain conversion mode includes low gain conversion mode and high gain conversion mode.

[0045] The short exposure frame and the long exposure frame are fused together;

[0046] The normal exposure parameters are obtained by determining the second exposure parameters of the fused exposure frame.

[0047] In one possible implementation, the device further includes:

[0048] The third and fourth exposure parameter acquisition modules are used to acquire the third exposure parameter of the camera sensor in the low-gain conversion mode and the fourth exposure parameter in the high-gain conversion mode when the first mode is the dual-gain conversion mode; wherein, the dual-gain conversion mode includes the low-gain conversion mode and the high-gain conversion mode, and the normal exposure parameters include the third exposure parameter and the fourth exposure parameter.

[0049] In one possible implementation, the acquisition module is specifically used for:

[0050] Acquire short exposure frames captured by the camera sensor under the third exposure parameter and long exposure frames captured under the fourth exposure parameter;

[0051] The preview module is specifically used to merge the short exposure frame and the long exposure frame; and to preview and display the merged frame.

[0052] In one possible implementation, the device further includes:

[0053] The output mode recovery module is used to trigger the camera sensor to switch the output mode from the second mode to the first mode after a preset number of variable exposure frames have been acquired.

[0054] The exposure parameter recovery module is used to restore the exposure parameters of the camera sensor to the corresponding exposure parameters in the first mode;

[0055] The acquisition module is used to drive the camera sensor to acquire image frames with the corresponding exposure parameters in the first mode.

[0056] In one possible implementation, the second mode is an interleaved high dynamic range imaging mode.

[0057] In one possible implementation, the device further includes:

[0058] The image output mode adjustment module is used to trigger the camera sensor to adjust its image output mode from the first mode to the second mode after receiving a shooting request for variable exposure photography.

[0059] Thirdly, embodiments of this application also provide an electronic device, including:

[0060] One or more processors and memory;

[0061] The memory is coupled to the one or more processors, and the memory is used to store computer program code, the computer program code including computer instructions, which the one or more processors invoke to cause the electronic device to execute the frame preview method described in any of the first aspects above.

[0062] Fourthly, embodiments of this application also provide a computer-readable storage medium including a computer program that, when run on an electronic device, causes the electronic device to perform any of the frame preview methods described in the first aspect.

[0063] The frame preview method provided in this application, in response to the camera sensor's output mode changing from a first mode to a second mode, sets the camera sensor using sensor exposure parameters, including normal exposure parameters and variable exposure parameters; acquires normal exposure frames captured by the camera sensor under normal exposure parameters, and variable exposure frames captured under variable exposure parameters; sends the variable exposure frames into the image capture path; and previews and displays the normal exposure frames. This enables image frame preview even when taking photos with variable exposure, and because the previewed frames are normal exposure frames, it also improves the user's preview experience. Attached Figure Description

[0064] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0065] Figure 1 A hardware structure diagram of an electronic device provided in an embodiment of this application;

[0066] Figure 2 A software structure block diagram of an electronic device provided in an embodiment of this application;

[0067] Figure 3 A user interface diagram provided for an embodiment of this application;

[0068] Figure 4 A schematic diagram of a shooting interface provided in an embodiment of this application;

[0069] Figure 5a A schematic diagram of a set of exposure frames with different exposure parameters provided for an embodiment of this application;

[0070] Figure 5b This is a set of preview images displayed after processing using existing technology.

[0071] Figure 5c This is a set of preview screens displayed after processing using the processing flow of this application;

[0072] Figure 6 A flowchart illustrating a frame preview method provided in this application embodiment;

[0073] Figure 7 A frame preview method in merging mode is provided in the embodiments of this application;

[0074] Figure 8 This application provides a frame preview method in dual-gain conversion mode for embodiments of the present application;

[0075] Figure 9 A frame preview method provided in this application embodiment;

[0076] Figure 10 This is a schematic diagram of a frame preview device provided in an embodiment of this application. Detailed Implementation

[0077] To better understand the technical solution of this application, the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0078] To facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with essentially the same function and effect. For example, "first instruction" and "second instruction" are used to distinguish different user instructions and do not limit their order. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0079] It should be noted that, in this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0080] The following describes some of the concepts involved in the frame preview method provided in the embodiments of this application.

[0081] Variable exposure frames: These are abnormal exposure frames, including long exposure frames and short exposure frames;

[0082] Long exposure frame: an exposure frame where EV is greater than EV0;

[0083] Short exposure frame: An exposure frame where EV is less than EV0;

[0084] EV: A quantity that reflects how much exposure is achieved. EV0 refers to the amount of exposure obtained at ISO (International Organization for Standardization) 100, with an aperture of F1 and an exposure time of 1 second.

[0085] Variable exposure shooting mode: refers to the shooting mode with HDR (High Dynamic Range) enabled. It is a shooting mode that captures image frames with multiple exposure parameters, such as long exposure frames, short exposure frames, and normal exposure frames, and then merges these image frames with multiple exposure parameters to obtain a high dynamic range image.

[0086] Backlighting and high-contrast environments are considered high dynamic range (HMR) scenes. In HMR scenes, variable exposure photography is often used to ensure the dynamic range and quality of the final image. Variable exposure photography adjusts the exposure time of the camera sensor through the automatic exposure (AE) module, thereby generating image frames with different exposures. However, when previewing image frames under variable exposure conditions, the different exposures can cause significant brightness changes in some frames, resulting in noticeable brightness jumps. To ensure a good preview experience, these variable exposure frames are discarded and not displayed in the preview. This causes a brief pause in the preview area on the screen after the user clicks the camera's shutter button.

[0087] For Live Photos, which consist of a short preview clip and a single captured photo, the generation of the captured photo follows the same process as a normal photo capture. To ensure the smoothness of the preview clip, variable exposure strategies are not allowed when shooting Live Photos. If a variable exposure strategy is used, the corresponding variable exposure frames will be discarded when sending the preview, thus affecting the smoothness of the preview clip. Therefore, Live Photos can only be taken using a non-variable exposure shooting method. However, non-variable exposure shooting does not achieve good results in high dynamic range scenes, and the captured photo in a high dynamic range scene is not optimal.

[0088] To address the issue of poor preview quality when taking photos with variable exposure, the frame preview method provided in this application embodiment will be described below.

[0089] This application provides a frame preview method applicable to electronic devices, such as smartphones, tablets, and smartwatches, that have cameras and provide photo-taking and display services. During variable exposure photography, the camera sensor's output mode is switched from a first mode to a second mode. In response to this switch, the camera sensor's exposure parameters are set using normal and variable exposure parameters. This process acquires both normal exposure frames and variable exposure frames. The variable exposure frames are then sent to the image capture path for image generation, while the normal exposure frames are sent to the preview path for preview display. When the camera sensor's output mode switches to the second mode, it simultaneously outputs both normal and variable exposure frames and sends the normal exposure frames for preview display. Since the exposure of each image frame is consistent during user preview, this method solves the problem of poor preview performance during variable exposure photography.

[0090] The structure of the electronic device to which the above frame preview method is applied is described below.

[0091] For example, Figure 1 A structural diagram of electronic device 100 is shown. Electronic device 100 may include a processor 110, a display screen 120, a camera 130, internal memory 140, a SIM (Subscriber Identification Module) card interface 150, a USB (Universal Serial Bus) interface 160, a charging management module 170, a battery management module 171, a battery 172 with battery cells and battery protection devices, a sensor module 180, a mobile communication module 190, a wireless communication module 200, antenna 1, and antenna 2, etc. The sensor module 180 may include a pressure sensor 180A, a fingerprint sensor 180B, a touch sensor 180C, an ambient light sensor 180D, etc.

[0092] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0093] Processor 110 may include one or more processing units, such as a CPU (Central Processing Unit), AP (Application Processor), modem processor, GPU (Graphics Processing Unit), ISP (Image Signal Processor), controller, video codec, DSP (Digital Signal Processor), baseband processor, and / or NPU (Neural-network Processing Unit). Different processing units may be independent components or integrated into one or more processors. In some embodiments, electronic device 100 may also include one or more processors 110. The controller can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution. In other embodiments, processor 110 may also include a memory for storing instructions and data. For example, the memory in processor 110 may be a cache memory. This memory can store instructions or data that processor 110 has just used or is repeatedly used. If processor 110 needs to reuse the instruction or data, it can directly retrieve it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the electronic device 100 in processing data or executing instructions.

[0094] In some embodiments, the processor 110 may include one or more interfaces. These interfaces may include an I2C (Inter-Integrated Circuit) interface, an I2S (Inter-Integrated Circuit Sound) interface, a PCM (Pulse Code Modulation) interface, a UART (Universal Asynchronous Receiver / Transmitter) interface, a MIPI (Mobile Industry Processor Interface) interface, a GPIO (General-Purpose Input / Output) interface, a SIM card interface, and / or a USB interface. The USB interface 160 is a USB standard-compliant interface, specifically a Mini USB interface, a Micro USB interface, a USB Type-C interface, etc. The USB interface 160 can be used to connect a charger to charge the electronic device 100, and can also be used for data transfer between the electronic device 100 and peripheral devices. The USB interface 160 can also be used to connect headphones for audio playback.

[0095] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are for illustrative purposes only and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also adopt different interface connection methods or a combination of multiple interface connection methods as described in the above embodiments.

[0096] The wireless communication function of electronic device 100 can be implemented through antenna 1, antenna 2, mobile communication module 190, wireless communication module 200, modem processor and baseband processor, etc.

[0097] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0098] Electronic device 100 implements display functions through a GPU, display screen 120, and application processor. The GPU is a microprocessor for image processing, connected to the display screen 120 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0099] The display screen 120 is used to display images, videos, etc. The display screen 120 includes a display panel. The display panel can be an LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), Active Matrix Organic Light-Emitting Diode, AMOLED (Active-Matrix Organic Light-Emitting Diode), FLED (Flexible Light-Emitting Diode), MiniLED, MicroLED, Micro-OLED, QLED (Quantum Dot Light-Emitting Diodes), etc. In some embodiments, the electronic device 100 may include one or more display screens 120.

[0100] In some embodiments of this application, when the display panel uses materials such as OLED, AMOLED, and FLED, the above-mentioned Figure 1 The display screen 120 can be bent. Here, "the display screen 120 can be bent" means that the display screen can be bent to any angle at any part and can maintain that angle. For example, the display screen 120 can be folded from the middle left to right. It can also be folded from the middle up and down.

[0101] The display screen 120 of electronic device 100 can be a flexible screen. Currently, flexible screens are attracting much attention due to their unique characteristics and enormous potential. Compared to traditional screens, flexible screens are highly flexible and bendable, providing users with new interaction methods based on their bendability and meeting more user needs for electronic devices. For electronic devices equipped with foldable displays, the foldable display can switch between a small screen in a folded state and a large screen in an unfolded state at any time. Therefore, users are increasingly using split-screen functionality on electronic devices equipped with foldable displays.

[0102] Electronic device 100 can perform shooting functions through ISP, camera 130, video codec, GPU, display 120 and application processor, wherein camera 130 includes a front camera and a rear camera.

[0103] The ISP is used to process data fed back from the camera 130. For example, during shooting, when the shutter is opened, light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can perform algorithmic optimization of image noise, brightness, and color. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 130.

[0104] Camera 130 is used to capture photos or videos. An object is projected onto a photosensitive element through a lens, generating an optical image. The photosensitive element can be a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide-Semiconductor) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to an ISP (Internet Service Provider) for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP (Digital Signal Processor) for processing. The DSP converts the digital image signal into image signals in standard RGB (Red, Green, Blue) or YUV (a color encoding method) formats. In some embodiments, the electronic device 100 may include one or N cameras 130, where N is a positive integer greater than 1.

[0105] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0106] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record video in various encoding formats, such as MPEG (Moving Picture Experts Group) 1, MPEG2, MPEG3, and MPEG4.

[0107] An NPU (Neural Processing Unit) is a neural network computing processor that, by drawing inspiration from the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, rapidly processes input information and can continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0108] Internal memory 140 can be used to store one or more computer programs, which include instructions. Processor 110 can execute the aforementioned instructions stored in internal memory 140, thereby causing electronic device 100 to perform the frame preview method provided in some embodiments of this application, as well as various applications and data processing. Internal memory 140 may include a program storage area and a data storage area. The program storage area may store the operating system; the program storage area may also store one or more applications (such as a gallery, contacts, etc.). The data storage area may store data created during the use of electronic device 100 (such as photos, contacts, etc.). In addition, internal memory 140 may include high-speed random access memory, and may also include non-volatile memory, such as one or more disk storage components, flash memory components, general-purpose flash memory, etc. In some embodiments, processor 110 can execute instructions stored in internal memory 140 and / or instructions stored in memory disposed in processor 110 to cause electronic device 100 to perform the frame preview method provided in embodiments of this application, as well as other applications and data processing.

[0109] The internal memory 140 can be used to store the relevant program of the frame preview method provided in the embodiments of this application. The processor 110 can be used to call the relevant program of the frame preview method stored in the internal memory 140 when displaying information, and execute the frame preview method of the embodiments of this application.

[0110] The sensor module 180 may include a pressure sensor 180A, a fingerprint sensor 180B, a touch sensor 180C, an ambient light sensor 180D, etc.

[0111] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be disposed on display screen 120. Pressure sensor 180A can be of many types, such as resistive pressure sensor, inductive pressure sensor, or capacitive pressure sensor. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 180A, the capacitance between the electrodes changes, and electronic device 100 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 120, electronic device 100 detects the touch operation based on pressure sensor 180A. Electronic device 100 can also calculate the touch position based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example, when a touch operation with an intensity less than a first pressure threshold is applied to the SMS application icon, a command to view an SMS is executed; when a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the SMS application icon, a command to create a new SMS is executed.

[0112] The fingerprint sensor 180B is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to perform functions such as unlocking, accessing application locks, taking photos, and answering calls.

[0113] Touch sensor 180C, also known as a touch device, can be disposed on display screen 120. The touch sensor 180C and display screen 120 together form a touchscreen, also known as a touch display. Touch sensor 180C is used to detect touch operations applied to or near it. Touch sensor 180C can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 120. In other embodiments, touch sensor 180C may also be disposed on the surface of electronic device 100 and in a different location from display screen 120.

[0114] The ambient light sensor 180D is used to sense the ambient light intensity. The electronic device 100 can adaptively adjust the brightness of the display screen 120 based on the sensed ambient light intensity. The ambient light sensor 180D can also be used to automatically adjust the white balance during shooting. The ambient light sensor 180D can also transmit environmental information about the device's location to the GPU.

[0115] The ambient light sensor 180D is also used to acquire the brightness, light ratio, color temperature, and other parameters of the environment in which the camera 130 captures images.

[0116] Figure 2This is a software architecture block diagram for an electronic device to which this application's embodiments apply. The software system of the electronic device can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. A layered architecture divides the software system of the electronic device into several layers, each with a clear role and division of labor, and the layers communicate with each other through software interfaces. In some embodiments, the software system can be divided into five layers: the application layer, the application framework layer, the system runtime layer, the HAL (Hardware Abstraction Layer), and the kernel layer.

[0117] The application layer can include cameras, photo galleries, and third-party applications with camera functionality. For example, the application layer can also include applications for calls, calendars, maps, navigation, music, video, and text messaging.

[0118] The application framework layer provides application programming interfaces and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0119] like Figure 2 As shown, the application framework layer may include a window manager, content provider, view system, resource manager, notification manager, camera service, etc.

[0120] The window manager is used to manage window programs. The window manager can obtain the screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0121] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0122] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0123] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0124] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. Examples include notifications of download completion and message alerts. Notifications can also appear as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Other notification methods include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0125] The camera service is used to invoke the camera (including the front-facing camera and / or the rear-facing camera) in response to a request from an application. Specifically, in this embodiment, the camera service can invoke the camera to capture images and obtain corresponding image frames in response to a photo-taking command. For example, if the user clicks the camera's shooting control, it is determined that a photo-taking command has been received, and the camera is invoked to capture an image.

[0126] The system runtime layer can include multiple functional modules. For example: surface manager, media library, 3D (Three Dimensional) graphics processing library (e.g., OpenGL ES), 2D (Two Dimensional) graphics engine, etc.

[0127] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0128] The media library supports playback and recording of various commonly used audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264 (a next-generation digital video compression format), MP3 (Moving Picture Experts Group Audio Layer III), AAC (Advanced Audio Coding), AMR (Adaptive Multi-Rate), JPG (Joint Photographic Experts Group), and PNG (Portable Network Graphics).

[0129] 3D graphics processing libraries are used to implement 3D graphics drawing, image rendering, compositing, and layer processing. 2D graphics engines are drawing engines for 2D graphics.

[0130] HAL (Host Alignment Layer) is the interface layer located between the operating system kernel and the hardware circuitry. HAL includes, but is not limited to, an image processing module. This image processing module is used to process image streams. In this embodiment, after the camera is started, the image processing module receives the image stream from the camera and can process the image stream, such as noise reduction, brightness adjustment, etc.

[0131] For example, when the camera application runs, it can send a user-triggered photo-taking command to the camera service. On one hand, the camera service can send the received photo-taking command to the image processing module, allowing the image processing module to call the camera driver in the kernel layer based on the received command. The camera driver then drives the camera and other hardware devices to respond to the command and capture image frames. For instance, the camera can transmit each captured image frame to the image processing module at a certain frame rate, where it is processed and then displayed on the screen. On the other hand, after receiving the photo-taking command, the camera service can determine the current shooting strategy based on the command. This strategy sets the specific image processing tasks to be performed on the captured image data. For example, in preview mode, the photo-taking command can set image processing task 1 in the shooting strategy to implement face detection. As another example, if the user enables a beautification function in preview mode, the camera service can also set image processing task 2 in the shooting strategy to implement the beautification function. Furthermore, the camera service can send the determined shooting strategy to the image processing module. After the image processing module receives each frame of image data captured by the camera, it can perform corresponding image processing tasks on the image data according to the shooting strategy issued by the camera service to obtain each frame of the captured image after image processing.

[0132] Subsequently, the image processing module can report each captured frame after image processing to the camera application through the camera service. The camera application can then display each captured frame in the preview area of ​​the screen, or save each captured frame in the electronic device as a photo or video.

[0133] The kernel layer is the layer between hardware and software. At a minimum, the kernel layer includes display drivers, camera drivers, image sensor drivers, and audio drivers. Hardware includes at a minimum, such as sensors, displays, cameras, and ISPs.

[0134] Understandable, Figure 2 The layers in the illustrated software structure and the components contained in each layer do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer layers than illustrated, and each layer may include more or fewer components; this application does not impose any limitations.

[0135] It is understood that, in order to implement the frame preview method provided in the embodiments of this application, an electronic device includes hardware and / or software modules that perform various functions. Based on the algorithm steps of the examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments, but such implementation should not be considered beyond the scope of this application.

[0136] The following describes an application scenario of the frame preview method provided in this application embodiment. In this scenario, a mobile phone is used as an example for explanation.

[0137] Figure 3 An exemplary illustration shows a user interface 300 on a mobile phone for displaying applications. The user interface 300 displays a page with application icons, which may include multiple application icons (e.g., weather app icon, calendar app icon, email app icon, settings app icon, app store app icon, notes app icon, photo album app icon, etc.). Below these multiple application icons, a page indicator may also be displayed to indicate the positional relationship between the currently displayed page and other pages. Below the page indicator are multiple tray icons (e.g., camera app icon 310, browser app icon, phone app icon, messaging app icon). The tray icons remain displayed when switching pages. This application embodiment does not limit the content displayed on the user interface 300.

[0138] The phone can detect user actions (such as touch / click) on the camera app icon 310, and in response to this action, the phone can display... Figure 4 The shooting interface 400 shown is a user interface for the default shooting mode of a camera application, where users can take photos. A camera application is an image-taking application on electronic devices such as smartphones and tablets; this application does not limit the name of the application. In other words, users can open the shooting interface 400 of the camera application by clicking the camera application icon 310. It is understood that the default camera in the default shooting mode is not limited to the rear camera; the phone can also set the front camera as the default camera. That is, after opening the camera application, the phone can display the image captured by the front camera in the preview area 420, which the user can use to take photos with the default front camera.

[0139] Figure 4 An example is shown of the shooting interface 400 of a camera application on a mobile phone. For example... Figure 4As shown, the shooting interface 400 may include a parameter adjustment area 410, a preview area 420, a camera mode option area 430, a gallery shortcut control 441, a shutter control 442, and a camera flip control 443. The controls in the parameter adjustment area 410 are used to adjust corresponding shooting parameters, including but not limited to: flash setting controls, HDR switch setting controls 411, and more detailed camera setting controls. The HDR switch setting control 411 can be used to trigger the electronic device to use an HDR algorithm to fuse multiple images with different exposure parameters when enabled.

[0140] The preview area 420 can be used to display a preview image, which is an image captured in real time by the phone's camera. The phone can refresh the display content in the preview area 420 in real time so that the user can preview the image currently captured by the camera.

[0141] The camera mode option area 430 may display one or more shooting mode options. These shooting mode options may include: night scene mode option 431, portrait mode option 432, photo mode option 433, video mode option 434, and more options 435. Understandably, these shooting mode options may be displayed as text information on the interface, such as "night scene," "portrait," "photo," "video," or "more," or as icons or other forms of interactive elements (IE), which this application does not limit.

[0142] When a user operation is detected on the shooting mode option, the phone can activate the user-selected shooting mode. Specifically, when a user operation is detected on the more options 435, the phone can further display more other shooting mode options, such as time-lapse mode options, burst mode options, etc., showcasing richer camera functions to the user. Understandably, the camera mode option area 430 can also contain more or fewer shooting mode options. Figure 4 The camera mode options shown are only one implementation of this application and should not be regarded as a limitation of this application.

[0143] The Gallery shortcut control 441 can be used to open the Gallery application. In response to user actions, such as touch operations, applied to the Gallery shortcut control 441, the phone can open the Gallery application.

[0144] The shutter control 442 can be used to listen for user actions that trigger taking a picture. The mobile phone can detect the user action applied to the shutter control 442, and in response to the action, the mobile phone acquires the corresponding captured image and combines the final output image to save it as a picture in the gallery application. In addition, the mobile phone can also display a thumbnail of the saved image in the gallery shortcut control 441. That is to say, the user can trigger taking a picture by acting on the shutter control 442. It is understood that the shutter control 442 can be a button or other form of control, and this application does not limit it.

[0145] The camera flip control 443 can be used to listen for user operations that trigger the camera flip. The phone can detect user operations, such as touch operations, applied to the camera flip control 443. In response to this operation, the phone can switch the camera used for shooting, for example, switching the rear camera to the front camera, or the front camera to the rear camera.

[0146] The shooting interface 400 may also contain more or fewer controls, and this application embodiment does not limit this.

[0147] The application scenarios of the solutions provided in the embodiments of this application will be introduced below.

[0148] In backlit, high-contrast environments, cameras typically enter variable exposure shooting mode to obtain photos with high dynamic range. In existing technology, in variable exposure shooting mode, the camera captures exposure frames at different exposure levels, i.e., the original frame sequence, and before sending the image frames to the display screen, the processor performs two processing steps.

[0149] The first processing step involves identifying variable exposure frames from the original frame sequence, discarding these frames, and sending the resulting preview frame sequence to the display screen. The display screen then previews the frames in its preview area. The second processing step involves fusing the long exposure frames, short exposure frames, and normal exposure frames from the original frame sequence using an algorithm to obtain a high dynamic range (HDR) photograph.

[0150] See Figure 5a If a user opens their phone's camera in a backlit scene and uses the variable exposure mode to take a picture of a person running, after the shooting command is generated, five frames are captured as shown in the figure. Frames 1, 2, and 5 are normal exposure frames, frame 3 is a short exposure frame, and frame 4 is a long exposure frame. (See also...) Figure 5bBefore sending the 5 frames to the display, the processor identifies and discards the variable exposure frames, specifically frames 3 and 4. The camera's preview area then displays frames 1, 2, and 5 sequentially. This missing frame between frames 2 and 5 causes a pause in the preview area and flickering in the preview image when the user clicks the shooting control, resulting in a poor user experience.

[0151] In view of this, the variable exposure photography processing flow in the prior art is improved in this application embodiment. In the improved processing flow, when taking a variable exposure photo, the output mode of the camera sensor needs to be adjusted from the first mode to the second mode. The exposure parameters of the camera sensor are set using normal exposure parameters and variable exposure parameters to obtain the normal exposure frame captured by the camera sensor under normal exposure parameters, and the variable exposure frame captured under variable exposure parameters. Then, the variable exposure frame is sent to the photography path to generate the final image, and the normal exposure frame is sent to the preview path for preview display. See also Figure 5c When the camera sensor switches to the second image output mode, it simultaneously outputs both normal exposure frames and variable exposure frames, and sends the normal exposure frames to the preview display. Since the exposure of each image frame is consistent when the user previews the image, it is evident that the method described in this embodiment solves the problem of poor preview quality when taking photos with variable exposure.

[0152] The frame preview method provided in this application will be described in detail below through specific embodiments.

[0153] See Figure 6 The flowchart of a frame preview method provided in this application embodiment may include steps S601-S604:

[0154] S601, in response to the camera sensor's output mode being adjusted from the first mode to the second mode, the camera sensor is set using sensor exposure parameters, wherein the sensor exposure parameters include normal exposure parameters and variable exposure parameters.

[0155] The function of a camera sensor is to output images in a specific image output mode. The first mode can be, for example, binning, dual conversion gain (DCG), stagger high dynamic range imaging (SHDR), etc. DCG mode is essentially a binning mode.

[0156] DCG mode refers to adding DCG to CMOS pixels, enabling the camera sensor to simultaneously possess high sensitivity and high dynamic range. DCG includes High Conversion Gain (HCG) and Low Conversion Gain (LCG). High HCG conversion gain corresponds to long exposure frames, resulting in excellent signal-to-noise ratio in dark areas. Low LCG conversion gain corresponds to short exposure frames, ensuring good highlight detail. When the camera sensor operates in DCG mode, long exposure frames and short exposure frames are acquired separately, and then the long and short exposure frames are fused. The fused image frame is used as the output image of the camera. In other words, the image output by the camera sensor in DCG mode is a fused image frame of long and short exposure frames. The fused image frame eliminates ghosting issues and improves dynamic range.

[0157] In this embodiment, after receiving a shooting request for variable exposure photography, the camera sensor's output mode may be switched from the first mode to the second mode. For example, in practical applications, this shooting request may be triggered by the user clicking the shooting control in the camera application with HDR shooting mode enabled.

[0158] In response to the camera sensor switching from the first mode to the second mode, the camera sensor is set using both normal and variable exposure parameters. In other words, the camera sensor simultaneously acquires image frames using both normal and variable exposure parameters.

[0159] For example, the variable exposure parameters can be parameters from a preset exposure parameter sequence, which can be set by relevant personnel based on experience. After obtaining a photo capture request (i.e., a variable exposure photo capture request) for image frames with different exposure parameters, the processor obtains each exposure parameter from the preset exposure parameter sequence and sends it to the camera sensor to drive the camera sensor to capture image frames with different exposure parameters.

[0160] It should be noted that the exposure parameters of each image frame can also be the exposure parameters in the exposure parameter sequence generated in real time by the processor after triggering the photo capture request for image frames with different exposure parameters. At this time, the processor sends the generated exposure parameter sequence to the camera sensor to drive the camera sensor to capture the image frame sequence with different exposure parameters.

[0161] In one example, the variable exposure parameter can be an exposure parameter whose EV value is not EV0, or an exposure parameter whose EV value is not within a preset range. This preset range can be a brightness range that is basically consistent with the brightness of EV0, set by relevant personnel based on experience. Both of these are reasonable.

[0162] S602, acquire the normal exposure frame collected by the camera sensor under the normal exposure parameters, and the variable exposure frame collected by the camera sensor under the variable exposure parameters.

[0163] In the second mode, the camera sensor outputs two frames simultaneously: one is a normal exposure frame obtained based on normal exposure parameters, and the other is a variable exposure frame obtained based on variable exposure parameters.

[0164] In normal exposure frames and variable exposure frames, "exposure" refers to exposure value (EV), not exposure time. Exposure value is determined by both exposure time and gain. For example, exposure value satisfies the formula: EV = Gain * Exposure Time. Here, EV represents exposure value. Factors affecting gain include, but are not limited to, CG and Iso. CG is conversion gain, and Iso is ISO.

[0165] S603, the variable exposure frame is sent into the imaging path.

[0166] The captured variable exposure frames are sent into the imaging path, and multiple variable exposure frames are fused by the imaging algorithm to obtain a high dynamic range image.

[0167] S604, the preview displays the normal exposure frame.

[0168] Since normal exposure frames are image frames acquired with normal exposure parameters, they can be previewed and displayed. The brightness of normal exposure frames will not change significantly, and users will not see any particular frame as particularly bright or dark during the preview.

[0169] In this embodiment, in response to the camera sensor's output mode changing from a first mode to a second mode, the camera sensor is set using sensor exposure parameters, which include normal exposure parameters and variable exposure parameters. Normal exposure frames captured by the camera sensor under normal exposure parameters and variable exposure frames captured under variable exposure parameters are acquired. The variable exposure frames are sent to the image capture path. Normal exposure frames are then previewed and displayed. This allows for previewing image frames even when taking photos with variable exposure, and because the previewed frames are normal exposure frames, it also improves the user's preview experience.

[0170] In one example, when the first mode is the merge mode, the normal exposure parameters are obtained by acquiring the first exposure parameters of the camera sensor in the merge mode.

[0171] Before receiving a shooting request for variable exposure photography, the camera sensor outputs an image in merge mode, obtaining the first exposure parameters of the camera sensor in merge mode as the normal exposure parameters. Here, merge mode means adding the charges sensed by adjacent pixels together and reading them out as a single pixel.

[0172] In this embodiment of the application, for the case where the first mode is the merge mode, the first exposure parameters of the camera sensor in the merge mode are obtained. In this way, when the camera sensor mode is switched to the second mode, the exposure parameters of the camera sensor can be set using the first exposure parameters and the variable exposure parameters.

[0173] In one example, when the first mode is a dual-gain conversion mode, short-exposure frames captured by the camera sensor in low-gain conversion mode and long-exposure frames captured in high-gain conversion mode are acquired; wherein, the dual-gain conversion mode includes low-gain conversion mode and high-gain conversion mode.

[0174] The short exposure frame and the long exposure frame are fused together;

[0175] The normal exposure parameters are obtained by determining the second exposure parameters of the fused exposure frame.

[0176] When the first mode is the dual-gain conversion mode, the camera sensor outputs two frames simultaneously: a short exposure frame corresponding to the low-gain conversion mode and a long exposure frame corresponding to the high-gain conversion mode. At this time, it is necessary to acquire the short exposure frame acquired by the camera sensor in the low-gain conversion mode and the long exposure frame acquired in the high-gain conversion mode, fuse the short exposure frame and the long exposure frame, and determine the second exposure parameter of the camera sensor in the dual-gain conversion mode based on the fused exposure frame.

[0177] In this embodiment, for the case where the first mode is the dual-gain conversion mode, short exposure frames acquired by the camera sensor in low-gain conversion mode and long exposure frames acquired in high-gain conversion mode are obtained. The short exposure frames and long exposure frames are fused to determine the exposure parameters of the fused exposure frame, which is the second exposure parameter of the camera sensor in dual-gain conversion mode, and the normal exposure parameters are obtained. In this way, when the camera sensor mode is switched to the second mode, the exposure parameters of the camera sensor can be set using the second exposure parameter and the variable exposure parameter.

[0178] In one example, when the first mode is a dual-gain conversion mode, the third exposure parameter of the camera sensor in the low-gain conversion mode and the fourth exposure parameter in the high-gain conversion mode are obtained; wherein, the dual-gain conversion mode includes a low-gain conversion mode and a high-gain conversion mode, and the normal exposure parameters include the third exposure parameter and the fourth exposure parameter.

[0179] In this embodiment of the application, when the first mode is the dual-gain conversion mode, the image output mode of the camera sensor is to output two frames simultaneously, namely the short exposure frame corresponding to the low-gain conversion mode and the long exposure frame corresponding to the high-gain conversion mode. At this time, it is necessary to obtain the third exposure parameter of the camera sensor in the low-gain conversion mode and the fourth exposure parameter in the high-gain conversion mode. In this way, when the camera sensor mode is switched to the second mode, the exposure parameters of the camera sensor can be set using the third exposure parameter, the fourth exposure parameter and the variable exposure parameter.

[0180] In one example, step S602 above, acquiring the normal exposure frame captured by the camera sensor under the normal exposure parameters, includes:

[0181] Acquire short exposure frames captured by the camera sensor under the third exposure parameter and long exposure frames captured under the fourth exposure parameter.

[0182] After switching the camera sensor's output mode to the second mode, the camera sensor simultaneously acquires image frames using the third exposure parameter, the fourth exposure parameter, and the variable exposure parameter. Since the third exposure parameter is the exposure parameter in the low-gain conversion mode, the camera sensor acquires short exposure frames when acquiring image frames using the third exposure parameter; since the fourth exposure parameter is the exposure parameter in the high-gain conversion mode, the camera sensor acquires long exposure frames when acquiring image frames using the fourth exposure parameter.

[0183] Step S604 above, previewing and displaying the normal exposure frame, includes:

[0184] The short exposure frame and the long exposure frame are fused together;

[0185] The preview shows the merged frames.

[0186] After acquiring the short exposure frame captured by the camera sensor under the third exposure parameter and the long exposure frame captured under the fourth exposure parameter, the short exposure frame and the long exposure frame can be fused and sent to the preview display.

[0187] In this embodiment of the application, for the case where the first mode is a dual-gain conversion mode, a short exposure frame is acquired using the third exposure parameter in the low-gain conversion mode, and a long exposure frame is acquired using the fourth exposure parameter in the high-gain conversion mode. The consistency of the preview effect can be ensured by using the short exposure frame and the long exposure frame.

[0188] In one example, the above method also includes:

[0189] S701, after acquiring a preset number of variable exposure frames, triggers the camera sensor to switch its output mode from the second mode to the first mode.

[0190] After the variable exposure shooting is completed, that is, after the preset number of variable exposure frames have been collected (the preset number can be determined according to the actual situation), the output mode of the camera sensor also needs to be adjusted to the first mode.

[0191] S702 restores the camera sensor's exposure parameters to the corresponding exposure parameters in the first mode.

[0192] S703 drives the camera sensor to acquire image frames with the corresponding exposure parameters in the first mode.

[0193] In this embodiment of the application, after the variable exposure photography is completed, the output mode of the camera sensor is adjusted from the second mode to the first mode, so that the camera sensor still acquires image frames with the corresponding exposure parameters in the first mode.

[0194] See Figure 7 This application provides a frame preview method in a merging mode. The camera sensor originally operates in Binning mode. When it receives a shooting request for variable exposure photography, it triggers the camera sensor's output mode to switch from Binning mode to SHDR (Short Exposure High Dynamic Range) mode. In SHDR mode, the camera sensor simultaneously outputs variable exposure frames and normal exposure frames. The normal exposure frame is obtained by configuring the exposure parameters and light ratio parameters unchanged through the automatic exposure (AE) module. The variable exposure frame (SNSL) is sent to the shooting path for image compositing, and the normal exposure frame is sent to the preview path for frame preview display. After the variable exposure photography is completed, the camera sensor's output mode is triggered to switch from SHDR mode to Binning mode. In the figure, S, N, and L represent image frames, where S is a short exposure frame, L is a long exposure frame, N is a normal exposure frame, and Nr is the normal exposure frame used as a reference. This application embodiment can achieve variable exposure photography preview without frame loss and can also support the use of variable exposure photography methods when shooting live photos.

[0195] See Figure 8This application provides a frame preview method in a dual-gain conversion mode. The camera sensor originally operates in dual-gain conversion mode (DCG), which includes high-gain conversion (HCG) and low-gain conversion (LCG). The camera sensor acquires long-exposure frames in HCG mode and short-exposure frames in LCG mode. When a variable-exposure shooting request is received, the camera sensor's output mode is triggered to switch from DCG mode to interlaced high dynamic range imaging mode (SHDR). In SHDR mode, the camera sensor simultaneously outputs variable-exposure frames and normal-exposure frames. The normal-exposure frames are obtained by configuring the exposure parameters and light ratio parameters unchanged through the automatic exposure (AE) module. The variable-exposure frames (SNSL) are sent to the image capture path for image composition; the normal-exposure frames are sent to the preview path for frame preview display. After the variable-exposure shooting is completed, the camera sensor's output mode is triggered to switch from SHDR mode to DCG mode. In the diagram, S, N, and L represent image frames. S is a short exposure frame, L is a long exposure frame, N is a normal exposure frame, Lr is the long exposure frame used as the reference, and Sr is the short exposure frame used as the reference. This embodiment of the application can achieve variable exposure preview without frame loss, and also supports the use of variable exposure shooting method when shooting live photos.

[0196] In one example, before triggering a variable exposure photo capture, the camera sensor's output mode can also be the staggered high dynamic range imaging mode (SHDR). In this way, when a variable exposure photo capture request is received, it is only necessary to configure the exposure parameters through the automatic exposure (AE) module to drive the camera sensor to capture image frames with the configured exposure parameters.

[0197] In one example, under Dual Gain Conversion (DCG) or Interleaved High Dynamic Range (SHDR) imaging modes, the contrast ratio is relatively high. If a single frame sent for preview has a high contrast ratio, it will lead to a decrease in the signal-to-noise ratio. However, when previewing the image frame, the resolution of the image frame is relatively low, so the user will not noticeably perceive the noise during preview. Of course, in this case, you can also choose to switch to SHDR mode that outputs three frames simultaneously. Two frames are used to maintain consistency with HCG and LCG for preview display, thus ensuring the consistency of the preview effect, while the other frame is a variable exposure frame used for final image processing.

[0198] See Figure 9This application provides a frame preview method whereby a user can click on a camera application. When the user clicks the camera to take a picture, the shooting command is sent to the camera hardware abstraction layer (HAL) through the camera access interface. The HAL can then send the shooting command to the automatic exposure module (AE). The AE calculates various exposure parameters and configures the output mode of the camera sensor to SHDR mode, along with the exposure and light ratio parameters of the frame in SHDR mode. After the camera sensor acquires the image frame, it sends the variable exposure frame to the HAL for image processing. Simultaneously, the AE configures the output mode of the camera sensor back to the original mode, along with the exposure parameters of the frame in the original output mode, and sends the acquired frame to the HAL.

[0199] On the other hand, embodiments of this application provide a frame preview device, see [link to relevant documentation]. Figure 10 The device includes:

[0200] The exposure parameter setting module 801 is used to set the camera sensor using sensor exposure parameters in response to the camera sensor's output mode being adjusted from a first mode to a second mode. The sensor exposure parameters include normal exposure parameters and variable exposure parameters.

[0201] The acquisition module 802 is used to acquire normal exposure frames collected by the camera sensor under the normal exposure parameters, and variable exposure frames collected by the camera sensor under the variable exposure parameters.

[0202] The camera module 803 is used to send the variable exposure frame into the camera path;

[0203] Preview module 804 is used to preview and display the normal exposure frame.

[0204] In one possible implementation, the device further includes:

[0205] The first exposure parameter acquisition module is used to obtain the first exposure parameters of the camera sensor in the merge mode to obtain the normal exposure parameters when the first mode is the merge mode.

[0206] In one possible implementation, the device further includes:

[0207] The second exposure parameter acquisition module is used to acquire, when the first mode is a dual-gain conversion mode, a short exposure frame captured by the camera sensor in low-gain conversion mode and a long exposure frame captured in high-gain conversion mode; wherein, the dual-gain conversion mode includes a low-gain conversion mode and a high-gain conversion mode; to fuse the short exposure frame and the long exposure frame; and to determine the second exposure parameter of the fused exposure frame to obtain the normal exposure parameter.

[0208] In one possible implementation, the device further includes:

[0209] The third and fourth exposure parameter acquisition modules are used to acquire the third exposure parameter of the camera sensor in the low-gain conversion mode and the fourth exposure parameter in the high-gain conversion mode when the first mode is the dual-gain conversion mode; wherein, the dual-gain conversion mode includes the low-gain conversion mode and the high-gain conversion mode, and the normal exposure parameters include the third exposure parameter and the fourth exposure parameter.

[0210] In one possible implementation, the acquisition module is specifically used for:

[0211] Acquire short exposure frames captured by the camera sensor under the third exposure parameter and long exposure frames captured under the fourth exposure parameter;

[0212] The preview module is specifically used to merge the short exposure frame and the long exposure frame; and to preview and display the merged frame.

[0213] In one possible implementation, the device further includes:

[0214] The output mode recovery module is used to trigger the camera sensor to switch the output mode from the second mode to the first mode after a preset number of variable exposure frames have been acquired.

[0215] The exposure parameter recovery module is used to restore the exposure parameters of the camera sensor to the corresponding exposure parameters in the first mode;

[0216] The acquisition module is used to drive the camera sensor to acquire image frames with the corresponding exposure parameters in the first mode.

[0217] In one possible implementation, the second mode is an interleaved high dynamic range imaging mode.

[0218] In one possible implementation, the device further includes:

[0219] The image output mode adjustment module is used to trigger the camera sensor to adjust its image output mode from the first mode to the second mode after receiving a shooting request for variable exposure photography.

[0220] This application also provides an electronic device including one or more processors and a memory; the memory is coupled to one or more processors and is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the electronic device to perform some or all of the steps in the above method embodiments.

[0221] This application also provides a computer-readable storage medium including a computer program that, when run on an electronic device, causes the electronic device to perform some or all of the steps described in the method embodiments. The storage medium may be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0222] Various embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or combinations of these implementation methods. Embodiments of this application can be implemented as computer programs or program code executable on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0223] Program code can be applied to input instructions to execute the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor such as, for example, a Digital Signal Processor (DSP), a microcontroller, an Application Specific Integrated Circuit (ASIC), or a microprocessor.

[0224] The program code can be implemented using a high-level procedural language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used when needed. In fact, the mechanisms described in this application are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0225] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored thereon on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed via a network or through other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine-readable (e.g., computer-readable) form, including but not limited to floppy disks, optical disks, CD-ROMs, compact disc read-only memory (CD-ROMs), magneto-optical disks, read-only memory, random access memory, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic cards or optical cards, flash memory, or tangible machine-readable storage for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in the form of electrical, optical, acoustic, or other forms of propagated signals. Therefore, machine-readable media includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a machine-readable (e.g., computer-readable) form.

[0226] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the accompanying drawings. Furthermore, including structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0227] It should be noted that all units / modules mentioned in the device embodiments of this application are logical units / modules. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important factor; the combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed in this application. Furthermore, to highlight the innovative aspects of this application, the above-described device embodiments of this application have not introduced units / modules that are not closely related to solving the technical problems proposed in this application. This does not mean that the above-described device embodiments do not contain other units / modules.

[0228] It should be noted that in the examples and description of this patent, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0229] Although this application has been illustrated and described with reference to certain preferred embodiments thereof, those skilled in the art should understand that various changes in form and detail may be made thereto without departing from the spirit and scope of this application.

Claims

1. A frame preview method, characterized in that, The method includes: After receiving a shooting request for variable exposure photography, the camera sensor's output mode is triggered to change from the first mode to the second mode, which is the interleaved high dynamic range imaging mode. In response to the camera sensor's output mode being adjusted from the first mode to the second mode, the camera sensor is set using sensor exposure parameters, wherein the sensor exposure parameters include normal exposure parameters and variable exposure parameters; The camera sensor acquires a normal exposure frame under the normal exposure parameters and a variable exposure frame under the variable exposure parameters. The normal exposure frame and the variable exposure frame are acquired simultaneously by the camera sensor. The normal exposure frame is obtained by setting the exposure parameters and light ratio parameters to remain constant through the automatic exposure (AE) module. The variable exposure frame is sent into the imaging path; The preview displays the normal exposure frame.

2. The method according to claim 1, characterized in that, The method further includes: When the first mode is the merge mode, the normal exposure parameters are obtained by acquiring the first exposure parameters of the camera sensor in the merge mode.

3. The method according to claim 1, characterized in that, The method further includes: When the first mode is a dual-gain conversion mode, short-exposure frames captured by the camera sensor in low-gain conversion mode and long-exposure frames captured in high-gain conversion mode are acquired; wherein, the dual-gain conversion mode includes low-gain conversion mode and high-gain conversion mode. The short exposure frame and the long exposure frame are fused together; The normal exposure parameters are obtained by determining the second exposure parameters of the fused exposure frame.

4. The method according to claim 1, characterized in that, The method further includes: When the first mode is a dual-gain conversion mode, the third exposure parameter of the camera sensor in the low-gain conversion mode and the fourth exposure parameter in the high-gain conversion mode are obtained; wherein, the dual-gain conversion mode includes a low-gain conversion mode and a high-gain conversion mode, and the normal exposure parameters include the third exposure parameter and the fourth exposure parameter.

5. The method according to claim 4, characterized in that, The step of acquiring the normal exposure frame captured by the camera sensor under the normal exposure parameters includes: Acquire short exposure frames captured by the camera sensor under the third exposure parameter and long exposure frames captured under the fourth exposure parameter; The preview displays the normal exposure frame, including: The short exposure frame and the long exposure frame are fused together; The preview shows the merged frames.

6. The method according to claim 1, characterized in that, The method further includes: After acquiring a preset number of variable exposure frames, the camera sensor's output mode is switched from the second mode to the first mode. Restore the camera sensor's exposure parameters to the corresponding exposure parameters in the first mode; The camera sensor is driven to acquire image frames with the exposure parameters corresponding to the first mode.

7. A frame preview device, characterized in that, The device includes: The output mode adjustment module is used to trigger the camera sensor to adjust the output mode from the first mode to the second mode after receiving a shooting request for variable exposure photography. The second mode is the staggered high dynamic range imaging mode. The exposure parameter setting module is used to set the camera sensor using sensor exposure parameters in response to the camera sensor's output mode being adjusted from a first mode to a second mode. The sensor exposure parameters include normal exposure parameters and variable exposure parameters. The acquisition module is used to acquire normal exposure frames collected by the camera sensor under the normal exposure parameters, and variable exposure frames collected by the camera sensor under the variable exposure parameters. The normal exposure frames and the variable exposure frames are acquired by the camera sensor simultaneously. The normal exposure frames are obtained by setting the exposure parameters and light ratio parameters to remain unchanged through the automatic exposure (AE) module. The camera module is used to send the variable exposure frame into the camera path; The preview module is used to preview and display the normal exposure frame.

8. An electronic device, characterized in that, include: One or more processors and memory; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the electronic device to perform the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, Includes a computer program that, when run on an electronic device, causes the electronic device to perform the method according to any one of claims 1-6.

Images