An exposure control method and electronic device

Abstract

This application provides an exposure control method and electronic device, relating to the imaging field, capable of executing exposure strategies based on a scene. For example, when the proportion of overexposed and underexposed areas to the overall area is greater than a preset proportion, a strategy to improve the dynamic range of the preview image and / or the captured image is executed. The method includes: initiating a camera to take a picture; determining an exposure strategy corresponding to the scene parameters in response to detecting scene parameters of the camera's preview image; the exposure strategy specifying the exposure duration of a first exposure frame and / or a second exposure frame, as well as specifying a strategy for generating the preview image and a strategy for generating the captured image; when the proportion information is greater than a preset proportion, the exposure strategy includes a strategy to improve the dynamic range of the captured image; and controlling the camera to generate the preview image and / or the captured image based on the exposure strategy corresponding to the scene parameters.

Description

Technical Field

[0001] This application relates to the imaging field, and more particularly to an exposure control method and electronic device. Background Technology

[0002] In the field of imaging, the dynamic range of an image refers to the range of brightness difference between the brightest and darkest parts of the image. It should be understood that the greater the dynamic range of an image, the more detail it retains in both highlights and shadows. Correspondingly, the dynamic range of a camera refers to the range of brightness that the camera can capture. For example, images captured by cameras that support High Dynamic Range (HDR) technology can retain more detail in both highlight and shadow scenes, resulting in better image quality compared to cameras that do not support HDR.

[0003] HDR technology can be divided into single-frame HDR technology and multi-frame HDR technology. Single-frame HDR technology can achieve high dynamic range of an image through a single exposure, while multi-frame HDR technology requires the continuous acquisition of two frames (referred to as long frames and short frames) or more frames with different exposure durations and their fusion to achieve high dynamic range.

[0004] Traditional multi-frame HDR technology typically produces high-resolution images, but motion blur can occur due to the temporal misalignment of frames with different exposure durations. To mitigate this issue, staggered HDR technology was developed. Staggered HDR captures long and short frames in a staggered manner; that is, after a long exposure for each line to obtain a long frame, a short exposure is immediately performed to obtain a short frame, instead of performing long exposures for all lines as in traditional multi-frame HDR. As can be seen, staggered HDR technology reduces the time difference between long and short frames, thus alleviating the motion blur problem faced by traditional multi-frame HDR technology.

[0005] However, since the images that the camera is trying to capture may be in relatively complex scenes, cameras that support staggered HDR technology may have poor dynamic range in these complex scenes. Summary of the Invention

[0006] To address this issue, embodiments of this application provide an exposure control method and electronic device capable of determining an exposure strategy based on scene parameters of the image to be captured by the camera. Specifically, when the proportion of overexposed and underexposed areas exceeds a preset ratio, the exposure strategy includes measures to enhance the dynamic range of the preview and / or captured images. This improves the display quality of the preview and / or captured images.

[0007] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0008] Firstly, an exposure control method is provided, applied to an electronic device. The electronic device includes a camera that supports line interleaving high dynamic range (HDR) technology. The line interleaving HDR technology supports outputting a set of exposure frames, including a first exposure frame and a second exposure frame, wherein the sum of the exposure durations of the first and second exposure frames is less than or equal to a preset duration. The method includes: initiating a camera to take a picture. In response to detecting scene parameters of the camera's preview image, determining an exposure strategy corresponding to the scene parameters. The scene parameters include proportion information, motion information, and stripe information in the preview image. The proportion information is the ratio of overexposed and underexposed areas to the overall area; the motion information indicates whether there is a moving object in the preview image and the speed of that object; and the stripe information indicates whether there are bright and dark stripes in the preview image. The exposure strategy specifies the exposure duration of the first and / or second exposure frames, and specifies the strategy for generating the preview image and the strategy for generating the captured image. When the proportion information is greater than a preset proportion, the exposure strategy includes a strategy to increase the dynamic range of the captured image. Based on the exposure strategy corresponding to the scene parameters, the camera is controlled to generate the preview image and / or the captured image.

[0009] Based on this scheme, electronic devices can execute different exposure strategies according to the different scenes being captured by the camera. For example, when the proportion of overexposed and underexposed areas in the overall area is greater than a preset proportion, a strategy that can improve the dynamic range of the preview and / or captured images is implemented. This helps to improve the display quality of the preview and / or captured images.

[0010] In one possible implementation, determining the exposure strategy corresponding to the scene parameters includes: when the scene parameters in the preview image indicate that the scale information in the preview image is greater than a preset scale, there is a moving object with a first speed, and there are no bright or dark stripes, the corresponding exposure strategy includes: reducing the exposure of the second exposure frame and reducing the exposure duration of the first exposure frame based on the first speed. A preview image and a captured image are generated based on the fused frame of the first and second exposure frames, or, a preview image is generated based on the fused frame of the first and second exposure frames, and either the fused frame of the first and second exposure frames or the first exposure frame is selected as a reference frame, and the first and second exposure frames are used to generate the captured image through a multi-frame fusion algorithm.

[0011] In one possible implementation, the scene parameters also include illuminance information from the preview image, which indicates the intensity of light. Determining the exposure strategy corresponding to the scene parameters further includes: when the scene parameters in the preview image indicate that the scale information in the preview image is less than a preset scale, there is a moving object with a first speed, there are no bright or dark stripes, and the light intensity is less than the preset illuminance, the corresponding exposure strategy is: setting the exposure duration of the second exposure frame to a preset exposure duration or one-nth of the exposure duration of the first exposure frame, and reducing the exposure duration of the first exposure frame based on the first speed. A preview image is generated based on the first exposure frame, the first exposure frame is selected as a reference frame, and the first exposure frame is used to generate the captured image through a multi-frame fusion algorithm. Here, the preset exposure duration is less than or equal to 13ms, and n is greater than or equal to 2.

[0012] In one possible implementation, the scene parameters also include illuminance information from the preview image, which indicates the intensity of light. Determining the exposure strategy corresponding to the scene parameters further includes: when the scene parameters in the preview image indicate that the scale information in the preview image is less than a preset scale, there is a moving object with a first speed, there are no bright or dark stripes, and the light intensity is greater than the preset illuminance, the corresponding exposure strategy is: based on the first speed, reduce the exposure duration of the first exposure frame, and set the exposure duration and gain value of the second exposure frame to be the same as the first exposure frame. A preview image is generated based on the first exposure frame, the first exposure frame is selected as a reference frame, and the first and second exposure frames are combined using a multi-frame fusion algorithm to generate the captured image.

[0013] In one possible implementation, the method further includes: when the scene parameters of the preview screen indicate that the scale information in the preview screen is less than a preset scale, there is a moving object with a first speed, and there are bright and dark stripes, the corresponding exposure strategy is: reducing the exposure duration of the second exposure frame based on the first speed, and setting the exposure duration of the first exposure frame to an integer multiple of the flicker period of the light source. A preview screen is generated based on the first exposure frame; the bright and dark stripes in the second exposure frame are corrected based on the stripe removal algorithm and the first exposure frame; the corrected second exposure frame is selected as the reference frame; and the first and second exposure frames are combined using a multi-frame fusion algorithm to generate the captured image.

[0014] In one possible implementation, the method further includes: when the scene parameters in the preview image indicate that the scale information in the preview image is greater than a preset scale, there is a moving object with a first speed, and there are bright and dark stripes, the corresponding exposure strategy is: setting the exposure duration of the first exposure frame to an integer multiple of the flicker period of the light source, and reducing the exposure duration of the second exposure frame based on the first speed. A preview image is generated based on the first exposure frame. The bright and dark stripes in the second exposure frame are corrected based on the stripe removal algorithm and the first exposure frame. The corrected second exposure frame is selected as the reference frame. The first exposure frame, the second exposure frame, and the subsequent frame are then combined using a multi-frame fusion algorithm to generate the captured image. The subsequent frame refers to the additional frame output by the camera after the electronic device takes a picture.

[0015] In one possible implementation, the scene parameters also include illuminance information from the preview image, which indicates the intensity of light. Determining the exposure strategy corresponding to the scene parameters further includes: when the scene parameters in the preview image indicate that the scale information in the preview image is less than a preset scale, there is no object movement, no bright or dark stripes, and the light intensity is greater than a preset illuminance, the corresponding exposure strategy is: setting the exposure duration and gain value of the second exposure frame to be the same as the first exposure frame. A preview image is generated based on the first exposure frame, the first exposure frame is selected as a reference frame, and the first and second exposure frames are fused using a multi-frame fusion algorithm to generate the captured image. Here, the preset exposure duration is less than or equal to 13ms, and n is greater than or equal to 2.

[0016] In one possible implementation, the scene parameters also include illuminance information from the preview image, which indicates the intensity of light. Determining the exposure strategy corresponding to the scene parameters further includes: when the scene parameters in the preview image indicate that the scale information in the preview image is less than a preset scale, there is no object movement, no bright or dark stripes, and the light intensity is less than the preset illuminance, the corresponding exposure strategy is: setting the exposure duration of the second exposure frame to a preset exposure duration or one-nth of the exposure duration of the first exposure frame, thus increasing the exposure duration of the first exposure frame. A preview image is generated based on the first exposure frame, and the first exposure frame is selected as a reference frame. The first exposure frame is then used to generate the captured image through a multi-frame fusion algorithm. Here, the preset exposure duration is less than or equal to 13ms, and n is greater than or equal to 2.

[0017] In one possible implementation, determining the exposure strategy corresponding to the scene parameters further includes: when the scene parameters in the preview image indicate that the scale information in the preview image is greater than a preset scale, there are no moving objects, and there are no bright or dark stripes, the corresponding exposure strategy is to reduce the exposure of the second exposure frame. A preview image and a captured image are generated based on the fused frame of the first and second exposure frames; or, a preview image is generated based on the fused frame of the first and second exposure frames, the first exposure frame is selected as a reference frame, and the first and second exposure frames are used to generate the captured image through a multi-frame fusion algorithm.

[0018] In one possible implementation, the method further includes: when the scene parameters of the preview screen indicate that there are bright and dark stripes in the preview screen, the exposure duration of the first exposure frame in the corresponding exposure strategy satisfies: the exposure duration of the first exposure frame is an integer multiple of the light source flicker period.

[0019] In one possible implementation, reducing the exposure duration of the first exposure frame based on the first speed includes: determining a mapping value for the reduction in exposure time corresponding to the first speed; calculating the quotient of the exposure duration of the first exposure frame and the mapping value for the reduction in exposure time to obtain the new exposure duration of the first exposure frame; and calculating the product of the gain value of the first exposure frame and the mapping value for the reduction in exposure time to obtain the new gain value of the first exposure frame.

[0020] In one possible implementation, reducing the exposure duration of the second exposure frame based on the first speed includes: determining a mapping value for the exposure time reduction magnitude corresponding to the first speed; calculating the quotient of the exposure duration of the second exposure frame and the mapping value for the exposure time reduction magnitude to obtain the new exposure duration of the second exposure frame; and calculating the product of the gain value of the second exposure frame and the mapping value for the exposure time reduction magnitude to obtain the new gain value of the second exposure frame.

[0021] In one possible implementation, the preset duration is equal to 33ms minus the readout duration of the first and second exposure frames.

[0022] In a second aspect, an electronic device is provided, comprising one or more processors, one or more memories, and a camera supporting line-interlaced high dynamic range technology. The one or more processors are connected to the camera, and the one or more memories are coupled to the one or more processors, storing a computer program. When the one or more processors execute the computer program, the electronic device performs an exposure control method as described in any of the first aspects.

[0023] Thirdly, a computer-readable storage medium is provided, comprising a computer program that, when executed, performs the steps of any of the methods in the first aspect.

[0024] Fourthly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to perform the steps of any of the methods in the first aspect.

[0025] Fifthly, a computer program product is provided, comprising a computer program, wherein the computer program, when executed by a processor, follows the steps of any of the methods in the first aspect.

[0026] In a sixth aspect, a chip system is provided, the chip including processing circuitry and an interface. The processing circuitry is used to retrieve and execute a computer program stored in a storage medium to perform an exposure control method as described in the first aspect and any of its possible designs.

[0027] It should be understood that the technical features of the technical solutions provided in the second to sixth aspects can all be corresponding to the exposure control methods provided in the first aspect and its possible designs, so the beneficial effects that can be achieved are similar, and will not be repeated here. Attached Figure Description

[0028] Figure 1 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application;

[0029] Figure 2 A schematic diagram of the software architecture of an electronic device provided in an embodiment of this application;

[0030] Figure 3 A flowchart illustrating an exposure control method provided in an embodiment of this application;

[0031] Figure 4 A schematic diagram of camera frame output provided for an embodiment of this application;

[0032] Figure 5 A schematic diagram illustrating another type of camera frame output provided in an embodiment of this application;

[0033] Figure 6 This is a schematic diagram of the composition of a chip system provided in an embodiment of this application. Detailed Implementation

[0034] In this application's embodiments, terms such as "first," "second," and "third" are used to distinguish different objects, not to limit a specific order. Furthermore, words such as "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application's embodiments should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0035] To facilitate understanding, some technical terms involved in the embodiments of this application will be explained below.

[0036] Exposure: The product of exposure time and gain. Exposure time refers to the duration the shutter needs to be open to allow light to reach the sensor of the camera's image sensor. Exposure time is typically controlled by the camera's shutter speed, while gain is determined by the sensitivity of the image sensor. Both exposure time and gain are positively correlated with the brightness of the captured image. The longer the exposure time, the brighter the captured image; the higher the gain, the brighter the captured image.

[0037] Sensitivity: This measures the sensitivity of a camera's image sensor to light. The higher the sensitivity of the image sensor, the more sensitive it is to light, and the stronger its ability to resolve light. In this application embodiment, sensitivity may also be referred to as ISO (International Standards Organization) sensitivity, ISO value, etc.

[0038] Automatic exposure (AE) algorithm: An algorithm that automatically adjusts the exposure and gain value of the camera based on the intensity of ambient light to mitigate overexposure or underexposure problems. Overexposure refers to an exposure exceeding a pre-set exposure threshold, which can be referred to as the first exposure threshold in this embodiment. Underexposure refers to an exposure below a pre-set exposure threshold, which can be referred to as the second exposure threshold in this embodiment. The first and second exposure thresholds can be the same or different. The AE algorithm can identify overexposed and underexposed areas in the image and determine the proportion of these areas to the overall image area. In this embodiment, this proportion can be referred to as the image proportion information.

[0039] Banding is a phenomenon where flickering stripes of varying brightness appear in the camera's preview or captured image. This phenomenon is related to the flicker period of the light source and the exposure time. Generally, banding does not occur when the exposure time is an integer multiple of the light source's flicker period. Banding may occur when the exposure time is not an integer multiple of the light source's flicker period. For example, if the light source is powered by 50Hz AC, its flicker period is 10ms. Therefore, when shooting a scene containing this light source, setting the exposure time to an integer multiple of 10ms will result in a banding-free image. If the exposure time is not an integer multiple of 10ms, banding may appear in both the preview and captured images. In this embodiment, the electronic device may include a flicker sensor that can detect the flicker frequency (i.e., the flicker period) of the light source in the currently captured or previewed scene. In this embodiment, the electronic device can determine whether banding exists in the preview and shooting images based on the flicker period of the light source detected by the flicker sensor and the exposure duration of each exposure frame, or it can directly detect whether banding exists in the preview image displayed by the camera.

[0040] Debanding algorithms are algorithms used to eliminate or reduce banding in images. For example, debanding algorithms can eliminate or reduce banding by adjusting the gamma curve parameters in the camera or the local tone mapping parameters in the image sensor. Alternatively, debanding algorithms can use deep neural networks to correct the color and brightness of banded frames using banded-free frames, thereby eliminating or reducing bright and dark stripes in the banded frames. Here, a banded-free frame is a frame without bright or dark stripes, while a banded frame is a frame containing bright and dark stripes.

[0041] Motion detection algorithms are used to detect the presence and speed of moving objects in captured and previewed images. In some possible implementations, this algorithm can determine the presence of moving objects by analyzing the positions of objects across multiple consecutive frames. For example, if the position of an object changes between two adjacent frames, it indicates the presence of a moving object; the speed of the moving object can then be determined based on the amount of position change and the duration between the two adjacent frames.

[0042] Hardware fusion algorithm: A hardware algorithm that can be executed by a camera's sensor. A camera supporting staggered HDR technology outputs a set of exposure frames at a time. The camera's sensor uses a hardware fusion algorithm to fuse these exposure frames into a single frame. When a set of exposure frames includes both long and short frames, the camera's sensor can use the hardware fusion algorithm to fuse the long and short frames into a single frame. In this embodiment, the image after fusing the long and short frames can be called the fused frame. The hardware fusion algorithm can preserve the advantages of both long and short frames. For example, if long and short frames have different dynamic ranges, the fused frame generated by the hardware fusion algorithm will retain the dynamic range of both long and short frames; that is, the fused frame has a higher dynamic range than either the long or short frames individually.

[0043] Multi-frame fusion algorithm: A software processing algorithm that can be executed by the GPU (graphics processing unit) of an electronic device. The multi-frame fusion algorithm can merge any number of exposure frames into a single frame. When the exposure levels of the multiple exposure frames are different, the multi-frame fusion algorithm can obtain a single image with a higher dynamic range. When the exposure levels of the multiple exposure frames are the same, the multi-frame fusion algorithm can obtain a single image with a higher signal-to-noise ratio and higher image quality. The input to the multi-frame fusion algorithm includes a reference frame and one or more exposure frames to be fused. The content of the output frame of the multi-frame fusion algorithm is consistent with that of the reference frame.

[0044] The application background of the exposure control method provided in the embodiments of this application will be introduced below based on the above-mentioned technical terms.

[0045] When users watch videos on mobile phones and other electronic devices, the frame rate threshold at which they perceive video stuttering is 30 frames per second. In other words, when the frame rate is higher than 30 frames per second, users will not perceive stuttering. However, when the frame rate is lower than 30 frames per second, users will perceive stuttering. Therefore, mobile phones should strive to ensure that the frame rate of videos played or displayed is above 30 frames per second.

[0046] For example, when the video frame is a preview displayed on the camera interface, the camera should ensure that the frame rate of the preview is greater than or equal to 30 frames per second. The rate at which the camera transmits display frames to the display screen should be greater than or equal to 30 frames per second, or in other words, the time it takes for the camera to generate display frames should be less than 1 / 30 of a second, approximately 33.3 ms. A display frame is a single frame of image transmitted by the camera to the display screen for display purposes.

[0047] In other words, to ensure the smoothness of the preview, the time it takes for the camera to generate a single display frame should be less than 33.3ms. For example, when a camera supporting line interleaving high dynamic range technology obtains a display frame using two different exposure frames—a long frame and a short frame—the sum of the exposure duration of the long frame, the exposure duration of the short frame, and the readout time of both the long and short frames must be less than or equal to 33.3ms. The readout time of the long and short frames is approximately 1.9ms. That is, the sum of the exposure durations of the long and short frames must be less than or equal to 31.4ms. In this embodiment, the long frame can also be referred to as the first exposure frame, and the short frame as the second exposure frame. It should be noted that a set of frames generated by a camera supporting staggered HDR technology can also include more exposure frames, such as three or four exposure frames. This embodiment uses two frames as an example.

[0048] The images the camera is trying to capture may be in complex scenes, such as high dynamic range scenes or scenes with moving objects. In these complex scenes, cameras supporting staggered HDR technology may suffer from poor dynamic range in their imaging.

[0049] To address the aforementioned issues, this application provides an exposure control method and electronic device that can apply a strategy to the camera to enhance the dynamic range of the preview and / or captured images when the proportion of overexposed and underexposed areas in the camera's preview image exceeds a preset ratio, thereby improving the display effect of the preview and / or captured images.

[0050] In this embodiment, the electronic device can be a portable terminal with a camera, such as a mobile phone, tablet computer, wearable device (such as a smartwatch), or in-vehicle device. The camera supports staggered HDR technology.

[0051] As an example, please refer to Figure 1 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. The exposure control method provided in this embodiment can be applied to, for example... Figure 1 In the electronic device 100 shown.

[0052] like Figure 1 As shown, the electronic device 100 may include a processor 101, a display screen 102, a power management module 103, a battery 104, a sensor module 105, buttons 106, a camera 107, and internal memory 108, etc.

[0053] The processor 101 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video stream codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors 101.

[0054] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0055] The processor 101 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 101 is a cache memory. This memory can store instructions or data that the processor 101 has just used or that are used repeatedly. If the processor 101 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 101, and thus improves the efficiency of the system.

[0056] When the processor 101 executes the instructions stored in the memory, it can cause the electronic device 100 to execute the exposure control method provided in the embodiments of this application.

[0057] Electronic device 100 implements image display function through GPU, display screen 102, and application processor. GPU is a microprocessor for image processing, which has functions such as geometric calculation and graphics rendering.

[0058] Display screen 102 is used to display images, video streams, etc. For example, display screen 102 may display images captured by camera 107.

[0059] The power management module 103 receives input from the battery 104 and supplies power to the processor 101, display screen 102, camera 107, etc. The power management module 103 can also monitor parameters such as battery 104 capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 103 may also be located within the processor 101.

[0060] The battery 104 is used to receive external power input and store power, and also to output the stored power to components such as the processor 101 and the display screen 102 through the power management module 103.

[0061] The sensor module 105 may include components such as a flicker sensor, motion sensing sensor, touch sensor, pressure sensor, gyroscope sensor, accelerometer, distance sensor, ambient light sensor, fingerprint sensor, and temperature sensor to realize the function of sensing and / or acquiring different signals.

[0062] Buttons 106 include a power button, volume buttons, etc. Buttons 106 can be mechanical buttons or touch-sensitive buttons. Electronic device 100 can receive input from buttons 106 and generate key signal inputs related to user settings and function control of electronic device 100. For example, electronic device 100 can activate the camera in response to a user double-clicking button 106.

[0063] Camera 107 is used to capture photos and videos. In this embodiment, camera 107 can be a camera that supports staggered HDR technology.

[0064] Internal memory 108 can be used to store computer executable program code, which includes instructions. Processor 101 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 108.

[0065] The internal memory 108 may also store one or more computer programs corresponding to the exposure control method provided in the embodiments of this application.

[0066] It should be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 100. In other embodiments, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0067] The above has been approved. Figure 1 The hardware structure of the electronic device provided in the embodiments of this application is described. Figure 1 The electronic device shown can be equipped with an operating system, which enables functions such as communication, audio and video playback, taking photos, and recording videos. This operating system may include, but is not limited to, [other functions]. And so on. The following uses a layered architecture. Taking the system as an example, the software architecture of the electronic device provided in the embodiments of this application is illustrated.

[0068] Please refer to Figure 2This is a schematic diagram of the software architecture of an electronic device provided in an embodiment of this application. Figure 2 As shown, the software architecture of electronic device 200 can be divided into four layers, from top to bottom: application layer 201, framework layer 202, hardware abstraction layer 203, and driver layer 204.

[0069] Application layer 201 may include all applications installed in the electronic device, such as camera applications.

[0070] For example, an electronic device may send a start command to the camera in response to a user's click, thereby activating the camera.

[0071] Applications in Application Layer 201 are all developed based on Framework Layer 202. Framework Layer 202 includes a large number of Application Programming Interfaces (APIs) available for developers to use. Developers can use these APIs to develop various applications in Application Layer 201. For example, APIs in Framework Layer 202 may include camera management services, etc.

[0072] The Hardware Abstraction Layer (HAL) 203, located between the Framework Layer 202 and the Driver Layer 204, is used to abstract the hardware in electronic devices. Specifically, the HAL 203 can encapsulate the drivers of the Driver Layer 204 into generic interfaces that can be called by the Framework Layer 202. These generic interfaces are compatible with various types of hardware. This enables the operating system of electronic devices to be hardware-independent, improving the applicability to various hardware types.

[0073] In this embodiment, the hardware abstraction layer 203 may include a camera hardware abstraction layer, etc. The driver corresponding to the camera hardware abstraction layer is a camera driver, the corresponding hardware is a camera, and the API in the corresponding framework layer 202 is a camera management service.

[0074] The driver layer 204 is used to drive the corresponding hardware. In this embodiment, the driver layer may include a camera driver, etc. The camera driver is used to drive the camera 107 to work.

[0075] The above has been approved. Figure 2 The software architecture of the electronic device provided in the embodiments of this application is described. It should be understood that... Figure 2 The software architecture shown does not constitute a specific limitation on the electronic device. In other embodiments, the electronic device may include... Figure 2 More or fewer layers are not specifically defined here.

[0076] The exposure control method provided in this application is applied to an electronic device. The hardware structure of this electronic device can be as follows: Figure 1 As shown, the software architecture of this electronic device can be as follows: Figure 2 As shown. For example, this exposure control method can be applied to camera applications in electronic devices.

[0077] It should be noted that the electronic device used in the exposure control method provided in this application includes a camera with staggered HDR technology. This staggered HDR technology supports outputting a set of exposure frames including at least two different exposure durations. Taking two as an example, the two exposure frames are referred to as the first exposure frame and the second exposure frame, respectively. In this application embodiment, the first exposure frame can also be referred to as a long frame, and the second exposure frame can also be referred to as a short frame.

[0078] The sum of the exposure duration of the first exposure frame and the exposure duration of the second exposure frame is less than a preset duration. The preset duration can be 33ms minus the readout time of the first and second exposure frames in the above embodiment, such as 31.4ms, etc., and is not specifically limited here.

[0079] Based on the above description of the hardware structure and software architecture of the electronic device, the exposure control method provided in the embodiments of this application will be introduced below.

[0080] Please refer to Figure 3 This is a flowchart illustrating an exposure control method provided in an embodiment of this application. Figure 3 As shown, the process may include the following steps.

[0081] S301, Start the camera to take a picture.

[0082] There are several ways to launch the camera to take a picture, such as clicking the camera icon on the desktop of the electronic device, or sliding the camera icon on the lock screen of the electronic device.

[0083] S302. In response to detecting scene parameters of the camera's preview image, determine the exposure strategy corresponding to the scene parameters.

[0084] In this context, the camera preview screen refers to the screen displayed in the preview interface of the camera application. Taking a mobile phone as an example, the phone can launch the camera application and enter the preview interface in response to the user clicking the camera icon on the phone's home screen; or, the phone can launch the camera application and enter the preview interface in response to the user double-clicking the power button. The screen displayed in the camera's preview interface is the camera preview screen. In this embodiment of the application, the camera preview screen may also be referred to as the preview screen, etc., and will not be elaborated further thereafter.

[0085] After the camera is activated to take a picture, the camera app controls the camera to output exposure frames based on the default exposure strategy. The camera app then generates either a preview image or the captured image based on this exposure frame. The captured image refers to the image generated by the camera after the user clicks the shooting control in the camera app. The exposure strategy specifies the exposure duration for each exposure frame, as well as the strategies for generating the preview image and the captured image.

[0086] Scene parameters in the camera preview may include, but are not limited to, at least one of the following: the proportion of overexposed and underexposed areas to the overall area of ​​the preview, the presence and speed of moving objects, the presence of bright and dark stripes, and the light intensity. The proportion of overexposed and underexposed areas to the overall area can also be referred to as proportion information; the presence and speed of moving objects can also be referred to as motion information; the presence and speed of bright and dark stripes can also be referred to as stripe information; and the light intensity can also be referred to as illuminance information.

[0087] In this embodiment, the electronic device can determine the proportion of overexposed and underexposed areas in the preview image to the overall area through the AE algorithm (i.e. the automatic exposure algorithm in the aforementioned embodiment), determine whether there is a moving object and the speed of the moving object in the preview image through the motion detection algorithm, determine whether there is banding in the preview image through the flicker sensor and the current exposure time, and determine the light intensity of the light in the preview image through the ambient light sensor.

[0088] Each scene parameter corresponds to a different scene. For example, if the proportion of overexposed and underexposed areas in the preview image is greater than a preset proportion, it indicates that the scene the camera intends to capture is a High Dynamic Range (HDR) scene. If the proportion of overexposed and underexposed areas in the preview image is less than a preset proportion, it indicates that the scene the camera intends to capture is a non-HDR scene. The presence of moving objects in the preview image indicates that the scene the camera intends to capture is a moving scene. The absence of moving objects in the preview image indicates that the scene the camera intends to capture is a non-moving scene. The presence of banding in the preview image indicates that the scene the camera intends to capture is a banding scene. The absence of banding in the preview image indicates that the scene the camera intends to capture is a non-banding scene. If the light intensity in the preview image is greater than a preset illuminance, it indicates that the scene the camera intends to capture is a high-illuminance scene. If the light intensity in the preview image is less than a preset illuminance, it indicates that the scene the camera intends to capture is a low-illuminance scene.

[0089] It should be understood that when scene parameters include multiple parameters such as proportion information, motion information, stripe information, and illuminance information, the scene indicated by the scene parameters also includes multiple scenes. For example, if the scene parameters include the proportion of overexposed and underexposed areas in the preview image to the overall area, the presence of moving objects, and the speed of the moving objects, then the scene indicated by the scene parameters includes HDR scene + motion scene, non-HDR scene + motion scene, HDR scene + non-motion scene, and non-HDR scene + non-motion scene.

[0090] In this embodiment of the application, the electronic device can pre-store a preset correspondence and determine the exposure strategy corresponding to the scene parameters in the camera preview screen in the preset correspondence.

[0091] The preset mapping relationships include the correspondence between different scene parameters and different exposure strategies, or they can include the correspondence between different shooting scenes and different exposure strategies. These preset mapping relationships can be stored in the electronic device in the form of tables, key-value pairs, or similar formats.

[0092] For example, some preset correspondences can be shown in Table 1 below. Here, a ratio information of 1 indicates that the ratio information in the preview image is greater than the preset ratio, meaning that the proportion of overexposed and underexposed areas in the preview image is greater than the preset ratio; a ratio information of 0 indicates that the ratio information in the preview image is less than the preset ratio. The preset ratio can be 10%, 15%, etc., and is not limited here. A motion information of 1 indicates that there is a moving object in the preview image; a motion information of 0 indicates that there is no moving object in the preview image. A stripe information of 1 indicates that there are bright and dark stripes in the preview image; a stripe information of 0 indicates that there are no bright and dark stripes in the preview image. An illuminance information of 1 indicates that the light intensity in the preview image is greater than the preset illuminance; an illuminance information of 0 indicates that the light intensity in the preview image is less than the preset illuminance. The preset illuminance can be 145 lv, 150 lv, 155 lv, etc., and is not limited here.

[0093]

[0094]

[0095]

[0096] Table 1

[0097] In Table 1 above, the preset exposure time is less than or equal to 13ms, and n is an integer greater than or equal to 2. It should be noted that the exposure time and gain value of the exposure frame are converged by the AE algorithm; adjusting the exposure time of the exposure frame in the above strategy will not change the exposure amount of the exposure frame. The following is a detailed introduction to the exposure strategies corresponding to each scene parameter in Table 1. First, we introduce the exposure strategy corresponding to the scene parameters where the scale information is less than the preset scale, there are no moving objects, no bright or dark stripes, and the light intensity is greater than the preset illuminance, which is Strategy 1 in Table 1 above.

[0098] As shown in Table 1, Strategy 1 involves setting the exposure duration and gain value of the second exposure frame to be the same as those of the first exposure frame; generating a preview image based on the first exposure frame; selecting the first exposure frame as a reference frame; and generating a shooting image by combining the first and second exposure frames using a multi-frame fusion algorithm.

[0099] Generating a preview image based on the first exposure frame means sending the first exposure frame to the display and generating a preview image.

[0100] After the camera's sensor detects light information, it continuously generates exposure frames based on this information, forming an exposure frame sequence. Selecting the first exposure frame means choosing one frame from this sequence. It should be understood that the first exposure frame sequence includes multiple consecutive first exposure frames.

[0101] For the reference frame, the multi-frame fusion algorithm can be described in the foregoing embodiments or related technologies, and will not be repeated here.

[0102] In Strategy 1, the first and second exposure frames use the same exposure duration and gain value. When generating the captured image, the first exposure frame is used as the reference frame, and a multi-frame fusion algorithm is employed to fuse the first and second exposure frames. Since the first and second exposure frames belong to the same set of exposure frames, the time interval between frames is short, which facilitates registration and can avoid or reduce ghosting in the captured image generated by the multi-frame fusion algorithm.

[0103] For example, the camera's image generation algorithm requires 4 exposure frames to generate the image. Please refer to... Figure 4 This is a schematic diagram of camera frame output provided in an embodiment of this application. Figure 4 As shown, the camera generates 7 sets of output frames over a period of time. These 7 sets of output frames include 7 first exposure frames and 7 second exposure frames. The 7 second exposure frames are numbered S1 to S7, and the 7 first exposure frames are numbered L1 to L7. First exposure frames and second exposure frames with the same number are from the same set of output frames.

[0104] In Strategy 1, the electronic device can send the first exposure frame to the display to generate a preview image, and select two sets of first and second exposure frames to input into a multi-frame fusion algorithm to generate the captured image. For example, the electronic device can select... Figure 4 The S5, S6, L5, and L6 inputs are used in the multi-frame fusion algorithm, where L5 or L6 can be selected as the reference frame.

[0105] The scene parameters for Strategy 2 are similar to those for Strategy 1, the difference being the different stripe information. Specifically, when the scene parameters include a scale less than a preset scale, no moving objects, the presence of bright and dark stripes, and light intensity exceeding a preset illuminance, Strategy 2 can be implemented as follows: Set the exposure duration and gain value of the second exposure frame to be the same as the first exposure frame, with the exposure duration of the first exposure frame required to be an integer multiple of the light source's flicker period; generate a preview image based on the first exposure frame, select the first exposure frame as a reference frame, and use a multi-frame fusion algorithm to generate the captured image from the first and second exposure frames. Strategy 2 has similar beneficial effects to Strategy 1, the difference being that by setting the exposure duration of the first exposure frame to an integer multiple of the light source's flicker period, Strategy 2 can also eliminate or reduce bright and dark stripes in the preview and captured images.

[0106] The following describes the exposure strategy corresponding to the scene parameters where the scale information is less than the preset scale, there are no moving objects, no bright or dark stripes, and the light intensity is less than the preset illuminance, namely Strategy 3 in Table 1 above.

[0107] As shown in Table 1, Strategy 3 is to set the exposure time of the second exposure frame to a preset exposure time or one-nth of the exposure time of the first exposure frame, thereby increasing the exposure time of the first exposure frame; generate a preview image based on the first exposure frame, select the first exposure frame as a reference frame, and generate the shooting image from the first exposure frame through a multi-frame fusion algorithm.

[0108] Specifically, generating a preview image based on the first exposure frame means sending the first exposure frame to the display to generate a preview image. Selecting the first exposure frame means choosing a specific first exposure frame from the sequence of first exposure frames.

[0109] In Strategy 3, either the preset exposure time or one-nth of the exposure time of the first exposure frame is a relatively small exposure time. Setting the exposure time of the second exposure frame to a smaller value expands the selectable range of exposure times for the second exposure frame. When generating the captured image, the first exposure frame is used as the reference frame, and a multi-frame fusion algorithm is employed to fuse the first exposure frame to obtain the captured image. This helps improve the signal-to-noise ratio of the captured image and enhances its image quality.

[0110] For example, the camera's image generation algorithm requires four exposure frames to generate the captured image. Therefore, in strategy 3, the electronic device can send the first exposure frame to the display to generate a preview image, and then select four of the first exposure frames to input into a multi-frame fusion algorithm to generate the captured image. For example, the electronic device can select... Figure 4 The L3, L4, L5, and L6 inputs are used in the multi-frame fusion algorithm, where L5 or L6 can be selected as the reference frame.

[0111] The scene parameters for Strategy 4 are similar to those for Strategy 3, except for the different stripe information. Similar to Strategy 3, when the scene parameters show a scale greater than a preset scale, no moving objects, bright and dark stripes, and light intensity less than a preset illuminance, Strategy 4 can be: setting the exposure time of the second exposure frame to a preset exposure time or one-nth of the exposure time of the first exposure frame; increasing the exposure time of the first exposure frame, which must be an integer multiple of the flicker period of the light source; generating a preview image based on the first exposure frame; selecting the first exposure frame as a reference frame; and generating the captured image from the first exposure frame using a multi-frame fusion algorithm.

[0112] The following describes the exposure strategy corresponding to the scene parameters where the scale information is less than the preset scale, there is a moving object with the first speed, there are no bright or dark stripes, and the light intensity is less than the preset illuminance, namely strategy 5 in Table 1 above.

[0113] As shown in Table 1, Strategy 5 sets the exposure time of the second exposure frame to a preset exposure time or one-nth of the exposure time of the first exposure frame, reduces the exposure time of the first exposure frame based on the first speed, generates a preview image based on the first exposure frame, selects the first exposure frame as a reference frame, and generates a shooting image from the first exposure frame through a multi-frame fusion algorithm.

[0114] Specifically, generating a preview image based on the first exposure frame means sending the first exposure frame to the display to generate a preview image. Selecting the first exposure frame means choosing a specific first exposure frame from the sequence of first exposure frames.

[0115] In Strategy 5, a preset exposure time or one-nth of the exposure time of the first exposure frame is considered a small exposure time. Setting the exposure time of the second exposure frame to a smaller exposure time expands the selectable range of the second exposure frame's exposure time. Reducing the exposure time of the first exposure frame based on the first speed can include: determining the exposure time reduction mapping value corresponding to the first speed; dividing the exposure time of the first exposure frame by the exposure time reduction mapping value to obtain the new exposure time of the first exposure frame. Reducing the exposure time of the first exposure frame helps to reduce motion blur in the preview and captured images. The correspondence between the first speed and the exposure time reduction mapping value can be pre-stored in the electronic device.

[0116] For example, the exposure time reduction mapping value corresponding to the first speed is ratio; the exposure time of the first exposure frame before adjustment is EL1, and the gain value is GL1; the exposure time of the first exposure frame after adjustment is EL2, and the gain value is GL2; then, without changing the exposure amount, after reducing the exposure time of the first exposure frame by the first speed, EL1, GL1, EL2, and GL2 satisfy the following formula (1).

[0117]

[0118] Furthermore, in Strategy 5, the first exposure frame is used as the reference frame when generating the captured image, and a multi-frame fusion algorithm is employed to fuse the first exposure frame to obtain the captured image. This helps to improve the signal-to-noise ratio of the captured image and enhance its image quality.

[0119] For example, the camera's image generation algorithm requires four exposure frames to generate the captured image. Therefore, in strategy 5, the electronic device can send the first exposure frame to the display to generate a preview image, and then select four of the first exposure frames to input into a multi-frame fusion algorithm to generate the captured image. For example, the electronic device can select... Figure 4 The L3, L4, L5, and L6 inputs are used in the multi-frame fusion algorithm, where L5 or L6 can be selected as the reference frame.

[0120] The following describes the exposure strategy corresponding to the scene parameters where the scale information is less than the preset scale, there is a moving object with the first speed, there are no bright or dark stripes, and the light intensity is greater than the preset illuminance, namely strategy 6 in Table 1 above.

[0121] As shown in Table 1, Strategy 6 is to reduce the exposure time of the first exposure frame based on the first speed, set the exposure time and gain value of the second exposure frame to be the same as the first exposure frame; generate a preview image based on the first exposure frame, select the first exposure frame as the reference frame, and generate the shooting image by combining the first exposure frame and the second exposure frame through a multi-frame fusion algorithm.

[0122] Specifically, generating a preview image based on the first exposure frame means sending the first exposure frame to the display to generate a preview image. Selecting the first exposure frame means choosing a specific first exposure frame from the sequence of first exposure frames.

[0123] In Strategy 6, the first and second exposure frames use the same exposure duration and gain value. When generating the captured image, the first exposure frame is used as the reference frame, and a multi-frame fusion algorithm is employed to fuse the first and second exposure frames. Since the first and second exposure frames are the same exposure frame, the time interval between the frames is short, which facilitates registration and can avoid or reduce ghosting in the captured image generated by the multi-frame fusion algorithm.

[0124] In addition, reducing the exposure time of the first exposure frame based on the first speed can reduce motion blur in the preview and shooting images. For specific solutions, please refer to the relevant explanation in Strategy 5, which will not be elaborated here.

[0125] For example, the camera's image generation algorithm requires four exposure frames to generate the captured image. Therefore, in strategy 6, the electronic device can send the first exposure frame to the display to generate a preview image, and select two sets of first and second exposure frames to input into a multi-frame fusion algorithm to generate the captured image. For example, the electronic device can select... Figure 4 The S5, S6, L5, and L6 inputs are used in the multi-frame fusion algorithm, where L5 or L6 can be selected as the reference frame.

[0126] The following describes the exposure strategy for scene parameters where the scale information is less than the preset scale, there is a moving object with the speed of first velocity, and there are bright and dark stripes, namely Strategy 7 in Table 1 above. It can be seen that Strategy 7 has no corresponding relationship with the illumination information.

[0127] As shown in Table 1, Strategy 7 is to reduce the exposure time of the second exposure frame based on the first speed, set the exposure time of the first exposure frame to an integer multiple of the flicker period of the light source; generate a preview image based on the first exposure frame, correct the bright and dark stripes in the second exposure frame based on the stripe removal algorithm and the first exposure frame, select the corrected second exposure frame as the reference frame, and generate the shooting image by combining the first exposure frame and the second exposure frame through a multi-frame fusion algorithm.

[0128] In Strategy 7, reducing the exposure time of the second exposure frame based on the first speed can eliminate or reduce motion blur in the preview and captured images. Setting the exposure time of the first exposure frame to an integer multiple of the light source's flicker period can eliminate or reduce bright and dark stripes in the preview image. Based on the stripe removal algorithm and the correction of bright and dark stripes in the second exposure frame using the first exposure frame, the corrected second exposure frame is selected as the reference frame. The first and second exposure frames are then fused using a multi-frame fusion algorithm to generate the captured image. This not only eliminates or reduces bright and dark stripes in the preview and captured images but also avoids or reduces ghosting in the captured image generated by the multi-frame fusion algorithm.

[0129] In addition, for specific solutions on reducing the exposure time of the second exposure frame based on the first speed, please refer to the relevant explanation in Strategy 5, which will not be elaborated here.

[0130] For example, the camera's image generation algorithm requires four exposure frames to generate the captured image. Therefore, in strategy 7, the electronic device can send the first exposure frame to the display to generate a preview image, and select two sets of first and second exposure frames to input into a multi-frame fusion algorithm to generate the captured image. For example, the electronic device can select... Figure 4 The S5, S6, L5, and L6 inputs are used in the multi-frame fusion algorithm. The S5 can be corrected based on the stripe removal algorithm and L5, and then the corrected S5 is used as the reference frame.

[0131] The following describes the exposure strategy corresponding to scene parameters where the scale information is greater than the preset scale, there are no moving objects, and there are no bright or dark stripes, i.e., strategy 8 in Table 1 above. It can be seen that strategy 8 has no corresponding relationship with the illumination information.

[0132] As shown in Table 1 above, Strategy 8 includes two strategies. One strategy is to reduce the exposure of the second exposure frame; a preview image and a shooting image are generated based on the fused frame of the first and second exposure frames. The other strategy is to reduce the exposure of the second exposure frame; a preview image is generated based on the fused frame of the first and second exposure frames, the first exposure frame is selected as the reference frame, and the first and second exposure frames are fused using a multi-frame fusion algorithm to generate the shooting image.

[0133] In the first strategy of Strategy 8, by reducing the exposure of the second exposure frame and generating a preview and captured image based on the fused frame of the first and second exposure frames, the dynamic range of the preview and captured images is improved. In the second strategy of Strategy 8, by reducing the exposure of the second exposure frame and generating a preview image based on the fused frame of the first and second exposure frames, the dynamic range of the preview image is improved. By reducing the exposure of the second exposure frame and selecting the first exposure frame as the reference frame, the captured image is generated using a multi-frame fusion algorithm, which improves the dynamic range of the captured image. Furthermore, since the first and second exposure frames are the same exposure frame, the time interval between frames is short, which is beneficial for registration and can avoid or reduce ghosting in the captured image generated by the multi-frame fusion algorithm.

[0134] In addition, the exposure amount of the second exposure frame is reduced and the exposure time is shorter, so the exposure time of the first exposure frame can be adjusted within a wide range in low-light scenes.

[0135] For example, the camera's image generation algorithm requires four exposure frames to generate the captured image. In the first strategy of Strategy 8, the electronic device can send a fused frame of four first exposure frames and four second exposure frames to the display to generate a preview image, and then send it to the camera to generate the captured image. In the second strategy of Strategy 8, the electronic device can send a fused frame of four first exposure frames and four second exposure frames to the display to generate a preview image, and then select two sets of first and second exposure frames to input into a multi-frame fusion algorithm to generate the captured image. For example, the electronic device can select... Figure 4The input multi-frame fusion algorithm uses S5, S6, L5, and L6, where L5 can be selected as the reference frame.

[0136] The scene parameters for Strategy 9 are similar to those for Strategy 8, the difference being the corresponding stripe information. Specifically, when the scene parameters have a scale greater than a preset scale, no moving objects, and bright and dark stripes exist, Strategy 9 can be: reducing the exposure of the second exposure frame and setting the exposure duration of the first exposure frame to an integer multiple of the light source's flicker period; generating a preview and a captured image based on the fused frame of the first and second exposure frames; or, selecting the first exposure frame as a reference frame and generating the captured image by fusing the first and second exposure frames using a multi-frame fusion algorithm. Strategy 9 has similar beneficial effects to Strategy 8, the difference being that by setting the exposure duration of the first exposure frame to an integer multiple of the light source's flicker period, Strategy 9 can also eliminate or reduce bright and dark stripes in the preview and captured images.

[0137] The following describes the exposure strategy for scene parameters where the scale information is greater than the preset scale, there is a moving object with the speed of the first velocity, and there are no bright or dark stripes, which is strategy 10 in Table 1 above. It can be seen that strategy 10 has no corresponding relationship with the illumination information.

[0138] As shown in Table 1 above, strategy 10 also includes two strategies. The first strategy is to reduce the exposure of the second exposure frame by reducing the exposure duration of the first exposure frame based on the first speed; and to generate a preview image and a shooting image based on the fused frame of the first and second exposure frames. The second strategy is to reduce the exposure of the second exposure frame by reducing the exposure duration of the first exposure frame based on the first speed; to generate a preview image based on the fused frame of the first and second exposure frames; and to select the fused frame of the first and second exposure frames or the first exposure frame as a reference frame, and to generate a shooting image by using a multi-frame fusion algorithm on the first and second exposure frames.

[0139] Reducing the duration of the first exposure frame based on the first speed can eliminate or reduce motion blur in the preview and captured images. Specific solutions can be found in the aforementioned embodiments and will not be elaborated upon here. By reducing the exposure of the second exposure frame, the dynamic range that the preview and captured images can present can be improved.

[0140] In the first strategy of Strategy 10, generating the preview and captured images based on the fused frame of the first and second exposure frames can improve the dynamic range of both. In the second strategy of Strategy 10, generating the preview image based on the fused frame of the first and second exposure frames can also improve the dynamic range of the preview image. Taking the fused frame of the first and second exposure frames or the first exposure frame as a reference frame, and generating the captured image from the first and second exposure frames using a multi-frame fusion algorithm, can not only improve the dynamic range of the preview image, but also, since the first and second exposure frames are the same exposure frame, the time interval between frames is short, which is beneficial for registration. It can also avoid or reduce ghosting in the captured image generated by the multi-frame fusion algorithm.

[0141] In addition, the exposure amount of the second exposure frame is reduced and the exposure time is shorter, so the exposure time of the first exposure frame can be adjusted within a wide range in low-light scenes.

[0142] For example, the camera's image generation algorithm requires four exposure frames to generate the captured image. In the first strategy of Strategy 10, the electronic device can send a fused frame of four first exposure frames and four second exposure frames to the display to generate a preview image, and then send it to the camera to generate the captured image. In the second strategy of Strategy 10, the electronic device can send a fused frame of four first exposure frames and four second exposure frames to the display to generate a preview image, and then select two sets of first and second exposure frames to input into a multi-frame fusion algorithm to generate the captured image. For example, the electronic device can select... Figure 4 The S5, S6, L5, and L6 inputs are used in the multi-frame fusion algorithm, where L5 or the fusion frame of L5 and S5 can be selected as the reference frame.

[0143] The following describes the exposure strategy corresponding to the scene parameters where the scale information is greater than the preset scale, there is a moving object with the speed of the first velocity, and there are bright and dark stripes, namely Strategy 11 in Table 1 above. It can be seen that Strategy 11 has no corresponding relationship with the illumination information.

[0144] As shown in Table 1, strategy 11 can be as follows: set the exposure time of the first exposure frame to an integer multiple of the flashing period of the light source, reduce the exposure time of the second exposure frame based on the first speed; generate a preview image based on the first exposure frame, correct the bright and dark stripes in the second exposure frame based on the stripe removal algorithm and the first exposure frame, select the corrected second exposure frame as the reference frame, and generate the shooting image by using a multi-frame fusion algorithm with the first exposure frame, the second exposure frame, and the subsequent frame.

[0145] In Strategy 11, setting the exposure duration of the first exposure frame to an integer multiple of the flashing period of the light source and generating a preview image based on the first exposure frame can avoid the appearance of bright and dark stripes in the preview image.

[0146] By reducing the exposure time of the second exposure frame based on the first speed, motion blur in the captured image can be avoided. For details, please refer to the aforementioned embodiments.

[0147] By using a stripe removal algorithm and correcting the bright and dark stripes in the second exposure frame using the first exposure frame, bright and dark stripes can be avoided in the captured image. The specific process can be referred to the relevant instructions in the aforementioned implementation, and will not be repeated here.

[0148] Selecting the corrected second exposure frame as the reference frame and generating the captured image using a multi-frame fusion algorithm from the first exposure frame, the second exposure frame, and the subsequent exposure frame means using the corrected second exposure frame as the reference frame, and inputting the first exposure frame sequence, the second exposure frame sequence, and the subsequent exposure frame sequence into the multi-frame fusion algorithm to obtain the captured image. The subsequent exposure frame refers to the extra frame output by the camera after the electronic device takes a picture in response to the user's shooting command. For example, the subsequent exposure frame includes a subsequent up-exposure frame and a subsequent down-exposure frame, where the up-exposure frame is used to brighten dark areas, and the down-exposure frame is used to suppress overexposure, thereby improving the dynamic range of the final image. This subsequent exposure frame is not used for display but only for the multi-frame fusion algorithm. In Strategy 10, this subsequent exposure frame can be used to improve the dynamic range of the captured image.

[0149] Please refer to Figure 5 This is a schematic diagram illustrating another type of camera frame output provided in an embodiment of this application. Figure 5 As shown, the camera generates 7 sets of output frames over a period of time. These 7 sets of output frames include 7 first exposure frames and 7 second exposure frames. The 7 second exposure frames are numbered S1 to S7, and the 7 first exposure frames are numbered L1 to L7. First exposure frames and second exposure frames with the same number are from the same set of output frames.

[0150] like Figure 5 As shown, if the shooting time of the electronic device is the time when frames S6 and L6 are output, then in strategy 11, the electronic device can send the first exposure frame to generate a preview screen, select two sets of first and second exposure frames before the shooting time, and one or more exposure frames after the shooting time (i.e., later output frames) to input into the multi-frame fusion algorithm to generate the captured image. For example, the electronic device can select... Figure 5 The S5, S6, L5, L6, and S7 inputs are used in the multi-frame fusion algorithm. The algorithm can be based on the stripe removal algorithm and L5 correction of the bright and dark stripes in S5, and the corrected S5 is selected as the reference frame.

[0151] The above describes the relationship between the scene parameters and exposure strategies provided in the embodiments of this application. After determining the exposure strategy based on the scene parameters in the camera preview, the exposure strategy can be applied in the camera.

[0152] S303, Based on the exposure strategy corresponding to the scene parameters, control the camera to generate preview images and / or capture images.

[0153] In other words, electronic devices can set the camera's exposure strategy to the exposure strategy corresponding to the scene parameters mentioned above. In this way, the camera can generate a preview image or capture an image based on the exposure strategy corresponding to the scene parameters.

[0154] Based on the foregoing description, it should be understood that the exposure control method provided in this application can detect when the proportion of overexposed and underexposed areas in the camera's preview image exceeds a preset proportion. In this case, the method applies a strategy to the camera that enhances the dynamic range of the preview image and / or the captured image, thereby improving the display effect of the preview image and / or the captured image.

[0155] Please refer to Figure 6 This application also provides a schematic diagram of a chip system. This chip system 600 can be installed in an electronic device, such as a mobile phone. Exemplarily, the chip system 600 may include a processor 601 and a communication interface 602, used to support the electronic device in implementing the functions involved in the above embodiments. In one possible design, the chip system 600 also includes a memory for storing necessary program instructions and data for the electronic device. This chip system can be composed of chips or may include chips and other discrete devices. It should be noted that in some implementations of this application, the communication interface 602 may also be referred to as an interface circuit.

[0156] It should be noted that all relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0157] This application also provides a computer storage medium storing computer instructions. When the computer instructions are executed on an electronic device, the electronic device performs the aforementioned method steps to implement the methods described in the above embodiments.

[0158] This application also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement the methods described in the above embodiments.

[0159] Furthermore, embodiments of this application also provide an apparatus, which may specifically be a chip, component, or module. This apparatus may include a connected processor and a memory; wherein the memory stores computer execution instructions, and when the apparatus is running, the processor can execute the computer execution instructions stored in the memory to cause the chip to perform the methods described in the above method embodiments. It should be noted that all related content of each step involved in the above method embodiments can be referenced to the functional description of the corresponding functional module, and will not be repeated here.

[0160] In this application, the electronic devices, computer storage media, computer program products or chips provided in the embodiments are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0161] The foregoing primarily describes the solutions provided by the embodiments of this application from the perspective of electronic devices. To achieve the aforementioned functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0162] This application embodiment can divide the device involved into functional modules according to the above method examples. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0163] The functions, actions, operations, or steps in the above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any combination thereof. When implemented using software programs, they can be implemented, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or include one or more data storage devices such as servers and data centers that can be integrated with the medium. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks, SSDs), etc.

[0164] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

Claims

1. An exposure control method, characterized in that, Applied to electronic devices; the electronic devices include cameras, the cameras supporting line interleaving high dynamic range technology; the line interleaving high dynamic range technology supports outputting a set of exposure frames including a first exposure frame and a second exposure frame, the sum of the exposure durations of the first exposure frame and the second exposure frame being less than or equal to a preset duration; The method includes: Start the camera to take a picture; In response to detecting scene parameters of the camera's preview image, an exposure strategy corresponding to the scene parameters is determined. The scene parameters include proportion information, motion information, and stripe information in the preview image. The proportion information is the ratio of overexposed and underexposed areas to the overall area; the motion information indicates whether there is a moving object in the preview image and the speed of that object; the stripe information indicates whether there are bright and dark stripes in the preview image. The exposure strategy specifies the exposure duration of the first exposure frame and / or the second exposure frame, as well as the strategy for generating the preview image and the strategy for generating the captured image. When the proportion information is greater than a preset proportion, the exposure strategy includes a strategy to improve the dynamic range of the captured image. Based on the exposure strategy corresponding to the scene parameters, the camera is controlled to generate a preview image and / or a captured image.

2. The method according to claim 1, characterized in that, The step of determining the exposure strategy corresponding to the scene parameters includes: when the scene parameters of the preview screen indicate that the proportion information in the preview screen is greater than a preset proportion, there is a moving object with a first speed, and there are no bright or dark stripes, the corresponding exposure strategy includes: reducing the exposure of the second exposure frame, reducing the exposure duration of the first exposure frame based on the first speed; generating a preview screen and a shooting screen based on the fused frame of the first exposure frame and the second exposure frame, or generating a preview screen based on the fused frame of the first exposure frame and the second exposure frame, selecting the fused frame of the first exposure frame and the second exposure frame or the first exposure frame as a reference frame, and generating a shooting screen by using a multi-frame fusion algorithm on the first exposure frame and the second exposure frame.

3. The method according to claim 1, characterized in that, The scene parameters also include illuminance information in the preview image, which is used to indicate the light intensity. The step of determining the exposure strategy corresponding to the scene parameters further includes: When the scene parameters of the preview screen indicate that the scale information in the preview screen is less than the preset scale, there is a moving object with a first speed, there are no bright or dark stripes, and the light intensity is less than the preset illuminance, the corresponding exposure strategy is as follows: set the exposure time of the second exposure frame to the preset exposure time or one-nth of the exposure time of the first exposure frame, reduce the exposure time of the first exposure frame based on the first speed; generate a preview screen based on the first exposure frame, select the first exposure frame as the reference frame, and generate a shooting screen from the first exposure frame through a multi-frame fusion algorithm; wherein, the preset exposure time is less than or equal to 13ms, and n is greater than or equal to 2.

4. The method according to claim 1, characterized in that, The scene parameters also include illuminance information in the preview image, which is used to indicate the light intensity. The step of determining the exposure strategy corresponding to the scene parameters further includes: When the scene parameters of the preview screen indicate that the scale information in the preview screen is less than the preset scale, there is a moving object with a first speed, there are no bright or dark stripes, and the light intensity is greater than the preset illuminance, the corresponding exposure strategy is as follows: reduce the exposure time of the first exposure frame based on the first speed, set the exposure time and gain value of the second exposure frame to be the same as the first exposure frame; generate a preview screen based on the first exposure frame, select the first exposure frame as the reference frame, and generate a shooting screen by combining the first exposure frame and the second exposure frame through a multi-frame fusion algorithm.

5. The method according to claim 1, characterized in that, The step of determining the exposure strategy corresponding to the scene parameters further includes: When the scene parameters in the preview screen indicate that the scale information in the preview screen is less than the preset scale, there is a moving object with a speed of a first speed, and there are bright and dark stripes, the corresponding exposure strategy is as follows: reduce the exposure time of the second exposure frame based on the first speed, set the exposure time of the first exposure frame to an integer multiple of the flicker period of the light source; generate a preview screen based on the first exposure frame, correct the bright and dark stripes in the second exposure frame based on the stripe removal algorithm and the first exposure frame, select the corrected second exposure frame as the reference frame, and generate the shooting screen by combining the first exposure frame and the second exposure frame through a multi-frame fusion algorithm.

6. The method according to claim 1, characterized in that, The step of determining the exposure strategy corresponding to the scene parameters further includes: When the scene parameters in the preview screen indicate that the scale information in the preview screen is greater than the preset scale, there is a moving object with a first speed, and there are bright and dark stripes, the corresponding exposure strategy is as follows: set the exposure time of the first exposure frame to an integer multiple of the flashing period of the light source, and reduce the exposure time of the second exposure frame based on the first speed; generate a preview screen based on the first exposure frame, correct the bright and dark stripes in the second exposure frame based on the stripe removal algorithm and the first exposure frame, select the corrected second exposure frame as the reference frame, and generate the shooting screen by using a multi-frame fusion algorithm with the first exposure frame, the second exposure frame, and the subsequent frame; the subsequent frame refers to the additional frame output by the camera after the electronic device takes a picture.

7. The method according to claim 1, characterized in that, The scene parameters also include illuminance information in the preview image, which is used to indicate the light intensity. The step of determining the exposure strategy corresponding to the scene parameters further includes: When the scene parameters of the preview screen indicate that the scale information in the preview screen is less than the preset scale, there is no object movement, no bright or dark stripes, and the light intensity is greater than the preset illuminance, the corresponding exposure strategy is as follows: set the exposure duration and gain value of the second exposure frame to be the same as the first exposure frame; generate a preview screen based on the first exposure frame, select the first exposure frame as the reference frame, and generate a shooting screen by combining the first exposure frame and the second exposure frame through a multi-frame fusion algorithm.

8. The method according to claim 1, characterized in that, The scene parameters also include illuminance information in the preview image, which is used to indicate the light intensity. The step of determining the exposure strategy corresponding to the scene parameters further includes: When the scene parameters of the preview screen indicate that the scale information in the preview screen is less than the preset scale, there is no object movement, no bright or dark stripes, and the light intensity is less than the preset illuminance, the corresponding exposure strategy is as follows: set the exposure time of the second exposure frame to the preset exposure time or one-nth of the exposure time of the first exposure frame, and increase the exposure time of the first exposure frame; generate a preview screen based on the first exposure frame, select the first exposure frame as the reference frame, and generate a shooting screen from the first exposure frame through a multi-frame fusion algorithm; wherein, the preset exposure time is less than or equal to 13ms, and n is greater than or equal to 2.

9. The method according to claim 1, characterized in that, The step of determining the exposure strategy corresponding to the scene parameters further includes: When the scene parameters of the preview screen indicate that the proportion information in the preview screen is greater than the preset proportion, there are no moving objects, and there are no bright or dark stripes, the corresponding exposure strategy is: reduce the exposure of the second exposure frame; generate a preview screen and a shooting screen based on the fused frame of the first exposure frame and the second exposure frame, or generate a preview screen based on the fused frame of the first exposure frame and the second exposure frame, select the first exposure frame as the reference frame, and generate a shooting screen by using a multi-frame fusion algorithm on the first exposure frame and the second exposure frame.

10. The method according to any one of claims 7-9, characterized in that, The method further includes: When the scene parameters of the preview screen indicate that there are bright and dark stripes in the preview screen, the exposure duration of the first exposure frame in the corresponding exposure strategy satisfies the following: the exposure duration of the first exposure frame is an integer multiple of the light source flicker period.

11. The method according to any one of claims 2-4, characterized in that, The reduction of the exposure duration of the first exposure frame based on the first speed includes: Determine the exposure time reduction mapping value corresponding to the first speed; The new exposure duration of the first exposure frame is obtained by calculating the quotient of the exposure duration of the first exposure frame and the mapping value of the reduction in exposure time. The new gain value of the first exposure frame is obtained by multiplying the gain value of the first exposure frame by the exposure time reduction mapping value.

12. The method according to any one of claims 5-6, characterized in that, The reduction of the exposure duration of the second exposure frame based on the first speed includes: Determine the exposure time reduction mapping value corresponding to the first speed; The new exposure duration of the second exposure frame is obtained by calculating the quotient of the exposure duration of the second exposure frame and the mapping value of the reduction in exposure time. The new gain value of the second exposure frame is obtained by multiplying the gain value of the second exposure frame by the exposure time reduction mapping value.

13. The method according to any one of claims 1-12, characterized in that, The preset duration is equal to 33ms minus the readout duration of the first exposure frame and the second exposure frame.

14. An electronic device, characterized in that, The electronic device includes one or more processors, one or more memories, and a camera that supports line interleaving high dynamic range technology; the one or more processors are connected to the camera, the one or more memories are coupled to the one or more processors, and the one or more memories store a computer program; When the one or more processors execute the computer program, the electronic device performs the exposure control method as described in any one of claims 1-13.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a computer program that, when executed, performs the exposure control method as described in any one of claims 1-13.

16. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method as described in any one of claims 1-13.

17. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, performs the steps of the method as described in any one of claims 1-13.

Images