Image processing method and related device

Abstract

The embodiment of the invention provides an image processing method and a related device, and relates to the technical field of terminals. The method is applied to the electronic equipment comprising a camera, and comprises the following steps: starting the camera in first illumination intensity at a first moment, and setting the exposure time of the camera for acquiring a first-frame image after the first moment as a first value; at the second moment, the camera is started in the second illumination intensity, and the exposure time of the camera for obtaining the first frame image after the second moment is set as a second value; at the third moment, the camera is started in the third illumination intensity, and the exposure time of the camera for obtaining the first frame image after the third moment is set as a third value; wherein the first illumination intensity is smaller than the second illumination intensity, the second illumination intensity is smaller than the third illumination intensity, and the first value, the second value and the third value are all smaller than or equal to a preset threshold value. Therefore, after the camera is started, the exposure time for obtaining the first frame image is smaller than or equal to the preset threshold value, and the interval time from starting of the camera to displaying of the first frame image can be shortened.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to image processing methods and related devices. Background Technology

[0002] Electronic devices, including those with cameras, can perform shooting functions through camera apps, such as taking photos, recording videos, or recording online videos. Regardless of the method used, users need to first open the camera app, activate the camera, and wait for the preview screen to appear.

[0003] In some situations, such as shooting in low-light scenes with low light intensity, such as at night, on cloudy days, or in dark rooms, users often have to wait a long time from opening the camera app to activating the camera and seeing the preview, which degrades the user experience of using the camera app. Summary of the Invention

[0004] The image processing method and related apparatus provided in this application are beneficial for shortening the time interval from starting the camera to displaying the first frame image.

[0005] In a first aspect, embodiments of this application provide an image processing method applied to an electronic device including a camera. The method includes: at a first moment, activating the camera under a first illumination intensity, and setting the exposure time of the first frame image acquired by the camera after the first moment to a first value; at a second moment, activating the camera under a second illumination intensity, and setting the exposure time of the first frame image acquired by the camera after the second moment to a second value; at a third moment, activating the camera under a third illumination intensity, and setting the exposure time of the first frame image acquired by the camera after the third moment to a third value; wherein the first illumination intensity is less than the second illumination intensity, the second illumination intensity is less than the third illumination intensity, and the first value, the second value, and the third value are all less than or equal to a preset threshold value.

[0006] Based on the embodiments of this application, when the camera is activated under the first light intensity, the second light intensity, or the third light intensity, an exposure time less than or equal to a preset threshold value is set for the camera. This can shorten the exposure time for acquiring the first frame image after the camera is activated, speed up the frame output of the first frame image, shorten the time interval from activating the camera to displaying the first frame image, and allow the user to see the preview screen after the camera is turned on more quickly, thereby improving the user's experience of using the camera application.

[0007] In one possible implementation, activating the camera in a first light intensity includes: activating the camera in response to an activation operation for a camera application in an electronic device in the first light intensity; or, activating the camera in response to an operation to switch the shooting mode of a camera application in the first light intensity; or, activating the camera in response to an operation to switch the camera in a camera application in the first light intensity.

[0008] Based on the embodiments of this application, for the operation of starting a camera application, switching the shooting mode of a camera application, or switching the camera in a camera application, the exposure time of the camera can be set to be less than or equal to a preset threshold value, thereby shortening the time interval from starting the camera to displaying the first frame image.

[0009] In one possible implementation, the first value is less than the exposure time of the first frame image after the first moment calculated according to the automatic exposure (AE) algorithm, and the gain value of the first frame image after the first moment acquired by the camera is greater than the gain value of the first frame image after the first moment acquired according to the AE algorithm.

[0010] Based on the embodiments of this application, relative to the exposure time and gain value calculated by the automatic exposure (AE) algorithm, if the first value is set to be less than the exposure time calculated by the AE algorithm and the gain value of the first frame image is greater than the calculated gain value, then the camera's exposure parameters can be configured by using a larger gain value in conjunction with a smaller exposure time. This ensures that the brightness of the first frame image acquired after the camera starts up is not too low and improves the frame output speed of the first frame image. This allows for a convenient way to improve the display speed of the first frame image while meeting the brightness requirements of the first frame image.

[0011] In one possible implementation, setting the exposure time of the first frame image acquired by the camera at the first moment as a first value includes: when it is determined that the image to be acquired at the first moment is the first frame image and the exposure time calculated according to the automatic exposure (AE) algorithm is greater than a preset threshold value, setting the exposure time of the first frame image acquired by the camera at the first moment as a first value; the method further includes: setting the gain value of the first frame image acquired by the camera at the first moment as a fourth value, wherein the brightness of the first frame image acquired based on the first value and the fourth value is the same as the brightness calculated by the AE algorithm.

[0012] Based on the embodiments of this application, when it is determined that the image to be acquired at the first moment is the first frame image, and the exposure time calculated according to the automatic exposure (AE) algorithm is greater than a preset threshold value, an exposure time less than or equal to the preset threshold value can be set for the camera, i.e., the exposure time is set to a first value; in addition, the gain value of the camera can be set to a fourth value. Acquiring the first frame image based on the first value and the fourth value can make the brightness of the first frame image the same as the brightness calculated by the AE algorithm, so that the brightness of the acquired first frame image is close to normal brightness.

[0013] In one possible implementation, setting the exposure time of the first frame image after the camera acquires the third time step as a third value includes: when it is determined that the image to be acquired at the third time step is the first frame image, and the exposure time calculated according to the automatic exposure (AE) algorithm is less than or equal to a preset threshold value, setting the exposure time of the first frame image after the camera acquires the third time step as a third value, wherein the third value is the exposure time calculated according to the automatic exposure (AE) algorithm.

[0014] Based on the embodiments of this application, when the camera is activated at a third light intensity and the exposure time calculated by the first frame image and the AE algorithm is less than or equal to a preset threshold, the exposure time calculated by the automatic exposure AE algorithm can be set for the camera to achieve the purpose of quickly configuring the camera's exposure parameters.

[0015] In one possible implementation, the electronic device includes a target module, which is used to set the exposure time of the first frame image after the camera is started, and to determine the image to be acquired at the first moment as the first frame image, including: determining the image to be acquired at the first moment as the first frame image when the target module has a preset identifier.

[0016] Based on the embodiments of this application, the target module can set the exposure time for acquiring the first frame image after the camera is started. If the target module has a preset identifier, the image to be acquired after the first moment can be conveniently determined as the first frame image based on the preset identifier.

[0017] In one possible implementation, after setting the exposure time of the first frame image after the camera acquires the first moment to a first value, the method further includes: setting the flag of the target module to another flag that is not a preset flag; or, deleting the preset flag of the target module.

[0018] Based on the embodiments of this application, other images acquired after the first frame image are not the first frame images. Therefore, after setting the first value, by setting the flag of the target module to another flag that is not a preset flag, or by deleting the preset flag of the target module, it is easy to determine that other images acquired after the first frame image are not the first frame images.

[0019] In one possible implementation, after the first moment, the method further includes setting the exposure time for the camera to acquire images other than the first frame to the exposure time calculated according to the automatic exposure (AE) algorithm.

[0020] Based on the embodiments of this application, since the user has already seen the preview of the first frame image, there is no problem of a long time interval between starting the camera and seeing the preview for non-first frame images. The exposure time calculated based on the AE algorithm is used to set the exposure time for the camera to acquire non-first frame images, enabling rapid setting of the exposure time.

[0021] Secondly, embodiments of this application provide an image processing apparatus, which may be an electronic device, a chip or chip system within an electronic device. The apparatus may include a processing unit. The processing unit is used to implement any processing-related method executed by the electronic device in the first aspect or any possible implementation of the first aspect. When the apparatus is an electronic device, the processing unit may be a processor. The apparatus may further include a storage unit, which may be a memory. The storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit to cause the electronic device to implement the method described in the first aspect or any possible implementation of the first aspect. When the apparatus is a chip or chip system within an electronic device, the processing unit may be a processor. The processing unit executes the instructions stored in the storage unit to cause the electronic device to implement the method described in the first aspect or any possible implementation of the first aspect. The storage unit may be a storage unit within the chip (e.g., a register, cache, etc.), or a storage unit located outside the chip within the electronic device (e.g., a read-only memory, random access memory, etc.).

[0022] For example, the processing unit is configured to: activate the camera at a first moment under a first illumination intensity and set the exposure time of the first frame image acquired by the camera at the first moment to a first value; activate the camera at a second moment under a second illumination intensity and set the exposure time of the first frame image acquired by the camera at the second moment to a second value; activate the camera at a third moment under a third illumination intensity and set the exposure time of the first frame image acquired by the camera at the third moment to a third value; wherein the first illumination intensity is less than the second illumination intensity, the second illumination intensity is less than the third illumination intensity, and the first value, the second value, and the third value are all less than or equal to a preset threshold value.

[0023] In one possible implementation, the processing unit is configured to activate the camera in response to an activation operation for a camera application in an electronic device at a first light intensity; or, activate the camera in response to an operation to switch the shooting mode of the camera application at a first light intensity; or, activate the camera in response to an operation to switch the camera in the camera application at a first light intensity.

[0024] In one possible implementation, the first value is less than the exposure time of the first frame image after the first moment calculated according to the automatic exposure (AE) algorithm, and the gain value of the first frame image after the first moment acquired by the camera is greater than the gain value of the first frame image after the first moment acquired according to the AE algorithm.

[0025] In one possible implementation, the processing unit is configured to, when determining that the image to be acquired at the first moment is the first frame image and the exposure time calculated according to the automatic exposure (AE) algorithm is greater than a preset threshold, set the exposure time of the first frame image acquired by the camera at the first moment to a first value; the processing unit is further configured to set the gain value of the first frame image acquired by the camera at the first moment to a fourth value, wherein the brightness of the first frame image acquired by the camera at the first moment calculated based on the first value and the fourth value is the same as the brightness calculated by the AE algorithm.

[0026] In one possible implementation, the processing unit is configured to, when determining that the image to be acquired at the third moment is the first frame image and the exposure time calculated according to the automatic exposure (AE) algorithm is less than or equal to a preset threshold, set the exposure time of the first frame image acquired by the camera at the third moment to a third value, wherein the third value is the exposure time calculated according to the automatic exposure (AE) algorithm.

[0027] In one possible implementation, the device includes a target module for setting the exposure time of the first frame image after the camera is started, and a processing unit for determining the image to be acquired at the first moment as the first frame image when the target module is marked with a preset identifier.

[0028] In one possible implementation, the processing unit is further configured to set the target module's flag to another flag that is not a preset flag; or, delete the target module's preset flag.

[0029] In one possible implementation, the processing unit is further configured to set the exposure time for the camera to acquire images other than the first frame to the exposure time calculated according to the automatic exposure (AE) algorithm.

[0030] Thirdly, embodiments of this application provide an electronic device, including a camera, a processor, and a memory. The camera is used to acquire images, the memory is used to store code instructions, and the processor is used to run the code instructions to perform the method described in the first aspect or any possible implementation of the first aspect.

[0031] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program or instructions that, when executed on a computer, cause the computer to perform the methods described in the first aspect or any possible implementation thereof.

[0032] Fifthly, embodiments of this application provide a computer program product including a computer program, which, when run on a computer, causes the computer to perform the methods described in the first aspect or any possible implementation thereof.

[0033] Sixthly, this application provides a chip or chip system including at least one processor and a communication interface. The communication interface and the at least one processor are interconnected via a circuit. The at least one processor is used to run computer programs or instructions to perform the methods described in the first aspect or any possible implementation of the first aspect. The communication interface in the chip can be an input / output interface, pins, or circuits, etc.

[0034] In one possible implementation, the chip or chip system described above in this application further includes at least one memory storing instructions. The memory can be an internal storage unit of the chip, such as a register or cache, or it can be a storage unit of the chip itself (e.g., read-only memory, random access memory, etc.).

[0035] It should be understood that the second to sixth aspects of this application correspond to the technical solutions of the first aspect of this application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation are similar, and will not be repeated here. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0037] Figure 2 A schematic diagram of the software architecture of an electronic device provided in an embodiment of this application;

[0038] Figure 3 A schematic diagram illustrating an image processing method provided in an embodiment of this application;

[0039] Figure 4 A schematic flowchart of an image processing method provided in an embodiment of this application;

[0040] Figure 5 A comparative schematic diagram of camera exposure parameters in the first illumination intensity provided in the embodiments of this application;

[0041] Figure 6 This is a schematic diagram of the structure of a chip provided in an embodiment of this application. Detailed Implementation

[0042] To facilitate a clear description of the technical solutions in the embodiments of this application, some terms and technologies involved in the embodiments of this application will be briefly introduced below:

[0043] 1. AE Algorithm: The Auto Exposure (AE) algorithm is used to automatically calculate and adjust the exposure parameters of the image acquisition device based on the light intensity in the scene and preset exposure tables, so that the image acquisition device can capture an image with appropriate brightness. Exposure parameters may include one or more of the following: aperture, shutter speed, ISO, exposure time, or gain value.

[0044] Exposure time: This refers to the actual duration of time the image sensor of an image acquisition device is exposed to light during image acquisition. Exposure time is usually related to shutter speed. Under the same conditions, a longer exposure time means more light is received by the image sensor, resulting in a brighter image. However, a longer exposure time also means a longer image acquisition time. Conversely, a shorter exposure time means less light is received by the image sensor, resulting in a lower image brightness. Again, a shorter exposure time means a shorter image acquisition time.

[0045] Gain value: This characterizes the degree to which the image signal is amplified by the image acquisition device during image acquisition. Under the same conditions, a smaller gain value results in a smaller amplification factor for the image signal and lower image brightness; conversely, a larger gain value results in a larger amplification factor for the image signal and higher image brightness.

[0046] RAW images refer to the raw images captured by an image acquisition device without any image processing. They reflect the original data captured by the sensor of the image acquisition device. Because RAW images have not undergone image processing by the image acquisition device or image optimization module, their image quality is typically lower. Image quality is generally reflected in aspects such as image sharpness, noise, color accuracy, and contrast.

[0047] ZSL photography: Zero shutter lag (ZSL) photography is a technology designed to eliminate the delay between pressing the shutter button and actually capturing the image. In traditional photography, when the user presses the shutter button, the camera begins to capture an image, which can cause some delay, such as in poor lighting conditions or when the camera needs to adjust its focus. ZSL photography continuously captures and buffers images before the user presses the shutter button, allowing the image to be acquired instantly when the shutter button is pressed, thus achieving a zero-latency effect.

[0048] 2. Terminology

[0049] In the embodiments of this application, terms such as "first" and "second" are used to distinguish identical or similar items with substantially the same function and purpose. For example, "first chip" and "second chip" are used only to distinguish different chips and do not limit their order of execution. Those skilled in the art will understand that terms such as "first" and "second" do not limit the quantity or execution order, and that "first" and "second" do not necessarily imply that they are different.

[0050] It should be noted that, in the embodiments of this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0051] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0052] 3. Electronic equipment

[0053] The electronic device in this application embodiment can be any electronic device including a camera. For example, the electronic device may include: a mobile phone, tablet computer, PDA, laptop computer, mobile internet device (MID), wearable device, virtual reality (VR) device, augmented reality (AR) device, wireless terminal in industrial control, wireless terminal in self-driving, wireless terminal in remote medical surgery, wireless terminal in smart grid, wireless terminal in transportation safety, wireless terminal in smart city, wireless terminal in smart home, cellular phone, cordless phone, session initiation protocol (SIP) phone, wireless local loop (WLL) station, personal digital assistant (PDA), handheld device with wireless communication function, computing device or other processing device connected to a wireless modem, in-vehicle device, electronic device in 5G network, or future evolved public land mobile communication network. The embodiments of this application do not limit the scope of electronic devices in a network (PLMN).

[0054] By way of example and not limitation, in this embodiment, the electronic device may be a wearable device. Wearable devices, also known as wearable smart devices, are a general term for devices that utilize wearable technology to intelligently design and develop everyday wearables, such as glasses, gloves, watches, clothing, and shoes. Wearable devices are portable devices that are worn directly on the body or integrated into the user's clothing or accessories. Wearable devices are not merely hardware devices, but also achieve powerful functions through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include those that are feature-rich, large in size, and can achieve complete or partial functions without relying on a smartphone, such as smartwatches or smart glasses, as well as those that focus on a specific type of application function and require the use of other devices such as smartphones, such as various smart bracelets and smart jewelry for vital sign monitoring.

[0055] Furthermore, in this application embodiment, the electronic device can also be an electronic device in the Internet of Things (IoT) system. IoT is an important part of the future development of information technology. Its main technical feature is to connect objects to the network through communication technology, thereby realizing an intelligent network of human-machine interconnection and object-to-object interconnection.

[0056] The electronic equipment in the embodiments of this application may also be referred to as: user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent, or user device, etc.

[0057] In this embodiment, the electronic device or various network devices include a hardware layer, an operating system layer running on top of the hardware layer, and an application layer running on top of the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and memory (also called main memory). The operating system can be any one or more computer operating systems that implement business processing through processes, such as Linux, Unix, Android, iOS, or Windows. The application layer includes applications such as browsers, address books, word processing software, and instant messaging software.

[0058] In some real-world scenarios, users will use electronic devices to take photos. For example, a user might tap the camera icon on their phone to activate the camera and take a picture of people or objects in the scene.

[0059] When shooting in low-light scenes with low light intensity, there may be a relatively long interval between when the user activates the camera and when the electronic device displays the first frame image. The first frame image can be understood as the first image that the camera waits to capture after an operation that activates the camera application, switches camera modes, or switches cameras.

[0060] Taking mobile phones as an example, in some implementations, to ensure that the preview image displayed on the phone screen has normal brightness after the camera is activated in low-light shooting conditions, the phone usually increases the exposure time. Specifically, after activating the phone's camera in a low-light scene, the camera is configured with a longer exposure time to allow the camera's photosensitive element to obtain sufficient imaging brightness, so that the brightness of the first frame image is normal.

[0061] However, the increased exposure time leads to a longer frame output time for the camera sensor, which in turn delays the image display time. This results in a longer interval between when the user clicks the camera app icon to start the camera and when they see the first frame on the screen. During this longer interval, some implementations will display a blurry animation, while others will display a blank screen. Regardless of the method, users will not be able to see the preview for a long time, which degrades the user experience of using the camera app.

[0062] In scenarios where electronic devices activate their cameras to capture images, users typically don't tap to take a picture very quickly. Therefore, the first raw frame is usually only used to display a preview and doesn't require a full scan (ZSL) shot. So even if the exposure time for the first frame is set short, resulting in lower image quality, it won't significantly impact the user's shooting experience. Therefore, to speed up the camera's image capture and reduce the time interval between activating the camera and displaying the first frame, a shorter exposure time can be set for the camera.

[0063] In view of this, the image processing method provided in this application sets an exposure time for the camera to be less than or equal to a preset threshold value in order to quickly acquire the first frame image after startup. Based on this, the exposure time of the camera can be effectively controlled. Especially in low-light shooting scenarios, it can shorten the light-sensing time of the camera's photosensitive element, that is, shorten the exposure time of the sensor in low light. Since the exposure time is shortened, the frame output speed during image acquisition can be accelerated, the delay between frame output and display can be reduced, and thus the time interval from starting the camera to displaying the first frame image can be reduced. Since this application embodiment shortens the time interval from starting the camera to displaying the first frame image, the time the user waits for the preview image after starting the camera can be reduced, thereby improving the user experience when shooting.

[0064] For example, Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0065] The electronic device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0066] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may include hardware, software, or a combination of software and hardware.

[0067] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0068] Internal memory 121 can be used to store executable program code, including instructions. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of the electronic device (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 110 executes various functional applications and data processing of the electronic device by running instructions stored in internal memory 121 and / or instructions stored in memory located within the processor.

[0069] For example, the internal memory 121 can be used to store executable program code that implements the image processing method of the embodiments of this application. When the processor 110 executes the program code, it can reduce the exposure time calculated by the AE algorithm to less than or equal to the preset threshold value when it is determined that the image to be acquired is the first frame image after the camera is started and the exposure time calculated by the AE algorithm is greater than the preset threshold value. Decision information is output based on the reduced exposure time, and the camera exposure parameters are configured based on the decision information.

[0070] Camera 193 is used to capture still images or videos. In some embodiments, an electronic device may include at least one camera 193. Taking a mobile phone as an example, a mobile phone may include multiple cameras 193, which may be distributed at any location on the phone, such as a front-facing camera on the front of the phone and a rear-facing camera on the back of the phone. Each camera 193 may have the same hardware parameters or different hardware parameters. Each camera 193 may perform actions such as camera activation, camera restart, camera switching, or camera deactivation in response to user operations.

[0071] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. In some embodiments, the electronic device may include one or N displays screens 194, where N is a positive integer greater than 1. The electronic device implements display functions through a GPU, display screen 194, and application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor.

[0072] The electronic device implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 194 and the application processor. The GPU performs mathematical and geometric calculations and is used for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information. The electronic device can implement shooting functions through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0073] For example, in this embodiment of the application, the raw image captured by the camera 193 is processed by rendering, drawing and compositing, and then sent to the display screen 194 for display.

[0074] The software system of an electronic device can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. A layered architecture can use the Android system, the Apple iOS system, or other operating systems; this application embodiment does not limit this. The following uses a layered Android system as an example to exemplify the software architecture of the electronic device provided in this application embodiment.

[0075] Figure 2 This is a schematic diagram of the software architecture of an electronic device provided in an embodiment of this application. Figure 2 As shown, a layered architecture divides the software system of an electronic device into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system can be divided into five layers, from top to bottom: the application layer, the application framework layer, the hardware abstraction layer (HAL), the driver layer, and the hardware layer. The application layer can also be called the application application layer, the application framework layer can also be called the application framework layer, and the driver layer can also be called the kernel layer.

[0076] The application layer can include a series of application packages. The application layer runs applications by calling the application programming interface (API) provided by the application framework layer. For example... Figure 2 As shown, the application package can include applications such as camera and gallery.

[0077] The application framework layer provides APIs and a programming framework for applications in the application layer. The application framework layer includes some predefined functions. For example... Figure 2 As shown, the application framework layer may include a camera access interface. This camera access interface provides an application programming interface and programming framework for camera applications, and may include camera management and camera devices.

[0078] like Figure 2 As shown, the HAL layer can include a camera hardware abstraction layer and a camera algorithm library. The camera hardware abstraction layer can provide virtual hardware for the camera device. For example, if an electronic device includes camera device 1, camera device 2, etc., the camera hardware abstraction layer can provide the corresponding virtual hardware.

[0079] In this embodiment, the camera algorithm library may include an AE algorithm module and a decision module. The AE algorithm module can be used to implement AE algorithms, and the decision module can be used to output decision information for the first frame image. Decision information may include, for example, exposure time and / or gain value. The decision module may also be called a target module, adjustment module, etc., and this embodiment does not impose specific limitations. Of course, the camera algorithm library may also include other processing algorithm modules to implement various types of image processing, such as image stabilization, bokeh, and / or beautification.

[0080] The driver layer is the layer between hardware and software. For example... Figure 2 As shown, this driver layer may include one or more of the following: camera device driver, digital signal processor driver, and graphics processor driver. The camera device driver is used to drive the camera's sensor to acquire images. The digital signal processor driver is used to drive the digital signal processor to process images. The graphics processor driver is used to drive the graphics processor to process images.

[0081] The hardware layer may include hardware such as a camera assembly, an image signal processor, a digital signal processor, a graphics processor, and a display screen. The camera assembly may include at least one camera.

[0082] For example, upon receiving a user-triggered operation to launch the camera application, the camera application at the application layer can access the camera hardware abstraction layer through the camera access interface of the application framework layer. The camera hardware abstraction layer can then execute the image processing method of this embodiment by calling the AE algorithm module and decision module in the camera algorithm library to output corresponding decision information to the driver layer. The output decision information can be used to configure the data stream for launching the camera for the camera device driver. The camera device driver configures the exposure time and / or gain value of the corresponding camera according to the data stream for launching the camera, and launches the corresponding camera to acquire images. After the camera acquires an image, the acquired image can be processed by an image signal processor and then further processed by the processing algorithm module in the hardware abstraction layer.

[0083] It should be understood that in some embodiments, layers that perform the same function may be called by other names, or layers that can perform the functions of multiple layers may be considered as one layer, or layers that can perform the functions of multiple layers may be divided into multiple layers. This application does not impose any limitations on this.

[0084] To better understand the image processing method provided in the embodiments of this application, the following will be combined with... Figure 3 Please provide a detailed explanation.

[0085] Figure 3 This is a schematic diagram of an image processing method provided in an embodiment of this application, as shown below. Figure 3 As shown, this image processing method can be applied to electronic devices including cameras, and the method includes:

[0086] S301, determine whether the image to be acquired is the first frame image.

[0087] In this embodiment of the application, the first frame image may be the first frame image to be acquired after the camera is started.

[0088] Scenarios that can trigger camera activation include: opening the camera application, switching shooting modes in preview, or switching between the front and rear cameras in preview or recording.

[0089] In this embodiment, the camera application can be any application capable of previewing and taking photos. The method for determining whether the image to be acquired is the first frame can differ in different scenarios. For example, the following examples illustrate several possible implementations for determining whether the image to be acquired is the first frame.

[0090] In some implementations, determining whether the image to be acquired is the first frame can be done by checking if a camera activation command has been received. For example, when a user clicks the camera app icon on an electronic device to open the camera app, the electronic device responds to the camera app's activation operation by calling the camera access interface to indicate a camera activation command (such as an open command) to the camera hardware abstraction layer. After receiving this activation command, the camera hardware abstraction layer can determine that the image to be acquired is the first frame.

[0091] In some implementations, determining whether the image to be acquired is the first frame can be done by checking if a mode switching command for switching shooting modes has been received. Shooting modes can include portrait mode, professional mode, and / or night mode, etc. For example, if a user clicks the professional mode button in the camera application interface while in portrait mode, the electronic device responds to this mode switching operation by having the camera application call the camera access interface to transmit a reconfiguration stream command for resetting camera parameters to the camera hardware abstraction layer. Upon receiving this reconfiguration stream command, the camera hardware abstraction layer can determine that the image to be acquired is the first frame.

[0092] In some implementations, determining whether the image to be acquired is the first frame can be done by checking if a camera switching command has been received. For example, if a user previously used the front-facing camera to take a picture, and now wants to switch to the rear camera by clicking the camera switch button in the camera application interface, the electronic device responds to this camera switch operation by having the camera application call the camera access interface to transmit a close command (such as a close command) to the camera hardware abstraction layer to close the front camera and send a start command (such as an open command) to start the rear camera. After receiving the start command, the camera hardware abstraction layer can determine that the image to be acquired is the first frame.

[0093] For example, the start command, mode switching command, and camera switching command can all include a preset identifier. Whether the image is the first frame can be quickly and accurately determined by detecting whether the command includes a preset identifier. The preset identifiers included in the start command, mode switching command, and camera switching command can be preset identifiers of the same format or preset identifiers of different formats.

[0094] It is understandable that in scenarios such as waking up the camera application when it is in a waiting state, the camera can also be activated. Therefore, if the instruction to wake up the camera application is received, the image to be acquired can be determined to be the first frame image. This application embodiment does not limit the specific implementation of the scenario of activating the camera and determining whether the image to be acquired is the first frame image.

[0095] In this embodiment of the application, if it is determined that the image to be acquired is the first frame image, S302 can be executed. If it is determined that the image to be acquired is not the first frame image, S304 and S305 can be executed.

[0096] S302 calculates exposure time and gain value using the AE algorithm.

[0097] For example, the AE algorithm can be loaded into the hardware abstraction layer, such as in the AE algorithm module configured in the camera algorithm library. The camera hardware abstraction layer can call the AE algorithm and use the AE algorithm to calculate exposure parameters based on the light intensity acquired in the scene and a preset exposure table. The exposure parameters include the exposure time and gain value calculated by the AE algorithm.

[0098] An exposure table can be a mapping table containing multiple correspondences, where the correspondence can be the relationship between light intensity and exposure time, and the gain value. Additionally, the exposure table can also include the relationship between exposure time and gain value, and the image brightness.

[0099] Of course, the AE algorithm can be any algorithm capable of calculating exposure time and gain value, and this application embodiment does not limit it.

[0100] S303, determine whether the exposure time calculated by the AE algorithm is greater than the preset threshold value.

[0101] After obtaining the exposure time calculated by the After Effects (AE) algorithm, the camera hardware abstraction layer can call the decision module in the camera algorithm library to compare the exposure time calculated by the AE algorithm with a preset threshold value to determine whether the exposure time calculated by the AE algorithm is greater than the preset threshold value. The preset threshold value can be a preset duration used to determine the magnitude of the exposure time calculated by the AE algorithm.

[0102] In some implementations, in low-light scenes with low illumination, the exposure time under that illumination intensity can be calculated by calling the AE algorithm, and the duration shorter than that exposure time can be preset as a preset threshold value.

[0103] In some implementations, in low-light scenes with low illumination, the exposure time and gain value under that illumination intensity, as well as the corresponding image brightness, can be calculated by calling an After Effects (AE) algorithm. Taking the maximum gain value achievable by the electronic device, and keeping the image brightness constant or slightly adjusted, the exposure time under this condition can be calculated using the AE algorithm, and this exposure time value can be preset as a threshold value.

[0104] The preset threshold value can also be determined in other ways. For example, similar to the above implementation, the second largest gain value achievable by the electronic device can be taken. While keeping the image brightness unchanged or slightly reducing it, the exposure time under this condition can be calculated using the AE algorithm, and the value of this exposure time can be preset as the preset threshold value.

[0105] The preset threshold value can be a value that is smaller than the exposure time calculated by the AE algorithm in a low-light scene. This application does not limit the method for determining the preset threshold value, nor does it limit the specific value of the preset threshold value.

[0106] In this embodiment of the application, if it is determined that the exposure time calculated by the AE algorithm is greater than a preset threshold, S3031 and S3032 can be executed. If it is determined that the calculated exposure time is less than or equal to the preset threshold, S3033 can be executed.

[0107] S3031, reduce the exposure time to less than or equal to the preset threshold value, and recalculate the gain value based on the reduced exposure time.

[0108] Understandably, after the After Effects (AE) algorithm calculates the exposure time and gain value, it can obtain the image brightness corresponding to the calculated exposure time and gain value. To maintain the same or approximately the same image brightness, when the exposure time is reduced, the image brightness and the gain value corresponding to the reduced exposure time can be recalculated.

[0109] In some implementations, if the exposure time calculated by the After Effects (AE) algorithm is determined to be greater than a preset threshold, the exposure time calculated by the AE algorithm can be modified to match the preset threshold, i.e., the reduced exposure time equals the preset threshold. Based on the image brightness corresponding to the original exposure time and gain value calculated by the AE algorithm, and the reduced exposure time, the gain value is recalculated using the image brightness calculated by the AE algorithm. For example, the recalculated gain value can include the image brightness calculated by the AE algorithm divided by the reduced exposure time. Alternatively, based on the original exposure time and gain value, and the corresponding image brightness, and the reduced exposure time, the corresponding gain value can be determined by querying the corresponding relationship in the exposure table; this gain value can be the recalculated gain value.

[0110] Based on this, the image brightness corresponding to the recalculated gain value and the reduced exposure time can be the same as or approximately the same as the image brightness corresponding to the exposure time and gain value calculated by the AE algorithm before the modification. This can keep the image brightness of the first frame image unchanged or approximately unchanged.

[0111] In some implementations, if the exposure time calculated by the AE algorithm is determined to be greater than a preset threshold, the exposure time can be modified to be less than the preset threshold. The gain value of an electronic device is limited by the sensor's performance limit and cannot be infinitely large; therefore, the electronic device has a maximum gain value. After modifying the exposure time calculated by the AE algorithm to be less than the preset threshold, if the recalculated gain value based on this value is greater than the electronic device's maximum gain value, then the camera is configured with the exposure time being less than the preset threshold and the maximum gain value being the gain value.

[0112] If the gain value recalculated based on the value less than the preset threshold is less than or equal to the maximum gain value of the electronic device, then the exposure time is based on the value less than the preset threshold, and the camera is configured with the recalculated gain value as the gain value. In this case, the image brightness of the first frame captured by the camera remains unchanged or approximately unchanged.

[0113] If the brightness of the first frame remains unchanged or approximately unchanged, the difference between the brightness of subsequent non-first frame images captured by the camera and the brightness of the first frame image will be small. When transitioning from the first frame image to the display of non-first frame images, the image brightness will not change significantly, resulting in a better user experience.

[0114] S3032 outputs decision information based on the reduced exposure time and the recalculated gain value.

[0115] After reducing the exposure time and recalculating the gain value, decision information can be output based on the reduced exposure time and the recalculated gain value. This decision information includes the reduced exposure time and the recalculated gain value. When the camera is started, the camera device driver can configure the camera's exposure parameters according to this decision information.

[0116] By setting an exposure time less than or equal to a preset threshold when activating the camera, the exposure time for acquiring the first frame can be effectively controlled. This is especially beneficial in low-light shooting scenarios, shortening the time the camera's sensor needs to process light, thus reducing the sensor's exposure time in low light. This shorter exposure time speeds up the frame capture process, reducing latency between frame capture and display. Consequently, it reduces the time interval between activating the camera and displaying the first frame, allowing users to see a preview image more quickly after activating the camera, thus improving the user experience when using the camera application.

[0117] As mentioned above, when executing S303, if it is determined that the exposure time calculated by the AE algorithm is not greater than the preset threshold value, S3033 can be executed.

[0118] S3033 outputs decision information based on the exposure time and gain value calculated by the AE algorithm.

[0119] If the exposure time calculated by the AE algorithm is not greater than the preset threshold, that is, if the exposure time calculated by the AE algorithm is less than or equal to the preset threshold, the exposure time and gain value calculated by the AE algorithm will not be adjusted. In this case, decision information can be output based on the exposure time and gain value calculated by the AE algorithm. This decision information includes the exposure time and gain value calculated by the AE algorithm. When the camera is started, the camera device driver can configure the camera's exposure parameters according to this decision information.

[0120] If it is determined during the execution of S301 that the image to be acquired is not the first frame image, then S304 and S305 will be executed.

[0121] S304 calculates exposure time and gain value using the AE algorithm.

[0122] S305 outputs decision information based on the exposure time and gain value calculated by the AE algorithm.

[0123] The image to be acquired is not the first frame; it can be understood as a non-first frame image. Since the image to be acquired is not the first frame image, the exposure time corresponding to this image will not affect the time interval from activating the camera to displaying the first frame image. Therefore, the exposure time of this image to be acquired does not need to be reduced.

[0124] For images that are not the first frame, a method similar to the steps in the above embodiments can be used to calculate the exposure time and gain value through the AE algorithm, and output decision information based on the exposure time and gain value calculated by the AE algorithm. The decision information includes the exposure time and gain value calculated by the AE algorithm, and the camera device driver can configure the camera's exposure parameters according to the decision information.

[0125] In some implementations, in order to improve the resource utilization efficiency of electronic devices, after setting the exposure time for acquiring the first frame image, the execution of S301 to S303 of the image processing method can be paused, and S304 and S305 can be executed. All steps of the above method will be executed again when the camera is confirmed to be started again.

[0126] For example, an identifier can be set for the decision module in the camera algorithm library. After detecting that the camera application has issued a start command, the camera hardware abstraction layer can call the AE algorithm module and the decision module to start execution from the program code corresponding to S301. After executing S3033 or S3032, an identifier is set for the decision module to indicate that execution is paused, pausing the execution of S301 to S303. When acquiring images later, S304 and S305 can be executed until the camera is started again, and the exposure time and / or gain value of the first frame image are decided again based on the decision module.

[0127] It is understandable that the above Figure 3 In this embodiment, the example illustrates the simultaneous adjustment of the exposure time and gain value of the first frame image. In some implementations, Figure 3 In one embodiment, the adjustment of the gain value can be removed, which can also achieve the effect of quickly displaying the first frame image.

[0128] The above combination Figure 3 The method provided in the embodiments of this application is described, based on Figure 3 The provided method, as described in the embodiments of this application, can achieve the following: Figure 4 The corresponding scenario.

[0129] For example, Figure 4 This is a schematic flowchart of an image processing method provided in an embodiment of this application, as shown below. Figure 4 As shown, this method is applied to an electronic device including a camera, and the method includes the following steps:

[0130] S401, At the first moment, the camera is activated under the first light intensity, and the exposure time of the first frame image after the camera acquires the first moment is set to a first value, wherein the first light intensity is less than the second light intensity, and the first value is less than or equal to a preset threshold value.

[0131] The ambient light intensity of an electronic device's environment can range from low-light to moderate-light, or even high-light. Illuminance, also known as illuminance or light intensity, refers to the luminous flux of visible light received per unit area, measured in lux (Lux or lx). Illuminance values ​​in a scene can be detected using light intensity detection devices such as photometers and light sensors.

[0132] The first moment can be any moment. The first light intensity can be the light intensity in a low-light scene. For example, a low-light scene with a light intensity between 0 Lux and 200 Lux, such as about 1 Lux under moonlight at night, or about 50 Lux in a family living room, etc.

[0133] The first frame image can refer to the first frame image in the above embodiments. The preset threshold value can refer to the preset threshold value in the above embodiments. The first value can be any time length value that is less than or equal to the preset threshold value.

[0134] S402, Second moment, the camera is activated at the second light intensity, and the exposure time of the first frame image after the camera acquires the second moment is set to the second value, wherein the second light intensity is less than the third light intensity, and the second value is less than or equal to a preset threshold value.

[0135] The second time point can be any time point different from the first time point. The second illumination intensity can be the illumination intensity in a normal lighting scene. For example, a normal lighting scene with moderate illumination intensity, between 200 Lux and 1000 Lux. The second value can be any time length value less than or equal to a preset threshold value. The second value can be equal to the first value.

[0136] S403, Third moment, the camera is activated at the third light intensity, and the exposure time of the first frame image after the camera acquires the third moment is set to the third value, wherein the third value is less than or equal to a preset threshold value.

[0137] The third time point can be any time point different from the first and second time points. The third light intensity can be the light intensity in a strong light scene. For example, a strong light scene with high light intensity, where the light intensity is greater than 1000 Lux, such as outdoor light intensity under direct sunlight which can reach 10000 Lux to 100000 Lux. The third value can be any time length value less than or equal to a preset threshold value. The third value can be equal to the first value or the second value.

[0138] It should be noted that the light intensity ranges for low-light, normal-light, and high-light scenes described above are merely illustrative, and the specific range values ​​can be adjusted. This application embodiment does not limit this.

[0139] Taking a mobile phone as an example, at the first moment, the user clicks the camera app icon on the phone to open the camera app under a first light intensity of 50 Lux. In response to this action, the camera app sends an "open" command to the camera access interface to activate the camera under that first light intensity. When the camera is activated, an exposure time is set for the first frame image acquired by the camera. This exposure time is set to a first value, which is less than or equal to a preset threshold. For example, if the preset threshold is 30 milliseconds, the first value can be 30 milliseconds or 25 milliseconds, etc. If the first value is 30 milliseconds, then when the camera is activated under that first light intensity, the exposure time for the first frame image acquired by the camera is set to 30 milliseconds.

[0140] At the second moment, the user taps the camera app icon on their phone to open the camera app within a second light intensity of 300 Lux. In response to this action, the camera app sends an "open" command to the camera access interface to activate the camera within this second light intensity. When the camera is activated, an exposure time is set for the first frame image acquired after the second moment. This exposure time is set to a second value, which is less than or equal to a preset threshold. For example, if the preset threshold is 30 milliseconds, the second value can be 30 milliseconds or 20 milliseconds, etc. If the second value is 20 milliseconds, then when the camera is activated within this second light intensity, the exposure time for the first frame image acquired after the second moment is set to 20 milliseconds.

[0141] It is understandable that the first, second, and third values ​​mentioned above can be the same or different, and the first, second, and third values ​​can be obtained through... Figure 3 The decisions made by the decision-making module in S3031-S3033 of the embodiment are not described in detail here. Additionally, S401, S402, and S403 are used to illustrate scenarios in this embodiment where the exposure time of the first frame image is less than or equal to a preset threshold value under different light intensities. The order of S401, S402, and S403 is not limited. The first light intensity, second light intensity, and third light intensity are all external light intensity conditions in this scenario. In implementation, the electronic device in this embodiment may not need to calculate the first light intensity, second light intensity, and third light intensity.

[0142] In summary, in the embodiments of this application, when the camera is activated under the first light intensity, the second light intensity, or the third light intensity, an exposure time less than or equal to a preset threshold value is set for the camera. This can shorten the exposure time for acquiring the first frame image after the camera is activated, speed up the frame output speed of the first frame image, shorten the time interval from activating the camera to displaying the first frame image, and allow the user to see the preview screen after the camera is turned on more quickly, thereby improving the user's experience of using the camera application.

[0143] Optionally, activating the camera in a first light intensity includes: activating the camera in response to an activation operation for a camera application in an electronic device in a first light intensity; or, activating the camera in response to an operation to switch the shooting mode of a camera application in a first light intensity; or, activating the camera in response to an operation to switch the camera in a camera application in a first light intensity.

[0144] In this embodiment, the startup operation can be the first startup of the camera application or the startup of the camera when waking up a camera application that is in a waiting-to-wake state. It can be understood that before starting the camera, all cameras of the electronic device are in a turned-off state. In response to the startup operation of the camera application, the electronic device starts at least one camera, bringing at least one camera into a running state.

[0145] In some examples, the camera application is not running before the first moment; at the first moment, in response to the user clicking the camera application icon on the electronic device to launch the camera application, the camera application runs and calls the camera access interface of the application framework layer to issue a start command to start the camera, so that one or more cameras enter the running state.

[0146] In other examples, before the first instant, the camera application has already started. If the user does not operate the camera application's shutter button for an extended period, the camera application enters a dormant, pending-wake state, during which all cameras are either off or in a dormant state. At the first instant, in response to a user's wake-up operation such as clicking the screen or a button on the electronic device, the camera application switches from the pending-wake state to the running state, calling the camera access interface to send a start command to the camera hardware abstraction layer to start the camera, thus putting at least one camera into the running state.

[0147] Switching the shooting mode of a camera app can be done by reconfiguring the camera's exposure parameters while the camera is already running, in order to restart the camera's operation.

[0148] In some examples, before the first instant, the camera application is already running and at least one camera is activated. At the first instant, in response to the user clicking the target shooting mode button in the camera application, the camera application calls the camera access interface to issue a reconfiguration stream command to reconfigure the camera's exposure parameters, thus restarting the camera. The target shooting mode can include any shooting mode such as portrait mode, professional mode, and night mode; this embodiment does not impose any limitations on this.

[0149] Switching between cameras in the camera app can be done by turning off the currently running camera and starting another camera while the current camera is already running.

[0150] In some examples, before the first moment, the camera application is already running, and camera 1 is already activated and running. At the first moment, in response to the user clicking the camera switch button in the camera application, the camera application calls the camera access interface, issuing a shutdown command to close the first camera and a startup command to start the second camera. Upon receiving the shutdown command, the camera hardware abstraction layer controls the camera device driver to shut down camera 1. Upon receiving the startup command, the camera hardware abstraction layer controls the camera device driver to start camera 2, which then enters the running state and will be used to acquire the first frame image after the first moment. Here, camera 1 can be a front-facing camera or a rear-facing camera, and camera 2 can be a rear-facing camera or a front-facing camera. Switching cameras in the camera application can be any two cameras on the electronic device, and the camera activated during switching can be specified by the user.

[0151] This embodiment can also refer to the above. Figure 3 The relevant descriptions in S301 of the corresponding embodiments will not be repeated.

[0152] For activating the camera under the second or third light intensity, a similar approach as activating the camera under the first light intensity can be used.

[0153] For example, activating a camera in a second light intensity includes: activating the camera in response to an activation operation for a camera application in an electronic device in the second light intensity; or, activating the camera in response to an operation to switch the shooting mode of a camera application in the second light intensity; or, activating the camera in response to an operation to switch the camera in a camera application in the second light intensity.

[0154] For example, activating a camera in a third light intensity includes: activating the camera in response to an activation operation for a camera application in an electronic device in the third light intensity; or, activating the camera in response to an operation to switch the shooting mode of a camera application in the third light intensity; or, activating the camera in response to an operation to switch the camera in a camera application in the third light intensity.

[0155] In this embodiment of the application, for the operation of starting the camera application, switching the shooting mode of the camera application, or switching the camera in the camera application, the exposure time of the camera can be set to be less than or equal to a preset threshold value, thereby shortening the time interval from starting the camera to displaying the first frame image.

[0156] Optionally, the first value is less than the exposure time of the first frame image after the first moment calculated according to the automatic exposure (AE) algorithm, and the gain value of the first frame image after the first moment is greater than the gain value of the first frame image after the first moment calculated according to the AE algorithm.

[0157] The After Effects (AE) algorithm calculates exposure time and gain values ​​based on an exposure table. When shooting in low-light conditions, the AE algorithm typically calculates a longer exposure time to increase the time the camera's sensor has to process light, resulting in a properly lit image. However, a longer exposure time leads to a slower first frame output. In this situation, if the camera's exposure time is still configured according to the AE algorithm's calculation, it will result in a longer time interval between camera activation and seeing the first frame.

[0158] Figure 5 This is a comparative diagram of camera exposure parameters in the first illumination intensity provided in an embodiment of this application, as shown below. Figure 5 As shown, the portion above the timeline represents the exposure time and gain value of the first frame image acquired after configuring and starting the camera based on the exposure time and gain value calculated by the AE algorithm; the portion below the timeline represents the exposure time and gain value of the first frame image acquired after configuring and starting the camera based on a first value and a gain value larger than the gain value calculated by the AE algorithm. In low-light scenes, to increase the amount of light received by the photosensitive element, the AE algorithm calculates a larger exposure time and a medium or smaller gain value. Configuring and starting the camera based on this exposure time and gain value can acquire a first frame image with normal brightness, but the time for acquiring and displaying the first frame image will be delayed, resulting in a longer waiting time for the user to see the preview of the first frame image.

[0159] like Figure 5 As shown, in order to reduce the exposure time and quickly obtain the first frame image, when the camera is started to acquire the first frame image, the camera's exposure time is set to a first value that is less than the exposure time calculated by the AE algorithm. Furthermore, in order to make the image brightness of the acquired first frame image the same as or close to the image brightness of the first frame image obtained according to the exposure time and gain value calculated by the AE algorithm, the gain value is set to be greater than the gain value calculated by the AE algorithm. This can compensate for the image brightness loss caused by the first value being less than the exposure time calculated by the AE algorithm. Based on this, a first frame image with normal or near-normal image brightness can be obtained, and the speed of displaying the first frame image can be accelerated. Figure 5 In the two methods shown, the camera exposure time and gain value are the same, but the exposure time and gain value for the camera to acquire the first frame image are set with the first value and the larger gain value, which shortens the time to output the first frame image in low-light scenes, allowing users to see the preview of the first frame image more quickly.

[0160] In this embodiment, relative to the exposure time and gain value calculated by the automatic exposure (AE) algorithm, the first value is set to be less than the exposure time calculated by the AE algorithm, and the gain value of the first frame image is greater than the calculated gain value. This allows the camera's exposure parameters to be configured with a larger gain value and a smaller exposure time, ensuring that the brightness of the first frame image acquired after the camera starts is not too low, and improving the frame output speed of the first frame image. This enables the camera to easily meet the brightness requirements of the first frame image while increasing the display speed of the first frame image.

[0161] For example, if the second value is less than the exposure time of the first frame image after the second moment calculated according to the automatic exposure (AE) algorithm, the gain value of the first frame image after the second moment acquired by the camera is greater than the gain value of the first frame image after the second moment calculated according to the AE algorithm.

[0162] For example, if the third value is less than the exposure time of the first frame image after the third moment calculated according to the automatic exposure (AE) algorithm, the gain value of the first frame image after the third moment obtained by the camera is greater than the gain value of the first frame image after the third moment calculated according to the AE algorithm.

[0163] Optionally, setting the exposure time of the first frame image after the camera acquires the first moment to a first value includes: when it is determined that the image to be acquired at the first moment is the first frame image and the exposure time calculated according to the automatic exposure (AE) algorithm is greater than a preset threshold value, setting the exposure time of the first frame image after the camera acquires the first moment to a first value; the method further includes: setting the gain value of the first frame image after the camera acquires the first moment to a fourth value, wherein the brightness of the first frame image after the first moment calculated based on the first value and the fourth value is the same as the brightness calculated by the AE algorithm.

[0164] For example, using the methods described above Figure 3 The method in embodiment S301 determines whether the image to be acquired is the first frame. The exposure time is calculated using the AE algorithm in S302. When it is determined to be the first frame, the exposure time calculated by the AE algorithm is compared with a preset threshold. If the exposure time calculated by the AE algorithm is greater than the preset threshold, the exposure time of the first frame image acquired by the camera after the first moment is set to a first value. This can be understood as follows: in this case, the exposure time calculated by the AE algorithm is not used to configure the exposure time for the first frame image acquired by the camera; instead, a duration less than or equal to the preset threshold is set as the exposure time.

[0165] After determining the first value, the AE algorithm can be used to calculate the gain value corresponding to the first value, which is the fourth value. The gain value of the first frame image after the camera acquires the first moment is set to the fourth value.

[0166] The brightness being the same as that calculated by the AE algorithm can be understood as the image brightness of the first frame image obtained by configuring the camera's exposure parameters using the first and fourth values ​​and acquiring the first frame image being the same as or approximately the same as the image brightness obtained by configuring the camera's exposure parameters using the exposure time and gain value calculated by the AE algorithm and acquiring the image. "Same" or "approximately the same" can be understood as the image brightness values ​​being identical or the difference in image brightness values ​​being within a preset brightness difference range. This brightness difference range can be set based on empirical values ​​or known image brightness data, and this application embodiment does not impose any limitations on it.

[0167] In this embodiment, if it is determined that the image to be acquired at the first moment is the first frame image and the exposure time calculated by the automatic exposure (AE) algorithm is greater than a preset threshold, an exposure time less than or equal to the preset threshold can be set for the camera, i.e., the exposure time is set to a first value; in addition, the gain value of the camera can be set to a fourth value. Acquiring the first frame image based on the first and fourth values ​​can make the brightness of the first frame image the same as the brightness calculated by the AE algorithm, so that the brightness of the acquired first frame image is close to normal brightness.

[0168] For example, setting the exposure time of the first frame image after the camera acquires the second moment as a second value includes: when it is determined that the image to be acquired at the second moment is the first frame image and the exposure time calculated according to the automatic exposure (AE) algorithm is greater than a preset threshold value, setting the exposure time of the first frame image after the camera acquires the second moment as a second value; the method further includes: setting the gain value of the first frame image after the camera acquires the second moment as a fifth value, wherein the brightness of the first frame image after the second moment calculated based on the second value and the fifth value is the same as the brightness calculated by the AE algorithm.

[0169] For example, after determining the second value, the AE algorithm can be used to calculate the gain value corresponding to the second value, which is the fifth value. The gain value of the first frame image after the camera acquires the second moment is set to the fifth value.

[0170] For example, setting the exposure time of the first frame image after the camera acquires the third time step as a third value includes: when it is determined that the image to be acquired at the third time step is the first frame image and the exposure time calculated according to the automatic exposure (AE) algorithm is greater than a preset threshold value, setting the exposure time of the first frame image after the camera acquires the third time step as a third value; the method further includes: setting the gain value of the first frame image after the camera acquires the third time step as a sixth value, wherein the brightness of the first frame image after the third time step calculated based on the third value and the sixth value is the same as the brightness calculated by the AE algorithm.

[0171] For example, after determining the third value, the AE algorithm can be used to calculate the gain value corresponding to the third value, which is the sixth value. The gain value of the first frame image after the camera acquires the third time point is the sixth value.

[0172] Optionally, setting the exposure time of the first frame image after the camera acquires the third time step as a third value includes: when it is determined that the image to be acquired at the third time step is the first frame image, and the exposure time calculated according to the automatic exposure (AE) algorithm is less than or equal to a preset threshold value, setting the exposure time of the first frame image after the camera acquires the third time step as a third value, wherein the third value is the exposure time calculated according to the automatic exposure (AE) algorithm.

[0173] The third moment corresponds to the third light intensity. When the camera is activated in a strong light scene, the AE algorithm will calculate a smaller exposure time due to the large light intensity in the scene. Usually, this exposure time will be less than or equal to the preset threshold value.

[0174] Adopt as described above Figure 3 The method in the illustrated embodiment determines whether the image to be acquired is the first frame image. After determining that the image to be acquired at the third moment is the first frame image, and comparing the exposure time calculated by the AE algorithm with a preset threshold value, if the exposure time calculated by the AE algorithm is less than or equal to the preset threshold value, then the exposure time calculated by the AE algorithm does not need to be adjusted. The exposure time calculated by the AE algorithm is determined as the third value, and this third value is set as the exposure time of the first frame image after the camera acquires the third moment. Based on this, the exposure time can be quickly set in strong light scenes.

[0175] In this embodiment, when the camera is activated at a third light intensity and the exposure time calculated by the first frame image and the AE algorithm is less than or equal to a preset threshold, the camera can be set with an exposure time calculated by the automatic exposure AE algorithm to achieve the purpose of quickly configuring the camera's exposure parameters.

[0176] For example, setting the exposure time of the first frame image after the camera acquires the second moment as a second value includes: when it is determined that the image to be acquired at the second moment is the first frame image and the exposure time calculated according to the automatic exposure AE algorithm is less than or equal to a preset threshold value, setting the exposure time of the first frame image after the camera acquires the second moment as a second value, wherein the second value is the exposure time calculated according to the automatic exposure AE algorithm.

[0177] For example, setting the exposure time of the first frame image after the camera acquires the first moment as a first value includes: when it is determined that the image to be acquired at the first moment is the first frame image and the exposure time calculated according to the automatic exposure AE algorithm is less than or equal to a preset threshold value, setting the exposure time of the first frame image after the camera acquires the first moment as a first value, wherein the first value is the exposure time calculated according to the automatic exposure AE algorithm.

[0178] Optionally, the electronic device includes a target module, which is used to set the exposure time of the first frame image after the camera is started, and to determine the image to be acquired at the first moment as the first frame image, including: when the target module has a preset identifier, determining the image to be acquired at the first moment as the first frame image.

[0179] The target module can be a hardware functional module in the hardware abstraction layer, for example, it can be... Figure 2 The decision module in the camera algorithm library, etc. After the decision module obtains the decision information, the exposure time included in the decision signal can be used to set the exposure time of the first frame image after the camera is started.

[0180] Preset identifiers can be used to indicate whether a target module needs to execute the first frame image. These identifiers can be any type of identifier, such as characters or numbers. For example, when a camera application sends a command to the hardware abstraction layer to start the camera, the electronic device adds a preset identifier to the target module. With this preset identifier, it's easy to determine that the image to be acquired at the first moment is the first frame image.

[0181] In this embodiment, the target module can set the exposure time for acquiring the first frame image after the camera is started. If the target module is marked with a preset identifier, the image to be acquired after the first moment can be conveniently determined as the first frame image based on the preset identifier.

[0182] When determining that the image to be acquired is the first frame image at the second or third time point, the same method can be used for determination.

[0183] For example, determining that the image to be acquired at the second moment is the first frame image includes: if the target module flag has a preset identifier, determining that the image to be acquired at the second moment is the first frame image.

[0184] For example, determining that the image to be acquired at the third moment is the first frame image includes: if the target module flag has a preset identifier, determining that the image to be acquired at the third moment is the first frame image.

[0185] Optionally, after setting the exposure time of the first frame image after the camera acquires the first moment to a first value, the method further includes: setting the flag of the target module to another flag that is not a preset flag; or, deleting the preset flag of the target module.

[0186] In this embodiment, after the electronic device marks a target module with a preset identifier, it can delete the preset identifier within a short period of time. This allows subsequent determinations based on the absence of the preset identifier on the target module, indicating that the subsequent image to be acquired is not the first frame image. Alternatively, after the electronic device marks a target module with a preset identifier, it can change the preset identifier to another identifier within a short period of time. This other identifier indicates that the target module does not need to execute the decision on the first frame image. This allows subsequent determinations based on the other identifiers of the target module, indicating that the subsequent image to be acquired is not the first frame image.

[0187] In this embodiment of the application, images acquired after the first frame image are not the first frame image. Therefore, after setting the first value, by setting the flag of the target module to another flag that is not the preset flag, or by deleting the preset flag of the target module, it is easy to determine that other images acquired after the first frame image are not the first frame image.

[0188] After setting the exposure time for the first frame image acquired by the camera at the second or third moment, the identification of the target module can be processed in the same way.

[0189] For example, after setting the exposure time of the first frame image after the camera acquires the second moment to a second value, the method further includes: setting the flag of the target module to another flag that is not a preset flag; or, deleting the preset flag of the target module.

[0190] For example, after setting the exposure time of the first frame image after the camera acquires the third time moment to the third value, the method further includes: setting the flag of the target module to another flag that is not a preset flag; or, deleting the preset flag of the target module.

[0191] Optionally, after the first moment, the method further includes: setting the exposure time for the camera to acquire images other than the first frame to the exposure time calculated according to the automatic exposure (AE) algorithm.

[0192] Non-first frame images are any frame images other than the first frame image captured by the camera after it is activated. For non-first frame images, the exposure time for capturing non-first frame images can be quickly set based on the exposure time calculated by the AE algorithm.

[0193] like Figure 5As shown, for non-first-frame images acquired in low-light scenes, the exposure time and gain value for the camera to acquire non-first-frame images can be set based on the exposure time and gain value calculated by the automatic exposure (AE) algorithm. Compared to the first-frame image, which has a lower exposure time and a larger gain value, the exposure time for non-first-frame images is increased, while the gain value is decreased. After obtaining the first-frame or non-first-frame image, it undergoes display processing so that the user can see either the first-frame or non-first-frame image.

[0194] In this embodiment, since the user has already seen a preview of the first frame image, there is no issue of a long time interval between activating the camera and seeing the preview for non-first frame images. The exposure time calculated using the AE algorithm is used to set the exposure time for the camera to acquire non-first frame images, allowing for rapid exposure time setting.

[0195] After the second or third moment, the exposure time for the camera to acquire images other than the first frame can be set in the same way.

[0196] For example, after the second moment, the method further includes: setting the exposure time for the camera to acquire images other than the first frame to the exposure time calculated according to the automatic exposure (AE) algorithm.

[0197] For example, after the third moment, the method further includes: setting the exposure time for the camera to acquire images other than the first frame to the exposure time calculated according to the automatic exposure (AE) algorithm.

[0198] It should be noted that the module names involved in the embodiments of this application can all be defined as other names, as long as they can achieve the function of each module, and no specific restrictions are placed on the module names.

[0199] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0200] The foregoing primarily describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the aforementioned functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the method steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0201] This application embodiment can divide the apparatus for implementing the method into functional modules based on the above method examples. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0202] Figure 6 This is a schematic diagram of a chip structure provided in an embodiment of this application. The chip 600 includes one or more processors 601, communication lines 602, communication interfaces 603, and memory 604.

[0203] In some implementations, memory 604 stores elements such as executable modules or data structures, or subsets thereof, or extended sets thereof.

[0204] The methods described in the embodiments of this application can be applied to, or implemented by, processor 601. Processor 601 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above methods can be completed by integrated logic circuits in the hardware of processor 601 or by instructions in software form. Processor 601 may be a general-purpose processor (e.g., a microprocessor or conventional processor), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates, transistor logic devices, or discrete hardware components. Processor 601 can implement or execute the various processing-related methods, steps, and logic block diagrams disclosed in the embodiments of this application.

[0205] The steps of the method disclosed in the embodiments of this application can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can be located in mature storage media in the art, such as random access memory, read-only memory, programmable read-only memory, or electrically erasable programmable read-only memory (EEPROM). This storage medium is located in memory 604, and processor 601 reads information from memory 604 and, in conjunction with its hardware, completes the steps of the above method.

[0206] The processor 601, memory 604 and communication interface 603 can communicate with each other through communication line 602.

[0207] In the above embodiments, the instructions stored in the memory for execution by the processor can be implemented in the form of a computer program product. This computer program product can be pre-written into the memory, or it can be downloaded and installed into the memory as software.

[0208] This application also provides a computer program product comprising one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions may be transmitted from a website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. For example, available media may include magnetic media (e.g., floppy disk, hard disk, or magnetic tape), optical media (e.g., digital versatile disc (DVD)), or semiconductor media (e.g., solid-state disk (SSD)).

[0209] This application also provides a computer-readable storage medium. The methods described in the above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any combination thereof. The computer-readable medium may include computer storage media and communication media, and may also include any medium capable of transferring a computer program from one place to another. The storage medium can be any target medium accessible by a computer.

[0210] As one possible design, computer-readable media may include compact disc read-only memory (CD-ROM), RAM, ROM, EEPROM, or other optical disc storage; computer-readable media may also include disk storage or other disk storage devices. Furthermore, any connecting cable may also be appropriately referred to as computer-readable media. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of media. As used herein, disks and optical discs include optical discs (CD), laser discs, optical discs, digital versatile discs (DVD), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs optically reproduce data using lasers.

[0211] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0212] The above specific embodiments further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. An image processing method, characterized in that, Applied to an electronic device including a camera, the method includes: At the first moment, the camera is activated under the first light intensity, and the exposure time of the first frame image acquired by the camera after the first moment is set to a first value; At the second moment, the camera is activated under the second light intensity, and the exposure time of the first frame image acquired by the camera after the second moment is set to the second value; At the third moment, the camera is activated under the third light intensity, and the exposure time of the first frame image acquired by the camera after the third moment is set to the third value. Wherein, the first light intensity is less than the second light intensity, the second light intensity is less than the third light intensity, and the first value, the second value, and the third value are all less than or equal to a preset threshold value.

2. The method according to claim 1, characterized in that, Activating the camera under the first light intensity includes: In response to a startup operation for a camera application in the electronic device at the first illumination intensity, the camera is activated; Alternatively, in response to switching the shooting mode of the camera application during the first light intensity, the camera is activated; Alternatively, in response to switching the camera in the camera application during the first light intensity, the camera is activated.

3. The method according to claim 1 or 2, characterized in that, The first value is less than the exposure time of the first frame image after the first moment calculated according to the automatic exposure (AE) algorithm, and the gain value of the first frame image after the first moment obtained by the camera is greater than the gain value of the first frame image after the first moment calculated according to the AE algorithm.

4. The method according to any one of claims 1-3, characterized in that, Setting the exposure time of the first frame image acquired by the camera after the first moment to a first value includes: If it is determined that the image to be acquired at the first moment is the first frame image and the exposure time calculated according to the automatic exposure (AE) algorithm is greater than the preset threshold value, the exposure time of the first frame image acquired by the camera at the first moment is set to the first value. The method further includes: setting the gain value of the first frame image after the first moment acquired by the camera to a fourth value, wherein the brightness of the first frame image after the first moment calculated based on the first value and the fourth value is the same as the brightness calculated by the AE algorithm.

5. The method according to any one of claims 1-4, characterized in that, Setting the exposure time of the first frame image acquired by the camera after the third moment to a third value includes: If the image to be acquired at the third moment is determined to be the first frame image, and the exposure time calculated by the automatic exposure (AE) algorithm is less than or equal to the preset threshold value, the exposure time of the first frame image acquired by the camera at the third moment is set as the third value, and the third value is the exposure time calculated by the automatic exposure (AE) algorithm.

6. The method according to claim 4 or 5, characterized in that, The electronic device includes a target module, which is used to set the exposure time of the first frame image after the camera is activated. Determining that the image to be acquired at the first moment is the first frame image includes: If the target module flag has a preset identifier, the image to be acquired at the first moment is determined to be the first frame image.

7. The method according to claim 6, characterized in that, After setting the exposure time of the first frame image acquired by the camera at the first moment to a first value, the method further includes: Set the flag of the target module to an identifier other than the preset identifier; Alternatively, delete the preset identifier of the target module.

8. The method according to any one of claims 1-7, characterized in that, After the first moment, the method further includes: The exposure time for the camera to acquire images other than the first frame is set to the exposure time calculated based on the automatic exposure (AE) algorithm.

9. An electronic device, characterized in that, include: A memory and a processor, the memory for storing a computer program and the processor for executing the computer program to perform the method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, cause a computer to perform the method as described in any one of claims 1-8.

11. A computer program product, characterized in that, Includes a computer program that, when run, causes an electronic device to perform the method as described in any one of claims 1-8.

Images