Image tone processing method and electronic equipment

Abstract

The invention discloses an image tone processing method and electronic equipment, and relates to the technical field of terminals. When an HDR video is played, a tone mapping processing method and data adopted by a GPU synthesis mode and a DPU synthesis mode are synchronized, so that images generated by the GPU synthesis mode and the DPU synthesis mode are ensured to realize the same display effect, the problem of picture flash occurring in the synthesis mode switching process is avoided, and the image quality is improved. The problem of display effect difference of two image layer synthesis modes is fundamentally solved, and better visual experience is provided for users.

Description

Technical Field

[0001] This application relates to the technical field of terminals, and in particular, to an image tone processing method and an electronic device. Background Art

[0002] Many electronic devices support playing high-dynamic range (HDR) videos. For example, a mobile phone photo album browses HDR videos, a video application plays HDR videos, and a mobile phone camera shooting preview involves HDR display effect enhancement features, etc. An image in an HDR video or an image with an HDR display effect enhancement feature is called an HDR image. During the process of synthesizing and sending an HDR image for display, tone mapping processing is required. Since image synthesis involves two different synthesis methods, and different synthesis methods have different processing methods during tone mapping processing, it will cause differences in the image synthesis and sending display effects. During the playback of an HDR video, the switching of the image synthesis method will cause the user to feel that the screen flickers and produces discomfort during viewing. Summary of the Invention

[0003] Embodiments of this application provide an image tone processing method and an electronic device, which can eliminate the differences in tone mapping processing between two different synthesis methods and avoid the problem of screen flickering caused by differences in image display effects during the playback of an HDR video.

[0004] To achieve the above object, the embodiments of this application adopt the following technical solutions:

[0005] In a first aspect, an image tone processing method is provided, which is applied to an electronic device. The electronic device includes a processor, and an operating system of the electronic device runs on the processor. The processor includes a graphics processing unit (GPU) and a display processing unit (DPU). The operating system obtains the layer information of the first layer of the first image; for example, the layer information includes at least one of the identifier of the layer, the control information included in the layer, and the control layout information; if it is determined that the first image is a high-dynamic range (HDR) image and the layer synthesis method corresponding to the first layer is the GPU synthesis method, the operating system obtains the first data, where the first data is the data used by the DPU for tone mapping processing; the operating system generates a shader rendering instruction according to the first data and the first tone mapping processing method (the tone mapping processing method used by the DPU for tone mapping processing); the GPU performs tone mapping processing on the first layer according to the shader rendering instruction.

[0006] In this method, the tone mapping processing processes of the GPU synthesis method and the DPU synthesis method (including the tone mapping processing method adopted and the data used in the tone mapping processing) are synchronized, so as to ensure that the images generated by the GPU synthesis method and the DPU synthesis method achieve the same display effect (for example, there will be no display differences, jumps in image color or brightness, etc.). The problem of screen flickering during the synthesis method switching process is avoided, fundamentally solving the problem of display effect differences between the two layer synthesis methods and providing a better visual experience for users.

[0007] Combined with the first aspect, in a possible implementation manner, the operating system includes a rendering engine. The operating system generating a shader rendering instruction according to the first data and the first tone mapping processing method includes: the rendering engine generating a shader rendering instruction according to the first data and the first tone mapping processing method.

[0008] Combined with the first aspect, in a possible implementation manner, the operating system further includes a surface compositor and a hardware compositor HWC. The operating system obtaining the first data includes: HWC obtaining the first data from the DPU; HWC sending the first data to the rendering engine through the surface compositor.

[0009] In this method, if it is recognized that it is an HDR layer, HWC synchronizes the data adopted by the tone mapping processing method of the DPU synthesis method to the rendering engine. The rendering engine logically packages the data used in the tone mapping processing of the DPU synthesis method and the tone mapping processing method adopted by the DPU synthesis method into a shader rendering instruction, and sends the shader rendering instruction to the GPU for rendering processing. It is realized that the GPU side performs tone mapping processing using the same tone mapping processing method as the DPU synthesis method, and the GPU synthesis method and the DPU synthesis method achieve a consistent display effect.

[0010] In a possible implementation manner, the first image is a frame image in an HDR video stored on the electronic device.

[0011] Combined with the first aspect, in a possible implementation manner, the operating system obtaining the first data includes: the rendering engine generating the first data according to the metadata of the first image.

[0012] In a possible implementation manner, the first image is a preview image for taking a photo with enhanced HDR display effect.

[0013] In this method, the electronic device plays a camera preview image and has not yet generated an HDR video. After the camera application obtains the image captured by the camera, a special mark is set for each layer with enhanced HDR display effects. For example, this special mark can be recorded in the layer information, and the rendering engine can determine that the layer is an HDR layer based on this special mark, and then data synchronization can be performed according to the metadata of the first image. In the camera preview scenario of taking pictures, compared with the method of transmitting the lookup table data through HWC, data synchronization through the metadata method saves the time of waiting for HWC to prepare the layer (such as determining the layer composition method, reading the lookup table data, etc.). The method of data synchronization through metadata in this scenario can reduce the display delay of the camera preview image and improve the user experience.

[0014] Combined with the first aspect, in a possible implementation, the rendering engine generates a first shader, which includes a first sub-shader and a second sub-shader. The first sub-shader is used to implement the algorithm logic of the first tone mapping processing method, and the second sub-shader is used to provide the first data.

[0015] In a possible implementation, the first sub-shader is pre-generated, and the second sub-shader is generated by the rendering engine after obtaining the first data.

[0016] In this method, the rendering engine pre-generates the algorithm logic (the first sub-shader) of the first tone mapping processing method. When the first data is obtained, the second sub-shader (the data sub-shader) is replaced according to the first data, without the need to reconstruct the entire linear effect shader (the first shader). This improves the efficiency and performance of the system while maintaining the consistency of tone mapping.

[0017] Combined with the first aspect, in a possible implementation, determining that the first image is a high dynamic range (HDR) image includes: determining that the first image is a high dynamic range (HDR) image according to the layer information of the first layer.

[0018] In a possible implementation, if the first image is a frame image in an HDR video saved locally, it is possible to determine whether the first layer is an HDR layer according to the attribute Dataspace in the layer information.

[0019] In a possible implementation, when the first image is a camera preview image, if it is determined that the value of the additional mark attribute in the layer information is "EnhanceToHDRLayer", then it is determined that the layer is an HDR layer.

[0020] Combined with the first aspect, in a possible implementation, the first tone mapping processing method is a 3D-LUT lookup table tone mapping algorithm.

[0021] In a second aspect, an electronic device is provided, which has the function of implementing the method described in the first aspect above. This function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0022] In a third aspect, an electronic device is provided, including: a processor and a memory; the memory is used to store computer-executable instructions, and when the electronic device runs, the processor executes the computer-executable instructions stored in the memory, so that the electronic device executes the method described in any one of the first aspect above.

[0023] In a fourth aspect, an electronic device is provided, including: a processor; the processor is used to be coupled with a memory, and after reading instructions in the memory, execute the method described in any one of the first aspect above according to the instructions.

[0024] In a fifth aspect, a computer-readable storage medium is provided, in which instructions are stored, and when it runs on a computer, the computer can execute the method described in any one of the first aspect above.

[0025] In a sixth aspect, a computer program product containing instructions is provided, and when it runs on a computer, the computer can execute the method described in any one of the first aspect above.

[0026] In a seventh aspect, a device (for example, the device can be a chip system) is provided, which includes a processor for supporting an electronic device to implement the functions involved in the first aspect above. In a possible design, the device further includes a memory for storing necessary program instructions and data of the electronic device. When the device is a chip system, it can be composed of chips or include chips and other discrete devices.

[0027] Among them, for the technical effects brought by any one of the design manners in the second aspect to the seventh aspect, reference can be made to the technical effects brought by different design manners in the first aspect, which will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present application;

[0029] Figure 2 It is a schematic diagram of the software architecture of an electronic device provided by an embodiment of the present application;

[0030] Figure 3 It is a schematic diagram of two image display effects;

[0031] Figure 4Schematic diagram of an image tone processing method;

[0032] Figure 5 Schematic diagram of an image tone processing method provided by an embodiment of the present application;

[0033] Figure 6 Schematic diagram of an implementation manner of an image tone processing method provided by an embodiment of the present application;

[0034] Figure 7 Schematic diagram of an implementation manner of an image tone processing method provided by an embodiment of the present application;

[0035] Figure 8 Schematic diagram of a process of an implementation manner in which a camera application transmits a special mark to a Render Engine;

[0036] Figure 9 Schematic diagram of the structural composition of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0037] In the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to limit the present application. As used in the specification and appended claims of the present application, the singular forms "a", "the", "above-mentioned", "this" and "this one" are also intended to include expressions such as "one or more", unless clearly indicated to the contrary in the context. It should also be understood that in the following embodiments of the present application, "at least one" and "one or more" mean one or more than two (including two). The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist; for example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B may be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0038] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, the phrases "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc., which appear in different places in this specification, are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "comprising", "including", "having" and their variants mean "including but not limited to", unless otherwise specifically emphasized. The term "connection" includes direct connection and indirect connection, unless otherwise stated. "First" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features.

[0039] In the embodiments of the present application, words such as "exemplary" or "for example" are used to represent examples, illustrations, or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0040] An image tone processing method provided by the embodiments of the present application can be applied to an electronic device including a display screen. The electronic device can play an HDR video on the display screen. The above-mentioned electronic device can include mobile phones, tablet computers, laptop computers, personal computers (PCs), ultra-mobile personal computers (UMPCs), handheld computers, netbooks, smart home devices (such as smart TVs, smart screens, large screens, smart speakers, smart air conditioners, etc.), personal digital assistants (PDAs), wearable devices (such as smart watches, smart bracelets, etc.), in-vehicle devices, virtual reality devices, etc. The embodiments of the present application do not impose any restrictions on this.

[0041] Exemplarily, Figure 1 A schematic diagram of a hardware structure of the above-mentioned electronic device is shown. As Figure 1As shown, the electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone interface 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a temperature sensor, an ambient light sensor, etc.

[0042] It can be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 100. In other embodiments, the electronic device 100 may include more or fewer components than shown, or combine certain components, or split certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0043] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a display processing unit (DPU), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0044] The controller may be the nerve center and command center of the electronic device 100. The controller may generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching and executing instructions.

[0045] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory may store instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be directly called from the said memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0046] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0047] It can be understood that the interface connection relationships between the modules illustrated in this embodiment are only illustrative and do not constitute a structural limitation on the electronic device. In other embodiments, the electronic device may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0048] The charging management module 140 is configured to receive a charging input from a charger. The charger may be a wireless charger or a wired charger. In some embodiments of wired charging, the charging management module 140 may receive the charging input from the wired charger through the USB interface 130. In some embodiments of wireless charging, the charging management module 140 may receive the wireless charging input through the wireless charging coil of the electronic device. While charging the battery 142, the charging management module 140 may also supply power to the electronic device through the power management module 141.

[0049] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives inputs from the battery 142 and / or the charging management module 140 to supply power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, the wireless communication module 160, etc. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 can also be disposed in the processor 110. In some other embodiments, the power management module 141 and the charging management module 140 can also be disposed in the same device.

[0050] The wireless communication function of the electronic device 100 can be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, and the baseband processor, etc.

[0051] The electronic device 100 implements the display function through the GPU, the display screen 194, and the application processor, etc. The GPU is a microprocessor for image processing, connecting the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering and plays a crucial role in improving graphics performance and efficiency. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0052] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini-LED, a Micro-OLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc.

[0053] The electronic device 100 can implement the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, and the application processor, etc.

[0054] The ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and light passes through the lens and is transmitted to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also optimize the noise, brightness, and skin tone of the image through algorithms. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be disposed in the camera 193.

[0055] The camera 193 is used to capture still images or videos. An object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard format such as RGB or YUV. In some embodiments, the electronic device may include one or N cameras 193, where N is a positive integer greater than 1.

[0056] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc.

[0057] The video codec is used to compress or decompress digital videos. The electronic device 100 can support one or more video codecs. In this way, the electronic device can play or record videos in multiple coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0058] The NPU is a neural-network (NN) computing processor. By learning from the structure of biological neural networks, such as the transmission pattern between human brain neurons, it can quickly process the input information and can also continuously self-learn. Through the NPU, applications such as intelligent cognition of the electronic device can be realized, such as image recognition, face recognition, speech recognition, text understanding, etc.

[0059] The DPU is a display processing unit whose responsibility is to process and optimize the displayed image and video content to provide a better visual experience.

[0060] The electronic device 100 can implement audio functions through the audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor, etc. For example, music playback, recording, etc. In the embodiments of the present application, the audio module 170 can be used to collect audio during video recording.

[0061] The external memory interface 120 can be used to connect to an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 110 through the external memory interface 120 to implement the data storage function. For example, audio, video and other files are saved in the external memory card.

[0062] The internal memory 121 can be used to store computer-executable program codes, and the executable program codes include instructions. The processor 110 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 121. For example, in the embodiments of the present application, the processor 110 can execute the instructions stored in the internal memory 121. The internal memory 121 can include a storage program area and a storage data area. Among them, the storage program area can store the operating system and application programs required for at least one function (such as the sound playback function, image playback function, etc.). The storage data area can store data created during the use of the electronic device (such as video files, etc.). In addition, the internal memory 121 can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0063] The keys 190 include a power-on key, volume keys, etc. The keys 190 can be mechanical keys or touch keys. The motor 191 can generate vibration prompts. The motor 191 can be used for incoming call vibration prompts and can also be used for touch vibration feedback. The indicator 192 can be an indicator light, which can be used to indicate the charging state, power change, and can also be used to indicate messages, missed calls, notifications, etc. The SIM card interface 195 is used to connect to the SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation from the electronic device. The electronic device can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc.

[0064] In the embodiments of the present application, the above-mentioned electronic device is an electronic device that can run an operating system and install application programs. Optionally, the operating system run by the electronic device can be system, system, Systems, etc.

[0065] In some embodiments, the software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, or a cloud architecture. In the embodiments of this application, the layered architecture system is taken as an example to exemplarily illustrate the software structure of the electronic device 100.

[0066] Figure 2 It is a schematic diagram of the software architecture of the electronic device provided in the embodiments of this application.

[0067] It can be understood that the layered architecture divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the system may include an application (App) layer, a framework (FWK) layer, a hardware abstraction layer (HAL), and a kernel layer.

[0068] The application layer may include a series of application packages. The application packages may include a camera application, a video playback application, etc. The camera application is used for taking pictures, shooting videos, etc. The video playback application is used for playing video files. In some embodiments, the application packages may also include applications such as a gallery, a calendar, a call, music, and a short message.

[0069] The framework layer provides application programming interfaces (APIs) and programming frameworks for the applications in the application layer. The framework layer includes some predefined functions. The framework layer provides programming services to the application layer for calling through the API interface, enabling the applications to interact with the hardware and the operating system. Exemplarily, the framework layer includes a camera service, a surface compositor, a render engine, etc. The corresponding services are used to manage the processing processes of the corresponding applications. The SurfaceFlinger is a graphics compositor in the system, responsible for compositing multiple graphics layers and displaying them on the display screen; the SurfaceFlinger coordinates and manages the rendering and display processes of the graphics. The RenderEngine is a graphics rendering engine in the system, responsible for presenting the graphics layers on the display screen; it implements the drawing and compositing operations of the graphics.

[0070] The hardware abstraction layer is used to abstract the hardware, encapsulate the drivers of the kernel layer, and provide interfaces upward. Exemplarily, such as Figure 2As shown, the hardware abstraction layer includes the Camera HAL, the Hardware Composer (HWComposer, HWC), etc. The Camera HAL (Camera Hardware Abstraction Layer) is an important software component in the system for processing camera hardware. It provides an interface for interacting with the camera hardware, allowing applications to communicate with and control the camera hardware (camera). The HWC is the hardware composer in the Android system, which is used to synthesize graphic layers into a displayable frame. It works in cooperation with SurfaceFlinger to achieve hardware-accelerated rendering and synthesis of graphics.

[0071] The kernel layer provides underlying drivers for various hardware of the electronic device. Exemplarily, the kernel layer may include a camera driver, a display driver, an audio driver, etc. The camera driver is a software layer below the Camera HAL, responsible for managing and controlling the camera hardware. It realizes camera functions through interaction with the hardware and provides an interface for the Camera HAL to use.

[0072] The electronic device can capture pictures or videos through the camera and save the captured pictures or videos in the storage space of the electronic device. A video is composed of frames of images. When the electronic device plays a video, it is a process of generating frames of images in sequence and displaying them on the display screen according to the playback timing. A frame of image usually includes one or more layers. The operating system of the electronic device obtains information about each layer in a frame of image (such as the elements included in the layer, the layout method of the elements, etc.), synthesizes each layer, generates an image and sends it to the display.

[0073] Exemplarily, refer to Figure 2 , taking the operating system as as an example. ①, The SurfaceFlinger in the system sends the complete list information of all the layers included in a frame of image to the HWC. ②. The HWC determines the composition method of these layers according to the hardware capabilities (DPU processing capabilities). The composition methods include HWC composition (usually using the hardware composition method) or GPU composition (usually using the software composition method). The HWC will label the corresponding composition method for each layer, whether it is composed by the GPU or by the HWC. ③. The SurfaceFlinger is responsible for calling the GPU to compose all the layers labeled for GPU composition (for example, the GPU uses OpenGL ES instructions to compose the layers) into an output buffer. ④. The SurfaceFlinger hands over this output buffer and other layers labeled for HWC composition to the HWC. ⑤. The HWC calls the DPU to complete the composition of the remaining layers (the layers labeled for HWC composition). Finally, the HWC is responsible for displaying the final composition result on the display screen.

[0074] To provide a better user experience, many electronic devices support playing HDR videos. During the layer composition stage, it is usually necessary to perform tone mapping processing on the images of HDR videos. Tone mapping processing is a process of adjusting HDR images to adapt to the display screen of electronic devices so as to better display HDR images according to the capabilities of the display screen. For example, mapping the brightness of the image to the corresponding display brightness of the display screen to conform to the display capabilities of the display screen. Tone mapping is usually represented by a brightness mapping curve and is performed during the composition stage. This processing step is crucial for ensuring that HDR videos present good display effects on different display devices. Exemplarily, common tone mapping processing methods include the equation fitting tone mapping algorithm, the 3D-LUT lookup table tone mapping algorithm, etc.

[0075] Generally speaking, different tone mapping processing methods are adopted for HWC composition (i.e., DPU composition) and GPU composition. For example, in the system, when using GPU composition, the tone mapping processing method is provided by the G platform that provides the system, and usually the "equation fitting tone mapping algorithm" is used to implement tone mapping; while when using DPU composition, the tone mapping processing method is usually provided by DPU chip manufacturers, such as the Q platform, and usually the "3D-LUT lookup table tone mapping algorithm" is used to implement tone mapping. Due to the different tone mapping processing methods adopted, there are differences in the display effects of GPU composition and DPU composition when sending to the display. Exemplarily, referring to Figure 3 , there are differences in display effects such as picture color and brightness between the images generated by using GPU composition and those generated by using DPU composition during layer composition.

[0076] In one example, referring to Figure 4 , when playing an HDR video, if the layer composition of the image is performed by processing unit A (such as a DPU), and this processing unit A uses tone mapping processing method A (such as a 3D-LUT lookup table tone mapping algorithm) to perform tone mapping processing on the layer, the display effect corresponding to the image is as shown in Figure 4 image A in; if the layer composition of the image is performed by processing unit B (such as a GPU), and this processing unit B uses tone mapping processing method B (such as an equation fitting tone mapping algorithm) to perform tone mapping processing on the layer, the display effect corresponding to the image is as shown in Figure 4 image B in. Since the tone mapping processing methods used by processing unit A and processing unit B are different, there are differences in display effects such as picture color and brightness between image A and image B. In this case, during the HDR video playback process, when switching the layer composition method, the user will perceive the difference in the display effect of the image and have a visual experience of picture flickering. Such abnormal display effect problems may lead to problems such as a decline in user experience, loss of users, a decline in product quality, and limited functions.

[0077] A common solution is to make the tone mapping effects between the GPU and the DPU as consistent as possible through parameter adjustment. For example, adjust the tone mapping fitting curve of the tone mapping processing method on the GPU side so that the result of the tone mapping processing on the GPU side is closer to the tone mapping curve of the 3D-LUT lookup table tone mapping algorithm used by the DPU, so that the processing effects of the GPU composition method and the DPU composition method can be made as close as possible. This method requires parameter tuning according to the specific hardware of the electronic device and the display device, usually requiring a large number of experiments and tests; and the adjustment results may vary depending on the device and display capabilities.

[0078] The embodiment of the present application provides an image tone processing method, which synchronizes the tone mapping processing processes (including the tone mapping processing methods used and the data used for tone mapping processing) of the GPU composition method and the DPU composition method, so as to ensure that the GPU composition method and the DPU composition method achieve the same display effect (for example, there will be no display differences, jumps in image color or brightness, etc.). Avoid the picture flickering problem that occurs during the switching of the composition method, fundamentally solve the display effect difference problem between the two layer composition methods, and provide a better visual experience for users.

[0079] In one implementation, referring to Figure 5, during the process of playing an HDR video, if the layer composition is performed by processing unit A (e.g., DPU), this processing unit A performs tone mapping processing using tone mapping processing method A (e.g., 3D-LUT lookup table tone mapping algorithm), and the corresponding display effect of the image is as Figure 5 shown in image A' in Figure 5 . If the layer composition is performed by processing unit B (e.g., GPU), this processing unit B also performs tone mapping processing using tone mapping processing method A (e.g., 3D-LUT lookup table tone mapping algorithm); and, the data used by processing unit A for tone mapping processing is synchronized to processing unit B, and processing unit B also uses this data for tone mapping processing; the corresponding display effect of the image is as shown in image B' in

[0080]

[0081] Next, in conjunction with the accompanying drawings, a method for image tone processing provided by an embodiment of the present application will be described in detail.

[0082] When performing layer composition on an image, some layers use the GPU composition method, and some layers use the DPU composition method.

[0083] Generally speaking, the tone mapping processing method used in the GPU composition method is provided by the platform that provides the operating system, which is generally called the software method; in one example, the GPU composition method uses the equation fitting tone mapping algorithm for tone mapping processing. The equation fitting tone mapping algorithm aims to adjust the color and tone of an image by applying mathematical equations to improve the visual effect of the image. Exemplarily, this process includes the following steps:

[0084] Data acquisition: First, data about different display devices and images needs to be acquired. For example, it includes the display characteristics of different display devices, the color information of the image, the brightness data, etc.

[0085] Equation fitting: According to the acquired data, use a mathematical equation to fit a tone mapping curve. This tone mapping curve is used to describe the correspondence between the color and brightness of the image and the color and brightness of the display device.

[0086] Applying the mapping: Once the fitted mathematical equation is obtained, it is applied to each pixel of the image, mapping the original color and brightness of the pixel to the color and brightness of the display device. This mapping process is usually highly computationally intensive, thus requiring efficient graphics processing.

[0087] Vision Correction: Finally, vision correction is performed to ensure that the image looks natural and balanced on the display device. This process involves fine-tuning the tone mapping curve to meet the user's visual expectations.

[0088] Generally speaking, the tone mapping processing method used in the DPU synthesis method is provided by the DPU chip manufacturer, which is generally called the hardware method; in one example, the DPU synthesis method uses a 3D-LUT lookup table tone mapping algorithm for tone mapping processing. The 3D-LUT lookup table tone mapping algorithm uses a 3D lookup table to perform tone mapping processing, and adjusts the color and tone through the values ​​in the lookup table to achieve color correction and optimization of the image.

[0089] Exemplarily, this process includes the following steps:

[0090] Create a lookup table: First, create a 3D lookup table, which usually includes dimensions such as brightness, saturation, hue, etc. The purpose of this 3D lookup table is to store the mapping relationship between different colors and brightness values.

[0091] Sample data: Sample the color and brightness information of a pixel from the image, and then find the corresponding entry in the 3D lookup table.

[0092] Lookup table mapping: Through the 3D lookup table, the sampled color and brightness values ​​are mapped to the color and brightness values ​​of the display device. This mapping is achieved through the values ​​in the 3D lookup table without the need for complex mathematical calculations.

[0093] Apply mapping: Apply the mapping according to the 3D lookup table to each pixel, correcting the color and brightness of each pixel to adapt to the display device. This process usually involves fast query of the 3D lookup table and is efficient.

[0094] Correction and Optimization: Finally, additional correction and optimization steps are performed to ensure that the image appears natural in color and tone on the display device.

[0095] An image tone processing method provided by an embodiment of the present application does not adopt the tone mapping processing method (such as the equation fitting tone mapping algorithm) default provided by the operating system platform in the GPU synthesis mode, but synchronizes the tone mapping processing method (such as the 3D-LUT lookup table tone mapping algorithm) and data adopted in the DPU synthesis mode to the GPU synthesis mode. On the one hand, the DPU adopts a hardware method, and it is more difficult to change its synthesis mode. Compared with the DPU, modifying the GPU synthesis mode that adopts a software method is simpler and more convenient. On the other hand, compared with the equation fitting tone mapping algorithm, the mapping relationship of the 3D-LUT lookup table tone mapping algorithm is more accurate, and the display effect of performing tone mapping processing on an image by using the 3D-LUT lookup table tone mapping algorithm is better. An image tone processing method provided by an embodiment of the present application performs tone mapping processing by using the 3D-LUT lookup table tone mapping algorithm in the GPU synthesis mode, which is convenient to implement, has a better display effect after processing, and eliminates the HDR video frame flashing problem caused by different layer synthesis modes.

[0096] Combined with Figure 2 , Figure 6 FIG. shows a schematic diagram of an implementation manner of an image tone processing method provided by an embodiment of the present application.

[0097] In some scenarios of HDR video playback, such as browsing HDR videos in the mobile phone photo album and playing HDR videos in a video App, the HDR videos are saved (or cached) in the storage space of the electronic device. The application program for playing the HDR video transmits the layer information of all the layers included in each frame of the HDR video to the operating system; for example, the layer information of a layer may include the identifier of the layer, the control information included in the layer, the control layout information, etc.

[0098] Taking the operating system as as an example. The SurfaceFlinger in the system obtains the layer information of all the layers (Layer) included in a frame of image, and sends the layer information of all the layers (Layer) included in a frame of image to the HWC.

[0099] The HWC can determine the synthesis mode of all the layers according to the DPU hardware capabilities, and mark the synthesis mode corresponding to each layer. The SurfaceFlinger can determine the synthesis mode of each layer according to the markings of the HWC, and distribute the layers in the GPU synthesis mode to the GPU for synthesis.

[0100] In one implementation, HWC can determine whether a layer is an HDR layer based on the Dataspace attribute in the layer information. Exemplarily, if the transfer information of Dataspace is TRANSFER_ST2084 or TRANSFER_HLG, it indicates that the layer is an HDR layer. This is because the conversion curves of HDR layers and standard dynamic range (SDR) video layers are different. Correspondingly, the transfer information of Dataspace corresponding to HDR layers and SDR video layers is also different.

[0101] If HWC determines that the layer is an HDR layer and the layer is distributed to the GPU for layer composition, HWC synchronizes the data used for tone mapping processing in the DPU composition mode to the SurfaceFlinger through the HWC Callback method. In one implementation, HWC includes a CLSTC (color look-up table and statistics correction) module. The CLSTC module contains the data (i.e., the look-up table data) used by the 3D-LUT look-up table tone mapping algorithm of the DPU. Specifically, the CLSTC module of HWC obtains the AIDL (Android interface definition language) service of the DisplayEngine, and then uses Binder to send the look-up table data to the DisplayEngine. Then, the look-up table data is called back to the SurfaceFlinger using Hardware::HalAidlCallback.

[0102] The SurfaceFlinger can also determine whether a layer is an HDR layer based on the Dataspace attribute in the layer information. If it is an HDR layer, the look-up table data is passed from the SurfaceFlinger to the RenderEngine. Specifically, the look-up table data is sent to the RenderEngine for processing together with the Display Settings (screen settings) in the composition and display process. In this way, the data (look-up table data) used by the 3D-LUT look-up table tone mapping algorithm that is called back to the SurfaceFlinger is sent to the RenderEngine along the native Android path.

[0103] Then, the RenderEngine packages the data (lookup table data) used by the 3D-LUT lookup table tone mapping algorithm and the 3D-LUT lookup table tone mapping algorithm logic into shader rendering instructions, and sends the shader rendering instructions to the GPU for rendering processing. In this way, the GPU side uses the 3D-LUT lookup table tone mapping algorithm to perform tone mapping processing on the HDR video image.

[0104] In one implementation, the RenderEngine can create a 3D-LUT lookup table tone mapping shader for the GPU side. This 3D-LUT lookup table tone mapping shader is used to implement the rendering instructions for tone mapping processing using the 3D-LUT lookup table tone mapping algorithm. Exemplarily, a special linear effect shader (SkRuntimeEffectShader) is created. The linear effect shader is a shader program for graphics rendering. It affects the appearance and effect of the image in a linear manner and is used to achieve the basic rendering effect of the image. In one example, this shader is designed for the logic of performing tone mapping processing on the HDR layer using the 3D-LUT lookup table tone mapping algorithm. Exemplarily, the shader contains at least two sub-shaders. One sub-shader is the "algorithm sub-shader" for implementing the 3D-LUT lookup table tone mapping algorithm, and the other is the "data sub-shader" for providing the lookup table data for the 3D-LUT lookup table tone mapping algorithm.

[0105] In one implementation, the RenderEngine pre-generates the linear effect shader (SkRuntimeEffectShader) and generates the "algorithm sub-shader" according to the logic of the 3D-LUT lookup table tone mapping algorithm. After obtaining the lookup table data corresponding to a layer, the "data sub-shader" is replaced according to the obtained lookup table data, without the need to reconstruct the entire linear effect shader (SkRuntimeEffectShader). This improves the efficiency and performance of the system while maintaining the consistency of tone mapping.

[0106] In this method, if an HDR layer is recognized, the HWC synchronizes the data used by the tone mapping processing method adopted by the DPU synthesis method to the rendering engine. The rendering engine packages the data used for tone mapping processing in the DPU synthesis method and the tone mapping processing method logic adopted by the DPU synthesis method into shader rendering instructions, and sends the shader rendering instructions to the GPU for rendering processing. It realizes that the GPU side uses the same tone mapping processing method as the DPU synthesis method for tone mapping processing, and the GPU synthesis method and the DPU synthesis method achieve consistent display effects.

[0107] In some scenarios, the HDR video has not been generated yet, and it is impossible to determine whether it is an HDR layer based on the property Dataspace in the layer information. For example, in the camera preview scenario during photo taking, the preview layer can be processed to enhance the HDR display effect. Since the video has not been generated yet in the camera preview scenario during photo taking, it is impossible to determine whether it is an HDR layer based on the property Dataspace in the layer information.

[0108] Combined with Figure 2 , Figure 7 FIG. shows a schematic diagram of an implementation of an image tone processing method provided by an embodiment of the present application in this scenario.

[0109] In some embodiments, after the camera captures an image, the captured image is sent to the camera HAL through the camera driver. The camera HAL sends the image to the camera application through the camera service. The camera application can generate a photo preview image based on the image captured by the camera.

[0110] After the camera captures each frame of the image, the camera HAL can provide the metadata of this frame of the image (Metadata), and the metadata (Metadata) contains information related to HDR image processing. These metadata (Metadata) can be stored in the graphic buffer (GraphicBuffer) allocated by the graphic memory allocator (Gralloc). Gralloc is The graphic memory allocator in the system, which is used to manage the memory allocation and release of graphic data, and it helps to achieve the efficient allocation and management of graphic resources. GraphicBuffer is an object used to manage graphic data in the system. GraphicBuffer is actually a wrapper of the graphic buffer allocated by Gralloc, and it contains information about the buffer and a reference to the buffer. GraphicBuffer can be accessed by the HAL layer and the framework layer. The RenderEngine in the framework layer can use the handle pointing to GraphicBuffer to read the metadata (Metadata) of the image. After processing the metadata (Metadata) of the image, the RenderEngine generates the data (lookup table data) used by the 3D-LUT lookup table tone mapping algorithm, so that the GPU can perform tone mapping processing using the 3D-LUT lookup table tone mapping algorithm. The method by which the above RenderEngine obtains the lookup table data from GraphicBuffer is called Gralloc mode data synchronization.

[0111] When playing the photo preview image with enhanced HDR display effect, the camera application sends the layer information of all the layers included in each frame of the photo preview image to the The SurfaceFlinger in the system obtains the layer information of all the layers included in a frame of image, and sends the layer information of all the layers included in a frame of image to the HWC.

[0112] The HWC can determine the composition method of all the layers according to the DPU hardware capabilities, and mark the corresponding composition method for each layer. The SurfaceFlinger can determine the composition method of each layer according to the markings of the HWC, and distribute the layers with the GPU composition method to the GPU through the RenderEngine for composition.

[0113] On the other hand, if the RenderEngine determines that a layer is an HDR layer, it performs data synchronization through the Gralloc method, and obtains the data used by the tone mapping processing method (such as the 3D-LUT lookup table tone mapping algorithm) adopted in the DPU composition method, such as lookup table data.

[0114] Then, the RenderEngine packs the data (lookup table data) adopted by the 3D-LUT lookup table tone mapping algorithm and the 3D-LUT lookup table tone mapping algorithm logic into a shader rendering instruction, and sends the shader rendering instruction to the GPU.

[0115] The GPU performs rendering processing on the layer according to the shader rendering instruction, so that the GPU side realizes tone mapping processing of the HDR video image by using the 3D-LUT lookup table tone mapping algorithm.

[0116] In this scenario, the electronic device plays a photo preview image, and the HDR video has not been generated yet. The RenderEngine cannot determine whether it is an HDR layer through the Dataspace attribute in the layer information. In one implementation, after the camera application obtains the image captured by the camera, it sets a special mark for each layer with enhanced HDR display effect. For example, this special mark can be recorded in the layer information. The RenderEngine can determine that the layer is an HDR layer according to this special mark, and then can perform data synchronization through the above Gralloc method. In the camera photo preview scenario, compared with the method of transmitting the lookup table data back through the HWC, data synchronization through the Gralloc method saves the time for waiting for the HWC to prepare the layer (such as determining the layer composition method, reading the lookup table data, etc.). The method of data synchronization through the Gralloc method in this scenario can reduce the display latency of the photo preview image and improve the user experience.

[0117] Exemplarily, Figure 8The flowchart shows an implementation of how a camera application passes special tags to a Render Engine.

[0118] As Figure 8 shown, the camera application obtains the camera-related environment, which contains various information related to camera functions. For example, it includes an instance of a SurfaceView used for rendering the camera preview part. The rendering part on this SurfaceView will ultimately become the camera preview layer in SurfaceFlinger for composition and display. If the camera preview layer is a "layer with enhanced HDR display effect", the camera application marks the SurfaceView corresponding to this camera preview layer with a special tag. In one example, the camera application adds an additional integer flag "EnhanceToHDRLayer" to the SurfaceView. This integer flag persists throughout the process of creating the SurfaceView as a Layer. The SurfaceView needs to use the SurfaceControl class for more underlying layer management. Among them, SurfaceControl is a class in the system for managing Surfaces. The camera application passes this special integer flag "EnhanceToHDRLayer" to the SurfaceComposerClient and LayerState in the LibGUI layer through the Transaction of SurfaceControl and the android_view_SurfaceControl in the JNI layer. During this process, the integer flag "EnhanceToHDRLayer" carried by Layerstate will become an additional tag attribute of the Layer when the Layer is created. This additional tag attribute is recorded in the layer information. If SurfaceFlinger or RenderEngine determines that the value of this additional tag attribute in the layer information is "EnhanceToHDRLayer", it determines that the layer is an HDR layer.

[0119] It can be understood that in order to implement the above functions, the electronic device provided in the embodiments of the present application includes the corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the embodiments of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the way of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments of the present application.

[0120] The embodiments of the present application can divide the functional modules of the above electronic device according to the above method examples. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of a software functional module. It should be noted that the division of modules in the embodiments of the present application is illustrative, only a logical function division, and there can be other division methods in actual implementation.

[0121] In one example, please refer to Figure 9 , which shows a possible structural schematic diagram of the electronic device involved in the above embodiments. The electronic device 900 includes: a processing unit 910, a storage unit 920, and a display unit 930.

[0122] Among them, the processing unit 910 is used to control and manage the actions of the electronic device 900; for example, for data synchronization, generating tone mapping algorithm logic, generating shader rendering instructions, performing tone mapping processing on the HDR layer, etc. The storage unit 920 is used to save the program code and data of the electronic device 900; for example, saving lookup table data, saving HDR video files, etc. The display unit 930 is used to display the interface of the electronic device 900; for example, playing HDR videos, displaying camera preview images for taking pictures, etc.

[0123] Of course, the unit modules in the above electronic device 900 include but are not limited to the above processing unit 910, storage unit 920, and display unit 930.

[0124] Optionally, the electronic device 900 may further include an image acquisition unit for acquiring images.

[0125] Optionally, the electronic device 900 may further include an audio unit for acquiring audio, playing audio, etc.

[0126] Optionally, the electronic device 900 may further include a communication unit for supporting communication between the electronic device 900 and other devices. For example, obtaining a video file from other devices, etc.

[0127] Among them, the processing unit 910 may be a processor or a controller. For example, it may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The storage unit 920 may be a memory. The display unit 930 may be a display screen, etc. The image acquisition unit may be a camera, etc. The audio unit may include a microphone, a speaker, etc. The communication unit may include a mobile communication unit and / or a wireless communication unit.

[0128] For example, the processing unit 910 is a processor (such as Figure 1 the processor 110 shown), the storage unit 920 may be a memory (such as Figure 1 the internal memory 121 shown), the display unit 930 may be a display screen (such as Figure 1 the display screen 194 shown). The image acquisition unit may be a camera (such as Figure 1 the camera 193 shown). The audio unit may be an audio module (such as Figure 1 the audio module 170 shown). The communication unit may include a mobile communication unit (such as Figure 1 the mobile communication module 150 shown) and a wireless communication unit (such as Figure 1 the wireless communication module 160 shown). The electronic device 900 provided by the embodiments of the present application may be Figure 1 the electronic device 100 shown. Among them, the above-mentioned processor, memory, display screen, camera, audio module, mobile communication unit, wireless communication unit, etc. may be connected together, for example, connected by a bus.

[0129] An embodiment of the present application also provides a chip system, which includes at least one processor and at least one interface circuit. The processor and the interface circuit can be interconnected through a line. For example, the interface circuit can be used to receive signals from other devices (such as the memory of an electronic device). For another example, the interface circuit can be used to send signals to other devices (such as a processor). Exemplarily, the interface circuit can read instructions stored in the memory and send the instructions to the processor. When the instructions are executed by the processor, the electronic device can execute each step in the above embodiments. Of course, the chip system can also include other discrete devices, and the embodiments of the present application do not make specific limitations on this.

[0130] An embodiment of the present application also provides a computer-readable storage medium, which includes computer instructions. When the computer instructions run on the above-mentioned electronic device, the electronic device can execute each function or step that the mobile phone executes in the above method embodiment.

[0131] An embodiment of the present application also provides a computer program product. When the computer program product runs on a computer, the computer can execute each function or step that the mobile phone executes in the above method embodiment.

[0132] Through the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and brevity of description, only the above division of each functional module is used as an example. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0133] In several embodiments provided by the present application, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the module or unit is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical, mechanical or other form.

[0134] The unit described as a separated component may or may not be physically separated. The component displayed as a unit may be a physical unit or multiple physical units, that is, it can be located in one place or distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0135] In addition, in each embodiment of the present application, each functional unit can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0136] If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions for causing a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods described in the embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0137] The above content is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An image tone processing method, characterized in that, Applied to an electronic device, the electronic device includes a processor, and an operating system of the electronic device runs on the processor. The processor includes a graphics processing unit (GPU) and a display processing unit (DPU). The method includes: The operating system obtains layer information of a first layer of a first image; the layer information includes at least one of an identifier of the layer, control information included in the layer, and control layout information; If it is determined that the first image is a high-dynamic range (HDR) image and the layer composition method corresponding to the first layer is a GPU composition method, the operating system obtains first data; the first data is data used by the DPU for tone mapping processing; The operating system generates a shader rendering instruction according to the first data and a first tone mapping processing method; the first tone mapping processing method is the tone mapping processing method used by the DPU for tone mapping processing; The GPU performs tone mapping processing on the first layer according to the shader rendering instruction.

2. The method according to claim 1, characterized in that, The operating system includes a rendering engine. The operating system generating a shader rendering instruction according to the first data and a first tone mapping processing method includes: The rendering engine generates a shader rendering instruction according to the first data and the first tone mapping processing method.

3. The method according to claim 2, wherein The operating system further includes a surface compositor and a hardware compositor (HWC). The operating system obtaining first data includes: The HWC obtains the first data from the DPU; The HWC sends the first data to the rendering engine through the surface compositor.

4. The method according to claim 3, wherein The first image is a frame image in an HDR video stored on the electronic device.

5. The method according to claim 2, wherein The operating system obtaining first data includes: The rendering engine generates the first data according to metadata of the first image.

6. The method according to claim 5, characterized in that The first image is a preview image of a photo with enhanced HDR display effect.

7. The method according to any one of claims 2-6, characterized in that, The method further includes: The rendering engine generates a first shader, the first shader includes a first sub-shader and a second sub-shader. The first sub-shader is used to implement the algorithm logic of the first tone mapping processing method, and the second sub-shader is used to provide the first data.

8. The method according to claim 7, wherein The first sub-shader is pre-generated, and the second sub-shader is generated after the rendering engine obtains the first data.

9. The method according to any one of claims 1 to 8, characterized in that, The determining that the first image is a high-dynamic range (HDR) image includes: Determining that the first image is a high-dynamic range (HDR) image according to the layer information of the first layer.

10. The method according to any one of claims 1-9, characterized in that, The first tone mapping processing method is a 3D-LUT lookup table tone mapping algorithm.

11. An electronic device, characterized in that, The electronic device includes: a processor, a memory, and a display screen; the processor is coupled to the memory and the display screen; the memory is used to store computer program code; the computer program code includes computer instructions. When the processor executes the above computer instructions, the electronic device executes the method according to any one of claims 1-10.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer instructions that, when run on an electronic device, cause the electronic device to perform the method according to any one of claims 1-10.

Images