Display method and electronic device

Abstract

The application discloses a display method and electronic equipment, the method comprises the following steps: at a first time point, a video channel module in a DPU channel acquires data of a first video from an HDR video frame; then a three-dimensional lookup table module processes the data of the first video to obtain a first picture of the first video, and the first picture is displayed through a first display module; at the same time, the CPU can store the three-dimensional lookup table in the DPU channel. At a second time point, in response to displaying a first window in a display interface of the first video, the CPU converts the three-dimensional lookup table into a two-dimensional image and transmits the two-dimensional image to a GPU channel. The data of the first video is processed through a restoration module and a three-dimensional mapping module in the GPU channel to obtain a second picture of the first video, and the second picture is displayed through a second display module. Through the application, the problem of brightness jump of a video picture can be avoided, and the visual effect is ensured.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the computer field, and in particular, to a display method and an electronic device. BACKGROUND

[0002] In computer graphics and cinematography, high-dynamic range imaging (HDR) is an image technology that can display a larger range of luminance and contrast. HDR can make dark details brighter and bright details not distorted, presenting a more natural and realistic picture, thereby improving the stereoscopic sense and comfort of the picture. The purpose of HDR is to correctly represent the range of luminance from direct sunlight to the darkest shadow in the real world, and better reflect the visual effects in the real environment.

[0003] However, in the process of playing an HDR video by a user using an electronic device, the following situations may occur:

[0004] (1) The user operates a physical volume button on the electronic device, at which time a volume window is popped up on the display interface.

[0005] (2) The user touches the picture of the HDR video, at which time a sidebar window is popped up on the display interface.

[0006] When these situations occur, the video picture is affected to some extent, thereby reducing the visual effect. SUMMARY

[0007] Embodiments of the present application provide a display method and an electronic device. Based on the method described in the present application, the problem of luminance jump of the video picture can be avoided, and the visual effect is ensured.

[0008] In a first aspect, the present application provides a display method, which comprises: displaying a picture of a first video at a first time point; displaying a first window in the display interface of the first video at a second time point; the picture luminance of the first video at the first time point is the same as the picture luminance of the first video at the second time point, and the second time point is later than the first time point.

[0009] Based on the method described in the first aspect, the picture luminance of the first video at the first time point is the same as the picture luminance of the first video at the second time point, which can avoid the problem of luminance jump of the video picture, and ensure the visual effect.

[0010] In a possible implementation, at the first time point, a picture of a first video is displayed, including: at the first time point, processing data of the first video based on a first channel to obtain a first picture of the first video, and displaying the first picture of the first video; the method further includes: at a second time point, in response to displaying a first window in a display interface of the first video, processing data of the first video based on a second channel to obtain a second picture of the first video; and displaying the second picture of the first video; wherein the second channel includes a two-dimensional image corresponding to a three-dimensional lookup table in the first channel. Based on this manner, the problem of brightness jump of a video picture can be avoided.

[0011] In a possible implementation, the method further includes: mapping the three-dimensional lookup table into a two-dimensional image; the data of the first video includes a first pixel value of the first video, processing the data of the first video based on the second channel to obtain the second picture of the first video includes: in the second channel, obtaining the first pixel value of the first video; processing the first pixel value based on the two-dimensional image to obtain a second pixel value; and determining the second picture of the first video based on the second pixel value. Based on this manner, the image can be ensured not to be distorted.

[0012] In a possible implementation, the two-dimensional image includes a first two-dimensional image and a second two-dimensional image, and mapping the three-dimensional lookup table into a two-dimensional image includes: determining first position information of an input pixel value in the three-dimensional lookup table mapping to the first two-dimensional image, and second position information of an output pixel value in the three-dimensional lookup table mapping to the second two-dimensional image; determining the first two-dimensional image based on the input pixel value and the first position information, and determining the second two-dimensional image based on the output pixel value and the second position information. Based on this manner, the accuracy of the two-dimensional image corresponding to the input pixel value and the two-dimensional image corresponding to the output pixel value can be ensured.

[0013] In a possible implementation, processing the first pixel value based on the two-dimensional image to obtain a second pixel value includes: determining index information corresponding to the first pixel value; determining an input pixel value corresponding to the index information in the first two-dimensional image, and determining an output pixel value corresponding to the index information in the second two-dimensional image; and performing interpolation calculation on the index information corresponding to the input pixel value, the index information corresponding to the output pixel value, and the first pixel value to obtain the second pixel value. Based on this manner, the interpolation manner of the three-dimensional lookup table is used to replace the traditional photoelectric conversion and electro-optical conversion function, the difference of the image can be avoided, and thus the problem of brightness jump of a video picture can be avoided.

[0014] In a possible implementation, the method further includes: closing the first window in the display interface of the first video at a third time point; the brightness of the picture of the first video at the second time point is the same as the brightness of the picture of the first video at the third time point, and the third time point is later than the second time point. Based on this manner, the problem that the brightness of the video picture jumps can be avoided.

[0015] In a possible implementation, the method further includes: switching the first video to a window mode at a fourth time point; the brightness of the picture of the first video at the third time point is the same as the brightness of the picture of the first video at the fourth time point, and the fourth time point is later than the third time point. Based on this manner, the problem that the brightness of the video picture jumps can be avoided.

[0016] In a possible implementation, the first video is a high dynamic range imaging (HDR) video. Since the HDR video is an image with high brightness and high saturation, when the HDR video is blocked, the problem of brightness jump is more likely to occur, and therefore the problem that the brightness of the video picture jumps can be more obviously avoided for the HDR video.

[0017] In a second aspect, the present application provides a display device, which can be an electronic device, a device in an electronic device, or a device that can be used with an electronic device; wherein the display device can also be a chip system, and the display device can execute the method performed by the electronic device in the first aspect. The functions of the display device can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more units corresponding to the above functions. The unit can be software and / or hardware. The operations and advantages of the display device can be referred to the method and advantages of the first aspect, and the repeated parts will not be described here.

[0018] In a third aspect, the present application provides a display device, which includes a processor, and when the processor invokes a computer program in a memory, the method in the first aspect is executed.

[0019] In a fourth aspect, the present application provides a display device, which includes a processor and a memory, and the processor and the memory are coupled; the processor is configured to implement the method in the first aspect.

[0020] In a fifth aspect, the present application provides a display device, which includes a processor, a memory, and a transceiver, and the processor and the memory are coupled; the transceiver is configured to transceive data, and the processor is configured to implement the method in the first aspect.

[0021] In a sixth aspect, the present application provides an electronic device, comprising one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are configured to store computer program codes, the computer program codes comprising computer instructions, which, when executed by the one or more processors, cause the electronic device to perform the display method in any possible implementation of the first aspect.

[0022] In a seventh aspect, the present application provides a chip, comprising a processor and an interface, the processor and the interface being coupled; the interface is configured to receive or output signals, and the processor is configured to execute code instructions, so that the method of the first aspect is executed.

[0023] In an eighth aspect, the present application provides a display system, comprising an electronic device; wherein the electronic device is configured to execute the method of the first aspect.

[0024] In a ninth aspect, the present application provides a display device, comprising a function or unit for executing the method of any one of the first aspect.

[0025] In a tenth aspect, the present application provides a computer readable storage medium, the computer readable storage medium storing a computer program, the computer program comprising program instructions, which, when executed on a display device, cause the display device to perform the display method in any possible implementation of the first aspect.

[0026] In an eleventh aspect, the present application provides a computer program product, which, when executed on a computer, causes the computer to perform the display method in any possible implementation of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1A is a schematic diagram of a three-dimensional lookup table provided by an embodiment of the present application;

[0028] Figure 1B is a schematic diagram of a DPU path and a GPU path provided by an embodiment of the present application;

[0029] Figure 1C is a schematic diagram of a display volume window provided by an embodiment of the present application;

[0030] Figure 1D is a schematic diagram of a display sidebar window provided by an embodiment of the present application;

[0031] Figure 1E is a schematic diagram of a video picture switching to a window mode provided by an embodiment of the present application;

[0032] Figure 2is a hardware structure schematic diagram of an electronic device provided by an embodiment of the present application;

[0033] Figure 3 is a software structure block diagram of an electronic device provided by an embodiment of the present application;

[0034] Figure 4 is another schematic diagram of a DPU channel and a GPU channel provided by an embodiment of the present application;

[0035] Figure 5 is a flow schematic diagram of a display method provided by an embodiment of the present application;

[0036] Figure 6A is a schematic diagram of a position relationship of a three-dimensional lookup table mapping to a two-dimensional image provided by an embodiment of the present application;

[0037] Figure 6B is a schematic diagram of a two-dimensional image provided by an embodiment of the present application;

[0038] Figure 7A is a schematic diagram of a smallest cube in which a first pixel value is located provided by an embodiment of the present application;

[0039] Figure 7B is a schematic diagram of an interpolation operation based on a smallest cube in which a first pixel value is located provided by an embodiment of the present application;

[0040] Figure 8 is another flow schematic diagram of a display method provided by an embodiment of the present application;

[0041] Figure 9A is a flow schematic diagram of determining a second picture of a first video provided by an embodiment of the present application;

[0042] Figure 9B is a flow schematic diagram of mapping a three-dimensional lookup table to a two-dimensional image provided by an embodiment of the present application;

[0043] Figure 9C is a flow schematic diagram of determining a second pixel value provided by an embodiment of the present application;

[0044] Figure 10 is another flow schematic diagram of a display method provided by an embodiment of the present application;

[0045] Figure 11 is a structure schematic diagram of a display device provided by an embodiment of the present application;

[0046] Figure 12 is a structure schematic diagram of a chip provided by an embodiment of the present application. DETAILED DESCRIPTION

[0047] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " represents the meaning of or, for example, A / B can represent A or B; the "and / or" in the text only describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which means that there are three cases of A alone, A and B together, and B alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0048] Hereinafter, the terms "first" and "second" are used for descriptive purposes only, and cannot be understood as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first" and "second" can explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, the meaning of "multiple" is two or more than two.

[0049] In the following embodiments of the present application, the term "user interface (UI)" is a medium interface for interaction and information exchange between an application or an operating system and a user, which realizes the conversion between the internal form of information and the form that the user can accept. The user interface is source code written in a specific computer language such as Java, extensible markup language (XML), etc. The interface source code is parsed, rendered on the terminal device, and finally presented as content that the user can recognize. The commonly used form of user interface is graphic user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be a visual interface element such as text, icon, button, menu, tab, text box, dialog box, status bar, navigation bar, Widget, etc. displayed in the display screen of the terminal device.

[0050] In order to facilitate understanding of the schemes provided by the embodiments of the present application, the related concepts related to the embodiments of the present application are introduced as follows:

[0051] 1. High-dynamic range imaging (high-dynamic range, HDR)

[0052] In computer graphics and cinematography, HDR is an image technology that can display a larger range of brightness and contrast, and can reproduce a larger dynamic range of luminance than standard digital imaging or photography technology. HDR can make dark details brighter, bright details not distorted, and present a more natural and realistic picture, thereby enhancing the stereoscopic sense and comfort of the picture. The purpose of HDR is to correctly represent the range of brightness from direct sunlight to the darkest shadow in the real world, and better reflect the visual effects in the real environment.

[0053] 2、3D look-up-table (3DLut)

[0054] Lut is the abbreviation of look-up-table, which is a container of data, similar to a dictionary, a map, etc. It is a description method of corresponding index and value. 3DLut means that the corresponding method between index and value is three-dimensional, which can be understood as the combination of the corresponding method between three-dimensional index and value, that is, through three-dimensional index, a value can be found. 3DLut is an algorithm for re-tuning the color of an image by establishing a color mapping table. The principle is to perform color correction and mapping on the image to achieve the desired effect. 3DLut can be understood as a color conversion matrix, in which each input pixel value (i.e. inputLut) has a corresponding output pixel value (outputLut). Each element in this conversion matrix represents the corresponding color conversion relationship. By performing color conversion on each pixel, the color of the image can be more accurate and realistic.

[0055] As shown in Figure 1A , the working principle of 3DLut is to divide the RGB color space into multiple small cubes (such as n*n*n 3DLut, where n represents the order of 3DLut), and each small cube corresponds to an output pixel value. Specifically, when the input pixel value enters the 3DLut, it will be mapped to the corresponding output pixel value according to its position in the color space, thereby realizing color correction and mapping.

[0056] 3、data processing unit (DPU), graphics processing unit (GPU)

[0057] DPU is a kind of processor used in data center to process data transmitted in data center. It can be used to build the main chip of smart NIC, in addition to completing the transmission task of traditional network, it can also complete part or all of the infrastructure management task originally completed by CPU, so it is called "smart" NIC. DPU is usually used to work with CPU and GPU, becoming one of the three processors in data center, to improve the performance and efficiency of data center.

[0058] GPU is another processor of computer, mainly used to meet the requirements of image calculation. Relatively speaking, CPU is good at logical judgment and serial data operation, while each pixel of a picture needs the same calculation processing, and GPU is good at parallel task of graphics calculation. Because of the high parallelism of GPU, it has also derived a general-purpose GPU that weakens the image capability and focuses on calculation. GPU can process a large amount of data at the same time, so it is faster than CPU in some application scenarios. GPU was originally used for graphics processing, but now it is also widely used in deep learning, scientific computing, cryptography, data mining and other fields. Currently, artificial intelligence and general large models can also use GPU for parallel training. In general, GPU is mainly used for graphics calculation, and DPU is mainly used for processing data transmitted in data center.

[0059] As shown in Figure 1B , when the user plays the HDR video, the HDR video frame is normally processed through the DPU channel. In the processing process of DPU, the video pipeline module first obtains the HDR video frame from the HDR video frame; then the three-dimensional lookup table module (3DLut module) calculates the corresponding three-dimensional lookup table (3DLut), uses the 3DLut to correct and map the HDR video frame, and then displays the converted video picture through the first display module (DPU RGB display). However, during the user's use of the electronic device to play the HDR video, the following situations may occur:

[0060] (1) As shown in Figure 1C , the user operates the physical volume button on the electronic device, at which time the display interface pops up a volume window;

[0061] (2) As shown in Figure 1D , the user touches the picture displaying the HDR video, at which time the display interface pops up a sidebar window.

[0062] (3) As shown in Figure 1E , the user switches the video picture to window mode by dragging the display interface of the HDR video.

[0063] When the above situations occur, the HDR video frame is processed through the GPU channel. In the process of the GPU, the HDR video frame is subjected to non-linear mapping of an electro-optical transfer function (EOTF), color conversion, tone mapping (ToneMapping), and inverse mapping of the OETF, and then displayed through a second display module (GPU RGB display) to achieve the display effect of the DPU as much as possible. However, it is found through tests that there is a large difference in display brightness between the two, thereby causing the problem of brightness jump of the video picture in the process of switching channels, which has a certain influence on the video picture and reduces the visual effect.

[0064] In order to avoid the problem of brightness jump of the video picture and ensure the visual effect, the present application provides a display method and an electronic device. In the specific implementation, the display method mentioned above can be executed by the electronic device 100. The electronic device 100 can be a mobile phone, a tablet computer, a notebook computer, or a wearable electronic device (such as a smart watch) with wireless communication function, but is not limited thereto. The electronic device 100 is configured with a display screen and can be installed with a preset application (APP), such as a gallery APP, a video APP, a camera APP, etc. The user can play the HDR video through the gallery APP or the video APP, or preview the HDR picture through the camera APP. Of course, the user can also switch the video picture to the window mode through the way of dragging the display interface.

[0065] The hardware structure of the electronic device 100 will be introduced as follows. Please refer to Figure 2 , Figure 2 is a hardware structure schematic diagram of the electronic device 100 provided by the present application.

[0066] The electronic device 100 can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headset interface 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 can include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0067] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than shown, or combine certain components, or split certain components, or different arrangement of components. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0068] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units can be independent devices, or can be integrated in one or more processors.

[0069] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching and executing instructions.

[0070] The processor 110 can also have internal memory for storing instructions and data. In some embodiments, the internal memory of the processor 110 is a cache memory. The cache memory can hold instructions or data that the processor 110 has recently accessed or is likely to access again.

[0071] If the processor 110 needs to use the instructions or data again, it can be directly called from the cache memory. This avoids repeated access and reduces the waiting time of the processor 110, thereby improving the efficiency of the system. The processor 110 calls the instructions or data stored in the memory to make the electronic device 100 perform the display method performed by the electronic device in the following method embodiments.

[0072] In some embodiments, the processor 110 can include one or more interfaces. The interface can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0073] The charging management module 140 is configured to receive a charging input from a charger. The charger can be a wireless charger or a wired charger.

[0074] The power management module 141 is configured to connect the battery 142 and the charging management module 140 to the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, and the wireless communication module 160, among other components. In some embodiments, the power management module 141 can also be disposed in the processor 110.

[0075] The wireless communication function of the electronic device 100 can be realized through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor, etc.

[0076] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0077] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation.

[0078] The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via the antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 can be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 can be housed in the same device.

[0079] A modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor.

[0080] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLAN) (such as Wi-Fi networks), Bluetooth (BT), BLE broadcasting, global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technology.

[0081] The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency-modulates and filters the electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module 160 can also receive signals to be transmitted from the processor 110, frequency-modulate them, amplify them, and convert them into electromagnetic wave radiation via the antenna 2.

[0082] In some embodiments, the antenna 1 and the mobile communication module 150 of the electronic device 100 are coupled, and the antenna 2 and the wireless communication module 160 are coupled, so that the electronic device 100 can communicate with a network and other devices through wireless communication technology.

[0083] The electronic device 100 implements a display function through a GPU, a display screen 194, and an application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs that execute program instructions to generate or change display information.

[0084] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the electronic device 100 can include 1 or N display screens 194, N being a positive integer greater than 1.

[0085] The electronic device 100 can implement a photographing function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor, etc. The ISP is used to process data fed back by the camera 193. The camera 193 is used to capture still images or videos. The digital signal processor is used to process digital signals, which can process not only digital image signals but also other digital signals. The video codec is used to compress or decompress digital videos. The electronic device 100 can support one or more video codecs.

[0086] The NPU is a neural-network (NN) computing processor that processes input information quickly by drawing on the structure of a biological neural network, such as the transmission mode between human brain neurons, and can also constantly self-learn.

[0087] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function.

[0088] The internal memory 121 can be used to store computer executable program codes including instructions. The processor 110 performs various functional applications and data processing of the electronic device 100 by executing the instructions stored in the internal memory 121. The internal memory 121 can include a program storage area and a data storage area. The program storage area can store an operating system, at least one application program (e.g., a sound play function) required for a function, and the like. The data storage area can store data (e.g., audio data) created during the use of the electronic device 100, and the like. In addition, the internal memory 121 can include a high-speed random access memory, and can further include a non-volatile memory such as a flash memory device, and the like.

[0089] The electronic device 100 can implement an audio function through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor, and the like. For example, music play, recording, and the like.

[0090] The audio module 170 is used to convert digital audio information into an analog audio signal output, and is also used to convert an analog audio input into a digital audio signal. The audio module 170 can also be used to encode and decode an audio signal. In some embodiments, the audio module 170 can be disposed in the processor 110, or part of the functions of the audio module 170 can be disposed in the processor 110.

[0091] The speaker 170A, also referred to as a "loudspeaker", is used to convert an audio electrical signal into a sound signal. The receiver 170B, also referred to as a "earpiece", is used to convert an audio electrical signal into a sound signal. The microphone 170C, also referred to as a "microphone", "sound transducer", is used to convert a sound signal into an electrical signal. The earphone interface 170D is used to connect a wired earphone. The pressure sensor 180A is used to sense a pressure signal, and can convert the pressure signal into an electrical signal.

[0092] In some embodiments, a pressure sensor 180A may be disposed on a display screen 194. A gyroscope sensor 180B may be used to determine the motion posture of the electronic device 100. A barometric pressure sensor 180C is used to measure barometric pressure. A magnetic sensor 180D includes a Hall effect sensor. An accelerometer sensor 180E can detect the magnitude of acceleration of the electronic device 100 in various directions (generally triaxial). A distance sensor 180F is used to measure distance. A proximity sensor 180G may include, for example, a light-emitting diode (LED) and a photodetector. An ambient light sensor 180L is used to sense ambient light intensity. A fingerprint sensor 180H is used to collect fingerprints. A temperature sensor 180J is used to detect temperature. A touch sensor 180K, also called a "touch panel," may be disposed on the display screen 194. The touch sensor 180K and the display screen 194 together form a touch screen, also called a "touchscreen." The touch sensor 180K is used to detect touch operations applied to or near it. A bone conduction sensor 180M can acquire vibration signals. Buttons 190 include a power button, volume buttons, etc. A motor 191 can generate vibration feedback. An indicator 192 can be an indicator light, used to indicate charging status, battery level changes, and also to indicate messages, missed calls, notifications, etc. A SIM card interface 195 is used to connect a SIM card.

[0093] Furthermore, an operating system runs on top of the aforementioned components. Examples include iOS and Android. The operating system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to exemplify the software structure of electronic device 100. It should be noted that although this application embodiment uses the Android system as an example for illustration, its basic principles are equally applicable to electronic devices with other operating systems.

[0094] Figure 3 This is a software structure block diagram of the electronic device 100 according to an embodiment of this application. The software structure adopts a layered architecture, which divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In this embodiment, the operating system (taking the Android system, which runs on an AP as an example) can be divided into six layers, from top to bottom: application layer (APP), application framework layer (FWK), Android runtime and system library, hardware abstraction layer (HAL), kernel layer, and hardware layer.

[0095] The application layer can include a series of application packages. For example... Figure 3As shown, the application package can include multimedia applications, camera applications, call applications, Bluetooth applications, and the like. The application layer can also include a system UI (systemUI) for displaying interfaces of the electronic device, such as a video screen, a photograph preview interface, and the like.

[0096] The application framework layer provides an application programming interface (API) and a programming framework for the applications of the application layer. The application framework layer includes some pre-defined functions. For example, Figure 3 As shown, the application framework layer can include a multimedia framework, a view system, a phone manager, a window manager, a notification manager, a content provider, a resource manager, and the like, without any limitation in the embodiments of the present application.

[0097] The multimedia framework is used for protocol, encapsulation, decoding, transcoding, and the like of multimedia data, and provides processing capabilities for multimedia data.

[0098] In the embodiments of the present application, as shown, Figure 4 The multimedia framework includes an HDR video frame, a DPU pipeline, a CPU, and a GPU pipeline. The DPU pipeline includes a video pipeline module, a three-dimensional lookup table module (3DLut module), and a first display module (DPU RGB display); the GPU pipeline includes a restoration module (3DLut restoration), a three-dimensional mapping module (Lut mapping), and a second display module (GPU RGB display). The CPU can store the three-dimensional lookup table in the DPU pipeline, and can also convert the three-dimensional lookup table into a two-dimensional image (2D 10bit image) to be transmitted to the GPU pipeline.

[0099] The view system includes visual controls, such as a control for displaying text, a control for displaying pictures, and the like. The view system can be used to build an application. A display interface can be composed of one or more views. For example, a display interface including a short message notification icon can include a view for displaying text and a view for displaying pictures.

[0100] The phone manager is used to provide communication functions of the electronic device 100. For example, management of call states (including call connection, call hang-up, and the like).

[0101] The window manager is used to manage window programs. The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, and the like.

[0102] The notification manager enables applications to display notification information in the status bar, which can be used to convey alert-type messages that can automatically disappear after a brief stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the form of a figure or a scroll bar text in the top status bar of the system, such as a notification of an application running in the background, and can also be a notification that appears in the form of a dialog window on the screen. For example, the status bar prompts text information, emits a prompt sound, the electronic device vibrates, the indicator light flashes, etc.

[0103] The content provider is used to store and obtain data, and enables the data to be accessed by applications. The data can include videos, images, audio, dialed and received calls, browsing history and bookmarks, phone books, etc.

[0104] The resource manager provides various resources for applications, such as localized strings, icons, pictures, layout files, video files, etc.

[0105] The Android runtime includes a core library and a virtual machine. The Android runtime is responsible for scheduling and managing the Android system.

[0106] The core library includes two parts: one part is the function function that the java language needs to call, and the other part is the core library of Android.

[0107] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the java files of the application layer and the application framework layer into binary files. The virtual machine is used to perform object lifecycle management, stack management, thread management, security and exception management, and garbage collection functions.

[0108] The system library can include multiple functional modules. For example: surface manager, media library, three-dimensional graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0109] The surface manager is used to manage the display subsystem and provides 2D and 3D layer fusion for multiple applications.

[0110] The media library supports multiple commonly used audio, video format playback and recording, and static image files, etc. The media library can support multiple audio and video encoding formats.

[0111] The three-dimensional graphics processing library is used to realize three-dimensional graphics drawing, image rendering, synthesis, and layer processing, etc.

[0112] The 2D graphics engine is a drawing engine for 2D drawing.

[0113] Hardware abstraction layer is an interface layer between the operating system kernel and hardware circuit, which aims to abstract hardware. It hides the hardware interface details of a specific platform and can provide a virtual hardware platform for the operating system. Hardware abstraction layer is a packaging of Linux kernel driver, which provides an interface to the upper layer and shields the implementation details of the low-level hardware. As shown in Figure 3 Hardware abstraction layer can include audio HAL, Wi-Fi HAL, etc. Among them, audio HAL is responsible for a part of the audio hardware abstraction layer, which provides an interface between the application and the audio hardware.

[0114] Kernel layer is the layer between hardware and software, which is the core of an operating system, the first layer of software expansion based on hardware, provides the most basic functions of the operating system, and is the basis of the operating system. It is responsible for managing the processes, memory, device drivers, files and network systems of the system, and determines the performance and stability of the system. The kernel layer can include display driver, audio driver, camera driver, sensor driver, Bluetooth driver, etc.

[0115] Hardware layer includes display, camera, sensor, etc.

[0116] Based on the above software structure, an embodiment of the present application provides a flowchart of a display method. As shown in Figure 5 The display method includes the following steps S501-S509. Among them:

[0117] S501, at a first time point, the video pipeline module obtains the data of the first video from the HDR video frame.

[0118] In the embodiment of the present application, the HDR video frame can be a video frame of the first video. Here, the first video can be an HDR video obtained from a cloud server, can be an HDR picture collected by a camera when a user uses an electronic device to preview a scene, or can be an HDR video obtained from other electronic devices, which is not limited here.

[0119] At the first time point, the user uses the electronic device to play the first video. Under normal circumstances, the first video will be processed through the DPU channel. In the processing process of DPU, the video pipeline module in the DPU channel will first obtain the data of the first video from the HDR video frame, and deliver the data of the first video to the three-dimensional lookup table module for processing.

[0120] It should be noted that the data transmission (data sending and receiving) between modules is actually realized by functions.

[0121] S502, a three-dimensional lookup table module processes data of the first video to obtain a first picture of the first video.

[0122] In the embodiment of the present application, the three-dimensional lookup table module (3DLut module) in the DPU channel calculates the 3DLut corresponding to the first video, uses the 3DLut to perform color correction and mapping on the data of the first video to obtain the first picture of the first video, and transmits the first picture of the first video to the first display module.

[0123] The data of the first video can include first pixel values of the first video, and the working principle of the 3DLut is to input the first pixel values of the first video as input pixel values of the 3DLut into the 3DLut, and then map to corresponding output pixel values according to the positions of the 3DLut in the color space, so as to realize color correction and mapping.

[0124] S503, the first display module displays the first picture of the first video.

[0125] In the embodiment of the present application, the first display module (DPU RGB display) in the DPU channel displays the first picture of the first video obtained by processing. That is, the picture of the first video viewed by the user through the display screen at the first time point is the first picture of the first video processed by the DPU channel.

[0126] S504, the CPU stores the three-dimensional lookup table obtained from the three-dimensional lookup table module.

[0127] In the embodiment of the present application, the three-dimensional lookup table can be stored in the CPU, or can be stored in other storage spaces (for example, the first storage space can store the three-dimensional lookup table obtained from the three-dimensional lookup table module), which is not limited here. The CPU can further store the 3DLut corresponding to the first video calculated in the DPU channel, so that when subsequent GPU channel processing is required, the CPU can convert the 3DLut corresponding to the first video into a two-dimensional image and transmit it to the GPU channel. Thus, the GPU channel can also be equivalent to using the 3DLut to correct and map the color, so that the same effect as the DPU channel processing can be presented when the GPU channel is processed, and the problem of brightness jump of the video picture is solved.

[0128] The execution order of steps S503 and S504 is not limited.

[0129] S505, at a second time point, in response to displaying a first window in a display interface of the first video, the CPU maps the three-dimensional lookup table into a two-dimensional image and transmits it to the GPU channel.

[0130] In this embodiment of the application, at the second time point, the user operates the physical volume button on the electronic device, at which time the display interface pops up a volume window (e.g. Figure 1C (as shown); or, the user touches the screen displaying the first video, at which point a sidebar window pops up on the display interface (as shown). Figure 1D (as shown), etc.

[0131] In these cases, the electronic device switches from the DPU path to the GPU path to process the first video data. At this time, the CPU maps the previously stored 3D lookup table into a 2D image and passes it to the GPU path. This is because the GPU path cannot directly use the 3D lookup table for color correction and mapping; therefore, the CPU needs to map the 3D lookup table into a 2D image. This 2D image retains 10 bits, which helps ensure the image remains undistorted. The second time point is later than the first time point. In this way, the GPU path can also be equivalent to using the same 3D lookup table in the DPU path for color correction and mapping, enabling the GPU path to produce the same effect as the DPU path, thus resolving the problem of brightness jumps in the video image.

[0132] Specifically, it can be Figure 1A Expanding the 3DLut in the image yields the following result: Figure 6A The list shown in (a) represents a small cube in 3DLut that divides the RGB color space, with each small cube corresponding to an output pixel value.

[0133] Each row has a three-dimensional index (i.e., the index of the R channel, the index of the G channel, and the index of the B channel). By determining the three-dimensional indices of the input pixel value, the corresponding small cube can be found in 3DLut, thus mapping the corresponding output pixel value. In summary, each row in the list corresponds to a three-dimensional pixel value (either the input pixel value or the output pixel value).

[0134] Furthermore, using formulas (1) and (2), the three-dimensional indices of each row in the list (i.e., the indices of the R channel, G channel, and B channel) are mapped to two-dimensional indices (i.e., the indices of x and y) in the two-dimensional image, thereby determining the positional information of the three-dimensional pixel values ​​(i.e., input pixel values ​​or output pixel values) in the three-dimensional lookup table mapped to the two-dimensional image. Figure 6A The positional relationship is shown in (b). Formulas (1) and (2) are as follows:

[0135] x = index_R + index_G*n (1)

[0136] y = index_B (2)

[0137] In formula (1), index_R is the index of the R channel, index_G is the index of the G channel, and n is the order of 3DLut. In formula (2), index_B is the index of the B channel.

[0138] Then, the three-dimensional pixel value corresponding to each row is determined as the pixel value corresponding to each position in the two-dimensional image, thus obtaining the final two-dimensional image. Specifically, as shown in formula (3), the pixel value value(x,y) corresponding to each position in the two-dimensional image is:

[0139] value(x,y)=(R&0x03FF<<0)|(G&0x03FF<<10)|(B&0x03FF<<20)|(0xc0000000)(3)

[0140] like Figure 6B As shown, 3DLut includes input 3DLut (input pixel value) and output 3DLut (output pixel value), thus mapping to two two-dimensional images: input 3DLut can be mapped to the first two-dimensional image, and output 3DLut can be mapped to the second two-dimensional image.

[0141] For example, taking the first row of the list as an example, the indices of the three dimensions of the first row are: the index of R is 1, the index of G is 0, and the index of B is 0. The input pixel value corresponding to the first row is 100, and the corresponding output pixel value is 190. Using formulas (1) and (2), the indices of the three dimensions corresponding to the first row are mapped to the indices of the two dimensions in the two-dimensional image, that is, the index of x is 1, and the index of y is 0. According to the input pixel value of 100 and the corresponding output pixel value of 190 in the first row, the input pixel value corresponding to the point with coordinate position (1, 0) in the first two-dimensional image mapped by input 3DLut is 100; and the output pixel value corresponding to the point with coordinate position (1, 0) in the second two-dimensional image mapped by output 3DLut is 190.

[0142] S506, the restoration module obtains data from the first video from the HDR video frame. This first video data includes the first pixel value of the first video.

[0143] S507, The restoration module determines the index information corresponding to the first pixel value of the first video, and determines the three-dimensional pixel value corresponding to the index information in the two-dimensional image.

[0144] In the embodiments of this application, such as Figure 7AAs shown, the restoration module (3DLut restoration) in the GPU channel can determine the smallest cube in which the first pixel value of the first video is located according to the position (i.e. point C) of the first pixel value of the first video, and can determine the index positions of each vertex (i.e. C 000 , C 001 , C 010 , C 011 , C 100 , C 101 , C 110 , C 111 of the smallest cube according to the first pixel value, i.e. the index information corresponding to the first pixel value.

[0145] For example, the input 3DLut of the three-dimensional lookup table is shown in Table 1, the value corresponding to index 0 is 0, the value corresponding to index 1 is 50, the value corresponding to index 2 is 100, the value corresponding to index 3 is 190, and the value corresponding to index 4 is 240.

[0146] Suppose the first pixel value of the first video is (90, 120, 225), the index positions of each vertex of the smallest cube in which the first pixel value is located in the three-dimensional lookup table are found according to the first pixel value, i.e. the index positions of C 000 are (1, 2, 3), the index positions of C 001 are (1, 3, 3), the index positions of C 010 are (1, 2, 4), the index positions of C 011 are (1, 3, 4), the index positions of C 100 are (2, 2, 3), the index positions of C 101 are (2, 3, 3), the index positions of C 110 are (2, 2, 4), and the index positions of C 111 are (2, 3, 4).

[0147] Suppose the first pixel value of the first video is (20, 180, 150), the index positions of each vertex of the smallest cube in which the first pixel value is located in the three-dimensional lookup table are found according to the first pixel value, i.e. the index positions of C 000 are (0, 1, 2), the index positions of C 001 are (0, 2, 2), the index positions of C 010 are (0, 1, 3), the index positions of C 011 are (0, 2, 3), the index positions of C 100 are (1, 1, 2), the index positions of C 101 are (1, 2, 2), the index positions of C 110 are (1, 1, 3), and the index positions of C 111 are (1, 2, 3).

[0148] Table 1

[0149] Index Parameter Value 0 0 1 50 2 100 3 190 4 240

[0150] Further, the position coordinates (x, y) of each vertex in the two-dimensional image are determined according to the index information corresponding to the first pixel value (i.e. the index positions of each vertex of the smallest cube), which can be calculated by the following formula (4) and formula (5):

[0151] x = ((float(RGBIndex.g) + 0.0) * n + (float(RGBIndex.r) + 0.5)) * stepWidth (4)

[0152] y = ((float(RGBIndex.b) + 0.5) * stepHeight (5)

[0153] In formula (4) and formula (5), RGBIndex.g is the index of the G channel of the vertex; RGBIndex.r is the index of the R channel of the vertex; stepWidth is the step length of the width of the image, which is usually 1.0; RGBIndex.b is the index of the B channel of the vertex; stepHeight is the step length of the height of the image, which is usually 1.0; x is the x-axis coordinate corresponding to each vertex in the two-dimensional image; y is the y-axis coordinate corresponding to each vertex in the two-dimensional image; float(·) is a single-precision floating-point number; n is the order of the 3DLut.

[0154] Further, since each position in the two-dimensional image corresponds to a three-dimensional pixel value (input pixel value or output pixel value), the three-dimensional pixel value corresponding to each vertex can be determined according to the position coordinates corresponding to each vertex in the two-dimensional image, i.e. the three-dimensional pixel value corresponding to the index information can be determined. It can be understood that the input pixel value corresponding to the index information is determined in the first two-dimensional image, and the output pixel value corresponding to the index information is determined in the second two-dimensional image.

[0155] For example, assuming that the index position of vertex C 000 is (1, 2, 3) and n is 4. The position coordinates (x, y) in the two-dimensional image are calculated by formula (4) and formula (5) as (9.5, 3.5); the input pixel value corresponding to the position coordinates (9.5, 3.5) in the first two-dimensional image is (60, 100, 190), and the output pixel value corresponding to the position coordinates (9.5, 3.5) in the second two-dimensional image is (50, 120, 220). Therefore, the input pixel value corresponding to vertex C 000 is (60, 100, 190), and the output pixel value corresponding to vertex C 000 is (50, 120, 220).

[0156] For example, assume that the index position of vertex C 111 is (2, 3, 4) and n is 4. The corresponding position coordinates in the two-dimensional image are (14.5, 4.5) through the calculation of formula (4) and formula (5); the input pixel value corresponding to the position coordinates (14.5, 4.5) in the first two-dimensional image is (80, 120, 185), and the output pixel value corresponding to the position coordinates (14.5, 4.5) in the second two-dimensional image is (90, 160, 235). Thus, the input pixel value corresponding to vertex C 000 is (80, 120, 185), and the output pixel value corresponding to vertex C 000 is (90, 160, 235).

[0157] S508, the three-dimensional mapping module performs interpolation calculation on the first pixel value of the first video based on the three-dimensional pixel value corresponding to the index information and the first pixel value, to obtain a second pixel value; and determines a second picture of the first video based on the second pixel value.

[0158] In the embodiments of the present application, as shown in Figure 7B , the three-dimensional mapping module (Lut mapping) in the GPU channel calculates the distance between the first pixel value and each vertex of the smallest cube as the interpolation weight information; the second pixel value is calculated by using the input pixel value, the output pixel value and the interpolation weight information corresponding to the vertex during the interpolation operation; and the second picture of the first video is determined according to the obtained second pixel value and is transmitted to the second display module. Since the GPU channel uses the two-dimensional image corresponding to the 3DLut in the DPU channel to correct and map the color, the same effect as the processing of the DPU channel can be achieved when the GPU channel is processed, and the problem of brightness jump of the video picture is solved.

[0159] Specifically, the interpolation weight information can be determined by using the following formula (6):

[0160]

[0161] In formula (6), r is the value of the R channel of the first pixel value, g is the value of the G channel of the first pixel value, and b is the value of the B channel of the first pixel value; R0 is the value of the R channel of the input pixel value corresponding to vertex C 000 , G0 is the value of the G channel of the input pixel value corresponding to vertex C 000 , and B0 is the value of the B channel of the input pixel value corresponding to vertex C 000 ; R1 is the value of the R channel of the input pixel value corresponding to vertex C 111 , G1 is the value of the G channel of the input pixel value corresponding to vertex C 111 , and B1 is the value of the B channel of the input pixel value corresponding to vertex C 111a value of a B channel of the corresponding input pixel value; and Ar, Ag, Ab are the interpolation weight information.

[0162] Further, the output pixel value corresponding to each vertex and the interpolation weight information can be calculated by using the following formula (7) to obtain the second pixel value:

[0163] V(r,g,b) = c0 + clAr + c2Ag + c3Ab + c4ArAg + c5ArAb + c6AgAb + c7ArAgAb b + c2Ag r + c3Ab g + c4ArAg b + c5ArAb r + c6AgAb r + c7ArAgAb g (7) g b r g b (7)

[0164] In the formula (7), Ar, Ag, Ab are the interpolation weight information calculated by the above formula (6); V(r,g,b) is the second pixel value;

[0165] c0 = V(R0, G0, B0), that is, the output pixel value corresponding to vertex C 000 ;

[0166] c1 = V(R0, G0, B1) - V(R0, G0, B0), that is, the output pixel value corresponding to vertex C 001 - the output pixel value corresponding to vertex C 000 ;

[0167] c2 = V(R1, G0, B0) - V(R0, G0, B0), that is, the output pixel value corresponding to vertex C 100 - the output pixel value corresponding to vertex C 000 ;

[0168] c3 = V(R0, G1, B0) - V(R0, G0, B0), that is, the output pixel value corresponding to vertex C 010 - the output pixel value corresponding to vertex C 000 ;

[0169] c4 = V(R1, G0, B1) - V(R1, G0, B0) - V(R0, G0, B1) + V(R0, G0, B0), that is, the output pixel value corresponding to vertex C 101 - the output pixel value corresponding to vertex C 100 - the output pixel value corresponding to vertex C 001 + the output pixel value corresponding to vertex C 000 ;

[0170] ​​​​c5 = V(R1,G1,B0) - V(R0,G1,B0) - V(R1,G0,B0) + V(R0,G0,B0), i.e. vertex C 110 corresponding output pixel value - vertex C 010 corresponding output pixel value - vertex C 100 corresponding output pixel value + vertex C 000 corresponding output pixel value

[0171] c6 = V(R0,G1,B1) - V(R0,G1,B0) - V(R0,G0,B1) + V(R0,G0,B0), i.e. vertex C 011 corresponding output pixel value - vertex C 010 corresponding output pixel value - vertex C 001 corresponding output pixel value + vertex C 000 corresponding output pixel value

[0172] c7 = V(R1,G1,B1) - V(R1,G1,B0) - V(R0,G1,B1) - V(R1,G0,B1) + V(R0,G0,B1) + V(R0,G1,B0)

[0173] + V(R1,G0,B0) - V(R0,G0,B0),

[0174] i.e. vertex C 111 corresponding output pixel value - vertex C 110 corresponding output pixel value - vertex C 011 corresponding output pixel value - vertex C 101 corresponding output pixel value + vertex C 001 corresponding output pixel value + vertex C 010 corresponding output pixel value + vertex C 100 corresponding output pixel value - vertex C 000 corresponding output pixel value

[0175] Based on the manner, the interpolation manner of the three-dimensional look-up table is adopted to replace the traditional photoelectric conversion and electro-optical conversion function, which can avoid the difference of images, thereby avoiding the problem of brightness jump of video pictures.

[0176] S509, the second display module displays the second picture of the first video.

[0177] In this embodiment, the second display module (GPU RGB display) in the GPU path displays the second frame of the processed first video. That is, the frame of the first video viewed by the user at the second time point is the second frame of the first video processed by the GPU path. At this time, the brightness of the first video at the first time point is the same as the brightness of the first video at the second time point, avoiding abrupt changes in video brightness and ensuring visual quality.

[0178] I. Regarding the situation where the first window is displayed on the first video display interface of the electronic device.

[0179] Based on the above, another display method provided by the embodiments of this application will be described in further detail below. For example... Figure 8 As shown, the display method includes the following steps S801 and S802. Figure 8 The method shown can be implemented by the aforementioned electronic device. Alternatively, Figure 8 The method shown can be executed by a chip in an electronic device, but this application does not limit the implementation. Figure 8 The method will be explained using an electronic device as the executing entity.

[0180] S801, At the first point in time, the electronic device displays the first video frame.

[0181] In this embodiment, the user plays the first video using an electronic device at a first point in time, at which time the electronic device displays the screen of the first video. The first video can be a video obtained from a cloud server, a video captured by the camera when the user takes a picture of the scene using the electronic device for preview, or a video obtained from other electronic devices; no limitation is made here.

[0182] Optionally, the first video is an HDR video. Since HDR videos use high-brightness and high-saturation images, they are more prone to brightness jumps when obscured. Therefore, HDR videos are more effective at avoiding brightness jumps.

[0183] In one possible implementation, when the electronic device displays the first video frame at the first time point, the specific implementation could be: at the first time point, processing the data of the first video based on the first path to obtain the first frame of the first video, and then displaying the first frame of the first video. It should be noted that when a user plays the first video using the electronic device, under normal circumstances, the first video will be processed through the DPU path. Here, the first path can refer to the DPU path.

[0184] It should be noted that in the software structure of the electronic device, the multimedia framework of the application program framework layer includes an HDR video frame, a DPU channel, a CPU, and a GPU channel. The DPU channel includes a video pipeline module, a three-dimensional lookup table module (3DLut module), and a first display module (DPU RGB display); the GPU channel includes a restoration module (3DLut restoration), a three-dimensional mapping module (Lut mapping), and a second display module (GPU RGB display). At the first time point, the specific implementation process of the electronic device displaying a picture of the first video can refer to the above steps S501-S504, that is:

[0185] At the first time point, the video pipeline module obtains data of the first video from the HDR video frame; the three-dimensional lookup table module processes the data of the first video to obtain a first picture of the first video; and the first display module displays the first picture of the first video. The CPU stores the three-dimensional lookup table obtained from the three-dimensional lookup table module.

[0186] S802. At a second time point, the electronic device displays a first window in a display interface of the first video; the picture brightness of the first video at the first time point is the same as the picture brightness of the first video at the second time point, and the second time point is later than the first time point.

[0187] In the embodiment of the present application, the first window here can be a volume window, a sidebar window, a notification window, etc., which is not limited here. As shown in Figure 1C , at the second time point, the user operates the physical volume key on the electronic device, and at this time the electronic device displays the volume window (i.e. the first window) in the display interface of the first video. As shown in Figure 1D , at the second time point, the user touches the picture displaying the first video, and at this time the electronic device displays the sidebar window (i.e. the first window) in the display interface of the first video.

[0188] In a possible implementation manner, the method further includes: at the second time point, in response to displaying the first window in the display interface of the first video, the electronic device processes data of the first video based on a second channel to obtain a second picture of the first video; and the electronic device displays the second picture of the first video; wherein the second channel contains a two-dimensional image corresponding to the three-dimensional lookup table in the first channel.

[0189] In these cases, the electronic device switches from the DPU pass to the GPU pass to process the data of the first video. At this time, the CPU maps the previously stored three-dimensional lookup table to a two-dimensional image and delivers it to the GPU pass. The reason is that the GPU pass cannot support direct use of the three-dimensional lookup table for color correction and mapping, so the CPU needs to map the three-dimensional lookup table to a two-dimensional image. At this time, the two-dimensional image can maintain 10 bits, which is beneficial to ensure that the image is not distorted. In this way, the GPU pass can also be equivalent to using the same 3DLut in the DPU pass for color correction and mapping, so that the same effect as the DPU pass processing can be achieved when the GPU pass processes, thereby solving the problem of brightness jump of the video picture.

[0190] In a possible implementation, the method further includes: mapping, by the electronic device, the three-dimensional lookup table to a two-dimensional image.

[0191] Optionally, the two-dimensional image includes a first two-dimensional image and a second two-dimensional image, and when the electronic device maps the three-dimensional lookup table to a two-dimensional image, the specific implementation can include the following steps 1 and 2, as shown in Figure 9A .

[0192] Step 1: The electronic device determines the first position information of the input pixel value in the three-dimensional lookup table mapped to the first two-dimensional image, and the second position information of the output pixel value in the three-dimensional lookup table mapped to the second two-dimensional image.

[0193] Step 2: The electronic device determines the first two-dimensional image based on the input pixel value and the first position information, and determines the second two-dimensional image based on the output pixel value and the second position information.

[0194] Wherein, the specific implementation of steps 1 and 2 can refer to the specific implementation of step S505 described above, that is, at the second time point, in response to displaying the first window in the display interface of the first video, the CPU maps the three-dimensional lookup table to a two-dimensional image and delivers it to the GPU pass.

[0195] Optionally, the data of the first video includes a first pixel value of the first video, and when the electronic device processes the data of the first video based on the second pass to obtain a second picture of the first video, the specific implementation can include the following steps s11-s13, as shown in Figure 9B .

[0196] s11, in the second pass, the electronic device acquires the first pixel value of the first video.

[0197] s12, the electronic device processes the first pixel value based on the two-dimensional image to obtain a second pixel value.

[0198] Further optionally, when the electronic device processes the first pixel value based on the two-dimensional image to obtain a second pixel value, the specific implementation manner can include the following steps a-c, as shown in the following table. Figure 9C

[0199] Step a, the electronic device determines the index information corresponding to the first pixel value.

[0200] Step b, the electronic device determines the input pixel value corresponding to the index information in the first two-dimensional image, and determines the output pixel value corresponding to the index information in the second two-dimensional image.

[0201] The specific implementation manner of steps a and b can refer to the specific implementation manner of steps S506 and S507 described above, that is:

[0202] The restoration module obtains the data of the first video from the HDR video frame; the restoration module determines the index information corresponding to the first pixel value of the first video, and determines the three-dimensional pixel value corresponding to the index information in the two-dimensional image. The two-dimensional image includes a first two-dimensional image and a second two-dimensional image, and the three-dimensional pixel value includes an input pixel value or an output pixel value, that is, the input pixel value corresponding to the index information is determined in the first two-dimensional image, and the output pixel value corresponding to the index information is determined in the second two-dimensional image.

[0203] Step c, the electronic device performs interpolation calculation on the input pixel value corresponding to the index information, the output pixel value corresponding to the index information, and the first pixel value based on the index information to obtain a second pixel value.

[0204] Based on this manner, the interpolation manner of the three-dimensional lookup table is used to replace the traditional photoelectric conversion and electro-optical conversion function, which can avoid the difference of the image, thereby avoiding the problem of brightness jump of the video picture.

[0205] s13, the electronic device determines a second picture of the first video based on the second pixel value.

[0206] The specific implementation manner of steps c and s13 can refer to the specific implementation manner of step S508 described above, that is:

[0207] The three-dimensional mapping module performs interpolation calculation on the three-dimensional pixel value corresponding to the index information and the first pixel value of the first video to obtain a second pixel value; and determines a second picture of the first video based on the second pixel value.

[0208] In addition, the specific implementation manner of the electronic device displaying the second picture of the first video can refer to the specific implementation manner of step S509 described above, that is, the second display module displays the second picture of the first video, which will not be described here.

[0209] ​In one possible implementation, the method further includes: at a third time point, the electronic device closes the first window in the display interface of the first video; the screen brightness of the first video at the second time point is the same as the screen brightness of the first video at the third time point, and the third time point is later than the second time point.

[0210] Optionally, the method further includes: at a third time point, in response to the first window disappearing from the display interface of the first video, the electronic device processes the data of the first video based on the first channel to obtain a third frame of the first video and displays the third frame of the first video.

[0211] This can be understood as follows: at the third time point, when the first window of the first video's display interface closes, the electronic device switches back from the GPU path to the DPU path to process the data of the first video, obtaining the third frame of the first video. Similarly, since the GPU path uses the three-dimensional lookup table in the DPU path to correspond to the two-dimensional image, it is equivalent to using the same three-dimensional lookup table for color correction and mapping. Therefore, there will be no problem of brightness jumps in the video frame. That is, the brightness of the first video frame at the second time point is the same as the brightness of the first video frame at the third time point, thus ensuring the visual effect.

[0212] In one possible implementation, the method further includes: at a fourth time point, the electronic device switches the first video to window mode; the brightness of the first video at the third time point is the same as the brightness of the first video at the fourth time point, and the fourth time point is later than the third time point.

[0213] Optionally, the method further includes: at a fourth time point, in response to the operation of switching the first video to window mode, the electronic device processes the data of the first video based on the second channel to obtain a fourth frame of the first video and displays the fourth frame of the first video.

[0214] This can be understood as, for example Figure 1E As shown, at the fourth time point, the user switches the first video to windowed mode by dragging the display interface. In this case, the electronic device switches from the DPU path to the GPU path to process the data of the first video, resulting in the fourth frame of the first video. Similarly, since the GPU path uses the three-dimensional lookup table in the DPU path to correspond to the two-dimensional image, it is equivalent to using the same three-dimensional lookup table for color correction and mapping. Therefore, there will be no problem of brightness jumps in the video frame. That is, the brightness of the first video frame at the third time point is the same as the brightness of the first video frame at the fourth time point, thus ensuring the visual effect.

[0215] As can be seen, based on the method described in this application, the brightness of the first video at the first time point is the same as the brightness of the first video at the second time point, which can avoid the problem of abrupt changes in the brightness of the video and ensure the visual effect.

[0216] 2. Regarding the situation where electronic devices switch the first video to window mode.

[0217] Based on the above, another display method provided by the embodiments of this application will be described in further detail below. For example... Figure 10 As shown, the display method includes the following steps S1001 and S1002. Figure 10 The method shown can be implemented by the aforementioned electronic device. Alternatively, Figure 10 The method shown can be executed by a chip in an electronic device, but this application does not limit the implementation. Figure 10 The method will be explained using an electronic device as the executing entity.

[0218] S1001. At the fifth time point, the electronic device displays the image of the first video.

[0219] In one possible implementation, when the electronic device displays the first video frame at the fifth time point, the specific implementation method could be: at the fifth time point, the data of the first video is processed based on the first path to obtain the first frame of the first video, and then the first frame of the first video is displayed. It should be noted that when a user plays the first video using the electronic device, under normal circumstances, the first video will be processed through the DPU path. Here, the first path can refer to the DPU path. Optionally, the first video is an HDR video. The specific implementation method of step S1001 can refer to the specific implementation method of step S801 above, and will not be repeated here.

[0220] It should be noted that in the software architecture of the electronic device, the multimedia framework of the application framework layer includes HDR video frames, the DPU path, and the CPU and GPU paths. The DPU path includes a video pipeline module, a 3D lookup table module (3DLut module), and a first display module (DPU RGB display); the GPU path includes a restoration module (3DLut restoration), a 3D mapping module (LUT mapping), and a second display module (GPU RGB display). The specific implementation process of the electronic device displaying the first video frame at the fifth time point can refer to steps S501 to S504 above, namely:

[0221] At the fifth time point, the video channel module obtains data of the first video from the HDR video frame; the three-dimensional lookup table module processes the data of the first video to obtain a first picture of the first video; and the first display module displays the first picture of the first video. The CPU stores the three-dimensional lookup table obtained from the three-dimensional lookup table module.

[0222] At the sixth time point, the electronic device switches the first video to the window mode; the brightness of the picture of the first video at the fifth time point is the same as the brightness of the picture of the first video at the sixth time point, and the sixth time point is later than the fifth time point.

[0223] In a possible implementation, the method further includes: at the sixth time point, in response to the operation of switching the first video to the window mode, the electronic device processes data of the first video based on the second channel to obtain a second picture of the first video, and displays the second picture of the first video. As shown in FIG. 6, at the sixth time point, the user switches the first video to the window mode by dragging the display interface of the first video. In this case, the electronic device also switches from the DPU channel to the GPU channel to process data of the first video. The second channel here can be the GPU channel. Figure 1E

[0224] In a possible implementation, the method further includes: the electronic device maps the three-dimensional lookup table into a two-dimensional image.

[0225] Optionally, the two-dimensional image includes a first two-dimensional image and a second two-dimensional image, and when the electronic device maps the three-dimensional lookup table into a two-dimensional image, the specific implementation can refer to steps 1 and 2 described above, that is, the electronic device determines first position information of an input pixel value in the three-dimensional lookup table mapping to the first two-dimensional image, and second position information of an output pixel value in the three-dimensional lookup table mapping to the second two-dimensional image; the electronic device determines the first two-dimensional image based on the input pixel value and the first position information, and determines the second two-dimensional image based on the output pixel value and the second position information.

[0226] Optionally, the data of the first video includes a first pixel value of the first video, and when the electronic device processes the data of the first video based on the second channel to obtain a second picture of the first video, the specific implementation can refer to steps s11-s13 described above, that is: in the second channel, the electronic device obtains the first pixel value of the first video; the electronic device processes the first pixel value based on the two-dimensional image to obtain a second pixel value; and the electronic device determines the second picture of the first video based on the second pixel value.

[0227] ​Further optionally, when the electronic device processes the first pixel value based on the two-dimensional image to obtain a second pixel value, the specific implementation manner can refer to steps a-c described above, that is, the electronic device determines the index information corresponding to the first pixel value; the electronic device determines the input pixel value corresponding to the index information in the first two-dimensional image, and determines the output pixel value corresponding to the index information in the second two-dimensional image; and the electronic device performs interpolation calculation on the input pixel value corresponding to the index information, the output pixel value corresponding to the index information, and the first pixel value based on the index information to obtain the second pixel value.

[0228] Specifically, it can be referred to the description of steps S505-S509 described above: at the sixth time point, in response to the operation of switching the first video to the window mode, the CPU maps the three-dimensional lookup table into a two-dimensional image and delivers it to the GPU channel; the restoration module obtains the data of the first video from the HDR video frame, wherein the data of the first video includes the first pixel value of the first video; the restoration module determines the index information corresponding to the first pixel value of the first video, and determines the three-dimensional pixel value corresponding to the index information in the two-dimensional image; the three-dimensional mapping module performs interpolation calculation on the three-dimensional pixel value corresponding to the index information and the first pixel value of the first video based on the index information to obtain a second pixel value, and determines a second picture of the first video based on the second pixel value; and the second display module displays the second picture of the first video.

[0229] It can be understood that at the sixth time point, the user can switch the first video to the window mode by dragging the display interface of the first video. At this time, the electronic device will also switch from the DPU channel to the GPU channel to process the data of the first video. Similarly, since the GPU channel uses the two-dimensional image corresponding to the three-dimensional lookup table in the DPU channel, it is equivalent to using the same three-dimensional lookup table for color correction and mapping, so there will be no problem of brightness jump of the video picture, that is, the picture brightness of the first video at the fifth time point is the same as the picture brightness of the first video at the sixth time point, thereby ensuring the visual effect. Specifically, it can be referred to the description in step S802 described above, which will not be repeated here.

[0230] It can be seen that based on the method described in the present application, the picture brightness of the first video at the fifth time point is the same as the picture brightness of the first video at the sixth time point, which can avoid the problem of brightness jump of the video picture and ensure the visual effect.

[0231] Please refer to Figure 11 , Figure 11 FIG. 1 shows a structural schematic diagram of a display device 1100 according to an embodiment of the present application. Figure 11 The display device shown can be an electronic device, a device in an electronic device, or a device that can be matched with an electronic device. Figure 11The display device shown can include a processing unit 1101 and a display unit 1102. Among them:

[0232] The display unit 1102 is configured to display a picture of a first video at a first time point.

[0233] The display unit 1102 is further configured to display a first window in a display interface of the first video at a second time point, the picture brightness of the first video at the first time point being the same as the picture brightness of the first video at the second time point, and the second time point being later than the first time point.

[0234] In a possible implementation, when the picture of the first video is displayed at the first time point, the processing unit 1101 is specifically configured to process data of the first video based on a first path to obtain a first picture of the first video at the first time point, and the display unit 1102 is configured to display the first picture of the first video.

[0235] The processing unit 1101 is further configured to, at the second time point, in response to displaying the first window in the display interface of the first video, process data of the first video based on a second path to obtain a second picture of the first video, and the display unit 1102 is further configured to display the second picture of the first video. The second path includes a two-dimensional image corresponding to a three-dimensional lookup table in the first path.

[0236] In a possible implementation, the processing unit 1101 is further configured to map the three-dimensional lookup table into a two-dimensional image, and the data of the first video includes a first pixel value of the first video. When the processing unit 1101 processes the data of the first video based on the second path to obtain the second picture of the first video, the processing unit 1101 is specifically configured to: in the second path, obtain the first pixel value of the first video; process the first pixel value based on the two-dimensional image to obtain a second pixel value; and determine the second picture of the first video based on the second pixel value.

[0237] In a possible implementation, the two-dimensional image includes a first two-dimensional image and a second two-dimensional image, and the processing unit 1101, when mapping the three-dimensional lookup table into a two-dimensional image, is specifically configured to: determine first position information in the three-dimensional lookup table that maps an input pixel value to the first two-dimensional image, and second position information in the three-dimensional lookup table that maps an output pixel value to the second two-dimensional image; determine the first two-dimensional image based on the input pixel value and the first position information, and determine the second two-dimensional image based on the output pixel value and the second position information.

[0238] In one possible implementation, when processing the first pixel value based on the two-dimensional image to obtain the second pixel value, the processing unit 1101 is specifically configured to: determine the index information corresponding to the first pixel value; determine the input pixel value corresponding to the index information in the first two-dimensional image, and determine the output pixel value corresponding to the index information in the second two-dimensional image; and perform interpolation calculation based on the input pixel value corresponding to the index information, the output pixel value corresponding to the index information, and the first pixel value to obtain the second pixel value.

[0239] In one possible implementation, the processing unit 1101 is further configured to: close the first window in the display interface of the first video at a third time point; the screen brightness of the first video at the second time point is the same as the screen brightness of the first video at the third time point, and the third time point is later than the second time point.

[0240] In one possible implementation, the processing unit 1101 is further configured to: switch the first video to window mode at a fourth time point; the brightness of the first video at the third time point is the same as the brightness of the first video at the fourth time point, and the fourth time point is later than the third time point.

[0241] In one possible implementation, the first video is an HDR video.

[0242] For cases where the display device can be a chip or a chip system, please refer to [link / reference]. Figure 12 The diagram shows the structure of the chip. Figure 12 The chip 1200 shown includes a processor 1201 and an interface 1202. Optionally, it may also include a memory 1203. The number of processors 1201 can be one or more, and the number of interfaces 1202 can be multiple.

[0243] For cases where the chip is used to implement the electronic device in the embodiments of this application:

[0244] The interface 1202 is used to receive or output signals;

[0245] The processor 1201 is used to perform data processing operations of the electronic device.

[0246] It is understood that some optional features in the embodiments of this application can be implemented independently in certain scenarios without relying on other features, such as the current solution on which they are based, to solve the corresponding technical problems and achieve the corresponding effects. Alternatively, they can be combined with other features as needed in certain scenarios. Accordingly, the data processing apparatus given in the embodiments of this application can also implement these features or functions, which will not be elaborated here.

[0247] It should be understood that the processor in the embodiments of the present application can be an integrated circuit chip with a processing capability of signals. In the implementation process, each step of the method embodiments described above can be completed by integrated logic circuits or instructions in the form of software in the processor. The processor described above can be a general processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0248] It can be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically EPROM (EEPROM) or a flash memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM) and direct memory bus random access memory (DRAM) (DRAM). It should be noted that the memory of the system and method described herein is intended to include but not limited to these and any other suitable types of memory.

[0249] The present application also provides a display system, comprising an electronic device; wherein the electronic device is configured to perform the method performed by the electronic device in any of the method embodiments described above.

[0250] The application further provides a computer readable storage medium, which stores a computer program, and the computer program includes program instructions, and when the program instructions are executed on an electronic device, functions of any of the method embodiments are implemented.

[0251] The application further provides a computer program product, which, when executed on a computer, causes the computer to implement functions of any of the method embodiments.

[0252] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (such as floppy disk, hard disk, magnetic tape), optical media (such as high-density digital video disc (digital video disc, DVD)), or semiconductor media (such as solid state disk (solid state disk, SSD)), etc.

[0253] The above is only a specific implementation of the application, but the protection scope of the application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical range disclosed in the application, which should be covered within the protection scope of the application. Therefore, the protection scope of the application should be subject to the protection scope of the claims.

Claims

1. A display method, characterized in that, The method includes: At the first point in time, the first frame of the first video is displayed; the first frame of the first video is obtained by processing the data of the first video based on a three-dimensional lookup table in the first path; At the second time point, in response to displaying a first window in the display interface of the first video, a second frame of the first video is displayed; the second frame of the first video is obtained by processing the data of the first video based on the two-dimensional image in the second path, the brightness of the first video at the first time point is the same as the brightness of the first video at the second time point, and the second time point is later than the first time point; The two-dimensional image in the second path is obtained based on the three-dimensional lookup table mapping in the first path.

2. The method according to claim 1, characterized in that, The data of the first video includes the first pixel value of the first video, and the method further includes: In the second path, the first pixel value of the first video is obtained; The second pixel value is obtained by processing the first pixel value based on the two-dimensional image; The second frame of the first video is determined based on the second pixel value.

3. The method according to claim 2, characterized in that, The two-dimensional image includes a first two-dimensional image and a second two-dimensional image, and the method further includes: Determine the first position information of the input pixel value in the three-dimensional lookup table mapped to the first two-dimensional image, and the second position information of the output pixel value in the three-dimensional lookup table mapped to the second two-dimensional image; The first two-dimensional image is determined based on the input pixel value and the first position information, and the second two-dimensional image is determined based on the output pixel value and the second position information.

4. The method according to claim 3, characterized in that, The step of processing the first pixel value based on the two-dimensional image to obtain the second pixel value includes: Determine the index information corresponding to the first pixel value; In the first two-dimensional image, the input pixel value corresponding to the index information is determined, and in the second two-dimensional image, the output pixel value corresponding to the index information is determined. The second pixel value is obtained by interpolation calculation based on the input pixel value corresponding to the index information, the output pixel value corresponding to the index information, and the first pixel value.

5. The method according to any one of claims 1-4, characterized in that, The method further includes: At the third time point, in response to closing the first window in the display interface of the first video, the third screen of the first video is displayed; the third screen of the first video is obtained by processing the data of the first video based on the three-dimensional lookup table in the first path, the screen brightness of the first video at the second time point is the same as the screen brightness of the first video at the third time point, and the third time point is later than the second time point.

6. The method according to claim 5, characterized in that, The method further includes: At the fourth time point, in response to switching the first video to windowed mode, the fourth frame of the first video is displayed; the fourth frame of the first video is obtained by processing the data of the first video based on the two-dimensional image in the second path, the brightness of the first video at the third time point is the same as the brightness of the first video at the fourth time point, and the fourth time point is later than the third time point.

7. The method according to any one of claims 1-4, characterized in that, The first video is a high dynamic range (HDR) video.

8. An electronic device, characterized in that, include: One or more processors, one or more memories; wherein the one or more memories are coupled to the one or more processors, the one or more memories being used to store computer program code, the computer program code including computer instructions, which, when the one or more processors execute the computer instructions, cause the electronic device to perform the method as described in any one of claims 1-7.

9. A display system, characterized in that, Includes an electronic device; wherein the electronic device is used to perform the method as described in any one of claims 1-7.

10. A chip, characterized in that, The device includes a processor and an interface, the processor and the interface being coupled; the interface is used to receive or output signals, and the processor is used to execute code instructions to cause the method of any one of claims 1-7 to be performed.

11. A computer storage medium, characterized in that, The computer storage medium stores a computer program, which includes program instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1-7.

12. A computer program product, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 1-7.

Images