Display method and electronic equipment

Abstract

The invention discloses a display method and electronic equipment, and the method comprises the steps: enabling a video channel module in a DPU channel to obtain the data of a first video from an HDR video frame at a first time point; then the three-dimensional lookup table module processes the data of the first video to obtain a first picture of the first video, and the first picture is displayed through the first display module; meanwhile, the CPU can store the three-dimensional lookup table in the DPU channel. And at a second time point, in response to a first window displayed in the display interface of the first video, the CPU converts the three-dimensional lookup table into a two-dimensional image and transmits the two-dimensional image to the GPU path. And processing the data of the first video through a restoration module and a three-dimensional mapping module in the GPU channel to obtain a second picture of the first video, and displaying the second picture through a second display module. Through the method and the device, the problem that the brightness of the video picture jumps can be avoided, and the visual effect is ensured.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computers, and in particular to a display method and an electronic device. Background Art

[0002] In computer graphics and cinematography, high-dynamic range (HDR) is an image technology that can display a wider range of brightness and contrast. HDR can brighten details in dark areas without distorting details in bright areas, presenting a more natural and realistic picture, thereby enhancing the three-dimensional sense and comfort of the picture. The purpose of HDR is to correctly represent the brightness of a large range in the real world, from direct sunlight to the darkest shadows, and better reflect the visual effects in the real environment.

[0003] However, when users use electronic devices to play HDR videos, the following situations may occur:

[0004] (1) The user presses the physical volume button on the electronic device, and a volume window pops up on the display interface;

[0005] (2) When the user touches the screen displaying the HDR video, a sidebar window will pop up on the display interface.

[0006] When these situations occur, the video picture will be affected to a certain extent, thereby reducing the visual effect. Summary of the invention

[0007] The embodiments of the present application provide a display method and an electronic device. Based on the method described in the present application, the problem of brightness jumps in the video screen can be avoided, thereby ensuring the visual effect.

[0008] In a first aspect, the present application provides a display method, the method comprising: displaying a screen of a first video at a first time point; displaying a first window in a display interface of the first video at a second time point; the screen brightness of the first video at the first time point is the same as the screen brightness of the first video at the second time point, and the second time point is later than the first time point.

[0009] Based on the method described in the first aspect, the screen brightness of the first video at the first time point is the same as the screen brightness of the first video at the second time point, which can avoid the problem of sudden changes in the brightness of the video screen and ensure the visual effect.

[0010] In a possible implementation, at a first time point, displaying a screen of a first video includes: at the first time point, processing the data of the first video based on the first path to obtain the first screen of the first video, and displaying the first screen of the first video; the method also includes: at a second time point, in response to displaying a first window in the display interface of the first video, processing the data of the first video based on the second path to obtain the second screen of the first video; displaying the second screen of the first video; wherein the second path includes a two-dimensional image corresponding to the three-dimensional lookup table in the first path. Based on this method, the problem of jumps in the brightness of the video screen can be avoided.

[0011] In a possible implementation, the method further includes: mapping the three-dimensional lookup table to a two-dimensional image; the data of the first video includes a first pixel value of the first video, and processing the data of the first video based on the second path to obtain a second picture of the first video, including: obtaining the first pixel value of the first video in the second path; processing the first pixel value based on the two-dimensional image to obtain a second pixel value; and determining the second picture of the first video based on the second pixel value. Based on this method, it is helpful to ensure that the image is not distorted.

[0012] In a possible implementation, the two-dimensional image includes a first two-dimensional image and a second two-dimensional image, and mapping the three-dimensional lookup table to a two-dimensional image includes: determining that an input pixel value in the three-dimensional lookup table is mapped to first position information of the first two-dimensional image, and that an output pixel value in the three-dimensional lookup table is mapped to second position information of the second two-dimensional image; determining the first two-dimensional image based on the input pixel value and the first position information, and determining the second two-dimensional image based on the output pixel value and the second position information. Based on this method, it is helpful to ensure the accuracy of the two-dimensional image corresponding to the input pixel value and the two-dimensional image corresponding to the output pixel value.

[0013] In a possible implementation, the first pixel value is processed based on the two-dimensional image to obtain the second pixel value, including: determining the index information corresponding to the first pixel value; determining the input pixel value corresponding to the index information in the first two-dimensional image, and determining the output pixel value corresponding to the index information in the second two-dimensional image; performing interpolation calculation based on the input pixel value corresponding to the index information, the output pixel value corresponding to the index information and the first pixel value to obtain the second pixel value. Based on this method, the interpolation method of the three-dimensional lookup table is used to replace the traditional photoelectric conversion and electro-optical conversion functions, which can avoid image differences and thus avoid the problem of jumps in the brightness of the video screen.

[0014] In a possible implementation, the method further includes: at a third time point, closing the first window in the display interface of the first video; the screen brightness of the first video at the second time point is the same as the screen brightness of the first video at the third time point, and the third time point is later than the second time point. Based on this method, the problem of sudden changes in the brightness of the video screen can be avoided.

[0015] In a possible implementation, the method further includes: at a fourth time point, switching the first video to a window mode; the screen brightness of the first video at the third time point is the same as the screen brightness of the first video at the fourth time point, and the fourth time point is later than the third time point. Based on this method, the problem of sudden changes in the brightness of the video screen can be avoided.

[0016] In a possible implementation, the first video is a high dynamic range imaging HDR video. Since HDR video uses high brightness and high saturation images, when the HDR video is blocked, it is more likely to cause brightness jump problems. Therefore, for HDR video, the brightness jump problem of the video screen can be more obviously avoided.

[0017] In a second aspect, the present application provides a display device, which may be an electronic device, or a device in an electronic device, or a device that can be used in combination with an electronic device; wherein the display device may also be a chip system, and the display device may execute the method executed by the electronic device in the first aspect. The functions of the display device may be implemented by hardware, or by hardware executing corresponding software implementations. The hardware or software includes one or more units corresponding to the above functions. The unit may be software and / or hardware. The operations and beneficial effects performed by the display device may refer to the methods and beneficial effects described in the first aspect above, and the repetitive parts will not be repeated.

[0018] In a third aspect, the present application provides a display device, comprising a processor, and when the processor calls a computer program in a memory, the method described in the first aspect is executed.

[0019] In a fourth aspect, the present application provides a display device, comprising a processor and a memory, wherein the processor and the memory are coupled; the processor is used to implement the method described in the first aspect.

[0020] In a fifth aspect, the present application provides a display device, which includes a processor, a memory and a transceiver, wherein the processor and the memory are coupled; the transceiver is used to send and receive data, and the processor is used to implement the method described in the first aspect.

[0021] In a sixth aspect, the present application provides an electronic device, comprising one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program codes, and the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the display method in any possible implementation of the first aspect.

[0022] In a seventh aspect, the present application provides a chip comprising a processor and an interface, wherein the processor and the interface are coupled; the interface is used to receive or output signals, and the processor is used to execute code instructions so that the method described in the first aspect is executed.

[0023] In an eighth aspect, the present application provides a display system, which includes an electronic device; wherein the electronic device is used to execute the method described in the first aspect.

[0024] In a ninth aspect, the present application provides a display device, comprising a function or unit for executing any one of the methods in the first aspect.

[0025] In a tenth aspect, the present application provides a computer-readable storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed on a display device, the display device executes the display method in any possible implementation of the first aspect.

[0026] In an eleventh aspect, the present application provides a computer program product. When the computer program product runs on a computer, it enables the computer to execute the display method in any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1A is a schematic diagram of a three-dimensional lookup table provided in an embodiment of the present application;

[0028] Figure 1B is a schematic diagram of a DPU path and a GPU path provided in an embodiment of the present application;

[0029] Figure 1C is a schematic diagram of a volume display window provided in an embodiment of the present application;

[0030] Figure 1D is a schematic diagram of displaying a sidebar window provided in an embodiment of the present application;

[0031] Figure 1E is a schematic diagram of switching a video screen to a window mode provided by an embodiment of the present application;

[0032] Figure 2It is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0033] Figure 3 It is a software structure block diagram of an electronic device provided in an embodiment of the present application;

[0034] Figure 4 is a schematic diagram of another DPU path and GPU path provided in an embodiment of the present application;

[0035] Figure 5 It is a flow chart of a display method provided in an embodiment of the present application;

[0036] Fig. 6A It is a schematic diagram of the positional relationship of mapping a three-dimensional lookup table to a two-dimensional image provided by an embodiment of the present application;

[0037] Figure 6B is a schematic diagram of a two-dimensional image provided in an embodiment of the present application;

[0038] Fig. 7A is a schematic diagram of a minimum cube in which a first pixel value is located provided in an embodiment of the present application;

[0039] Figure 7B is a schematic diagram of performing an interpolation operation based on a minimum cube in which a first pixel value is located, provided in an embodiment of the present application;

[0040] Figure 8 is a flow chart of another display method provided in an embodiment of the present application;

[0041] Fig. 9A is a schematic diagram of a process of determining a second picture of a first video provided by an embodiment of the present application;

[0042] Fig. 9B It is a schematic diagram of a process of mapping a three-dimensional lookup table into a two-dimensional image provided by an embodiment of the present application;

[0043] Fig. 9C is a schematic diagram of a process for determining a second pixel value provided by an embodiment of the present application;

[0044] Fig.10 is a flow chart of another display method provided in an embodiment of the present application;

[0045] Fig.11 is a structural schematic diagram of a display device provided in an embodiment of the present application;

[0046] Fig.12 It is a schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The technical solutions in the embodiments of the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0048] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0049] The term "user interface (UI)" in the following embodiments of the present application refers to a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on the terminal device, and finally presented as content that the user can recognize. The commonly used form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation that is displayed in a graphical manner. It can be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of a terminal device.

[0050] In order to facilitate understanding of the solution provided by the embodiment of the present application, the following introduces the relevant concepts involved in the embodiment of the present application:

[0051] 1. High-dynamic range (HDR)

[0052] In computer graphics and cinematography, HDR is an image technology that can display a wider range of brightness and contrast, and can reproduce a wider dynamic range of brightness than standard digital imaging or photography technology. HDR can brighten details in dark areas without distorting details in bright areas, presenting a more natural and realistic picture, thereby enhancing the three-dimensional sense and comfort of the picture. The purpose of HDR is to correctly represent the wide range of brightness in the real world, from direct sunlight to the darkest shadows, and better reflect the visual effects in the real environment.

[0053] 2. 3D look-up-table (3DLut)

[0054] Lut is the abbreviation of look-up table. It is a container of data, similar to a dictionary, map, etc., and a description method of matching indexes with values. 3DLut means that the description of the correspondence between indexes and values ​​is three-dimensional, which can be understood as a combination of the correspondence between indexes and values ​​in three dimensions, that is, a value can be found through the indexes in three dimensions. 3DLut is an algorithm that readjusts the hue of an image by establishing a color mapping table. Its principle is to achieve the desired effect by color correction and mapping the image. 3DLut can be understood as a color conversion matrix, in which each input pixel value (i.e., inputLut) has a corresponding output pixel value (outputLut). Each element in this conversion matrix represents the corresponding color conversion relationship. By performing color conversion on each pixel, the color of the image can be made more accurate and realistic.

[0055] like Figure 1A As shown in the figure, the working principle of 3DLut is to divide the RGB color space into multiple small cubes (such as n*n*n 3DLut, where n represents the order of 3DLut), and each small cube corresponds to an output pixel value. Specifically, when the input pixel value enters 3DLut, it will be mapped to the corresponding output pixel value according to its position in the color space, thereby achieving color correction and mapping.

[0056] 3. Data processing unit (DPU), graphics processing unit (GPU)

[0057] DPU is a processor used in data centers to process data transmitted in data centers. It can be used to build the main chip of smart network cards. In addition to completing the transmission tasks of traditional networks, it can also complete some or all of the infrastructure management tasks originally completed by the central processing unit (CPU), so it is called a "smart" network card. DPU is usually used to work side by side with CPU and GPU, becoming one of the three major processors in data centers to improve the performance and efficiency of data centers.

[0058] GPU is another processor of the computer, which is mainly used to meet the requirements of image computing. Relatively speaking, CPU is good at logical judgment and serial data operation, and each pixel of an image requires the same calculation and processing. GPU is good at parallel tasks such as graphics computing. Because of the high parallelism of GPU, the category has also derived general-purpose GPUs that weaken image capabilities and focus on computing. GPU can process large amounts of data at the same time, making it faster than CPU in some application scenarios. GPU was originally used for graphics processing, but is now also widely used in deep learning, scientific computing, cryptography, data mining and other fields. At present, artificial intelligence and general large models can also use GPU for parallel training. In general, GPU is mainly used for graphics computing, and DPU is mainly used for processing data transmission in data centers.

[0059] like Figure 1B As shown, when the user plays an HDR video, the HDR video frame is normally processed through the DPU pipeline. During the DPU processing, the video pipeline module (videopipeline) first obtains the HDR video frame from the HDR video frame; then the three-dimensional lookup table module (3DLut module) calculates its corresponding three-dimensional lookup table (3DLut), uses 3DLut to perform color correction and mapping on the HDR video frame, and then displays the converted video screen through the first display module (DPU RGB display). However, when the user uses an electronic device to play an HDR video, the following situations may occur:

[0060] (1) Figure 1C As shown, the user operates the physical volume button on the electronic device, and a volume window pops up on the display interface;

[0061] (2) Figure 1D As shown, when the user touches the screen displaying the HDR video, a sidebar window will pop up on the display interface.

[0062] (3) Figure 1E As shown, the user switches the video screen to window mode by dragging the display interface of the HDR video.

[0063] When the above situations occur, the HDR video frame will be processed through the GPU channel. During the GPU processing, the HDR video frame will be subjected to nonlinear mapping of the electro-optical transfer function (EOTF), color conversion, tone mapping, and inverse mapping of the OETF, and then displayed through the second display module (GPU RGB display) to achieve the display effect of DPU processing as much as possible. However, through testing and comparison, it is found that there is a large difference in display brightness between the two, which leads to the problem of jump in the brightness of the video screen during the switching process, which has a certain impact on the video screen and reduces the visual effect.

[0064] In order to avoid the problem of brightness jumps in the video screen and ensure the visual effect, the present application provides a display method and an electronic device. In a specific implementation, the above-mentioned display method can be performed by an electronic device 100. Among them, the electronic device 100 can be a mobile phone, a tablet computer, a laptop computer, or a wearable electronic device with wireless communication function (such as a smart watch), etc., but is not limited to this. The electronic device 100 is configured with a display screen and can be installed with a preset application (application, APP), such as a gallery APP, a video APP, a camera APP, etc. Users can play HDR videos through the gallery APP or the video APP, or take pictures through the camera APP to preview the HDR picture. Of course, the user can also switch the video screen to window mode by dragging the display interface.

[0065] The hardware structure of the electronic device 100 is introduced below. Figure 2 , Figure 2 It is a schematic diagram of the hardware structure of the electronic device 100 provided in an embodiment of the present application.

[0066] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0067] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0068] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0069] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0070] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or is cyclically used.

[0071] If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system. The processor 110 calls the instruction or data stored in the memory, so that the electronic device 100 executes the display method executed by the electronic device in the following method embodiment.

[0072] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0073] The charging management module 140 is used to receive charging input from a charger, where the charger can be a wireless charger or a wired charger.

[0074] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. In some other embodiments, the power management module 141 can also be set in the processor 110.

[0075] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0076] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0077] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, etc. the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation.

[0078] The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 can be set in the same device as at least some modules of the processor 110.

[0079] The modulation and demodulation processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is transmitted to the application processor.

[0080] The wireless communication module 160 can provide wireless communication solutions for application in the electronic device 100, including wireless local area networks (WLAN) (such as Wi-Fi networks), Bluetooth (BT), BLE broadcasting, global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc.

[0081] The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 may also receive a signal to be sent from the processor 110, modulate the signal, amplify the signal, and convert it into an electromagnetic wave for radiation via the antenna 2.

[0082] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0083] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0084] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0085] The electronic device 100 can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194 and the application processor. The ISP is used to process the data fed back by the camera 193. The camera 193 is used to capture static images or videos. The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. The video codec is used to compress or decompress digital videos. The electronic device 100 can support one or more video codecs.

[0086] NPU is a neural-network (NN) computing processor. It can quickly process input information by drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, and can also continuously self-learn.

[0087] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement a data storage function.

[0088] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function), etc. The data storage area may store data (such as audio data) created during the use of the electronic device 100, etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as a flash memory device, etc.

[0089] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0090] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0091] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The receiver 170B, also called a "earpiece", is used to convert an audio electrical signal into a sound signal. The microphone 170C, also called a "microphone" or "microphone", is used to convert a sound signal into an electrical signal. The headphone jack 170D is used to connect a wired headphone. The pressure sensor 180A is used to sense a pressure signal and can convert the pressure signal into an electrical signal.

[0092] In some embodiments, the pressure sensor 180A may be provided on the display screen 194. The gyroscope sensor 180B may be used to determine the motion posture of the electronic device 100. The air pressure sensor 180C is used to measure air pressure. The magnetic sensor 180D includes a Hall sensor. The acceleration sensor 180E may detect the magnitude of the acceleration of the electronic device 100 in various directions (generally three axes). The distance sensor 180F is used to measure the distance. The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector. The ambient light sensor 180L is used to sense the brightness of ambient light. The fingerprint sensor 180H is used to collect fingerprints. The temperature sensor 180J is used to detect the temperature. The touch sensor 180K, also known as a "touch panel". The touch sensor 180K may be provided on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also known as a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The bone conduction sensor 180M may obtain a vibration signal. The buttons 190 include a power button, a volume button, etc. The motor 191 can generate a vibration prompt. The indicator 192 can be an indicator light, which can be used to indicate the charging status, power change, messages, missed calls, notifications, etc. The SIM card interface 195 is used to connect a SIM card.

[0093] In addition, an operating system is running on the above components. For example, operating systems such as iOS and Android. The operating system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to illustrate the software structure of the electronic device 100. It should be noted that although the embodiment of the present application is described by taking the Android system as an example, its basic principles are also applicable to electronic devices of other operating systems.

[0094] Figure 3 It is a software structure block diagram of the electronic device 100 of the embodiment of the present application. The software structure adopts a layered architecture, which divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In the embodiment of the present application, the operating system (taking the Android system, the Android system running on the AP as an example) can be divided into six layers, from top to bottom, respectively, the application layer (application, APP), the application framework layer (framework, FWK), the Android runtime (Android runtime) and the system library, the hardware abstraction layer (hardware abstraction layer, HAL), the kernel layer and the hardware layer.

[0095] The application layer may include a series of application packages. Figure 3As shown, the application package may include multimedia applications, camera applications, call applications, Bluetooth applications, etc. The application layer may also include a system UI (system UI), which is used to display the interface of the electronic device, such as a video screen, a camera preview interface, etc.

[0096] The application framework layer provides application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions. Figure 3 As shown, the application framework layer may include a multimedia framework, a view system, a phone manager, a window manager, a notification manager, a content provider, a resource manager, etc., and the embodiments of the present application do not impose any restrictions on this.

[0097] The multimedia framework is used to perform operations such as protocol decompression, decapsulation, decoding, and transcoding on multimedia data, providing the ability to process multimedia data.

[0098] In the embodiments of the present application, Figure 4 As shown, the multimedia framework includes HDR video frames, DPU pathways, CPU and GPU pathways. The DPU pathway includes a video pipeline module (videopipeline), a three-dimensional lookup table module (3DLut module) and a first display module (DPU RGB display); the GPU pathway includes a restoration module (3DLut restoration), a three-dimensional mapping module (Lut mapping) and a second display module (GPU RGB display). The CPU can store the three-dimensional lookup table in the DPU pathway, and can also convert the three-dimensional lookup table into a two-dimensional image (2D 10bit image) and pass it to the GPU pathway.

[0099] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0100] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connecting, hanging up, etc.).

[0101] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0102] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0103] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0104] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0105] Android runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0106] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0107] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0108] The system library may include multiple functional modules, such as surface manager, media libraries, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0109] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0110] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats.

[0111] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0112] A 2D graphics engine is a drawing engine for 2D drawings.

[0113] The hardware abstraction layer is an interface layer between the operating system kernel and the hardware circuit. Its purpose is to abstract the hardware. It hides the hardware interface details of a specific platform and can provide a virtual hardware platform for the operating system. The hardware abstraction layer is an encapsulation of the Linux kernel driver, providing an interface to the upper layer and shielding the implementation details of the low-level hardware. Figure 3 As shown, the hardware abstraction layer may include audio HAL, Wi-Fi HAL, etc. Among them, the audio HAL is responsible for a part of the audio hardware abstraction layer, which provides an interface between the application and the audio hardware.

[0114] The kernel layer is the layer between hardware and software. It is the core of an operating system and the first layer of software expansion based on hardware. It provides the most basic functions of the operating system and is the basis for the operation of the operating system. It is responsible for managing the system's processes, memory, device drivers, files, and network systems, and determines the system's performance and stability. The kernel layer can include display drivers, audio drivers, camera drivers, sensor drivers, Bluetooth drivers, etc.

[0115] The hardware layer includes displays, cameras, sensors, etc.

[0116] Based on the above software structure, the present application embodiment provides a flow chart of a display method. Figure 5 As shown, the display method includes the following steps S501 to S509. Among them:

[0117] S501. At a first time point, a video channel module obtains data of a first video from an HDR video frame.

[0118] In an embodiment of the present application, the HDR video frame may be a video frame of a first video, where the first video may be an HDR video obtained from a cloud server, or an HDR picture captured by a camera when a user uses an electronic device to take a photo of a scene for preview, or an HDR video obtained from other electronic devices, without limitation herein.

[0119] At a first time point, a user uses an electronic device to play the first video. Under normal circumstances, the first video will be processed through the DPU pipeline. During the DPU processing, the video pipeline module in the DPU pipeline first obtains the data of the first video from the HDR video frame, and passes the data of the first video to the three-dimensional lookup table module for processing.

[0120] It should be noted that the data transmission between modules (sending and receiving data) is actually implemented through functions.

[0121] S502: The three-dimensional lookup table module processes the data of the first video to obtain a first frame of the first video.

[0122] In an embodiment of the present application, the three-dimensional lookup table module (3DLut module) in the DPU path calculates the 3DLut corresponding to the first video, uses 3DLut to perform color correction and mapping on the data of the first video to obtain the first picture of the first video; and passes the first picture of the first video to the first display module.

[0123] Among them, the data of the first video may include the first pixel value of the first video. The working principle of 3DLut is to input the first pixel value of the first video into 3DLut as the input pixel value of 3DLut, and then map it to the corresponding output pixel value according to its position in the color space, thereby realizing color correction and mapping.

[0124] S503: The first display module displays the first frame of the first video.

[0125] In the embodiment of the present application, the first display module (DPU RGB display) in the DPU channel displays the first frame of the processed first video. That is, the frame of the first video viewed by the user through the display screen at the first time point is the first frame of the first video processed by the DPU channel.

[0126] S504: The CPU stores the three-dimensional lookup table obtained from the three-dimensional lookup table module.

[0127] In the embodiment of the present application, the three-dimensional lookup table can be stored in the CPU or in other storage spaces (for example, the first storage space can store the three-dimensional lookup table obtained from the three-dimensional lookup table module), which is not limited here. The CPU can further store the 3DLut corresponding to the first video calculated in the DPU path, so that when the GPU path needs to be processed later, the CPU can convert the 3DLut corresponding to the first video into a two-dimensional image and pass it to the GPU path. In this way, the GPU path can also be equivalent to using the 3DLut for color correction and mapping, so that when the GPU path is processed, it can also present the same effect as the DPU path processing, solving the problem of brightness jump of the video screen.

[0128] The execution order of step S503 and step S504 is not limited.

[0129] S505. At a second time point, in response to displaying the first window in the display interface of the first video, the CPU maps the three-dimensional lookup table into a two-dimensional image and transmits it to the GPU path.

[0130] In the embodiment of the present application, at the second time point, the user operates the physical volume button on the electronic device, and a volume window (such as Figure 1C Or, the user touches the screen showing the first video, and a sidebar window (such as Figure 1D shown), etc.

[0131] In these cases, the electronic device will switch from the DPU pathway to the GPU pathway to process the data of the first video. At this time, the CPU will map the previously stored three-dimensional lookup table into a two-dimensional image and pass it to the GPU pathway. The reason is that the GPU pathway cannot support the direct use of the three-dimensional lookup table for color correction and mapping, so the CPU is required to map the three-dimensional lookup table into a two-dimensional image. The two-dimensional image at this time can maintain 10 bits, which is conducive to ensuring that the image is not distorted. Among them, the second time point is later than the first time point. In this way, the GPU pathway can also be equivalent to using the same 3DLut in the DPU pathway for color correction and mapping, which can make the GPU pathway present the same effect as the DPU pathway processing, thereby solving the problem of brightness jumps in the video screen.

[0132] Specifically, Figure 1A Expand the 3DLut in the equation to get Fig. 6A As shown in (a), each row in the list can represent a small cube in which the RGB color space is divided in 3DLut, and each small cube corresponds to an output pixel value.

[0133] Each row has a three-dimensional index (i.e., the index of the R channel, the index of the G channel, and the index of the B channel). By determining the three-dimensional index of the input pixel value, the corresponding small cube can be found in 3DLut, thereby mapping the corresponding output pixel value. In general, each row in the list corresponds to a three-dimensional pixel value (input pixel value or output pixel value).

[0134] Further, using formula (1) and formula (2), the three-dimensional index of each row in the list (i.e., the index of the R channel, the index of the G channel, and the index of the B channel) is mapped to the two-dimensional index in the two-dimensional image (i.e., the index of x and the index of y), so as to determine the location information of the three-dimensional pixel value (i.e., the input pixel value or the output pixel value) in the three-dimensional lookup table mapped to the two-dimensional image, that is, Fig. 6A The positional relationship is shown in (b). Wherein, formula (1) and formula (2) are as follows:

[0135] x=index_R+index_G*n (1)

[0136] y=index_B (2)

[0137] In formula (1), index_R is the index of the R channel, index_G is the index of the G channel, and n is the order of 3DLut. In formula (2), index_B is the index of the B channel.

[0138] Then, the three-dimensional pixel value corresponding to each row is determined as the pixel value corresponding to each position in the two-dimensional image, thereby obtaining the final two-dimensional image. Specifically, as shown in formula (3), the pixel value value(x,y) corresponding to each position in the two-dimensional image is:

[0139] value(x,y)=(R&0x03FF<<0)|(G&0x03FF<<10)|(B&0x03FF<<20)|(0xc0000000)(3)

[0140] like Figure 6B As shown, 3DLut includes input 3DLut (input pixel value) and output 3DLut (output pixel value), so two two-dimensional images are mapped, that is, input 3DLut can be mapped to a first two-dimensional image, and output 3DLut can be mapped to a second two-dimensional image.

[0141] Exemplarily, taking the first row in the list as an example, the indexes of the three dimensions of the first row are: the index of R is 1, the index of G is 0, and the index of B is 0. The input pixel value corresponding to the first row is 100 and the corresponding output pixel value is 190. Using formula (1) and formula (2), the indexes of the three dimensions corresponding to the first row are mapped to the indexes of two dimensions in the two-dimensional image, that is, the index of x is 1 and the index of y is 0. According to the input pixel value of the first row is 100 and the corresponding output pixel value is 190; then in the first two-dimensional image mapped by input 3DLut, the input pixel value corresponding to the point with coordinate position (1, 0) is 100; in the second two-dimensional image mapped by output 3DLut, the output pixel value corresponding to the point with coordinate position (1, 0) is 190.

[0142] S506: The restoration module obtains data of the first video from the HDR video frame, wherein the data of the first video includes a first pixel value of the first video.

[0143] S507: The restoration module determines index information corresponding to the first pixel value of the first video, and determines a three-dimensional pixel value corresponding to the index information in the two-dimensional image.

[0144] In the embodiments of the present application, Fig. 7AAs shown, the restoration module (3DLut restoration) in the GPU path can determine the minimum cube where the first pixel value of the first video is located in 3DLut according to the position of the first pixel value (i.e., point C). Furthermore, each vertex of the minimum cube (i.e., C 000 , C 001 , C 010 , C 011 , C 100 , C 101 , C 110 , C 111 ), that is, the index information corresponding to the first pixel value.

[0145] For example, the input3DLut of the three-dimensional lookup table is shown in Table 1, the value corresponding to index 0 is 0, the value corresponding to index 1 is 50, the value corresponding to index 2 is 100, the value corresponding to index 3 is 190, and the value corresponding to index 4 is 240.

[0146] Assume that the first pixel value of the first video is (90, 120, 225), find the index position of each vertex of the smallest cube in the three-dimensional lookup table according to the first pixel value, that is, C 000 The index position of C is (1, 2, 3). 001 The index position of C is (1, 3, 3). 010 The index position of C is (1, 2, 4). 011 The index position of C is (1, 3, 4). 100 The index position of C is (2, 2, 3). 101 The index position of C is (2, 3, 3). 110 The index position of C is (2, 2, 4). 111 The index position is (2, 3, 4).

[0147] Assume that the first pixel value of the first video is (20, 180, 150), find the index position of each vertex of the smallest cube in the three-dimensional lookup table according to the first pixel value, that is, C 000 The index position of C is (0, 1, 2). 001 The index position of C is (0, 2, 2). 010 The index position of C is (0, 1, 3). 011 The index position of C is (0, 2, 3). 100 The index position of C is (1, 1, 2). 101 The index position of C is (1, 2, 2). 110 The index position of C is (1, 1, 3). 111 The index position is (1, 2, 3).

[0148] Table 1

[0149] index Parameter Value 0 0 1 50 2 100 3 190 4 240

[0150] Further, the position coordinates (x, y) corresponding to each vertex in the two-dimensional image are determined according to the index information corresponding to the first pixel value (i.e., the index position of each vertex of the minimum cube), which can be specifically calculated using the following formulas (4) and (5):

[0151] x=((float(RGBIndex.g)+0.0)*n+(float(RGBIndex.r)+0.5))*stepWidth (4)

[0152] y=((float(RGBIndex.b)+0.5)*stepHeight (5)

[0153] In formula (4) and formula (5), RGBIndex.g is the index of the G channel of the vertex; RGBIndex.r is the index of the R channel of the vertex; stepWidth is the step size of the width of the image, usually 1.0; RGBIndex.b is the index of the B channel of the vertex; stepHeight is the step size of the height of the image, usually 1.0; x is the x-axis coordinate corresponding to each vertex in the two-dimensional image; y is the y-axis coordinate corresponding to each vertex in the two-dimensional image; float(·) is a single-precision floating point number; and n is the order of 3DLut.

[0154] Furthermore, since each position in the two-dimensional image corresponds to a three-dimensional pixel value (input pixel value or output pixel value), the three-dimensional pixel value corresponding to each vertex can be determined according to the position coordinates corresponding to each vertex in the two-dimensional image, and the three-dimensional pixel value corresponding to the index information can be determined. It can be understood that the input pixel value corresponding to the index information is determined in the first two-dimensional image, and the output pixel value corresponding to the index information is determined in the second two-dimensional image.

[0155] For example, suppose vertex C 000 The index position of is (1, 2, 3), and n is 4. The corresponding position coordinates in the two-dimensional image are calculated by formula (4) and formula (5) to be (9.5, 3.5); the input pixel value corresponding to the position coordinate (9.5, 3.5) in the first two-dimensional image is (60, 100, 190), and the output pixel value corresponding to the position coordinate (9.5, 3.5) in the second two-dimensional image is (50, 120, 220). Therefore, vertex C 000 The corresponding input pixel value is (60, 100, 190), vertex C 000 The corresponding output pixel value is (50, 120, 220).

[0156] For example, suppose vertex C 111 The index position of is (2, 3, 4), and n is 4. The corresponding position coordinates in the two-dimensional image are calculated by formula (4) and formula (5) to be (14.5, 4.5); the input pixel value corresponding to the position coordinates (14.5, 4.5) in the first two-dimensional image is (80, 120, 185), and the output pixel value corresponding to the position coordinates (14.5, 4.5) in the second two-dimensional image is (90, 160, 235). Therefore, vertex C 000 The corresponding input pixel value is (80, 120, 185), vertex C 000 The corresponding output pixel value is (90, 160, 235).

[0157] S508. The three-dimensional mapping module performs interpolation calculation based on the three-dimensional pixel value corresponding to the index information and the first pixel value of the first video to obtain a second pixel value; and determines a second picture of the first video based on the second pixel value.

[0158] In the embodiments of the present application, Figure 7B As shown, the three-dimensional mapping module (Lut mapping) in the GPU path calculates the distance between the first pixel value and each vertex of the minimum cube, and uses it as the interpolation weight information; during the interpolation operation, the input pixel value, output pixel value and interpolation weight information corresponding to the vertex are used to calculate to obtain the second pixel value; and the second picture of the first video is determined based on the processed second pixel value and passed to the second display module. Since the GPU path uses the two-dimensional image corresponding to the 3DLut in the DPU path for color correction and mapping, the GPU path can present the same effect as the DPU path processing, solving the problem of brightness jumps in the video image.

[0159] Specifically, the interpolation weight information can be determined using the following formula (6):

[0160]

[0161] In formula (6), r is the value of the R channel of the first pixel value, g is the value of the G channel of the first pixel value, and b is the value of the B channel of the first pixel value; R 0 Vertex C 000 The value of the R channel of the corresponding input pixel value, G 0 Vertex C 000 The value of the G channel of the corresponding input pixel value, B 0 Vertex C 000 The value of the B channel of the corresponding input pixel value; R 1 Vertex C 111 The value of the R channel of the corresponding input pixel value, G 1 Vertex C111 The value of the G channel of the corresponding input pixel value, B 1 Vertex C 111 The value of the B channel of the corresponding input pixel value; Δr, Δg, and Δb are the interpolation weight information.

[0162] Furthermore, the output pixel value and interpolation weight information corresponding to each vertex can be calculated using the following formula (7) to obtain the second pixel value:

[0163] V(r,g,b)=c 0 +c 1 Δ b +c 2 Δ r +c 3 Δ g +c 4 Δ b Δ r +c 5 Δ r Δ g +c 6 Δ g Δ b +c 7 Δ r Δ g Δ b (7)

[0164] In formula (7), Δr, Δg, Δb are the interpolation weight information calculated by the above formula (6); V(r, g, b) is the second pixel value;

[0165] c 0 =V(R 0 ,G 0 ,B 0 ), that is, vertex C 000 The corresponding output pixel value;

[0166] c 1 =V(R 0 ,G 0 ,B 1 )-V(R 0 ,G 0 ,B 0 ), that is, vertex C 001 Corresponding output pixel value - vertex C 000 The corresponding output pixel value;

[0167] c 2 =V(R 1 ,G 0 ,B 0 )-V(R 0 ,G 0 ,B 0), that is, vertex C 100 Corresponding output pixel value - vertex C 000 The corresponding output pixel value;

[0168] c 3 =V(R 0 ,G 1 ,B 0 )-V(R 0 ,G 0 ,B 0 ), that is, vertex C 010 Corresponding output pixel value - vertex C 000 The corresponding output pixel value;

[0169] c 4 =V(R 1 ,G 0 ,B 1 )-V(R 1 ,G 0 ,B 0 )-V(R 0 ,G 0 ,B 1 )+V(R 0 ,G 0 ,B 0 ), that is, vertex C 101 Corresponding output pixel value - vertex C 100 Corresponding output pixel value - vertex C 001 The corresponding output pixel value + vertex C 000 The corresponding output pixel value;

[0170] c 5 =V(R 1 ,G 1 ,B 0 )-V(R 0 ,G 1 ,B 0 )-V(R 1 ,G 0 ,B 0 )+V(R 0 ,G 0 ,B 0 ), that is, vertex C 110 Corresponding output pixel value - vertex C 010 Corresponding output pixel value - vertex C 100 The corresponding output pixel value + vertex C 000 The corresponding output pixel value;

[0171] c 6 =V(R 0 ,G 1 ,B 1 )-V(R0 ,G 1 ,B 0 )-V(R 0 ,G 0 ,B 1 )+V(R 0 ,G 0 ,B 0 ), that is, vertex C 011 Corresponding output pixel value - vertex C 010 Corresponding output pixel value - vertex C 001 The corresponding output pixel value + vertex C 000 The corresponding output pixel value;

[0172] c 7 =V(R 1 ,G 1 ,B 1 )-V(R 1 ,G 1 ,B 0 )-V(R 0 ,G 1 ,B 1 )-V(R 1 ,G 0 ,B 1 )+V(R 0 ,G 0 ,B 1 )+V(R 0 ,G 1 ,B 0 )

[0173] +V(R 1 ,G 0 ,B 0 )-V(R 0 ,G 0 ,B 0 ),

[0174] That is, vertex C 111 Corresponding output pixel value - vertex C 110 Corresponding output pixel value - vertex C 011 Corresponding output pixel value - vertex C 101 The corresponding output pixel value + vertex C 001 The corresponding output pixel value + vertex C 010 The corresponding output pixel value + vertex C 100 Corresponding output pixel value - vertex C 000 The corresponding output pixel value.

[0175] Based on this method, the traditional photoelectric conversion and electro-optical conversion functions are replaced by the interpolation method of a three-dimensional lookup table, which can avoid image differences and thus avoid the problem of sudden changes in the brightness of the video screen.

[0176] S509: The second display module displays the second picture of the first video.

[0177] In the embodiment of the present application, the second display module (GPU RGB display) in the GPU path displays the second screen of the processed first video. That is, the screen of the first video viewed by the user through the display screen at the second time point is the second screen of the first video processed by the GPU path. At this time, the screen brightness of the first video at the first time point is the same as the screen brightness of the first video at the second time point, avoiding the problem of jump in the brightness of the video screen and ensuring the visual effect.

[0178] 1. Regarding the situation where the electronic device displays the first window in the display interface of the first video.

[0179] Based on the above, another display method provided by the embodiment of the present application is further described in detail below. Figure 8 As shown, the display method includes the following steps S801 and S802. Figure 8 The method execution subject shown can be the electronic device mentioned above. Or, Figure 8 The method execution entity shown may be a chip in an electronic device, which is not limited in the embodiments of the present application. Figure 8 The method is described by taking an electronic device as an example of an execution subject.

[0180] S801. At a first time point, an electronic device displays a screen of a first video.

[0181] In the embodiment of the present application, a user uses an electronic device to play a first video at a first time point, and the electronic device will display the screen of the first video. The first video can be a video obtained from a cloud server, or a video captured by a camera when the user uses the electronic device to take a photo of a scene for preview, or a video obtained from other electronic devices, which is not limited here.

[0182] Optionally, the first video is an HDR video. Since HDR video is a high-brightness and high-saturation image, when the HDR video is blocked, it is more likely to cause a brightness jump problem. Therefore, for HDR video, the brightness jump problem of the video screen can be more obviously avoided.

[0183] In a possible implementation, at a first time point, when an electronic device displays a screen of a first video, a specific implementation may be: at the first time point, data of the first video is processed based on a first path to obtain a first screen of the first video, and the first screen of the first video is displayed. It should be noted that when a user uses an electronic device to play the first video, under normal circumstances, the first video will be processed through the DPU path. The first path here may refer to the DPU path.

[0184] It should be noted that in the software structure of the electronic device, the multimedia framework of the application framework layer includes HDR video frames, DPU pathways, CPU and GPU pathways. Among them, the DPU pathway includes a video pathway module (video pipeline), a three-dimensional lookup table module (3DLut module) and a first display module (DPU RGB display); the GPU pathway includes a restoration module (3DLut restoration), a three-dimensional mapping module (Lut mapping) and a second display module (GPU RGB display). At the first time point, the specific implementation process of the electronic device displaying the screen of the first video can refer to the above steps S501 to S504, that is:

[0185] At a first time point, the video path module obtains data of the first video from the HDR video frame; the three-dimensional lookup table module processes the data of the first video to obtain a first picture of the first video; the first display module displays the first picture of the first video. The CPU stores the three-dimensional lookup table obtained from the three-dimensional lookup table module.

[0186] S802. At a second time point, the electronic device displays a first window in the display interface of the first video; the screen brightness of the first video at the first time point is the same as the screen brightness of the first video at the second time point, and the second time point is later than the first time point.

[0187] In the embodiment of the present application, the first window here can be a volume window, a sidebar window, a notification window, etc., which is not limited here. Figure 1C As shown, at the second time point, the user operates the physical volume button on the electronic device, and the electronic device displays a volume window (ie, the first window) in the display interface of the first video. Figure 1D As shown, at the second time point, the user touches the screen displaying the first video, and the electronic device displays a sidebar window (ie, the first window) in the display interface of the first video.

[0188] In one possible implementation, the method further includes: at a second time point, in response to displaying a first window in a display interface of the first video, the electronic device processes data of the first video based on a second path to obtain a second picture of the first video; the electronic device displays the second picture of the first video; wherein the second path includes a two-dimensional image corresponding to the three-dimensional lookup table in the first path.

[0189] In these cases, the electronic device will switch from the DPU pathway to the GPU pathway to process the data of the first video. At this time, the CPU will map the previously stored three-dimensional lookup table into a two-dimensional image and pass it to the GPU pathway. The reason is that the GPU pathway cannot support the direct use of the three-dimensional lookup table for color correction and mapping, so the CPU is required to map the three-dimensional lookup table into a two-dimensional image. At this time, the two-dimensional image can be kept at 10 bits, which is conducive to ensuring that the image is not distorted. In this way, the GPU pathway can also be equivalent to using the same 3DLut in the DPU pathway for color correction and mapping, which can make the GPU pathway present the same effect as the DPU pathway processing, thereby solving the problem of brightness jumps in the video screen.

[0190] In a possible implementation manner, the method further includes: the electronic device mapping the three-dimensional lookup table into a two-dimensional image.

[0191] Optionally, the two-dimensional image includes a first two-dimensional image and a second two-dimensional image. When the electronic device maps the three-dimensional lookup table to the two-dimensional image, a specific implementation method may include the following steps 1 and 2: Fig. 9A shown.

[0192] Step 1: The electronic device determines that an input pixel value in the three-dimensional lookup table is mapped to first position information of a first two-dimensional image, and that an output pixel value in the three-dimensional lookup table is mapped to second position information of a second two-dimensional image.

[0193] Step 2: The electronic device determines a first two-dimensional image based on the input pixel value and the first position information, and determines a second two-dimensional image based on the output pixel value and the second position information.

[0194] Among them, the specific implementation method of step 1 and step 2 can refer to the specific implementation method of the above-mentioned step S505, that is: at the second time point, in response to displaying the first window in the display interface of the first video, the CPU maps the three-dimensional lookup table into a two-dimensional image and passes it to the GPU path.

[0195] Optionally, the data of the first video includes a first pixel value of the first video, and when the electronic device processes the data of the first video based on the second path to obtain a second picture of the first video, a specific implementation method may include the following steps s11 to s13, such as Fig. 9Bshown.

[0196] s11. In the second path, the electronic device obtains a first pixel value of the first video.

[0197] s12. The electronic device processes the first pixel value based on the two-dimensional image to obtain a second pixel value.

[0198] Further optionally, when the electronic device processes the first pixel value based on the two-dimensional image to obtain the second pixel value, a specific implementation method may include the following steps a to c: Fig. 9C shown.

[0199] Step a: The electronic device determines index information corresponding to the first pixel value.

[0200] Step b: The electronic device determines the input pixel value corresponding to the index information in the first two-dimensional image, and determines the output pixel value corresponding to the index information in the second two-dimensional image.

[0201] The specific implementation of step a and step b may refer to the specific implementation of steps S506 and S507 above, that is:

[0202] The restoration module obtains data of the first video from the HDR video frame; the restoration module determines index information corresponding to a first pixel value of the first video, and determines a three-dimensional pixel value corresponding to the index information in a two-dimensional image. The two-dimensional image includes a first two-dimensional image and a second two-dimensional image, and the three-dimensional pixel value includes an input pixel value or an output pixel value, that is, the input pixel value corresponding to the index information is determined in the first two-dimensional image, and the output pixel value corresponding to the index information is determined in the second two-dimensional image.

[0203] Step c: The electronic device performs interpolation calculation based on the input pixel value corresponding to the index information, the output pixel value corresponding to the index information, and the first pixel value to obtain a second pixel value.

[0204] Based on this method, the traditional photoelectric conversion and electro-optical conversion functions are replaced by the interpolation method of a three-dimensional lookup table, which can avoid image differences and thus avoid the problem of sudden changes in the brightness of the video screen.

[0205] s13. The electronic device determines a second frame of the first video based on the second pixel value.

[0206] The specific implementation of step c and step s13 may refer to the specific implementation of step S508 above, that is:

[0207] The three-dimensional mapping module performs interpolation calculation based on the three-dimensional pixel value corresponding to the index information and the first pixel value of the first video to obtain a second pixel value; and determines a second picture of the first video based on the second pixel value.

[0208] In addition, the specific implementation method of the electronic device displaying the second screen of the first video can refer to the specific implementation method of the above-mentioned step S509, that is: the second display module displays the second screen of the first video, which is not repeated here.

[0209] In a possible implementation, the method also includes: at a third time point, the electronic device closes the first window in the display interface of the first video; the screen brightness of the first video at the second time point is the same as the screen brightness of the first video at the third time point, and the third time point is later than the second time point.

[0210] Optionally, the method also includes: at a third time point, in response to the first window disappearing in the display interface of the first video, the electronic device processes the data of the first video based on the first path, obtains a third picture of the first video, and displays the third picture of the first video.

[0211] It can be understood that at the third time point, when the first window in the display interface of the first video is closed, the electronic device switches from the GPU path to the DPU path to process the data of the first video and obtains the third picture of the first video. Similarly, since the GPU path uses the three-dimensional lookup table in the DPU path to correspond to the two-dimensional image, it is equivalent to using the same three-dimensional lookup table for color correction and mapping, so there will be no problem of brightness jumps in the video picture, that is, the picture brightness of the first video at the second time point is the same as the picture brightness of the first video at the third time point, thereby ensuring the visual effect.

[0212] In a possible implementation, the method also includes: at a fourth time point, the electronic device switches the first video to window mode; the screen brightness of the first video at the third time point is the same as the screen brightness of the first video at the fourth time point, and the fourth time point is later than the third time point.

[0213] Optionally, the method also includes: at a fourth time point, in response to the operation of switching the first video to window mode, the electronic device processes the data of the first video based on the second path to obtain a fourth frame of the first video, and displays the fourth frame of the first video.

[0214] It can be understood that, Figure 1EAs shown, at the fourth time point, the user switches the first video to window mode by dragging the display interface of the first video. In this case, the electronic device will switch from the DPU path to the GPU path to process the data of the first video to obtain the fourth picture of the first video. Similarly, since the GPU path uses the three-dimensional lookup table in the DPU path to correspond to the two-dimensional image, it is equivalent to using the same three-dimensional lookup table for color correction and mapping, so there will be no problem of brightness jumps in the video picture, that is, the picture brightness of the first video at the third time point is the same as the picture brightness of the first video at the fourth time point, thereby ensuring the visual effect.

[0215] It can be seen that based on the method described in this application, the screen brightness of the first video at the first time point is the same as the screen brightness of the first video at the second time point, which can avoid the problem of sudden changes in the brightness of the video screen and ensure the visual effect.

[0216] 2. Regarding the case where the electronic device switches the first video to the window mode.

[0217] Based on the above, another display method provided by the embodiment of the present application is further described in detail below. Fig.10 As shown, the display method includes the following steps S1001 and S1002. Fig.10 The method execution subject shown can be the electronic device mentioned above. Or, Fig.10 The method execution entity shown may be a chip in an electronic device, which is not limited in the embodiments of the present application. Fig.10 The method is described by taking an electronic device as an example of an execution subject.

[0218] S1001. At a fifth time point, the electronic device displays a screen of a first video.

[0219] In one possible implementation, at the fifth time point, when the electronic device displays the screen of the first video, the specific implementation method may be: at the fifth time point, the data of the first video is processed based on the first path to obtain the first screen of the first video, and the first screen of the first video is displayed. It should be noted that when the user uses the electronic device to play the first video, under normal circumstances, the first video will be processed through the DPU path. The first path here may refer to the DPU path. Optionally, the first video is an HDR video. Among them, the specific implementation method of step S1001 can refer to the specific implementation method of the above-mentioned step S801, which will not be repeated here.

[0220] It should be noted that in the software structure of the electronic device, the multimedia framework of the application framework layer includes HDR video frames, DPU pathways, CPU and GPU pathways. Among them, the DPU pathway includes a video pathway module (video pipeline), a three-dimensional lookup table module (3DLut module) and a first display module (DPU RGB display); the GPU pathway includes a restoration module (3DLut restoration), a three-dimensional mapping module (Lut mapping) and a second display module (GPU RGB display). At the fifth time point, the specific implementation process of the electronic device displaying the screen of the first video can refer to the above steps S501 to S504, that is:

[0221] At the fifth time point, the video path module obtains data of the first video from the HDR video frame; the three-dimensional lookup table module processes the data of the first video to obtain a first picture of the first video; the first display module displays the first picture of the first video. The CPU stores the three-dimensional lookup table obtained from the three-dimensional lookup table module.

[0222] S1002. At a sixth time point, the electronic device switches the first video to a window mode; the screen brightness of the first video at the fifth time point is the same as the screen brightness of the first video at the sixth time point, and the sixth time point is later than the fifth time point.

[0223] In a possible implementation, the method further includes: at a sixth time point, in response to an operation of switching the first video to the window mode, the electronic device processes the data of the first video based on the second path to obtain a second picture of the first video, and displays the second picture of the first video. Figure 1E As shown, at the sixth time point, the user switches the first video to the window mode by dragging the display interface of the first video. In this case, the electronic device will also switch from the DPU path to the GPU path to process the data of the first video. The second path here can be the GPU path.

[0224] In a possible implementation manner, the method further includes: the electronic device mapping the three-dimensional lookup table into a two-dimensional image.

[0225] Optionally, the two-dimensional image includes a first two-dimensional image and a second two-dimensional image. When the electronic device maps the three-dimensional lookup table to a two-dimensional image, the specific implementation method can refer to the above steps 1 and 2, that is, the electronic device determines the first position information of the first two-dimensional image mapped to the input pixel value in the three-dimensional lookup table, and the second position information of the second two-dimensional image mapped to the output pixel value in the three-dimensional lookup table; the electronic device determines the first two-dimensional image based on the input pixel value and the first position information, and determines the second two-dimensional image based on the output pixel value and the second position information.

[0226] Optionally, the data of the first video includes a first pixel value of the first video, and when the electronic device processes the data of the first video based on the second path to obtain a second picture of the first video, a specific implementation method may refer to the above steps s11 to s13, that is: in the second path, the electronic device obtains the first pixel value of the first video; the electronic device processes the first pixel value based on the two-dimensional image to obtain a second pixel value; and the electronic device determines the second picture of the first video based on the second pixel value.

[0227] Further optionally, when the electronic device processes the first pixel value based on the two-dimensional image to obtain the second pixel value, the specific implementation method can refer to the above steps a to c, that is: the electronic device determines the index information corresponding to the first pixel value; the electronic device determines the input pixel value corresponding to the index information in the first two-dimensional image, and determines the output pixel value corresponding to the index information in the second two-dimensional image; the electronic device performs interpolation calculation based on the input pixel value corresponding to the index information, the output pixel value corresponding to the index information and the first pixel value to obtain the second pixel value.

[0228] Specifically, reference may be made to the description of steps S505 to S509 above: at a sixth time point, in response to the operation of switching the first video to the window mode, the CPU maps the three-dimensional lookup table to a two-dimensional image and passes it to the GPU path; the restoration module obtains data of the first video from the HDR video frame, wherein the data of the first video includes a first pixel value of the first video; the restoration module determines the index information corresponding to the first pixel value of the first video, and determines the three-dimensional pixel value corresponding to the index information in the two-dimensional image; the three-dimensional mapping module performs interpolation calculation based on the three-dimensional pixel value corresponding to the index information and the first pixel value of the first video to obtain a second pixel value, and determines the second picture of the first video based on the second pixel value; the second display module displays the second picture of the first video.

[0229] It can be understood that at the sixth time point, the user can switch the first video to window mode by dragging the display interface of the first video. At this time, the electronic device will also switch from the DPU path to the GPU path to process the data of the first video. Similarly, since the GPU path uses the three-dimensional lookup table in the DPU path to correspond to the two-dimensional image, it is equivalent to using the same three-dimensional lookup table for color correction and mapping, so there will be no problem of brightness jumps in the video screen, that is, the screen brightness of the first video at the fifth time point is the same as the screen brightness of the first video at the sixth time point, thereby ensuring the visual effect. Specifically, please refer to the description in the above step S802, which will not be repeated here.

[0230] It can be seen that based on the method described in the present application, the screen brightness of the first video at the fifth time point is the same as the screen brightness of the first video at the sixth time point, which can avoid the problem of sudden changes in the brightness of the video screen and ensure the visual effect.

[0231] See also Fig.11 , Fig.11 A schematic structural diagram of a display device 1100 according to an embodiment of the present application is shown. Fig.11 The display device shown may be an electronic device, or a device in an electronic device, or a device that can be used in conjunction with an electronic device. Fig.11 The display device shown may include a processing unit 1101 and a display unit 1102. Among them:

[0232] The display unit 1102 is used to display a picture of a first video at a first time point;

[0233] The display unit 1102 is also used to display a first window in the display interface of the first video at a second time point; the screen brightness of the first video at the first time point is the same as the screen brightness of the first video at the second time point, and the second time point is later than the first time point.

[0234] In a possible implementation, at a first time point, when displaying a picture of a first video: a processing unit 1101 is specifically configured to process data of the first video based on a first path at the first time point to obtain a first picture of the first video, and a display unit 1102 is configured to display the first picture of the first video;

[0235] The processing unit 1101 is further used to: at a second time point, in response to displaying a first window in the display interface of the first video, process the data of the first video based on the second path to obtain a second picture of the first video; the display unit 1102 is also used to display the second picture of the first video; wherein the second path includes a two-dimensional image corresponding to the three-dimensional lookup table in the first path.

[0236] In a possible implementation, the processing unit 1101 is further used to: map the three-dimensional lookup table into a two-dimensional image; the data of the first video includes a first pixel value of the first video, and the processing unit 1101, when processing the data of the first video based on the second path to obtain the second picture of the first video, is specifically used to: obtain the first pixel value of the first video in the second path; process the first pixel value based on the two-dimensional image to obtain a second pixel value; and determine the second picture of the first video based on the second pixel value.

[0237] In one possible implementation, the two-dimensional image includes a first two-dimensional image and a second two-dimensional image. The processing unit 1101, when mapping the three-dimensional lookup table to a two-dimensional image, is specifically used to: determine the first position information in which the input pixel value in the three-dimensional lookup table is mapped to the first two-dimensional image, and the second position information in which the output pixel value in the three-dimensional lookup table is mapped to the second two-dimensional image; determine the first two-dimensional image based on the input pixel value and the first position information, and determine the second two-dimensional image based on the output pixel value and the second position information.

[0238] In one possible implementation, the processing unit 1101, when processing the first pixel value based on the two-dimensional image to obtain the second pixel value, is specifically used to: determine the index information corresponding to the first pixel value; determine the input pixel value corresponding to the index information in the first two-dimensional image, and determine the output pixel value corresponding to the index information in the second two-dimensional image; perform interpolation calculation based on the input pixel value corresponding to the index information, the output pixel value corresponding to the index information, and the first pixel value to obtain the second pixel value.

[0239] In one possible implementation, the processing unit 1101 is also used to: at a third time point, close the first window in the display interface of the first video; the screen brightness of the first video at the second time point is the same as the screen brightness of the first video at the third time point, and the third time point is later than the second time point.

[0240] In one possible implementation, the processing unit 1101 is also used to: switch the first video to window mode at a fourth time point; the screen brightness of the first video at the third time point is the same as the screen brightness of the first video at the fourth time point, and the fourth time point is later than the third time point.

[0241] In a possible implementation, the first video is an HDR video.

[0242] For the case where the display device may be a chip or a chip system, see Fig.12 Schematic diagram of the chip structure shown. Fig.12 The chip 1200 shown includes a processor 1201 and an interface 1202. Optionally, it may also include a memory 1203. The number of the processor 1201 may be one or more, and the number of the interface 1202 may be multiple.

[0243] For the case where the chip is used to implement the electronic device in the embodiment of the present application:

[0244] The interface 1202 is used to receive or output signals;

[0245] The processor 1201 is used to execute data processing operations of the electronic device.

[0246] It is understandable that some optional features in the embodiments of the present application may be implemented independently in certain scenarios without relying on other features, such as the solution on which they are currently based, to solve corresponding technical problems and achieve corresponding effects, or may be combined with other features according to needs in certain scenarios. Accordingly, the data processing device provided in the embodiments of the present application may also implement these features or functions accordingly, which will not be described in detail here.

[0247] It should be understood that the processor in the embodiment of the present application can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, a digital signal processor (digital signal processor, DSP), an application specific integrated circuit (application specific integrated circuit, ASIC), a field programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0248] It can be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0249] The present application also provides a display system, which includes an electronic device; wherein the electronic device is used to execute the method executed by the electronic device in any of the above method embodiments.

[0250] The present application also provides a computer-readable storage medium, in which a computer program is stored. The computer program includes program instructions. When the program instructions are executed on an electronic device, the functions of any of the above method embodiments are implemented.

[0251] The present application also provides a computer program product. When the computer program product is executed on a computer, the computer can implement the functions of any of the above method embodiments.

[0252] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0253] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A display method, It is characterized in that The method comprises: At a first time point, displaying a picture of a first video; At a second time point, a first window is displayed in the display interface of the first video; the screen brightness of the first video at the first time point is the same as the screen brightness of the first video at the second time point, and the second time point is later than the first time point.

2. The method according to claim 1, It is characterized in that The step of displaying a first video at a first time point includes: At a first time point, data of the first video is processed based on a first path to obtain a first frame of the first video, and the first frame of the first video is displayed; The method further comprises: At the second time point, in response to displaying a first window in the display interface of the first video, processing data of the first video based on a second path to obtain a second picture of the first video; Displaying a second picture of the first video; The second path includes a two-dimensional image corresponding to the three-dimensional lookup table in the first path.

3. The method according to claim 2, It is characterized in that The method further comprises: Mapping the three-dimensional lookup table into a two-dimensional image; The data of the first video includes a first pixel value of the first video, the data of the first video includes a first pixel value of the first video, and the processing of the data of the first video based on the second path to obtain a second picture of the first video includes: In a second path, obtaining a first pixel value of the first video; Processing the first pixel value based on the two-dimensional image to obtain a second pixel value; A second picture of the first video is determined based on the second pixel value.

4. The method according to claim 3, It is characterized in that The two-dimensional image includes a first two-dimensional image and a second two-dimensional image, and mapping the three-dimensional lookup table into the two-dimensional image includes: Determine first position information of the first two-dimensional image mapped to an input pixel value in the three-dimensional lookup table, and second position information of the second two-dimensional image mapped to an output pixel value in the three-dimensional lookup table; The first two-dimensional image is determined based on the input pixel value and the first position information, and the second two-dimensional image is determined based on the output pixel value and the second position information.

5. The method according to claim 4, It is characterized in that The processing of the first pixel value based on the two-dimensional image to obtain a second pixel value includes: Determining index information corresponding to the first pixel value; Determine an input pixel value corresponding to the index information in the first two-dimensional image, and determine an output pixel value corresponding to the index information in the second two-dimensional image; An interpolation calculation is performed based on the input pixel value corresponding to the index information, the output pixel value corresponding to the index information, and the first pixel value to obtain a second pixel value.

6. The method according to any one of claims 1 to 5, It is characterized in that The method further comprises: At a third time point, close the first window in the display interface of the first video; the screen brightness of the first video at the second time point is the same as the screen brightness of the first video at the third time point, and the third time point is later than the second time point.

7. The method according to any one of claims 1 to 6, It is characterized in that The method further comprises: At a fourth time point, the first video is switched to window mode; the screen brightness of the first video at the third time point is the same as the screen brightness of the first video at the fourth time point, and the fourth time point is later than the third time point.

8. The method according to any one of claims 1 to 7, It is characterized in that The first video is a high dynamic range imaging HDR video.

9. An electronic device, It is characterized in that include: One or more processors, one or more memories; wherein the one or more memories are coupled to the one or more processors, the one or more memories are used to store computer program codes, the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the method as described in any one of claims 1-8.

10. A display system, It is characterized in that Comprising an electronic device; wherein, the electronic device is used to execute the method as described in any one of claims 1-8.

11. A chip, It is characterized in that It comprises a processor and an interface, the processor and the interface are coupled; the interface is used to receive or output signals, and the processor is used to execute code instructions so that the method described in any one of claims 1 to 8 is executed.

12. A computer storage medium, It is characterized in that The computer storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 8.

13. A computer program product, It is characterized in that When the computer program product is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 8.

Images