Display device and page loading method

Abstract

The invention provides a display device and a page loading method. When the display device receives to-be-decoded data returned by the server, whether the decoding data format of the to-be-decoded data is within the hard decoding format range of the display device or not is judged firstly, and when the decoding data format is within the hard decoding format range, page loading data is decoded based on a hard decoding mode to obtain first video decoding data. And controlling the display to display a media resource picture corresponding to the first video decoding data. By reasonably calling the hard decoding function of the display equipment, the pressure of a processor and memory resources can be relieved, so that the display equipment can keep high performance in the page loading process, and the page loading speed is increased.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and a page loading method. Background Art

[0002] When loading a page, the display device can obtain data from the server and load the page according to the data sent back by the server. When loading a page, the display device decodes the data sent back by the server through the soft decoding function provided by the browser, and then renders the page through the decoded data.

[0003] However, the processor performance of display devices is weaker than that of computers and mobile phones, and the memory resources are also more limited. Therefore, if the display device only decodes the data sent back by the server through the soft decoding function provided by the browser during operation, the processor and memory usage will increase, which will lead to a decrease in the performance of the display device and a decrease in decoding efficiency. Summary of the invention

[0004] The present application provides a display device and a page loading method to solve the problem that the display device only decodes audio and video data by soft decoding, resulting in a decrease in the performance of the display device.

[0005] In a first aspect, the present application provides a display device, comprising: a display; a communicator configured to communicate with a server; and a controller. The controller is coupled to the display and the communicator, and is configured to:

[0006] In response to the received page access instruction, controlling the communicator to send a data request instruction to the server, wherein the data request instruction is used to obtain page loading data;

[0007] When controlling the communicator to receive the page loading data from the server, acquiring the decoding data format of the page loading data and the hard decoding format range of the display device;

[0008] If the decoded data format is within the hard decoding format range, decoding the page loading data based on the hard decoding mode to obtain first video decoding data;

[0009] The display is controlled to display a media asset picture corresponding to the first video decoding data.

[0010] When receiving the page loading data sent back by the server, the display device can determine whether the decoding data format supported by the page loading data is within the hard decoding format range of the display device. When the decoding data format is within the hard decoding format range, the page loading data can be decoded by hard decoding to relieve the operating pressure of the processor and memory, thereby improving the page loading speed based on the decoding efficiency of hard decoding and relieving the operating pressure of the display device.

[0011] In some feasible embodiments, after the controller compares the decoded data format with the decoded format range, the controller further includes:

[0012] If the decoded data format is outside the decoding format range, decoding the page loading data based on a soft decoding mode to obtain second video decoded data;

[0013] The display is controlled to display a media asset picture corresponding to the second video decoding data.

[0014] When the decoded data format of the page loading data is outside the hard decoding format range, the display device can continue to use the soft decoding mode provided by the browser to decode the page loading data, so as to decode the page loading data based on the soft decoding method and display the media asset screen.

[0015] In some feasible embodiments, the controller performs decoding of the page loading data based on the hard decoding mode to obtain the first video decoding data, and is further configured to:

[0016] Set up the video layer and graphics layer;

[0017] When the hard decoding interface performs hard decoding on the page loading data, the display is controlled to superimpose the video layer and the graphic layer, and a local media picture with an area smaller than the video layer is displayed in the video layer, and an occluded picture is displayed in the graphic layer; the display level of the graphic layer is higher than the display level of the video layer; the local media picture is a media picture corresponding to part of the first video decoding data obtained through the hard decoding mode.

[0018] The display device displays the decoded data obtained by hard decoding in the video layer, and displays the decoded data obtained by soft decoding in the graphics layer. Then, the graphics layer can be superimposed on the video layer, so that the occluded image displayed by the graphics layer can occlude the partial media image of the video layer when the complete hard decoding data is not obtained.

[0019] In some feasible embodiments, the controller performs decoding of the page loading data based on the hard decoding mode to obtain the first video decoding data, and is further configured to:

[0020] Copying the page loading data to obtain copied page data identical to the page loading data;

[0021] Sending the copied page data to a data consumption interface through a first process, so as to obtain a size parameter included in the copied page data through the data consumption interface;

[0022] The display size of the occlusion picture is set according to the size parameter; the display size of the occlusion picture is the same as the display size of the media asset picture corresponding to the first video decoding data.

[0023] The copied page loading data has the same data information as the page loading data, and may include data format and display size. In this way, by inputting the copied page loading data into the data consumption interface, on the one hand, the integrity of the browser architecture can be maintained, so that the browser can operate normally. On the other hand, the display size of the media data can be obtained from the data consumption interface, and then the size of the occlusion screen can be controlled according to the display size to effectively occlude the partial media screen displayed by the video layer.

[0024] In some feasible embodiments, when the controller performs decoding of the page loading data based on the hard decoding mode to obtain the first decoded data, the controller is specifically configured as follows:

[0025] Sending the page loading data to a hard decoding interface through a first process, so that the hard decoding interface performs hard decoding on the page loading data to obtain the first video decoding data;

[0026] The display is controlled to display a media asset picture corresponding to the first video decoding data in the video layer.

[0027] When the display device receives the page loading data sent back by the server, it can establish a first process through the browser, and send the page loading data to the hard decoding interface through the first process, and the hard decoding interface performs hard decoding on the page loading data to obtain the first video decoding data, and control the display to display the media asset picture corresponding to the first video decoding data in the video layer.

[0028] In some feasible embodiments, the controller executes sending the page loading data to the hard decoding interface through the first process, which is specifically configured as follows:

[0029] Obtaining a first communication identifier of the first process and a second communication identifier of the second process;

[0030] Based on the communication connection relationship determined by the first communication identifier and the second communication identifier, the page loading data is sent to the second process through the first process; the first process is associated with the soft decoding mode; the second process is associated with the hard decoding mode;

[0031] The page loading data is sent to the hard decoding interface through the second process, so that the hard decoding interface decodes the page loading data.

[0032] The display device can also send the page loading data to the hard decoding interface through the second process. Since calling the hard decoding interface requires higher permissions, the second process with higher permissions can be used to receive the page loading data sent by the first process, and then send the page loading data to the hard decoding interface to achieve the migration of page loading data.

[0033] In some feasible embodiments, when the controller controls the display to display the media asset picture corresponding to the first video decoding data, the controller is configured to:

[0034] When the hard decoding interface completes hard decoding of the page loading data, determining an overlapping area of ​​the video layer and the graphics layer according to display sizes of the video layer and the graphics layer;

[0035] A transparency process is performed on a portion of the graphic layer corresponding to the overlapping area to display the media asset picture displayed in the video layer.

[0036] When hard decoding is completed, the display device can cancel the occlusion of the video layer by the graphic layer, that is, transparently process the overlapping area of ​​the graphic layer and the video layer to completely display the complete media asset picture corresponding to the first decoded data in the video layer.

[0037] In some feasible embodiments, an audio player coupled to the controller is further included, and the page loading data includes audio encoding data; the controller is further configured to:

[0038] Sending the audio encoding data to the audio decoding interface through the first process to obtain audio decoding data;

[0039] Based on the time stamps of the audio decoded data and the video decoded data, when controlling the display to display the media asset picture corresponding to the first video decoded data, the audio player is controlled to play the audio data corresponding to the time stamp of the media asset picture among all the audio data corresponding to the audio decoded data.

[0040] When the display device loads the page, it can also decode and load the audio data. According to the time stamp of the audio decoding data and the video decoding data, the video and audio can be played synchronously.

[0041] In some feasible embodiments, when the controller performs rendering of a page based on the second video decoding data rendering band, the controller is specifically configured as follows:

[0042] When the page loading data is decoded based on the soft decoding mode, a media asset picture is rendered based on the second video decoding data through a first process;

[0043] The display is controlled to play a media resource image rendered by the second video decoding data.

[0044] The display device can establish a first process through a browser, and render a media screen through the second video decoding data through the first process. Then, the display is controlled to play the media screen rendered by the second video decoding data. Then, when the display device receives the page loading data returned by the server, it can adopt a software and hardware collaboration method to improve the page loading efficiency.

[0045] In a second aspect, the present application further provides a page loading method, which is applied to the display device described in the first aspect, comprising:

[0046] In response to the received page access instruction, sending a data request instruction to the server, wherein the data request instruction is used to obtain page loading data;

[0047] When receiving the page loading data from the server, obtaining a decoding data format of the page loading data and a hard decoding format range of a display device;

[0048] If the decoded data format is within the hard decoding format range, decoding the page loading data based on the hard decoding mode to obtain first video decoding data;

[0049] A media asset screen corresponding to the first video decoding data is displayed.

[0050] It can be seen from the above technical content that the present application provides a display device and a page loading method. When the display device receives the data to be decoded sent back by the server, it first determines whether the decoded data format of the data to be decoded is within the hard decoding format range of the display device. When the decoded data format is within the hard decoding format range, the page loading data is decoded based on the hard decoding mode to obtain the first video decoding data. And the display is controlled to display the media resource screen corresponding to the first video decoding data. By reasonably calling the hard decoding function of the display device, the pressure on the processor and memory resources can be relieved, so that the display device can maintain a high performance during the page loading process and improve the page loading speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0052] Figure 1A schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

[0053] Figure 2 A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;

[0054] Figure 3 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0055] Figure 4 A timing diagram of a display device provided in some embodiments of the present application decoding page loading data based on a hard decoding function;

[0056] Figure 5 A timing diagram of a display device provided in some embodiments of the present application decoding page loading data based on a soft decoding function;

[0057] Figure 6 A flow chart of determining the decoding of page loading data based on the data format of the page loading data provided in some embodiments of the present application;

[0058] Figure 7 A schematic diagram of the hierarchical relationship between the graphics layer and the video layer provided in some embodiments of the present application;

[0059] Figure 8 A schematic diagram of the operation of a first process provided in some embodiments of the present application;

[0060] Fig. 9 A schematic diagram of migrating page loading data from a first process to a second process provided in some embodiments of the present application;

[0061] Fig.10 A schematic diagram of a transparent processing graphic layer provided in some embodiments of the present application. DETAILED DESCRIPTION

[0062] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0063] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0064] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0065] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0066] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0067] In the embodiment of the present application, the display device 200 generally refers to a device with image display and data processing capabilities. For example, the display device 200 includes but is not limited to a smart TV, a mobile terminal, a computer, a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, etc.

[0068] Figure 1 This is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG. 1 , a user can operate the display device 200 through touch operation, the mobile terminal 300 and the control device 100. The control device 100 is used to receive operation instructions input by the user and convert the operation instructions into control instructions that can be recognized and responded to by the display device 200. For example, the control device 100 can be a remote controller, a stylus pen, a handle, etc.

[0069] The mobile terminal 300 can be used as a control device for performing human-computer interaction between the user and the display device 200. The mobile terminal 300 can also be used as a communication device for establishing a communication connection with the display device 200 and performing data interaction. In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.

[0070] In some embodiments, the mobile terminal 300 or other electronic devices may also simulate the functions of the control device 100 by running an application program for controlling the display device 200 .

[0071] like Figure 1As also shown in FIG. 4 , the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0072] The display device 200 may provide a broadcast receiving television function, and may also additionally provide an intelligent network television function with a computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), and the like.

[0073] Figure 2 Some embodiments of the present application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0074] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0075] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0076] In some embodiments, the display 260 includes a display function component for presenting a picture, and a driving component for driving an image display. The display 260 is used to receive an image signal output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, and components of a menu control interface and a user control UI interface.

[0077] In some embodiments, the communication device 220 is a component for communicating with an external device or server 400 according to various communication protocol types. The display device 200 may be provided with a plurality of communication devices 220 according to different supported communication modes. For example, when the display device 200 supports wireless network communication, the display device 200 may be provided with a communication device 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including a Bluetooth function.

[0078] The communication device 220 can enable the display device 200 to communicate with the external device or server 400 by wireless or wired connection. Among them, the wired connection can connect the display device 200 with the external device through components such as data cables and interfaces. The wireless connection can connect the display device 200 with the external device through wireless signals or wireless networks. The display device 200 can establish a connection relationship with the external device directly, or indirectly establish a connection relationship through a gateway, a router, a connection device, etc.

[0079] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0080] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0081] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0082] In some embodiments, the audio output device 270 may be a local speaker of the display device 200, or may be an external audio output device of the display device 200. In particular, for the external audio output device of the display device 200, the display device 200 may also be provided with an external audio output terminal, and the audio output device may be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0083] In some embodiments, the user input interface 280 may be used to receive instructions from a user.

[0084] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling hardware resources and software resources in the display device 200. The operating system can control the display device to provide a user interface, for example, the operating system can directly control the display device to provide a user interface, or can provide a user interface by running an application program. The operating system also allows the user to interact with the display device 200.

[0085] It should be noted that the operating system may be a native operating system based on a specific operating platform, or a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for a display device.

[0086] The operating system can be divided into different modules or layers according to the functions implemented, such as Figure 3 As shown, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the system library layer and the kernel layer.

[0087] In some embodiments, the application layer is used to provide services and interfaces for applications so that the display device 200 can run applications and interact with users based on the applications. At least one application can be run in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. that come with the operating system; they can also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0088] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Applications can access system resources and obtain system services during execution through the API interface.

[0089] like Figure 3 As shown, Figure 3 A software configuration diagram of a display device provided for some embodiments of the present application. In some embodiments, the system of the display device 200 can be divided into three layers, namely, an application layer, a middleware layer, and a hardware layer from top to bottom.

[0090] The application layer mainly includes applications on the TV and the application framework. Among them, applications are mainly applications developed based on the browser, such as HTML5 APPs and native applications.

[0091] The Application Framework is a complete program model that has all the basic functions required by standard application software, such as file access, data exchange, etc., as well as the user interfaces of these functions (toolbars, status bars, menus, dialog boxes).

[0092] Native apps can support online or offline, message push or local resource access.

[0093] The middleware layer includes various TV protocols, multimedia protocols, system components and other middleware. The middleware can use the basic services (functions) provided by the system software to connect various parts of the application system or different applications on the network, and can achieve the purpose of resource sharing and function sharing.

[0094] The hardware layer mainly includes HAL interface, hardware and driver. Among them, HAL interface is the unified interface for all TV chips to connect, and the specific logic is implemented by each chip. Drivers mainly include: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0095] It should be noted that the above example is only a simple division of the operating system functions and does not constitute a limitation on the specific operating system form of the display device 200 in the embodiment of the present application. Depending on factors such as the function of the display device and the type of operating system, the number of levels and specific level types contained in the operating system may be expressed in other forms.

[0096] After starting the display device 200, the user can select a preset control in the display device to start the page corresponding to the preset control. In some embodiments, the page corresponding to the preset control is a pre-set picture, so there is no need to load it. In other embodiments, the page corresponding to the preset control is a page that needs to be loaded. For example, in a cloud game scenario, the user selects a cloud game-related control through a remote control, voice, or other input method. The display device 200 displays the page corresponding to the preset control in response to the selection instruction input by the user.

[0097] It is understandable that the display device 200 needs to request page loading data from the server, load the page through the page loading data returned by the server, and display it to the user. In some embodiments, the display device 200 can establish a first process (render) through a built-in browser, and the first process can create a WebRTC instance, and then receive the page loading data returned by the server through WebRTC. The page loading data may include audio data and video data. Then the first process can decode the page loading data based on the soft decoding mode that the browser can provide, obtain the decoded data, and then render the page according to the decoded data to display the media asset screen corresponding to the page loading data to the user.

[0098] However, decoding the page loading data based on the soft decoding mode provided by the browser requires occupying the CPU, memory and other resources of the display device. However, the CPU and memory resources of the display device are more scarce than those of high-computing computers, mobile phones and other terminal devices, and the display device needs to perform a large number of interface interactions, data loading and other processes during operation. Therefore, decoding the page loading data only through the soft decoding mode is likely to cause the performance of the display device 200 to degrade due to occupying too many hardware resources, and will also affect the page loading speed, resulting in a degraded user experience.

[0099] In order to solve the above problems, the present application provides a display device, including: a display 260, a communicator 220 and a controller 250. The communicator is configured to establish a communication connection with a server. Figure 4 As shown, the controller 250 is configured as follows:

[0100] S100: In response to a received page access instruction, control the communicator to send a data request instruction to a server, where the data request instruction is used to obtain page loading data.

[0101] In some embodiments, when a user selects a preset control in a user interface through an input device such as a remote control or a touch pad, the controller 250 receives a page access instruction from the user and can control the communication device 220 to send a data request instruction to the server, thereby displaying a media resource screen corresponding to the page loading data to the user through the page loading data returned by the server.

[0102] It is understandable that when the controller 250 receives the page loading data sent back by the server, it needs to decode the page loading data to obtain decoded data, and then display the media asset screen according to the decoded data. Therefore, the controller 250 can receive the page loading data through the native browser of the display device and decode the page loading data.

[0103] It should be noted that if Figure 6As shown, in some embodiments of the present application, a hard decoding preprocessing module is added based on the basic architecture of the browser, so that when the browser receives the page loading data, the hard decoding preprocessing module transmits the page loading data to the hard decoding interface, so as to process part of the data based on the hard decoding mode of the display device 200, thereby improving the page loading efficiency and maintaining the operating performance of the display device 200.

[0104] In this way, when the browser receives the page loading data sent back by the server, the controller 250 can first determine the decoding data format supported by the page loading data, and then determine which method to use to decode the page loading data.

[0105] S200: When controlling the communicator to receive the page loading data from the server, obtaining a decoding data format of the page loading data and a hard decoding format range of the display device.

[0106] In some embodiments, when receiving the page loading data, the controller 250 may obtain the decoded data format of the page loading data and the hard decoding format range supported by the display device 200. In this way, by comparing the decoded data format with the hard decoding format range, it can be determined whether the display device 200 can use the hard decoding mode to decode the page loading data.

[0107] S300: If the decoded data format is within the hard decoding format range, the page loading data is decoded based on a hard decoding mode to obtain first video decoding data.

[0108] In some embodiments, the decoded data format of the page loading data is within the hard decoding format range supported by the display device 200. The controller 250 can send the page loading data to the hard decoding interface through the hard decoding preprocessing module to decode the page loading data through the hard decoding interface to obtain the first video decoding data.

[0109] It should be noted that the hard decoding function integrated in the display device 200 is for page loading data within the hard decoding format range, and the page loading data can be processed by a pre-integrated hard decoding circuit, so there is no need to occupy additional system resources of the display device 200, and the decoding efficiency is high. Therefore, the efficiency of the display device 200 in decoding page loading data through the hard decoding mode is higher than the efficiency of decoding page loading data through the soft decoding mode.

[0110] S400: Control the display to display a media resource picture corresponding to the first video decoding data.

[0111] After obtaining the first video decoding data, the controller 250 can control the display to display the media asset screen corresponding to the first video decoding data. In some embodiments, when the user starts the cloud game application scenario, the CPU (Central Processing Unit) occupancy rate of the display device 200 needs to reach 75%, the page loading delay is 1-2 seconds, and the screen is frequently stuck before the cloud game scene is fully imported, and it takes 3-5 seconds to respond to the operation instructions of the input device such as the remote control and the handle.

[0112] In other embodiments, Figure 6 As shown, by adding a hard decoding preprocessing module to the decoding process of the display device 200, and adding a step of determining whether the decoding data format supported by the page loading data is within the hard decoding format range supported by the display device 200, the page loading data can be decoded based on the software and hardware collaboration. In the application scenario where the user starts the cloud game, the CPU occupancy rate of the display device 200 is reduced to 40%, the page loading delay is reduced to 120ms, and the operation instructions input by the user through the input device can be responded to without delay.

[0113] It is understandable that some embodiments of the present application still retain the method of decoding the page loading data by the browser in a soft decoding manner, such as Figure 5 As shown, in some embodiments, after the controller compares the decoded data format with the decoded format range, it also includes:

[0114] If the decoded data format is outside the decoding format range, the page loading data is decoded based on a soft decoding mode to obtain second video decoded data.

[0115] The display is controlled to display a media asset picture corresponding to the second video decoding data.

[0116] like Figure 6 As shown, the controller 250 can determine whether the page loading data supports the hardware decoding mode through a preset judgment interface, that is, determine whether the data decoding format of the page loading data is within the decoding format range. When the judgment interface determines that the data decoding format of the page loading data is outside the decoding format range, the controller 250 can decode the page loading data based on the soft decoding mode provided by the browser to obtain the second video decoding data.

[0117] It should be noted that the media asset screens corresponding to the first video decoding data and the second video decoding data are the same, and are both media asset screens corresponding to the page loading data returned by the server. Here, only "first video decoding data" and "second video decoding data" are used to distinguish the decoding data obtained by hard decoding mode and the decoding data obtained by soft decoding mode, but the type of decoding data and the content represented are not limited.

[0118] After obtaining the second video decoded data, the controller 250 can control the display 260 to display the media asset screen corresponding to the second video decoded data. In this way, the display device 200 can flexibly adjust the decoding method according to the decoded data format of the page loading data, thereby releasing system resources to improve the stability of the operation of the display device 200. In addition, when the page loading data supports the decoding method of hard decoding, the page loading efficiency can also be improved.

[0119] It should be noted that when the display device 200 decodes the page loading data based on the hard decoding mode, as the hard decoding process proceeds, part of the first video decoding data will be generated, and the local media image corresponding to the part of the first video decoding data will be displayed on the video layer, but this part of the media image is unclear. From the user's viewing perspective, it may manifest as a variety of conditions such as freezes and blurred images, so if it is seen by the user, it will affect the user experience.

[0120] Therefore, in some embodiments, the controller performs decoding of the page loading data based on the hard decoding mode to obtain the first video decoding data, and is further configured to:

[0121] Set up the video layer and graphics layer;

[0122] When the hard decoding interface performs hard decoding on the page loading data, the display is controlled to overlay the video layer and the graphic layer, and a partial media picture with an area smaller than the video layer is displayed in the video layer, and an occlusion picture is displayed in the graphic layer. The partial media picture is a media picture corresponding to a portion of the first video decoding data obtained through the hard decoding mode.

[0123] It should be noted that the hard decoding interface is a hardware module integrated in the CPU. The hardware module corresponding to the hard decoding interface can directly output the decoded data to the video layer for display, and through customized design of the hardware module, the page loading data can not only be decoded and processed during the decoding process, but also can be adaptively optimized, so it can be directly displayed in the video layer, thereby improving the page loading efficiency.

[0124] In some embodiments, the video layer is a display layer corresponding to the hard decoding interface, and the graphic layer is a display layer used to display the occluded image. Figure 7As shown, the display level of the graphic layer can be set higher than the display level of the video layer. And the display mode of the video layer and the graphic layer can be set to be superimposed display, so that when a partial media asset picture is displayed in the video layer, the occlusion picture of the graphic layer can occlude the partial media asset picture displayed in the video layer.

[0125] It should be noted that in some embodiments of the present application, by adding a hard decoding preprocessing module, the page loading data can be migrated to the hard decoding interface for decoding. In order to keep the overall operation architecture of the browser from being affected by the hard decoding preprocessing module, such as Figure 8 As shown, in some embodiments of the present application, a data consumption module is also provided.

[0126] In some embodiments, the controller performs decoding of the page loading data based on the hard decoding mode to obtain the first video decoding data, and is further configured to:

[0127] The page loading data is copied to obtain copied page data identical to the page loading data.

[0128] The copied page data is sent to a data consumption interface through a first process, so as to obtain a size parameter included in the copied page data through the data consumption interface.

[0129] The display size of the occlusion picture is set according to the size parameter; the display size of the occlusion picture is the same as the display size of the media asset picture corresponding to the first video decoding data.

[0130] In some embodiments, the first process may be a render process created by a browser. When the display device 200 requests page loading data from a server through the communication device 220, the controller 250 may create the first process through the browser. A WebRTC instance is further created through the first process to receive the page loading data sent back by the server and received by the communication device 220 through the WebRTC instance. The WebRTC instance then forwards the page loading data and performs other processing.

[0131] When processing page loading data, the WebRTC instance may copy the page loading data to obtain copied page data, and then send the copied page data to the data consumption interface, and then obtain the size parameter contained in the copied page data through the data consumption interface.

[0132] The data consumption interface may refer to a data consumption module created by a browser. The data consumption module may obtain information such as the data format and size parameters of the copied page data from the copied page data. The data consumption module may output data to the graphics layer, but the graphics layer is mainly used to display the occlusion screen to cover the partial media asset screen in the video layer. Therefore, the data consumption module only needs to perform packet loss processing on the received page loading data. In this way, the graphics layer can continue to serve as the occlusion screen in the form of a blank curtain to cover the partial media asset screen in the video layer.

[0133] In this way, the data consumption module can maintain the browser's operating architecture during the hard decoding process of the display device 200, so that the browser can still use the initial soft decoding control logic. It will not affect the browser's operating architecture, and will not affect other functions such as interface display and data decoding, which is conducive to maintaining the operating stability of the display device 200.

[0134] Furthermore, the copied page data and the page loading data are exactly the same data, so the information such as the media asset display size and data format contained in the copied page data are the same as the information contained in the page loading data. Furthermore, the WebRTC instance can control the display size of the blank screen in the graphics layer based on the size parameters obtained by the data consumption module. This makes the display size of the blank screen consistent with the display size of the media asset screen in the video layer, so as to achieve the occlusion of the video layer.

[0135] In some embodiments, the cloud game page accessed by the user is a small window page, so when the display device 200 requests page loading data from the server through the communication device 220, it can request page loading data that matches the small window page size from the server, so that the display size will also be included in the page loading data fed back by the server. Therefore, when the data consumption module receives the copied page data, it can obtain the display size contained in the copied page data, and the WebRTC instance can obtain the display size information contained in the copied page data from the data consumption module. In this way, the WebRTC instance can control the display size of the blank curtain in the graphics layer through the obtained display size information to achieve occlusion of the video layer.

[0136] By controlling the display size of the blank screen in the graphics layer, the problem of additional obstruction of other images in the user interface due to the large size of the blank screen in various playback situations such as small window playback and picture-in-picture playback can be avoided, which is beneficial to improving the user experience.

[0137] It is understandable that the copied page data is sent by the WebRTC instance to the data consumption module to control the display size of the blank screen in the graphics layer. The page loading data needs to be sent to the hard decoding interface to decode the page loading data by the hard decoding interface to obtain the first video decoding data.

[0138] In some embodiments, when the controller decodes the page loading data based on the hard decoding mode to obtain the first decoded data, it is specifically configured as follows:

[0139] The page loading data is sent to a hard decoding interface through a first process, so that the hard decoding interface performs hard decoding on the page loading data to obtain the first video decoding data.

[0140] The display is controlled to display a media asset picture corresponding to the first video decoding data in the video layer.

[0141] It is understandable that the page loading data can be sent to the hard decoding interface through the render process provided by the browser, so that the hard decoding interface can perform hard decoding on the page loading data. When the hard decoding interface receives the page loading data, it can immediately perform hard decoding on the page loading data to obtain the first video decoding data. While obtaining the first video decoding data, the hard decoding interface can output the first video decoding data to the video layer. After the hard decoding interface completely decodes the page loading data, the controller 250 can control the display 260 to display the complete media asset screen corresponding to the first video decoding data on the video layer.

[0142] It should be noted that the render process provided by the browser has a lower scope of authority, and the hard decoding interface belongs to the hardware module, which needs to be called by the Browser process provided by the browser. Therefore, in some scenarios, it is difficult for the render process to directly transfer the page loading data to the hard decoding interface, and the Browser process needs to migrate the page loading data to the hard decoding interface.

[0143] like Fig. 9 As shown, in some embodiments, the controller executes the first process to send the page loading data to the hard decoding interface, which is specifically configured as follows:

[0144] Obtaining a first communication identifier of the first process and a second communication identifier of the second process;

[0145] Based on the communication connection relationship determined by the first communication identifier and the second communication identifier, sending the page loading data to the second process through the first process;

[0146] The page loading data is sent to the hard decoding interface through the second process, so that the hard decoding interface decodes the page loading data.

[0147] In some embodiments, the first process refers to the render process provided by the browser, and the second process refers to the browser process provided by the browser. The render process supports module calls and data transmission related to the soft decoding function. The browser process has a higher authority than the render process and supports the call of hardware modules and data transmission. Therefore, when calling the hard decoding interface, it is necessary to regulate the page loading data and the hard decoding interface based on the browser process.

[0148] It should be noted that the page loading data is received by the first process through the WebRTC instance, so the page loading data needs to be migrated from the first process to the second process. In some embodiments of the present application, a hard decoding preprocessing module is added to realize the process of migrating the page loading data from the first process to the second process.

[0149] In some embodiments, when the WebRTC instance inputs the copied page data into the data consumption module, it can also input the page loading data into the hard decoding preprocessing module. The hard decoding preprocessing module can migrate the page loading data from the render process to the browser process based on the mojo cross-process communication method.

[0150] Among them, the hard decoding preprocessing module can determine the communication connection relationship between the Browser process and the render process based on the communication identifiers of the two processes, thereby migrating the page loading data from the render process to the Browser process.

[0151] In this way, the controller 250 can further call the hard decoding interface through the Browser process to perform hard decoding processing on the page loading data.

[0152] It should be noted that the specific method of cross-process communication based on mojo is not the focus of the description of the embodiments of this application, and other cross-process communication methods can also be applied to the embodiments of this application. The description in the embodiments of this application focuses on migrating the page loading data from the process with a lower authority range to the process with a higher authority corresponding to the hard decoding module in this way, so that the hard decoding module can decode the page loading data.

[0153] After the controller 250 transfers the page loading data from the render process to the browser process, the hard decoding interface can perform hard decoding processing on the page loading data in the browser process and output the first video decoding data to the video layer. During the decoding process, the video layer is blocked by the blank curtain provided by the graphics layer. When the decoding is completed, the media asset screen in the video layer can be displayed by making the graphics layer transparent.

[0154] like Fig.10 As shown, in some embodiments, when the controller controls the display to display the media asset picture corresponding to the first video decoding data, the controller is configured to:

[0155] When the hard decoding interface completes hard decoding of the page loading data, an overlapping area of ​​the video layer and the graphic layer is determined according to display sizes of the video layer and the graphic layer.

[0156] A transparency process is performed on a portion of the graphic layer corresponding to the overlapping area to display the media asset picture displayed in the video layer.

[0157] It can be understood that the overlapping area of ​​the video layer and the graphics layer is determined according to the display size of the video layer and the graphics layer, that is, the overlapping area is determined according to the display size of the blank screen in the graphics layer and the display size of the media screen in the video layer. After the WebRTC instance inputs the copied page data into the data consumption module, the WebRTC instance can obtain the display size of the blank screen from the data small module, and then determine

[0158] Because the display level of the graphic layer is higher than that of the video layer, the media asset picture corresponding to the first video decoding data in the video layer can be displayed by only making the graphic layer transparent.

[0159] It should be noted that the video layer also corresponds to the hardware display module integrated in the controller 250, so the first video decoded data obtained by the hardware decoding module can be directly input into the hardware display module, and the video layer displays the media asset screen corresponding to the first video decoded data. Based on the display mode of the video layer, the occupancy rate of the graphics layer can also be alleviated, thereby saving system resources and improving display efficiency.

[0160] In application scenarios such as cloud gaming, the target page accessed by the user includes both the media asset screen and the playback audio corresponding to the media asset screen. Therefore, the page loading data includes not only video data but also audio data. In this way, the controller 250 can display the media asset screen based on the video data and play the audio corresponding to the media asset screen based on the audio data.

[0161] In some embodiments, the display device 200 further includes an audio player coupled to the controller, and the page loading data includes audio encoding data. The controller is further configured to:

[0162] The audio encoding data is sent to the audio decoding interface through the first process to obtain audio decoding data.

[0163] Based on the time stamps of the audio decoded data and the video decoded data, when controlling the display to display the media asset picture corresponding to the first video decoded data, the audio player is controlled to play the audio data corresponding to the time stamp of the media asset picture among all the audio data corresponding to the audio decoded data.

[0164] It should be noted that the audio encoding data is the audio data contained in the page loading data, which needs to be decoded by the display device 200 before it can be played through the music player. The audio data consumes very few resources during the decoding process, so the audio data can be decoded by soft decoding. However, when the media asset picture corresponding to the first video decoded data is finally played, the corresponding relationship between the audio data and the media asset picture needs to be ensured.

[0165] In some embodiments, both the audio data and the video data carry a time stamp, and the audio decoded data and the video decoded data obtained by decoding also carry a time stamp. In this way, when the controller controls the video layer to play the media data, it can align the audio decoded data and the video decoded data according to the time stamp in advance, and then play the media data. In this way, while playing the media picture, the audio data corresponding to the media picture can also be played synchronously to prevent the problem of reduced user experience due to asynchrony between audio and video.

[0166] It should be noted that, since the hard decoding module is a circuit designed based on a specific data format, its decoding process includes optimization processing of data, and the hard decoding module and the hardware display module can be directly connected, the hardware display module can display the media asset picture corresponding to the decoded data in the video layer according to the decoding order of the hard decoding module. However, when the display device 200 decodes data through the soft decoding function, it is necessary to optimize the graphics and render the page based on the optimized data to obtain the media asset picture corresponding to the decoded data.

[0167] In some embodiments, when the controller performs rendering of a page based on the second video decoding data rendering band, the controller is specifically configured to:

[0168] When the page loading data is decoded based on the soft decoding mode, a media asset picture is rendered based on the second video decoding data through a first process.

[0169] The display is controlled to play a media resource image rendered by the second video decoding data.

[0170] It is understandable that the controller 250 can perform the soft decoding function through the first process, that is, through the render process. That is, when the WebRTC instance created by the render process receives the page loading data, the soft decoding function can be called to decode the page loading data to obtain the second video decoding data.

[0171] After obtaining the second video decoded data, the render process will create a graphics rendering module, and render and play the second video decoded data through the graphics rendering module. In this way, soft decoding and playing the media asset screen corresponding to the page loading data can be performed by the render process. Even if the display device 200 cannot hard decode the page loading data, the data can be decoded through the soft decoding function provided by the browser, so that when the user enters the page access instruction, the media asset screen can be displayed to the user in time.

[0172] Some embodiments of the present application also provide a page loading method, including:

[0173] In response to the received page access instruction, sending a data request instruction to the server, wherein the data request instruction is used to obtain page loading data;

[0174] When receiving the page loading data from the server, obtaining a decoding data format of the page loading data and a hard decoding format range of a display device;

[0175] If the decoded data format is within the hard decoding format range, decoding the page loading data based on the hard decoding mode to obtain first video decoding data;

[0176] A media asset screen corresponding to the first video decoding data is displayed.

[0177] It can be seen from the above technical content that the present application provides a display device and a page loading method. When the display device receives the data to be decoded sent back by the server, it first determines whether the decoded data format of the data to be decoded is within the hard decoding format range of the display device. When the decoded data format is within the hard decoding format range, the page loading data is decoded based on the hard decoding mode to obtain the first video decoding data. And the display is controlled to display the media resource screen corresponding to the first video decoding data. By reasonably calling the hard decoding function of the display device, the pressure on the processor and memory resources can be relieved, so that the display device can maintain a high performance during the page loading process and improve the page loading speed.

Claims

1. A display device, characterized in that: include: monitor; a communicator configured to communicate with the server; A controller is coupled to the display and the communicator and is configured to: In response to the received page access instruction, controlling the communicator to send a data request instruction to the server, wherein the data request instruction is used to obtain page loading data; When controlling the communicator to receive the page loading data from the server, acquiring the decoding data format of the page loading data and the hard decoding format range of the display device; If the decoded data format is within the hard decoding format range, decoding the page loading data based on the hard decoding mode to obtain first video decoding data; The display is controlled to display a media asset picture corresponding to the first video decoding data.

2. The display device according to claim 1, characterized in that After the controller compares the decoded data format with the decoded format range, the controller further includes: If the decoded data format is outside the decoding format range, decoding the page loading data based on a soft decoding mode to obtain second video decoded data; The display is controlled to display a media asset picture corresponding to the second video decoding data.

3. The display device according to claim 1, characterized in that The controller performs decoding of the page loading data based on the hard decoding mode to obtain first video decoding data, and is further configured to: Set up the video layer and graphics layer; When the hard decoding interface performs hard decoding on the page loading data, the display is controlled to superimpose the video layer and the graphic layer, and a local media picture with an area smaller than the video layer is displayed in the video layer, and an occluded picture is displayed in the graphic layer; the display level of the graphic layer is higher than the display level of the video layer; the local media picture is a media picture corresponding to part of the first video decoding data obtained through the hard decoding mode.

4. The display device according to claim 3, characterized in that The controller performs decoding of the page loading data based on the hard decoding mode to obtain first video decoding data, and is further configured to: Copying the page loading data to obtain copied page data identical to the page loading data; Sending the copied page data to a data consumption interface through a first process, so as to obtain a size parameter included in the copied page data through the data consumption interface; The display size of the occlusion picture is set according to the size parameter; the display size of the occlusion picture is the same as the display size of the media asset picture corresponding to the first video decoding data.

5. The display device according to claim 3, characterized in that When the controller decodes the page loading data based on the hard decoding mode to obtain the first decoded data, the controller is specifically configured as follows: Sending the page loading data to a hard decoding interface through a first process, so that the hard decoding interface performs hard decoding on the page loading data to obtain the first video decoding data; The display is controlled to display a media asset picture corresponding to the first video decoding data in the video layer.

6. The display device according to claim 5, characterized in that The controller executes sending the page loading data to the hard decoding interface through the first process, and is specifically configured as follows: Obtaining a first communication identifier of the first process and a second communication identifier of the second process; Based on the communication connection relationship determined by the first communication identifier and the second communication identifier, the page loading data is sent to the second process through the first process; the first process is associated with the soft decoding mode; the second process is associated with the hard decoding mode; The page loading data is sent to the hard decoding interface through the second process, so that the hard decoding interface decodes the page loading data.

7. The display device according to claim 3, characterized in that When the controller controls the display to display a media asset picture corresponding to the first video decoding data, the controller is configured to: When the hard decoding interface completes hard decoding of the page loading data, determining an overlapping area of ​​the video layer and the graphics layer according to display sizes of the video layer and the graphics layer; A transparency process is performed on a portion of the graphic layer corresponding to the overlapping area to display the media asset picture displayed in the video layer.

8. The display device according to claim 1, characterized in that Also included is an audio player coupled to the controller, wherein the page loading data includes audio encoding data; and the controller is further configured to: Sending the audio encoding data to the audio decoding interface through the first process to obtain audio decoding data; Based on the time stamps of the audio decoded data and the video decoded data, when controlling the display to display the media asset picture corresponding to the first video decoded data, the audio player is controlled to play the audio data corresponding to the time stamp of the media asset picture among all the audio data corresponding to the audio decoded data.

9. The display device according to claim 2, characterized in that: When the controller executes rendering of a page based on the second video decoding data rendering band, the controller is specifically configured as follows: When the page loading data is decoded based on the soft decoding mode, a media asset picture is rendered based on the second video decoding data through a first process; The display is controlled to play a media resource image rendered by the second video decoding data.

10. A page loading method, characterized in that: The display device according to any one of claims 1 to 9 comprises: In response to the received page access instruction, sending a data request instruction to the server, wherein the data request instruction is used to obtain page loading data; When receiving the page loading data from the server, obtaining a decoding data format of the page loading data and a hard decoding format range of a display device; If the decoded data format is within the hard decoding format range, decoding the page loading data based on the hard decoding mode to obtain first video decoding data; A media asset screen corresponding to the first video decoding data is displayed.

Images