Display device and method for improving playing speed of media assets

Abstract

The embodiment of the invention provides a display device and a method for improving the playing starting speed of media assets, and the method comprises the steps: responding to a touch instruction of a user for a playing control corresponding to the media asset data, executing silent playing starting processing on the media asset data, and caching the media asset data; after the rendering of the first video frame of the media asset data is completed and the cache of the media asset data reaches a threshold storage amount, generating a playing instruction of the media asset data; and in response to the playing instruction, controlling the display to play the media asset data according to the first video frame. According to the method, by introducing a mechanism of silent play starting processing and parallel caching of the media asset data, operations such as play pipeline creation, decoder creation, decoding and first frame rendering are completed in advance, so that the waiting time of a user from click play to picture appearance is shortened, the play starting speed of the streaming media application to the media asset data is optimized, and the user experience is improved. The blank screen time in the playing starting process is shortened, and the problem that the playing starting speed is low in the media asset data playing process of the streaming media application at present is solved.

Description

Technical Field

[0001] This application relates to the technical field of display devices, and particularly to a display device and a method for improving the start-up speed of media assets. Background Art

[0002] In order to play media asset data, a display device may deploy multiple streaming media applications, and the start-up performance of the streaming media applications will directly affect the user's viewing experience. For example, a fast start-up time can reduce the user's waiting time and thus improve the user's viewing experience.

[0003] In the process of the start-up process of the streaming media application, the time-consuming links are mainly reflected in two parts. The first part of the time-consuming is the time from when the underlying SDK is told that it can start the start-up to when the SDK officially issues the start-up command; the second part of the time-consuming is the time from when the start-up command is issued to when the video starts playing. For the first part of the time-consuming, the issuance of the start-up command depends on the network condition, and the SDK will only issue the start-up command after the downloaded buffer data is sufficient, and this process is difficult to optimize. For the second part of the time-consuming, the video start-playing time after the start-up command is issued mainly depends on the implementation of the streaming media playback framework, including the creation of the playback pipeline, the creation of the decoder, the decoding of the decoder, the rendering of video frames, etc. These steps are particularly time-consuming during the first play, resulting in a slow start-up speed and an obvious black screen phenomenon during the start-up process, affecting the user's visual experience.

[0004] Therefore, currently, there is a problem of slow start-up speed in the process of streaming media applications playing media asset data. Summary of the Invention

[0005] This application provides a display device and a method for improving the start-up speed of media assets to solve the problem of slow start-up speed in the process of current streaming media applications playing media asset data.

[0006] In a first aspect, some embodiments of this application provide a display device, including.

[0007] A display configured to display a user interface;

[0008] A controller configured to:

[0009] In response to a touch instruction of a play control corresponding to the media asset data, perform a silent start-up process on the media asset data and cache the media asset data; the silent start-up process of the media asset data is executed in parallel with the caching of the media asset data; the silent start-up process is executed under the condition that the player does not display a picture and the sound is in a mute state;

[0010] After the first frame of the video frame of the media asset data is rendered, and after the cache of the media asset data reaches the threshold storage amount, generate a play instruction for the media asset data;

[0011] In response to the playback instruction, control the display to play the media data according to the first video frame.

[0012] The above technical solution has the following advantages or beneficial effects: By introducing the mechanisms of silent startup processing and parallel caching of media data, the display device completes operations such as creating a playback pipeline, creating a decoder, decoding, and rendering the first frame in advance, thereby reducing the waiting time between the user clicking play and the picture appearing, optimizing the startup speed of the media data in the streaming media application, reducing the black screen time during startup, and solving the problem of slow startup speed in the current streaming media application when playing media data.

[0013] In some embodiments, the controller performs silent startup processing on the media data, and is specifically configured to:

[0014] Create a playback pipeline and a decoder for the media data;

[0015] After receiving the video data including the first video frame of the media data in the playback pipeline, perform decoding on the video data through the decoder;

[0016] Render the decoded video data under the condition that the player does not display the picture;

[0017] Generate the first video frame of the media data according to the rendered video data.

[0018] The above technical solution has the following advantages or beneficial effects: By completing operations such as creating a playback pipeline, creating a decoder, decoding, and rendering the first frame in advance in the background, when the user triggers the playback instruction, the display device can immediately switch to the normal playback state and display the first frame. This significantly reduces the black screen time during startup, and the user hardly feels the delay between clicking play and the picture appearing. Since the silent startup has completed the media data preparation work, the execution of the playback instruction hardly requires additional waiting time. Therefore, the silent startup processing can not only improve the startup speed of the media data, but also lay a foundation for the subsequent playback smoothness.

[0019] In some embodiments, after the step of the controller creating a playback pipeline and a decoder for the media data, it is further configured to:

[0020] During the silent startup processing, detect the creation status of the playback pipeline or the decoder;

[0021] If it is detected that the creation status of the playback pipeline or the decoder is creation failure, terminate the silent startup processing and re-initialize the playback pipeline or the decoder;

[0022] After the playback pipeline or the decoder is successfully re-initialized, continue to perform the silent startup process.

[0023] The above technical solution has the following advantages or beneficial effects: By adding an error handling mechanism during the silent startup process, the robustness of the startup process can be ensured, enabling the media asset data to be played normally.

[0024] In some embodiments, after the step of the controller generating the first video frame of the media asset data based on the rendered video data, it is further configured to:

[0025] During the silent startup process, pre-load the playback pipeline resources, decoder resources, and rendering resources of the media asset data;

[0026] After the first video frame of the media asset data is rendered, call the pre-loaded playback pipeline resources, decoder resources, and rendering resources to play the media asset data.

[0027] The above technical solution has the following advantages or beneficial effects: By pre-loading resources during the silent startup process, the display device can reduce the resource initialization time during startup, thereby improving the startup speed of the media asset data.

[0028] In some embodiments, before the controller controls the display to play the media asset data according to the first video frame, it is further configured to:

[0029] Perform unmute processing on the media asset data after the silent startup process;

[0030] Control the display to display the media asset screen corresponding to the media asset data.

[0031] The above technical solution has the following advantages or beneficial effects: Before playing the media asset data, the display device needs to perform unmute processing on the media asset data to ensure that the audio can be played normally. At the same time, the display device can control the display to display the media asset screen corresponding to the media asset data, enabling the user to see the video content to ensure that the video and audio can be presented to the user synchronously and smoothly.

[0032] In some embodiments, the controller caches the media asset data, specifically configured to:

[0033] Set a caching policy according to the network bandwidth information and the total storage amount of the media asset data;

[0034] Receive the audio data and video data in the media asset data based on the caching policy;

[0035] Save the audio data and the video data in segments.

[0036] The above technical solution has the following advantages or beneficial effects: The caching mechanism for caching media asset data can dynamically adapt to network changes. Under different network conditions, the display device can adjust the caching focus to ensure smooth playback. Saving the audio data and video data in segments by the display device facilitates the management and rapid retrieval of media asset data.

[0037] Before the controller performs the silent start-up processing on the media asset data, it is further configured to:

[0038] Create a player instance for the media asset data;

[0039] Configure playback parameters for the player instance.

[0040] The above technical solution has the following advantages or beneficial effects: By pre-creating a player instance and configuring parameters, the display device can ensure that the player is already in a ready state during silent start-up, so as to avoid delays or lags caused by player initialization and parameter settings during playback, thereby improving the stability and smoothness of playback.

[0041] Before the controller generates a playback instruction for the media asset data, it is further configured to:

[0042] Monitor the storage amount of the media asset data cache;

[0043] When the storage amount reaches the threshold storage amount, generate an instruction generation packet; the instruction generation packet is used to trigger the generation of the playback instruction.

[0044] The above technical solution has the following advantages or beneficial effects: By monitoring the cache storage amount and triggering playback when the threshold is reached, the display device can ensure that sufficient data has been cached at the start of playback, so as to avoid playback lags or interruptions caused by insufficient cache, thereby improving the smoothness of playback.

[0045] In some embodiments, the controller controls the display to play the media asset data according to the first video frame, and is specifically configured to:

[0046] Obtain the first video frame;

[0047] Perform decoding processing on the first video frame;

[0048] Send the decoded first video frame to the display;

[0049] Control the display to play the media asset data according to the first video frame.

[0050] The above technical solution has the following advantages or beneficial effects: By acquiring and decoding the first video frame, the display device can present the video picture to the user in the shortest time, reduce the waiting time for the user for the video to start, improve the response speed of media asset startup, and solve the problem of slow startup speed in the process of playing media asset data in current streaming media applications.

[0051] In a second aspect, some embodiments of the present application provide a method for improving the startup speed of media assets, the method comprising:

[0052] In response to a touch instruction of a playback control corresponding to media asset data, perform a silent startup process on the media asset data and cache the media asset data; the silent startup process of the media asset data is executed in parallel with the caching of the media asset data; the silent startup process is executed under the condition that the player does not display a picture and the sound is in a mute state;

[0053] After the first video frame of the media asset data is rendered, and after the cache of the media asset data reaches the threshold storage amount, generate a playback instruction for the media asset data;

[0054] In response to the playback instruction, control the display to play the media asset data according to the first video frame.

[0055] The above technical solution has the following advantages or beneficial effects: By introducing a mechanism of silent startup processing and parallel caching of media asset data, operations such as creating a playback pipeline, creating a decoder, decoding, and rendering the first frame are completed in advance, thereby reducing the waiting time between the user clicking play and the picture appearing, optimizing the startup speed of the streaming media application for media asset data, reducing the black screen time during startup, and solving the problem of slow startup speed in the process of playing media asset data in current streaming media applications.

[0056] As can be seen from the above technical solutions, some embodiments of the present application provide a display device and a method for improving the start-up speed of media assets. The method includes: in response to a touch instruction on a play control corresponding to media asset data, performing a silent start-up process on the media asset data and caching the media asset data; the silent start-up process of the media asset data and the caching of the media asset data are executed in parallel; the silent start-up process is executed under the condition that the player does not display a picture and the sound is in a mute state; after the first video frame of the media asset data is rendered, and after the cache of the media asset data reaches the threshold storage amount, generating a play instruction for the media asset data; in response to the play instruction, controlling the display to play the media asset data according to the first video frame. The method reduces the waiting time between the user clicking play and the picture appearing by introducing a mechanism of silent start-up process and parallel caching of media asset data, optimizes the start-up speed of the media asset data by the streaming media application, reduces the black screen time during the start-up process, and solves the problem of slow start-up speed of the current streaming media application when playing media asset data. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0058] Figure 1 It is a schematic diagram of the operation scenario between the display device and the control device provided by some embodiments of the present application;

[0059] Figure 2 It is a schematic diagram of the hardware configuration of the display device provided by some embodiments of the present application;

[0060] Figure 3 It is a schematic diagram of the software configuration of the display device provided by some embodiments of the present application;

[0061] Figure 4 It is a schematic diagram of the start-up process link of the streaming media application shown by some embodiments of the present application;

[0062] Figure 5 It is a schematic diagram of the method flow for the display device to execute the method for improving the start-up speed of media assets provided by some embodiments of the present application;

[0063] Figure 6 It is a comparison schematic diagram of the traditional start-up process of the streaming media application and the start-up process of the embodiments of the present application provided by some embodiments of the present application;

[0064] Figure 7Schematic diagram of the timing of the method for a display device provided in some embodiments of the present application to improve the start-up speed of media assets;

[0065] Figure 8 Schematic diagram of the process for a display device provided in some embodiments of the present application to perform silent start-up processing;

[0066] Figure 9 Schematic diagram of the process for a display device provided in some embodiments of the present application to cache media asset data;

[0067] Figure 10 Schematic diagram of the process for a display device provided in some embodiments of the present application to play media asset data according to the first video frame. Detailed implementation manners

[0068] The embodiments will be described in detail below, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following embodiments do not represent all implementation manners consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application detailed in the claims.

[0069] It should be noted that the brief description of the terms in the present application is only for facilitating the understanding of the following described implementation manners, rather than intending to limit the implementation manners of the present application. Unless otherwise specified, these terms should be understood in their ordinary and common meanings.

[0070] The terms "first", "second", "third", etc. in the specification, claims and the above drawings of the present application are used to distinguish similar or like objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that such terms can be interchanged under appropriate circumstances.

[0071] The terms "comprising" and "having" and any variations thereof are intended to cover but not be exclusive of inclusion. For example, a product or device comprising a series of components does not necessarily have to be limited to all the components clearly listed, but may include other components not clearly listed or inherent to these products or devices.

[0072] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or a combination of hardware or / and software code that can perform the functions related to the element.

[0073] In the embodiments of the present application, the display device 200 generally refers to a device with the capabilities of displaying images and processing data. For example, the display device 200 includes, but is not limited to, smart TVs, mobile terminals, computers, monitors, advertising screens, wearable devices, virtual reality devices, augmented reality devices, etc.

[0074] Figure 1 A schematic diagram of an operation scenario between a display device and a control device provided for some embodiments of the present application. As Figure 1 shown, a user can operate the display device 200 through touch operations, a mobile terminal 300, and a control device 100. Among them, the control device 100 is used to receive operation instructions input by the user and convert the operation instructions into control instructions recognizable and responsive by the display device 200. For example, the control device 100 can be a remote control, a stylus, a gamepad, etc.

[0075] The mobile terminal 300 can be used as a control device to perform human-computer interaction between the user and the display device 200. The mobile terminal 300 can also be used as a communication device to establish a communication connection with the display device 200 for data interaction. In some embodiments, software applications can be installed on the mobile terminal 300 and the display device 200, and connection communication can be achieved through network communication protocols to achieve the purpose of one-to-one control operations and data communication. It is also possible to transmit the audio and video content displayed on the mobile terminal 300 to the display device 200 to achieve a synchronous display function.

[0076] In some embodiments, the mobile terminal 300 or other electronic devices can also simulate the function of the control device 100 by running an application program for controlling the display device 200.

[0077] As Figure 1 also shown, the display device 200 also communicates with the server 400 for data communication through various communication methods. The display device 200 is allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0078] The display device 200 can provide a broadcast receiving television function, and can also additionally provide an intelligent network television function with computer support functions, including but not limited to, network television, smart television, Internet Protocol Television (IPTV), etc.

[0079] Figure 2 Provided for some embodiments of the present application Figure 1 The hardware configuration block diagram of the display device 200 in

[0080] In some embodiments, the display device 200 may include at least one of a tuner demodulator 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0081] In some embodiments, the detector 230 is configured to collect signals of the external environment or for external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the ambient light intensity; alternatively, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes, or user interaction gestures. Or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0082] In some embodiments, the display 260 includes a display function component for presenting a picture and a driving component for driving image display. The display 260 is configured to receive an image signal output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, components of a menu manipulation interface, and a user manipulation UI interface, etc.

[0083] In some embodiments, the communication device 220 is a component for communicating with an external device or server 400 according to various communication protocol types. The display device 200 can be provided with a plurality of communication devices 220 according to different supported communication methods. For example, when the display device 200 supports wireless network communication, the display device 200 can be provided with a communication device 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including a Bluetooth function.

[0084] The communication device 220 can enable the display device 200 to communicate with an external device or server 400 through a wireless or wired connection. Among them, the wired connection can connect the display device 200 with an external device through components such as a data cable and an interface. The wireless connection can connect the display device 200 with an external device through a wireless signal or a wireless network. The display device 200 can directly establish a connection relationship with an external device, or can indirectly establish a connection relationship through a gateway, a router, a connection device, etc.

[0085] In some embodiments, the controller 250 can include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and first to n interfaces for input / output. The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0086] In some embodiments, the controller 250 and the tuner demodulator 210 can be located in different split devices, that is, the tuner demodulator 210 can also be in an external device of the main device where the controller 250 is located, such as an external set-top box, etc.

[0087] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0088] In some embodiments, the audio output device 270 may be a local speaker of the display device 200, or may be an external audio output device of the display device 200. In particular, for the external audio output device of the display device 200, the display device 200 may also be provided with an external audio output terminal, and the audio output device may be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0089] In some embodiments, the user input interface 280 may be used to receive instructions from a user.

[0090] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling hardware resources and software resources in the display device 200. The operating system can control the display device to provide a user interface, for example, the operating system can directly control the display device to provide a user interface, or can provide a user interface by running an application program. The operating system also allows the user to interact with the display device 200.

[0091] It should be noted that the operating system may be a native operating system based on a specific operating platform, or a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for a display device.

[0092] The operating system can be divided into different modules or layers according to the functions implemented, such as Figure 3 As shown, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications layer for short), the middleware layer, and the hardware layer.

[0093] In some embodiments, the application layer is used to provide services and interfaces for applications so that the display device 200 can run applications and interact with users based on the applications. At least one application can be run in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. that come with the operating system; they can also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0094] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0095] like Figure 3 As shown, Figure 3 Some embodiments of the present application provide Figure 1 Schematic diagram of software configuration in the display device. In some embodiments, the system of the display device 200 can be divided into three layers, namely, application layer, middleware layer and hardware layer from top to bottom.

[0096] The application layer mainly includes commonly used applications on TV and application frameworks. Common applications are mainly applications developed based on browsers, such as HTML5 APPs and native APPs.

[0097] The Application Framework is a complete program model that has all the basic functions required by standard application software, such as file access, data exchange, etc., as well as the user interfaces of these functions (toolbars, status bars, menus, dialog boxes).

[0098] Native apps can support online or offline, message push or local resource access.

[0099] The middleware layer includes various TV protocols, multimedia protocols, system components and other middleware. The middleware can use the basic services (functions) provided by the system software to connect various parts of the application system or different applications on the network, and can achieve the purpose of resource sharing and function sharing.

[0100] The hardware layer mainly includes the hardware abstraction layer (HAL) interface, hardware and drivers. The HAL interface is a unified interface for all TV chips to connect, and the specific logic is implemented by each chip. Drivers mainly include: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver.

[0101] It should be noted that the above examples are only simple divisions of the functions of the operating system, and do not limit the specific form of the operating system of the display device 200 in the embodiments of the present application. Depending on factors such as the functions of the display device and the type of the operating system, the number of levels and the specific level types included in the operating system may be presented in other forms.

[0102] The display device 200 is an intelligent device that can present a user interface and support user interaction. Taking a smart TV as an example, a smart TV is a TV product based on Internet application technology, with an open operating system and a chip, an open application platform, and can realize two-way human-computer interaction functions, integrating various functions such as audio-visual, entertainment, and data, to meet the diverse and personalized needs of users. The display device can present different media recommendation interfaces, and the media recommendation interfaces can include multiple media options for users to select and play.

[0103] In some embodiments, the display device 200 may be deployed with multiple streaming media applications, and the start-up performance of the streaming media applications will directly affect the user's viewing experience. For example, a fast start-up time can reduce the user's waiting time, thereby improving the user's viewing experience.

[0104] Figure 4 Schematic diagram of the start-up process of a streaming media application shown in some embodiments of the present application, as Figure 4 shown, in some embodiments, in order to implement the start-up process of the streaming media application, an SDK JS layer, an SDK device docking layer such as MPB, an implementation of the application at the device layer such as HSMediaBackend, and an implementation of the application at the multimedia framework layer such as HmcMediaPlayer may be deployed in the display device 200. Among them, the SDKjs layer is the JavaScript layer of the streaming media application SDK, mainly responsible for processing logic and commands related to playback, such as playback control, data request, etc. The SDK device docking layer MPB is the docking layer between the SDK and the device layer, responsible for converting the commands of the SDK into instructions that the device layer can understand, interacting with the device layer, and coordinating the operations of the device layer; the implementation of the application at the device layer, HSMediaBackend, is responsible for implementing specific device-related media functions, such as video decoding, audio processing, etc.; the implementation of the application at the multimedia framework layer, HmcMediaPlayer, is responsible for the overall process control and coordination of multimedia playback, including player initialization, resource preparation, playback operation execution, etc.

[0105] Figure 4Describes the various steps from system initialization to the execution of a play command. The entire process can include an initialization phase, a data caching phase, a play command execution phase, and a video display phase. Among them, the initialization phase can include initializing media asset playback, processing audio / video metadata, opening media streams, etc. The data caching phase can include starting the audio / video stream phase, processing audio / video access units, pushing ES (Elementary Stream) data, waiting for the first packet of audio and video data, preparing callbacks for playback, receiving the event of preparation completion, etc.; the play command execution phase can include sending a play command, actual playback operations, receiving the event of display start, callbacks for playback start, etc.; the video display phase can include video frame display, etc. Throughout the process, the time-consuming links are mainly reflected in Figure 4 the two parts indicated by the dashed lines. The time consumption of the first part is the time from when the underlying layer tells the SDK that it can start playing to when the SDK officially sends the start-play command, that is, Figure 4 the time consumed for caching data before the SDK issues the play command as shown in one of the dashed boxes in Figure 4 The time consumption of the second part is the time from the issuance of the start-play command to the video start-up, that is,

[0106] In some embodiments, for the time consumption of the first part, the issuance of the start-play command depends on the network condition. The SDK will only issue the start-play command after sufficient download buffer data. This process is difficult to optimize. For the time consumption of the second part, the video start-up time after the issuance of the start-play command mainly depends on the implementation of the streaming media playback framework, including play pipeline creation, decoder creation, decoder decoding, video frame rendering, etc. These steps are particularly time-consuming during the first playback, resulting in a slow start-play speed and an obvious black screen phenomenon during the start-play process, affecting the user's visual experience. For example, the time from the issuance of the start-play command to the video start-up, the implementation logic is at the device layer. This part mainly depends on the implementation of the streaming media playback framework and also depends on the chip and hardware capabilities. During the first playback, since it is necessary to initialize the playback and start the underlying solution provider's decoder (such as the time from the issuance of the start-play command to the video start-up, the implementation logic is at the device layer. This part mainly depends on the implementation of the streaming media playback framework and also depends on the chip and hardware capabilities. During the first playback, since it is necessary to initialize the playback and start the underlying solution provider's decoder), the time from the issuance of the start-play command to the video start-up is relatively long.

[0107] Therefore, currently, there is a problem of slow start-play speed in the process of streaming media applications playing media asset data.

[0108] Based on the above scenario, some embodiments of the present application provide a display device 200, which includes a display 260 and a controller 250. Among them, the display 260 is configured to display a user interface, and the controller 250 enables the display device 200 to execute a method for improving the start-up speed of media assets by running specific computer execution instructions. The display device 200 adds a silent start-up process, and starts up in an implicit manner in advance before the actual play instruction corresponding to the media asset data. The play resources are pulled up during the buffering process of the media asset data, and the buffering of the media asset data and the preparation for playing the media resources are executed in parallel, making full use of the time gap during the buffering of the media asset data to execute the play preparation process. In this way, when the play instruction is specifically executed, since silent start-up such as first-frame rendering has been performed in advance, only the resume play operation and the opening need to be performed, thereby improving the start-up speed of the media asset data, reducing the start-up time and the black screen duration during start-up, and enhancing the user experience.

[0109] To facilitate the understanding of the technical solutions in some embodiments of the present application, the following will describe each step in detail with reference to some specific embodiments and the accompanying drawings. Figure 5 Schematic diagram of the method for a display device provided in some embodiments of the present application to improve the start-up speed of media assets, as Figure 5 shown, in some embodiments, when the display device 200 executes the method for improving the start-up speed of media assets, it may include the following steps:

[0110] Step S1: In response to a touch instruction of a user on a play control corresponding to media asset data, the display device 200 performs a silent start-up process on the media asset data and caches the media asset data.

[0111] In some embodiments, before the display device 200 performs a silent start-up process on the media asset data, it may create a player instance for the media asset data and configure play parameters for the player instance.

[0112] Exemplarily, before the display device 200 performs a silent start-up process, it may create a player instance for the media asset data. This player instance will serve as the core object for subsequent play operations and be responsible for managing the decoding and rendering of audio and video data. After the player instance is created, the display device 200 may configure a series of play parameters for it to adapt to different play requirements and environments. For example, the play parameters may include volume, play rate, screen display mode (such as full screen or small window), buffering policy, etc. By pre-creating a player instance and configuring parameters, the display device 200 can ensure that the player is in a ready state during silent start-up, avoiding delays or lags caused by player initialization and parameter setting during play, thereby improving the stability and smoothness of play.

[0113] In some embodiments, when a user performs a touch operation on a playback control corresponding to media asset data through a user interface (UI) of the display device 200, the display device 200 can respond to the touch instruction, immediately initiate a silent start-up process for the media asset data, and simultaneously start caching the media asset data. The silent start-up process and the caching operation of the media asset data are executed in parallel, which means that they are synchronized in time, thus minimizing the waiting time during the start-up process.

[0114] Exemplarily, Figure 6 FIG. is a comparison schematic diagram of the traditional start-up process of a streaming media application and the start-up process of an embodiment of the present application provided by some embodiments of the present application. As Figure 6 shown, in the traditional start-up process of a streaming media application, after the user clicks the play button of the playback control, the display device 200 can perform the initialization of the player and the opening of the player, i.e., open, and then the streaming media application starts to send down the media asset data. When the sent-down media asset data reaches a certain number of digits and a start-up signal thrown by the Device layer (referring to the underlying implementation layer related to the hardware device, which is responsible for interacting with the hardware and processing operations such as decoding and rendering of audio and video data) is received, that is, after the Device layer completes necessary resource preparations such as decoder initialization and data buffering, it sends a signal to the upper layer (such as the SDK), indicating that it can start playing, and the SDK will send down a play command. After the SDK sends down the play command, the remaining operations will be performed by the streaming media framework for the start-up operation, including play pipeline creation, decoder creation, decoder decoding, video frame rendering, etc., and this process is relatively time-consuming and will affect the start-up speed of the media asset data.

[0115] Compared with the startup process of traditional streaming media applications, the startup process of the streaming media application in this application, that is, the improved startup, adds a silent startup process. In some embodiments, the silent startup is equivalent to a normal playback, except that this process is carried out under the condition that the player is in the mute state, that is, mute. In this state, the sound is muted and the picture is not displayed at the same time. During the silent startup process, no sound and picture will be output during the execution of this process, which is equivalent to silently performing relevant preparatory work in the background, such as creating a playback pipeline, creating and decoding a decoder, rendering video frames, etc., but the results of these operations will not be presented to the user temporarily. For example, no audio and video pictures will be output until the unmute and resume operations are executed and then the normal playback will start. Moreover, the silent startup process and the data stream distribution are parallel. Since the data stream distribution requires buffering the data stream for a period of time, which is also a relatively time-consuming operation, basically when the data stream is distributed to the predetermined storage capacity, the silent startup has been completed, so the silent startup no longer takes time alone. In this way, the time for creating the playback pipeline, creating the decoder, decoding the decoder, and rendering the first video frame in the improved startup process can be saved, and the startup speed of the media asset data can be optimized.

[0116] That is to say, after the user clicks the play button, since the silent startup and data caching are carried out in parallel, the user can hardly feel an obvious waiting time. The silent startup process is executed when the player does not display the picture and the sound is in the mute state, which means that although the player has started to perform operations such as creating the playback pipeline, initializing the decoder, and rendering the first video frame, the results of these operations will not be immediately presented to the user, thus avoiding the user seeing a black screen or hearing a silent picture during the startup process. At the same time, the silent startup process is executed in parallel with the caching of the media asset data, so that after the data stream is distributed to the predetermined storage capacity, the silent startup has been completed, thereby optimizing the startup speed and reducing the user waiting time.

[0117] Figure 7 The timing schematic diagram of the method for the display device provided by some embodiments of this application to improve the startup speed of media assets is as Figure 7 shown, where 7-1 to 7-3 are the initialization processes of the player; 7-4 to 7-6 are the player open processes, which may include operations such as creating a player instance, configuring playback parameters, and initializing the decoder. The role of the player open process is to ensure that the player has prepared all necessary resources, such as a decoder, a playback pipeline, etc., before starting to play, so that the subsequent playback operations can proceed smoothly.

[0118] 7-7 to 7-11 are the process of pushing the first frame of audio and video data stream. After getting the first frame of video data, it can be sent to the player to render the first frame. 7-12 to 7-16 are the process of preparing player resources and informing the SDK that it can be played. Figure 4 Compared with the traditional method, the improved streaming media application playback process of this application adds silent start, as shown in parts 7-13 of the figure. Before the streaming media application sends the play playback instruction, after receiving the first frame data of audio and video, it can decode and render in advance, render the first frame in the mute state but not display it, and after completing these, inform the SDK that it can send the playback, and inform the application SDK that it can issue the play instruction through callbacks such as ready_to_play.

[0119] 7-17 to 7-20 are the data flow under SDK, which downloads and buffers data. It should be noted that in the embodiment of the present application, the two parts of the process 7-12 to 716 and 7-17 to 7-20 are executed in parallel, that is, the silent start and the media data push are executed in parallel, so that the initialization of the first play and pulling up the decoder are no longer time-consuming separately, thereby optimizing the time-consuming link of the start.

[0120] 7-21 to 7-29 are the process from SDK playing instructions to actual picture output. Since 7-12 to 7-16 have already prepared the playback resources of the media data and rendered the first frame, here we only need to resume playback and start the picture, as shown in 7-24, which can optimize the black screen duration and start speed. For example, the display device 200 can directly execute unmute, that is, release the mute state, resume audio output and resume playback operations, such as resuming from a pause or silent state to a normal playback state, thereby achieving extremely fast picture opening and reducing the black screen time at the start of playback.

[0121] The above content will be introduced in detail later.

[0122] In some embodiments, the display device 200 may perform silent start processing on the media asset data including the following contents. Figure 8 A flow chart of a display device performing silent start processing provided in some embodiments of the present application, such as Figure 8 As shown, the display device 200 can first create a playback pipeline and a decoder for the media data. After receiving the video data including the first video frame of the media data in the playback pipeline, the video data is decoded by the decoder; when the player does not display the picture, the decoded video data is rendered, and the first video frame of the media data is generated according to the rendered video data.

[0123] Exemplarily, the display device 200 performs silent start-up processing on media asset data, aiming to complete a series of playback preparation tasks in the background without outputting pictures and sounds to the user. After everything is ready, it will instantaneously switch to normal playback, reducing waiting and black screen time, and improving playback smoothness and user experience. Specifically, during the silent start-up process, the display device 200 can create a playback pipeline and a decoder for the media asset data. The creation of the playback pipeline can include a data transmission module, a decoding module, a rendering module, etc. These modules need to be configured and connected before playback starts to ensure that data can be smoothly transmitted from the source to the display 260. After the decoder is created, it can correctly parse and process the video coding format in the media asset data. Different video formats require corresponding decoders for decoding. Creating the decoder in advance can avoid delays caused by decoder initialization during playback. When the video data containing the first video frame of the media asset data is received in the playback pipeline, the decoder can perform a decoding operation on the video data. The decoding process can convert the compressed video data into a format that can be recognized and displayed by the display 260. The silent start-up processing is carried out under the condition that the player does not display pictures, which means that the user cannot see any playback-related picture output on the operation interface, and all operations of the player are silently completed in the background.

[0124] In this way, by completing operations such as creating the playback pipeline, creating the decoder, decoding, and rendering the first frame in the background in advance, when the user triggers the playback instruction, the display device 200 can immediately switch to the normal playback state and display the first frame picture. This significantly reduces the black screen time during the start-up process, and the user hardly feels the delay between clicking play and the appearance of the picture. Since the silent start-up has completed the media asset data preparation work, the execution of the playback instruction hardly requires additional waiting time. Therefore, the silent start-up processing can not only improve the start-up speed of the media asset data but also lay a foundation for subsequent playback smoothness.

[0125] Figure 9 The flowchart shows the process of the display device caching media asset data provided by some embodiments of this application. As Figure 9 shown, in some embodiments, when the display device 200 caches media asset data, it can first set a caching policy according to the network bandwidth information and the total storage amount of the media asset data, and then receive the audio data and video data in the media asset data based on the caching policy, and then save the audio data and video data in segments.

[0126] Exemplarily, when the display device 200 caches media asset data, it can dynamically adjust the caching policy by monitoring the network bandwidth and the total storage amount of media asset data. For example, in a low-bandwidth network environment, it can preferentially cache audio data and key frames of videos to ensure basic playback quality; while in a high-bandwidth environment, it can increase the resolution and frame rate of the cached data to enhance the viewing experience. In this way, the caching mechanism for caching media asset data can dynamically adapt to network changes. Under different network conditions, the display device 200 can adjust the caching focus to ensure smooth playback. After receiving audio data and video data based on the caching policy, the display device 200 can save them in segments. For example, the audio data is segmented according to the playback order, and the video data is segmented according to scenes or key frames, etc., which is convenient for managing and quickly retrieving media asset data.

[0127] In some embodiments, after the display device 200 creates a playback pipeline and a decoder for media asset data, during the silent startup process, it can detect the creation status of the playback pipeline or the decoder; if it detects that the creation status of the playback pipeline or the decoder is a creation failure, it terminates the silent startup process and re-initializes the playback pipeline or the decoder; after the playback pipeline or the decoder is successfully re-initialized, it continues to execute the silent startup process.

[0128] Exemplarily, during the silent startup process, if the resource creation of the playback pipeline and the decoder fails, it will affect subsequent operations. If it detects that the creation status of the playback pipeline or the decoder is a failure, the display device 200 can terminate the current silent startup process to avoid continuing subsequent operations when the resources are not properly initialized, thereby preventing possible errors or abnormal behaviors. After the playback pipeline or the decoder is successfully re-initialized, the display device 200 can continue to execute the silent startup process to ensure the continuity and integrity of the playback preparation work, so that subsequent playback operations can proceed smoothly. In this way, by adding an error handling mechanism during the silent startup process, the robustness of the startup process can be ensured, enabling the media asset data to be played normally. After step S1 is executed, the following step S2 can be executed.

[0129] Step S2: After the first video frame of the media asset data is rendered and after the cache of the media asset data reaches the threshold storage amount, the display device 200 generates a playback instruction for the media asset data.

[0130] In some embodiments, after the display device 200 generates the first video frame of the media asset data based on the rendered video data, during the silent startup process, it can pre-load the playback pipeline resources, decoder resources, and rendering resources of the media asset data; after the first video frame of the media asset data is rendered, it calls the pre-loaded playback pipeline resources, decoder resources, and rendering resources to play the media asset data.

[0131] Exemplarily, during the silent startup process, preloading the playback pipeline, decoder, and rendering resources in advance can reduce the resource initialization time when the user actually starts playing. This can significantly reduce the startup latency, avoid excessive waiting time for the user, and enhance the viewing experience. After the first video frame is rendered, it indicates that the video is ready to play. At this time, calling the preloaded resources can start playing immediately, ensuring a smooth startup effect. Therefore, by preloading resources during the silent startup process, the display device 200 can reduce the resource initialization time at startup, thereby enhancing the startup speed of media asset data.

[0132] In some embodiments, after the display device 200 completes the silent startup process, that is, after the first video frame is rendered, and at the same time the cache of the media asset data reaches a predetermined threshold storage amount, the display device 200 can generate a playback instruction for the media asset data. In this way, it can be ensured that when the player starts playing, it already has sufficient buffered data and the rendered first frame picture, so that it can start playing immediately.

[0133] Exemplarily, after the user clicks the play button, the display device 200 can perform silent startup and caching operations in the background. When the first video frame is rendered and the cache reaches the threshold, the system will automatically trigger the generation of a playback instruction. When detecting whether the first video frame is rendered, it can be achieved through the internal status monitoring mechanism of the player. After the first video frame is rendered, the player can send a signal to the system. When detecting whether the cache of the media asset data reaches the threshold storage amount, the cache module inside the display device 200 can monitor the cache data volume in real time. When the data volume reaches the preset threshold, the cache module can send a signal to the system. When the above two conditions are met simultaneously, the display device 200 can generate a playback instruction and send it to the player.

[0134] In some embodiments, before the display device 200 generates a playback instruction for the media asset data, it can monitor the storage amount of the media asset data cache. When the storage amount reaches the threshold storage amount, it generates an instruction generation packet, where the instruction generation packet is used to trigger the generation of a playback instruction.

[0135] Exemplarily, when the display device 200 caches media asset data, it can start a background process or thread to monitor the storage capacity of the cache in real time. For example, this monitoring mechanism can periodically (e.g., every second or every few seconds) check the current storage capacity of the cache and compare it with a preset threshold storage capacity. When the cache storage capacity reaches or exceeds the threshold storage capacity, the monitoring mechanism can trigger an event, which is responsible for generating an instruction generation packet. After receiving the instruction generation packet, the player instance can start playing the media asset data according to the instructions therein. The player instance can continue to monitor the cache storage capacity during the playback process to ensure smooth playback and dynamically adjust the cache policy as needed. In this way, by monitoring the cache storage capacity and triggering playback when the threshold is reached, the display device 200 can ensure that sufficient data has been cached at the start of playback to avoid playback stuttering or interruption caused by insufficient cache, thereby improving the smoothness of playback. After step S2 is completed, the following step S3 can be executed.

[0136] Step S3: In response to the play instruction, the display device 200 controls the display 260 to play the media asset data according to the first video frame.

[0137] In some embodiments, after generating the play instruction, the display device 200 can control the display 260 to play the media asset data according to the first video frame. Before controlling the display to play the media asset data according to the first video frame, the display device 200 can perform an unmute process on the media asset data after the silent start process and control the display to display the media asset screen corresponding to the media asset data.

[0138] Exemplarily, since the display device 200 has previously performed a silent start process on the media asset data, before playing the media asset data, the display device 200 needs to perform an unmute process on the media asset data to ensure that the audio can be played normally. At the same time, the display device 200 can control the display to display the media asset screen corresponding to the media asset data so that the user can see the video content to ensure that the video and audio can be presented to the user synchronously and smoothly.

[0139] Figure 10 The flowchart of the display device playing media asset data according to the first video frame provided by some embodiments of the present application is as follows Figure 10 As shown, when the display device 200 plays the media asset data according to the first video frame, it can first obtain the first video frame and perform decoding processing on the first video frame; then send the decoded first video frame to the display, and then control the display to play the media asset data according to the first video frame.

[0140] Exemplarily, after muting the media asset data and restoring the media asset picture, the display device 200 may render the first video frame onto the display 260 to present the media asset data after the silent start-up process and cache preparation to the user in a normal manner. Specifically, the display device 200 may first accurately obtain the first video frame from the media asset data stream. After obtaining the first video frame, it may perform decoding processing on it to convert the first video frame into a data format that can be recognized and displayed by the display. After the decoding is completed, the display device 200 may send the decoded first video frame to the display 260 through a specific communication interface or data transmission channel, instructing the display 260 to start playing the media asset data according to the received first video frame. By obtaining and decoding the first video frame, the display device 200 can present the video picture to the user in the shortest time, reduce the user's waiting time for the video to start, improve the response speed of the media asset start-up, and solve the problem of slow start-up speed in the current streaming media application during the process of playing media asset data.

[0141] As can be seen from the above technical solutions, the above embodiment provides a display device 200, which responds to a touch instruction of a playback control corresponding to media asset data, performs a silent start-up process on the media asset data, and caches the media asset data; the silent start-up process of the media asset data and the caching of the media asset data are executed in parallel; the silent start-up process is executed under the condition that the player does not display a picture and the sound is in a muted state; after the first video frame of the media asset data is rendered, and after the cache of the media asset data reaches the threshold storage amount, a playback instruction for the media asset data is generated; in response to the playback instruction, the display is controlled to play the media asset data according to the first video frame. The display device 200 reduces the waiting time between the user's click to play and the appearance of the picture by introducing the mechanism of silent start-up process and parallel caching of media asset data, optimizes the start-up speed of the streaming media application for media asset data, reduces the black screen time during the start-up process, and solves the problem of slow start-up speed in the current streaming media application during the process of playing media asset data.

[0142] Based on the above display device 200, some embodiments of the present application further provide a method for improving the start-up speed of media assets, and this method can be applied to the display device 200 in the above embodiment. In some embodiments, the method may include the following content:

[0143] In response to a touch instruction of a playback control corresponding to media asset data, perform a silent start-up process on the media asset data, and cache the media asset data; the silent start-up process of the media asset data and the caching of the media asset data are executed in parallel; the silent start-up process is executed under the condition that the player does not display a picture and the sound is in a muted state;

[0144] After the first video frame of the media asset data is rendered and after the cache of the media asset data reaches the threshold storage amount, a play instruction for the media asset data is generated;

[0145] In response to the play instruction, control the display to play the media asset data according to the first video frame.

[0146] As can be seen from the above technical solutions, the above embodiments provide a method for improving the start-up speed of media assets. The method reduces the waiting time between the user clicking play and the picture appearing by introducing a silent start-up process and a mechanism for parallel caching of media asset data, and optimizes the start-up speed of the media asset data in the streaming media application, reducing the black screen time during the start-up process, and solving the problem of slow start-up speed in the current streaming media application when playing media asset data.

[0147] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solutions in the embodiments of the present invention, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods in various embodiments or some parts of the embodiments of the present invention.

[0148] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

[0149] For the sake of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. According to the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A display device, characterized in that, Comprising: A display configured to display a user interface; A controller configured to: In response to a touch instruction of a user on a play control corresponding to media data, perform a silent start-up process on the media data and cache the media data; the silent start-up process of the media data is executed in parallel with the caching of the media data; the silent start-up process is executed under the condition that the player does not display a picture and the sound is in a mute state; After the first frame of video frame of the media data is rendered and after the cache of the media data reaches a threshold storage amount, generate a play instruction for the media data; In response to the play instruction, control the display to play the media data according to the first frame of video frame.

2. The display device according to claim 1, wherein The controller performs the silent start-up process on the media data, and is specifically configured to: Create a play pipeline and a decoder for the media data; After receiving video data including the first frame of video frame of the media data in the play pipeline, perform decoding on the video data through the decoder; Under the condition that the player does not display a picture, render the decoded video data; Generate the first frame of video frame of the media data according to the rendered video data.

3. The display device according to claim 2, wherein After the step that the controller creates a play pipeline and a decoder for the media data, it is further configured to: During the silent start-up process, detect the creation status of the play pipeline or the decoder; If it is detected that the creation status of the play pipeline or the decoder is a creation failure, terminate the silent start-up process and re-initialize the play pipeline or the decoder; After the play pipeline or the decoder is successfully re-initialized, continue to execute the silent start-up process.

4. The display device according to claim 2, wherein After the step that the controller generates the first frame of video frame of the media data according to the rendered video data, it is further configured to: During the silent start-up process, pre-load the play pipeline resources, decoder resources and rendering resources of the media data; After the first frame of video frame of the media data is rendered, call the pre-loaded play pipeline resources, decoder resources and rendering resources to play the media data.

5. The display device according to claim 2, wherein Before the controller controls the display to play the media data according to the first frame of video frame, it is further configured to: Perform an unmute process on the media data after the silent start-up process; Control the display to display the media picture corresponding to the media data.

6. The display device according to claim 1, wherein The controller caches the media data, and is specifically configured to: Set a cache policy according to network broadband information and the total storage amount of the media data; Receive the audio data and video data in the media data based on the cache policy; Save the audio data and the video data in segments.

7. The display device according to claim 1, wherein Before the controller performs the silent start-up process on the media data, it is further configured to: Create a player instance for the media data; Configure play parameters for the player instance.

8. The display device according to claim 1, wherein Before the controller generates a play instruction for the media data, it is further configured to: Monitor the storage amount of the media data cache; When the storage amount reaches the threshold storage amount, an instruction generation packet is generated; the instruction generation packet is used to trigger the generation of the play instruction.

9. The display device according to claim 1, wherein The controller controls the display to play the media data according to the first video frame, and is specifically configured to: Obtain the first video frame; Perform decoding processing on the first video frame; Send the decoded first video frame to the display; Control the display to play the media data according to the first video frame.

10. A method for improving the start-up speed of media assets, which is applied to the display device according to any one of claims 1-9, characterized in that, The method includes: In response to a touch instruction on a play control corresponding to media data, perform a silent start-up process on the media data and cache the media data; the silent start-up process of the media data is executed in parallel with the caching of the media data; the silent start-up process is executed under the condition that the player does not display a picture and the sound is in a mute state; After the first video frame of the media data is rendered and after the cache of the media data reaches the threshold storage amount, generate a play instruction for the media data; In response to the play instruction, control the display to play the media data according to the first video frame.

Images