Display device and video switching method

Abstract

Some embodiments of the present application show a display device and a video switching method, the method comprising: when playing first video data, if the first video data is a target video frame, saving the target video frame to a preset storage location; in response to a user input instruction of switching video, decoding the target video frame by a soft decoding module to obtain image data; controlling the display to display an image drawn based on the image data; stopping playing the first video data; starting playing second video data, and after receiving a start message of the second video data, controlling the display to cancel displaying the image. After receiving a video switching instruction, the saved video frame is displayed on the screen by soft decoding, which can block the black screen picture generated in the process of switching video, and the video frame display is cancelled when the next video is played, which can avoid the problems of black screen and incomplete picture display.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of display devices, and in particular to a display device and a video switching method. BACKGROUND

[0002] With the continuous development of intelligent hardware, the performance of hardware is continuously improved, and users have higher requirements for video application experience, especially for the fluency experience in the video switching process.

[0003] When receiving a user input instruction to switch videos, it is often necessary to close the current player and then create a new player to play the next video. However, during the process of closing the current player and waiting for the next video data to be rendered, there is a short period of time without picture display, i.e., a black screen phenomenon, or it can be understood that the newly created player is difficult to obtain the last frame of the previous player, thereby causing a black screen phenomenon. If the previous player is used during the video switching process, i.e., a new player is not created to play the next video, although the black screen problem is solved, the player suitable for the previous video is not suitable for the next video due to different encoding parameters such as resolution of the two videos, thereby causing a problem of incomplete picture display during the playing process. SUMMARY

[0004] Some embodiments of the present application provide a display device and a video switching method. After receiving a video switching instruction, the saved video frame is displayed on the screen through soft decoding, which can block the black screen picture generated during the video switching process, and the video frame display is cancelled when the next video is played, thereby avoiding the problems of black screen and incomplete picture display.

[0005] In a first aspect, some embodiments of the present application provide a display device, comprising:

[0006] a display;

[0007] a controller configured to:

[0008] when playing first video data, if the first video data is a target video frame, saving the target video frame to a preset storage location to replace a target video frame already saved in the preset storage location, the target video frame including a key frame;

[0009] in response to a user input instruction to switch videos, decoding the target video frame through a soft decoding module to obtain image data;

[0010] controlling the display to display an image drawn based on the image data;

[0011] stopping playing the first video data;

[0012] start playing the second video data, and after receiving a start message of the second video data, control the display to stop displaying the image.

[0013] In some embodiments, the controller performs saving the target video frame to a preset storage location, and is further configured to:

[0014] remove a key frame saved in the preset storage location, and save the key frame to the preset storage location.

[0015] In some embodiments, the target video frame further includes a preset number of video frames after the key frame, and the controller performs saving the target video frame to a preset storage location, and is further configured to:

[0016] remove the target video frame saved in the preset storage location, and save the key frame and the preset number of video frames after the key frame to the preset storage location.

[0017] In some embodiments, the controller includes an application layer and a player; the controller performs playing the first video data, and is further configured to:

[0018] control the application layer to execute an instruction of playing the first video data in response to a user input, and send the instruction of playing the first video data to the player;

[0019] control the player to execute, after receiving the instruction of playing the first video data, creating a decoder, decoding the first video data through the decoder, and displaying the decoded first video data on a video layer after rendering.

[0020] In some embodiments, the controller performs decoding the target video frame through a soft decoding module to obtain image data in response to an instruction of switching videos from a user input, and is further configured to:

[0021] control the application layer to execute the instruction of switching videos from the user input, and send the instruction of switching videos to the player;

[0022] control the player to execute, after receiving the instruction of switching videos, calling the soft decoding module to decode the target video frame to obtain image data, and sending the image data to the application layer.

[0023] In some embodiments, the controller performs controlling the display to display an image drawn based on the image data, and is further configured to:

[0024] control the application layer to execute drawing the image based on an open graphics library and the image data, and control the display to display the image on a user interface layer, which is above the video layer.

[0025] In some embodiments, the controller's action to stop playing the first video data is further configured to:

[0026] The application layer is controlled to send an instruction to stop playing the first video data to the player;

[0027] Upon receiving an instruction to stop playing the first video data, the player is controlled to release the resources of the decoder and send a stop message to the application layer.

[0028] In some embodiments, the controller executes the start of playback of the second video data, which is further configured to:

[0029] The application layer is controlled to send an instruction to the player to play the second video data after receiving the stop message;

[0030] Upon receiving an instruction to play the second video data, the player is controlled to recreate the decoder, decode the second video data using the decoder, and then render and display the decoded second video data on the video layer.

[0031] In some embodiments, the controller, upon receiving a start message for the second video data, controls the display to cancel the display of the image, which is further configured to:

[0032] The player is controlled to send a start-up message to the application layer after the second video data is successfully started.

[0033] Upon receiving the start-up message, the application layer controls the display to cancel the display of the image at the user interface layer.

[0034] Secondly, some embodiments of this application provide a video switching method, including:

[0035] When playing the first video data, if the first video data is a target video frame, the target video frame is saved to a preset storage location to replace the target video frame already saved in the preset storage location. The target video frame includes keyframes.

[0036] In response to a user's instruction to switch videos, the target video frame is decoded via a software decoding module to obtain image data;

[0037] Control the display to show an image drawn based on the image data;

[0038] Stop playing the first video data;

[0039] start playing the second video data, and after receiving a start message of the second video data, control the display to stop displaying the image.

[0040] Some embodiments of the present application provide a display device and a video switching method. When playing a first video data, if the first video data is a target video frame, save the target video frame to a preset storage location to replace a target video frame already saved in the preset storage location. Wherein, the target video frame includes a key frame; after receiving a user input instruction of switching video, decode the target video frame by a soft decoding module to obtain image data. Control a display to display an image drawn based on the image data; after stopping playing the first video data, start playing a second video data, and after receiving a start message of the second video data, control the display to stop displaying the image. The embodiments of the present application can display the saved video frame on the screen by soft decoding after receiving the video switching instruction, which can block the black screen picture generated in the video switching process, and can stop displaying the video frame when playing the next video, which can avoid the problems of black screen and incomplete picture display. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 An operation scenario between a display device and a control device according to some embodiments is shown;

[0042] Figure 2 A hardware configuration block diagram of a control device according to some embodiments is shown;

[0043] Figure 3 A hardware configuration block diagram of a display device according to some embodiments is shown;

[0044] Figure 4 A software configuration diagram of a display device according to some embodiments is shown;

[0045] Figure 5 A timing diagram of a first video switching method according to some embodiments is shown;

[0046] Figure 6 A timing diagram of a second video switching method according to some embodiments is shown;

[0047] Figure 7 A timing diagram of a third video switching method according to some embodiments is shown;

[0048] Figure 8 A timing diagram of a fourth video switching method according to some embodiments is shown;

[0049] Figure 9 A flowchart of a video switching method according to some embodiments is shown;

[0050] Figure 10 A schematic diagram illustrating a user interface provided according to some embodiments is shown;

[0051] Figure 11 A schematic diagram illustrating another user interface provided according to some embodiments is shown;

[0052] Figure 12 A software architecture diagram provided according to some embodiments is shown;

[0053] Figure 13 A timing diagram of a fifth video switching method provided according to some embodiments is shown. DETAILED DESCRIPTION

[0054] For the purpose of making the objects and embodiments of the present application more clear, the following will combine the drawings in the exemplary embodiments of the present application to clearly and completely describe the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only some of the embodiments of the present application, but not all of the embodiments of the present application.

[0055] It should be noted that the brief description of the terms in the present application is only for the convenience of understanding the following described embodiments, and is not intended to limit the embodiments of the present application. Unless otherwise specified, these terms should be understood according to their ordinary and general meanings.

[0056] The terms "first" and "second" and the like in the specification and claims of the present application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit the specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchanged under appropriate circumstances.

[0057] The term "includes" and any variation thereof, is intended to cover but not exclusive inclusion, for example, a product or device including a series of components does not necessarily limit to all components clearly listed, but can include other components not clearly listed or inherent to these products or devices.

[0058] The display device provided by the embodiments of the present application can have various implementation forms, for example, can be a television, a smart television, a laser projection device, a monitor, an electronic bulletin board, an electronic table, etc. Figure 1 And Figure 2 A specific embodiment of the display device of the present application.

[0059] Figure 1 A schematic diagram of the operation scenario between the display device and the control device according to the embodiments is shown. As shown inFigure 1 As shown, the user can operate the display device 200 through the smart device 300 or the control device 100.

[0060] In some embodiments, the control device 100 can be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication or Bluetooth protocol communication, and other short-distance communication modes, to control the display device 200 through wireless or wired mode. The user can input user instructions through the keys on the remote controller, voice input, control panel input, etc., to control the display device 200.

[0061] In some embodiments, the smart device 300 (such as a mobile terminal, a tablet computer, a computer, a notebook computer, etc.) can also be used to control the display device 200. For example, the display device 200 is controlled using an application running on the smart device.

[0062] In some embodiments, the display device can not receive instructions using the smart device or the control device described above, but can receive user control through touch or gesture, etc.

[0063] In some embodiments, the display device 200 can also be controlled in ways other than the control device 100 and the smart device 300, for example, the user's voice instructions can be received directly through the voice instruction acquisition module configured inside the display device 200 device, or the user's voice instructions can be received through the voice control device set outside the display device 200 device.

[0064] In some embodiments, the display device 200 also communicates data with the server 400. The display device 200 can be allowed to communicate through a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 400 can provide various content and interaction to the display device 200. The server 400 can be a cluster, or multiple clusters, and can include one or more types of servers.

[0065] Figure 2 An exemplary configuration block diagram of the control device 100 according to an exemplary embodiment is shown. As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface 140, a memory, a power supply. The control device 100 can receive user input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, acting as an intermediary between the user and the display device 200. Figure 2 As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface 140, a memory, a power supply. The control device 100 can receive user input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, acting as an intermediary between the user and the display device 200.

[0066] Figure 3 ​The display device 200 includes at least one of a tuner and demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a storage, a power supply, and a user interface.

[0067] In some embodiments, the controller includes a processor, a video processor, an audio processor, a graphic processor, a RAM, a ROM, a first interface to an n-th interface for input / output.

[0068] The display 260 includes a display screen component for presenting a picture, and a driving component for driving the image display, a component for receiving an image signal from the output of the controller, and displaying video content, image content, and a menu control interface, and a user control UI interface.

[0069] The display 260 can be a liquid crystal display, an OLED display, and a projection display, and can also be a projection device and a projection screen.

[0070] The display 260 further includes a touch screen, which is used to receive a control instruction input by a user sliding or clicking a finger on the touch screen.

[0071] The communicator 220 is a component for communicating with external devices or servers according to various communication protocol types. For example, the communicator can include at least one of a Wifi module, a Bluetooth module, a wired Ethernet module, other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The display device 200 can establish transmission and reception of control signals and data signals with the external control device 100 or the server 400 through the communicator 220.

[0072] The user interface can be used to receive a control signal of the control device 100 (such as an infrared remote controller).

[0073] The detector 230 is used to collect signals of the external environment or interaction with the outside. For example, the detector 230 includes a light receiver for collecting ambient light intensity, or an image collector such as a camera for collecting external environment scenes, user attributes, or user interaction gestures, or a sound collector such as a microphone for receiving external sounds.

[0074] The external device interface 240 can include, but is not limited to, any one or more of the following: a high-definition multimedia interface (HDMI), an analog or data high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, and the like. It can also be a composite input / output interface formed by the above multiple interfaces.

[0075] The tuner and demodulator 210 receives broadcast television signals through wired or wireless reception, and demodulates audio and video signals, such as EPG data signals, from a plurality of broadcast television signals.

[0076] In some embodiments, the controller 250 and the tuner and demodulator 210 can be located in different separate devices, i.e., the tuner and demodulator 210 can also be located in an external device of the main device in which the controller 250 is located, such as an external set-top box, etc.

[0077] The controller 250 controls the operation of the display device and responds to the user's operation by storing various software control programs in the memory. The controller 250 controls the overall operation of the display device 200. For example, in response to receiving a user command for selecting a UI object displayed on the display 260, the controller 250 can perform an operation related to the object selected by the user command.

[0078] In some embodiments, the controller includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), a random access memory (RAM), a read-only memory (ROM), a first to nth interface for input / output, a communication bus, etc.

[0079] The user can input a user command through a graphic user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphic user interface (GUI). Alternatively, the user can input a user command by inputting a specific sound or gesture, and the user input interface receives the user input command by recognizing the sound or gesture through a sensor.

[0080] The "user interface" is a medium interface for interaction and information exchange between an application or an operating system and a user, and it implements conversion between an internal form of information and a form that a user can accept. A commonly used form of the user interface is a graphic user interface (GUI), which is a user interface that displays information in a graphic manner. It can be an icon, a window, a control, etc. displayed on a display screen of an electronic device, and the control can include an icon, a button, a menu, a tab, a text box, a dialog box, a status bar, a navigation bar, a widget, etc.

[0081] As shown in FIG. 1, the system of the display device is divided into three layers from top to bottom, i.e., an application layer, a middleware layer, and a hardware layer. Figure 4 As shown in FIG. 1, the system of the display device is divided into three layers from top to bottom, i.e., an application layer, a middleware layer, and a hardware layer.

[0082] The application layer mainly includes common applications on the television and an application framework. The common applications are mainly applications developed based on a browser, such as HTML5 APPs, and native applications (Native APPs).

[0083] The application framework is a complete program model, which has all basic functions required by standard application software, such as file access, data exchange, and use interfaces (toolbars, status columns, menus, dialog boxes) of the functions.

[0084] The native applications (Native APPs) can support online or offline, message pushing or local resource access.

[0085] The middleware layer includes various television protocols, multimedia protocols, and system components. The middleware can use basic services (functions) provided by system software to connect various parts of the application system on the network or different applications, so as to achieve the purpose of resource sharing and function sharing.

[0086] The hardware layer mainly includes a HAL interface, hardware, and a driver. The HAL interface is a unified interface for all television chips, and the specific logic is implemented by each chip. The driver mainly includes an audio driver, a display driver, a Bluetooth driver, a camera driver, a WIFI driver, a USB driver, an HDMI driver, a sensor driver (such as a fingerprint sensor, a temperature sensor, a pressure sensor, etc.), and a power supply driver.

[0087] With the continuous development of smart hardware, the hardware performance level is continuously improved, and users have higher requirements for video application experience, especially for the fluency experience in the video switching process.

[0088] In some embodiments, as Figure 5As shown, after receiving the instruction of playing the first video data from the user, the application layer sends the instruction of playing the first video data to the player, the player creates a decoder, decodes the first video data through the decoder, and displays the first video data on the display after rendering. After receiving the instruction of switching the video, i.e., playing the second video data from the user, the application layer sends the instruction of stopping playing the first video data to the player. The player releases the resource of the decoder, i.e., stops playing the first video, and sends a stop message to the application layer. After receiving the stop message, the application layer sends the instruction of playing the second video to the player. The player creates a decoder again, decodes the second video data through the decoder, and displays the second video data on the display after rendering. However, there is no picture display during the time when the player stops playing the first video data to the first frame of the second video, or it can be understood that the newly created decoder (player) is difficult to obtain the last frame of the previous decoder (player), and a black screen phenomenon occurs.

[0089] In some embodiments, in order to avoid the occurrence of the black screen phenomenon, the previous decoder (player) is used during the video switching process, i.e., a decoder (player) is not created again, and the next video is played. The video switching method is as shown in Figure 6 As shown, after receiving the instruction of playing the first video data from the user, the application layer sends the instruction of playing the first video data to the player, the player creates a decoder, decodes the first video data through the decoder, and displays the first video data on the display after rendering. After receiving the instruction of switching the video, i.e., playing the second video data from the user, the application layer sends the instruction of playing the second video data to the player, the player directly decodes the second video data through the decoder created before, and displays the second video data on the display after rendering. However, due to the fact that the encoding information such as resolution of the first video data and the second video data may be different and the compatibility of the hardware chip is not strong, the decoder (player) parameters suitable for the first video data may not be suitable for the second video data, resulting in the problem of incomplete picture display after the video switching. For example, the resolution of the first video data is 1080p, and the resolution of the second video data is 720p. After switching the video, the decoder (player) suitable for 1080p video cannot adapt to the 720p video, resulting in incomplete picture display.

[0090] In some embodiments, as Figure 7As shown, after receiving the instruction of playing the first video data from the user, the application layer sends the instruction of playing the first video data to the player, the player creates a decoder, and the first video data is decoded by the decoder and displayed by the display after rendering. After receiving the instruction of switching the video, i.e., playing the second video data from the user, the application layer sends a request for obtaining a video poster to the server, and receives the video poster image issued by the server based on the request. The application layer controls the display to display the video poster image, and sends an instruction of stopping playing the first video data to the player. The video poster can be the poster of the first video or the poster of the second video. The player releases the resource of the decoder, i.e., stops playing the first video data, and sends a stop message to the application layer. After receiving the stop message, the application layer sends an instruction of playing the second video data to the player. The player re-creates the decoder, and the second video data is decoded by the decoder and displayed by the display after rendering. The application layer controls the display to cancel displaying the video poster image when the second video data is played. Although the video poster can cover the black screen phenomenon of the switching video, the video poster has no direct correlation with the last frame of the first video and the first frame of the second video, and the user can obviously feel the fragmentation of the video switching, and the user experience is poor.

[0091] In some embodiments, as shown, Figure 8 As shown, after receiving the instruction of playing the first video data from the user, the application layer sends the instruction of playing the first video data to the player, the player creates a decoder, and the first video data is decoded by the decoder and displayed by the display after rendering. After receiving the instruction of switching the video, i.e., playing the second video data from the user, the application layer sends a request for obtaining a video poster to the server, and receives the video poster image issued by the server based on the request. The application layer controls the display to display the video poster image, and sends an instruction of stopping playing the first video data to the player. The video poster can be the poster of the first video or the poster of the second video. The player releases the resource of the decoder, i.e., stops playing the first video data, and sends a stop message to the application layer. After receiving the stop message, the application layer sends an instruction of playing the second video data to the player. The player re-creates the decoder, and the second video data is decoded by the decoder and displayed by the display after rendering. The application layer controls the display to cancel displaying the video poster image when the second video data is played. Although the video poster can cover the black screen phenomenon of the switching video, the video poster has no direct correlation with the last frame of the first video and the first frame of the second video, and the user can obviously feel the fragmentation of the video switching, and the user experience is poor.

[0092] TextureView is a View that can draw its content into an external texture. TextureView is a normal View in the View hierarchy, so it can be animated just like any other View, with translation, rotation, scaling, etc. However, TextureView must be in a hardware-accelerated window, and the content it draws can come from either the app process or a remote process. TextureView inherits from View, and is managed and drawn in the View hierarchy just like any other View, requiring CPU to process, which can consume CPU to some extent.

[0093] To solve the above technical problems, an embodiment of the present application provides a display device 200, the structure and functions of each part of the display device 200 can be referred to the above embodiments. In addition, based on the display device 200 shown in the above embodiments, some functions of the display device 200 are further improved in this embodiment. As shown in the figure, the controller 250 performs the following steps: Figure 9

[0094] Step S901: playing first video data;

[0095] The controller includes an application layer and a player, and the step of playing the first video data includes:

[0096] The controller controls the application layer to execute an instruction of playing the first video data in response to a user input, and sends the instruction of playing the first video data to the player;

[0097] The controller controls the player to execute, after receiving the instruction of playing the first video data, creating a decoder, decoding the first video data through the decoder, and displaying the decoded first video data on a video layer after rendering.

[0098] The decoder is a hardware decoder. The hardware decoder includes a Mediacodec. The Mediacodec is mainly used for video decoding and encoding operations, and can realize functions such as video fast playback, full key frame conversion, and video reverse playback. The Mediacodec generates output data after processing input data, processes data asynchronously, and uses a set of input and output buffers. For example, an empty input buffer is requested, it is filled with data and sent to the codec for processing. The codec processes the data and converts it to an empty output buffer. Finally, a full output buffer is requested, its content is used and it is released back to the codec.

[0099] The decoded first video data is input to a rendering module, and the rendering module renders the decoded first video data, and the display displays the rendered image on the video layer. ​

[0100] The rendering module includes a SurfaceView. The update of a conventional View and its derived classes can only be performed in a UI thread, however, the UI thread also simultaneously processes other interactive logic, which cannot guarantee the speed and frame rate of View update, while the SurfaceView can use an independent thread for drawing, thus providing a higher frame rate.

[0101] For video decoding, Mediacodec is used together with the SurfaceView, which can enable the GPU to perform efficient processing, improve performance, and reduce power consumption.

[0102] It should be noted that after the player receives the instruction to play the first video data, the player also needs to decode the first audio data corresponding to the first video data through a hardware decoder or a software decoding module, and play the decoded audio data through an audio output interface. The first video data and the first audio data are played synchronously.

[0103] To clarify the process of playing the first video data according to the present application, one embodiment is disclosed as follows:

[0104] After the application layer receives the instruction of the user to play the first video data, the application layer sends the instruction to play the first video data to the player, and the instruction to play the first video data includes the URL (Uniform Resource Locator, Uniform Resource Locator) corresponding to the first video data.

[0105] The player is initialized, that is, the player is created. After the player parses the URL corresponding to the first video data, the player obtains the streaming media data corresponding to the first video data from the URL, and parses the audio and video stream corresponding to the first video data to obtain the encoding information of the audio and video stream, that is, the encoding information of the first video data. The encoding information includes resolution, frame rate, code rate, and the like.

[0106] The player starts to create a decoder, that is, a hardware decoder, based on the encoding information. After the creation of the decoder is successful, the audio and video stream corresponding to the first video data is injected into the decoder to start the decoding process.

[0107] The decoded first video data is input into the rendering module for rendering to obtain a rendered image, and the rendered image is displayed on the video layer.

[0108] In some embodiments, the first video data includes video data of a network video.

[0109] In some embodiments, the first video data includes video data of a broadcast program.

[0110] Step S902: When playing the first video data, it is judged whether the first video data is a target video frame, wherein the target video frame comprises a key frame.

[0111] I frame, also known as intra picture. I frame is usually the first frame of each GOP (MPEG uses a kind of video compression technology), which is moderately compressed and used as a random access reference point, and can be used as an image. Frame is the basic unit of video image. Key frame, also known as I frame, is an important frame in inter-frame compression coding; it is a full-frame compressed coded frame; when decoding, only the data of I frame can reconstruct a complete image; I frame does not need to refer to other pictures to generate.

[0112] If the first video data is the target video frame, step S903 is executed: the target video frame is saved to a preset storage location to replace the target video frame already saved in the preset storage location.

[0113] In some embodiments, the target video frame is a key frame. It is judged whether the first video data is a key frame, i.e. I frame.

[0114] If the first video data is a key frame, the key frame already saved in the preset storage location is removed, and the key frame is saved to the preset storage location, i.e. the key frame is saved to the preset storage location to replace the key frame already saved in the preset storage location, which can also be expressed as saving the key frame to variable I.

[0115] If the first video data is not a key frame, it is continued to be judged whether the next first video data is a key frame.

[0116] The preset storage location can only save one frame of key frame data, and when a new key frame is stored, the stored key frame data needs to be removed, i.e. the player retains the last video key frame I frame to variable I in real time.

[0117] In some embodiments, the target video frame is a key frame and a preset number of video frames after the key frame. It is judged whether the received first video data is a key frame, i.e. I frame.

[0118] If the first video data is a key frame, the target video frame already saved in the preset storage location is removed, and the key frame and the preset number of video frames after the key frame are saved to the preset storage location, so as to replace the target video frame already saved in the preset storage location.

[0119] In order to clarify the process of saving the key frame and the preset number of video frames after the key frame to the preset storage location in the present application, one embodiment is disclosed as follows:

[0120] If the first video data is a key frame, removing the target video frame already saved in the preset storage location, saving the key frame to the preset storage location, and setting the storage frame number to 0.

[0121] If the first video data is not a key frame, determining whether the storage frame number is less than or equal to a preset value.

[0122] If the storage frame number is less than or equal to the preset value, saving the first video data to the preset storage location, and adding 1 to the storage frame number. The storage frame number less than or equal to the preset value indicates that the target video frame has not reached the preset value.

[0123] If the storage frame number is greater than the preset value, continuing to determine whether the first video data is a key frame. The storage frame number greater than the preset value indicates that the target video frame has reached the preset value, and no further storage is needed.

[0124] The embodiments of the present application can save only the key frame and a preset number of video frames after the key frame in the preset storage location, and can increase the dynamic effect when the subsequent occlusion black screen occurs.

[0125] The player is controlled to execute the following steps: determining whether the first video data is a target video frame when playing the first video data, saving the target video frame to the preset storage location to replace the target video frame already saved in the preset storage location if the first video data is the target video frame.

[0126] Step S904: In response to the user input instruction for switching the video, decoding the target video frame by the soft decoding module to obtain image data.

[0127] In some embodiments, the step of receiving the user input instruction for switching the video includes receiving the user input instruction for switching the channel by pressing the up and down keys of the control device 100. For example, as shown in FIG. 10, when playing the program of broadcast channel 1, the instruction for playing the program of channel 2 is input by pressing the down key of the control device 100. Figure 10

[0128] In some embodiments, the step of receiving the user input instruction for switching the video includes receiving the user input instruction for switching the network video by pressing the right key of the control device 100. For example, as shown in FIG. 11, when playing video 1, the side bar includes a plurality of video controls, and the instruction for playing video 2 is input by moving the focus to the side bar and pressing the down key of the control device 100. Figure 11

[0129] ​​It is to be noted that the controls refer to visual objects displayed in each display area of the user interface in the display device to represent corresponding content such as icons, thumbnails, video clips, links, etc. These controls can provide the user with various conventional program content received through data broadcasting, as well as various application and service content set by the content manufacturer.

[0130] The display form of the controls is usually diversified. For example, the controls can include text content and / or an image for displaying a thumbnail related to the text content, or a video clip related to the text. For another example, the controls can be text and / or icons of an application program.

[0131] The focus is used to indicate that any one of the controls has been selected. In one aspect, the controls can be selected or controlled according to the user's input through the control device 100 to control the movement of the focus object displayed in the display device. For example, the user can control the movement of the focus object between the controls through the up direction key on the control device 100 to select and control the controls. In another aspect, the controls can be selected or controlled according to the user's input through the control device 100 to control the movement of each control displayed in the display device. For example, the user can control each control to move left and right together through the up direction key on the control device 100 to select and control the controls while keeping the position of the focus object unchanged.

[0132] The identification form of the focus is usually diversified. For example, the position of the focus object can be realized or identified by enlarging the control, or by setting the background color of the control, or by changing the border line, size, color, transparency, and outline of the text or image of the focused control, and / or the font, etc.

[0133] In some embodiments, the step of receiving the instruction of switching the video input by the user includes:

[0134] The instruction of switching the video formed by the voice data collected by the user after pressing the voice key of the control device 100. For example, the user inputs the instruction of "playing channel 23" by pressing the voice key of the control device 100.

[0135] In some embodiments, the step of receiving the instruction of switching the video input by the user includes:

[0136] The instruction of switching the video formed by the voice data collected by the sound collector after the voice assistant is woken up by the far-field wake-up word. For example, the user wakes up the voice assistant by "HX Xiao J", and speaks the voice instruction of "playing channel 23".

[0137] In response to the instruction of switching the video input by the user, the step of decoding the target video frame by the soft decoding module to obtain the image data includes:

[0138] controlling the application layer to execute an instruction of switching the video in response to the user input, and sending the instruction of switching the video to the player;

[0139] controlling the player to execute, after receiving the instruction of switching the video, calling a soft decoding module to decode the target video frame to obtain image data, and sending the image data to the application layer.

[0140] The soft decoding module includes FFmpeg. FFmpeg is a set of open source computer programs that can be used to record, convert digital audio, video, and convert them into streams. FFmpeg provides a complete solution for recording, converting, and streaming audio and video. FFmpeg includes a very advanced audio / video codec library libavcodec, which can ensure high portability and codec quality.

[0141] In order to clarify the process of decoding the target video frame by the soft decoding module in the present application, one embodiment is disclosed as follows:

[0142] The application layer sends the instruction of switching the video to the player after receiving the instruction of switching the video in response to the user input.

[0143] The player calls the soft decoding module to decode the target video frame in the preset storage location into YUV image data after receiving the instruction of switching the video. The player transmits the YUV image data to the application layer through JNI (Java Native Interface, Java Native Interface) callback.

[0144] Step S905: controlling the display to display an image drawn based on the image data;

[0145] The step of controlling the display to display the image drawn based on the image data includes:

[0146] controlling the application layer to draw the image based on the image data and OpenGL (Open Graphics Library), and controlling the display to display the image on the user interface layer, wherein the user interface layer is located above the video layer. The display content of the user interface can cover the display content of the video layer.

[0147] In some embodiments, a video display area of the video layer is obtained;

[0148] an area corresponding to the video display area on the user interface layer draws the image based on the image data and the OpenGL; or

[0149] Based on the open graphics library and the image data, an image is drawn, the image is scaled and translated to a target region, so that the target region of the user interface layer coincides with the video display region of the video layer.

[0150] As shown in the example, Figure 10 The drawn image can be displayed full screen. Figure 11 As shown in the example,

[0151] In some embodiments, the target video frame is a key frame, and only the image drawn after decoding the key frame is displayed.

[0152] In some embodiments, the target video frame is a key frame and a preset number of video frames after the key frame, and the images drawn after decoding the target video frames are displayed in sequence, achieving a dynamic playing effect and avoiding the feeling of picture lag for the user.

[0153] Step S906: stop playing the first video data;

[0154] The step of stopping playing the first video data includes:

[0155] The application layer is controlled to send an instruction to stop playing the first video data to the player.

[0156] The player is controlled to release the resources of the hardware decoder after receiving the instruction to stop playing the first video data, i.e., to stop playing the first video data, or to stop the player, and to send a stop message to the application layer.

[0157] Step S907: start playing the second video data, and control the display to cancel displaying the image after receiving the start message of the second video data.

[0158] The step of starting to play the second video data includes:

[0159] The application layer is controlled to send an instruction to play the second video data to the player after receiving the stop message.

[0160] The player is controlled to re-create a decoder after receiving the instruction to play the second video data, to decode the second video data through the decoder, and to display the decoded second video data on the video layer after rendering.

[0161] It should be noted that after receiving the instruction to play the second video data, the player also needs to decode the second audio data corresponding to the second video data through the hardware decoder or the soft decoding module, and play the decoded audio data through the audio output interface. The second video data and the second audio data are played synchronously.

[0162] To make the process of playing the second video data clear, one embodiment is disclosed as follows:

[0163] After receiving the instruction of playing the second video data from the user, the application layer sends the instruction of playing the second video data to the player, and the instruction of playing the second video data includes the URL corresponding to the second video data.

[0164] The player is initialized, i.e., re-created. After obtaining the URL corresponding to the second video data through parsing, the player acquires the streaming media data corresponding to the second video data from the URL, and parses the audio and video stream corresponding to the second video data to obtain the encoding information of the audio and video stream, i.e., the encoding information of the second video data. The encoding information includes resolution, frame rate, code rate and the like.

[0165] The player re-creates the decoder, i.e., the hardware decoder, based on the encoding information. After the decoder is successfully created, the audio and video stream corresponding to the second video data is injected into the decoder to start the decoding process.

[0166] The decoded second video data is input into the rendering module for rendering to obtain the rendered image, and the rendered image is displayed on the video layer.

[0167] In some embodiments, the second video data includes the video data of a network video.

[0168] In some embodiments, the second video data includes the video data of a broadcast program.

[0169] After receiving the start message of the second video data, the step of controlling the display to cancel displaying the image includes:

[0170] The player is controlled to send the start message to the application layer after the decoding operation starts successfully.

[0171] The application layer is controlled to control the display to cancel displaying the image on the user interface layer after receiving the start message.

[0172] To make the process of canceling displaying the image clear, one embodiment is disclosed as follows:

[0173] The player sends the start message to the application layer after the decoder is initialized and the decoding operation starts successfully. The start message is FFP_MSG_VIDEO_DECODED_START, which is used to notify the application layer that the first frame of the video frame has been rendered.

[0174] After receiving the start message, the application layer controls the display to cancel displaying the image on the user interface layer, so that the second video data played on the video layer can be displayed to the user.

[0175] In some embodiments, such as Figure 12 As shown in the diagram, the software architecture includes an application layer, a middleware layer, and a hardware layer. The application layer receives user input commands and displays the user interface. It includes an open graphics library and a data processing module related to the user interface display. The data processing module draws images based on the image data sent by the player. The middleware layer includes the player. The player performs operations related to video playback, saves target video frames, and calls the software decoding module. The player includes a software decoding module, which decodes the saved target video frames. The hardware layer includes a hardware decoder and a rendering module. The hardware decoder decodes video data, and the rendering module renders the decoded video data and displays it on the screen.

[0176] In some embodiments, the timing diagram of the video switching method is as follows: Figure 13 As shown. After receiving a user's instruction to play the first video data, the application layer sends the instruction to the player. The player initializes and parses the instruction to obtain the encoding information, creates a decoder based on the encoding information, and saves the target video frame. The decoder decodes the first video data and inputs it to the rendering module (SurfaceView). The rendering module renders and displays the decoded first video data. After receiving a user's instruction to switch videos, the application layer calls the software decoding module to decode the target video frame, obtains the image data, and sends the image data to the application layer. The application layer processes the image data and displays it. The application layer sends an instruction to stop playing the first video data to the player, and the player releases the decoder's resources. The application layer sends an instruction to play the second video data to the player. The player reinitializes and parses the instruction to obtain the encoding information, recreates the decoder based on the encoding information, and saves the target video frame. The decoder decodes the second video data and inputs it to the rendering module, which renders and displays the decoded second video data.

[0177] Some embodiments of the present application provide a video switching method, which is suitable for a display device including a display and a controller, the controller is configured to: when playing a first video data, if the first video data is a target video frame, save the target video frame to a preset storage location to replace a target video frame already saved in the preset storage location, the target video frame including a key frame; in response to a user input instruction of switching video, decode the target video frame by a soft decoding module to obtain image data; control the display to display an image drawn based on the image data; stop playing the first video data; start playing a second video data, and after receiving a start message of the second video data, control the display to cancel displaying the image. After receiving the instruction of switching video, the saved video frame is displayed on the screen by soft decoding according to the embodiments of the present application, which can block the black screen picture generated in the process of switching video, and cancel the display of the video frame when the next video is played, which can avoid the problems of black screen and incomplete picture display, and the video switching effect is smoother.

[0178] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

[0179] In order to facilitate explanation, the above description has been made in combination with specific embodiments. However, the above exemplary discussion is not intended to exhaust or limit the embodiments to the specific forms disclosed above. Various modifications and variations can be derived according to the above teachings. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different modified embodiments suitable for specific use considerations.

Claims

1. A display device, characterized by comprising: The application relates to a video switching method and device. The application comprises: a display; a controller configured to: save a target video frame to a preset storage location to replace a target video frame already saved in the preset storage location if the first video data is the target video frame when playing the first video data, the target video frame comprising a key frame; decode the target video frame by a soft decoding module to obtain image data in response to an instruction of switching videos input by a user; control the display to display an image drawn based on the image data; stop playing the first video data; 2. The display device of claim 1, wherein, start playing second video data, and control the display to cancel displaying the image after receiving a start message of the second video data. The controller performing the step of saving the target video frame to the preset storage location is further configured to:

3. The display device of claim 1, wherein, remove the key frame already saved in the preset storage location and save the key frame to the preset storage location. The target video frame further comprises a preset number of video frames after the key frame, and the controller performing the step of saving the target video frame to the preset storage location is further configured to:

4. The display device of claim 1, wherein, remove the target video frame already saved in the preset storage location and save the key frame and the preset number of video frames after the key frame to the preset storage location. The controller comprises an application layer and a player, and the controller performing the step of playing the first video data is further configured to: control the application layer to send an instruction of playing the first video data to the player in response to an instruction of playing the first video data input by a user; 5. The display device of claim 4, wherein, control the player to create a decoder, decode the first video data by the decoder, and display the first video data after being decoded and rendered on a video layer after receiving the instruction of playing the first video data. The controller performing the step of decoding the target video frame by the soft decoding module to obtain the image data in response to the instruction of switching videos input by the user is further configured to: control the application layer to send an instruction of switching videos to the player in response to the instruction of switching videos input by the user; 6. The display device of claim 5, wherein, control the player to decode the target video frame by the soft decoding module to obtain the image data and send the image data to the application layer after receiving the instruction of switching videos. The controller performing the step of controlling the display to display the image drawn based on the image data is further configured to:

7. The display device of claim 6, wherein, control the application layer to draw the image based on an open graphics library and the image data, and control the display to display the image on a user interface layer, the user interface layer being above the video layer. The controller performing the step of stopping playing the first video data is further configured to: control the application layer to send an instruction of stopping playing the first video data to the player; 8. The display device of claim 7, wherein, control the player to release resources of the decoder and send a stop message to the application layer after receiving the instruction of stopping playing the first video data. The controller performing the step of starting playing the second video data is further configured to: control the application layer to send an instruction of playing the second video data to the player after receiving the stop message. The controller controls the player to perform, after receiving the instruction of playing the second video data, re-creating a decoder, decoding the second video data by the decoder, and displaying the decoded second video data on the video layer after rendering.

9. The display device of claim 8, wherein, The controller controls the display to cancel displaying the image after receiving the start message of the second video data, and is further configured to: The controller controls the player to send a start message to the application layer after the start of the second video data is successful. The controller controls the display to cancel displaying the image on the user interface layer after receiving the start message.

10. A video switching method characterized by comprising: Comprise: When playing the first video data, if the first video data is a target video frame, save the target video frame to a preset storage location to replace the target video frame already saved in the preset storage location, and the target video frame comprises a key frame; In response to the instruction of the user input switching the video, decode the target video frame by a soft decoding module to obtain image data; Control the display to display the image drawn based on the image data; Stop playing the first video data; Start playing the second video data, and control the display to cancel displaying the image after receiving the start message of the second video data.

Images