Display device and video multi-speed playing method

Abstract

The invention provides a display device and a video multi-speed playing method, and the method can respond to a video multi-speed playing instruction when a target video is played, a multi-speed rate and a first playing time point are obtained, and the first playing time point is the current playing time point of the target video. And calculating a second playing time point according to the multi-speed rate and the first playing time point, wherein the second playing time point is a playing time point before or after the first playing time point when the target video is played according to the multi-speed rate. And extracting a target key frame in the target video, and setting the display timestamp of the target key frame as a second playing time point, the target key frame being a key frame closest to the second playing time point. And finally, playing the target video at multiple speeds according to the target key frame, and controlling a display to display the video playing progress of the target video according to the display timestamp of the target key frame. According to the method, the display of the video playing progress is optimized by adjusting the display timestamp of the key frame.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and a video speed-up playback method. Background Art

[0002] Display devices refer to terminal devices that can output specific display images, which can be terminal devices such as smart TVs, communication terminals, smart advertising screens, projectors, etc. Taking smart TVs as an example, smart TVs are based on Internet application technology, have open operating systems and chips, have open application platforms, can realize two-way human-computer interaction functions, and are TV products that integrate multiple functions such as audio and video, entertainment, and data to meet the diverse and personalized needs of users.

[0003] When a display device plays a video, it can process the video at a double speed to achieve double speed playback. However, in the double speed video playback scenario, the video playback progress is displayed according to the display timestamp of the key frame in the video stream. There will be a problem that the display timestamp of the key frame does not match the actual video playback progress, resulting in a large difference between the display of the video playback progress and the actual set video speed rate, which reduces the user experience. Summary of the invention

[0004] The present application provides a display device and a video speed playback method to solve the problem of mismatch between the display of video playback progress and the actual set video speed rate.

[0005] In a first aspect, the present application provides a display device, including a display and a controller. The display is configured to display a user interface; the controller is configured to:

[0006] When playing the target video, in response to the video speed playback instruction, obtain the speed rate and the first playback time point, where the first playback time point is the current playback time point of the target video;

[0007] Calculate a second playback time point according to the speed ratio and the first playback time point, the second playback time point being a previous playback time point or a subsequent playback time point of the first playback time point when the target video is played at the speed ratio;

[0008] Extracting a target key frame from the target video, where the target key frame is the key frame closest to the second playback time point;

[0009] Setting the display timestamp of the target key frame to the second playback time point;

[0010] The target video is played at a multiple speed according to the target key frame, and the display is controlled to display the video playback progress of the target video according to the display timestamp of the target key frame.

[0011] The above technical solution has the following advantages or beneficial effects: by adjusting the distribution of key frames, the display timestamp of the key frames is made to correspond to the actual playback position when the video is sped up, so that the display of the video playback progress is proportional to the actual set speed rate.

[0012] In an optional implementation manner, after the controller executes the step of calculating the second playback time point according to the multiple speed rate and the first playback time point, the controller is further configured to:

[0013] Get the playback time range of the target video;

[0014] If the second playback time point exceeds the playback time range, end the playback of the target video;

[0015] If the second playback time point does not exceed the playback time range, a target key frame is extracted from the target video.

[0016] The above technical solution has the following advantages or beneficial effects: after calculating the second playback time point, check whether it exceeds the playback time range. When the second playback time point exceeds the playback time range, end the video playback to avoid invalid operations and waste of resources.

[0017] In an optional implementation manner, after executing the step of setting the display timestamp of the target key frame to the second playback time point, the controller is further configured to:

[0018] Store the target key frame into the cache queue;

[0019] Play the target video at the target key frame speed in the cache queue.

[0020] The above technical solution has the following advantages or beneficial effects: storing the target key frame in the cache queue and playing it at a multiple speed according to the target key frame in the cache queue can improve the fluency and continuity of video playback. The use of the cache queue can reduce the burden of real-time extraction and processing of key frames and improve the efficiency of video playback.

[0021] In an optional implementation, the controller executes storing the target key frame into a cache queue, and is further configured to:

[0022] Obtain the initial display timestamp of the target key frame and the initial display timestamp of the historical key frame, where the historical key frame is the key frame corresponding to the initial display timestamp closest to the first playback time point; the initial display timestamp is the display timestamp corresponding to the key frame when the target video is played at normal speed;

[0023] If the initial display timestamp of the history keyframe is different from the initial display timestamp of the target keyframe, the target keyframe is stored in the cache queue.

[0024] The above technical solution has the following advantages or beneficial effects: by comparing the initial display timestamps of the target key frame and the historical key frame, the same key frame can be avoided from being stored repeatedly, thus saving storage space and processing resources. At the same time, when the video is subsequently played at a multiple speed according to the key frame stored in the cache queue, the resource consumption for decoding the same key frame can be reduced.

[0025] In an optional embodiment, the controller is further configured to:

[0026] In response to a play instruction of a target video, a video play window and a play progress control are created, wherein a display level of the play progress control is higher than a display level of the video play window;

[0027] Control the display to display the video playback window and playback progress controls;

[0028] The target video is played through the video playing window, and the video playing progress of the target video is displayed through the playing progress control.

[0029] The above technical solution has the following advantages or beneficial effects: creating a video playback window and a playback progress control, and displaying the video playback progress through the playback progress control, providing an intuitive user interface, and facilitating users to understand the progress of video playback at any time. The display level of the playback progress control is higher than the video playback window, ensuring that users can clearly see the playback progress, and improving user experience.

[0030] In an optional implementation, the controller plays the target video at a multiple speed according to the target key frame, and is specifically configured as follows:

[0031] Decode the target keyframe;

[0032] The display is controlled to display the decoded target key frame through a video playback window.

[0033] The above technical solution has the following advantages or beneficial effects: by decoding the target key frame and displaying the decoded key frame, smooth picture output is achieved when the video is played at double speed.

[0034] In an optional implementation, the controller controls the display to display the video playback progress of the target video according to the display timestamp of the target key frame, and is specifically configured as follows:

[0035] Read the display timestamp of the target keyframe;

[0036] Update the video playback progress of the target video according to the display timestamp;

[0037] The display is controlled to display the updated video playback progress of the target video through the playback progress control.

[0038] The above technical solution has the following advantages or beneficial effects: by reading the display timestamp of the target key frame and updating the video playback progress, the real-time and accuracy of the video playback progress is ensured, and accurate playback progress information is provided to the user.

[0039] In an optional implementation manner, the controller calculates the second playback time point according to the multiple speed rate and the first playback time point, and is further configured to:

[0040] Detect video speed type;

[0041] If the video speed type is fast-forward playback, the second playback time point is the previous playback time point of the first playback time point when the target video is played at the speed rate;

[0042] If the video speed type is fast forward playback, the second playback time point is a playback time point after the first playback time point when the target video is played at the multiple speed rate.

[0043] The above technical solution has the following advantages or beneficial effects: by detecting the video speed type, the direction of the second playback time point is determined, and the speed playback direction is accurately controlled. At the same time, the corresponding target key frame is subsequently extracted according to the second playback time point to ensure the correctness of the video playback progress.

[0044] In an optional implementation manner, after the controller executes the step of calculating the second playback time point according to the multiple speed rate and the first playback time point, the controller is further configured to:

[0045] Separate the target video from audio and video to generate an audio stream and a video stream;

[0046] Extract target key frames in video stream;

[0047] Play the video stream at the target key frame speed, and do not play the audio stream.

[0048] The above technical solution has the following advantages or beneficial effects: separating the target video into audio and video, and playing the video stream at the target key frame speed instead of the audio stream, thereby avoiding the problem of obvious changes in the speech rate and tone of the audio data after the playback speed changes, which reduces the listening effect.

[0049] In a second aspect, the present application further provides a video speed playback method, which is applied to the above-mentioned display device, and the method includes:

[0050] When playing the target video, in response to the video speed playback instruction, obtain the speed rate and the first playback time point, where the first playback time point is the current playback time point of the target video;

[0051] Calculate a second playback time point according to the speed ratio and the first playback time point, the second playback time point being a previous playback time point or a subsequent playback time point of the first playback time point when the target video is played at the speed ratio;

[0052] Extracting a target key frame from the target video, where the target key frame is the key frame closest to the second playback time point;

[0053] Setting the display timestamp of the target key frame to the second playback time point;

[0054] The target video is played at a multiple speed according to the target key frame, and the display is controlled to display the video playback progress of the target video according to the display timestamp of the target key frame.

[0055] It can be seen from the above technical solutions that the present application provides a display device and a video speed playback method. The method can obtain the speed rate and the first playback time point in response to the video speed playback instruction when playing the target video. The first playback time point is the current playback time point of the target video. Then calculate the second playback time point based on the speed rate and the first playback time point. The second playback time point is the previous playback time point or the next playback time point when the target video is played at the speed rate. Extract the target key frame from the target video, and set the display timestamp of the target key frame to the second playback time point. The target key frame is the key frame closest to the second playback time point. Finally, play the target video at the speed of the target key frame, and control the display to display the video playback progress of the target video according to the display timestamp of the target key frame. The method adjusts the display timestamp of the key frame to realize the display of the video playback progress in direct proportion to the video speed rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 A schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

[0058] Figure 2 A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;

[0059] Figure 3 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0060] Figure 4 A schematic diagram of the architecture of video playback provided in some embodiments of the present application;

[0061] Figure 5 A schematic diagram of the distribution of video frame types in a video stream provided by some embodiments of the present application;

[0062] Figure 6 A schematic diagram of a video playback interface of a target video provided in some embodiments of the present application;

[0063] Figure 7 A schematic diagram of the distribution of key frames provided in some embodiments of the present application;

[0064] Figure 8 A schematic diagram of changes in video playback progress presented according to the initial distribution of key frames provided in some embodiments of the present application;

[0065] Fig. 9 A schematic diagram of a flow chart of a video speed playback method provided in some embodiments of the present application;

[0066] Fig.10 A schematic diagram of a speed selection menu provided in some embodiments of the present application;

[0067] Fig.11 A schematic diagram of the distribution of adjusted key frames provided in some embodiments of the present application;

[0068] Fig.12 A schematic diagram of changes in video playback progress presented according to the distribution of adjusted key frames provided in some embodiments of the present application;

[0069] Fig.13 A schematic diagram of key frame deduplication filtering provided in some embodiments of the present application;

[0070] Fig.14 A schematic diagram of the architecture of video speed playback provided in some embodiments of the present application;

[0071] Fig.15 A schematic diagram of a video speed playback process provided for some embodiments of the present application. DETAILED DESCRIPTION

[0072] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0073] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0074] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0075] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0076] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0077] In the embodiment of the present application, the display device 200 generally refers to a device with image display and data processing capabilities. For example, the display device 200 includes but is not limited to a smart TV, a mobile terminal, a computer, a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, etc.

[0078] Figure 1 This is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG. 1 , a user can operate the display device 200 through touch operation, the mobile terminal 300 and the control device 100. The control device 100 is used to receive operation instructions input by the user and convert the operation instructions into control instructions that can be recognized and responded to by the display device 200. For example, the control device 100 can be a remote controller, a stylus pen, a handle, etc.

[0079] The mobile terminal 300 can be used as a control device for performing human-computer interaction between the user and the display device 200. The mobile terminal 300 can also be used as a communication device for establishing a communication connection with the display device 200 and performing data interaction. In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.

[0080] In some embodiments, the mobile terminal 300 or other electronic devices may also simulate the functions of the control device 100 by running an application program for controlling the display device 200 .

[0081] like Figure 1 As also shown in FIG. 4 , the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0082] The display device 200 may provide a broadcast receiving television function, and may also additionally provide an intelligent network television function with a computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), and the like.

[0083] Figure 2 Some embodiments of the present application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0084] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0085] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0086] In some embodiments, the display 260 includes a display function component for presenting a picture, and a driving component for driving an image display. The display 260 is used to receive an image signal output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, and components of a menu control interface and a user control UI interface.

[0087] In some embodiments, the communication device 220 is a component for communicating with an external device or server 400 according to various communication protocol types. The display device 200 may be provided with a plurality of communication devices 220 according to different supported communication modes. For example, when the display device 200 supports wireless network communication, the display device 200 may be provided with a communication device 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including a Bluetooth function.

[0088] The communication device 220 can enable the display device 200 to communicate with the external device or server 400 by wireless or wired connection. Among them, the wired connection can connect the display device 200 with the external device through components such as data cables and interfaces. The wireless connection can connect the display device 200 with the external device through wireless signals or wireless networks. The display device 200 can establish a connection relationship with the external device directly, or indirectly establish a connection relationship through a gateway, a router, a connection device, etc.

[0089] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0090] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0091] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0092] In some embodiments, the audio output device 270 may be a local speaker of the display device 200, or may be an external audio output device of the display device 200. In particular, for the external audio output device of the display device 200, the display device 200 may also be provided with an external audio output terminal, and the audio output device may be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0093] In some embodiments, the user input interface 280 may be used to receive instructions from a user.

[0094] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling hardware resources and software resources in the display device 200. The operating system can control the display device to provide a user interface, for example, the operating system can directly control the display device to provide a user interface, or can provide a user interface by running an application program. The operating system also allows the user to interact with the display device 200.

[0095] It should be noted that the operating system may be a native operating system based on a specific operating platform, or a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for the display device.

[0096] The operating system can be divided into different modules or layers according to the functions implemented, such as Figure 3 As shown, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the system library layer and the kernel layer.

[0097] In some embodiments, the application layer is used to provide services and interfaces for applications so that the display device 200 can run applications and interact with users based on the applications. At least one application can be run in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. that come with the operating system; they can also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0098] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0099] like Figure 3As shown, in the embodiment of the present application, the application framework layer includes a view system, managers, content providers, etc., wherein the view system can design and implement the interface and interaction of the application, and the view system includes lists, grids, text boxes, buttons, etc. The manager includes at least one of the following modules: the activity manager is used to interact with all activities running in the system; the location manager is used to provide system services or applications with access to the system location service; the package manager is used to retrieve various information related to the application package currently installed on the device; the notification manager is used to control the display and clearing of notification messages; the window manager is used to manage icons, windows, toolbars, wallpapers and desktop components on the user interface.

[0100] In some embodiments, the activity manager is used to manage the life cycle of each application and the usual navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes, for example, reducing the display window, shaking the display, distorting the display, etc.

[0101] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system will run the instruction library contained in the system runtime layer, such as the C / C++ instruction library, to implement the functions to be implemented by the framework layer.

[0102] In some embodiments, the kernel layer is a functional layer between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. Figure 3 As shown, the kernel layer may be configured with hardware drivers, and the drivers included in the kernel layer may be at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0103] It should be noted that the above example is only a simple division of the operating system functions and does not constitute a limitation on the specific operating system form of the display device 200 in the embodiment of the present application. Depending on factors such as the function of the display device and the type of operating system, the number of levels and specific level types contained in the operating system may be expressed in other forms.

[0104] Based on the above, the display device 200 can present a specific screen. For example, a playback screen, a control interface, and other application interfaces. Depending on the developer or operator of the application, the application installed on the display device 200 may include various forms, such as system applications, third-party applications, etc. When the application list is displayed in the display 260, the user can select one of the applications and control the display device 200 to run the application to implement the function of the application.

[0105] In some embodiments, the display device 200 may be installed with a player application, which may play corresponding video files based on the video player in the framework layer to implement functions such as local video playback, online on-demand, and TV live broadcast. Figure 4 As shown, it is a schematic diagram of the architecture of video playback provided by an embodiment of the present application. During the video playback process, the video player of the display device 200 includes multiple modules, namely, a parsing module, a data caching module, a video decoding module, an audio decoding module, an audio-visual synchronization control module, a video rendering module, and an audio rendering module, through which a series of tasks such as obtaining video data from a media source, parsing, decoding, and audio-visual synchronization are performed.

[0106] Among them, the parsing module is used to obtain video files (film sources), separate the video data and audio data in the video files, and also obtain the encoding information such as the length, resolution, bit rate, sampling rate, etc. of the video files for subsequent playback.

[0107] The data cache module is used to cache the video data and audio data separated by the parsing module. At this time, the video data and audio data are undecoded elementary stream (es) data.

[0108] The video decoding module is used to obtain undecoded video es data from the data cache module, decode the video es data, and output decoded video data, such as data in YUV format, RGB format, and other data formats.

[0109] The audio decoding module is used to obtain undecoded audio es data from the data cache module, decode the audio es data, and output the decoded audio data, such as PCM format, WAV format, etc.

[0110] The audio and video synchronization control module is used to synchronize the decoded audio data and video data so that the decoded audio data and video data can be played synchronously. The audio and video synchronization control module can use the clock to perform clock correction according to the actual playback volume of the audio in the synchronization control algorithm based on audio processing (audio master), that is, the audio data stream is used as a reference to synchronize the video data stream.

[0111] The video rendering module is used to display the decoded video data on the screen of the display 260 to play the video.

[0112] The audio rendering module is used to output the decoded audio data to an audio output device 270 such as a speaker or earphone to play the audio.

[0113] In some embodiments, when the display device 200 plays a video, the video can be played at a multiple speed to achieve multiple speed playback of the video. Since a video file is composed of multiple continuous images, a frame is a basic unit that composes a video image. Video frames include multiple types, such as Figure 5 As shown, it is a schematic diagram of the distribution of video frame types in the video stream provided by the embodiment of the present application, including key frames (Intra-codedframe, I frame), predictive frames (Predictive-codedframe, P frame), bidirectional predictive frames (Bi-directional-codedframe, B frame), etc. It can be understood that the key frame is a coded frame compressed by the whole frame, and the full frame image information can be compressed, encoded and transmitted. When decoding, only the data of the key frame needs to be decoded to reconstruct the complete image. Therefore, the display device 200 can discard video frames such as bidirectional predictive frames and predictive frames in the video stream, and the video speed playback can be achieved only based on the key frames.

[0114] In some embodiments, when the display device plays the video at 200 times the speed, the speech speed and intonation of the audio data after the playback speed changes will also change significantly, thereby reducing the user's listening experience. Therefore, in order to improve the user experience, when the display device plays the target video at 200 times the speed, the target video can be separated into audio and video to generate an audio stream and a video stream, and the audio stream is discarded, and only the video stream is subsequently played at multiple times the speed.

[0115] In some embodiments, the display device 200 can achieve video speed playback through a seek operation. During video playback, the seek operation refers to jumping to the target time point of the video stream so that the video player can start playing the video from the target time point. The seek operation is implemented by finding key frames. When the display device 200 plays the target video at a speed, the direction of the seek operation can be set according to the video speed type, and the step length of the seek operation can be set according to the speed rate. For example, when the speed rate is 2x, the step length of the seek operation is set to 2s; when the speed rate is 4x, the step length of the seek operation is set to 4s; when the speed rate is 8x, the step length of the seek operation is set to 8s; when the speed rate is 16x, the step length of the seek operation is set to 16s. The seek operation locates the key frame closest to the target time point, so that the video player decodes and plays the key frame.

[0116] For example, taking 2x fast forward playback as an example, when the display device 200 plays the target video to the 2nd second, it receives a video speed playback instruction, sets the step size of the seek operation to 2s, and sets the direction of the seek operation to backward. If the content of the 4th second in the video stream needs to be played at the 3rd second, then the seek operation is used to jump backward 2s from the 2nd second as the starting point, jump to the 4th second, and locate the key frame closest to the 4th second, and decode and play the key frame. If the content of the 6th second in the video stream needs to be played at the 4th second, then the seek operation can be used to jump backward 2s from the 4th second as the starting point, jump to the 6th second, and locate the key frame closest to the 6th second, and decode and play the key frame, and so on, so as to achieve video speed playback.

[0117] In some embodiments, the display device 200 can achieve video speed playback by resetting the presentation time stamp (PTS) of the key frame in the video stream. During the video encoding process, the encoder assigns a presentation time stamp to each video frame, which is used to indicate the time when the video frame should be displayed. When the player plays the video, it reads the presentation time stamp of the video frame, and decodes and displays the video content of the video frame according to the display time specified by the presentation time stamp.

[0118] Therefore, when the display device plays the target video at 200 times the speed, the initial display timestamp of the key frame that needs to be played in the target video can be reset to the target display timestamp according to the speed rate, so that the target display timestamp meets the time interval required for the speed playback, and the key frame is played according to the adjusted target display timestamp.

[0119] The initial display timestamp is the display timestamp corresponding to the key frame when the target video is played at a normal speed, and the target display timestamp is the display timestamp corresponding to the key frame adjusted according to the speed rate of the target video.

[0120] For example, taking 4x fast forward playback as an example, the initial display timestamps of each key frame in the target video are 0 seconds, 1 second, 2 seconds, 3 seconds, etc. When playing at 4x speed, the interval between two adjacent key frames should be adjusted to 1 / 4 second. Then the display timestamp of the first key frame is 0, the display timestamp of the second key frame is adjusted to 0.25, the display timestamp of the third key frame is adjusted to 0.5, the display timestamp of the fourth key frame is adjusted to 0.75, and so on. That is, the number of key frames played per second changes from 1 at normal speed playback to 4 at 4x speed playback, thereby achieving video speed playback without discarding key frames.

[0121] In some embodiments, in order to improve user experience, when the display device 200 plays a video, the video playback progress of the player can be recorded. When the user inputs a display instruction for indicating the display of the video playback progress, the video playback progress can be obtained and the display 260 can be controlled to display the video playback progress according to a preset display method.

[0122] In some embodiments, the display device 200 may create a video playback window and a playback progress control in response to a playback instruction of the target video, and control the display 260 to display the video playback window and the playback progress control. The display level of the playback progress control is higher than the display level of the video playback window, and the display device 200 may play the target video through the video playback window, and display the video playback progress of the target video through the playback progress control. For example, Figure 6 As shown, it is a schematic diagram of the video playback interface of the target video provided in the embodiment of the present application. The playback progress control in the playback interface displays the video playback progress in the form of text and progress bar. The text "00:16 / 00:50" in the figure indicates that the video duration is 50s and the current playback is to the 16th second, so that the user can intuitively understand the playback progress information.

[0123] However, since the video playback progress is calculated based on the playback volume of the audio stream when the video is playing normally, and the audio stream is discarded in the video speed playback scenario, the video playback progress is displayed based on the display timestamp of the video frame in the video stream. However, in the above-mentioned implementation methods for realizing video speed, there is a problem that the display timestamp of the key frame does not match the actual video playback progress, which will cause a large difference between the display of the video playback progress and the actual set video speed rate, reducing the user experience. If the audio and video are also decoded in a variable speed but not invariant tone manner, higher decoding performance is required, and this method only supports low speeds, which is difficult to implement in high speed scenarios.

[0124] For example, when resetting the display timestamp of a key frame to achieve speed playback of a video, since the display timestamp of the key frame is adjusted directly according to the speed rate of the target video, if the video playback progress is displayed according to the adjusted display timestamp of the key frame, not only will the display of the video playback progress be not proportional to the actual speed rate of the video, but the video duration will also change.

[0125] For example, if the target video has a playback duration of 12 seconds and there are two key frames in the target video, at the 1st second and the 12th second respectively, when the video is fast-forwarded at 2x speed, the display timestamp of the key frame at the 1st second will be set to the 1st second, and the display timestamp of the key frame at the 12th second will be set to the 2nd second. When the video is fast-forwarded at 2x speed, the actual playback duration is only 2 seconds.

[0126] For another example, when playing a video at double the speed through a seek operation, since the actual position of the seek operation depends on the position of the key frames stored inside the source, the positions of the key frames stored in the video are not evenly distributed. If the video playback progress is displayed directly according to the display timestamp of the key frames, uneven progress will occur.

[0127] like Figure 7As shown, it is a schematic diagram of the distribution of key frames provided in an embodiment of the present application. The playback duration of the target video is 50s. Taking 16x fast forward playback as an example, the display device 200 will play the contents of the 16th, 32nd, and 48th seconds of the target video. When it is necessary to display the contents of the 16th second of the target video, the first key frame closest to 16s will be located through a seek operation, and the video playback progress displayed according to the display timestamp of the first key frame is the 14th second; when it is necessary to display the contents of the 32nd second of the target video, the second key frame closest to 32s will be located through a seek operation, and the video playback progress displayed according to the display timestamp of the second key frame is the 26th second; when it is necessary to display the contents of the 48th second of the target video, the third key frame closest to 48s will be located through a seek operation, and the video playback progress displayed according to the display timestamp of the third key frame is the 38th second. The effect changes of the implementation of the above-mentioned 16x fast forward playback on the interface are as follows: Figure 8 As shown, the video playback progress will jump from 14s to 26s, and then from 26s to 38s. The video playback progress is not displayed in direct proportion to the actual set speed rate.

[0128] In some embodiments, some embodiments of the present application provide a video speed playback method, which can adjust the distribution of key frames in a way that video speed playback is achieved based on a seek operation, and correspond the display timestamp of the key frames to the playback position that requires a seek operation, thereby achieving a display of the video playback progress that is proportional to the actual set speed rate. Among them, the display device 200 that can apply the video speed playback method includes at least: a display 260 and a controller 250. Among them, the display 260 is configured to display a user interface. As shown in FIG. Fig. 9 FIG. 1 is a flow chart of a method for playing a video at a multiple speed provided in an embodiment of the present application, which specifically includes the following steps:

[0129] S901: When playing a target video, in response to a video speed-up playback instruction, obtain a speed-up rate and a first playback time point.

[0130] The first playback time point is the current playback time point of the target video. Figure 6 The video playback interface shown in FIG. 1 includes a speed control. The display device 200 may display a video playback interface as shown in FIG. 1 in response to a click instruction of the speed control input by the user. Fig.10 The speed selection menu shown includes multiple speed options, such as 0.5 times, 1 times, 1.5 times, 2 times, 4 times, 8 times, 16 times, 32 times, etc., for the user to select. When the user selects the 16 times speed option, the display device 200 can generate a video speed playback instruction indicating a speed rate of 16 to perform 16 times playback processing of the target video.

[0131] S902: Calculate a second playback time point according to the multiple speed rate and the first playback time point.

[0132] The second playback time point is a playback time point before or after the first playback time point when the target video is played at a multiple speed rate.

[0133] In some embodiments, the display device 200 can detect the video speed type. If the video speed type is fast-forward playback, the second playback time point is the previous playback time point of the first playback time point when the target video is played at the speed rate. If the video speed type is fast-forward playback, the second playback time point is the next playback time point of the first playback time point when the target video is played at the speed rate.

[0134] For example, taking 2x speed as an example, set the step length of the seek operation to 2s, and the current playback time point of the target video is the 5th second, that is, the first playback time point is the 5th second. If the video speed type is fast forward playback, set the direction of the seek operation to backward, then the seek operation will jump backward 2s from the 5th second to the 7th second, and the playback position after the seek operation is the second playback time point, which is the 7th second. If the video speed type is fast reverse playback, set the direction of the seek operation to forward, then the seek operation will jump forward 2s from the 5th second to the 3rd second, and the playback position after the seek operation is the second playback time point, which is the 3rd second.

[0135] In some embodiments, the display device 200 can obtain the playback time range of the target video, detect whether the second playback time point exceeds the playback time range, and end the playback of the target video if the second playback time point exceeds the playback time range. If the second playback time point does not exceed the playback time range, perform subsequent video speed playback processing.

[0136] For example, if the target video's playback duration is 30 seconds, the corresponding playback time range is 0-30 seconds. If the calculated second playback time point does not exceed the range of 0-30 seconds, the subsequent video speed playback process continues. If the calculated second playback time point exceeds the range of 0-30 seconds, the playback of the target video ends.

[0137] S903: Extract target key frames from the target video.

[0138] The target key frame is the key frame closest to the second playback time point.

[0139] S904: Setting the display timestamp of the target key frame as the second playback time point.

[0140] The display device 200 can locate the key frame closest to the second playback time point through a seek operation, and modify the display timestamp of the key frame to the second playback time point to make the display timestamp of the key frame correspond to the position requiring the seek operation.

[0141] In some embodiments, the display timestamp of a key frame may be adjusted based on the following time correction algorithm:

[0142] pts_temp'=start_pos+gap*((pts_temp-start_pos) / gap+direction*1);

[0143] Among them, pts_temp is the display timestamp of the key frame before adjustment. pts_temp' is the display timestamp of the key frame after adjustment. start_pos is the starting position, that is, the first playback time point. gap is the step size of the seek operation. direction represents the direction of the seek operation. If the video speed type is fast-forward playback, direction is -1. If the video speed type is fast-forward playback, direction is 1.

[0144] For example, using Figure 7 Taking the distribution of key frames during 16x fast-forward playback as shown, the first key frame closest to 16s is located through a seek operation, the first key frame is extracted, and the display timestamp of the first key frame is changed from 14s to 16s; the second key frame closest to 32s is located through a seek operation, the second key frame is extracted, and the display timestamp of the second key frame is changed from 26s to 32s; the third key frame closest to 48s is located through a seek operation, the third key frame is extracted, and the display timestamp of the third key frame is changed from 38s to 48s.

[0145] like Fig.11 , which is a distribution diagram of the adjusted target display timestamps of the key frames provided in the embodiment of the present application. It can be seen that the adjusted target display timestamps of the key frames correspond to the actual playback positions that need to be sought.

[0146] S905: Play the target video at a multiple speed according to the target key frame, and control the display 260 to display the video playback progress of the target video according to the display timestamp of the target key frame.

[0147] In some embodiments, Figure 6As shown, after extracting the target key frame and modifying the display timestamp of the target key frame, the display device 200 decodes the target key frame and controls the display 260 to display the decoded target key frame through the video playback window. At the same time, the display timestamp of the target key frame can be read, the video playback progress of the target video can be updated according to the display timestamp, and the display 260 can be controlled to display the updated video playback progress of the target video through the playback progress control.

[0148] For example, using Fig.11 The target display timestamp example shown in the figure shows that the effect of the above 16x fast forward playback on the interface changes as follows: Fig.12 As shown, the video playback progress will jump from 16s to 32s, and then from 32s to 48s, so that the display of the video playback progress is proportional to the actual set speed rate.

[0149] In some embodiments, a target key frame closest to the second playback time point is located through a seek operation. There may be a situation where two adjacent seek operations locate the same target key frame. In order to reduce the resource consumption of decoding, the display device 200 can obtain the initial display timestamp of the target key frame and the initial display timestamp of the historical key frame. The historical key frame is the key frame corresponding to the initial display timestamp closest to the first playback time point, and the initial display timestamp is the display timestamp corresponding to the key frame when the target video is played at a normal speed.

[0150] Compare the initial display timestamp of the target key frame with the initial display timestamp of the historical key frame. If the initial display timestamp of the historical key frame is different from the initial display timestamp of the target key frame, it means that the target key frame and the historical key frame are not the same key frame, and the subsequent decoding operation is performed on the target key frame. If the initial display timestamp of the historical key frame is the same as the initial display timestamp of the target key frame, it means that the target key frame and the historical key frame are the same key frame, and the subsequent decoding operation is not performed on the target key frame.

[0151] For example, Fig.13As shown, it is a schematic diagram of the key frame deduplication filtering provided by the embodiment of the present application. Taking 4x fast forward playback as an example, the display device 200 can set the step size of the seek operation to 4s, and set the direction of the seek operation to backward. Jump to the 12th second through the seek operation, and locate the key frame 11 closest to the 12th second. Jump to the 16th second again through the seek operation, and locate the key frame I2 closest to the 16th second. Jump to the 20th second again through the seek operation, and locate the key frame I2 closest to the 20th second. Jump to the 24th second again through the seek operation, and locate the key frame I3 closest to the 24th second. At the same time, the display timestamp of the key frame can be adjusted according to the playback position of the seek operation, so that the display timestamp of the key frame corresponds to the playback position of the seek operation.

[0152] After adjusting the distribution of key frames, the display device 200 can perform deduplication filtering on the key frames. According to the initial display timestamp of the key frames, it can be determined that the key frames corresponding to the 16th and 20th positions are repeated and are both key frames I2. In this case, the decoding operation of the key frame I2 corresponding to the 20th position is canceled to obtain the final distribution of key frames.

[0153] That is, when the display device 200 needs to display the 16th second video content, it decodes and displays the key frame I2. When it needs to display the 20th second video content, it does not need to decode the key frame I2 again, and can directly display the decoded key frame I2.

[0154] In some embodiments, the display device 200 may set a cache queue for storing the target key frame. After setting the display timestamp of the target key frame to the second play time point, the display device 200 may store the target key frame in the cache queue and play the target video at the target key frame speed in the cache queue.

[0155] Accordingly, the display device 200 compares the initial display timestamp of the target key frame with the initial display timestamp of the historical key frame, and if the initial display timestamp of the historical key frame is different from the initial display timestamp of the target key frame, the target key frame is stored in the cache queue. If the initial display timestamp of the historical key frame is the same as the initial display timestamp of the target key frame, the target key frame is discarded and is not stored in the cache queue.

[0156] In some embodiments, Fig.14 As shown, it is a schematic diagram of the architecture of the video speed playback provided by the embodiment of the present application. Figure 4 Based on the architecture of the video player shown in FIG. 1 , a video speed-up module is added between the parsing module and the data cache module to perform video speed-up processing. Fig.15 As shown, Fig.14The video player architecture of the present application is used to illustrate the video speed playback process provided in the embodiment of the present application.

[0157] When the player application plays the target video based on the video player, it can be based on Figure 6 The video playback interface shown interacts with the user, receives the video speed playback instruction input by the user, and sends an identifier representing the video speed type and speed rate to the parsing module in response to the video speed playback instruction. For example, the identifier "1" indicates that the video speed type is fast forward playback, the identifier "-1" indicates that the video speed type is fast rewind playback, and the identifier "16" indicates that the speed rate is 16 times the speed.

[0158] The parsing module sets the direction of the seek operation according to the video speed type and the step size of the seek operation according to the speed rate. The seek operation is performed starting from the current playback time point, and the seek operation is detected whether it is out of bounds, that is, whether the playback time point after the seek operation (the second playback time point) exceeds the playback time range of the target video. If it exceeds the playback time range, the playback of the target video is terminated. If it does not exceed the playback time range, the target key frame closest to the second playback time point located by the seek operation is extracted, and the target key frame is sent to the video speed module.

[0159] It is understandable that in order to improve the user's viewing experience, the parsing module separates the audio stream and video stream in the target video, discards the audio stream, and only sends the target key frame in the video stream located by the seek operation to the video speed module, so that when the video is played at double speed, only the video stream is played at double speed, and the audio stream is not played.

[0160] The video speed module receives the target key frame, modifies the display timestamp of the target key frame to the second playback time point, and sends the modified target key frame to the data cache module.

[0161] The data cache module receives the target key frame and determines whether the currently received target key frame is the same as the previously received target key frame based on the initial display timestamp of the target key frame. If they are the same, the currently received target key frame is discarded; if they are not the same, the target key frame is stored in the cache queue.

[0162] The video decoding module reads the target key frame from the cache queue in the data cache module, decodes the target key frame, and outputs the decoded video data. The video rendering module displays the decoded video data on the screen of the display 260 to achieve double-speed playback of the target video.

[0163] When the user wants to restore the normal playback speed of the video, the normal playback instruction can be input. The player application responds to the normal playback instruction and sends an instruction to the parsing module to cancel the execution of the seek operation. The parsing module cancels the execution of the seek operation and plays the video according to the normal video playback process from the current playback time point. That is, the video data and audio data in the video file are parsed and sent to the data cache module. The video decoding module obtains the video data from the data cache module for decoding. The audio decoding module obtains the audio data from the data cache module for decoding. The audio and video synchronization control module synchronizes the decoded audio data and video data. Finally, the video rendering module displays the decoded video data on the screen of the display 260, and the audio rendering module outputs the decoded audio data to the audio output device 270, thereby switching the target video from the double-speed playback back to normal playback.

[0164] Based on the above-mentioned video speed playback method, some embodiments of the present application provide a display device 200. It includes: a display 260 and a controller 250. The display 260 is configured to display a user interface. The controller 250 is configured to:

[0165] When playing the target video, in response to the video speed playback instruction, the speed rate and the first playback time point are obtained. The first playback time point is the current playback time point of the target video.

[0166] The second playback time point is calculated according to the speed ratio and the first playback time point, and the second playback time point is a previous playback time point or a subsequent playback time point of the first playback time point when the target video is played at the speed ratio.

[0167] A target key frame is extracted from the target video, where the target key frame is a key frame closest to the second playback time point.

[0168] Set the display timestamp of the target keyframe to the second playback time point.

[0169] The target video is played at a multiple speed according to the target key frame, and the display 260 is controlled to display the video playback progress of the target video according to the display timestamp of the target key frame.

[0170] The same and similar parts between the various embodiments in this specification can be referenced to each other and will not be described again here.

[0171] It can be seen from the above technical scheme that the display device and the video speed playback method provided by the above embodiment can obtain the speed rate and the first playback time point in response to the video speed playback instruction when playing the target video. The first playback time point is the current playback time point of the target video. Then calculate the second playback time point according to the speed rate and the first playback time point. The second playback time point is the previous playback time point or the next playback time point when the target video is played at the speed rate. Extract the target key frame from the target video, and set the display timestamp of the target key frame to the second playback time point. The target key frame is the key frame closest to the second playback time point. Finally, play the target video according to the speed of the target key frame, and control the display to display the video playback progress of the target video according to the display timestamp of the target key frame. The method adjusts the display timestamp of the key frame to realize the display of the video playback progress in direct proportion to the video speed rate.

[0172] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention or certain parts of the embodiments.

[0173] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0174] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A display device, characterized in that: include: a display configured to display a user interface; The controller is configured as: When playing the target video, in response to a video speed playback instruction, obtaining a speed rate and a first playback time point, where the first playback time point is a current playback time point of the target video; Calculate a second playback time point according to the speed rate and the first playback time point, the second playback time point being a previous playback time point or a subsequent playback time point of the first playback time point when the target video is played at the speed rate; Extracting a target key frame from the target video, the target key frame being a key frame closest to the second playback time point; Setting the display timestamp of the target key frame as the second playback time point; The target video is played at a multiple speed according to the target key frame, and the display is controlled to display the video playback progress of the target video according to the display timestamp of the target key frame.

2. The display device according to claim 1, characterized in that After the controller executes the step of calculating the second playback time point according to the multiple speed rate and the first playback time point, the controller is further configured to: Obtaining the playback time range of the target video; If the second playback time point exceeds the playback time range, ending the playback of the target video; If the second playback time point does not exceed the playback time range, a target key frame is extracted from the target video.

3. The display device according to claim 1, characterized in that After executing the step of setting the display timestamp of the target key frame to the second playback time point, the controller is further configured to: Storing the target key frame in a cache queue; The target video is played at a speed multiple of the target key frames in the cache queue.

4. The display device according to claim 3, characterized in that The controller executes storing the target key frame into the cache queue, and is further configured to: Obtaining an initial display timestamp of a target key frame and an initial display timestamp of a historical key frame, wherein the historical key frame is a key frame corresponding to an initial display timestamp closest to the first playback time point; The initial display timestamp is the display timestamp corresponding to the key frame when the target video is played at a normal speed; If the initial display timestamp of the historical key frame is different from the initial display timestamp of the target key frame, the target key frame is stored in the cache queue.

5. The display device according to claim 1, characterized in that The controller is also configured to: In response to a play instruction of a target video, a video play window and a play progress control are created, wherein a display level of the play progress control is higher than a display level of the video play window; Controlling the display to display the video playback window and the playback progress control; The target video is played through the video playback window, and the video playback progress of the target video is displayed through the playback progress control.

6. The display device according to claim 5, characterized in that The controller plays the target video at a multiple speed according to the target key frame, and is specifically configured as follows: Decoding the target key frame; The display is controlled to display the decoded target key frame through the video playback window.

7. The display device according to claim 5, characterized in that The controller controls the display to display the video playback progress of the target video according to the display timestamp of the target key frame, and is specifically configured as follows: Reading the display timestamp of the target key frame; Update the video playback progress of the target video according to the display timestamp; The display is controlled to display the updated video playback progress of the target video through the playback progress control.

8. The display device according to claim 1, characterized in that The controller is configured to calculate the second playback time point according to the multiple speed rate and the first playback time point, and is further configured to: Detect video speed type; If the video speed type is fast-forward playback, the second playback time point is a playback time point before the first playback time point when the target video is played at the speed rate; If the video speed type is fast forward playback, the second playback time point is a playback time point after the first playback time point when the target video is played at the speed rate.

9. The display device according to claim 1, characterized in that: After the controller executes the step of calculating the second playback time point according to the multiple speed rate and the first playback time point, the controller is further configured to: Separating the target video into audio and video to generate an audio stream and a video stream; Extracting a target key frame from the video stream; The video stream is played at a speed multiple according to the target key frame, and the audio stream is not played.

10. A video speed playback method, characterized in that: The method comprises: When playing the target video, in response to a video speed playback instruction, obtaining a speed rate and a first playback time point, where the first playback time point is a current playback time point of the target video; Calculate a second playback time point according to the speed rate and the first playback time point, the second playback time point being a previous playback time point or a subsequent playback time point of the first playback time point when the target video is played at the speed rate; Extracting a target key frame from the target video, the target key frame being a key frame closest to the second playback time point; Setting the display timestamp of the target key frame as the second playback time point; The target video is played at a multiple speed according to the target key frame, and a display is controlled to display the video playback progress of the target video according to the display timestamp of the target key frame.

Images