Implementation method of HLS (HTTP Live Streaming) video-on-demand player supporting reverse playing and player

Abstract

The invention discloses an implementation method of an HLS (HTTP Live Streaming) on-demand player supporting reverse playing and the player, belongs to the technical field of front-end client development, and aims to solve the technical problem of realizing smooth video reverse playing, speed-multiplying playing and free switching between forward playing and reverse playing on the premise of low resource occupation. Comprising the steps that a playing control mechanism is set for a player, a preloading mechanism is set for the player, a resource optimization mechanism is set for the player, in the preloading mechanism, RAW data, stored in the past direction, of GOP is optimized so as to reduce memory occupation, overall playing function implementation is guaranteed through the playing control mechanism, playing fluency is guaranteed through the preloading mechanism, and the resource optimization mechanism is set for the player. And low resource occupation in the playing process is guaranteed through a resource optimization mechanism.

Description

Technical Field

[0001] The present invention relates to the technical field of front-end client development, and in particular to an implementation method and a player of an HLS on-demand player supporting reverse playback. Background Art

[0002] As a playback protocol natively supported by the Web, HLS has inherent advantages in the promotion of remote audio and video playback. In fields such as security, there has long been a demand for double-speed playback and reverse playback.

[0003] Traditional HLS playback components only support playback control methods such as normal playback, multiple playback, and pause, but do not provide support for reverse playback scenarios such as normal reverse playback and multiple reverse playback. A playback control method with low resource usage and the ability to achieve smooth video reverse playback, multiple-speed reverse playback, and free switching between forward and reverse playback is particularly necessary.

[0004] Under the premise of low resource usage, achieving smooth video reverse playback, double-speed playback, and free switching between forward and reverse playback is a technical problem that needs to be solved. Summary of the invention

[0005] The technical task of the present invention is to address the above shortcomings and provide an HLS on-demand player implementation method and player that supports reverse playback, so as to solve the technical problems of achieving smooth video reverse playback, double-speed playback, and free switching between forward and reverse playback under the premise of low resource occupancy.

[0006] In a first aspect, the present invention provides a method for implementing an HLS on-demand player supporting reverse playback, comprising the following steps:

[0007] Set up a playback control mechanism for the player: take the start time of the recording as the reference point, calculate the relative timestamp of each frame, and calculate the current playback time point based on the playback parameters and the negative playback speed parameter to be compatible with reverse playback. Control the entire playback progress by comparing the relative timestamp of the frame with the current playback time point. The playback parameters include the playback state, playback direction and speed, and the actual elapsed time.

[0008] Set up a preloading mechanism for the player: when playing forward, preload a segment file in the future direction, pre-decode one frame in advance, and keep two segments and one GOP cropped RAW data in the past direction; when playing backward, pre-load two segment files in the past direction, pre-decode and crop one GOP RAW data in advance, and keep one segment in the future direction;

[0009] Set up a resource optimization mechanism for the player: In the preloading mechanism, optimize the RAW data of the GOP stored in the past direction to reduce memory usage.

[0010] Preferably, the resource optimization mechanism set for the player includes:

[0011] Activate when in use: Since the reverse playback function is used much less frequently than the normal playback function, the GOP cache in the past direction is enabled when the reverse playback mode is first entered;

[0012] Data cropping: scaling the RAW data in the GOP in both width and height dimensions to the minimum size that can meet the clarity of the display area, so as to reduce the amount of data;

[0013] Frame extraction mechanism: Since people perceive the frame rate lower during reverse playback than during forward playback, the GOP cache data is extracted based on user configuration.

[0014] Preferably, the player includes three workflows: a main workflow, a download workflow and a real workflow;

[0015] Provides playback status initialization, user operation response, playback direction judgment, frame data check, timestamp calculation, frame display and cache management through the main workflow;

[0016] The download workflow provides segment download requirements in forward and reverse modes, and continuously checks user operations and decoding status during segment download. It also performs a short sleep before the decoding workflow is completed to reduce resource usage.

[0017] The decoding workflow provides decoding requirements in forward playback and reverse playback modes, continuously checks user operations and file end status during the decoding process, and performs a short sleep when the file is not finished to reduce resource usage.

[0018] Preferably, the main workflow includes the following steps:

[0019] a. Initialize the playback state to forward playback, initialize the playback speed, and set the current playback time to the current frame timestamp;

[0020] b. Determine whether the user has finished playing. If the user has finished playing, the workflow ends; otherwise, proceed to the next step;

[0021] c. Determine the current playback direction. If it is forward playback, jump to step d; otherwise, jump to step g.

[0022] d. If there is no next frame of data and the file ends, end the workflow, otherwise proceed to the next step;

[0023] e. Recalculate the current timestamp. If the current timestamp is greater than or equal to the timestamp of the next frame, display the next frame, remove the next frame from the future RAW cache, add it to the past RAW cache, and then jump to step b, otherwise proceed to the next step;

[0024] f. Sleep for a predetermined time and jump to step b;

[0025] g. If there is no next frame of data and the file ends, the workflow ends, otherwise proceed to the next step;

[0026] h. Recalculate the current timestamp. If the current timestamp is less than or equal to the timestamp of the next frame, display the next frame, remove the next frame from the past RAW cache, add it to the future RAW cache, and then jump to step b, otherwise go to the next step;

[0027] i. Sleep for a predetermined time and jump to step b.

[0028] 5. The method for implementing an HLS on-demand player supporting reverse playback according to claim 4, wherein the current timestamp is calculated as follows:

[0029] Current timestamp = previous current timestamp + actual interval time * speed

[0030] The speed is negative in reverse playback mode.

[0031] Preferably, the download workflow includes the following operations:

[0032] a. Determine whether the user stops playing. If the user stops playing, the workflow ends. Otherwise, determine whether the current mode is reverse playback. If it is reverse playback mode, execute step c. Otherwise, execute step b.

[0033] b. Determine whether the next segment in the future direction of the current playback position has been downloaded. If not, download the segment and jump to step a; otherwise, jump to step d;

[0034] c. Determine whether the next segment in the direction past the current playback position has been downloaded. If not, download the segment and jump to step a; otherwise, jump to step d.

[0035] d. Determine whether the current decoding workflow is finished. If the decoding workflow is finished, then end the downloading workflow. Otherwise, jump to step a after sleeping for a predetermined time.

[0036] Preferably, the decoding workflow includes the following operations:

[0037] a. Determine whether the user stops playing. If the user stops playing, the workflow ends. Otherwise, determine whether the current mode is reverse playback. If it is reverse playback mode, execute step c. Otherwise, execute step b.

[0038] b. Determine whether the undisplayed RAW data frame in the future direction is a full frame and whether there is an undecoded data frame. If so, decode and crop the next coded frame, add it to the future direction RAW frame list, and then jump to step a. Otherwise, skip this step and jump directly to step d.

[0039] c. Determine the amount of RAW data cached in the past direction and whether the previous GOP fragment data has been downloaded. If the RAW data is less than 1 GOP and the previous GOP fragment data has been downloaded, decode the next GOP data and add it to the RAW data cache in the past direction after cropping. Otherwise, skip this step and jump directly to step d.

[0040] d. Determine whether the file is finished. If the file is finished, end the workflow. Otherwise, sleep for 10ms and jump to step a.

[0041] Preferably, when cropping RAW data, only image scaling by a power of 2 is performed to ensure that scaling only requires integer addition and shift operations. The width and height after scaling should be greater than or equal to the width and height of the display area to ensure that display clarity is not affected.

[0042] Preferably, when extracting RAW data frames, each time a new RAW frame is added to the past direction RAW cache, if the number of frames in the cache pool is greater than or equal to a predetermined number of frames, and the timestamps of the last frame and the third-to-last frame are less than a predetermined time, the second-to-last frame is removed and discarded, otherwise no processing is performed.

[0043] In a second aspect, the present invention provides an HLS on-demand player that supports reverse playback, wherein the player is provided with a playback control mechanism, a preloading mechanism, and a resource optimization mechanism set by an implementation method of an HLS on-demand player that supports reverse playback as described in any one of the first aspects.

[0044] The HLS on-demand player implementation method supporting reverse playback and the player of the present invention have the following advantages: the overall playback function is guaranteed by a playback control mechanism, the smoothness of the playback is guaranteed by a preloading mechanism, and the low resource occupancy during the playback process is guaranteed by a resource optimization mechanism. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] The present invention is further described below in conjunction with the accompanying drawings.

[0047] Figure 1This is a forward playback data flow diagram in a method for implementing an HLS on-demand player supporting reverse playback in Example 1;

[0048] Figure 2 This is a reverse playback data flow diagram in a method for implementing an HLS on-demand player that supports reverse playback in Example 1. DETAILED DESCRIPTION

[0049] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments may be combined with each other.

[0050] The embodiment of the present invention provides an HLS on-demand player implementation method and player that supports reverse playback, which are used to solve the technical problem of achieving smooth video reverse playback, double-speed playback, and free switching between forward and reverse playback under the premise of low resource occupation.

[0051] Embodiment 1:

[0052] The present invention discloses a method for realizing an HLS on-demand player supporting reverse playback, comprising three operations of setting a playback control mechanism for the player, setting a preloading mechanism for the player, and setting a resource optimization mechanism for the player.

[0053] Set up a playback control mechanism for the player: take the start time of the recording as the reference point, calculate the relative timestamp of each frame, calculate the current playback time point based on the playback parameters and the negative playback speed parameter to be compatible with reverse playback, and control the entire playback progress by comparing the relative timestamp of the frame with the current playback time point. The playback parameters include the playback status, playback direction and speed, and the actual elapsed time.

[0054] Set up a preloading mechanism for the player: When playing forward, preload a segment file in the future direction, pre-decode one frame in advance, and retain two segments and a GOP of cropped RAW data in the past direction. When playing in reverse, load two segment files in the past direction, pre-decode and crop a GOP of RAW data in advance, and retain one segment in the future direction.

[0055] Set up a resource optimization mechanism for the player: In the preloading mechanism, optimize the RAW data of the GOP stored in the past direction to reduce memory usage.

[0056] Among them, the resource optimization mechanism set for the player includes the following three mechanisms:

[0057] (1) Activation when in use: Since the reverse playback function is used much less frequently than the normal playback function, the GOP cache is used in the past and is enabled when the reverse playback mode is first entered;

[0058] (2) Data cropping: scaling the RAW data in the GOP in both width and height dimensions to the minimum size that can meet the clarity of the display area, thereby reducing the amount of data;

[0059] (3) Frame extraction mechanism: Since people perceive the frame rate lower during reverse playback than during forward playback, the GOP cache data is extracted based on user configuration.

[0060] The player of this embodiment includes three workflows: main workflow, download workflow and real workflow. The main workflow provides playback status initialization, user operation response, playback direction judgment, frame data check, timestamp calculation, frame display and cache management;

[0061] The download workflow provides segment download requirements in forward and reverse modes, and continuously checks user operations and decoding status during segment download. It also performs a short sleep before the decoding workflow is completed to reduce resource usage.

[0062] The decoding workflow provides decoding requirements in forward playback and reverse playback modes, continuously checks user operations and file end status during the decoding process, and performs a short sleep when the file is not finished to reduce resource usage.

[0063] The main workflow includes the following steps:

[0064] a. Initialize the playback state to forward playback, initialize the playback speed, and set the current playback time to the current frame timestamp;

[0065] b. Determine whether the user has finished playing. If the user has finished playing, the workflow ends; otherwise, proceed to the next step;

[0066] c. Determine the current playback direction. If it is forward playback, jump to step d; otherwise, jump to step g.

[0067] d. If there is no next frame of data and the file ends, end the workflow, otherwise proceed to the next step;

[0068] e. Recalculate the current timestamp. If the current timestamp is greater than or equal to the timestamp of the next frame, display the next frame, remove the next frame from the future RAW cache, add it to the past RAW cache, and then jump to step b, otherwise proceed to the next step;

[0069] f. Sleep for a predetermined time and jump to step b;

[0070] g. If there is no next frame of data and the file ends, the workflow ends, otherwise proceed to the next step;

[0071] h. Recalculate the current timestamp. If the current timestamp is less than or equal to the timestamp of the next frame, display the next frame, remove the next frame from the past RAW cache, add it to the future RAW cache, and then jump to step b, otherwise go to the next step;

[0072] i. Sleep for a predetermined time and jump to step b.

[0073] The current timestamp calculation method is as follows:

[0074] Current timestamp = previous current timestamp + actual interval time * speed

[0075] The speed is negative in reverse playback mode.

[0076] The download workflow includes the following operations:

[0077] a. Determine whether the user stops playing. If the user stops playing, the workflow ends. Otherwise, determine whether the current mode is reverse playback. If it is reverse playback mode, execute step c. Otherwise, execute step b.

[0078] b. Determine whether the next segment in the future direction of the current playback position has been downloaded. If not, download the segment and jump to step a; otherwise, jump to step d;

[0079] c. Determine whether the next segment in the direction past the current playback position has been downloaded. If not, download the segment and jump to step a; otherwise, jump to step d.

[0080] d. Determine whether the current decoding workflow is finished. If the decoding workflow is finished, then end the downloading workflow. Otherwise, jump to step a after sleeping for a predetermined time.

[0081] The decoding workflow includes the following operations:

[0082] a. Determine whether the user stops playing. If the user stops playing, the workflow ends. Otherwise, determine whether the current mode is reverse playback. If it is reverse playback mode, execute step c. Otherwise, execute step b.

[0083] b. Determine whether the undisplayed RAW data frame in the future direction is a full frame and whether there is an undecoded data frame. If so, decode and crop the next coded frame, add it to the future direction RAW frame list, and then jump to step a. Otherwise, skip this step and jump directly to step d.

[0084] c. Determine the amount of RAW data cached in the past direction and whether the previous GOP fragment data has been downloaded. If the RAW data is less than 1 GOP and the previous GOP fragment data has been downloaded, decode the next GOP data and add it to the RAW data cache in the past direction after cropping. Otherwise, skip this step and jump directly to step d.

[0085] d. Determine whether the file is finished. If the file is finished, end the workflow. Otherwise, sleep for 10ms and jump to step a.

[0086] When cropping RAW data, only image scaling by powers of 2 is performed to ensure that scaling only requires integer addition and shift operations. The width and height after scaling should be greater than or equal to the width and height of the display area to ensure that display clarity is not affected.

[0087] When extracting RAW data frames, each time a new RAW frame is added to the RAW cache in the past direction, if the number of frames in the cache pool is greater than or equal to the predetermined number of frames, and the timestamps of the last frame and the third-to-last frame are less than the predetermined time, the second-to-last frame is removed and discarded, otherwise no processing is performed.

[0088] The method of this embodiment, such as Figure 1 As shown in the data flow diagram, during forward playback, a segment file is preloaded in the future direction, one frame is pre-decoded in advance, and two segments and a GOP cropped RAW data are retained in the past direction. Figure 2 As shown in the data flow diagram, when playing in reverse, two segment files are loaded in advance in the past direction, the RAW data of a GOP is pre-decoded and cropped in advance, and one segment is reserved in the future direction. The method of this embodiment ensures the realization of the overall playback function through a unique playback control mechanism; ensures the smoothness of playback through a unique preloading mechanism; and ensures low resource usage during playback through a unique resource optimization mechanism.

[0089] Embodiment 2:

[0090] The present invention discloses an HLS on-demand player supporting reverse playback, wherein the player is provided with a playback control mechanism, a preloading mechanism, and a resource optimization mechanism set by the method disclosed in Example 1.

[0091] The above is a detailed introduction to the HLS on-demand player implementation method and player supporting reverse playback provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for implementing an HLS on-demand player that supports reverse playback, characterized in that: The steps include: Set up a playback control mechanism for the player: take the start time of the recording as the reference point, calculate the relative timestamp of each frame, and calculate the current playback time point based on the playback parameters and the negative playback speed parameter to be compatible with reverse playback. Control the entire playback progress by comparing the relative timestamp of the frame with the current playback time point. The playback parameters include the playback state, playback direction and speed, and the actual elapsed time. Set up a preloading mechanism for the player: when playing forward, preload a segment file in the future direction, pre-decode one frame in advance, and keep two segments and one GOP cropped RAW data in the past direction; when playing backward, pre-load two segment files in the past direction, pre-decode and crop one GOP RAW data in advance, and keep one segment in the future direction; Set up a resource optimization mechanism for the player: In the preloading mechanism, optimize the RAW data of the GOP stored in the past direction to reduce memory usage.

2. The method for implementing an HLS on-demand player supporting reverse playback according to claim 1, characterized in that: The resource optimization mechanisms set for the player include: Activate when in use: Since the reverse playback function is used much less frequently than the normal playback function, the GOP cache in the past direction is enabled when the reverse playback mode is first entered; Data cropping: scaling the RAW data in the GOP in both width and height dimensions to the minimum size that can meet the clarity of the display area, so as to reduce the amount of data; Frame extraction mechanism: Since people perceive the frame rate lower during reverse playback than during forward playback, the GOP cache data is extracted based on user configuration.

3. The method for implementing an HLS on-demand player supporting reverse playback according to claim 1 or 2, characterized in that: The player includes three workflows: main workflow, download workflow and real workflow; Provides playback status initialization, user operation response, playback direction judgment, frame data check, timestamp calculation, frame display and cache management through the main workflow; The download workflow provides segment download requirements in forward and reverse modes, and continuously checks user operations and decoding status during segment download. It also performs a short sleep before the decoding workflow is completed to reduce resource usage. The decoding workflow provides decoding requirements in forward playback and reverse playback modes, continuously checks user operations and file end status during the decoding process, and performs a short sleep when the file is not finished to reduce resource usage.

4. The method for implementing an HLS on-demand player supporting reverse playback according to claim 3, characterized in that: The main workflow includes the following steps: a. Initialize the playback state to forward playback, initialize the playback speed, and set the current playback time to the current frame timestamp; b. Determine whether the user has finished playing. If the user has finished playing, the workflow ends; otherwise, proceed to the next step; c. Determine the current playback direction. If it is forward playback, jump to step d; otherwise, jump to step g. d. If there is no next frame of data and the file ends, end the workflow, otherwise proceed to the next step; e. Recalculate the current timestamp. If the current timestamp is greater than or equal to the timestamp of the next frame, display the next frame, remove the next frame from the future RAW cache, add it to the past RAW cache, and then jump to step b, otherwise proceed to the next step; f. Sleep for a predetermined time and jump to step b; g. If there is no next frame of data and the file ends, the workflow ends, otherwise proceed to the next step; h. Recalculate the current timestamp. If the current timestamp is less than or equal to the timestamp of the next frame, display the next frame, remove the next frame from the past RAW cache, add it to the future RAW cache, and then jump to step b, otherwise go to the next step; i. Sleep for a predetermined time and jump to step b.

5. The method for implementing an HLS on-demand player supporting reverse playback according to claim 4, characterized in that: The current timestamp is calculated as follows: Current timestamp = previous current timestamp + actual interval time * speed The speed is negative in reverse playback mode.

6. The method for implementing an HLS on-demand player supporting reverse playback according to claim 4, characterized in that: The download workflow includes the following operations: a. Determine whether the user stops playing. If the user stops playing, the workflow ends. Otherwise, determine whether the current mode is reverse playback. If it is reverse playback mode, execute step c. Otherwise, execute step b. b. Determine whether the next segment in the future direction of the current playback position has been downloaded. If not, download the segment and jump to step a; otherwise, jump to step d; c. Determine whether the next segment in the direction past the current playback position has been downloaded. If not, download the segment and jump to step a; otherwise, jump to step d. d. Determine whether the current decoding workflow is finished. If the decoding workflow is finished, then end the downloading workflow. Otherwise, jump to step a after sleeping for a predetermined time.

7. The method for implementing an HLS on-demand player supporting reverse playback according to claim 4, characterized in that: The decoding workflow includes the following operations: a. Determine whether the user stops playing. If the user stops playing, the workflow ends. Otherwise, determine whether the current mode is reverse playback. If it is reverse playback mode, execute step c. Otherwise, execute step b. b. Determine whether the undisplayed RAW data frame in the future direction is a full frame and whether there is an undecoded data frame. If so, decode and crop the next coded frame, add it to the future direction RAW frame list, and then jump to step a. Otherwise, skip this step and jump directly to step d. c. Determine the amount of RAW data cached in the past direction and whether the previous GOP fragment data has been downloaded. If the RAW data is less than 1 GOP and the previous GOP fragment data has been downloaded, decode the next GOP data and add it to the RAW data cache in the past direction after cropping. Otherwise, skip this step and jump directly to step d. d. Determine whether the file is finished. If the file is finished, end the workflow. Otherwise, sleep for 10ms and jump to step a.

8. The method for implementing an HLS on-demand player supporting reverse playback according to claim 7, characterized in that: When cropping RAW data, only image scaling by powers of 2 is performed to ensure that scaling only requires integer addition and shift operations. The width and height after scaling should be greater than or equal to the width and height of the display area to ensure that display clarity is not affected.

9. The method for implementing an HLS on-demand player supporting reverse playback according to claim 7, characterized in that: When extracting RAW data frames, each time a new RAW frame is added to the RAW cache in the past direction, if the number of frames in the cache pool is greater than or equal to the predetermined number of frames, and the timestamps of the last frame and the third-to-last frame are less than the predetermined time, the second-to-last frame is removed and discarded, otherwise no processing is performed.

10. An HLS on-demand player supporting reverse playback, characterized in that: The player is provided with a playback control mechanism, a preloading mechanism and a resource optimization mechanism set by a method for implementing an HLS on-demand player supporting reverse playback as described in any one of claims 1-9.

Images