Hls on-demand player implementation method supporting upside-down playing and player

Abstract

The application discloses an HLS on-demand player implementation method and player supporting reverse playing, and belongs to the technical field of front-end client development, and aims to solve the technical problem of realizing fluent video reverse playing, speed playing and free switching between normal playing and reverse playing under the premise of low resource occupation. The method comprises the following steps: setting a playing control mechanism for the player, setting a preloading mechanism for the player, and setting a resource optimization mechanism for the player. In the preloading mechanism, the RAW data of the GOP stored in the past direction is optimized to reduce memory occupation. The playing control mechanism guarantees the overall function realization of playing, the preloading mechanism guarantees the fluency of playing, and the resource optimization mechanism guarantees low resource occupation in the playing process.

Description

Technical Field

[0001] The present invention relates to the technical field of front-end client development, and in particular to an implementation method and a player of an HLS on-demand player supporting reverse playback. Background Art

[0002] As a playback protocol natively supported by the Web, HLS has inherent advantages in remote audio and video playback. In fields such as security, there has long been demand for double-speed playback and reverse playback.

[0003] Traditional HLS playback components only support playback control methods such as normal playback, multiple playback, and pause, and do not provide support for reverse playback scenarios such as normal reverse playback and multiple reverse playback. A playback control method with low resource usage that can achieve smooth video reverse playback, video multiple-speed reverse playback, and free switching between forward and reverse playback is particularly necessary.

[0004] Under the premise of low resource usage, achieving smooth video reverse playback, double-speed playback, and free switching between forward and reverse playback is a technical problem that needs to be solved. Summary of the Invention

[0005] The technical task of the present invention is to address the above shortcomings and provide an HLS on-demand player implementation method and player that supports reverse playback, so as to solve the technical problems of achieving smooth video reverse playback, double-speed playback, and free switching between forward and reverse playback under the premise of low resource occupation.

[0006] In a first aspect, the present invention provides a method for implementing an HLS on-demand player that supports reverse playback, comprising the following steps:

[0007] Set up a playback control mechanism for the player: Using the recording start time as a reference point, calculate the relative timestamp of each frame. Based on playback parameters and a negative playback speed parameter compatible with reverse playback, calculate the current playback time point. Control the entire playback progress by comparing the relative timestamp of each frame with the current playback time point. Playback parameters include playback status, playback direction and speed, and actual elapsed time.

[0008] Set up a preloading mechanism for the player: During forward playback, preload one fragment file in the future direction and pre-decode one frame in advance, while retaining two fragments and one GOP of cropped RAW data in the past direction. During reverse playback, preload two fragment files in the past direction and pre-decode and crop one GOP of RAW data in advance, while retaining one fragment in the future direction.

[0009] Set up a resource optimization mechanism for the player: In the preloading mechanism, optimize the RAW data of the GOP stored in the past direction to reduce memory usage.

[0010] Preferably, the resource optimization mechanism set for the player includes:

[0011] Activate when used: Since the reverse playback function is used much less frequently than the normal playback function, the reverse GOP cache is used and is enabled when the reverse playback mode is first entered;

[0012] Data cropping: scaling the RAW data in GOP in both width and height dimensions to the minimum size that can meet the clarity of the display area, in order to reduce the amount of data;

[0013] Frame extraction mechanism: Since people perceive the frame rate lower during reverse playback than during forward playback, frame extraction is performed on the GOP cache data based on user configuration.

[0014] Preferably, the player includes three workflows: main workflow, download workflow and decoding workflow;

[0015] The main workflow provides playback status initialization, user operation response, playback direction judgment, frame data inspection, timestamp calculation, frame display, and cache management.

[0016] The download workflow supports segmented downloading in forward and reverse modes. During the segmented download process, the system continuously checks user operations and decoding status, and performs a short sleep before the decoding workflow is completed to reduce resource usage.

[0017] The decoding workflow provides decoding requirements in forward and reverse modes, continuously checks user operations and file end status during the decoding process, and performs a short sleep when the file is not finished to reduce resource usage.

[0018] Preferably, the main workflow includes the following steps:

[0019] a. Initialize the playback state to forward playback, initialize the playback speed, and set the current playback time to the current frame timestamp;

[0020] b. Determine whether the user has finished playing. If so, the workflow ends; otherwise, proceed to the next step.

[0021] c. Determine the current playback direction. If it is forward playback, jump to step d; otherwise, jump to step g.

[0022] d. If there is no next frame of data and the file ends, end the workflow, otherwise proceed to the next step;

[0023] e. Recalculate the current timestamp. If the current timestamp is greater than or equal to the timestamp of the next frame, display the next frame, remove the next frame from the future direction RAW buffer, add it to the past direction RAW buffer, and then jump to step b. Otherwise, proceed to the next step.

[0024] f. Sleep for a predetermined time and jump to step b;

[0025] g. If there is no next frame of data and the file ends, the workflow ends, otherwise proceed to the next step;

[0026] h. Recalculate the current timestamp. If the current timestamp is less than or equal to the timestamp of the next frame, display the next frame, remove the next frame from the past direction RAW buffer, add it to the future direction RAW buffer, and then jump to step b. Otherwise, go to the next step.

[0027] i. Sleep for a predetermined time and jump to step b.

[0028] 5. The method for implementing an HLS on-demand player supporting reverse playback according to claim 4, wherein the current timestamp is calculated as follows:

[0029] Current timestamp = previous current timestamp + actual interval time * speed

[0030] The speed is negative in reverse playback mode.

[0031] Preferably, the download workflow includes the following operations:

[0032] a. Determine whether the user has stopped playing. If so, the workflow ends. Otherwise, determine whether the current mode is reverse playback. If so, execute step c; otherwise, execute step b.

[0033] b. Determine whether the next segment in the future direction of the current playback position has been downloaded. If not, download the segment and jump to step a; otherwise, jump to step d.

[0034] c. Determine whether the next segment in the direction past the current playback position has been downloaded. If not, download the segment and jump to step a; otherwise, jump to step d.

[0035] d. Determine whether the current decoding workflow is finished. If the decoding workflow is finished, then end the downloading workflow. Otherwise, jump to step a after sleeping for a predetermined time.

[0036] Preferably, the decoding workflow includes the following operations:

[0037] a. Determine whether the user has stopped playing. If so, the workflow ends. Otherwise, determine whether the current mode is reverse playback. If so, execute step c; otherwise, execute step b.

[0038] b. Determine whether the undisplayed RAW data frame in the future direction is a full frame and whether there is any undecoded data frame. If so, decode and crop the next encoded frame, add it to the future direction RAW frame list, and then jump to step a. Otherwise, skip this step and jump directly to step d.

[0039] c. Determine the amount of RAW data cached in the past direction and whether the previous GOP fragment data has been downloaded. If the RAW data is less than one GOP and the previous GOP fragment data has been downloaded, decode the next GOP data and add it to the RAW data cache in the past direction after cropping. Otherwise, skip this step and jump directly to step d.

[0040] d. Determine whether the file is finished. If so, end the workflow. Otherwise, sleep for 10ms and jump to step a.

[0041] Preferably, when cropping RAW data, only image scaling by powers of 2 is performed to ensure that scaling only requires integer addition and shift operations. The width and height after scaling should be greater than or equal to the width and height of the display area to ensure that display clarity is not affected.

[0042] Preferably, when extracting RAW data frames, each time a new RAW frame is added to the past direction RAW cache, if the number of frames in the cache pool is greater than or equal to the predetermined number of frames, and the timestamps of the last frame and the third-to-last frame are less than the predetermined time, the second-to-last frame is removed and discarded, otherwise no processing is performed.

[0043] In a second aspect, the present invention provides an HLS on-demand player that supports reverse playback, wherein the player is provided with a playback control mechanism, a preloading mechanism, and a resource optimization mechanism set by an implementation method of an HLS on-demand player that supports reverse playback as described in any one of the first aspects.

[0044] The HLS on-demand player implementation method and player supporting reverse playback of the present invention have the following advantages: ensuring the realization of the overall playback function through a playback control mechanism, ensuring the smoothness of playback through a preloading mechanism, and ensuring low resource usage during playback through a resource optimization mechanism. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] The present invention will be further described below with reference to the accompanying drawings.

[0047] Figure 1This is a forward playback data flow diagram in a method for implementing an HLS on-demand player supporting reverse playback in Example 1;

[0048] Figure 2 This is a reverse playback data flow diagram in a method for implementing an HLS on-demand player that supports reverse playback in Example 1. DETAILED DESCRIPTION

[0049] The present invention will be further described below with reference to the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments given are not intended to limit the present invention. Unless there is a conflict, the embodiments of the present invention and the technical features in the embodiments may be combined with each other.

[0050] The embodiment of the present invention provides an implementation method and player for an HLS on-demand player that supports reverse playback, which is used to solve the technical problem of achieving smooth video reverse playback, double-speed playback, and free switching between forward and reverse playback under the premise of low resource usage.

[0051] Example 1:

[0052] The present invention provides an implementation method of an HLS on-demand player supporting reverse playback, comprising three operations of setting a playback control mechanism for the player, setting a preloading mechanism for the player, and setting a resource optimization mechanism for the player.

[0053] Set up a playback control mechanism for the player: Using the recording start time as a reference point, calculate the relative timestamp of each frame. Based on the playback parameters and a negative playback speed parameter compatible with reverse playback, calculate the current playback time point. Control the entire playback progress by comparing the relative timestamp of the frame with the current playback time point. The playback parameters include playback status, playback direction and speed, and the actual elapsed time.

[0054] Set up a preloading mechanism for the player: During forward playback, preload one fragment file in the future direction and pre-decode one frame in advance. Two fragments and the RAW data of one GOP after cropping are retained in the past direction. During reverse playback, two fragment files are pre-loaded in the past direction and the RAW data of one GOP after pre-decoding is cropped in advance. One fragment is retained in the future direction.

[0055] Set up a resource optimization mechanism for the player: In the preloading mechanism, optimize the RAW data of the GOP stored in the past direction to reduce memory usage.

[0056] The resource optimization mechanisms set for the player include the following three mechanisms:

[0057] (1) Activation when used: Since the reverse playback function is used much less frequently than the normal playback function, the GOP cache is used in the past and is enabled when the reverse playback mode is first entered;

[0058] (2) Data cropping: scaling the RAW data in the GOP in both width and height dimensions to the minimum size that can meet the clarity of the display area to reduce the amount of data;

[0059] (3) Frame extraction mechanism: Since people perceive the frame rate lower during reverse playback than during forward playback, the GOP cache data is extracted based on user configuration.

[0060] The player in this embodiment includes three workflows: main workflow, download workflow, and decoding workflow. The main workflow provides playback status initialization, user operation response, playback direction judgment, frame data inspection, timestamp calculation, frame display, and cache management.

[0061] The download workflow supports segmented downloading in forward and reverse modes. During the segmented download process, the system continuously checks user operations and decoding status, and performs a short sleep before the decoding workflow is completed to reduce resource usage.

[0062] The decoding workflow provides decoding requirements in forward and reverse modes, continuously checks user operations and file end status during the decoding process, and performs a short sleep when the file is not finished to reduce resource usage.

[0063] The main workflow includes the following steps:

[0064] a. Initialize the playback state to forward playback, initialize the playback speed, and set the current playback time to the current frame timestamp;

[0065] b. Determine whether the user has finished playing. If so, the workflow ends; otherwise, proceed to the next step.

[0066] c. Determine the current playback direction. If it is forward playback, jump to step d; otherwise, jump to step g.

[0067] d. If there is no next frame of data and the file ends, end the workflow, otherwise proceed to the next step;

[0068] e. Recalculate the current timestamp. If the current timestamp is greater than or equal to the timestamp of the next frame, display the next frame, remove the next frame from the future direction RAW buffer, add it to the past direction RAW buffer, and then jump to step b. Otherwise, proceed to the next step.

[0069] f. Sleep for a predetermined time and jump to step b;

[0070] g. If there is no next frame of data and the file ends, the workflow ends, otherwise proceed to the next step;

[0071] h. Recalculate the current timestamp. If the current timestamp is less than or equal to the timestamp of the next frame, display the next frame, remove the next frame from the past direction RAW buffer, add it to the future direction RAW buffer, and then jump to step b. Otherwise, go to the next step.

[0072] i. Sleep for a predetermined time and jump to step b.

[0073] The current timestamp is calculated as follows:

[0074] Current timestamp = previous current timestamp + actual interval time * speed

[0075] The speed is negative in reverse playback mode.

[0076] The download workflow includes the following operations:

[0077] a. Determine whether the user has stopped playing. If so, the workflow ends. Otherwise, determine whether the current mode is reverse playback. If so, execute step c; otherwise, execute step b.

[0078] b. Determine whether the next segment in the future direction of the current playback position has been downloaded. If not, download the segment and jump to step a; otherwise, jump to step d.

[0079] c. Determine whether the next segment in the direction past the current playback position has been downloaded. If not, download the segment and jump to step a; otherwise, jump to step d.

[0080] d. Determine whether the current decoding workflow is finished. If the decoding workflow is finished, then end the downloading workflow. Otherwise, jump to step a after sleeping for a predetermined time.

[0081] The decoding workflow includes the following operations:

[0082] a. Determine whether the user has stopped playing. If so, the workflow ends. Otherwise, determine whether the current mode is reverse playback. If so, execute step c; otherwise, execute step b.

[0083] b. Determine whether the undisplayed RAW data frame in the future direction is a full frame and whether there is any undecoded data frame. If so, decode and crop the next encoded frame, add it to the future direction RAW frame list, and then jump to step a. Otherwise, skip this step and jump directly to step d.

[0084] c. Determine the amount of RAW data cached in the past direction and whether the previous GOP fragment data has been downloaded. If the RAW data is less than one GOP and the previous GOP fragment data has been downloaded, decode the next GOP data and add it to the RAW data cache in the past direction after cropping. Otherwise, skip this step and jump directly to step d.

[0085] d. Determine whether the file is finished. If so, end the workflow. Otherwise, sleep for 10ms and jump to step a.

[0086] When cropping RAW data, only image scaling by powers of 2 is performed to ensure that scaling only requires integer addition and shift operations. The width and height after scaling should be greater than or equal to the width and height of the display area to ensure that display clarity is not affected.

[0087] When extracting RAW data frames, each time a new RAW frame is added to the past direction RAW buffer, if the number of frames in the buffer pool is greater than or equal to the predetermined number of frames, and the timestamps of the last frame and the third-to-last frame are less than the predetermined time, the second-to-last frame is removed and discarded; otherwise, no processing is performed.

[0088] The method of this embodiment, such as Figure 1 As shown in the data flow diagram, during forward playback, a segment file is preloaded in the future direction and a frame is pre-decoded in advance, while two segments and a GOP of cropped RAW data are retained in the past direction. Figure 2 As shown in the data flow diagram, during reverse playback, two fragment files are pre-loaded in the past direction, and the RAW data of one GOP is pre-decoded and cropped in advance, while one fragment is retained in the future direction. The method of this embodiment ensures the overall playback function through a unique playback control mechanism; ensures playback smoothness through a unique pre-loading mechanism; and ensures low resource usage during playback through a unique resource optimization mechanism.

[0089] Example 2:

[0090] The present invention provides an HLS on-demand player that supports reverse playback, wherein the player is provided with a playback control mechanism, a preloading mechanism, and a resource optimization mechanism set by the method disclosed in Example 1.

[0091] The above is a detailed introduction to the HLS on-demand player implementation method and player supporting reverse playback provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method and core ideas of the present invention; at the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for implementing an HLS on-demand player that supports reverse playback, characterized in that: The steps include: Set up a playback control mechanism for the player: Using the recording start time as a reference point, calculate the relative timestamp of each frame. Based on playback parameters and a negative playback speed parameter compatible with reverse playback, calculate the current playback time point. Control the entire playback progress by comparing the relative timestamp of each frame with the current playback time point. Playback parameters include playback status, playback direction and speed, and actual elapsed time. Set up a preloading mechanism for the player: During forward playback, preload one fragment file in the future direction and pre-decode one frame in advance, while retaining two fragments and one GOP of cropped RAW data in the past direction. During reverse playback, preload two fragment files in the past direction and pre-decode and crop one GOP of RAW data in advance, while retaining one fragment in the future direction. Set up a resource optimization mechanism for the player: In the preloading mechanism, the RAW data of the GOP stored in the past direction is optimized to reduce memory usage; The player includes three workflows: main workflow, download workflow and decoding workflow; The main workflow provides playback status initialization, user operation response, playback direction judgment, frame data inspection, timestamp calculation, frame display, and cache management. The download workflow supports segmented downloading in forward and reverse modes. During the segmented download process, the system continuously checks user operations and decoding status, and performs a short sleep before the decoding workflow is completed to reduce resource usage. The decoding workflow supports decoding requirements in forward and reverse modes. During the decoding process, the system continuously checks user operations and file end status, and performs a short sleep when the file is not finished, thus reducing resource usage. The main workflow includes the following steps: a. Initialize the playback state to forward playback, initialize the playback speed, and set the current playback time to the current frame timestamp; b. Determine whether the user has finished playing. If so, the workflow ends; otherwise, proceed to the next step. c. Determine the current playback direction. If it is forward playback, jump to step d; otherwise, jump to step g. d. If there is no next frame of data and the file ends, end the workflow, otherwise proceed to the next step; e. Recalculate the current timestamp. If the current timestamp is greater than or equal to the timestamp of the next frame, display the next frame, remove the next frame from the future direction RAW buffer, add it to the past direction RAW buffer, and then jump to step b. Otherwise, proceed to the next step. f. Sleep for a predetermined time and jump to step b; g. If there is no next frame of data and the file ends, the workflow ends, otherwise proceed to the next step; h. Recalculate the current timestamp. If the current timestamp is less than or equal to the timestamp of the next frame, display the next frame, remove the next frame from the past direction RAW buffer, add it to the future direction RAW buffer, and then jump to step b. Otherwise, go to the next step. i. Sleep for a predetermined time and jump to step b; The download workflow includes the following operations: a. Determine whether the user has stopped playing. If so, the workflow ends. Otherwise, determine whether the current mode is reverse playback. If so, execute step c; otherwise, execute step b. b. Determine whether the next segment in the future direction of the current playback position has been downloaded. If not, download the segment and jump to step a; otherwise, jump to step d. c. Determine whether the next segment in the direction past the current playback position has been downloaded. If not, download the segment and jump to step a; otherwise, jump to step d. d. Determine whether the current decoding workflow is finished. If the decoding workflow is finished, then end the downloading workflow. Otherwise, jump to step a after sleeping for a predetermined time.

2. The method for implementing an HLS on-demand player supporting reverse playback according to claim 1, wherein: The resource optimization mechanisms set for the player include: Activate when used: Since the reverse playback function is used much less frequently than the normal playback function, the reverse GOP cache is used and is enabled when the reverse playback mode is first entered; Data cropping: scaling the RAW data in GOP in both width and height dimensions to the minimum size that can meet the clarity of the display area, in order to reduce the amount of data; Frame extraction mechanism: Since people perceive the frame rate lower during reverse playback than during forward playback, frame extraction is performed on the GOP cache data based on user configuration.

3. The method for implementing an HLS on-demand player supporting reverse playback according to claim 1, wherein: The current timestamp is calculated as follows: Current timestamp = previous current timestamp + actual interval time * speed The speed is negative in reverse playback mode.

4. The method for implementing an HLS on-demand player supporting reverse playback according to claim 1, wherein: The decoding workflow includes the following operations: a. Determine whether the user has stopped playing. If so, the workflow ends. Otherwise, determine whether the current mode is reverse playback. If so, execute step c; otherwise, execute step b. b. Determine whether the undisplayed RAW data frame in the future direction is a full frame and whether there is any undecoded data frame. If so, decode and crop the next encoded frame, add it to the future direction RAW frame list, and then jump to step a. Otherwise, skip this step and jump directly to step d. c. Determine the amount of RAW data cached in the past direction and whether the previous GOP fragment data has been downloaded. If the RAW data is less than one GOP and the previous GOP fragment data has been downloaded, decode the next GOP data and add it to the RAW data cache in the past direction after cropping. Otherwise, skip this step and jump directly to step d. d. Determine whether the file is finished. If so, end the workflow. Otherwise, sleep for 10ms and jump to step a.

5. The method for implementing an HLS on-demand player supporting reverse playback according to claim 4, wherein: When cropping RAW data, only image scaling by powers of 2 is performed to ensure that scaling only requires integer addition and shift operations. The width and height after scaling should be greater than or equal to the width and height of the display area to ensure that display clarity is not affected.

6. The method for implementing an HLS on-demand player supporting reverse playback according to claim 4, wherein: When extracting RAW data frames, each time a new RAW frame is added to the past direction RAW buffer, if the number of frames in the buffer pool is greater than or equal to the predetermined number of frames, and the timestamps of the last frame and the third-to-last frame are less than the predetermined time, the second-to-last frame is removed and discarded; otherwise, no processing is performed.

7. An HLS on-demand player that supports reverse playback, characterized in that: The player is provided with a playback control mechanism, a preloading mechanism, and a resource optimization mechanism set by the method for implementing an HLS on-demand player supporting reverse playback as described in any one of claims 1-6.

Images