A multi-step normalized short video preloading method

Abstract

The patent discloses a multi-step normalized short video preloading method. In a short video application scenario, a client constructs a Markov decision process according to a current video playing progress, a past network bandwidth, a video buffer state, a video block code rate and a user retention probability, calculates expected rewards of different video blocks and different code rate versions in a unified time scale, and selects a video block and a code rate version corresponding to a maximum expected reward to construct a request sent to a server, so that the effect of normalizing all decisions is achieved. Experimental results show that compared with existing methods, the short video preloading method proposed by the patent can improve user experience quality in different types of user behaviors, network bandwidths and video scenarios, and reduce bandwidth waste.

Description

Technical Field

[0001] This invention relates to the field of streaming media transmission, and in particular to a multi-step normalized short video preloading method. Background Technology

[0002] With the rapid growth of the short video market, achieving a satisfactory Quality of Experience (QoE) is becoming increasingly important, and short video preloading methods are key to improving the quality of user experience.

[0003] Existing video-on-demand and live streaming platforms use Adaptive Bitrate Streaming (ABR) algorithms to determine the bitrate of downloaded video blocks. Examples include buffer-based methods like BBA and Stick, rate-based methods like Festive and Lumos, and methods combining buffer and rate such as MPC, BOLA, SENSEI, and Fugu. However, ABR does not determine the order in which video blocks are preloaded. Loading video blocks sequentially would cause continuous buffering for users who tend to scroll quickly, making it unsuitable for short video platforms.

[0004] TikTok uses Fixed for short video preloading. For video chunk selection, Fixed limits the buffer size of each video to no more than four seconds. Similarly, the No-save algorithm sets a threshold for each video at each decision, limiting the total size of video chunks in each video buffer to no more than the threshold. For bitrate selection, both Fixed and No-save use the MPC algorithm to determine the bitrate of the selected video chunks. Neither approach considers the impact of user behavior on the preloading method; choosing a fixed preloading order can cause frequent buffering for users who rarely watch the entire video.

[0005] Since the Fixed and No-save preloading algorithms do not consider the impact of user viewing behavior, a rule-based adaptive preloading method was subsequently proposed. This method uses user retention probability to predict actual user behavior, obtains the probability that downloaded video blocks will be played, and dynamically adjusts the priority of decisions based on this probability. The user retention probability is obtained by the video provider counting the number of user swipes. The adaptive preloading method overcomes the shortcomings of the fixed preloading algorithm by comprehensively considering user viewing behavior, network bandwidth, and video block size, thus solving the drawbacks of the fixed preloading method. In the rule-based adaptive preloading scheme, PDAS sets a threshold for each video based on user retention probability and video block download time. When the video buffer length exceeds the threshold, the current decision will not select this video. For the remaining selectable videos, PDAS calculates the bandwidth used by the current decision to download the video block and the QoE for future playback, obtaining a score for each decision. The video and video block bitrate corresponding to the highest score are taken as the final output. This decision-making method is called single-step decision-making. When there are no selectable video blocks, PDAS will pause the download. QoE-Aware is similar in concept to PDAS, setting thresholds for each video block to select downloadable blocks and calculating scores for each block in a single-step decision-making process to determine the downloaded video and its bitrate. The difference lies in that QoE-Aware sets two thresholds for each video block, and the coefficients for each parameter in its evaluation metric differ from PDAS. While these adaptive preloading methods have achieved relatively satisfactory QoE, they make each decision independently and cannot guarantee that these decisions are globally optimal. In fact, the impact of a single decision is difficult to quantify because the current decision alters the video buffer, and the size of the video buffer limits the choice of subsequent decisions. Furthermore, because different video blocks have different sizes and download times, and QoE and bandwidth usage are time-dependent, it is impossible to make fair comparisons across different time scales when evaluating decisions based on QoE and bandwidth usage. Therefore, each decision may be suboptimal. Summary of the Invention

[0006] To address the problem of local optimal decision-making caused by single-step decision-making in current preloading methods, this invention provides a short video preloading method that can achieve global optimization and ensure a consistent evaluation decision time scale.

[0007] To achieve the above-mentioned technical objectives, the technical solution of the present invention is as follows:

[0008] A multi-step normalized short video preloading method is proposed. A unified time scale is determined based on the current state and optional actions. The client calculates all future action sequences, state transition probabilities, and rewards within the given time scale, starting from each optional action, based on the current state and the user retention probability of the video block. For each optional action in the client's current state, the expected total value of all future action sequences within the unified time scale starting from that action is calculated. The optional action corresponding to the maximum expected total value is selected as the optimal decision. Based on this action, a request is sent to the server to download the video block or pause the download.

[0009] The current state in the client is defined as follows in the method described:

[0010]

[0011] in Indicates the client's first The state when making the preloading decision for the next time. This indicates the sequence number of the currently playing video. This indicates the sequence number of the video block currently playing. It's the past The throughput of a video block This represents the buffer for each video. This indicates the bitrate of each video block downloaded in the past.

[0012] The method described, the current state in the client The number of optional actions is as follows indivual, as well as These represent the number of videos in the recommendation queue and the number of video block bitrate versions, respectively. Indicates the optional pause time; let Representing state The following options are available One action, A unified time scale The calculation formula is:

[0013]

[0014] in, This refers to taking the maximum value. Indicates action The predicted download time. It is obtained by dividing the known video block data size by the predicted bandwidth, which is calculated from past data. The average throughput is obtained from the size of a video block.

[0015] The method described, for each future action sequence The following constraints must be met:

[0016]

[0017] in, Indicates action Predicted download time The first action in the sequence of actions One action; because the action sequence is not actually executed in the client, so use The client that predicts the number of clients is number 1. The state when making the preloading decision for the first time. Indicates the state The following are optional actions, each action sequence It can be represented as , Represents an action sequence The number of actions in the sequence.

[0018] The method is characterized by calculating each action sequence based on the current state of the client and the user retention probability of the video. The process of state transition probability is as follows:

[0019] The client is in state Execute action After transitioning to state The probability can be expressed by the following formula:

[0020]

[0021] Among them, due to the diversity of user behavior, Memory exists A video block playback sequence, Indicates the action Execution time Inner A sequence of video blocks for playback. for The total number of video blocks in the video. express The first in A video segment that was played. Represents video block After the video is finished playing, play the video segment. The probability, that is, the probability of user retention for a given video block is measured in advance.

[0022] The method described, in state Take action transition to state The formula for calculating the reward is as follows:

[0023]

[0024]

[0025] in, It refers to the action Execution time The QoE obtained by playing video blocks within the loop. refers to actions The network bandwidth used The weighting parameters for the influencing factors; refer to The video blocks played during execution It is a video block bitrate, It is playing a video block Bitrate switching during time It refers to the action The pause time during execution time, The parameters are the weights of the corresponding influencing factors. The values ​​are 0.5, 1, 1, and 1.85.

[0026] The method described above, for each optional action, calculates the expected total value of all future action sequences starting from that action within a uniform time scale as follows:

[0027] Step 1: Represent the client's state using the following formula. The state value obtained by choosing the optimal action at the right time :

[0028]

[0029] Indicates action On a unified time scale The expected total value within;

[0030] Step 2: Current Status Below, each optional action On a unified time scale The total intrinsic expected value can be expressed by the following formula:

[0031]

[0032] in, Indicates the execution of an action Post-predicted transition state, Indicates the state Execute action Transition to state The probability, Indicates the state Take action Transition to state The reward Indicates from state Starting on a new time scale The expected total value of the remaining action sequence.

[0033] When the new time scale is 0, the client's experience time scale is calculated using the following formula. Then reach the state At that time, the expected total reward for unplayed video blocks in the buffer is:

[0034]

[0035] in, Indicates that the prediction client passes through The final state reached, Indicates the state The number of playback sequences consisting of video blocks in the buffer. This indicates the number of video blocks played in each playback sequence. Indicates the sequence number of the video block being played. and These represent the video blocks being played. The probability and reward.

[0036] The technical advantage of this invention lies in that by setting a unified time scale and multi-step evaluation criteria, it helps the algorithm make globally optimal decisions, thereby solving the local optimum problem caused by single-step decision-making and the unfair time scale problem in current preloading methods.

[0037] The invention will now be further described with reference to the accompanying drawings. Attached Figure Description

[0038] Figure 1 This is a general framework diagram of the present invention;

[0039] Figure 2 A comparison chart of scores for single-step, multi-step, and multi-step normalized decision-making;

[0040] Figure 3 For the present invention in different The following is a comparison chart of performance metrics data;

[0041] Figure 4 A comparison chart of scores for preloading methods under different network bandwidths;

[0042] Figure 5 A comparison chart of QoE and bandwidth usage for preloading methods under different bandwidth conditions; Detailed Implementation

[0043] See Figure 1 In this embodiment, the client determines a unified time scale based on the current state and optional actions. The client calculates all future action sequences, state transition probabilities after future actions, and rewards starting from each optional action within a given time scale based on the current state and the user retention probability of the video block. For each optional action of the client in the current state, the expected total value of all future action sequences starting from that action within the unified time scale is calculated. The optional action corresponding to the maximum expected total value is selected as the optimal decision. Based on this action, a request to download the video block or pause the download is sent to the server, thereby solving the local optimal decision problem existing in the existing short video preloading methods.

[0044] Step 1: Current state in the client The number of optional actions is as follows indivual, as well as These represent the number of videos in the recommendation queue and the number of video block bitrate versions, respectively. Indicates the optional pause time; let Representing state The following options are available One action, A unified time scale The calculation formula is:

[0045]

[0046] in, This refers to taking the maximum value. Indicates action Predicted download time, It is obtained by dividing the known video block data size by the predicted bandwidth, which is calculated from past data. The average throughput is obtained from the size of a video block.

[0047] Step 2: The client calculates all future action sequences, state transition probabilities after future actions are executed, and rewards for each available action within a given time scale, based on the current state and the user retention probability of the video block.

[0048] The specific client-side operations in step two are as follows:

[0049] Step A1: Starting from each optional action, generate a sequence of all future actions within a given time scale. The following constraints must be met:

[0050]

[0051] in, Indicates action Predicted download time The first action in the sequence of actions One action; because the action sequence is not actually executed in the client, so use The client that predicts the number of clients is number 1. The state when making the preloading decision for the first time. Indicates the state The following are optional actions, each action sequence It can be represented as , Represents an action sequence The number of actions in the sequence.

[0052] Step A2: Calculate each action sequence based on the current client state and the user retention probability of the video. The process of state transition probability is as follows:

[0053] The client is in state Execute action After transitioning to state The probability can be expressed by the following formula:

[0054]

[0055] Among them, due to the diversity of user behavior, Memory exists A sequence of video blocks for playback. Indicates the action Execution time Inner A sequence of video blocks for playback. for The total number of video blocks in the video. express The first in A video segment that was played. Represents video block After the video is finished playing, play the video segment. The probability, that is, the probability of user retention for a given video block is measured in advance.

[0056] Step A3, in state Take action transition to state The formula for calculating the reward is as follows:

[0057]

[0058]

[0059] in, It refers to the action Execution time The QoE obtained by playing video blocks within the loop. refers to actions The network bandwidth used The weighting parameters for the influencing factors; refer to The video blocks played during execution It is a video block bitrate, It is playing a video block Bitrate switching during time It refers to the action The pause time during execution time, The parameters are the weights of the corresponding influencing factors. The values ​​are 0.5, 1, 1, and 1.85.

[0060] Step 3: For each optional action, calculate the expected total value of all future action sequences starting from that action within a uniform time scale.

[0061] The specific steps for the client in step three are as follows:

[0062] Step A1, the client's state is represented by the following formula. The state value obtained by choosing the optimal action at the right time :

[0063]

[0064] in, Indicates action On a unified time scale The expected total value within;

[0065] Step A2, Current State Below, each optional action On a unified time scale The total intrinsic expected value can be expressed by the following formula:

[0066]

[0067] in, Indicates the execution of an action Post-predicted transition state, Indicates the state Execute action Transition to state The probability, Indicates the state Take action Transition to state The reward Indicates from state Starting on a new time scale The expected total value of the remaining action sequence;

[0068] When the new time scale is 0, the client's experience time scale is calculated using the following formula. Then reach the state At that time, the expected total reward for unplayed video blocks in the buffer is:

[0069]

[0070] in, Indicates that the prediction client passes through The final state reached, Indicates the state The number of playback sequences consisting of video blocks in the buffer. This indicates the number of video blocks played in each playback sequence. Indicates the sequence number of the video block being played. and These represent the video blocks being played. The probability and reward.

[0071] Step 4: The client sends a request to the server to download video blocks or pause the download based on the optimal action.

[0072] Figure 2 This example demonstrates the score of the method (Unified Multi-step Preload strategy for Short Video, UMPS) under different network bandwidths. The score is calculated using the following formula:

[0073]

[0074] The score is composed of the QoE obtained after a user watches all videos and the bandwidth used to download video chunks. The user's QoE is mainly determined by the average video quality. Average mass change and lag time Decision: Use bandwidth It is determined by the size of the downloaded video block.

[0075] When decision-making changes from single-step to multi-step, taking two-step as an example, the score improved by 24.1% under low bandwidth, 84.7% under medium bandwidth, and 18.5% under high bandwidth. However, when decision-making changes from two steps to three steps, the improvement is only 3.0% under low bandwidth, 4.7% under medium bandwidth, and 16.8% under high bandwidth. The improvement in experimental results is relatively small, mainly due to the increase in the number of steps. The issue of unfairness is more pronounced. The multi-step normalized UMPS algorithm shows a significant improvement over the unnormalized multi-step decision-making algorithm. Compared to the three-step decision-making algorithm, the UMPS algorithm's score is improved by 21.8% under low bandwidth, 12.3% under medium bandwidth, and 6.3% under high bandwidth.

[0076] Figure 3 This example demonstrates the methods provided in different... The values ​​represent changes in bit rate, bit rate switching, stuttering, and bandwidth usage.

[0077] Select respectively Values , and By observing the experimental data, Values ,contrast and The bitrate was increased by 6.7% and 20.5% respectively, stuttering was reduced by 35.6% and 20.5%, and bitrate switching was reduced by 20.3% and 5.5%.

[0078] Experimental results show that, under the premise of relatively accurate bandwidth prediction, Generally speaking, the bigger the better. It allows algorithms to anticipate more scenarios, thereby making more reasonable decisions and avoiding local optima as much as possible.

[0079] Figure 4 , Figure 5 This example demonstrates the comparison of scores, QoE, and bandwidth usage under different network bandwidths using the method provided.

[0080] Figure 4 The cumulative distribution function (CDF) plot of the scores across all bandwidths is displayed. From Figure 4As can be seen, under all three bandwidth types, UMPS improved its score compared to PDAS and Fixed. At low bandwidth, UMPS improved user experience by 23.4% and 13.6% compared to Fixed and PDAS, respectively, while reducing bandwidth usage by 9.0% and 7.2%. At medium bandwidth, user experience improved by 51.5% and 1.0%, respectively, while reducing bandwidth usage by 2.5% and 23%. At high bandwidth, user experience improved by 38.9% and 19.0%, respectively, while reducing bandwidth usage by 10.8% and 15.2%. Figure 5 It can be seen that when the Fixed score is used as the baseline, the UMPS score is 43.6%, 36.6% and 34.0% higher than that of PDAS under the three bandwidths.

[0081] Experimental results show that UMPS, compared to Fixed and PDAS, not only effectively improves the user experience but also reduces bandwidth usage. This effectively confirms the superior performance of UMPS under different bandwidth conditions.

Claims

1. A multi-step normalization method for preloading short videos, characterized in that, Includes the following steps: Step 1: The preloading method determines a uniform time scale based on the current state and optional actions; The current state in the client is defined as follows: Indicates the client's first The state when making the preloading decision for the next time. This indicates the sequence number of the currently playing video. This indicates the sequence number of the video block currently playing. It's the past The throughput of a video block This represents the buffer for each video. This indicates the bitrate of each video block downloaded previously; Current state in the client The number of optional actions is as follows indivual, as well as These represent the number of videos in the recommendation queue and the number of video block bitrate versions, respectively. Indicates the optional pause time; let Representing state Next One optional action, A unified time scale The calculation formula is: in, This refers to taking the maximum value. Indicates action Predicted download time, It is obtained by dividing the known video block data size by the predicted bandwidth, which is calculated from past data. The average throughput obtained from the size of a video block; Step 2: The client calculates all future action sequences starting from each optional action within a given time scale, as well as the state transition probability and reward after the execution of the future actions, based on the current state and the user retention probability of the video block; In this action sequence, the first action is the current state. The next action is optional, and subsequent actions are optional actions in the future state; each action sequence The following constraints must be met: Indicates optional action The predicted download time is calculated because the action sequence is not actually executed on the client side. The client that predicts the number of clients is number 1. The state when making the preloading decision for the next time. Indicates the state Next One optional action; The first action in the sequence of actions One action; each action sequence It can be represented as , Represents an action sequence The number of actions in the sequence; The client is in state Execute action Subsequently, due to the diversity of user behavior, the execution time... Memory exists A possible video block playback sequence, Indicates the first A video block playback sequence, transitioning to the state according to the playback sequence. The probability can be expressed by the following formula: in, for The total number of video blocks in the video. express The first in A video segment that was played. Represents video block After the video is finished playing, play the video segment. The probability of user retention for a given video segment is measured in advance. In state Take action transition to state The formula for calculating the reward is as follows: in, It refers to the action Execution time The QoE obtained by playing video blocks within the loop. refers to actions The network bandwidth used The weighting parameters for the influencing factors; refer to The video blocks played during execution It is a video block bitrate, It is playing a video block Bitrate switching during time It refers to the action The pause time during execution time, These are the weighting parameters for the corresponding influencing factors; Step 3: For each available action of the client in the current state, calculate the expected total value of all action sequences starting from that action within a uniform time scale. This includes the following sub-steps: Sub-step A1: The client's state is represented by the following formula. The state value obtained by choosing the optimal action at the right time : in, Indicates action On a unified time scale The expected total value within; Sub-step A2, Current state Below, each optional action On a unified time scale The total intrinsic expected value can be expressed by the following formula: in, Indicates the execution of an action Post-predicted transition state, Indicates the state Execute action Transition to state The probability, Indicates the state Take action Transition to state The reward Indicates from state Starting on a new time scale The expected total value of the remaining action sequence; Sub-step A3: When the new time scale is 0, calculate the time scale experienced by the client using the following formula. Then reach the state At that time, the expected total reward for unplayed video blocks in the buffer is: in, Indicates the completion of the action sequence All The final state reached after a series of actions. Indicates the state The number of playback sequences consisting of video blocks in the buffer. This indicates the number of video blocks played in each playback sequence. Indicates the sequence number of the video block being played. and These represent the video blocks being played. The probability and reward; Sub-step A4: Select the action corresponding to the maximum expected total value as the optimal decision, and construct a request to send a video block to the server or pause the download based on the action.

2. The method according to claim 1, characterized in that, The weight parameters The values ​​are 0.5, 1, 1, and 1.85.

Images