A streaming media code rate self-adaption method and its action pruning system

Abstract

The application provides a streaming media code rate self-adaption method and an action pruning system thereof, comprising: a network throughput prediction module, taking network historical throughput information as input and outputting network throughput prediction value in a future period of time; an optional code rate space pruning rule module, determining pruning results of each optional code rate version according to the network throughput prediction value and a streaming media linear viewing experience index expression; and a code rate selection mask module, based on the pruning results of each optional code rate version, acting on various code rate self-adaption method decision processes, and used for reducing decision calculation complexity and improving decision robustness. The action pruning system of the application, as an auxiliary module of the existing code rate self-adaption method, can improve the robust performance of the method and effectively reduce the calculation complexity of the model-based code rate self-adaption method.

Description

Technical Field

[0001] This invention relates to the field of video communication technology, and in particular to a streaming media bitrate adaptive method and its action pruning system. Background Technology

[0002] Dynamic streaming bitrate adaptation is a network service optimization scheme proposed to maximize the user's online video viewing experience. Specifically, on the video server side, the video service provider divides a given video into a sequence of video slices of fixed playback length, and each video slice is encoded with multiple versions at different bitrates. On the user side, the streaming media player, by observing the user's network environment and the player's video buffer status, uses a bitrate adaptive allocation method to select the optimal bitrate version for each video slice for download and playback, aiming to maximize the overall experience quality for the user watching the video.

[0003] Because network throughput is dynamic and difficult to predict, and the optimization variables are highly dimensional and complex, model-based bitrate adaptation methods optimize bitrate selection by solving an NP-hard problem. However, this method requires traversing all possible bitrate version combinations, which results in very high computational complexity. In real-world environments, this can lead to significant computational latency and affect the user's viewing experience. While existing model-free bitrate adaptive methods based on deep neural networks can make optimal inferences in polynomial time with relatively low computational overhead, these methods are not robust to dynamic and uncertain environmental factors such as network bandwidth and user preferences. Currently, reducing the solution complexity of model-based bitrate adaptation methods or improving the decision robustness of model-free bitrate adaptation methods is a major challenge facing current streaming bitrate adaptation schemes. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the purpose of this invention is to provide a streaming media bitrate adaptive method and its action pruning system.

[0005] According to one aspect of the present invention, an action pruning system based on streaming bitrate adaptation is provided, comprising: The network throughput prediction module takes historical network throughput information as input and outputs a predicted network throughput value for a future period of time. An optional bitrate space pruning rule module is configured to prune rules based on the network throughput prediction value. The expression for the streaming linear viewing experience metric determines the pruning results for each available bitrate version; A bitrate selection mask module, based on the pruning results of each selectable bitrate version, is applied to various bitrate adaptive decision-making processes to reduce computational complexity and improve decision robustness.

[0006] Preferably, the network throughput prediction value Network throughput in a time sliding window The average value within, where, For the current moment, This is the length of the sliding window.

[0007] Preferably, the expression for the streaming linear viewing experience metric is as follows: , in, Indicates that the user watched the first... Viewing experience value when streaming media slices are used. For the current number The download bitrate version of each streaming media slice is The quality of the streaming video at that time For users from Start downloading the bitrate version at any time. The The stuttering time in streaming client playback caused by individual streaming media slices It is the first Download bitrate version of each streaming media slice, , These are the weighting coefficients.

[0008] Preferably, in the optional bitrate space pruning rule module, a judgment rule is used to determine the pruning result for each optional bitrate version, and the judgment rule is: No. The bitrate version corresponding to each streaming media slice The condition for being selected for download is that its corresponding ,in, In the formula, , Network throughput The trend of change is determined. The pruning results obtained The bitrate selection mask module outputs a bitrate selection mask vector to assist the bitrate adaptation process in making the optimal download bitrate decision, that is, selecting the optimal download bitrate version.

[0009] Preferably, calculation When the value is given, the calculation required is... The value is based on the network throughput forecast. Confirmed, the calculation formula is: ,in, For bitrate version The Size of each streaming media slice file For the first The duration of playable streaming media stored in the user's player buffer at the moment the streaming media slice file begins downloading. ; Predicting within the sliding window Network throughput fluctuation range and The value is directly proportional to the throughput; when the throughput fluctuates small, .

[0010] Preferably, the various bitrate adaptive decision-making processes include: model-based bitrate adaptive methods or model-free bitrate adaptive methods; When the bitrate selection mask module is applied to the model-based bitrate adaptation method, it performs the following operations on all available bitrate versions: When performing traversal, according to Result confirmed Does it need to be traversed? This prevents model-based bitrate adaptation methods from traversing bitrate versions. ; When the bitrate selection mask module is applied to a model-free bitrate adaptation method, it adjusts the probability vectors of all bitrate versions being selected, thus preventing all... bitrate version The corresponding probability of being selected is adjusted to 0.

[0011] Preferably, adjusting the probability vector for selecting all bitrate versions includes: The probability vector of all bitrate versions selected before adjustment is ,in, For bitrate version The probability of being selected for download; After adjustment by the bitrate selection mask module, the probability vector for the selection of all bitrate versions is: ,in: 。

[0012] According to a second aspect of the present invention, a streaming media bitrate adaptive method is provided, comprising: On the server side, the streaming media file is divided into segments based on time. The playback duration is fixed as follows: slice sequence Each streaming media slice is encoded as Versions with different bitrates And cache; When the user initiates an action against the first When a streaming media slice is requested, the video slice number k to be downloaded, historical network throughput information, and user viewing information are fed back to the streaming media bitrate adaptive action pruning system to obtain the optimal download bitrate. ; Combine the request and the optimal download bitrate The request is sent to the server, which then sets the bitrate version accordingly. The Each streaming media slice is sent to the user's terminal, among which When the first When a streaming media segment begins downloading, the playable streaming media duration stored in the user's player buffer is [duration missing]. .

[0013] According to a third aspect of the present invention, a terminal is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, can be used to run the aforementioned streaming media bitrate adaptive action pruning system or the aforementioned streaming media bitrate adaptive method.

[0014] According to a fourth aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, can be used to run the aforementioned streaming media bitrate-adaptive action pruning system or the aforementioned streaming media bitrate-adaptive method.

[0015] Compared with the prior art, the embodiments of the present invention have at least one of the following beneficial effects: The action pruning system based on streaming bitrate adaptation in this embodiment of the invention serves as an auxiliary component of existing bitrate adaptation methods. It can improve the robustness of bitrate adaptation methods and effectively reduce the computational complexity of model-based bitrate adaptation methods.

[0016] The streaming bitrate adaptive method in this embodiment of the invention reduces the computational complexity of the model-based bitrate adaptive method by pruning the selection of traversal variables in the process of solving the optimization problem, without affecting the solution performance of this type of method. The streaming bitrate adaptive method in this embodiment of the invention can avoid selecting unsuitable bitrate versions by adjusting the probability vector of all bitrate versions output by the model-free bitrate adaptive method being selected for download. This reduces the bitrate selection probability calculation error caused by the inability of the neural network model in the model-free method to generalize to the actual deployment environment, improves the robustness of the method's decision-making, and ensures the user's viewing experience. Attached Figure Description

[0017] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a schematic diagram of an action pruning system based on adaptive streaming bitrate in one embodiment of the present invention; Figure 2 This is a flowchart of a preferred embodiment of the streaming media bitrate adaptive method of the present invention. Detailed Implementation

[0018] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0019] To facilitate understanding of this invention, a brief introduction to the video server and client information is provided first.

[0020] This paper presents a case study analyzing the segmentation and encoding of online video streaming media on the server side. It assumes that the video streaming media files stored on the server are segmented into... The playback time is video slices ( (where is any positive integer greater than 2). For each video slice, dynamic adaptive streaming encoding is used to encode it as... M Versions with different bitrates ( M (where is any positive integer greater than 2), denoted as the set of video slice versions. Furthermore, the set is arranged in descending order of coding bitrate, i.e. .

[0021] when At any time, the user requests to download the first When processing video slices, the user-side information includes: 1) By constructing a network throughput predictor that takes historical network throughput information as input and predicts network throughput over a future period as output, the network throughput within a time sliding window is predicted. The average value within, using Indicates, 2) Streaming media file size mapping function The output bitrate version is The 3) Size of each video slice stream file, and the current player buffer size. 4) The downloaded number Bitrate version of each video slice In addition, the user client will also send a video slice request message, that is, the current requested download segment number. Index information of each video slice in the video server.

[0022] Based on the above background, the present invention provides an embodiment, see [link to embodiment]. Figure 1 An action pruning system based on streaming bitrate adaptation includes: The network throughput prediction module takes historical network throughput information as input and outputs a predicted network throughput value for a future period of time. The optional bitrate space pruning rule module determines the pruning result for each optional bitrate version based on the network throughput prediction and the expression of the streaming linear viewing experience metric. The bitrate selection mask module, based on the pruning results of each optional bitrate version, is applied to the decision-making process of various bitrate adaptive methods to reduce the computational complexity of decision-making and improve decision robustness.

[0023] This embodiment can be used to assist existing bitrate adaptation methods in decision-making, improve the robustness of the methods, and effectively reduce the computational complexity of model-based bitrate adaptation methods.

[0024] This invention does not limit the method for calculating network throughput predictions; any method for estimating network throughput predictions can be deployed. In a preferred embodiment, the network throughput prediction... Network throughput in a time sliding window The average value within, where, For the current moment, The sliding window length is used. Using the average value within the sliding window as the predicted network throughput simplifies the calculation of streaming media download latency and yields more robust prediction results.

[0025] In a preferred embodiment of the present invention, the general mathematical expression for the streaming media linear viewing experience index is: ,in, Indicates that the user watched the first... Viewing experience value when streaming media slices are used. For the current number The download bitrate version of each streaming media slice is The quality of the streaming video at that time For users from Start downloading the bitrate version at any time. The The stuttering time in streaming client playback caused by individual streaming media slices It is the first Download bitrate version of each streaming media slice, , represents the weighting coefficient. The method proposed in this invention is effective for all streaming media viewing experience metrics that satisfy this general mathematical expression.

[0026] In a preferred embodiment of the present invention, the pruning result for each optional bitrate version is determined by the following specific judgment rule: The first... The bitrate version corresponding to each streaming media slice The conditions for being selected for download are: its corresponding , The calculation formula is: in, , Network throughput The trend of change is determined. The pruning results obtained The bitrate selection mask module outputs a bitrate selection mask vector, which is used to assist the bitrate adaptive method in selecting the optimal download bitrate version.

[0027] Furthermore, in the rules for judging pruning results, calculation When the value is given, the calculation required is... The value is based on the network throughput forecast. Confirmed, the calculation formula is: ,in, For bitrate version The Size of each streaming media slice file ; When predicting within the sliding window The network throughput fluctuation range is related to formula (1) The value is directly proportional to the throughput; when the throughput fluctuation is small... .

[0028] In a preferred embodiment of the present invention, the masking module can be applied to both model-based bitrate adaptation methods and model-free bitrate adaptation methods. When used for model-based rate adaptation methods, the masking module can support all available bitrate versions in this type of method. When performing traversal, according to Result confirmed Does it need to be traversed? This prevents model-based bitrate adaptation methods from traversing bitrate versions. ; When used for model-free rate adaptation methods, the masking module can adjust the probability vector of all rate versions selected from the output of this type of method, thus affecting all... bitrate version The corresponding probability of being selected is adjusted to 0.

[0029] Furthermore, in a preferred embodiment, the probability vector for selecting all bitrate versions before adjustment is: ,in, For bitrate version The probability of being selected for download; after adjustment by the bitrate selection mask module, the probability vector of all bitrate versions being selected is... ,in: After the adjustment, it resulted in bitrate version The corresponding probability value of being selected is adjusted to This avoids being selected for download; conversely, after the above normalization operation, the range of the selection probability values ​​for other bitrate versions becomes... Furthermore, the sum of the probability values ​​is 1, thereby increasing the probability of these bitrate versions being selected.

[0030] Based on the same inventive concept, in other embodiments of the present invention, a streaming media bitrate adaptive method is provided, which can be found in [reference needed]. Figure 2 This method involves the user end, a streaming media bitrate adaptive action pruning system, and the server end. The method process is as follows: Step 1: On the server side, the streaming media file is divided into segments based on time. The playback duration is fixed as follows: slice sequence Each streaming media slice is encoded as Versions with different bitrates And cache; Step 2, when the user initiates an action against the first... When a streaming media slice is requested, the sequence number k of the video slice to be downloaded, historical network throughput information, and user viewing information are fed back to the streaming media bitrate adaptive action pruning system. The bitrate adaptive method deployed on the user end obtains the optimal download bitrate with the assistance of the bitrate selection mask module of the action pruning system. Specifically, this could be: for model-based bitrate adaptation methods, performing pruning operations on the traversal search during the bitrate adaptation optimization problem solving process; or, for model-free bitrate adaptation methods, adjusting the probability of all bitrate versions output by the method strategy module being selected for download. Step 3: Combine the request and the optimal download bitrate. Send it to the server, and the server will send the bitrate version according to the request. The Each streaming media slice is sent to the user's terminal, among which When the first When a streaming media segment begins downloading, the playable streaming media duration stored in the user's player buffer is [duration missing]. .

[0031] In a preferred embodiment, the model-based rate adaptation method mainly includes the following steps: S11: When the user Please request to download the first one. Video slices At that time, the network throughput forecast value As the download number The predicted value of user network throughput within the time period of the video slice is used as the basis for future downloads of the [number]th video slice. Simulated network throughput within a video slice time period; and the duration of the video buffer at the user's player at that time. ; S12: Predicted value obtained from S11 Duration of unplayed video in the user player buffer And the mathematical expression for the viewing experience index is: Given the conditions, calculate the simulated download of the first... When processing video slices, each optional bitrate version corresponding The value is calculated using the following formula: in, , , , ; S13: Traversing the... For any video slice, all possible download bitrate versions are possible. The corresponding bitrate version Excluding possible combinations of download bitrate versions, satisfying The bitrate versions constitute the candidate bitrate version combination set. ; S14: Traverse using a model-based bitrate adaptive method. From all combinations, select the optimal bitrate version combination. ; S15: Select the video server A video slice Download the bitrate version; S16: Order Repeat S11~S15.

[0032] This embodiment reduces the computational complexity of the model-based bitrate adaptive method by pruning the selection of traversal variables during the optimization process of solving the model-based bitrate adaptive method, without affecting the solution performance of this type of method. In another preferred embodiment, the model-free bitrate adaptive method mainly includes the following steps: S21: When the user Please request to download the first one. Video slices At that time, calculate the predicted network throughput. As the download number The predicted network throughput of the user within each video slice time period is obtained, along with the duration of the video buffer in the player at that time. ; S22: For the first Video slices The model-free bitrate adaptive method outputs a probability vector indicating which bitrate version is selected for download. S23: For the first Video slices The predicted value obtained from S21 Duration of unplayed video in the user player buffer And the mathematical expression for the viewing experience index is: Given the conditions, calculate the simulation download of the first... When processing video slices, each optional bitrate version corresponding The value is calculated using the following formula: in, , , , ; S24: Obtained from S23 Value, relative to the probability vector obtained in S22 After adjustment, a new probability vector is obtained. ,in: S25: Based on the probability vector obtained in S24 Select the bitrate version with the highest probability value. As the first Download bitrate version of each video slice The information is uploaded to the streaming media server for download. S26: Order Repeat S21~S25.

[0033] This embodiment helps to adjust the probability vector of all bitrate versions output by the model-free bitrate adaptive method to be selected for download. This avoids selecting unsuitable bitrate versions, reduces the bitrate selection probability calculation error caused by the inability of the neural network model in the model-free method to generalize to the actual deployment environment, improves the robustness of the method's decision-making, and ensures the user's viewing experience.

[0034] Based on the same inventive concept, in other embodiments of the present invention, a terminal is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it can be used to run the above-described streaming media bitrate adaptive method.

[0035] Optionally, the memory is used to store programs; the memory may include volatile memory, such as random-access memory (RAM), such as static random-access memory (SRAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), etc.; the memory may also include non-volatile memory, such as flash memory. The memory is used to store computer programs (such as application programs, functional modules, etc. that implement the above methods), computer instructions, etc., and the above-mentioned computer programs, computer instructions, etc., may be partitioned and stored in one or more memories.

[0036] A processor is used to execute a computer program stored in memory to implement the various steps of the methods involved in the above embodiments. For details, please refer to the relevant descriptions in the preceding method embodiments.

[0037] The processor and memory can be separate structures or integrated structures. When the processor and memory are separate structures, they can be coupled together via a bus.

[0038] Based on the same inventive concept, in other embodiments of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, performs the above-described streaming media bitrate adaptive method.

[0039] Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of computer programs from one place to another. Storage media can be any available medium accessible to a general-purpose or special-purpose computer. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Alternatively, the ASIC can reside in a user device. Of course, the processor and storage medium can also exist as discrete components in a communication device.

[0040] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0041] This application is described with reference to flowchart illustrations and / or block diagrams of the methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0042] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0043] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0044] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention. The above preferred features can be used in any combination without conflict.

Claims

1. An action pruning system based on adaptive streaming bitrate, characterized in that, include: The network throughput prediction module takes historical network throughput information as input and outputs a predicted network throughput value for a future period of time. An optional bitrate space pruning rule module is configured to prune rules based on the network throughput prediction value. The expression for the streaming linear viewing experience metric determines the pruning results for each available bitrate version; A bitrate selection mask module, based on the pruning results of each selectable bitrate version, is applied to various bitrate adaptive decision-making processes to reduce the computational complexity of decision-making and improve decision robustness. The expression for the streaming media linear viewing experience metric is as follows: ， in, Indicates that the user watched the first... Viewing experience value when streaming media slices are used. For the current number The download bitrate version of each streaming media slice is The quality of the streaming video at that time For users from Start downloading the bitrate version immediately. The The stuttering time in streaming client playback caused by individual streaming media slices It is the first Download bitrate version of each streaming media slice, , These are the weighting coefficients; In the optional bitrate space pruning rule module, a judgment rule is used to determine the pruning result for each optional bitrate version. The judgment rule is as follows: No. The bitrate version corresponding to each streaming media slice The condition for being selected for download is that its corresponding ,in, ； In the formula, , Network throughput The trend of change is determined. ; The pruning results obtained The bitrate selection mask module outputs a bitrate selection mask vector to assist the bitrate adaptation process in making the optimal download bitrate decision, that is, selecting the optimal download bitrate version.

2. The action pruning system based on adaptive streaming bitrate as described in claim 1, characterized in that, The network throughput forecast Network throughput in a time sliding window The average value within, where, For the current moment, This is the length of the sliding window.

3. The action pruning system based on adaptive streaming bitrate as described in claim 1, characterized in that, calculate When the value is given, the calculation required is... The value is based on the network throughput forecast. Confirmed, the calculation formula is: ,in, For bitrate version The Size of each streaming media slice file For the first The duration of playable streaming media stored in the user's player buffer at the moment the streaming media slice file begins downloading. ; Predicting within the sliding window Network throughput fluctuation range and The value is directly proportional to the throughput; when the throughput fluctuation is small, .

4. The action pruning system based on adaptive streaming bitrate as described in claim 3, characterized in that, The various bitrate adaptive decision-making processes include: model-based bitrate adaptive methods or model-free bitrate adaptive methods; When the bitrate selection mask module is applied to the model-based bitrate adaptation method, it performs the following operations on all available bitrate versions: When performing traversal, according to Result confirmed Does it need to be traversed? This prevents model-based bitrate adaptation methods from traversing bitrate versions. ; When the bitrate selection mask module is applied to a model-free bitrate adaptation method, it adjusts the probability vectors of all bitrate versions being selected, thus preventing all... bitrate version The corresponding probability of being selected is adjusted to 0.

5. The action pruning system based on adaptive streaming bitrate as described in claim 4, characterized in that, The adjustment of the probability vector for selecting all bitrate versions includes: The probability vector of all bitrate versions selected before adjustment is ,in, For bitrate version The probability of being selected for download; After adjustment by the bitrate selection mask module, the probability vector for the selection of all bitrate versions is: ,in: 。 6. A streaming media bitrate adaptive method, employing the system described in claim 5, characterized in that, include: The streaming media file is divided into time segments on the server side. The playback duration is fixed as follows: slice sequence Each streaming media slice is encoded as Versions with different bitrates And cache; When the user initiates an action against the first When a streaming media slice is requested, the video slice number k to be downloaded, historical network throughput information, and user viewing information are fed back to the streaming media bitrate adaptive action pruning system described in claim 1 to obtain the optimal download bitrate. ; Combine the request and the optimal download bitrate The request is sent to the server, and the server, based on the request, provides the bitrate version as follows: The A streaming media slice is sent to the user terminal, wherein When the first When a streaming media segment begins downloading, the playable streaming media duration stored in the user's player buffer is [duration missing]. .

7. A terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it can be used to run the method of claim 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, this program can be used to run the method described in claim 6.

Images