Multi-user video transmission fairness improvement method based on double-layer optimization

Abstract

The invention discloses a multi-user video transmission fairness improvement method NetABR based on double-layer optimization. According to the method, a multi-agent adaptive bit rate (ABR) algorithm framework aiming at different terminal devices is provided, and the service quality of different users is optimized in a targeted manner by deploying an ABR module with a device sensing capability at a client. Besides, the optimization of global fairness is realized by introducing a bandwidth allocation module, and a training architecture based on a double-layer optimization model is provided, so that the optimization effect is remarkably improved. According to the method, the ABR module with the equipment perception capability and the training architecture based on the double-layer optimization model are successfully combined, the optimization effect of global fairness is improved while fine-grained service quality optimization is achieved, and high robustness is shown in different network scenes.

Description

Technical Field

[0001] The present invention relates to the field of network communication technologies, and in particular, to an improved method for multi-user video transmission fairness based on double-layer optimization. Background Art

[0002] After decades of development, watching videos has become an important way for people to obtain information and carry out entertainment activities in their daily lives. More and more streaming providers, such as Youtube, Netflix, BiliBili, etc., are working hard to improve the user experience during video viewing. The Adaptive Bitrate (ABR) algorithm is an important means to achieve the above goal. At the video provider, each video is divided into video chunks of a certain length. These video chunks will be processed into different bitrates and correspond to different video qualities. At the client side, users can select the corresponding video bitrate according to their own needs to achieve the expected viewing experience. Generally speaking, the larger the bitrate of the video chunk, the higher its picture quality, the larger the file size, and the higher the transmission delay. The Quality of Experience (QoE) of users is an important indicator to measure user satisfaction in the above process. Generally speaking, it consists of three parts: video bitrate size, buffering time, and video bitrate fluctuation. The goal of the ABR algorithm is to select the appropriate bitrate for the next video chunk for the user according to the current video playback state of the user to maximize the user QoE. In the academic community, a series of ABR algorithms have emerged. One prominent method is the ABR algorithm based on Reinforcement Learning (RL), which can fully explore the potential relationship between the video playback state and QoE and exhibits good performance in complex network environments. However, previous research on ABR algorithms has mostly focused on single-user scenarios. In the real environment, multiple users often access the video server through a bottleneck link. Therefore, how to achieve fairness in multi-user scenarios has become a key focus.

[0003] In a multi-user streaming scenario, it is challenging to achieve fairness among users without centralized management because each user aims to maximize their individual QoE. This greedy control strategy lacking global information undermines fairness. Although ABR algorithms can combine congestion control algorithms like Cubic to mitigate the above issues, such solutions ignore the specific requirements of the streaming transmission scenario and do not specifically optimize fairness and user QoE. Existing solutions considering multi-user streaming scenarios mainly include Minerva and Muabr. Minerva was proposed in the paper "End-to-End Transport for Video QoE Fairness". In Minerva, the video server collects users' video request information and optimizes multi-user fairness through a dynamic bandwidth allocation strategy. In the scheme design, the author constructs a utility function to estimate the QoE of users currently and in a future period. Based on the utility function values of all online users, the video server can allocate corresponding bandwidth to each user. Through mathematical proof, the author shows that the final system will converge and tend to be stable, at which time the utility function values of all users are equal, thus achieving fairness. In terms of scheme deployment, the author modified the code of the Cubic protocol to achieve dynamic bandwidth allocation by updating the transmission window size in real time; Muabr was proposed in the paper "MUABR: Multi-user Adaptive Bitrate Algorithm based Multi-agent Deep Reinforcement Learning". In terms of system architecture, Muabr is similar to Minerva. The video server collects the video playback status of all users and optimizes fairness through dynamic bandwidth allocation. Bandwidth allocation is achieved through a bandwidth allocation coefficient that takes into account four parameters, namely the user's current buffer size, the user's current priority, the video block size, and the user's bandwidth allocation in the past period. On the client side, each user uses the ABR algorithm of RL to select the video block bitrate and also outputs the user priority to indicate the user's current demand for high bandwidth. Correspondingly, the video server will allocate higher bandwidth to users with higher priorities to meet their needs.

[0004] Although the above solutions have a certain degree of feasibility, however, they still have the following defects:

[0005] 1. Poor performance in a dynamic network environment. Existing solutions such as Minerva and Muabr assume that the throughput of the bottleneck link is fixed, which does not conform to the real network scenario. Due to the complexity of the network environment and the competition of multiple traffic flows for link resources, the available bandwidth for streaming video transmission fluctuates rapidly. Minerva and Muabr can be classified as rule-based optimization solutions, that is, such solutions construct a fixed utility function and optimize global fairness based on the utility function. Placing existing solutions in such a dynamic network environment, it can be found that their performance degrades sharply. The experimental results of our previous work show that taking the average bandwidth allocation as the benchmark algorithm, under the Norway dataset, the performance of Minerva is significantly worse than the benchmark algorithm, while the performance of Muabr is only slightly better than the benchmark algorithm; under the FCC dataset, the performances of Minerva and Muabr are both worse than the benchmark algorithm. This further shows that existing solutions do not optimize for the dynamic network environment specifically, making it difficult for them to be applied in real network scenarios.

[0006] 2. Poor fairness optimization results. To solve the problem that rule-based fairness optimization solutions are difficult to adapt to dynamic network scenarios, some scholars have proposed RL-based fairness optimization solutions. However, simply introducing RL does not bring an improvement in fairness. In the traditional architecture, the client and the video server separately train their ABR algorithms and bandwidth allocation algorithms. This architecture is simple and easy to implement, only requiring both parties to collect the data they need to complete the training of their own models. However, in the actual verification process, training the two algorithms separately does not make the fairness performance of the system higher than the benchmark algorithm. This is because this training architecture severs the relationship between the ABR algorithm and the bandwidth allocation algorithm. In reality, these two parts affect each other. For example, when a user requests a video block with a higher bitrate, the video server needs to balance the current QoE of this user and the fairness of the QoE of multiple users over a period of time, and allocate as appropriate bandwidth to it without causing a huge impact on the current QoE of this user (such as causing rebuffering) to ensure that the QoE of other users is not damaged.

[0007] To solve the above problems, the present invention combines RL and a bi-level optimization model, and proposes a fairness improvement method for multi-user video transmission based on bi-level optimization - NetABR. Summary of the Invention

[0008] Aiming at the problems existing in the prior art, such as poor performance in a dynamic network environment and poor fairness optimization effect, the present invention proposes a fairness improvement method for multi-user video transmission based on bi-level optimization - NetABR.

[0009] To achieve the above object, the present invention provides the following technical solutions:

[0010] The present invention provides an improved method for multi - user video transmission fairness based on double - layer optimization. This solution decomposes the multi - user streaming video transmission process into the following steps:

[0011] S1. User i, based on network status, including information such as network bandwidth prediction, buffer length, and the number of remaining video blocks, uses the ABR algorithm to select the bitrate of the next video block and transmits this request to the video server using the HTTP protocol;

[0012] S2. The video server collects the network status of current online users, including information such as user QoE, user buffer length, and the bitrate of the previous video block in the past period of time, and implements user grouping based on the K - means algorithm. Subsequently, the information within the group will be aggregated to generate an aggregated information matrix of a fixed size. This matrix will be used as the input of the bandwidth allocation module and output the bandwidth allocation coefficient w for the next time slice;

[0013] S3. Implement bandwidth allocation based on w and transmit the video block;

[0014] S4. Repeat steps S1 - S3 until user i finishes watching the current video.

[0015] Furthermore, in step S1, user i uses the RL - based ABR algorithm to select the bitrate of the next video block based on the network status. Specifically, the state space, action space, and reward in the ABR module are set as follows:

[0016] State space: The state space of the ABR module can be represented by the vector where yt is an estimate of the network throughput during the transmission of the past v video blocks, dt is the transmission time of the past v video blocks, zt is the file size corresponding to each available bitrate of the available video blocks at the next moment, ft is the current buffer size, ht is the number of remaining video blocks of the current video, and xt -1 is the bitrate of the previous video block.

[0017] Action space: The action space of the ABR module is the video bitrate at the next moment, that is xt is obtained by the policy network based on the current state This conversion process is expressed as where θ is the parameter of the policy network.

[0018] Reward: After making an action decision the ABR module will obtain the feedback reward from the environment This feedback will guide the ABR module to conduct subsequent learning, thereby optimizing its policy network. The goal of the ABR module is to maximize user QoE, so is defined as Here, QoE is defined as follows:

[0019] QoEi(xi ,t ,bi ,t )=q(xi ,t )-μ1T(xi ,t ,bi ,t )-μ2|q(xi ,t-1 )-q(xi ,t )|

[0020] Where q(xi ,t ) represents the video block bit rate xi selected at time t ,t The corresponding user perceived quality, generally speaking, q(xi ,t ) and xi ,t is positively correlated, that is, the higher the video block bit rate, the higher the user perceived quality q(xi ,t ) is larger. Considering the heterogeneity of terminal devices, the present invention uses the number of pixels per unit screen area (Normalized Pixels-Per-Degree, N-PPD) to measure q(xi ,t ), which is related to the device screen size and video block resolution. With this design, NetABR can capture the impact of different playback devices on QoE and achieve device-aware, targeted QoE optimization. In addition, in the above formula, T(xi ,t ,bi ,t ) means downloading video chunk xi ,t The rebuffering time caused by |q(xi ,t-1 )-q(xi ,t )| represents the bit rate difference between the previous and next two video blocks. The above formula shows that user QoE is related to three parts, namely, the bit rate of a single video block, the rebuffering time, and the bit rate fluctuation. μ1 and μ2 are weight parameters used to achieve a trade-off between the above three parts.

[0021] Furthermore, after completing the selection of an action, the client transmits the request to the video server via the HTTP protocol.

[0022] Furthermore, in step S2, the video server collects status information of the current online user, including the user's QoE in the past period of time, the current buffer length, the bit rate of the previous video block, etc.

[0023] Furthermore, the video server uses a grouping method based on K-means to achieve scalability for any number of users. This algorithm groups users based on the current user status information. Users within the same group are considered to have similar QoE expectations and are assigned the same bandwidth.

[0024] Furthermore, the information within the same group will be aggregated, and this process is shown in the following formula:

[0025]

[0026] where Gc is the within-group aggregated information of group c, gi is the individual user feature vector, C is the set of groups, and nc is the number of users within the current group c. gi contains a series of lower-layer status and decision information, and this vector is defined as gi = (qi ,t , yi ,t , di ,t , fi ,t , xi ,t ), where qi ,t is the QoE of the user at time t.

[0027] Furthermore, the aggregated within-group information will be used as the input to the bandwidth allocation module and output the bandwidth allocation coefficients for the next time slice. The state space, action space, and reward setting in the bandwidth allocation module are as follows:

[0028] State space: The state space of the bandwidth allocation module can be represented by the vector where Gi is the within-group aggregated message of group i. For simplicity of expression, the final bandwidth allocation vector bt is represented as where η is the policy network parameter.

[0029] Action space: The action space of the bandwidth allocation module is represented as where wi represents the bandwidth coefficient allocated to group i. The bandwidth allocated to each user can be obtained through the following formula:

[0030]

[0031] where nj ,t represents the number of users belonging to group j at time t. For simplicity of expression, the final bandwidth allocation vector bt is represented as where η is the policy network parameter.

[0032] Reward: The optimization objective is to maximize the QoE fairness among multiple users. Based on max-min fairness, the upper-layer reward is defined as

[0033] Furthermore, in combination with the above definitions, the problem of optimizing the policy parameters of the ABR module and the bandwidth allocation module during the training process can be transformed into the following bilevel optimization problem:

[0034]

[0035] where represents the objective function of the upper-level optimization. The optimization of this objective function is related to the parameter η, represents the objective function of the lower-level optimization. The optimization of this objective function is related to the parameters η and θi. The above has been proven to be differentiable for iterative optimization. Therefore, through continuous exploration, RL can achieve the joint optimization of the upper and lower layer objectives.

[0036] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0037] The present invention proposes a method for improving the fairness of multi-user video transmission based on bilevel optimization, NetABR. Compared with the existing solutions, the present invention has achieved improvements in multi-user fairness, adaptation to dynamic available bandwidth, and QoE optimization for user devices.

[0038] 1. In terms of multi-user fairness, NetABR models the multi-user global fairness optimization problem as a bilevel optimization problem. By combining the ABR module and the bandwidth allocation module for joint training, NetABR avoids the problem of training the two separately in the previous solutions. This design enables NetABR to fully explore and exploit the decision-making correlation between the upper and lower layers, enabling the ABR module and the bandwidth allocation module to make a more accurate estimation of the future expectations of the optimization objective and achieve better system performance.

[0039] 2. In terms of adaptation to dynamic available bandwidth, compared with existing solutions such as Minerva and Muabr, which are rule-based bandwidth allocation solutions, NetABR realizes a learning-based bandwidth allocation solution by introducing RL. With the powerful exploration and trial-and-error ability of RL, NetABR can fit an excellent bandwidth allocation strategy and show good performance in dynamic available bandwidth scenarios. In addition, NetABR fully considers the difficulty of RL in handling scenarios with variable numbers of multi-users. Through the K-means algorithm and the aggregation of intra-group user information, NetABR realizes the expansion for any number of users, ensuring the usability of the system in real scenarios.

[0040] 3. In terms of QoE optimization for user devices, NetABR takes into account that different user devices may have different QoEs for the same video chunk, and introduces N-PPD as an important parameter for measuring QoE. Based on this design, NetABR has the ability to sense user devices and can achieve targeted QoE optimization.

[0041] Generally speaking, NetABR has obvious advantages compared with existing solutions. Whether it is in terms of the QoE of a single user and the fairness of multiple users, NetABR has achieved corresponding optimizations and can show good performance in real and dynamic network scenarios. Brief Description of the Drawings

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required to be used in the embodiments. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings.

[0043] Figure 1 It is an architecture diagram of a multi-user streaming media video transmission method based on joint optimization of adaptive bitrate and bandwidth allocation provided by an embodiment of the present invention.

[0044] Figure 2 It is a flowchart of a multi-user streaming media video transmission method based on joint optimization of adaptive bitrate and bandwidth allocation provided by an embodiment of the present invention. Detailed Embodiment

[0045] Most existing solutions for multi-user streaming media video transmission scenarios have problems such as poor performance in dynamic network environments and poor fairness optimization. The present invention combines reinforcement learning and a two-layer optimization model to design an improved method for multi-user video transmission fairness based on two-layer optimization.

[0046] Specifically, most traditional multi-user streaming media video transmission solutions adopt rule-based bandwidth allocation strategies, and this strategy degrades severely in network scenarios with large fluctuations in real available bandwidth. The present invention proposes an RL-based bandwidth allocation strategy and models the global fairness optimization problem as a two-layer optimization problem for solution. With the powerful exploration ability of RL, the bandwidth allocation module can better adapt to dynamic network environments and show good robustness in different test scenarios. In addition, the training architecture of two-layer optimization enables the ABR module located in the lower layer of the architecture and the bandwidth allocation module located in the upper layer of the architecture to achieve joint optimization, thus significantly improving the optimization effect of fairness.

[0047] In addition, to address the problem of single QoE modeling in existing solutions, the present invention proposes an ABR module with device awareness capabilities. This module introduces the N-PPD attribute associated with the user device during the QoE modeling process, thereby achieving targeted and fine-grained QoE optimization.

[0048] To better understand the technical solution, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described examples are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those of ordinary skill in the art based on this application belong to the scope protected by the present invention.

[0049] The multi-user streaming video transmission method NetABR based on the joint optimization of adaptive bitrate and bandwidth allocation proposed by the present invention has an overall architecture diagram as Figure 1 shown, including two parts: the ABR module located at the user side and the bandwidth allocation module located at the video server side. At the user side, each ABR module determines the bitrate of the next video block based on the user's current and past video playback states for a period of time. These video states include the current buffer length, the file sizes corresponding to each bitrate of the next video block, the download times of the past few video blocks, and the number of remaining video blocks. At the video server side, the bandwidth allocation module determines the bandwidth size allocated to each user based on the user's decision and the QoE state in the past period of time, achieving fairness optimization among multiple users.

[0050] For the process of NetABR, specifically, as Figure 2 shown, it includes the following steps:

[0051] S1. User i selects the bitrate of the next video block using the ABR algorithm based on network states, including information such as network bandwidth prediction, buffer length, and the number of remaining video blocks, and transmits this request to the video server using the HTTP protocol.

[0052] The present invention uses an RL-based ABR algorithm, and its state space, action space, and reward setting are as follows:

[0053] State space: The state space of the ABR module can be represented by the vector where yt is an estimate of the network throughput during the transmission of the past v video blocks, dt is the transmission time of the past v video blocks, zt is the file sizes corresponding to all available bitrates of the available video blocks at the next moment, ft is the current buffer size, ht is the number of remaining video blocks of the current video, and xt -1 is the bitrate of the previous video block.

[0054] Action Space: The action space of the ABR module is the video bitrate at the next moment, that is xt is obtained by the policy network based on the current state This conversion process is expressed as where θ is the parameter of the policy network.

[0055] Reward: After making an action decision the ABR module will obtain the feedback reward from the environment This feedback will guide the ABR module to perform subsequent learning, thus realizing the optimization of its policy network. The goal of the ABR module is to maximize the user QoE, so is defined as Here, the definition of QoE is as follows:

[0056] QoEi(xt ,i ,bi ,t ) = q(xi ,t ) - μ1T(xi ,t ,bi ,t ) - μ2|q(xi ,t-1 ) - q(xi ,t )|

[0057] where q(xi ,t ) represents the user perceived quality corresponding to the video chunk bitrate xi ,t selected at time t. Generally speaking, q(xi ,t ) is positively correlated with xi ,t , that is, the higher the video chunk bitrate, the greater the user perceived quality q(xi ,t ). Considering the heterogeneity of terminal devices, the present invention uses the number of pixels per unit screen area (Normalized Pixels-Per-Degree, N-PPD) to measure q(xi ,t ), and this function is related to the device screen size and the video chunk resolution. With such a design, NetABR can capture the impact of different playback devices on QoE and achieve targeted QoE optimization with device awareness. In addition, in the above formula, T(xi ,t ,bi ,t ) represents the rebuffering time brought by downloading the video chunk xi ,t , and |q(xi ,t-1 ) - q(xi ,t )| represents the bitrate difference between two consecutive video chunks. The above formula shows that the user QoE is related to three parts, namely, the single video chunk bitrate, the rebuffering time, and the bitrate fluctuation. μ1 and μ2 are weight parameters used to achieve a trade-off among the above three parts.

[0058] S2. The video server collects the network status of current online users, including information such as user QoE, user buffer length, and the bitrate of the previous video chunk over a period of time, and implements user grouping based on the K-means algorithm. Subsequently, the information within the group will be aggregated to generate an aggregated information matrix of a fixed size, which will be used as the input of the bandwidth allocation module and output the bandwidth allocation coefficient w for the next time slice.

[0059] The video server uses the K-means-based grouping method to achieve scalability for any number of users. This algorithm groups users based on the current user status information. Users within the same group are considered to have similar QoE expectations and are assigned the same bandwidth. After grouping, the information within the same group will be aggregated, and this process is shown in the following formula:

[0060]

[0061] where Gc is the within-group aggregation information of group c, gi is the individual user feature vector, C is the set of groups, and nc is the number of users in the current group c. gi contains a series of lower-layer status and decision information, and this vector is defined as gi = (qi ,t , yi ,t , di ,t , fi ,t , xi ,t ), where qi ,t is the QoE of the user at time t.

[0062] The aggregated within-group information will be used as the input of the bandwidth allocation module and output the bandwidth allocation coefficient for the next time slice. The state space, action space, and reward setting in the bandwidth allocation module are as follows:

[0063] State space: The state space of the bandwidth allocation module can be represented by the vector where Gi is the within-group aggregation message of group i. For simplicity of expression, the final bandwidth allocation vector bt is represented as where η is the policy network parameter.

[0064] Action space: The action space of the bandwidth allocation module is represented as where wi represents the bandwidth coefficient allocated to group i. The bandwidth allocated to each user can be obtained through the following formula:

[0065]

[0066] where nj ,t represents the number of users belonging to group j at time t. For simplicity of expression, the final bandwidth allocation vector bt is represented as where η is the policy network parameter.

[0067] Reward: The optimization objective is to maximize the QoE fairness among multiple users. Based on max-min fairness, the upper-layer reward is defined as

[0068] S3. Implement bandwidth allocation based on w and transmit video chunks;

[0069] S4. Repeat steps S1 - S3 until user i finishes watching the current video.

[0070] The present invention proposes an improved method for multi-user video transmission fairness based on two-layer optimization. Compared with existing solutions, NetABR has achieved improvements in multi-user fairness, adaptation to dynamic available bandwidth, and QoE optimization for user devices.

[0071] In terms of multi-user fairness, NetABR models the multi-user global fairness optimization problem as a two-layer optimization problem. By combining the ABR module and the bandwidth allocation module for joint training, NetABR avoids the problem of separating the two for training in previous solutions. This design enables NetABR to fully explore and exploit the decision-making correlation between the upper and lower layers, allowing the ABR module and the bandwidth allocation module to make a more accurate estimation of the future expectations of the optimization objective and achieve better system performance.

[0072] In terms of adaptation to dynamic available bandwidth, compared with existing rule-based bandwidth allocation solutions such as Minerva and Muabr, NetABR implements a learning-based bandwidth allocation solution by introducing RL. With the powerful exploration and trial-and-error ability of RL, NetABR can fit an excellent bandwidth allocation strategy and perform well in dynamic available bandwidth scenarios. In addition, NetABR fully considers the difficulty of RL in handling scenarios with variable numbers of users. Through the K-means algorithm and intra-group user information aggregation, NetABR achieves scalability for any number of users, ensuring the usability of the system in real scenarios.

[0073] In terms of QoE optimization for user devices, NetABR takes into account that different user terminal devices may have different QoEs for the same video chunk and introduces N-PPD as an important parameter for measuring QoE. Based on this design, NetABR has the ability to perceive user devices and can achieve targeted QoE optimization.

[0074] Generally speaking, NetABR has obvious advantages compared with existing solutions. Whether in terms of the QoE of a single user and the fairness of multiple users, NetABR has achieved corresponding optimizations and can perform well in real and dynamic network scenarios.

[0075] The above description only details the preferred embodiments and principles of the present invention, and is not intended to limit the protection scope of the present invention. For those of ordinary skill in the art, any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be regarded as falling within the protection scope of the present invention.

Claims

1. A multi-user video transmission fairness improvement method NetABR based on double-layer optimization, characterized in that: It includes the following steps: S1. User i selects the bitrate of the next video chunk based on the network state, including information such as network bandwidth prediction, buffer length, and the number of remaining video chunks, and uses the HTTP protocol to transmit this request to the video server; S2. The video server collects the network states of the currently online users, including information such as the QoE of users in the past period, the buffer lengths of users, and the bitrate of the previous video chunk, and implements user grouping based on the K-means algorithm. Subsequently, the information within the group will be aggregated to generate an aggregated information matrix of a fixed size, which will be used as the input of the bandwidth allocation module and output the bandwidth allocation coefficient w for the next time slice; S3. Implement bandwidth allocation based on w and transmit the video chunk; S4. Repeat steps S1 - S3 until user i finishes watching the current video.

2. The multi-user streaming video transmission method based on the joint optimization of adaptive bitrate and bandwidth allocation according to claim 1, wherein In step S1, user i selects the bitrate of the next video chunk based on the network state. The present invention designs an ABR module based on Reinforcement Learning (RL). This module receives the current network state information, outputs an action selection, that is, the video bitrate for the next moment, and corrects the policy through the rewards feedback from the environment, and fits a better policy in multiple rounds of iteration. Specifically, the state space, action space, and reward setting in the ABR module are as follows: State space: The state space of the ABR module can be represented by the vector where y t is the estimated network throughput during the transmission of the past v video chunks, d t is the transmission time of the past v video chunks, z t is the file size corresponding to each available bitrate of the available video chunks at the next moment, f t is the current buffer size, h t is the number of remaining video chunks of the current video, x t-1 is the bitrate of the previous video chunk. Action space: The action space of the ABR module is the video bitrate at the next moment, that is x t is obtained by the policy network based on the current state and this conversion process is expressed as where θ are the parameters of the policy network. Reward: When making action decisions the ABR module will receive feedback rewards from the environment This feedback will guide the ABR module in subsequent learning to optimize its policy network. The goal of the ABR module is to maximize the user QoE, so is defined as 3. The multi-user streaming video transmission method based on the joint optimization of adaptive bitrate and bandwidth allocation according to claim 1, characterized in that, In step S1, the QoE is defined as follows: QoE i (x i,t ,b i,t ) = q(x i,t ) - μ1T(x i,t ,b i,t ) - μ2|q(x i,t-1 ) - q(x i,t )| where q(x i,t ) represents the user perceived quality corresponding to the video chunk bitrate x i,t selected at time t. Generally speaking, q(x i,t ) is positively correlated with x i,t , that is, the higher the video chunk bitrate, the greater the user perceived quality q(x i,t ). Considering the heterogeneity of terminal devices, the present invention uses the number of pixels per unit screen area (Normalized Pixels-Per-Degree, N-PPD) to measure q(x i,t ), and this function is related to the device screen size and the video chunk resolution. With such a design, NetABR can capture the impact of different playback devices on QoE and achieve targeted QoE optimization with device awareness. In addition, in the above formula, T(x i,t , b i,t ) represents the rebuffering time brought by downloading the video chunk x i,t , and |q(x i,t-1 ) - q(x i,t )| represents the bitrate difference between two consecutive video chunks. The above formula shows that the user QoE is related to three parts, namely, the single video chunk bitrate, the rebuffering time, and the bitrate fluctuation. μ1 and μ2 are weight parameters used to achieve a trade-off among the above three parts.

4. The multi-user streaming video transmission method based on the joint optimization of adaptive bitrate and bandwidth allocation according to claim 1, characterized in that In step S1, after completing the selection of an action, the client will transmit this request to the video server through the HTTP protocol.

5. The multi - user streaming video transmission method based on the joint optimization of adaptive bitrate and bandwidth allocation according to claim 1, wherein In step 2, the video server collects the network states of the currently online users and determines the bandwidth allocation coefficient for the next time slice. To achieve the expansion for any number of users, the present invention uses a grouping method based on K-means. This algorithm groups users based on the current playback state of users. Users within the same group are considered to have the same QoE expectation and are allocated the same bandwidth.

6. The multi - user streaming video transmission method based on the joint optimization of adaptive bitrate and bandwidth allocation according to claim 1, characterized in that, In step 2, after completing user grouping, the user information within the same group will be aggregated to generate a vector of a fixed length. The process of information aggregation is shown in the following formula: Among them, G c is the intra-group aggregation information of group c, and g i is a single user feature vector. C is the set of groups, and n c is the number of users within the current group c. g i contains a series of lower-layer state and decision information, and this vector is defined as g i =(q i,t , y i,t , d i,t , f i,t , x i,t ), where q i,t is the QoE of the user at time t.

7. The multi-user streaming video transmission method based on the joint optimization of adaptive bitrate and bandwidth allocation according to claim 1, characterized in that In step 2, the bandwidth allocation module receives the aggregated information within the group and makes a decision based on RL. The state space, action space, and reward setting in the bandwidth allocation module are as follows: State space: The state space of the bandwidth allocation module can be represented by the vector where G i is the intra-group aggregation message of group i. To simplify the expression, the final bandwidth allocation vector b t is represented as where η is the policy network parameter. Action space: The action space of the bandwidth allocation module is represented as where w i represents the bandwidth coefficient allocated to group i. The bandwidth allocated to each user can be obtained through the following formula: Among them, n j,t represents the number of users belonging to group j at time t. To simplify the expression, the final bandwidth allocation vector b t is expressed as where η is the policy network parameter. Reward: The optimization objective is to maximize the QoE fairness among multiple users. Based on max-min fairness, the upper-layer reward is defined as 8. The multi-user streaming video transmission method based on the joint optimization of adaptive bitrate and bandwidth allocation according to claim 1, characterized in that In step S3, implement bandwidth allocation according to the bandwidth allocation coefficient w output in step S2 and transmit the video chunk with the corresponding bitrate requested by the user.

9. The multi-user streaming video transmission method based on the joint optimization of adaptive bitrate and bandwidth allocation according to claim 1, characterized in that The termination condition of step S4 is: Once user i finishes watching the current video, it ends.

Images