Edge cloud video block scheduling method driven by experience quality
By adopting experience-quality-driven video block scheduling method in the edge cloud system, using reinforcement learning to optimize cache replacement and transcoding decisions, the problem of difficult user experience quality and delay can be met simultaneously, and high-quality and low-latency video blocks are achieved Get it.
Patent Information
- Application Number
- CN202411432197.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-14
- Publication Date
- 2025-02-14
AI Technical Summary
The prior art is difficult to effectively improve the user experience quality, especially in the process of obtaining video blocks, and delay and video quality are difficult to meet high user expectations at the same time.
Adopt the experience-quality-driven edge cloud video block scheduling method, by making cache replacement and transcoding decisions on edge servers, selecting a video block delivery method that maximizes user experience quality, and sending video blocks from the cloud if necessary. This method uses reinforcement learning neural network to optimize video block scheduling strategies based on user behavior and system state.
It improves the quality of users' acquisition of video blocks and reduces latency, while reducing the cost of obtaining video blocks from the cloud and improving the quality of user experience.
Smart Images
Figure CN119450086A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of video block scheduling, and in particular to an edge-cloud video block scheduling method driven by quality of experience. Background Art
[0002] In recent years, with the rapid growth of video traffic, video content has become one of the main components of the Internet, occupying a large amount of network bandwidth. In 2017, video traffic accounted for 75% of global network traffic and will continue to grow. At the same time, with the improvement of video resolution and frame rate, as well as the continuous pursuit of smoothness and clarity, users' expectations for video quality are also increasing. Although 5G technology can increase network bandwidth, the Internet may soon be overwhelmed by a large amount of video traffic, which will affect users' viewing experience.
[0003] For users, the video viewing experience has gone beyond simple content provision. Users are more concerned about being able to watch video content in a high-quality, seamless, and smooth manner. In this context, Quality of Experience (QoE) has become one of the hot research topics in academia and industry.
[0004] Researchers generally recognize that providing high-quality user experience involves not only optimizing technical performance, but also requires a deep understanding of user perception, expectations, and behavior. User experience quality measures the user's satisfaction with service performance, that is, the user's subjective feeling about the performance of video streaming services provided by mobile networks. Therefore, how to improve the quality of video blocks obtained by users and reduce latency is an urgent problem to be solved. Summary of the invention
[0005] In view of the above-mentioned technical problems, the present invention provides an edge-cloud video block scheduling method driven by experience quality, which can improve the quality of video blocks obtained by users and reduce delays.
[0006] The technical solution of the present invention is: an edge-cloud video block scheduling method driven by experience quality, comprising the following steps:
[0007] S01: Each edge server makes cache replacement and transcoding decisions based on the state of the previous time slot;
[0008] S02: The user moves within an edge server or switches from one edge server to another;
[0009] S03: The user requests a video block from the local edge server;
[0010] S04: The edge server selects a video block delivery method that maximizes the user experience quality to satisfy the user's request. If the nearby edge server cannot satisfy the user's request, the video block will be sent from the cloud.
[0011] S05: The system records the status and reward of this operation and updates the parameters of the reinforcement learning neural network.
[0012] Preferably, an edge cloud system is included, and the edge cloud system consists of three components: a user, a cloud end, and an edge server; among the three components of the edge cloud system:
[0013] (1) The content server in the cloud stores all the video chunks with different bit rates requested by the user. It communicates with the edge server and user devices through a high-bandwidth network and has computing and storage capabilities.
[0014] (2) Users request and receive the video chunks they need from the system. Different users have different preferences for the bit rate and latency of video streams. In addition, users may move and switch to the area covered by another edge server.
[0015] (3) The edge server can cache and transcode video blocks for the user's video request. Since receiving video blocks from the cloud may result in high latency, the edge server will provide video block services nearby. When an edge server does not cache a video block requested by a user, it will request and receive the required video block from an adjacent edge server. Only when all adjacent edge servers do not cache the required video block will it seek help from the cloud.
[0016] Preferably, assuming that time is evenly divided into T time slots, a system including J cache servers is established, j=0 represents the content server on the cloud, and the cache size, computing resources, number of users, number of video blocks and bandwidth of the jth edge server are represented as W j , P j , U j , I j , R j ,These edge servers are connected to each other through the network. The set of all video blocks and the set of different bit rates of each video block are represented by I and Q respectively. Denotes the qth bit rate of the ith video chunk, the set index of all users is U, assuming that the video stream consists of consecutive video chunks, and users watch these video chunks in sequence. The edge server to which the user belongs is called the local edge server.
[0017] Preferably, the edge server needs to meet the following cache restrictions:
[0018]
[0019] ,in Respectively indicate the cache status, size and whether the video block is replaced.
[0020] Preferably, the edge server needs to meet the following transcoding restrictions:
[0021]
[0022] ,in Indicates whether the video chunk is transcoded.
[0023] Preferably, the delay of the user receiving the video block needs to satisfy:
[0024]
[0025] ,in represents the delay in sending video chunks from the edge server to the user. represents the bandwidth allocated to the user in time slot t, and denote the time and resources required to transcode a video chunk from bit rate q′ to q, Represents computing resources allocated to a user.
[0026] Preferably, in addition to directly requesting video chunks from the local edge server, users have multiple ways to obtain video chunks. Different methods will result in different delays, and the system will select the method with the shortest delay. The following are four ways for users to obtain video chunks:
[0027] (1) If the local edge server caches the video block, the user can obtain the video block directly from the local edge server with a delay equal to the time required to send the video block;
[0028] (2) If the local edge server caches the high bit rate version of the video chunk, the user can also directly obtain the video chunk from the local edge server. The latency includes the time required to send and transcode the video chunk.
[0029] (3) If the neighboring edge server caches the video block, the video block needs to be transmitted from the neighboring edge server to the local edge server and then sent to the user. The delay includes the time of sending the video block and transmitting it from the neighboring edge server. Since transcoding at the neighboring edge server will cause significant delay, this chapter does not consider the case where the neighboring edge server caches the high bit rate version of the video block;
[0030] (4) If none of the above situations apply, it can be assumed that the user cannot obtain the video block in the edge server and needs to obtain it from the cloud. In this case, the latency is the time required to send the video block from the cloud;
[0031] Therefore, the delay for users to obtain video chunks can be expressed as:
[0032]
[0033] , where δ is a small constant used to avoid division by zero.
[0034] Preferably, the probability that a user enters the next edge server at time t is:
[0035]
[0036] , where the current edge server has a range of x meters and the vehicle speed distribution follows the distribution from v 0 m / s to v 1 Uniform distribution of m / s.
[0037] Preferably, a joint optimization target is also included, and the joint optimization target consists of three components: video block bit rate, delay, and cost of obtaining video blocks from the cloud. u，1 and x u，2 to represent the user's preference for video bitrate and delay, Q(·) represents the user's satisfaction function regarding bitrate variation, and the cost of obtaining video chunks from the cloud is expressed as Therefore, the joint optimization objective is expressed as:
[0038]
[0039] Preferably, a training method of a reinforcement learning algorithm is also included, which is specifically as follows:
[0040] At each time slot t, each agent selects actions based on the current state and random exploration probability and executes these actions. Then, the user moves and requests a video chunk. Subsequently, the agent responds to the user request and the environment returns the next state and reward. The samples are stored in the experience replay buffer. Finally, each agent updates its parameters based on the randomly selected samples.
[0041] Compared with the prior art, the advantages of the present invention are:
[0042] (1) The present invention provides an edge-cloud video block scheduling method driven by quality of experience. By providing a scheduling strategy, it ensures that the user's quality of experience can be improved. It can not only ensure the quality of the video blocks obtained by the user and reduce the delay, but also reduce the cost of obtaining video blocks from the cloud.
[0043] (2) Based on game theory, the present invention models the scheduling problem of video blocks as a multi-agent game model, and proves that the proposed multi-agent game is a state-based potential game. By maximizing the reward function of each agent separately, the algorithm can achieve the goal of maximizing the global potential function. Therefore, the multi-agent game problem can be decomposed into individual agent decision-making problems. At the same time, the present invention takes into account the resource limitation problem of each edge server, which can be more in line with practical applications. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:
[0045] Figure 1 is a flow chart of the scheduling method of the present invention;
[0046] Figure 2 It is the reward, hit rate, user experience quality, and cloud cost saving rate of the present invention and the two comparison algorithms;
[0047] Figure 3 The performance of the present invention and the two comparison algorithms under different storage scales is shown in Figure 2.
[0048] Figure 4 The performance of the present invention and the two comparison algorithms under different zipf parameters is shown in Figure 2. DETAILED DESCRIPTION
[0049] The above scheme is further described below in conjunction with specific examples. It should be understood that these examples are used to illustrate the present invention and are not limited to the scope of the present invention. The implementation conditions adopted in the examples can be further adjusted according to the conditions of the specific manufacturer, and the unspecified implementation conditions are usually the conditions in conventional experiments.
[0050] Example:
[0051] The edge-cloud video block scheduling method driven by the quality of experience of the present invention includes an edge-cloud system, and the edge-cloud system consists of three main components:
[0052] (1) The content server in the cloud stores all the video chunks with different bitrates requested by users. It communicates with edge servers and user devices through a high-bandwidth network and has powerful computing and storage capabilities.
[0053] (2) Users will request and receive the video chunks they need from the system. Different users have different preferences for the bitrate and latency of the video stream. In addition, users may move and switch to an area covered by another edge server.
[0054] (3) The edge server is able to cache and transcode video chunks for the user's video request. Since receiving video chunks from the cloud may result in high latency, the edge server will provide video chunk services nearby. When an edge server does not cache a video chunk requested by a user, it will request and receive the required video chunk from a neighboring edge server. Only when all neighboring edge servers do not cache the required video chunk will it seek help from the cloud.
[0055] The edge-cloud video block scheduling method driven by the quality of experience of the present invention comprises the following steps:
[0056] S01: Each edge server makes cache replacement and transcoding decisions based on the state of the previous time slot; the reinforcement learning model in step S01 needs to input five different states:
[0057] (1) User location L(t), which represents the edge server to which the user is currently connected.
[0058] (2) Video block R(t) requested by the user.
[0059] (3) User preference U(t).
[0060] (4) User movement probability P(t).
[0061] (5) The cache status C'(t) of the edge server. Then, based on the above status, the cache replacement decision C(t) and transcoding decision T(t) are output.
[0062] S02: The user moves within an edge server or switches from one edge server to another;
[0063] S03: The user requests a video block from the local edge server;
[0064] S04: The edge server selects the video chunk delivery method that maximizes the user experience quality to satisfy the user's request. If the nearby edge server cannot satisfy the user's request, the video chunk will be sent from the cloud; the system will select the method with the shortest delay. The following are four ways for the user to obtain the video chunk:
[0065] (1) If the local edge server caches the video chunk, the user can obtain the video chunk directly from the local edge server with a delay equal to the time required to send the video chunk.
[0066] (2) If the local edge server caches a high bit rate version of the video chunk, the user can also get the video chunk directly from the local edge server. The latency includes the time required to send and transcode the video chunk.
[0067] (3) If the neighboring edge server caches the video chunk, the video chunk needs to be transferred from the neighboring edge server to the local edge server and then sent to the user. The delay includes the time to send the video chunk and transfer it from the neighboring edge server. Since transcoding at the neighboring edge server will cause significant delay, this chapter does not consider the case where the neighboring edge server caches a high bit rate version of the video chunk.
[0068] (4) If none of the above situations apply, it can be assumed that the user cannot obtain the video chunk in the edge server and needs to obtain it from the cloud. In this case, latency is the time required to send the video chunk from the cloud.
[0069] The delay for users to obtain video chunks can be expressed as:
[0070]
[0071] , where δ is a small constant used to avoid division by zero.
[0072] S05: The system records the status and reward of this operation and updates the parameters of the reinforcement learning neural network.
[0073] The experimental test description of this case is as follows Figure 2 to Figure 4 This scheduling strategy not only improves the user experience quality, but also reduces the cloud cost, and has high use value and a wide range of application backgrounds.
[0074] In another technical solution, assuming that time is evenly divided into T time slots, a system including J cache servers is established, j=0 represents the content server on the cloud, and the cache size, computing resources, number of users, number of video blocks and bandwidth of the jth edge server are represented as W j , P j , U j , I j , R j ,These edge servers are connected to each other through the network. The set of all video blocks and the set of different bit rates of each video block are represented by I and Q respectively. Denotes the qth bit rate of the ith video chunk, the set index of all users is U, assuming that the video stream consists of consecutive video chunks, and users watch these video chunks in sequence. The edge server to which the user belongs is called the local edge server.
[0075] Edge servers need to meet the following cache restrictions:
[0076]
[0077]
[0078] ,in Respectively indicate the cache status, size and whether the video block is replaced.
[0079] Edge servers need to meet the following transcoding restrictions:
[0080]
[0081] ,in Indicates whether the video chunk is transcoded.
[0082] The delay for users to receive video chunks must satisfy:
[0083]
[0084] ,in represents the delay in sending video chunks from the edge server to the user. represents the bandwidth allocated to the user in time slot t, and denote the time and resources required to transcode a video chunk from bit rate q′ to q, Represents computing resources allocated to a user.
[0085] The probability that a user enters the next edge server at time t is:
[0086]
[0087] , where the current edge server has a range of x meters and the vehicle speed distribution follows the distribution from v 0 m / s to v 1 Uniform distribution of m / s.
[0088] It also includes a joint optimization objective, which consists of three components: video chunk bitrate, latency, and the cost of getting video chunks from the cloud. u，1 and x u，2 to represent the user's preference for video bitrate and delay, Q(·) represents the user's satisfaction function regarding bitrate variation, and the cost of obtaining video chunks from the cloud is expressed as Therefore, the joint optimization objective is expressed as:
[0089]
[0090] It also includes the training method of reinforcement learning algorithm, as follows:
[0091] At each time slot t, each agent selects actions based on the current state and random exploration probability and executes these actions. Then, the user moves and requests a video chunk. Subsequently, the agent responds to the user request and the environment returns the next state and reward. The samples are stored in the experience replay buffer. Finally, each agent updates its parameters based on the randomly selected samples.
[0092] The above examples are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit of the present invention should be included in the protection scope of the present invention.
Claims
1. An edge-cloud video block scheduling method driven by quality of experience, characterized in that: The following steps are involved: S01: Each edge server makes cache replacement and transcoding decisions based on the state of the previous time slot; S02: The user moves within an edge server or switches from one edge server to another; S03: The user requests a video block from the local edge server; S04: The edge server selects a video block delivery method that maximizes the user experience quality to satisfy the user's request. If the nearby edge server cannot satisfy the user's request, the video block will be sent from the cloud. S05: The system records the status and reward of this operation and updates the parameters of the reinforcement learning neural network.
2. The edge-cloud video block scheduling method driven by quality of experience according to claim 1, characterized in that: Including edge cloud system, the edge cloud system consists of three components: user, cloud, edge server; the three components of the edge cloud system: (1) The content server in the cloud stores all the video chunks with different bit rates requested by the user. It communicates with the edge server and user devices through a high-bandwidth network and has computing and storage capabilities. (2) Users request and receive the video chunks they need from the system. Different users have different preferences for the bit rate and latency of video streams. In addition, users may move and switch to the area covered by another edge server. (3) The edge server can cache and transcode video blocks for the user's video request. Since receiving video blocks from the cloud may result in high latency, the edge server will provide video block services nearby. When an edge server does not cache a video block requested by a user, it will request and receive the required video block from an adjacent edge server. Only when all adjacent edge servers do not cache the required video block will it seek help from the cloud.
3. The edge-cloud video block scheduling method driven by quality of experience according to claim 1, characterized in that: Assuming that time is evenly divided into T time slots, a system consisting of J cache servers is set up, j = 0 represents the content server on the cloud, and the cache size, computing resources, number of users, number of video blocks, and bandwidth of the jth edge server are represented as W j ,P j ,U j ,I j ,R j ,These edge servers are connected to each other through the network. The set of all video blocks and the set of different bit rates of each video block are represented by I and Q respectively. Denotes the qth bit rate of the ith video chunk, the set index of all users is U, assuming that the video stream consists of consecutive video chunks, and users watch these video chunks in sequence. The edge server to which the user belongs is called the local edge server.
4. The edge-cloud video block scheduling method driven by quality of experience according to claim 3, characterized in that: Edge servers need to meet the following cache restrictions: ,in Respectively indicate the cache status, size and whether the video block is replaced.
5. The edge-cloud video block scheduling method driven by quality of experience according to claim 3, characterized in that: Edge servers need to meet the following transcoding restrictions: ,in Indicates whether the video chunk is transcoded.
6. The edge-cloud video block scheduling method driven by quality of experience according to claim 3, characterized in that: The delay for users to receive video chunks must satisfy: ,in represents the delay in sending video chunks from the edge server to the user. represents the bandwidth allocated to the user in time slot t, and denote the time and resources required to transcode a video chunk from bit rate q′ to q, Represents computing resources allocated to a user.
7. The edge-cloud video block scheduling method driven by quality of experience according to claim 3, characterized in that: In addition to directly requesting video chunks from the local edge server, users have multiple ways to obtain video chunks. Different methods will result in different delays. The system will choose the method with the shortest delay. The following are four ways for users to obtain video chunks: (1) If the local edge server caches the video block, the user can obtain the video block directly from the local edge server with a delay equal to the time required to send the video block; (2) If the local edge server caches the high bit rate version of the video chunk, the user can also directly obtain the video chunk from the local edge server. The latency includes the time required to send and transcode the video chunk. (3) If the neighboring edge server caches the video block, the video block needs to be transmitted from the neighboring edge server to the local edge server and then sent to the user. The delay includes the time of sending the video block and transmitting it from the neighboring edge server. Since transcoding at the neighboring edge server will cause significant delay, this chapter does not consider the case where the neighboring edge server caches the high bit rate version of the video block; (4) If none of the above situations apply, it can be assumed that the user cannot obtain the video block in the edge server and needs to obtain it from the cloud. In this case, the latency is the time required to send the video block from the cloud; Therefore, the delay for users to obtain video chunks can be expressed as: ， where δ is a small constant used to avoid division by zero.
8. The edge-cloud video block scheduling method driven by quality of experience according to claim 3, characterized in that: The probability that a user enters the next edge server at time t is: ， The range of the current edge server is x meters, and the vehicle speed distribution follows a uniform distribution from v0m / s to v1m / s.
9. The edge-cloud video block scheduling method driven by quality of experience according to claim 3, characterized in that: It also includes a joint optimization objective, which consists of three components: video chunk bitrate, latency, and the cost of getting video chunks from the cloud. u,1 and x u,2 to represent the user's preference for video bitrate and latency, Q(·) represents the user's satisfaction function regarding bitrate variation, and the cost of obtaining video chunks from the cloud is expressed as Therefore, the joint optimization objective is expressed as:
10. The edge-cloud video block scheduling method driven by quality of experience according to claim 3, characterized in that: It also includes the training method of reinforcement learning algorithm, as follows: At each time slot t, each agent selects actions based on the current state and random exploration probability and executes these actions. Then, the user moves and requests a video chunk. Subsequently, the agent responds to the user request and the environment returns the next state and reward. The samples are stored in the experience replay buffer. Finally, each agent updates its parameters based on the randomly selected samples.