A dynamic video transmission scheduling method based on reinforcement learning

Abstract

A dynamic video transmission scheduling algorithm based on reinforcement learning, characterized in that it includes the following steps: 1) Acquisition of communication system parameters. 2) Establishment of optimization problem function: Let K be the number of users served by the system, and L be the number of OFDM resource blocks contained in the system. Then the optimization problem function is the user number assigned to each resource block l, so as to maximize the number of users who meet the QoS requirements. 3) Update the OFDM resource block allocation method of the communication system: Establish state, action and target reward functions, use the reinforcement learning network to output the user number assigned to each resource block l, and form the OFDM resource block allocation method of the communication system. 4) Update the reinforcement learning network parameters: Use the gradient descent method to update the reinforcement learning network parameters. 5) Determine whether the optimization end condition is met: After the algorithm outputs the resource block allocation result, observe whether the system has reached the target number of users that meet the QoS requirements. If not, return to 3) for the next iteration.

Description

Technical Field

[0001] The present invention belongs to the communication field, and in particular is aimed at resource allocation scenarios in the video transmission field. Background Art

[0002] With the rapid development of the new generation of wireless communication technology, the gradual implementation of applications such as robot clusters, vehicle-road collaboration, and virtual reality, as well as the increasing demand for high-definition video, people's demand for high-speed, high-reliability, and low-latency real-time video data transmission has increased year by year. In a wireless channel environment, multiple users use orthogonal frequency division multiplexing (OFDM) in the same base station for video transmission. Each user has certain requirements for the quality of service (QoS), including video frame transmission delay and frame loss rate. However, OFDM's communication resources such as the number of resource sub-blocks and frequency bands are very limited, which poses a huge challenge to the current wireless transmission of high-definition video. When communication resources are fixed, people hope to maximize the number of users in the system that meet the video service quality requirements, especially when communication resources are not sufficient. Therefore, for this business scenario, there is an urgent need for a multi-user dynamic video transmission cross-layer joint scheduling optimization solution to maximize the number of users in the communication system that meet the video service quality requirements. Summary of the invention

[0003] The present invention aims to provide a dynamic cross-layer joint scheduling optimization algorithm for video transmission, so that more users can meet their video service quality requirements. If only the traditional resource scheduling algorithm (such as the proportional fairness algorithm, etc.) is used, the video service quality of all users cannot meet the requirements when communication resources are insufficient or the number of users is large. The present invention can dynamically allocate communication resource blocks in the communication system according to the current user's QoS status, channel status, video frame size and delay requirements and other parameters, so that the QoS of more users can be met.

[0004] Technical Solution

[0005] A dynamic video transmission scheduling method based on reinforcement learning is characterized by being a cross-layer joint scheduling optimization algorithm for multi-user dynamic video transmission, comprising the following steps:

[0006] S1 obtains communication system parameters for algorithm training.

[0007] S2 establishes the optimization problem function: let K be the number of users served by the system, L be the number of OFDM resource blocks contained in the system, then the optimization problem function is the user number assigned to each resource block l, so as to maximize the number of users who meet the QoS requirements.

[0008] S3 updates the OFDM resource block allocation method of the communication system: establishes state, action and target reward functions, uses the reinforcement learning network to output the user number assigned to each resource block l, and forms the OFDM resource block allocation method of the communication system.

[0009] S4 updates the reinforcement learning network parameters: updates the reinforcement learning network parameters using the gradient descent method.

[0010] S5 determines whether the optimization end condition is met: after the algorithm outputs the resource block allocation result, observe whether the system reaches the user number target that meets the QoS requirements. If not, return to 3) for the next iteration.

[0011] S1. Obtain communication system parameters for algorithm training, including the number of system users K, the number of system OFDM resource blocks L, the wireless frequency band, the transmission time interval TTI, and the frame loss rate requirement of each user k (k = 1, 2, ..., K) in the system Maximum delay tolerance The number of subframes contained in the transmitted video frame And the channel signal-to-noise ratio h of each user on each OFDM resource block k,l (k=1,2,…,K;l=1,2,…,L) and other parameters for reinforcement learning algorithm training.

[0012] S2. Establish an optimization problem function. Let K be the number of users served by the system and L be the number of OFDM resource blocks contained in the system. Then we need to find a resource block allocation scheme to maximize the number of users that meet QoS requirements. Let A be a resource block allocation matrix with L rows and K columns, K QoS is the number of users that have met the QoS requirements, then the optimization function is

[0013] max K QoS

[0014] a ij ∈{0,1},a ij ∈A,i∈[0,L],j∈[0,K]

[0015]

[0016] In the above formula, a ij is the element of the resource block allocation matrix A, where i and j are subscripts.

[0017] S3. Update the OFDM resource block allocation method of the communication system.

[0018] (1) The present invention uses a reinforcement learning algorithm to update the matrix A. In combination with the application characteristics of the present system, the reinforcement learning state space, action space and reward designed by the present invention are as follows:

[0019] ① State space s: a set of parameters used to characterize the system state. Let the actual frame loss rate of K users at time t be O t = {O t,1 ,O t,2 ,…,O t,K}, the remaining transmission delay is T t ={T t,1 ,T t,2 ,…,T t,K}、The number of remaining sub-frames of the video frame is n t ={n t,1 ,n t,2 ,…,n t,K}, the channel state is h t =[h1,h2,…,h K ], where h k ={h t,1 ,h t,2 ,…,h t,L} represents the signal-to-noise ratio of the kth user on each OFDM resource block. Then the state space can be expressed as

[0020]

[0021] In the above formula, Indicates h t The transposed matrix of .

[0022] ② Action space: Matrix A is used to represent the allocation of OFDM resource blocks by the communication system. It is an L×K dimensional matrix.

[0023] ③ Reward: used to guide the learning process of the reinforcement learning network to achieve the goal of maximizing the number of users who meet QoS requirements. The reward function used in this invention is

[0024] r=∑ k R 1,k +R 2,k

[0025] In the above formula, R 1,k The reward for the frame loss rate of user k is given only when the video frame is transmitted. When the user just meets the frame loss rate requirement, the reward is given, and when it does not meet the requirement, the reward is not given; if the actual frame loss rate is much lower than the frame loss rate requirement, it means that the user occupies too many communication resources, so a decaying reward is given. 2,k In order to represent the reward of user k's current transmission rate, a reward is given after each TTI. If the transmission speed given by the current system to the user can be transmitted within the delay requirement, a reward is given, otherwise a penalty is given.

[0026] For the first part of the reward function, the remaining transmission delay T of user k ist,k = 0, if the number of remaining subframes is n t,k ≤0, it is determined that the video frame has been transmitted. Then R 1,k The expression is

[0027]

[0028] In the above formula, r1 is the reward value, p is the average number of OFDM resource blocks occupied by the user per frame transmission, and q k It is used to limit the frame loss rate of user k to the required level. It is set by the algorithm user. m is the proportional coefficient and needs to be adjusted according to the training results.

[0029] For the second part of the reward function, a reward is calculated and given at each transmission time interval (TTI), R 2,k The expression is

[0030]

[0031] In the above formula, r2 is the reward value, v need,k is the minimum transmission rate that user k needs to allocate if there is no frame loss, v k is the transmission rate currently allocated to user k by the system.

[0032] (2) The reinforcement learning algorithm of the present invention is a deep Q network. For each OFDM resource block, a deep Q network model is established to output the user number assigned to the resource block, that is, a row of the matrix A. The model is as follows: Figure 2 As shown in Figure 2, the outputs of all deep Q networks are assembled into a matrix A, which is the way the communication system allocates resource blocks.

[0033] Figure 2 In DQN1~DQN L is a reinforcement learning network, each of which corresponds to a resource block RB in the communication system. l ={a l,1 ,a l,2 ,…,a l,K}(l=0,1,2,…L) is the row vector of the resource block allocation matrix A, dimension K, and its elements satisfy

[0034]

[0035] {s t-1 ,A t ,r t ,s t} is a tuple that stores the system state at the last moment, the allocation of resource blocks, the reward after executing a TTI based on the allocation action, and the system state at the next moment. This tuple is called "experience" and is stored in the experience replay cache for use by the reinforcement learning network. After each TTI, the system extracts a portion of experience from the experience replay cache, forms a miniBatch, and sends it to the reinforcement learning network for the next step of learning.

[0036] S4. Update reinforcement learning network parameters.

[0037] (1) The action A output by the reinforcement learning network interacts with the environment and stores the result in the experience replay cache. A portion of the experience data is extracted from the experience replay cache and the network parameters of each deep Q network are updated using the gradient descent method.

[0038] (2) Update the state space parameters. For user k, use the following formula to update the state space parameters of each user:

[0039] Frame loss rate:

[0040] Remaining delay:

[0041] Number of remaining sub-frames in the video frame

[0042] In the above formula, n sent,k is the number of video subframes transmitted by the user in the current TTI. Channel state h t,k Updated based on current user feedback.

[0043] S5. Determine whether the optimization end condition is met: After the algorithm outputs the resource block allocation result, observe whether the target number of users that meet the QoS requirements of the communication system is reached. If not, return to S3 for the next iteration. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 Flow chart of the method of the present invention

[0045] Figure 2 The deep Q network model of the present invention

[0046] Figure 3 The performance comparison diagram of the method of the present invention and the currently commonly used video transmission scheduling method is shown in FIG.

[0047] Figure 4 The service quality change curve of the user using the algorithm of the present invention is

[0048] Figure 5 This is the service quality change curve of users using the polling algorithm

[0049] Figure 6 This is the service quality change curve of users using the maximum carrier-to-interference ratio algorithm

[0050] Figure 7 This is the service quality change curve of users using the proportional fairness algorithm

[0051] Figure 8 This is the user service quality change curve of the proportional fairness algorithm DETAILED DESCRIPTION

[0052] Since the video transmission service quality QoS of each user is highly correlated with the current user's location, channel quality, and the QoS of other users in the wireless channel scenario, it is necessary to reasonably allocate communication resources from a system macro perspective according to the current system status in order to maximize the number of users who meet QoS requirements. The present invention utilizes the frame loss rate, remaining transmission delay, number of remaining video subframes, channel status and other information of each user in the system at each moment, and proposes a dynamic video transmission scheduling algorithm based on deep reinforcement learning, which completes real-time dynamic allocation of the OFDM resource blocks of the communication system according to the current communication system status, thereby maximizing the number of users who meet QoS requirements in the system.

[0053] The algorithm implementation process proposed by the present invention is as follows:

[0054] S1. Obtain communication system parameters for algorithm training, including the number of system users K, the number of system OFDM resource blocks L, the wireless frequency band, the transmission time interval TTI, and the frame loss rate requirement of each user k (k = 1, 2, ..., K) in the system Maximum delay tolerance The number of subframes contained in the transmitted video frame And the channel signal-to-noise ratio h of each user on each OFDM resource block k,l (k=1,2,…,K;l=1,2,…,L) and other parameters for reinforcement learning algorithm training.

[0055] S2. Establish an optimization problem function. Let K be the number of users served by the system and L be the number of OFDM resource blocks contained in the system. Then we need to find a resource block allocation scheme to maximize the number of users that meet QoS requirements. Let A be a resource block allocation matrix with L rows and K columns, K QoS is the number of users that have met the QoS requirements, then the optimization function is

[0056] max K QoS

[0057] a ij ∈{0,1},a ij ∈A,i∈[0,L],j∈[0,K]

[0058]

[0059] In the above formula, a ij is the element of the resource block allocation matrix A, where i and j are subscripts.

[0060] S3. Update the OFDM resource block allocation method of the communication system.

[0061] (1) The present invention uses a reinforcement learning algorithm to update the matrix A. In combination with the application characteristics of the present system, the reinforcement learning state space, action space and reward designed by the present invention are as follows:

[0062] ① State space s: a set of parameters used to characterize the system state. Let the actual frame loss rate of K users at time t be O t = {O t,1 ,O t,2 ,…,O t,K}, the remaining transmission delay is T t ={T t,1 ,T t,2 ,…,T t,K}、The number of remaining sub-frames of the video frame is n t ={n t,1 ,n t,2 ,…,n t,K}, the channel state is h t =[h1,h2,…,h K ], where h k ={h t,1 ,h t,2 ,…,h t,L} represents the signal-to-noise ratio of the kth user on each OFDM resource block. Then the state space can be expressed as

[0063]

[0064] In the above formula, Indicates h t The transposed matrix of .

[0065] ② Action space: Matrix A is used to represent the allocation of OFDM resource blocks by the communication system. It is an L×K dimensional matrix.

[0066] ③ Reward: used to guide the learning process of the reinforcement learning network to achieve the goal of maximizing the number of users who meet QoS requirements. The reward function used in this invention is

[0067] r=∑ k R 1,k +R 2,k

[0068] In the above formula, R1,k The reward for the frame loss rate of user k is given only when the video frame is transmitted. When the user just meets the frame loss rate requirement, the reward is given, and when it does not meet the requirement, the reward is not given; if the actual frame loss rate is much lower than the frame loss rate requirement, it means that the user occupies too many communication resources, so a decaying reward is given. 2,k In order to represent the reward of user k's current transmission rate, a reward is given after each TTI. If the transmission speed given by the current system to the user can be transmitted within the delay requirement, a reward is given, otherwise a penalty is given.

[0069] For the first part of the reward function, the remaining transmission delay T of user k is t,k = 0, if the number of remaining subframes is n t,k ≤0, it is determined that the video frame has been transmitted. Then R 1,k The expression is

[0070]

[0071] In the above formula, r1 is the reward value, p is the average number of OFDM resource blocks occupied by the user per frame transmission, and q k It is used to limit the frame loss rate of user k to the required level. It is set by the algorithm user. m is the proportional coefficient and needs to be adjusted according to the training results.

[0072] For the second part of the reward function, a reward is calculated and given at each transmission time interval (TTI), R 2,k The expression is

[0073]

[0074] In the above formula, r2 is the reward value, v need,k is the minimum transmission rate that user k needs to allocate if there is no frame loss, v k is the transmission rate currently allocated to user k by the system.

[0075] (2) The reinforcement learning algorithm of the present invention is a deep Q network. For each OFDM resource block, a deep Q network model is established to output the user number assigned to the resource block, that is, a row of the matrix A. The model is as follows: Figure 2 As shown in Figure 2, the outputs of all deep Q networks are assembled into a matrix A, which is the way the communication system allocates resource blocks.

[0076] Figure 2 In DQN1~DQN L is a reinforcement learning network, each of which corresponds to a resource block RB in the communication system. l ={al,1 ,a l,2 ,…,a l,K}(l=0,1,2,…L) is the row vector of the resource block allocation matrix A, dimension K, and its elements satisfy

[0077]

[0078] {s t-1 ,A t ,r t ,s t} is a tuple that stores the system state at the last moment, the allocation of resource blocks, the reward after executing a TTI based on the allocation action, and the system state at the next moment. This tuple is called "experience" and is stored in the experience replay cache for use by the reinforcement learning network. After each TTI, the system extracts a portion of experience from the experience replay cache, forms a miniBatch, and sends it to the reinforcement learning network for the next step of learning.

[0079] S4. Update reinforcement learning network parameters.

[0080] (1) The action A output by the reinforcement learning network interacts with the environment and stores the result in the experience replay cache. A portion of the experience data is extracted from the experience replay cache and the network parameters of each deep Q network are updated using the gradient descent method.

[0081] (2) Update the state space parameters. For user k, use the following formula to update the state space parameters of each user:

[0082] Frame loss rate:

[0083] Remaining delay:

[0084] Number of remaining sub-frames in the video frame

[0085] In the above formula, n sent,k is the number of video subframes transmitted by the user in the current TTI. Channel state h t,k Updated based on current user feedback.

[0086] S5. Determine whether the optimization end condition is met: After the algorithm outputs the resource block allocation result, observe whether the target number of users that meet the QoS requirements of the communication system is reached. If not, return to S3 for the next iteration.

[0087] S6. The algorithm proposed in the present invention is simulated and compared with the traditional video transmission scheduling algorithm. The results are as follows: Figures 3 to 8 As shown,

[0088] from Figure 3 It can be seen that the algorithm proposed in the present invention can achieve better performance than the currently commonly used algorithms such as random selection algorithm (Random), polling algorithm (RandomRobin), maximum signal-to-interference ratio algorithm (Maximun C / I) and proportional fairness algorithm (Proportional Fair), so that more users in the communication system can meet QoS requirements.

[0089] from Figure 4 It can be seen that the algorithm proposed in the present invention can well maintain the actual QoS value of the user above its required value, and make full use of the QoS tolerance of the user to allow more users to meet the QoS requirements. At the same time, in the case of insufficient communication resources, the algorithm proposed in the present invention can abandon some users according to the system status to ensure the QoS requirements of most users in the system.

[0090] from Figure 5 It can be seen that the communication resource allocation strategy of the random selection algorithm is random allocation, so the QoS of users with good channel conditions is more likely to be met.

[0091] from Figure 6 It can be seen that in the polling algorithm, each user takes turns to occupy the system resources for the same time. This overly fair algorithm not only does not take into account the channel conditions of each user, but also does not take into account the transmission conditions of the current user. As a result, the transmission rate of users in the system is completely related to the channel conditions itself, and there are a large number of frame losses due to incomplete transmission. Ultimately, no user's QoS is met.

[0092] from Figure 7 It can be seen that the maximum carrier-to-interference ratio algorithm will allocate communication resources to users with the best channel status, but due to the influence of user mobility, the user's channel status changes over time. In this case, the maximum carrier-to-interference ratio algorithm cannot adapt to the user's channel changes, and it is difficult to continuously maintain the QoS of users in the system.

[0093] from Figure 8 It can be seen that the proportional fairness algorithm takes into account both system throughput and fairness, and the QoS of each user is maintained at around 60%. However, this will result in no user in the system having their QoS met, and all users must endure a poor communication environment, which does not meet the communication system's goal of "maximizing the number of users who meet QoS requirements."

[0094] In summary, the above results can prove that the proposed algorithm can indeed significantly increase the number of users meeting QoS requirements in the communication system.

[0095] The communication system parameters assumed in the simulation are shown in the following table.

[0096] Number of resource blocks 38 TTI duration 1ms Number of users 12 Transmitting power Pt 43dBm System bandwidth 30M Wireless frequency band 3400MHz User QoS requirements >＝0.9 Path loss exponent α 3.5 Maximum tolerable delay <=20ms Subcarrier spacing 60KHz Video Type 4K video, H.265 encoding Noise Power Spectral Density -174dBm

[0097] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A dynamic video transmission scheduling method based on reinforcement learning, characterized in that: The cross-layer joint scheduling optimization algorithm for multi-user dynamic video transmission includes the following steps: S1 obtains communication system parameters for algorithm training; S2 establishes the optimization problem function: let K be the number of users served by the system, L be the number of OFDM resource blocks contained in the system, then the optimization problem function is the user number assigned to each resource block l, so as to maximize the number of users who meet the QoS requirements; S3 updates the OFDM resource block allocation method of the communication system: establishes the state, action and target reward functions, uses the reinforcement learning network to output the user number assigned to each resource block l, and forms the OFDM resource block allocation method of the communication system; S4 updates the reinforcement learning network parameters: updates the reinforcement learning network parameters using the gradient descent method; S5 determines whether the optimization end condition is met: after the algorithm outputs the resource block allocation result, observe whether the number of users that meet the QoS requirements of the system is reached. If not, return to 3) for the next iteration; Among them, S2, establish the optimization problem function; let K be the number of users served by the system, L be the number of OFDM resource blocks contained in the system, then it is necessary to find a resource block allocation scheme to maximize the number of users who meet the QoS requirements; let A be the resource block allocation matrix with L rows and K columns, K Qos is the number of users that have met the QoS requirements, then the optimization function is max K QoS a ij ∈{0,1},a ij ∈A,i∈[0,L],j∈[0,K] In the above formula, a ij is the element of the resource block allocation matrix A, i and j are subscripts; Wherein, S3, updating the OFDM resource block allocation method of the communication system; (1) Using the reinforcement learning algorithm to update the matrix A; combined with the application characteristics of this system, the reinforcement learning state space, action space and reward designed by the present invention are as follows: ① State space s: a set of parameters used to characterize the system state; let the actual frame loss rate of K users at time t be O t = {O t，1 , O t，2 , ..., O t，K }, the remaining transmission delay is T t ={T t，1 , T t，2 , ..., T t，K }、The number of remaining sub-frames of the video frame is n t ={n t,1 , n t，2 , ..., n t，K }, the channel state is h t =[h1, h2, ..., h K ], where h k ={h t，1 ,h t，2 , ..., h t，L } represents the signal-to-noise ratio of the kth user on each OFDM resource block; then the state space can be expressed as In the above formula, Indicates h t The transposed matrix of ② Action space: Matrix A is used to represent the allocation of OFDM resource blocks by the communication system, which is an L×K dimensional matrix; ③ Reward: used to guide the learning process of the reinforcement learning network to achieve the goal of maximizing the number of users who meet QoS requirements; the reward function used in the present invention is r = ∑ k R 1，k +R 2，k In the above formula, R 1，k The reward for the frame loss rate of user k is given only when the video frame is transmitted. The reward is given when the user just meets the frame loss rate requirement, and no reward is given when it does not meet the requirement. If the actual frame loss rate is much lower than the frame loss rate requirement, it means that the user occupies too many communication resources, so a decaying reward is given. 2，k To represent the reward of user k’s current transmission rate, a reward is given after each TTI. If the current system can provide the user with a transmission speed that can be transmitted within the delay requirement, a reward is given; otherwise, a penalty is given. For the first part of the reward function, the remaining transmission delay T of user k is t，k = 0, if the number of remaining subframes is n t，k ≤0, it is determined that the video frame has been transmitted; at this time, Then R 1，k The expression is In the above formula, r1 is the reward value, p is the average number of OFDM resource blocks occupied by the user per frame transmission, and q k It is used to limit the frame loss rate of user k to the required value. It is set by the algorithm user. m is the proportional coefficient and needs to be adjusted according to the training results. For the second part of the reward function, a reward is calculated and given at each transmission time interval (TTI), R 2，k The expression is In the above formula, r2 is the reward value, v need，k is the minimum transmission rate that user k needs to allocate if there is no frame loss, v k is the transmission rate currently allocated by the system to user k; (2) The reinforcement learning algorithm is a deep Q network. For each OFDM resource block, a deep Q network model is established to output the user number assigned to the resource block, that is, a row of the matrix A. The outputs of all deep Q networks are assembled into the matrix A, which is the allocation method of the communication system for the resource block. DQN1~DQN L is a reinforcement learning network, each network corresponds to a resource block RB in the communication system; a l ={a l，1 , a l，2 , ..., a l，K }(l=0,1,2,...L) is the row vector of the resource block allocation matrix A, dimension K, whose elements satisfy {S t-1 , A t , r t ,s t } is a tuple, which is used to store the system state at the previous moment, the allocation of resource blocks, the reward after executing a TTI according to the allocation action, and the system state at the next moment; this tuple is called "experience" and is stored in the experience replay cache for use by the reinforcement learning network; after each TTI, the system will extract a part of the experience from the experience replay cache, form a miniBatch, and send it to the reinforcement learning network for the next step of learning; Among them, S4, update reinforcement learning network parameters; (1) The action A output by the reinforcement learning network interacts with the environment and stores the result in the experience replay cache; extracts a portion of the experience data from the experience replay cache and uses the gradient descent method to update the network parameters of each deep Q network; (2) Update the state space parameters: For user k, use the following formula to update the state space parameters of each user: Frame loss rate: Remaining delay: Number of remaining sub-frames in the video frame In the above formula, n sent，k is the number of video subframes transmitted by the user in the current TTI; channel state h t，k Updated based on current user feedback.

2. A dynamic video transmission scheduling method based on reinforcement learning as claimed in claim 1, characterized in that ； S1. Obtain communication system parameters for algorithm training, including the number of system users K, the number of system OFDM resource blocks L, the wireless frequency band, the transmission time interval TTI, and the frame loss rate requirement of each user k (k = 1, 2, ..., K) in the system Maximum delay tolerance The number of subframes contained in the transmitted video frame And the channel signal-to-noise ratio h of each user on each OFDM resource block k，l (k=1,2,...,K;l=1,2,...,L) and other parameters for use in reinforcement learning algorithm training.

3. A dynamic video transmission scheduling method based on reinforcement learning as described in claim 1, characterized in that ； S5. Determine whether the optimization end condition is met: After the algorithm outputs the resource block allocation result, observe whether the target number of users that meet the QoS requirements of the communication system is reached. If not, return to S3 for the next iteration.

Images