Method for mac layer packet delay prediction and scheduling optimization based on generative learning

Abstract

This invention discloses a generative learning-based method for MAC layer packet delay prediction and scheduling optimization, applicable to time-sensitive network systems. The method first constructs a wireless communication simulation environment to obtain an initial dataset. Then, it utilizes a generative learning framework to model delay behavior: a conditional variational autoencoder samples synthetic samples from the latent space, which are then merged with the original dataset to form a unified dataset. This dataset is trained to extract latent feature representations and characterize the main distribution of communication delay. Next, a generalized Pareto distribution is used to fit tail delays exceeding a threshold, describing tail characteristics in extreme scenarios, thereby obtaining a complete delay probability distribution. Finally, the prediction results are introduced into the scheduling framework as part of the state information. A reinforcement learning mechanism is used to dynamically adjust scheduling priorities and resource allocation strategies, achieving parameterized and intelligent scheduling control, thus effectively improving system performance.

Description

Technical Field

[0001] This invention relates to the field of wireless communication resource management technology, and in particular to a method for MAC layer packet delay prediction and scheduling optimization based on generative learning. Background Technology

[0002] Time-Sensitive Networking (TSN) is a deterministic communication technology for typical scenarios such as the Industrial Internet, Vehicle-to-Everything (V2X), and remote control. Its core objective is to achieve strict time synchronization, low latency, low jitter, and extremely high transmission reliability in complex network environments. To meet TSN's requirements for end-to-end latency control and extremely low packet loss rate, wireless communication systems must possess accurate latency awareness and efficient resource scheduling mechanisms. However, most existing wireless scheduling algorithms primarily aim to maximize system throughput or optimize average performance, typically relying on static or semi-static resource allocation based on user channel conditions or data arrival rates. This makes it difficult to capture the dynamic characteristics of latency over short timescales, especially in tail-end latency modeling, leading to significant deficiencies in extreme cases and making it difficult to avoid severe latency violations. Furthermore, at the MAC layer, packet scheduling and retransmission mechanisms have a decisive impact on latency fluctuations. A lack of accurate modeling and response to MAC layer queue dynamics and transmission characteristics will directly cause a mismatch between scheduling and actual link performance, thereby increasing the probability of latency violations. Therefore, how to effectively introduce a delay distribution prediction mechanism and feed the prediction results back to the real-time resource scheduling process to achieve delay-sensitive scheduling optimization for TSN requirements has become a key technical challenge facing current wireless systems. Summary of the Invention

[0003] This invention provides a generative learning-based method for MAC layer packet latency prediction and scheduling optimization, addressing the problems of existing wireless systems' inability to effectively predict tail latency and dynamically adjust resource scheduling strategies in TSN scenarios, thereby improving the system's low-latency transmission capability and latency guarantee level. This method focuses on latency prediction and optimization during MAC layer scheduling, particularly the impact of the remaining latency and service order of packets in the scheduling queue on scheduler behavior.

[0004] The technical solution adopted in this invention is:

[0005] A generative learning-based method for MAC layer packet delay prediction and scheduling optimization includes the following steps:

[0006] Step 1: In the established wireless communication simulation or actual system environment, collect communication data of users in the MAC layer scheduling queue, including data packet arrival time, data packet size, transmission deadline, resource share, and specific transmission delay, to comprehensively characterize the transmission delay characteristics of users; to obtain a representative and widely covered delay distribution prediction dataset, construct an initial wireless communication simulation environment, set the initial parameter configuration of the scheduler, and set an adjustable range for its key scheduling parameters; during the simulation, for users with different communication needs, dynamically adjust the scheduler parameters within a preset range at set time intervals to simulate system behavior under various scheduling conditions, and collect the corresponding data packet delay information to construct the initial delay distribution prediction dataset;

[0007] Step 2: Based on the initial time delay distribution prediction dataset obtained in Step 1, data augmentation is performed using a Conditional Variational Autoencoder (CVAE): Taking time delay and corresponding conditions as input, a latent representation is learned, and synthetic samples are generated from the latent space. The synthetic samples are merged with the initial dataset to form a synthetic overall dataset. On this overall dataset, the main distribution model and latent feature extraction are completed. High-delay samples exceeding a preset threshold are extracted as tail data and fitted using a Gaussian Digital Probability (GPD) model to obtain the tail delay distribution parameters. Subsequently, the main time delay distribution obtained from CVAE modeling and the tail delay parameters obtained from GPD are fused and concatenated to construct a complete time delay probability distribution model, thereby achieving time delay prediction based on the CVAE–GPD model.

[0008] Step 3: Construct a reinforcement learning model. The delay distribution feature parameters of each user output by the CVAE-GPD model and the environmental state information are combined to form the state space input model; the action space consists of the set of scheduling parameters. Composition, each user Has independent parameters and global parameters , and Shared by all users, it is used to collectively describe the overall scheduling strategy; the reward function comprehensively considers system throughput and packet loss rate to balance performance and reliability.

[0009] Step 4: Based on the reinforcement learning model in the time slot Output action A parameterized scheduler is modeled, which receives action parameters output by the reinforcement learning model and calculates user priority functions and resource allocation coefficients based on the parameters to determine the resource allocation and scheduling scheme within the current time slot. The execution result is used as feedback information to affect the subsequent training and update of the reinforcement learning model.

[0010] Furthermore, the collection of the initial time delay distribution prediction dataset in step 1 includes the following steps:

[0011] Step 1.1: Construct the initial wireless communication simulation environment, set the initial parameter configuration of the scheduler, and set an adjustable range for its key scheduling parameters to support dynamic changes in parameters during the scheduling process. The key scheduling parameters include weighting coefficients used for priority calculation. Proportional fairness factor Delay sensitivity parameter And resource allocation weight parameters, used to dynamically adjust the priority and resource allocation ratio among users during the scheduling process;

[0012] Step 1.2: In the initial wireless communication simulation environment, for users with different arrival rates, deadlines, packet sizes and resource shares, perform the resource scheduling process, and legally adjust the scheduler parameters within a preset range within a set time interval to simulate diverse scheduling behaviors.

[0013] Step 1.3: Collect data packet latency information generated under diverse scheduling behaviors, and summarize it to construct an initial latency distribution prediction dataset for training.

[0014] Furthermore, the modeling of the CVAE-GPD model in step 2 includes the following steps:

[0015] Step 2.1: Train CVAE by maximizing the variational evidence lower bound ELBO to learn the data packet transmission delay data of users in the MAC layer scheduling queue under different conditions collected in Step 1. On this basis, generate new samples to achieve overall data augmentation and complete the main distribution modeling. Improve the modeling accuracy by regularizing the latent space.

[0016] Step 2.2: For tail delay data exceeding the set threshold, GPD is used for modeling, and the scale and shape parameters of GPD are optimized using the weighted maximum likelihood estimation (MLE) method to achieve accurate fitting of tail distribution.

[0017] Step 2.3: Use neural network training to obtain the key parameters of the CVAE and GPD models. Among them, the key parameters of CVAE include the mean of the latent space. ,variance And network weight parameters, key parameters of GPD include scale parameters With shape parameters , used to describe the tail delay distribution;

[0018] Step 2.4: The main distribution function With tail distribution function The components are then combined to form a complete time delay probability density function.

[0019] Specifically, a threshold is set to divide the data into main and tail data. Tail samples exceeding the threshold are extracted and modeled using a Generalized Pareto Distribution (GPD). The main data distribution obtained from CVAE is then concatenated with the tail data distribution obtained from GPD to obtain a complete latency probability distribution. Combining the state characteristics of each user in the current environment, the CVAE–GPD model predicts the latency distribution parameters of each user in each time slot, and these parameters are used as one of the user's states as input to a PPO-based reinforcement learning model to generate resource allocation actions.

[0020] CVAE learns the latent structure of data by maximizing the variational evidence lower bound ELBO, and its optimization objective function is defined as follows:

[0021] Equation (1);

[0022] This represents the input delay sample. Indicates conditional features, Indicates the corresponding latent variable; The approximate posterior distribution established for the encoder, These are the network parameters of the encoder; The generation distribution established for the decoder, These are the network parameters for the decoder; For the conditional prior distribution of the potential space; Let represent the Kullback–Leibler divergence, used to constrain the consistency between the latent distribution and the prior distribution. By maximizing equation (1), it is possible to make the generated distribution... For input samples This maximizes the reconstruction accuracy, thereby enabling effective modeling of the potential structural features of time-delay data.

[0023] Based on this, by [doing something] in the potential space By sampling to generate synthetic samples, the overall communication latency data is augmented, thereby constructing a comprehensive dataset with higher distribution coverage. The main part of the latency distribution is then modeled and its features extracted. This generation mechanism not only improves the modeling accuracy of the main distribution but also alleviates the training insufficiency problem caused by sparse tail samples, providing data support for subsequent GPD tail modeling.

[0024] For the tail portion of the delay distribution, GPD is used to model the delay of data samples exceeding a set threshold and fit its shape parameters. Scale parameters To improve the modeling and prediction capabilities for extreme time delays, its probability density function is:

[0025] Equation (2);

[0026] To further optimize the model parameters, weighted maximum likelihood estimation (MLE) is used to solve for the tail parameters. The objective function is:

[0027] Equation (3);

[0028] In the formula, The scale parameter representing GPD. Indicates shape parameters; For the first A number exceeding the set threshold The over-threshold delay sample is defined as This is used to describe the degree of deviation of the tail data; The weighting coefficients corresponding to the sample are set according to the frequency of the sample or its confidence level, and are used to balance the contributions of different tail samples in the parameter solution process. This represents the number of tail data. By maximizing the objective function in equation (3), the optimal parameters for the tail distribution can be obtained, thereby achieving accurate modeling of extreme delay events.

[0029] Ultimately, the complete time delay probability density function is derived from the main distribution. Tail and tail distribution The concatenation structure is as follows:

[0030] Equation (4);

[0031] in, This represents the probability distribution function of the main body at the threshold, used to ensure continuity at the splicing point.

[0032] After obtaining the prediction results of the delay distribution in step 2, the scheduling optimization stage begins. Because the scheduling environment of wireless communication systems is highly time-varying and involves multiple users, relying solely on fixed rules is insufficient to achieve globally optimal resource allocation. Therefore, a reinforcement learning model is needed to dynamically optimize the strategy.

[0033] Furthermore, the scheduling parameter set mentioned in step 3 The policy gradient-based reinforcement learning algorithm PPO is used for online or offline updates, and the resource allocation strategy is adjusted periodically to achieve adaptive optimization of the scheduling strategy. The state, action, and reward settings for reinforcement learning are as follows:

[0034] Step 3.1, State Space Setup: Following the framework of deep reinforcement learning, the set of all users is considered as a single agent, and the agent's state is defined. The agent's state includes the head-of-queue delay for each user, queue length, signal-to-interference-plus-noise ratio (SINNR), and the delay distribution parameters output by the CVAE-GPD predictor for each time slot: mean. ,variance Scale parameters and shape parameters ;

[0035] Step 3.2, Action Space Settings: All parameters involved in the execution of the parameterized scheduler, including... ;

[0036] Step 3.3, Reward Function Setting: Maximize total throughput while minimizing packet loss rate. The calculation method is as follows:

[0037] Equation (5);

[0038] in, and This is the proportionality coefficient. This represents the total throughput of the current time slot. This indicates the number of data packets dropped due to timeout.

[0039] Furthermore, the parameterized scheduler modeling in step 4 includes the following steps:

[0040] Step 4.1: Set the scheduling priority index for each user. The calculation method is as follows:

[0041] Equation (6);

[0042] in, For parameterized variables, Total number of users For users Remaining delay at the head of the queue, The latency violation rate threshold, For users The deadline;

[0043] Step 4.2: Based on the priority indicators in Step 4.1 Filter out specific scheduling users and calculate user... Resource allocation coefficient The calculation method is as follows:

[0044] Equation (7);

[0045] in, For parameterized variables, For users Instantaneous rate, For users Moving average rate;

[0046] Step 4.3: Calculate the coefficients. Normalize and allocate resources;

[0047] Step 4.4: Use reinforcement learning to train the scheduling parameters to maximize the reward function and continuously update and optimize them.

[0048] This invention introduces a generative learning-based CVAE–GPD delay modeling method and a parameterized scheduling mechanism to achieve accurate modeling and dynamic optimization of data packet delay in wireless communication systems. CVAE is used to learn the main features of the delay distribution and generate synthetic samples to enhance data diversity, while GPD is used to fit tail-end extreme delays to capture high-risk events, thus obtaining a complete delay probability distribution. This prediction result is then input as one of the states into the parameterized scheduler, and the scheduling parameters are adaptively updated through a reinforcement learning algorithm. This technical solution effectively solves the problems of traditional scheduling methods failing to accurately characterize delay distributions and responding slowly to sudden tail delays. It possesses predictive and policy-adaptive capabilities, can identify and intervene in high-risk delay events in advance, and significantly improves the stability, delay control capabilities, and service reliability of wireless communication systems in TSN scenarios. It is suitable for low-latency, high-precision industrial communication tasks. Attached Figure Description

[0049] Figure 1 The scheduling system structure block diagram provided by the present invention;

[0050] Figure 2 This is a schematic diagram of a scenario provided by the present invention;

[0051] Figure 3 The schematic diagram of CVAE-GPD delay prediction provided by this invention;

[0052] Figure 4 A schematic diagram of the deep reinforcement learning training model provided by this invention;

[0053] Figure 5 This is a schematic diagram illustrating the training reward for the scheduling model provided by the present invention.

[0054] Figure 6 This is a schematic diagram illustrating the packet loss rate of the scheduling model provided by the present invention. Detailed Implementation

[0055] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings.

[0056] The present invention provides a MAC layer packet delay prediction and scheduling optimization method based on generative learning, such as... Figure 1 The steps shown are as follows:

[0057] Step 1: In the constructed wireless communication simulation scenario, collect communication data from each user in the MAC layer scheduling queue, including the arrival time, size, transmission deadline, resource share, and corresponding actual transmission delay of data packets, to characterize the end-to-end delay characteristics of users. To obtain a representative and widely covered delay distribution prediction dataset, construct an initial wireless communication simulation environment, set the initial parameter configuration of the scheduler, and set adjustable ranges for its main scheduling control parameters. During the simulation operation, for users with different service needs, dynamically adjust the scheduler parameters within a preset interval according to the set time intervals to simulate diverse scheduling strategies and system operating states, thereby constructing the initial delay distribution prediction dataset.

[0058] Step 2, see Figure 3 The time delay prediction model employs a joint generative modeling framework consisting of Conditional Variational Autoencoder (CVAE) and Generalized Pareto Distribution (GPD). Based on the initial time delay distribution prediction dataset obtained in step 1, data augmentation is first performed using CVAE: using the time delays in the initial dataset... and its conditions As input, CVAE learns the mean and variance of latent variables through probabilistic encoding to generate latent representations. Subsequently, synthetic samples are obtained by sampling and decoding the latent space. These synthetic samples are merged with the original dataset to form the overall dataset for subsequent modeling, thereby achieving an expansion and diversification of the representation of the time delay distribution. During the training phase, CVAE jointly optimizes the latent distribution by maximizing the variational ELBO. With the generated distribution This study aims to learn the underlying generation mechanism of communication delay data. This generation process not only improves the fitting accuracy of the main distribution but also effectively mitigates training bias caused by the sparsity of tail samples. Its optimization objective can be expressed as:

[0059] Equation (1);

[0060] For the long tail portion, GPD is specifically designed to fit the tail data and is suitable for extreme value modeling. Due to the high requirement for delay determinism in TSN networks, modeling the tail delay is particularly important. Therefore, it is necessary to separate the tail data and model it separately, optimizing the parameters of GPD to improve its predictive ability for delay tail effects. The probability density function of GPD is as follows:

[0061] Equation (2);

[0062] in, These are the scale parameter, shape parameter, and threshold parameter, respectively. To further optimize the parameter estimation of the tail distribution, a weighted MLE method is used, with the objective function being:

[0063] Equation (3);

[0064] As weight, For exceeding the threshold The sample values. Ultimately, the complete probability density function is derived from the main distribution. Tail and tail distribution The composition is as follows:

[0065] Equation (4);

[0066] This indicates that the main body is distributed at the threshold. The probability distribution function at a given location. This method, combining CVAE and GPD, can not only effectively learn the latent characteristics of the data, but also characterize the latency characteristics of the long tail, enabling wireless communication systems to have stronger latency prediction capabilities in URLLC and other low-latency application scenarios.

[0067] Step 3: To improve the adaptability of the scheduling strategy, it can be achieved through methods such as... Figure 4 The example shown illustrates online or offline updates of scheduling parameter sets based on policy gradient classes in reinforcement learning. The algorithm adjusts resource allocation strategies based on timely feedback. It treats the entire user set as a single agent, whose state includes user queue head delay, queue length, signal-to-interference-plus-noise ratio (SINNR), and delay distribution parameters predicted by the CVAE-GPD model for each time slot based on the current environment. The action space consists of the parameters of the resource scheduler. The objective reward is to maximize total throughput while minimizing packet loss rate:

[0068] Equation (5);

[0069] in, and This is the proportionality coefficient. This represents the total throughput of the current time slot. This indicates the number of packets dropped due to timeout. The scheduling period can be set to a fixed interval or a sliding window mode, depending on system settings. The system supports rolling updates and dynamic optimization of policy parameters.

[0070] Step 4: Establish a parameterized scheduler, which is based on a reinforcement learning model in time slots. Output action By combining the queue status information of the MAC layer and the predicted latency distribution parameters, the data packet priority and resource allocation coefficient of each user are calculated to determine the resource allocation and scheduling scheme for the current time slot. After determining the scheduling parameters, the remaining time of each user can be used as a basis for scheduling. Delay violation rate threshold and user deadline Substitute the following scheduling priorities into the calculation to determine the priority. The priority function is defined as follows: Equation (6);

[0071] in, These are adjustable policy parameters. After normalizing the priority values, they are combined with scheduling parameters. Calculate the resource allocation ratio coefficient to satisfy:

[0072] Equation (7);

[0073] in, For users Instantaneous rate, For users The moving average rate is normalized and the available wireless resources are allocated proportionally to each user.

[0074] The method provided by this invention exhibits significant advantages in performance. For example... Figure 5 As shown, this method can continuously improve the reward level while ensuring overall stability during training, and it is always superior to the latency-aware Deep Q-Network (DQN) method and the traditional Earliest Deadline First (EDF) method. Figure 6 Furthermore, this method demonstrates better convergence characteristics and robustness in delay violation rate control, achieving improved system rewards while satisfying constraints. In contrast, EDF exhibits limited overall performance, and while delay-aware DQN improves rewards to some extent, it still lags slightly behind in stability and constraint satisfaction. In summary, the proposed method achieves a more balanced and efficient performance between reward optimization and constraint control.

[0075] The above embodiments are used to illustrate the technical implementation process of the present invention and should not be used to limit the scope of protection of the present invention. All equivalent modifications or substitutions made based on the technical concept of the present invention shall fall within the scope of protection of the present invention.

Claims

1. A method for MAC layer packet delay prediction and scheduling optimization based on generative learning, characterized in that, Includes the following steps: Step 1: In the set wireless communication simulation or actual system environment, collect the communication data of users in the MAC layer scheduling queue under different conditions, including data packet arrival time, data packet size, transmission deadline, resource share and specific transmission delay, to form an initial delay distribution prediction dataset, which is used to comprehensively characterize the transmission delay characteristics of users. Step 2: On the initial latency distribution prediction dataset obtained in Step 1, data augmentation is performed using CVAE: taking latency and its corresponding conditions as input, where conditions include packet arrival rate, deadline, packet size, and resource share; CVAE learns latent representations and generates synthetic samples from the latent space, merging these synthetic samples with the initial latency distribution prediction dataset into a unified dataset, and performing latent feature extraction and main distribution modeling on this unified dataset; high latency data exceeding a set threshold are extracted as tail data, and GPD is used for fitting to obtain tail latency parameters; the main distribution modeling obtained from CVAE and the tail latency parameters obtained from GPD are concatenated and fused to form a complete latency probability distribution, realizing latency prediction based on the CVAE–GPD model; Step 3: Construct a reinforcement learning model, combining the delay distribution feature parameters of each user output by the CVAE-GPD model with the environmental state information to form a state space input model; The action space consists of a set of scheduling parameters. Composition, each user Has independent parameters and global parameters , and Shared by all users, it is used to collectively describe the overall scheduling strategy; the reward function comprehensively considers system throughput and packet loss rate to balance performance and reliability. Step 4: Based on the reinforcement learning model in the time slot Output action A parameterized scheduler is modeled, which receives action parameters output by the reinforcement learning model and calculates user priority functions and resource allocation coefficients based on the parameters to determine the resource allocation and scheduling scheme in the current time slot. The execution result is used as feedback information to affect the subsequent training and update of the reinforcement learning model. Step 2, modeling the CVAE-GPD model, includes the following steps: Step 2.1: Train CVAE by maximizing the variational evidence lower bound ELBO to learn the data packet transmission delay data of users in the MAC layer scheduling queue under different conditions collected in Step 1. On this basis, generate new samples to achieve overall data augmentation and complete the main distribution modeling. Improve the modeling accuracy by regularizing the latent space. Step 2.2: For tail delay data exceeding the set threshold, GPD is used for modeling, and the scale and shape parameters of GPD are optimized using the weighted maximum likelihood estimation (MLE) method to achieve accurate fitting of tail distribution. Step 2.3: Use neural network training to obtain the key parameters of the CVAE and GPD models. Among them, the key parameters of CVAE include the mean of the latent space. ,variance And network weight parameters, key parameters of GPD include scale parameters With shape parameters , used to describe the tail delay distribution; Step 2.4: The main distribution function With tail distribution function The components are then combined to form a complete time delay probability density function.

2. The MAC layer packet delay prediction and scheduling optimization method based on generative learning according to claim 1, characterized in that, The collection of the initial time delay distribution prediction dataset in step 1 includes the following steps: Step 1.1: Construct the initial wireless communication simulation environment, set the initial parameter configuration of the scheduler, and set an adjustable range for its key scheduling parameters to support dynamic changes in parameters during the scheduling process. The key scheduling parameters include weighting coefficients used for priority calculation. Proportional fairness factor Delay sensitivity parameter And resource allocation weight parameters, used to dynamically adjust the priority and resource allocation ratio among users during the scheduling process; Step 1.2: In the initial wireless communication simulation environment, for users with different arrival rates, deadlines, packet sizes and resource shares, perform the resource scheduling process, and legally adjust the scheduler parameters within a preset range within a set time interval to simulate diverse scheduling behaviors. Step 1.3: Collect data packet latency information generated under diverse scheduling behaviors, and summarize it to construct an initial latency distribution prediction dataset for training.

3. The MAC layer packet delay prediction and scheduling optimization method based on generative learning according to claim 1, characterized in that, The scheduling parameter set described in step 3 The policy gradient-based reinforcement learning algorithm PPO is used for online or offline updates, and the resource allocation strategy is adjusted periodically to achieve adaptive optimization of the scheduling strategy. The state, action, and reward settings for reinforcement learning are as follows: Step 3.1, State Space Setup: Following the framework of deep reinforcement learning, the set of all users is considered as a single agent, and the agent's state is defined. The agent's state includes the head-of-queue delay for each user, queue length, signal-to-interference-plus-noise ratio (SINNR), and the delay distribution parameters output by the CVAE-GPD predictor for each time slot: mean. ,variance Scale parameters and shape parameters ; Step 3.2, Action Space Settings: All parameters involved in the execution of the parameterized scheduler, including... ; Step 3.3, Reward Function Setting: Maximize total throughput while minimizing packet loss rate. The calculation method is as follows: Equation (1) in, and This is the proportionality coefficient. This represents the total throughput of the current time slot. This indicates the number of data packets dropped due to timeout.

4. The MAC layer packet delay prediction and scheduling optimization method based on generative learning according to claim 1, characterized in that, Step 4, the parameterized scheduler modeling, includes the following steps: Step 4.1: Set the scheduling priority index for each user. The calculation method is as follows: Equation (2); in, For parameterized variables, Total number of users For users Remaining delay at the head of the queue, The latency violation rate threshold, For users The deadline; Step 4.2: Based on the priority indicators in Step 4.1 Filter out specific scheduling users and calculate user... Resource allocation coefficient The calculation method is as follows: Equation (3); in, For parameterized variables, For users Instantaneous rate, For users Moving average rate; Step 4.3: Calculate the coefficients. Normalize and allocate resources; Step 4.4: Use reinforcement learning to train the scheduling parameters to maximize the reward function and continuously update and optimize them.

Images