Deep Learning-Based Network Traffic Scheduling Optimization Method

Abstract

This invention provides a deep learning-based network traffic scheduling optimization method. Applied to the field of communication network technology, the method includes: S1, real-time collection of traffic data, link status, latency, and packet loss rate indicators deployed on network nodes and construction of a sample database; S2, based on the sample database, construction of a traffic prediction model fusing a retrieval-enhanced diffusion model and a hybrid linear expert model for multimodal traffic prediction; S3, construction of the state space of a reinforcement learning agent based on real-time network status and the predicted future traffic; S4, policy optimization based on a multi-objective optimization mechanism; S5, deployment of the trained and optimized model to a network controller or edge computing node to achieve real-time perception and dynamic scheduling control of network resources. This invention significantly improves traffic prediction accuracy and updates and optimizes scheduling strategies in a timely manner according to changes in the network environment, thereby improving overall network performance.

Description

Technical Field

[0001] This invention relates to the field of communication network technology, and in particular to a network traffic scheduling optimization method based on deep learning. Background Technology

[0002] As network scale and complexity continue to grow, traditional traffic scheduling methods are increasingly unable to meet the demands for dynamic, efficient, and intelligent management. Especially in network environments with multiple concurrent service types (such as voice, video, and big data synchronization), achieving efficient and refined scheduling based on real-time traffic conditions, network topology, and service requirements has become a key challenge in network optimization.

[0003] In recent years, deep learning has made significant progress in areas such as temporal modeling and decision optimization, demonstrating its enormous potential in network scheduling. However, existing deep learning-based traffic prediction methods generally suffer from weak generalization ability and slow response to sudden events, failing to fully adapt to the complexities of real-world networks. Furthermore, scheduling strategies often rely on statically trained models, making it difficult to update and optimize them in a timely manner according to changes in the network environment. In addition, in multi-objective optimization scenarios, there are conflicts between objectives such as latency, bandwidth utilization, and energy consumption, necessitating the design of a strategy to coordinate these objectives and improve overall performance.

[0004] Therefore, there is an urgent need for a deep learning method with intelligent perception, accurate prediction, and adaptive scheduling capabilities to address traffic scheduling optimization problems in complex network environments. Summary of the Invention

[0005] To address the aforementioned problems, this invention proposes a deep learning-based network traffic scheduling optimization method to solve technical issues such as low prediction accuracy, rigid strategies, and resource waste in traditional network traffic scheduling.

[0006] To achieve the above objectives, the following technical solution is adopted:

[0007] This invention provides a deep learning-based network traffic scheduling optimization method, comprising the following steps:

[0008] S1. Collect traffic data, link status, latency and packet loss rate metrics deployed on network nodes in real time, and build a sample database;

[0009] S2. Based on the sample database, construct a traffic prediction model that integrates a retrieval-enhanced diffusion model and a hybrid linear expert model to perform multimodal traffic prediction, and output the future traffic prediction results through weighted fusion.

[0010] S3. Construct the state space of the reinforcement learning agent based on the real-time network status and the future traffic prediction results, train the model using a multi-round online reinforcement learning mechanism, optimize the scheduling strategy in stages, guide the strategy improvement through reward bonus items, and dynamically adjust the action space according to the state space.

[0011] S4. Based on the multi-objective optimization mechanism, the strategy is optimized. Combining the weighted linear normalization method and the Pareto front approximation algorithm, the Pareto front solution set that satisfies the multi-objective trade-off is selected from the candidate solutions formed by the action space for actual deployment.

[0012] S5. Deploy the trained and optimized model to the network controller or edge computing node to achieve real-time perception and dynamic scheduling control of network resources, and update the parameters of the traffic prediction model online based on the scheduling effect data.

[0013] Furthermore, the construction of the retrieval enhancement diffusion model includes:

[0014] The sample time series in the sample database are truncated into fixed-length segments according to a unified time window to form a candidate sample set D;

[0015] The pre-trained encoder maps the current input sample time series to the first embedding vector.

[0016] For each sample in the sample database, the first π time steps are extracted, and a second embedding vector is generated using the same encoder;

[0017] Calculate the similarity between the first embedding vector and all the second embedding vectors, and select the k sample indices with the smallest similarity to form a retrieval sample set;

[0018] A diffusion probability model framework is constructed, and the retrieved sample set is encoded and guided to denoise through a mechanism that destroys the data distribution by forward diffusion and reconstructs the target sequence by reverse diffusion, thereby generating denoised prediction results.

[0019] Furthermore, the construction based on a diffusion probability model framework, which encodes and guides denoising of the retrieved sample set through a mechanism of forward diffusion disrupting data distribution and reverse diffusion reconstructing the target sequence, to generate denoised prediction results, includes:

[0020] In the forward diffusion phase: starting from the real flow sequence x0, Gaussian noise is gradually added over time step T to generate a noisy sequence x1, x2, ..., x T initial noise vector , It approximately follows a standard normal distribution N(0,I) and its dimension is consistent with the prediction target;

[0021] In the reverse diffusion prediction stage:

[0022] By using a pre-trained time-series encoder, each reference sample in the retrieval sample set is mapped to an embedding vector, and its deep temporal features are extracted to obtain the retrieval sample encoding features. ;

[0023] The diffusion state characteristics at the current diffusion time step t Features after encoding the input time series The retrieval sample encoding features The current diffusion time step t is input to the reference modulation attention module (RMA), which calculates the dynamic weights between features through a multi-head attention mechanism, and dynamically adjusts the contribution ratio of the reference sample to the denoising process.

[0024] Denoising is performed at each step of the reverse diffusion from t=T to t=1, gradually generating denoised predictions.

[0025] After T-step denoising, the first flow prediction result is obtained. .

[0026] Furthermore, the structure of the hybrid linear expert model includes: a multi-expert parallel prediction layer, a dynamic weight router, and a weighted fusion and post-processing layer;

[0027] The multi-expert parallel prediction layer contains n independent linear expert models, each of which is a linear regression model with different structures or parameters, and is used to capture the long-term trend, periodic or sudden characteristics of the time series respectively.

[0028] The dynamic weight router includes an encoder for timestamp embedding encoding of the input time series and a hybrid layer consisting of two fully connected networks; the hybrid layer is used to map the timestamp embedding encoding features into a channel-specific weight matrix.

[0029] The weighted fusion and post-processing layer includes a channel-weighted fusion unit and a post-processing layer. The channel-weighted fusion unit is used to calculate the weight of each expert prediction result for each channel according to the channel-specific weight matrix to obtain the fusion result. The post-processing layer is used to perform nonlinear correction on the fusion result and output the final prediction result.

[0030] Furthermore, the construction of the hybrid linear expert model includes:

[0031] Set up n linear expert models Each expert For the input sample time series Perform independent predictions and output the prediction results for a future time window of length p. Where c represents the number of channels for traffic monitoring;

[0032] Time stamp embedding features based on input sequence , m represents the dimension of the timestamp embedding feature; a hybrid layer consisting of two MLP layers. Perform dynamic weighted routing and generate channel-specific weight matrices. The calculation formula is: The prediction results of each expert model The summation is performed based on the specific weights corresponding to each channel, and then processed by the post-processing layer. Adjust the output trend and output the final second prediction result. .

[0033] Furthermore, the post-processing layer Adjust the output trend and output the final second prediction result. ,include:

[0034] ;in, Defined as a channel-specific multiplication operation; The outputs of each expert model are weighted and combined to form a comprehensive prediction representation; The post-processing layer includes residual connections and layer normalization operations to compensate for prediction errors in weighted fusion, further fit errors, and adjust trends. The final second prediction result;

[0035] The timestamp embedding feature The encoding method is as follows: normalize the day of the week, hour, and holiday markers respectively, and then concatenate them into a vector.

[0036] Furthermore, the weighted fusion formula for the traffic prediction model is:

[0037] Where Z is the predicted future traffic; α and β are dynamically adjusted fusion coefficients, and satisfy α+β=1.

[0038] Furthermore, the phased optimization scheduling strategy includes:

[0039] Phase 1: Decoupling Attempt Initialization

[0040] Policy generation; Generate the action policy for the first scheduling attempt based on the initial state, including route selection and bandwidth allocation; Improve the action policy for the second scheduling attempt based on the performance metrics fed back after the first scheduling attempt;

[0041] Optimization objective: Maximize the reward of the second scheduling attempt, while constraining the policy distribution of the first scheduling attempt through KL divergence regularization to prevent the policy from deviating from the reasonable distribution of the base model;

[0042] Phase Two: Joint Optimization of Reward Shaping

[0043] Based on the optimization results of the first stage, the second stage of joint optimization is carried out. Through the reward shaping mechanism, the optimization process drives the iterative optimization of the policy by the feedback difference between the two scheduling attempts, encouraging the model to effectively improve the first policy in the second scheduling attempt.

[0044] Furthermore, S4 involves strategy optimization based on a multi-objective optimization mechanism, combining a weighted linear normalization method and a Pareto front approximation algorithm to select a set of Pareto front solutions that satisfy the multi-objective trade-offs from the candidate solutions for actual deployment, including:

[0045] During the training phase, the weight coefficients of each optimization objective are dynamically allocated according to the current network scenario requirements, and a joint objective function is constructed. :

[0046] ;in, Let be the weight coefficient of the i-th indicator; The normalized score for the q-th indicator;

[0047] A Pareto optimal solution set is generated through a multi-objective optimization algorithm to provide an unbiased scheduling reference set during the deployment phase, ensuring that each scheduling strategy in the solution set cannot be further optimized on a certain objective without harming other objectives;

[0048] Based on the real-time network status, the optimal strategy is selected from the Pareto solution set. The priority rules include: in high-load scenarios, low-latency solutions are selected first; in energy-saving mode, high-efficiency solutions are selected first; and in balanced mode, the solution with the highest comprehensive score is selected.

[0049] Furthermore, the online update mechanism in step S5 includes:

[0050] (1) Store historical state-action-reward samples in the experience replay pool and periodically sample and fine-tune the model parameters;

[0051] (2) The PPO algorithm is used for policy gradient update, combined with the target network stable training process;

[0052] (3) Deploy multiple model redundancies and gradually update online model parameters through soft replacement.

[0053] Compared with the prior art, the present invention achieves the following beneficial effects:

[0054] 1. This invention constructs a retrieval-enhanced diffusion model, combining a reference sample retrieval mechanism with the generation capability of the diffusion model. In the prediction stage, a reference-guided reverse diffusion process is introduced, which significantly enhances the model's stability and generalization ability in predicting future traffic trends, and improves prediction accuracy. It is especially suitable for network traffic scenarios with strong sudden changes.

[0055] 2. This invention constructs a hybrid linear expert model: a lightweight and scalable multi-expert system is designed, which uses multiple parallel linear experts to capture long-term trend features and improves the fitting effect on periodic and trend flows through dynamic weight fusion, effectively supplementing the shortcomings of nonlinear models in long-term modeling.

[0056] 3. This invention addresses the issues of high state-action space dimensionality and feedback delay in the scheduling policy formulation process by constructing a multi-round online reinforcement learning method. It introduces a multi-round online reinforcement learning mechanism to dynamically adjust the policy based on prediction results, achieving continuous optimization and rapid adaptation. Simultaneously, it incorporates self-reinforcement based on historical scheduling feedback, improving policy stability and long-term returns.

[0057] 4. This invention constructs a reward bonus design mechanism, which acts as an auxiliary signal in the optimization process of reinforcement learning strategy. By modeling the relationship between intermediate states (such as sub-policies or transitional flow states) and the optimal goal, it guides the learning process to focus on key decision points, effectively alleviates the sparse reward problem, and improves the convergence speed and performance of the strategy.

[0058] It should be understood that the description in the Summary of the Invention is not intended to limit the key or essential features of the embodiments of the present invention, nor is it intended to restrict the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0059] The above and other features, advantages, and aspects of the various embodiments of the present invention will become more apparent from the accompanying drawings and the following detailed description. The drawings are provided for a better understanding of the invention and are not intended to limit the invention. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0060] Figure 1 A flowchart illustrating a deep learning-based network traffic scheduling optimization method according to an embodiment of the present invention is shown.

[0061] Figure 2 The diagram illustrates the architecture of a deep learning-based network traffic scheduling optimization method according to an embodiment of the present invention. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0064] Figure 1 A flowchart illustrating a deep learning-based network traffic scheduling optimization method according to an embodiment of the present invention is shown. Figure 2 This diagram illustrates the architecture of a deep learning-based network traffic scheduling optimization method according to an embodiment of the present invention. Figure 1 and Figure 2 As shown, a network traffic scheduling optimization method 100 based on deep learning includes the following steps:

[0065] S1. Collect traffic data, link status, latency and packet loss rate metrics deployed on network nodes in real time, and build a sample database;

[0066] In step S1, firstly, key indicators such as traffic data, link status, latency, and packet loss rate are collected in real time through monitoring modules deployed at various network nodes. Then, two database construction strategies are proposed for time-series datasets with different characteristics. For datasets that are insufficient in size and difficult to label with a single class, the entire training set is defined as the database; for datasets containing complete class labels but exhibiting class imbalance, a subset of samples containing all classes is selected as the database.

[0067] S2. Based on the sample database, construct a traffic prediction model that integrates a retrieval-enhanced diffusion model and a hybrid linear expert model to perform multimodal traffic prediction, and output the future traffic prediction results through weighted fusion.

[0068] In step S2, a traffic prediction model integrating two different modeling paradigms is constructed: a retrieval-enhanced diffusion model and a hybrid linear expert model, which are used to enhance the model's stability and multi-modal modeling capabilities, respectively. Furthermore, the fusion of the hybrid linear expert model (lightweight plug-in architecture) and the retrieval-enhanced diffusion model (pre-trained encoder) reduces computational overhead and adapts to the resource limitations of edge devices; by weighted fusion of the prediction results from both models, the overall prediction performance and robustness are effectively improved.

[0069] Step S2 specifically includes the following steps:

[0070] Step S2.1: Construct a retrieval-enhanced diffusion model

[0071] To address the issues of large performance fluctuations and weak generalization ability faced by traditional time series forecasting models in traffic prediction tasks, this invention introduces a retrieval-enhanced diffusion model for modeling and predicting future traffic. Based on the diffusion probability modeling framework, the retrieval-enhanced diffusion model incorporates a reference sample auxiliary mechanism. By embedding two main modules—retrieval and reference-guided diffusion—it achieves enhanced modeling of the temporal characteristics of traffic / network traffic, effectively improving prediction stability and accuracy.

[0072] Furthermore, the construction of the retrieval enhancement diffusion model includes: truncating the sample time series in the sample database into fixed-length segments according to a unified time window to form a candidate sample set D; mapping the currently input sample time series into a first embedding vector through a pre-trained encoder; for each sample in the sample database, truncating its first π time steps and generating a second embedding vector through the same encoder; calculating the similarity between the first embedding vector and all the second embedding vectors, selecting the k sample indices with the lowest similarity to form a retrieval sample set; constructing a diffusion probability model framework, encoding and guiding denoising of the retrieval sample set through a mechanism of forward diffusion disrupting data distribution and reverse diffusion reconstructing the target sequence, and generating denoised prediction results.

[0073] Step S2.1 specifically includes the following sub-steps:

[0074] Step S2.1.1: Embedding the retrieval mechanism

[0075] The retrieval-enhanced diffusion model first uses a sample database containing a large number of historical traffic time series data in step S1. Each time series segment is truncated into a fixed-length sequence according to a uniform time window, serving as the candidate set for subsequent retrieval. Then, the input historical traffic time series X (e.g., traffic over the past 24 hours or 7 days) is encoded, specifically encoding each time component within the timestamp. Taking the encoding of the day of the week as an example, the encoder extracts the embedding vector of the current input sequence and matches it with sequences in the database.

[0076]

[0077] ;in ;in, The input historical traffic time series represents the observed network traffic data of c channels (e.g., multiple network interfaces, sensor channels) within the past s time steps (e.g., the number of observation points in the past 24 hours, 7 days, etc.), which serves as the primary modeling object. or The input historical traffic time series X is embedded after being encoded (Embedding function) to extract its temporal semantic information. : The i-th historical sample sequence in the database, where D represents the set of all stored samples used as a retrieval reference. The first π time steps of the i-th sample are used to calculate the embedding (i.e., only the initial part of the sample is taken as the reference basis). or : A pre-trained time series encoder function used to map database sample sequences to an embedding space representation. This facilitates similarity comparison. Sim(a,b) represents the similarity metric function between two embedding vectors, commonly used functions include negative Euclidean distance, the inverse of cosine similarity, etc. (the smaller the value, the more similar the vectors). : This represents the set of k reference time series samples that are most similar to the currently input historical time series X, retrieved from database D. : This indicates that the sample index with the lowest similarity (i.e., closest) is selected to select the l-th reference sample from the database.

[0078] In step S2.1.1 above, a pre-trained encoder is used to embed and map the traffic sequence within the current time window to obtain a low-dimensional feature vector. ; Perform a distance metric (such as Euclidean distance) between this vector and the embedding representation of each candidate sample in the database, and retrieve the k most similar reference samples from it. This is used for subsequent diffusion guidance.

[0079] Step S2.1.2: Construct a framework based on the diffusion probability model

[0080] Furthermore, the construction based on a diffusion probability model framework, which encodes and guides denoising of the retrieved sample set through a mechanism of forward diffusion disrupting data distribution and reverse diffusion reconstructing the target sequence, to generate denoised prediction results, includes:

[0081] In the forward diffusion phase: starting from the true flow sequence x0, Gaussian noise is gradually added over T diffusion time steps to generate a noisy sequence x1, x2, ..., x T initial noise vector , It approximately follows a standard normal distribution N(0,I) and its dimension is consistent with the prediction target;

[0082] To prevent the model from over-relying on the reference sample, the retrieval-enhanced diffusion model also introduces an adjustment parameter in the reference guidance. The weights of the reference information are constrained to maintain the robustness and generalization of the prediction. Assume there are T diffusion time steps, and at each step, the model generates a prediction:

[0083]

[0084] : The initial state of the diffusion process, also known as the "noise vector" or "reverse denoising starting point". It is the input after T diffusion time steps (i.e., the state furthest from the target prediction), and is usually a random variable with the same dimension as the target prediction sequence. T: The maximum number of time steps in the diffusion process, representing the number of denoising stages required to go from the most chaotic state to a clear prediction. :express It is randomly sampled from a multidimensional standard normal distribution with a mean of 0 and a covariance of identity matrix I. This distribution defines the starting point of the model in the informationless state.

[0085] In the reverse diffusion prediction stage:

[0086] By using a pre-trained time-series encoder, each reference sample in the retrieval sample set is mapped to an embedding vector, and its deep temporal features are extracted to obtain the retrieval sample encoding features. ;

[0087] The retrieval-enhanced diffusion model employs a diffusion model framework in the prediction phase. In the forward diffusion phase, it operates similarly to the traditional diffusion model, gradually adding noise to disrupt the target predicted flow sequence and construct a noise distribution. During the backward diffusion process, the retrieval-enhanced diffusion model utilizes the reference samples obtained from the previous retrieval and guides the denoising process through a Reference Modulation Attention (RMA) module. Specifically, the RMA module fuses features of the current prediction state, features of the reference samples, and lateral information, dynamically adjusting the contribution weights of each feature through an adaptive modulation attention mechanism.

[0088] The diffusion state characteristics at the current diffusion time step t Features after encoding the input time series The retrieval sample encoding features The current diffusion time step t is input to the reference modulation attention module (RMA). A multi-head attention mechanism is used to calculate the dynamic weights between features, dynamically adjusting the contribution ratio of the reference sample to the denoising process. Denoising is performed at each step of the reverse diffusion from t=T to t=1, progressively generating denoised predictions.

[0089]

[0090]

[0091]

[0092] The predicted mean of the backdiffusion denoising function is the denoising function that integrates the current state, input sequence features, and reference sample features. : The diffusion state at the current diffusion time step t, i.e., the noisy predicted flow sequence, which is an intermediate variable that is gradually denoised during backdiffusion. RMA: Reference Modulation Attention Module, an attention mechanism or variant structure that dynamically generates a denoised prediction result that fuses the context and external reference. t: The current diffusion time step, used to control the diffusion or denoising progress. The RMA module uses it to implement temporal modulation (such as temporal embedding). The k reference time series samples most similar to X, retrieved from the database through an embedded retrieval mechanism, are used to provide external "experience" guidance for prediction. : Represents the embedding representation obtained after encoding the input time series X (usually through a pre-trained time series encoder), used to extract sequence feature vectors. : This represents the set of embedded data after encoding the k retrieved reference samples. It is used as external knowledge input into the RMA module to participate in the denoising guidance. The covariance (or standard deviation) term corresponding to the current diffusion time step t is used to control the magnitude of the added noise. It is usually the output of a predefined time scheduling function, such as a linear, cosine, or learned noise scale table. The noise vector sampled from the standard normal distribution is the core random variable used in the diffusion model to generate diversity and simulate uncertainty.

[0093] After multiple rounds of backdiffusion iterations and T-step denoising, the diffusion model finally outputs a denoised future flow sequence. This refers to the prediction of traffic flow over a future period, i.e., the first traffic prediction result. : First flow prediction results 'p' represents the length of the future time window predicted by the model, and 'c' represents the number of future time steps predicted, indicating the time dimension. For example, if the input historical traffic is the past 24 hours (s=24), and the traffic for the next 6 hours is predicted, then p=6; if the traffic for the next 1 hour is predicted (sampled per minute), then p=60. Each time step corresponds to a fixed time interval (e.g., 1 minute, 1 hour, etc.), specifically determined by the data collection frequency. 'c' represents the number of traffic monitoring channels (e.g., multiple network interfaces, different sensors, etc.), representing different data sources, interfaces, or paths in the network that need to be independently monitored or scheduled, indicating the spatial dimension. For example: multiple network interfaces: such as multiple network interface cards (NICs) on a server, each NIC corresponding to one channel; sensor nodes: different monitoring points distributed in the network (e.g., router, switch ports); virtual channels: logically divided independent data transmission paths (e.g., VPN tunnels, QoS priority queues). Traffic data for each channel is collected independently, and the model needs to predict the future state of all channels simultaneously. For example... This indicates that the model predicts the traffic data for the three channels over the next 24 time steps (e.g., 24 hours).

[0094] Based on the retrieval enhancement diffusion model constructed in step S2.1 above, historical similar sample retrieval is performed ( The diffusion model, along with the Reference Modulation Attention (RMA) module, can significantly improve the prediction accuracy of burst traffic and abnormal fluctuations. The multi-step denoising mechanism of the diffusion model, combined with external reference guidance, reduces the dependence on the distribution of training data and enhances the model's generalization ability in unknown network scenarios.

[0095] Step S2.2: Construct a hybrid linear expert model

[0096] Hybrid linear expert models are a type of linearly centered multi-expert system. They are lightweight, scalable, and pluggable architectures suitable for long-term traffic trend modeling. They utilize multiple parallel linear expert models to learn time series features of different dimensions / periods and fuse them through a dynamic weight router to finally output predicted values.

[0097] Furthermore, the structure of the hybrid linear expert model includes: a multi-expert parallel prediction layer, a dynamic weight router, and a weighted fusion and post-processing layer;

[0098] Among them, the multi-expert parallel prediction layer contains n independent linear expert models, each of which is a linear regression model with different structure or parameters, and is used to capture the long-term trend, periodic or sudden characteristics of time series respectively.

[0099] The dynamic weighted router includes an encoder for timestamp embedding encoding of the input time series and a hybrid layer consisting of two fully connected network (MLP) layers; the hybrid layer is used to map the timestamp embedding encoding features into a channel-specific weight matrix.

[0100] The weighted fusion and post-processing layer includes a channel-weighted fusion unit and a post-processing layer. The channel-weighted fusion unit is used to calculate the weight of each expert prediction result for each channel according to the channel-specific weight matrix to obtain the fusion result. The post-processing layer is used to perform nonlinear correction on the fusion result and output the final prediction result.

[0101] Furthermore, the construction of the hybrid linear expert model includes:

[0102] Given the historical traffic time series as input The starting timestamp information is preprocessed, that is, each time component in the timestamps of the historical traffic time series X is encoded to obtain multi-dimensional timestamp embedding features. Where m is the dimension of the timestamp embedded features, then each expert model The output is:

[0103] ; X represents the input historical traffic time series; c represents the number of traffic monitoring channels (e.g., multiple network traffic, traffic at different intersections, etc.); s represents the historical time step, such as the past 24 hours or 7 days. The i'-th expert model is typically a linear model with different structures or parameters; n linear expert models are set. Each expert Perform independent predictions on the historical traffic time series X, and output the traffic prediction results for future time windows with lengths of p and c channels. .

[0104] Timestamp embedding features based on the input historical traffic time series X A hybrid layer composed of two MLP layers Perform dynamic weighted routing and generate channel-specific weight matrices. The calculation formula is:

[0105] ;in, A hybrid layer (also known as a router) consisting of two MLP layers is used to generate the weighting coefficients for each channel to each expert model. W is the channel-specific weight matrix; n is the number of expert models.

[0106] Specifically, the starting timestamp information of the input historical traffic time series X is encoded to obtain multi-dimensional timestamp embedding features. The encoding method is as follows: for example, the day of the week, hour, and holiday markers are normalized and then concatenated into a vector. Typically, it includes a combination of the following time components: day of the week (1-dimensional, normalized encoding); time period (e.g., hour, normalized to a scalar of 0 and 1); holiday indicator (1-dimensional, 0 / 1 indicating whether it is a holiday); season or month (optional, 1-dimensional, normalized encoding of the 12 months). Each time component is encoded and converted into numerical features, then concatenated into a vector. Taking the day of the week as an example, the formula is: Here, "index" represents the index of the day of the week, with Monday's index being 0 and Sunday's index being 6. Using this formula, Monday is encoded as -0.5, Tuesday as approximately -0.33, Wednesday as approximately -0.17, and so on, with Sunday encoded as 0.5. This encoding method transforms time information into numerical form, facilitating model learning and utilization of periodic features in time series data.

[0107] The prediction results of each expert model The data is weighted and summed according to the specific weights corresponding to each channel, and the output trend is adjusted by the post-processing layer P(⋅). The final second flow prediction result is then output after weighted fusion. :

[0108] ;in, Defined as a channel-specific multiplication operation; The outputs of each expert model are weighted and combined to form a comprehensive prediction representation; Post-processing layer: used to further fit the error, adjust the trend, and improve the prediction accuracy (e.g., residual connection, regularization, etc.).

[0109] This method can dynamically adjust the contribution of different experts to the final prediction, and has good interpretability and stability, making it particularly suitable for long-term sequence modeling tasks.

[0110] The hybrid linear expert model constructed in step S2.2, through a process of "first retrieval, then guidance, and finally prediction," combines historical experience knowledge with current contextual features, demonstrating stronger time-series modeling capabilities and prediction stability in traffic sequence prediction tasks. Multiple experts work in parallel to capture long-term trends, periodicity, and suddenness features, while a weighted fusion mechanism (dynamic router) adapts to different time scenarios (such as weekdays / holidays), improving long-term prediction stability. The post-processing layer compensates for prediction errors through residual connections, further optimizing output quality.

[0111] Step S2.3: Weighted fusion of prediction results

[0112] To balance the predictive advantages of both models, a simple yet effective weighted fusion mechanism is proposed as follows:

[0113] The traffic prediction model obtains the future traffic prediction result Z using the following weighted fusion formula:

[0114] in, The first flow prediction result obtained through step S2.1; The second flow prediction result is obtained after step S2.2; the weighting coefficients are α and β, and α+β=1.

[0115] α and β can be dynamically adjusted based on performance on the validation set, for example, by learning the optimal combination coefficients through metrics such as MSE / MAE, or by introducing an attention mechanism for adaptive weighting.

[0116] S3. Construct the state space of the reinforcement learning agent based on the real-time network status and the future traffic prediction results, train the model using a multi-round online reinforcement learning mechanism, optimize the scheduling strategy in stages, guide the strategy improvement through reward bonus items, and dynamically adjust the action space according to the state space.

[0117] Based on a multi-round online reinforcement learning approach, this method addresses common network environment issues such as distribution shift and behavioral collapse through a two-stage training and reward shaping mechanism. This allows the scheduling strategy to self-correct using self-generated data, improving robustness and generalization ability. This method does not rely on external feedback; it improves the correctness of the final response solely through multiple trials and optimizations by the model itself. The agent selects the optimal scheduling strategy based on the current network state to maximize long-term benefits (such as throughput and load balancing). Through continuous interaction and learning, the system can adapt to changes in the network environment and achieve self-optimization.

[0118] Furthermore, the phased optimization scheduling strategy in step S3 includes:

[0119] Phase 1: Decoupling Attempt Initialization

[0120] Input: Current network status (such as traffic load, link latency, topology information).

[0121] Strategy generation; based on initial state Generate the first scheduling attempt ( The action strategy for the second scheduling attempt includes routing and bandwidth allocation; based on the performance metric p1 fed back after the first scheduling attempt, the action strategy for the second scheduling attempt is improved, resulting in the second scheduling attempt ( );

[0122] Optimization objective: Maximize the reward of the second scheduling attempt. Simultaneously constrained by the KL divergence regularization term First scheduling attempt ( The strategy distribution is designed to prevent strategies from deviating from the reasonable distribution of the basic model.

[0123] In the initialization process of the aforementioned Phase 1 decoupling attempt, the basic scheduling policy model is first fine-tuned. Using a reinforcement learning framework, while optimizing the accuracy of the second scheduling attempt, a Kullback-Leibler divergence regularization term is introduced to constrain the behavior distribution of the first attempt, making it closer to the output distribution of the basic model. This avoids the policy deviating from its original reasonable region and reduces the risk of behavioral collapse. The optimization objectives are as follows:

[0124] Where θ: parameters of the current scheduling strategy model; Current network status, such as user demand, link load, channel status, and other characteristics; The action strategy output during the first scheduling attempt, such as resource block allocation, task migration, etc. Feedback information after the first attempt, such as performance metrics (real-time data such as latency, packet loss rate, bandwidth utilization, and energy consumption) or reward signals, i.e., the first scheduling action. Instant reward values ​​(such as load balancing score, service quality score) or execution Updates to the network status (such as changes in link load and node resource utilization). The improved strategy output from the second scheduling attempt; : The probability distribution of the current model's output scheduling strategy under input state x; : Reference policy model (basic policy) in state The output distribution is as follows; Second scheduling result With ideal scheduling target Similarity score or performance score between them; Kullback-Leibler divergence is used to measure the difference between two policy distributions. : The regularization coefficient of the Kullback-Leibler divergence, used to control whether the behavior of the output on the first attempt deviates from the reference model.

[0125] Phase Two: Joint Optimization of Reward Shaping

[0126] Based on the results of the first stage of optimization, the second stage of joint optimization is performed. Through a reward-shaping mechanism, the model is encouraged to effectively improve the first strategy in the second attempt, avoiding the learning process from falling into a non-self-correcting path. The original optimization objective was:

[0127] ;in, The input state for the first scheduling attempt is usually the original network state (such as traffic load, link state, etc.). The input state for the second scheduling attempt is usually a combination of the state of the first attempt and some of its feedback or response, which helps the agent adjust subsequent strategies based on the effects of historical actions. : Input state of the i-th attempt It is used to guide the generation of scheduling strategies for each round. The action policy output of the first scheduling attempt, i.e., the model based on the initial state. The generated scheduling decision. The action strategy output of the second scheduling attempt takes into account the state feedback information after the first attempt. Based on this, a new strategy is developed after self-correction. The policy output of the i-th attempt With ideal goal strategy The performance matching score between them can be defined comprehensively based on indicators such as throughput, load balancing, and energy consumption. The current policy model is in state. The probability distribution of the generation strategy. The reference policy (such as the base model or previous policy) in the state The probability distribution of the generation strategy is used as a reference target for the Kullback-Leibler divergence. In the i-th attempt, the Kullback-Leibler divergence between the current policy distribution and the reference policy distribution is used to prevent the policy from shifting too far. : Controls the weight of the Kullback-Leibler regularization term in the overall loss. A larger value indicates a greater emphasis on preventing the policy behavior from deviating from the reference policy, in order to avoid "policy collapse" or overfitting.

[0128] To further encourage "improvements" in performance in the second attempt, a bonus item will be introduced in addition to the original reward. The performance difference between two scheduling attempts is quantified and defined as:

[0129] ;

[0130] : Reward bonus, used to measure the performance improvement of the second scheduling compared to the first. Reward amplification factor This is used to enhance the incentive strength for the model's self-correcting behavior. , : These represent the scores for the second and first scheduling results, respectively.

[0131] In step S3 above, in phase one (decoupling attempt initialization), the first scheduling is as follows: based on the initial state x1, a policy y1 is generated, and feedback p1 is obtained after execution. The second scheduling occurs when the input state changes to x2=[x1,p1], and an improved policy y2 is generated. The optimization objective is to maximize the reward of y2 while constraining the behavioral distribution of y1 (using KL divergence regularization).

[0132] The traffic prediction result Z in step S2 is the core input feature for reinforcement learning scheduling decisions and directly affects policy generation: the prediction result serves as a state representation, meaning the state space (x1, x2) of the reinforcement learning agent must contain the predicted future traffic trend (Z), for example: x1 = [real-time network state, Z]. The agent generates scheduling actions (such as route adjustment and bandwidth allocation) based on this state.

[0133] Dynamic adjustment of the action space: If the forecast indicates a future surge in traffic (e.g., a significant increase in Z), the scheduling strategy may expand links or migrate tasks in advance to avoid congestion. For example, if the prediction accuracy in step S2 is insufficient (e.g., failing to capture sudden traffic spikes), the strategy in step S3 may generate suboptimal scheduling actions due to "misjudging" the network state (e.g., insufficient resource allocation). Conversely, high-precision prediction can provide a reliable basis for strategy optimization and improve scheduling efficiency.

[0134] Step S3 generates multiple possible scheduling actions (such as y1, y2) through multiple rounds of trials as candidate solutions for multi-objective optimization.

[0135] S4. Based on the multi-objective optimization mechanism, the strategy is optimized. Combining the weighted linear normalization method and the Pareto front approximation algorithm, the Pareto front solution set that satisfies the multi-objective trade-off is selected from the candidate solutions formed by the action space for actual deployment.

[0136] Network scheduling is essentially a complex decision-making problem with multiple performance constraints and optimization objectives, involving indicators including but not limited to communication latency, link bandwidth utilization, system throughput, energy consumption, load balancing, and quality of service. To achieve a reasonable balance among these performance objectives, this invention introduces a multi-objective optimization mechanism, modeling the optimization of scheduling strategies as a multi-objective problem and constructing a joint objective function containing multiple loss terms.

[0137] Specifically, this system employs a strategy combining weighted linear normalization and the Pareto front approximation algorithm, allowing for dynamic trade-offs among multiple objectives such as latency, energy consumption, and throughput. The weighted method dynamically adjusts the weight coefficients of various performance indicators during the training phase based on scenario preferences, enabling flexible control over different objectives. Meanwhile, the Pareto optimal solution set provides an unbiased scheduling reference set during the deployment phase, allowing the agent to select the most suitable scheduling strategy for the current network state. Step S4 filters the candidate solutions to select the Pareto front solution set that satisfies the multi-objective trade-offs for actual deployment.

[0138] Furthermore, a normalized scoring mechanism is introduced during the scheduling strategy evaluation process to unify indicators with different dimensions (such as millisecond-level latency and kW-level energy consumption) to ensure the fairness and convergence stability of the optimization process. This mechanism allows the system to switch optimization priorities according to actual needs when facing high-reliability, low-latency scenarios (such as vehicle-to-everything (V2X) and high-energy-efficiency scenarios (such as low-power edge devices), thereby improving the system's adaptability and service resilience. The normalized scoring mechanism eliminates differences in indicator dimensions, ensuring optimization fairness. For example, high-load scenarios automatically prioritize low-latency strategies; low-power requirements prioritize energy-efficient strategies.

[0139] Furthermore, in step S4 during the training phase, the weight coefficients of each optimization objective are dynamically allocated according to the current network scenario requirements, thus constructing a joint objective function. :

[0140] ;in, Let be the weight coefficient of the i-th indicator; The normalized score for the q-th indicator;

[0141] A Pareto optimal solution set is generated through a multi-objective optimization algorithm to provide an unbiased scheduling reference set during the deployment phase, ensuring that each scheduling strategy in the solution set cannot be further optimized on a certain objective without harming other objectives;

[0142] Based on the real-time network status, the optimal strategy is selected from the Pareto solution set. The priority rules include: in high-load scenarios, low-latency solutions are selected first; in energy-saving mode, high-efficiency solutions are selected first; and in balanced mode, the solution with the highest comprehensive score is selected.

[0143] S5. Deploy the trained and optimized model to the network controller or edge computing node to achieve real-time perception and dynamic scheduling control of network resources, and update the parameters of the traffic prediction model online based on the scheduling effect data.

[0144] The trained reinforcement learning scheduling model is deployed on network controllers (such as SDN controllers) or edge computing nodes to achieve real-time awareness and dynamic scheduling control of network resources. In actual deployment, the model automatically generates scheduling decisions by continuously sensing the current network status (including link load, node status, traffic patterns, etc.), achieving intelligent scheduling response with millisecond-level latency.

[0145] The scheduling effect data collected in actual deployment (such as actual latency and energy consumption) can be fed back into step S2 to update the prediction model (such as online fine-tuning of diffusion model parameters); at the same time, the scheduling feedback data can also be used to correct the multi-objective weights, forming a closed-loop optimization of "prediction → decision → feedback → re-prediction".

[0146] In some embodiments, to address continuous changes in the network environment, such as dynamic business demands, fluctuating link states, and node online / offline status, this invention supports an online learning and incremental update mechanism for the model. Specifically, it introduces an experience replay mechanism and a policy soft update mechanism, extracting samples from new experiences accumulated during operation to periodically fine-tune model parameters, maintaining the model's long-term adaptability. Reinforcement learning algorithms (such as DQN, PPO, or Actor-Critic) are used for lightweight iterative updates on edge nodes, ensuring self-evolution capabilities even with low computing resources. While ensuring uninterrupted service, a smooth transition between old and new policies is achieved through a model hot-swap mechanism or a multi-model redundancy switching mechanism, effectively preventing model degradation or catastrophic forgetting.

[0147] In summary, the deep learning-based network traffic scheduling optimization method provided by the above embodiments of the present invention solves the problems of low prediction accuracy, rigid strategies, and resource waste in traditional network traffic scheduling through a prediction-decision-optimization closed-loop system. It achieves high-precision prediction, adaptive decision-making, low-energy operation, and high-reliability service in complex dynamic environments, and is applicable to scenarios such as 5G, IoT, and cloud computing, significantly improving network service quality and operational economy.

[0148] Furthermore, embodiments of this application also provide an electronic device, including: a processor, a memory, and a system bus; the processor and the memory are connected via the system bus; the memory is used to store one or more programs, the one or more programs including instructions, which, when executed by the processor, cause the processor to perform any of the methods described above.

[0149] Furthermore, this application also provides a computer program product that, when run on a terminal device, causes the terminal device to execute any of the methods described above.

[0150] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0151] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0152] It should also be noted that, in the embodiments of this application, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0153] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined in the embodiments of this application may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown in this application, but is to be accorded the widest scope consistent with the principles and novel features disclosed in the embodiments of this application.

Claims

1. A network traffic scheduling optimization method based on deep learning, characterized in that, Includes the following steps: S1. Collect traffic data, link status, latency and packet loss rate metrics deployed on network nodes in real time, and build a sample database; S2. Based on the sample database, construct a traffic prediction model that integrates a retrieval-enhanced diffusion model and a hybrid linear expert model to perform multimodal traffic prediction, and output the future traffic prediction results through weighted fusion. The construction of the retrieval-enhanced diffusion model includes: The sample time series in the sample database are truncated into fixed-length segments according to a unified time window to form a candidate sample set D; The pre-trained encoder maps the current input sample time series to the first embedding vector. For each sample in the sample database, the first π time steps are extracted, and a second embedding vector is generated using the same encoder; Calculate the similarity between the first embedding vector and all the second embedding vectors, and select the k sample indices with the smallest similarity to form a retrieval sample set; A diffusion probability model framework is constructed, and the retrieved sample set is encoded and guided to denoise through a mechanism of forward diffusion destroying data distribution and reverse diffusion reconstructing target sequence, thereby generating denoised prediction results; S3. Construct the state space of the reinforcement learning agent based on the real-time network status and the future traffic prediction results, train the traffic prediction model using a multi-round online reinforcement learning mechanism, optimize the scheduling strategy in stages, guide the strategy improvement through reward bonus items, and dynamically adjust the action space according to the state space. S4. Based on the multi-objective optimization mechanism, the strategy is optimized. Combining the weighted linear normalization method and the Pareto front approximation algorithm, the Pareto front solution set that satisfies the multi-objective trade-off is selected from the candidate solutions formed by the action space for actual deployment. S5. Deploy the trained and optimized traffic prediction model to the network controller or edge computing node to achieve real-time perception and dynamic scheduling control of network resources, and update the parameters of the traffic prediction model online based on the scheduling effect data.

2. The network traffic scheduling optimization method based on deep learning according to claim 1, characterized in that, in, The construction is based on a diffusion probability model framework. It encodes and guides denoising of the retrieved sample set through a mechanism that disrupts data distribution via forward diffusion and reconstructs the target sequence via reverse diffusion, generating denoised prediction results, including: In the forward diffusion phase: starting from the real flow sequence x0, Gaussian noise is gradually added over time step T to generate a noisy sequence x1, x2, ..., x T initial noise vector , It approximately follows a standard normal distribution N(0,I) and its dimension is consistent with the prediction target; In the reverse diffusion prediction stage: By using a pre-trained time-series encoder, each reference sample in the retrieval sample set is mapped to an embedding vector, and its deep temporal features are extracted to obtain the retrieval sample encoding features. ; The diffusion state characteristics at the current diffusion time step t Features after encoding the input time series The retrieval sample encoding features The current diffusion time step t is input to the reference modulation attention module (RMA), which calculates the dynamic weights between features through a multi-head attention mechanism, and dynamically adjusts the contribution ratio of the reference sample to the denoising process. Denoising is performed at each step of the reverse diffusion from t=T to t=1, gradually generating denoised predictions. After T-step denoising, the first flow prediction result is obtained. .

3. The network traffic scheduling optimization method based on deep learning according to claim 2, characterized in that, in, The structure of the hybrid linear expert model includes: a multi-expert parallel prediction layer, a dynamic weight router, and a weighted fusion and post-processing layer; The multi-expert parallel prediction layer contains n independent linear expert models, each of which is a linear regression model with different structures or parameters, and is used to capture the long-term trend, periodic or sudden characteristics of the time series respectively. The dynamic weight router includes an encoder for timestamp embedding encoding of the input time series and a hybrid layer consisting of two fully connected networks; the hybrid layer is used to map the timestamp embedding encoding features into a channel-specific weight matrix. The weighted fusion and post-processing layer includes a channel-weighted fusion unit and a post-processing layer. The channel-weighted fusion unit is used to calculate the weight of each expert prediction result for each channel according to the channel-specific weight matrix to obtain the fusion result. The post-processing layer is used to perform nonlinear correction on the fusion result and output the final prediction result.

4. The network traffic scheduling optimization method based on deep learning according to claim 3, characterized in that, in, The construction of the hybrid linear expert model includes: Set up n linear expert models Each expert For the input sample time series Perform independent predictions and output the prediction results for a future time window of length p. Where c represents the number of channels for traffic monitoring; Time stamp embedding features based on input sequence , m represents the dimension of the timestamp embedding feature; a hybrid layer consisting of two MLP layers. Perform dynamic weighted routing and generate channel-specific weight matrices. The calculation formula is: ； The prediction results of each expert model The summation is performed based on the specific weights corresponding to each channel, and then processed by the post-processing layer. Adjust the output trend and output the second flow prediction result. .

5. The network traffic scheduling optimization method based on deep learning according to claim 4, characterized in that, The post-processing layer Adjust the output trend and output the second flow prediction result. ,include: ； in, Defined as a channel-specific multiplication operation; The outputs of each expert model are weighted and combined to form a comprehensive prediction representation; The post-processing layer includes residual connections and layer normalization operations to compensate for prediction errors in weighted fusion, further fit errors, and adjust trends. Second flow prediction results; The timestamp embedding feature The encoding method is as follows: normalize the day of the week, hour, and holiday markers respectively, and then concatenate them into a vector.

6. The method according to claim 5, characterized in that, The weighted fusion formula for the traffic prediction model is: ； Where Z is the predicted future traffic; α and β are dynamically adjusted fusion coefficients, and satisfy α+β=1.

7. The network traffic scheduling optimization method based on deep learning according to claim 1, characterized in that, The phased optimization scheduling strategy includes: Phase 1: Decoupling Attempt Initialization Policy generation; Generate the action policy for the first scheduling attempt based on the initial state, including route selection and bandwidth allocation; Improve the action policy for the second scheduling attempt based on the performance metrics fed back after the first scheduling attempt; Optimization objective: Maximize the reward of the second scheduling attempt, while constraining the policy distribution of the first scheduling attempt through KL divergence regularization to prevent the policy from deviating from the reasonable distribution of the base model; Phase Two: Joint Optimization of Reward Shaping Based on the optimization results of the first stage, the second stage of joint optimization is carried out. Through the reward shaping mechanism, the optimization process drives the iterative optimization of the strategy by the feedback difference between the two scheduling attempts, which encourages the traffic prediction model to effectively improve the first strategy in the second scheduling attempt.

8. The network traffic scheduling optimization method based on deep learning according to claim 1, characterized in that, S4 involves strategy optimization based on a multi-objective optimization mechanism, combining a weighted linear normalization method and a Pareto front approximation algorithm to select a set of Pareto front solutions that satisfy the multi-objective trade-offs from the candidate solutions for actual deployment. This includes: During the training phase, the weight coefficients of each optimization objective are dynamically allocated according to the current network scenario requirements, and a joint objective function is constructed. : ； in, Let be the weight coefficient of the i-th indicator; The normalized score for the q-th indicator; A Pareto optimal solution set is generated through a multi-objective optimization algorithm to provide an unbiased scheduling reference set during the deployment phase, ensuring that each scheduling strategy in the solution set cannot be further optimized on a certain objective without harming other objectives; Based on the real-time network status, the optimal strategy is selected from the Pareto solution set. The priority rules include: in high-load scenarios, low-latency solutions are selected first; in energy-saving mode, high-efficiency solutions are selected first; and in balanced mode, the solution with the highest comprehensive score is selected.

Images