A dynamic time slot allocation method based on prediction and reinforcement learning and related devices

Abstract

The application belongs to the technical field of wireless communication network resource management, and discloses a dynamic time slot allocation method based on prediction and reinforcement learning and related devices, which predicts future time slot demand through a variational mode decomposition and deep learning hybrid model, and solves a Markov decision process with prediction information and a current state as input by using an integrated attention mechanism near-optimal algorithm to generate a dynamic time slot allocation action. The method significantly improves the time slot resource utilization efficiency, enhances the adaptability of the network to a high-dynamic strong-interference environment, effectively reduces communication conflicts and transmission delay, improves throughput and overall robustness, and meets the urgent needs for resource optimization and communication guarantee in complex scenarios.

Description

Technical Field

[0001] This invention belongs to the field of wireless communication network resource management technology, and particularly relates to a dynamic time slot allocation method and related apparatus based on prediction and reinforcement learning. Background Technology

[0002] Data links, acting as the "nervous system" of modern information exchange, are crucial infrastructure for enabling interconnection and collaborative task completion among communication nodes, occupying a central position in modern communication networks. Currently, typical data link systems such as Link-16 widely employ Time Division Multiple Access (TDMA) technology. This technology divides the time axis into continuous time slots, allocating specific time slots to different communication nodes to achieve orderly and conflict-free communication transmission. The quality of the time slot allocation strategy directly determines the core performance indicators of the data link network, such as throughput, transmission latency, and reliability, thus having a critical impact on overall communication efficiency. As modern communication networks evolve towards integrated land, sea, air, and space communication and intelligent collaboration, the importance of time slot allocation becomes increasingly prominent.

[0003] Currently, traditional time slot allocation schemes are mainly divided into three categories: fixed allocation, contention-based random access, and reservation-based on-demand allocation. Each type of scheme has obvious problems and drawbacks. Among them, fixed allocation schemes, such as classic TDMA, are simple to control and have no communication conflicts, but they cannot adapt to the dynamic changes in node traffic, easily leading to wasted time slot resources or congestion during sudden traffic surges, resulting in low resource utilization and difficulty in meeting the needs of highly dynamic communication scenarios. Contention-based random access schemes, such as Carrier Sense Multiple Access (CSMA), have a certain degree of flexibility, but they are prone to frequent communication conflicts under high load environments, leading to a sharp increase in transmission latency, a sharp drop in throughput, and poor stability. Reservation-based on-demand allocation schemes, such as DAMA, are more flexible than fixed allocation, but existing schemes of this type mostly make decisions based on historical traffic averages or simple prediction models, making it difficult to accurately capture the nonlinearity, non-stationarity, and suddenness of business demands in complex environments. Moreover, the allocation logic relies on manually set static rules, lacking online learning and autonomous optimization capabilities, and cannot achieve a globally optimal trade-off between multiple objectives. Meanwhile, in highly dynamic and uncertain complex environments characterized by high node mobility, strong electromagnetic interference, and sudden and frequent changes in service traffic, existing time slot allocation methods still suffer from core defects such as insufficient "perception" and limited "decision-making" capabilities, making it difficult to adapt to the rapid changes in network status.

[0004] Existing time slot allocation methods cannot achieve a deep integration of accurate forward-looking perception and intelligent autonomous decision-making, and are insufficient to meet the demands of data link networks in complex dynamic environments for improved resource utilization efficiency, communication assurance capabilities, and overall robustness. Summary of the Invention

[0005] This invention provides a dynamic time slot allocation method and related apparatus based on prediction and reinforcement learning. This method can achieve a deep integration of accurate forward perception and intelligent autonomous decision-making, thereby meeting the needs of data link networks in complex dynamic environments for improved resource utilization efficiency, communication assurance capabilities and overall robustness.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: A dynamic time slot allocation method based on prediction and reinforcement learning includes: Historical time slot demand characteristic sequences were obtained based on simulated communication datasets. Based on the historical time slot demand feature sequence, a pre-constructed variational mode decomposition and deep learning hybrid model is used to make multi-step predictions of future time slot demand, resulting in a predicted demand sequence. The time slot allocation problem is modeled as a Markov decision process; wherein, the state space of the Markov decision process includes the current allocation information and the predicted demand sequence, and the action space is the time slot allocation ratio corresponding to each communication node; A proximal policy optimization algorithm with integrated attention mechanism is used to solve the Markov decision process and obtain the time slot allocation action; The time slot allocation action is transformed into an actual time slot allocation scheme that satisfies network constraints to complete dynamic time slot allocation; wherein, the near-end policy optimization algorithm integrating the attention mechanism is updated according to the feedback reward; the reward is calculated based on the network performance after execution.

[0007] Furthermore, based on the historical time slot demand feature sequence, a pre-constructed variational mode decomposition and deep learning hybrid model is used to perform multi-step prediction of future time slot demand to obtain a predicted demand sequence, including: The variational mode decomposition method is used to decompose the historical time slot demand characteristic sequence of each node into multiple intrinsic mode function components; The original features are combined with the intrinsic mode function components, and the combined features are input into a deep learning model to obtain intermediate output results; the deep learning model adopts an improved temporal convolutional network. The intermediate output results are fed into the improved Transformer encoder module, which synchronously outputs the prediction requirement sequence for all node types through a multi-task learning architecture.

[0008] Furthermore, the improved temporal convolutional network integrates: Gated dilated causal convolution mechanism is used to achieve adaptive adjustment of input features; A multi-scale skip connection aggregation mechanism is used to fuse local temporal features from different receptive fields; A dual attention mechanism is used to recalibrate the importance of the feature map in both the channel and temporal dimensions. The improved Transformer encoder module includes: Temporally enhanced location coding is used to fuse absolute time, periodicity, and trend information; A local-global hybrid attention architecture, comprising local causal convolutional layers for capturing recent associations and sparse global attention layers for modeling long-term dependencies; The improved temporal convolutional network and the improved Transformer encoder module are deeply integrated through feature injection and knowledge distillation loss.

[0009] Furthermore, the state vector of the Markov decision process adopts a node-feature two-layer structure. For a network containing N types of nodes, the state vector... for:

[0010] in, For node type The 7-dimensional feature vector includes at least: the current time slot allocation ratio, the future demand ratio based on the predicted demand sequence, the node relay capability index, the node communication capability index, the operational priority, the historical network average utilization rate, and the dynamic communication quality factor.

[0011] Furthermore, the policy network and value network of the proximal policy optimization algorithm with integrated attention mechanism both adopt an architecture that includes a multi-head attention mechanism; wherein, the training process of the proximal policy optimization algorithm adopts a course learning mechanism, and the training environment parameters gradually transition from simple static scenarios to complex dynamic scenarios including high interference, high mobility and burst traffic.

[0012] Furthermore, the reward function of the Markov decision process Weighted sum for multiple objectives Specifically as follows:

[0013] in, These represent the normalized throughput reward, latency penalty, fairness reward, priority guarantee reward, and stability penalty, respectively. , , , , These are the corresponding weight coefficients, all determined using the analytic hierarchy process (AHP).

[0014] Furthermore, the step of transforming the time slot allocation action into an actual time slot allocation scheme that satisfies network constraints to complete dynamic time slot allocation includes: Perform node-level minimum / maximum ratio pruning on time slot allocation actions; The ratio is mapped to an integer number of time slots to generate a preliminary allocation scheme; If the total number of time slots in the initial allocation scheme exceeds the total number of time slots in the system, the weights are adjusted based on node priority, and iterative reduction is performed until the total system constraint is met. The actual time slot allocation scheme is then output, and dynamic time slot allocation is completed.

[0015] A dynamic time slot allocation system based on prediction and reinforcement learning includes: The data acquisition module is used to acquire historical time slot demand characteristic sequences based on simulated communication datasets. The time slot prediction module is used to perform multi-step prediction of future time slot demand based on the historical time slot demand feature sequence, using a pre-built variational mode decomposition and deep learning hybrid model to obtain the predicted demand sequence. A construction module is used to model the time slot allocation problem as a Markov decision process; wherein, the state space of the Markov decision process includes the current allocation information and the predicted demand sequence, and the action space is the time slot allocation ratio corresponding to each communication node; The solution module is used to solve the Markov decision process using a proximal policy optimization algorithm with an integrated attention mechanism to obtain the time slot allocation action; The output module is used to transform the time slot allocation action into an actual time slot allocation scheme that satisfies network constraints, so as to complete the dynamic time slot allocation; wherein, the near-end policy optimization algorithm with integrated attention mechanism is updated according to the feedback reward; the reward is calculated based on the network performance after execution.

[0016] A dynamic time slot allocation device based on prediction and reinforcement learning, comprising: Memory, used to store computer programs; A processor is used to implement the above-described dynamic time slot allocation method based on prediction and reinforcement learning when executing the computer program.

[0017] A computer-readable storage medium storing a computer program, which, when executed by a processor, is used to implement the above-described dynamic time slot allocation method based on prediction and reinforcement learning.

[0018] Compared with the prior art, the present invention has the following beneficial effects: This invention provides a dynamic time slot allocation method based on prediction and reinforcement learning. It predicts future time slot demands using a hybrid model combining variational mode decomposition and deep learning, and employs a proximal policy optimization algorithm with an integrated attention mechanism to solve a Markov decision process with the predicted information and the current state as input, thereby generating dynamic time slot allocation actions. The prediction model accurately captures the nonlinearity and suddenness of business demands, providing forward-looking basis for decision-making; the reinforcement learning framework focuses on key information through an attention mechanism, enabling online learning and autonomous optimization of policies, achieving a global trade-off among multiple objectives. This method significantly improves the efficiency of time slot resource utilization, enhances the network's adaptability to highly dynamic and interference-prone environments, effectively reduces communication conflicts and transmission latency, improves throughput and overall robustness, and meets the urgent needs for resource optimization and communication assurance in complex scenarios. Attached Figure Description

[0019] Figure 1 A flowchart for generating a dataset provided in an embodiment of the present invention; Figure 2 The time slot demand decomposition spectrum diagram provided in the embodiments of the present invention; Figure 3 The following is a time slot demand distribution diagram provided in an embodiment of the present invention; wherein, (a) is a time slot demand distribution histogram; (b) is a time slot demand box plot; (c) is the time slot demand distribution for different node types; and (d) is the relationship between time slot demand and the number of service messages. Figure 4 A flowchart of the improved PPO time slot allocation algorithm provided in an embodiment of the present invention; Figure 5 A flowchart illustrating the architecture implementation of a dynamic time slot allocation system based on prediction and reinforcement learning, provided in an embodiment of the present invention; Figure 6 The core flowchart of a dynamic time slot allocation method based on prediction and reinforcement learning provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of a dynamic time slot allocation system based on prediction and reinforcement learning, provided as an embodiment of the present invention. Detailed Implementation

[0020] To further understand the content of this invention, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments are merely illustrative and not limiting of the invention.

[0021] As described in the background section, existing dynamic time slot allocation methods typically rely on real-time negotiation between nodes. In highly dynamic and complex network environments, as the scale of network nodes expands, the negotiation collision rate between nodes increases dramatically, leading to a decrease in overall system throughput and resource utilization efficiency. Furthermore, traditional methods often employ planar network structures, which are difficult to adapt to the dynamic expansion requirements of network scale, and control overhead increases significantly with the number of nodes, failing to meet the urgent requirements for real-time performance, reliability, and global optimization of communication resources in large-scale dynamic networking scenarios. To address these technical problems, this invention proposes an intelligent time slot allocation method based on a "prediction-decision" collaborative framework. This method constructs an integrated "perception-decision" intelligent closed loop, first achieving advanced and accurate prediction of future time slot demands, and then using this prediction information as key input to drive an intelligent decision-making mechanism capable of online learning and autonomous optimization, generating a dynamic time slot allocation strategy. Ultimately, this improves the overall utilization efficiency of communication resources, critical service assurance capabilities, and network robustness of data link communication systems in highly dynamic, highly uncertain, and complex environments.

[0022] For example, such as Figure 6 As shown, this embodiment provides a dynamic time slot allocation method based on prediction and reinforcement learning, including: Historical time slot demand characteristic sequences were obtained based on simulated communication datasets. Based on the historical time slot demand feature sequence, a pre-constructed variational mode decomposition and deep learning hybrid model is used to make multi-step predictions of future time slot demand, resulting in a predicted demand sequence. The time slot allocation problem is modeled as a Markov decision process; wherein, the state space of the Markov decision process includes the current allocation information and the predicted demand sequence, and the action space is the time slot allocation ratio corresponding to each communication node; A proximal policy optimization algorithm with integrated attention mechanism is used to solve the Markov decision process and obtain the time slot allocation action; The time slot allocation action is transformed into an actual time slot allocation scheme that satisfies network constraints to complete dynamic time slot allocation; wherein, the near-end policy optimization algorithm integrating the attention mechanism is updated according to the feedback reward; the reward is calculated based on the network performance after execution.

[0023] Therefore, this embodiment provides a dynamic time slot allocation method based on prediction and reinforcement learning. By constructing a three-in-one technical framework of "high-fidelity simulation data-driven," "VMD-TCN-Transformer accurate prediction," and "improved PPO intelligent decision-making," it achieves advanced perception and autonomous optimization scheduling of communication data link time slot resources. First, by introducing variational mode decomposition and an improved TCN-Transformer hybrid model, this method can deeply analyze and accurately predict non-stationary and nonlinear communication service demands, with a prediction determination coefficient R0. 2 Achieving a latency of over 0.997 provides reliable forward-looking input for resource scheduling, fundamentally solving the "perception lag" problem of traditional methods. Secondly, by modeling the problem as an MDP (Markov Decision Process) and solving it using an improved PPO (Proximal Policy Optimization) algorithm with integrated attention mechanisms, this method enables the system to learn online, make multi-objective trade-offs, and make autonomous decisions in complex and dynamic environments. It can adaptively generate strategies to optimize the long-term overall network performance without relying on rigid predefined rules. It exhibits excellent robustness, especially in complex scenarios with high interference, high mobility, and burst traffic, showing an average throughput increase of 25%-40% and an average latency reduction of 30%-50% compared to traditional DAMA and STDMA algorithms. Finally, by designing a complete "prediction-decision-execution-feedback" closed loop, this system can continuously optimize itself using interactive experience, forming an environmentally adaptable intelligent resource management entity. This significantly improves the resource utilization efficiency, critical service assurance capabilities, and overall network resilience of communication networks in highly dynamic and complex environments, meeting the urgent needs of modern intelligent communication networks for autonomous, accurate, and reliable resource management.

[0024] For example, the specific steps of the dynamic time slot allocation method based on prediction and reinforcement learning provided in this embodiment are as follows: Step 1: Construct a communication simulation dataset.

[0025] Due to the difficulty in acquiring communication data in real-world, complex environments, we first construct a communication simulation dataset that closely matches actual dynamic communication scenarios and possesses high dynamism and complexity. This dataset will be used for subsequent model training and validation. The dataset construction follows the principles of "scenario-driven, comprehensive elements, and dynamic evolution."

[0026] First, a heterogeneous node system containing seven typical communication node types is constructed, as shown in Table 1. Each type of node is configured with differentiated initial parameters based on its functional role and operational characteristics in the network, including communication priority, speed range, formation type, and main functions.

[0027] Table 1 shows the node type and basic parameter settings.

[0028] Node attributes are divided into static attributes (node ​​type, communication priority, formation mode) and dynamic attributes (geographic location, motion state, health status, energy remaining). This method systematically models from four levels: node, environment, service, and global state. By introducing differentiated energy consumption, health decay, and environmental resistance coefficients for each type of node, and designing a hierarchical management mechanism that integrates deterministic scripts and random damage events, the individual differences and highly dynamic evolution characteristics of communication nodes are simulated. A three-dimensional coupled channel model is constructed by comprehensively integrating position updates based on spherical motion, diverse formation modes, Friis transmission model, and multiple weather influencing factors. Service generation adopts a three-level architecture of "priority-type-quantity," dynamically generating multiple types of communication loads with sudden characteristics based on Poisson distribution and weather adjustment factors. Through global state tracking and condition-triggered event system, nodes, environment, and service modules are organically linked, and a closed-loop model of consumption and state decay of motion state, environmental conditions, and global resource level is established.

[0029] Dataset generation follows a modular simulation process, such as... Figure 1 As shown. The process begins with environment and parameter initialization, followed by the time-element-time-frame main loop, traversing 20 time elements and 64 time frames in a two-level nested structure, totaling 1280 simulation moments. Within each time frame, the following are executed sequentially: event triggering and processing, node state and location updates, service message generation (based on Poisson distribution), time slot demand calculation (through a three-level adjustment process: basic demand calculation, channel quality adjustment, and environmental factor correction), data recording, and periodic state persistence. Figure 3 As shown, a standardized CSV dataset containing 15 dimensions (spatiotemporal identifier, node attributes, business features, and environmental parameters) is finally generated, providing a high-quality and reproducible verification platform for subsequent algorithm research. The descriptions of each field are shown in Table 2.

[0030] Table 2 provides a description of the dataset fields.

[0031] Step 2: Time slot service demand prediction based on VMD-TCN-Transformer.

[0032] This step aims to proactively and accurately predict future business needs for time slots, providing forward-looking intelligence for resource scheduling.

[0033] The time slot demand forecasting is modeled as a multivariate time series multi-step forecasting problem. The model input is a multidimensional feature sequence within a sliding window consisting of the N most recent consecutive time frames:

[0034] Among them, the feature vector of each time frame It includes historical values ​​of the core prediction target (slot demand), node status characteristics (node ​​type encoding, quantity, priority, health status, fuel remaining, formation type encoding), and environmental and spatial characteristics (weather condition encoding, average distance to key nodes, latitude and longitude). The model's prediction target is the future. The time slot requirement for each time frame:

[0035] in Indicates to Predicted values ​​of time slot demand.

[0036] like Figure 2 As shown, to reduce the non-stationarity and noise interference of the original time slot demand sequence, variational mode decomposition (VMD) is first introduced to preprocess the sequence. The normalized historical time slot demand sequence is then... As a one-dimensional discrete-time signal, it is input into the VMD algorithm. The number of decomposed modes is set. Punishment factor Convergence tolerance The algorithm performs ADMM iterations, iteratively updating each IMF component and its center frequency until the convergence criterion is met. The algorithm ultimately outputs K eigenmode function components arranged from high to low frequency. and a residual trend term ,satisfy The length of each output sequence is equal to the length of the original sequence. The signal and noise are kept consistent. The high-frequency IMF captures random noise and sudden fluctuations, the mid-frequency IMF reflects periodic change patterns, and the low-frequency IMF characterizes long-term trends, thus achieving effective separation of signal and noise.

[0037] The high-dimensional feature sequence, composed of the VMD decomposition components and other related features, is input into an improved temporal convolutional network module for multi-scale local temporal feature extraction. This module consists of... The system consists of stacked residual blocks, each containing a gated dilated causal convolutional layer. Blocks are aggregated via multi-scale skip connections, and a dual attention mechanism is introduced at the end. Specific improvements are as follows: Gated dilated causal convolution mechanism: This introduces a gating mechanism based on dilated causal convolution. For input features... First, the number of channels is expanded to 2C using dilated causal convolution:

[0038] Then Feature representations are evenly distributed along the channel dimension. With gate signal :

[0039] Apply a sigmoid activation function to the gated signal to generate gate weights. And then, it is multiplied element-wise with the feature representation to achieve adaptive adjustment of the features:

[0040] Finally, residual connections are used to preserve the original information and ensure gradient flow.

[0041] Multi-scale skip connection aggregation mechanism: To fuse feature representations from different receptive fields, the output of each residual block is... Weighted summation is performed using learnable aggregate weights. First, learnable parameters are initialized. The contribution of each layer is obtained by normalization using the Softmax function. The final multi-scale aggregation features for:

[0042] Weight It is updated along with other network parameters during training, enabling the model to adaptively assign importance to features at different levels.

[0043] Dual attention mechanism: Parallel channel attention and temporal attention are introduced at the end of the TCN module.

[0044] Channel attention: for feature maps Global average pooling is performed along the time dimension to obtain channel statistical descriptors. Then, channel attention weights are generated using two fully connected layers (with an intermediate layer dimension of C / 8) and a sigmoid function. Weight the original feature map:

[0045] Temporal attention: Features at each time step are encoded using a one-dimensional convolution (kernel size 3, output channel 1), and then normalized in the temporal dimension using a softmax function to obtain the temporal attention weights. Weight the original feature map:

[0046] Finally, the attention-enhanced features are fused with the original features via residual connections, and a learnable scaling factor is used to balance the ratio of original to enhanced information to obtain the final output. .

[0047] The features output by the TCN module are passed to the improved Transformer encoder module to capture long-term dependencies in the sequence. Improvements include: Timing-enhanced position coding: This involves encoding standard sinusoidal position codes. Compared with the standardized absolute timestamp Periodic coding (Period is 64) and moving average trend characteristics (Window size 5) Fusion, aligning dimensions through a learnable linear projection matrix, yields temporally enhanced location encoding. :

[0048] It is added to the input features and then fed into the Transformer layer.

[0049] Local-Global Hybrid Attention Architecture: Employs a hierarchical hybrid attention mechanism. First, it captures tight correlations within the local neighborhood through one-dimensional causal convolution, with the kernel size... Then, sparse global attention is employed, calculating only the query and the selected queries based on importance scores (such as historical demand variance). Key time steps ( Attention between the weights. Finally, an adaptive gating weight is used. Combine two layers of output:

[0050] This mechanism reduces computational complexity to a minimum while retaining the ability to model long-term dependencies. .

[0051] TCN-Transformer Deep Fusion Strategy: Employs a feature interaction fusion strategy. The TCN module's... The output of each residual block , through a After the convolutional adaptation layer, a skip connection is made to the Transformer's first... The input to the layer encoder. Simultaneously, an auxiliary distillation loss is introduced during training. Encourage TCN and Transformer intermediate layer feature representations to be close to each other:

[0052] in For Transformer The output of the layer, This is an optional projection layer.

[0053] Channel-temporal dual attention: Parallel channel attention and temporal attention are also integrated within the Transformer layer to enhance the modeling of multivariate interactions and key timing sequences.

[0054] To simultaneously output future demand sequences for multiple node types, this method employs a multi-task learning architecture. General high-level features encoded by Transformer are input into the multi-task output layer, with an independent output branch for each node type to be predicted. Each branch contains a task-specific fully connected layer. During training, an adaptive multi-task loss weighting mechanism is used. The total loss function is a weighted sum of the losses from each task:

[0055] in Number of node types For the first Mean squared error loss for each task. Weights The training process is dynamically updated: the initial weights are set based on the historical verification error of each task, and are smoothly updated after each batch according to the current loss. Temperature coefficient scaling is used to prevent the weights of some tasks from being too small and causing them to stagnate in learning.

[0056] To further improve prediction accuracy, a three-level residual correction module was designed. Gradient boosting regression trees were used to model the residuals of the initial prediction results, and an iterative correction strategy was employed to gradually reduce the system's prediction bias. The residual correction process was repeated three times.

[0057] The model training employs a stochastic search strategy for hyperparameter optimization, with the search space including the number of TCN filters (64-256), the number of Transformer attention heads (2-8), and the learning rate (1e-5 to 1e-3). During training, an early stopping mechanism (stopping after 30 epochs without improvement) and dynamic learning rate adjustment (halving the learning rate during loss plateaus) are used. The batch size is set to 256, and the maximum training epochs are 400. After training, the model can simultaneously output the predicted slot demand values ​​for all node types in the next M time frames.

[0058] Step 3: Dynamic time slot intelligent allocation based on improved PPO.

[0059] like Figure 4 As shown, this step uses the predicted information as the key input and utilizes deep reinforcement learning algorithms to learn online and generate a dynamic allocation strategy that can maximize the long-term overall performance of the network.

[0060] The time slot allocation problem is modeled as a Markov decision process quintuple. .

[0061] State space S: The state vector is constructed using a two-layer node-feature structure. For a network containing N types of nodes, the state vector is defined as:

[0062] in Is a node type The 7-dimensional feature vector has seven dimensions, namely: the current allocation ratio. (Previous allocation ratio), demand ratio (Based on predicted future demand), relay capability index (determined by the number of relay nodes), communication capability index (determined by the number of antennas, transmit power, etc.), node priority (static priority), historical network utilization (average utilization of the most recent time slot), and communication quality factor (combining environmental factors such as interference and mobility). All features have been normalized.

[0063] Action space A: Designed as a continuous space, action vectors ,in Represents a node The allocated time slot ratio. To ensure the feasibility of the actions, a multi-level constraint processing mechanism is designed to convert continuous actions into actual time slot allocations. First, node-specific minimum and maximum constraints are applied to each action component: a pruning operation is used to ensure... Then, the ratio is mapped to an integer number of time slots: If the initial allocation totals exceed the total number of time slots. Then, Algorithm 1 (time slot total constraint adjustment algorithm) is executed for iterative adjustment. This algorithm calculates the adjustment weight based on node priority (the lower the priority, the higher the weight), allocates the reduction amount according to the weight, and ensures that the allocation of each node is not lower than its minimum requirement through iterative fine-tuning, and finally strictly meets the total constraint.

[0064] reward function A multi-objective reward function is constructed using a linear weighted combination approach, unifying five key metrics—throughput, latency, fairness, priority guarantee, and stability—within a single optimization framework.

[0065] The calculations for each component are as follows: Throughput Bonus: = ,in This is the channel quality factor.

[0066] Delay penalty: .

[0067] Fairness rewards: calculated using Jain's fairness index. .

[0068] Priority guarantee reward: .

[0069] Stability penalty: .

[0070] Weight vector of each component The weights are determined using the Analytic Hierarchy Process (AHP). First, a judgment matrix is ​​constructed (based on expert evaluation). Then, the normalized eigenvector corresponding to the largest eigenvalue is calculated. After consistency verification, the weights are adjusted according to the numerical range and optimization direction of each component to obtain the final weights. In this embodiment, the weights are configured as follows: throughput 1.5, latency -0.5, fairness 0.3, priority guarantee 0.2, and stability -0.1, reflecting the core requirements of "ensuring connectivity, stability, and criticality."

[0071] State transition function The overall state transition is decomposed into three relatively independent probabilistic processes: allocation transition, demand transition, and environmental transition. Their joint probability distribution can be expressed as:

[0072] The allocation transfer is deterministic (determined by the mapping from action to actual allocation), the demand transfer is based on the VMD-TCN-Transformer prediction model and Gaussian noise is introduced to simulate prediction error, and the environmental transfer (including combat intensity, interference status, and mobility) is characterized by random walk, Markov chain and other models.

[0073] To address the problem that multilayer perceptrons in traditional PPO struggle to capture complex dependencies between high-dimensional state features, a multi-head attention mechanism is introduced into the policy network and value network of PPO, constructing a three-layer architecture of "feature extraction - attention enhancement - task-specific output".

[0074] Basic feature extraction layer: Consists of two fully connected layers, each with 256 neurons, followed by a ReLU activation function and layer normalization. Input state vector. After extraction, a high-level feature representation is obtained. .

[0075] Multi-head attention enhancement layer: Reshapes H into a sequence (assuming N nodes, each node has the following features). The input is a 4-head self-attention mechanism, with each attention head having a dimension of 64. The multi-head attention output is then processed through residual connections and layer normalization to obtain the enhanced feature representation. .

[0076] Strategy output layer: The dimensions are compressed to 128 through a fully connected layer, and then the mean vector of the action distribution is output through two independent linear layers. Sum of logarithmic and standard deviation vectors The mean is constrained to the [0,1] interval using the Sigmoid function, and the standard deviation is obtained through exponential operations. Action sampling was obtained .

[0077] Value Network: Employs a similar architecture, but its output layer is a single-neuron linear layer, outputting scalar state values. The parameters of the value network and the policy network are independent and not shared.

[0078] Course learning mechanism: The entire training process is divided into four stages, totaling 1000 rounds. The environmental parameter settings for each stage are shown in Table 3. The parameters remain stable within each stage, and linear interpolation is used for smooth transitions when switching stages.

[0079] Table 3 shows the parameter configurations for the course learning phase.

[0080] Adaptive learning rate scheduling: The Adam optimizer and ReduceLROnPlateau scheduler are configured for the policy network and value network, respectively. The scheduler monitors the moving average reward (window size 10) on the validation set, and determines the scheduler to respond when the improvement falls below a threshold for 10 consecutive rounds. When the learning rate is below a preset minimum (1e), the learning rate is multiplied by a decay factor of 0.5. -6 And when performance stagnates, trigger a warm restart: reset the learning rate to 10% of the initial value, and moderately increase the optimizer's momentum parameter. ).

[0081] Improved generalized advantage estimation: Adaptive Parameters: Dynamically adjusted based on the standard deviation of the current batch advantage estimate:

[0082] in The target standard deviation is set to 0.5.

[0083] Hierarchical advantage normalization: Nodes are divided into two layers according to priority: high priority (priority 1) and normal priority (priority 2-3). Advantage normalization is performed independently within each layer (subtract the mean and divide by the standard deviation).

[0084] Vectorized GAE: Expands recursive computation into matrix operations by constructing a lower triangular weight matrix. (element Dominance estimation vector From TD error vector Multiplying by the weight matrix yields: .

[0085] PPO Update: Sample small batches of data (batch size 64) from the experience replay buffer and perform multiple rounds of PPO updates (10 iterations per round). A pruning mechanism is used to limit the update magnitude (pruning factor). The value loss is calculated using mean squared error, with an entropy reward (coefficient 0.01) added to encourage exploration. The total loss function is:

[0086] in The ratio of new to old strategies, S represents the loss of value, and S represents the strategy of incitement.

[0087] Step 4: Closed-loop application and network topology maintenance.

[0088] After successful initial network formation, each node performs conflict-free data communication in fixed time slots according to the time slot allocation scheme generated by the PPO algorithm. During node communication and movement, if a new node joins the network or an existing node leaves the two-hop cluster range, the node remains silent and listens, re-booking a time slot during the competition for network entry (isolated node competition phase). Simultaneously, the system rebroadcasts control information at regular intervals (e.g., every 100 frames), triggering a global topology update. This involves repeating steps two through four, forming an intelligent closed loop of "perception-prediction-decision-feedback," ensuring continuous optimization and reliable operation of network topology and resource allocation.

[0089] Through the above steps, this embodiment provides high-precision prediction and intelligent dynamic allocation of communication data link time slot resources, which can adapt to complex communication environments with high dynamics, strong interference, and multiple constraints, significantly improve network throughput, reduce latency and packet loss rate, and meet the urgent need for accurate, real-time, and reliable management and control of communication resources in large-scale complex communication scenarios.

[0090] like Figure 7 As shown, this embodiment also provides a dynamic time slot allocation system based on prediction and reinforcement learning, including: a data acquisition module, used to acquire a historical time slot demand feature sequence based on a simulated communication dataset; The time slot prediction module is used to perform multi-step prediction of future time slot demand based on the historical time slot demand feature sequence, using a pre-built variational mode decomposition and deep learning hybrid model to obtain the predicted demand sequence. A construction module is used to model the time slot allocation problem as a Markov decision process; wherein, the state space of the Markov decision process includes the current allocation information and the predicted demand sequence, and the action space is the time slot allocation ratio corresponding to each communication node; The solution module is used to solve the Markov decision process using a proximal policy optimization algorithm with an integrated attention mechanism to obtain the time slot allocation action; The output module is used to transform the time slot allocation action into an actual time slot allocation scheme that satisfies network constraints, so as to complete the dynamic time slot allocation; wherein, the near-end policy optimization algorithm with integrated attention mechanism is updated according to the feedback reward; the reward is calculated based on the network performance after execution.

[0091] The present invention also provides a dynamic time slot allocation device based on prediction and reinforcement learning, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the dynamic time slot allocation method based on prediction and reinforcement learning.

[0092] The present invention also provides a computer program product, including a computer program / instruction that, when executed by a processor, implements the steps of the dynamic time slot allocation method based on prediction and reinforcement learning.

[0093] When the processor executes the computer program, it implements the above-mentioned dynamic time slot allocation steps based on prediction and reinforcement learning, for example: obtaining a historical time slot demand feature sequence based on a simulated communication dataset; Based on the historical time slot demand feature sequence, a pre-constructed variational mode decomposition and deep learning hybrid model is used to make multi-step predictions of future time slot demand, resulting in a predicted demand sequence. The time slot allocation problem is modeled as a Markov decision process; wherein, the state space of the Markov decision process includes the current allocation information and the predicted demand sequence, and the action space is the time slot allocation ratio corresponding to each communication node; A proximal policy optimization algorithm with integrated attention mechanism is used to solve the Markov decision process and obtain the time slot allocation action; The time slot allocation action is transformed into an actual time slot allocation scheme that satisfies network constraints to complete dynamic time slot allocation; wherein, the near-end policy optimization algorithm integrating the attention mechanism is updated according to the feedback reward; the reward is calculated based on the network performance after execution.

[0094] For example, the computer program can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing preset functions, the instruction segments describing the execution process of the computer program in the prediction and reinforcement learning-based dynamic time slot allocation device. For example, the computer program can be divided into a data acquisition module, a time slot prediction module, a construction module, a solution module, and an output module; the specific functions are as follows: the data acquisition module is used to acquire historical time slot demand feature sequences based on simulated communication datasets; the time slot prediction module is used to perform multi-step prediction of future time slot demands based on historical time slot demand feature sequences using a pre-built variational mode decomposition and deep learning hybrid model to obtain a predicted demand sequence; the construction module is used to model the time slot allocation problem as a Markov decision process; wherein, the state space of the Markov decision process includes current allocation information and the predicted demand sequence, and the action space is the time slot allocation ratio corresponding to each communication node; the solution module is used to solve the Markov decision process using a proximal policy optimization algorithm with an integrated attention mechanism to obtain time slot allocation actions; the output module is used to transform the time slot allocation actions into actual time slot allocation schemes that satisfy network constraints to complete dynamic time slot allocation; wherein, the proximal policy optimization algorithm with an integrated attention mechanism is updated according to the feedback reward; the reward is calculated based on the network performance after execution.

[0095] The dynamic time slot allocation device based on prediction and reinforcement learning can be a computing device such as a desktop computer, laptop, handheld computer, or cloud server. The dynamic time slot allocation device based on prediction and reinforcement learning may include, but is not limited to, processors and memory. Those skilled in the art will understand that the above examples of dynamic time slot allocation devices based on prediction and reinforcement learning do not constitute a limitation on such devices. The device may include more components than described above, or combine certain components, or use different components. For example, the dynamic time slot allocation device based on prediction and reinforcement learning may also include input / output devices, network access devices, buses, etc.

[0096] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or any conventional processor. This processor is the control center of the prediction and reinforcement learning-based dynamic time slot allocation system, connecting all parts of the system via various interfaces and lines.

[0097] The memory can be used to store the computer program and / or modules. The processor implements various functions of the prediction and reinforcement learning-based dynamic time slot allocation device by running or executing the computer program and / or modules stored in the memory and calling the data stored in the memory.

[0098] The memory may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function (such as sound playback, image playback, etc.). The data storage area may store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). Furthermore, the memory may include high-speed random access memory and non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart media cards (SMC), secure digital cards (SD cards), flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0099] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the dynamic time slot allocation method based on prediction and reinforcement learning.

[0100] If the modules / units integrated by the dynamic time slot allocation system based on prediction and reinforcement learning are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0101] Based on this understanding, the present invention can implement all or part of the processes in the above-described dynamic time slot allocation method based on prediction and reinforcement learning, or it can be accomplished by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of the above-described dynamic time slot allocation method based on prediction and reinforcement learning. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or a preset intermediate form, etc.

[0102] The computer-readable storage medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc.

[0103] It should be noted that the content contained in the computer-readable storage medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.

[0104] To implement the dynamic time slot allocation method provided in the above embodiments, such as Figure 5 As shown, this embodiment also provides an architecture for a dynamic time slot allocation system based on prediction and reinforcement learning. The specific structure and execution flow are as follows: Dataset Construction Module: This module executes the method described in step one to construct a communication simulation dataset that closely resembles a real, complex communication environment and possesses high dynamics and complexity. This module incorporates a heterogeneous node modeling unit, an environment parameter configuration unit, a service generation unit, and an event triggering unit. Through a modular simulation process, it generates a standardized CSV dataset containing 15-dimensional features (spatiotemporal identifier, node attributes, service features, and environment parameters). The output of the Dataset Construction Module is used for the subsequent training and validation of the prediction model.

[0105] Time slot demand forecasting module: Used to execute the method described in step two, with a built-in VMD-TCN-Transformer hybrid forecasting model. This module includes: Variational Mode Decomposition Unit: Adaptively decomposes the historical time-slot demand sequence and outputs K intrinsic mode function components and residual trend terms; The improved TCN feature extraction unit consists of a gated dilated causal convolutional layer, a multi-scale skip connection aggregation layer, and a dual attention mechanism to extract multi-scale local temporal features. Improved Transformer coding unit: integrates temporally enhanced positional coding, local-global hybrid attention mechanism, TCN-Transformer deep fusion strategy and channel-temporal dual-path attention to capture long-term dependencies; Multi-task output unit: Sets an independent output branch for each node type, adopts an adaptive multi-task loss weighting mechanism, and synchronously outputs the time slot demand prediction values ​​of all node types for the next M time frames.

[0106] This module takes historical multidimensional feature sequences as input and outputs high-precision time slot demand prediction results.

[0107] Time slot intelligent allocation module: used to execute the method described in step three, with a built-in improved PPO reinforcement learning agent. This module includes: State construction unit: Based on real-time dynamic environment perception data and the output of the time slot demand prediction module, construct a 7N-dimensional state vector with a two-layer structure of nodes and features; Action generation and constraint processing unit: Based on a policy network enhanced by multi-head attention mechanism, it generates continuous action vectors (time slot allocation ratio), and converts continuous actions into actual executable integer time slot allocation schemes through a multi-level constraint processing mechanism (minimum and maximum pruning, integer mapping, total amount constraint adjustment algorithm). Reward Calculation Unit: Calculates immediate rewards based on a multi-objective reward function (throughput, latency, fairness, priority guarantee, stability) and weights determined by the analytic hierarchy process. Training optimization unit: Integrates curriculum learning mechanism, adaptive learning rate scheduling, improved generalized advantage estimation (adaptive λ parameter, hierarchical advantage normalization, vectorized GAE) and PPO pruning update to achieve stable and efficient policy optimization.

[0108] This module takes a state vector as input and outputs an optimized time slot allocation scheme.

[0109] Network communication and execution module: This module executes time slot allocation according to the scheme generated by the intelligent time slot allocation module, and performs conflict-free data communication in the corresponding time slots. Simultaneously, this module monitors network status changes in real time (such as new node joining the network, node leaving the network, channel quality fluctuations, etc.) and feeds back the monitoring data to the time slot demand prediction module and the intelligent time slot allocation module, providing a basis for subsequent decision-making.

[0110] Network topology maintenance module: This module initiates the network topology update process periodically (e.g., every 100 time frames) or when triggered by an event. It calls the time slot demand prediction module and the time slot intelligent allocation module to re-predict demand and reallocate time slots, forming an intelligent closed loop of "perception-prediction-decision-feedback" to ensure continuous optimization and reliable operation of network topology and resource allocation.

[0111] The above modules are interconnected. The dataset construction module provides offline training data for the time slot demand prediction module; the time slot demand prediction module inputs the real-time prediction results into the time slot intelligent allocation module; the allocation scheme generated by the time slot intelligent allocation module is executed by the network communication and execution module; the network status monitored by the network communication and execution module is fed back to the time slot demand prediction module and the time slot intelligent allocation module for online adjustment or triggering retraining; the network topology maintenance module triggers the reinitialization and update of the entire system according to a preset period or event.

[0112] The workflow of the intelligent time slot allocation system provided in this embodiment will be described in detail below, taking a specific typical communication scenario as an example.

[0113] The specific example is as follows: Suppose a multi-node collaborative task includes four types of nodes: a ground control center (1), two airborne surveillance platforms, eight airborne mission platforms, and two surface mobile platforms, forming a data link network. During system initialization, the dataset construction module generates a simulation dataset based on historical operational data, which includes the service requirements, environmental parameters, and node state changes of the aforementioned nodes over 1280 consecutive time frames.

[0114] Step 1: Dataset Construction The dataset construction module generates a dataset containing 15-dimensional features based on the simulation process in step one. Taking an air mission platform node as an example, the number of its service messages follows a Poisson distribution (λ=5), and the time slot requirements, after channel quality adjustment and environmental factor correction, form a sequence that changes over time. Some sample records in the dataset are shown in Table 4; Table 4 shows an example of a simulation dataset (airborne mission platform).

[0115] The dataset construction module uses 80% of the data as the training set and 20% as the validation set for use by the subsequent prediction module.

[0116] Step 2: Time Slot Demand Forecasting The time slot demand prediction module loads the pre-trained VMD-TCN-Transformer model. Taking the air mission platform node as an example, the model input is a historical feature sequence of the past 24 time frames (including time slot demand, number of service messages, weather conditions, health status, etc.). First, the variational mode decomposition unit decomposes the time slot demand sequence of the air mission platform into 7 IMF components, such as... Figure 2As shown. Then, the improved TCN unit extracts multi-scale local features, and the improved Transformer unit captures long-term dependencies. Finally, the multi-task output unit synchronously outputs the predicted time slot demand values ​​for the next 8 time frames (corresponding to M=8). For the current time t, the model outputs the predicted demand of the air mission platform in future time slots: [7, 9, 8, 10, 12, 11, 9, 8]. Similarly, the prediction results for other node types are also generated synchronously.

[0117] Step 3: Intelligent Time Slot Allocation The time slot intelligent allocation module receives the above prediction results and real-time network environment data. At current time t, the system state vector... The structure is as follows: Ground Control Center: Current allocation ratio 0.15, predicted demand ratio 0.10, relay capacity 0.8, communication capacity 1.0, priority 1, historical utilization rate 0.65, communication quality 0.9; Airborne relay platform: Current allocation ratio 0.20, predicted demand ratio 0.18, relay capacity 0.9, communication capacity 0.9, priority 1, historical utilization rate 0.70, communication quality 0.8; Airborne mission platform: Current allocation ratio 0.40, predicted demand ratio 0.45, relay capacity 0.5, communication capacity 0.7, priority 2, historical utilization rate 0.75, communication quality 0.7; Surface mobile platform: Current allocation ratio 0.25, predicted demand ratio 0.27, relay capacity 0.6, communication capacity 0.6, priority 2, historical utilization rate 0.60, communication quality 0.8.

[0118] The normalized state vector is then input into the improved PPO policy network.

[0119] The multi-head attention layer of the policy network interacts with the features of the above four types of nodes to generate the mean vector of the action distribution. (summing to 1) and logarithmic standard deviation vector Sampling yields continuous motion. .

[0120] The action generation and constraint processing unit executes multi-level constraints: Minimum and maximum pruning: The minimum demand ratios for each node are [0.08, 0.15, 0.35, 0.20], and the maximum capacity ratios are [0.20, 0.25, 0.50, 0.30]. After pruning... Remain unchanged (all within the range).

[0121] Integer mapping: Assuming a total number of time slots T = 256, the initial allocation... The total is 33 + 51 + 108 + 64 = 256, which is exactly equal to T, so no total adjustment is needed.

[0122] The final allocation scheme is as follows: 33 time slots for the ground control center, 51 time slots for the air relay platform, 108 time slots for the air mission platform, and 64 time slots for the waterborne mobile platform.

[0123] Step 4: Network Communication and Execution The network communication and execution module distributes the above allocation scheme to each node, and each node sends and receives data in its corresponding fixed time slot. Simultaneously, this module monitors network performance in real time. For example, in a subsequent time slot, the air mission platform actually generates 10 service messages, allocates 108 time slots, and predicts a demand of 12. Therefore, the satisfaction rate is 108 / 12 = 9.0 (the actual number of time slots should be divided by the service volume). It's important to clarify here that the time slot demand prediction is the required number of time slots. Allocating 108 time slots and demanding 12 indicates that the allocation is far greater than the demand, potentially leading to waste. However, in reality, time slot demand is the required number of time slots, and the allocation is also the number of time slots; the two should be comparable. The data needs to be adjusted to make it reasonable. Assuming a predicted demand of 12, allocating 108 is clearly unreasonable, and we need to adjust the data. In simulations, the predicted time slot demand is usually a single digit, and the allocated time slots should also be single digits. Therefore, the total number of time slots in the above integer mapping should be a smaller value. For simplicity, we assume a total of 64 time slots, which would be allocated as [8, 13, 27, 16], totaling 64. Demand forecasts are assumed to be [7, 12, 25, 15]. Throughput bonuses, etc., can then be calculated. To maintain consistency, we reset: Total number of time slots T=64.

[0124] Preliminary allocation: The total is 8 + 13 + 27 + 16 = 64.

[0125] Demand forecast: 7 ground control centers, 12 air relay platforms, 25 air mission platforms, and 15 waterborne mobile platforms.

[0126] The satisfaction rates are: 8 / 7≈1.14, 13 / 12≈1.08, 27 / 25≈1.08, 16 / 15≈1.07, all slightly higher than the demand, reflecting the efficient use of resources.

[0127] Communication quality factor All are set to 1.0. The throughput reward is then calculated as a weighted average of the satisfaction levels of each node (multiplied by the priority weight). Latency penalties (0 for unsatisfied portions) and a fairness index are also calculated.

[0128] The reward function module calculates the immediate reward in real time and outputs the state transition quadruple. Store it in the experience replay buffer for subsequent training and optimization.

[0129] Step 5: Network Topology Maintenance After 100 time frames of system operation, the network topology maintenance module triggers a periodic update. By this time, the network environment may have changed (e.g., some nodes have failed, or environmental interference has increased). This module restarts the time slot demand prediction module and the time slot intelligent allocation module, performing a new round of prediction and allocation based on the latest sensing data to achieve adaptive adjustment.

[0130] Through the collaborative work of the above modules, the intelligent time slot allocation system provided in this embodiment can achieve high-precision prediction and dynamic optimization allocation of data link network time slot resources, significantly improve network throughput in complex dynamic environments, reduce latency and packet loss rate, and meet the needs of large-scale, highly dynamic networking scenarios for accurate, real-time, and reliable management and control of communication resources.

[0131] Compared with existing time slot allocation methods, this invention provides a dynamic time slot allocation method based on prediction and reinforcement learning, which has the following advantages: This invention constructs a communication simulation dataset that closely resembles a real, complex, and dynamic environment. Based on variational mode decomposition and an improved TCN-Transformer hybrid model, it performs multi-step predictions of future time slot service demands. Using the predicted information as key input, it leverages an improved near-end policy optimization reinforcement learning algorithm, enhanced network architecture through multi-head attention, multi-level constraint processing, and a multi-objective reward function weighted by the analytic hierarchy process (AHP), to generate an optimized dynamic time slot allocation scheme. Data communication is then performed according to the allocation scheme, and network topology maintenance and model updates are conducted periodically. The system includes a dataset construction module, a time slot demand prediction module, a time slot intelligent allocation module, a network communication and execution module, and a network topology maintenance module. Through a collaborative optimization framework of "accurate prediction first, then intelligent decision-making," this invention achieves high-precision prediction and dynamic optimization allocation of data link time slot resources, significantly improving network throughput in highly dynamic and interference-prone environments, reducing latency and packet loss rates, and meeting the needs of large-scale dynamic networking scenarios for accurate, real-time, and reliable management of communication resources.

[0132] The above embodiments are merely one of the implementation methods for achieving the technical solution of the present invention. The scope of protection claimed by the present invention is not limited to this embodiment, but also includes any variations, substitutions and other implementation methods that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the present invention.

Claims

1. A dynamic time slot allocation method based on prediction and reinforcement learning, characterized in that, include: Historical time slot demand characteristic sequences were obtained based on simulated communication datasets. Based on the historical time slot demand feature sequence, a pre-constructed variational mode decomposition and deep learning hybrid model is used to make multi-step predictions of future time slot demand, resulting in a predicted demand sequence. The time slot allocation problem is modeled as a Markov decision process; wherein, the state space of the Markov decision process includes the current allocation information and the predicted demand sequence, and the action space is the time slot allocation ratio corresponding to each communication node; A proximal policy optimization algorithm with integrated attention mechanism is used to solve the Markov decision process and obtain the time slot allocation action; The time slot allocation action is transformed into an actual time slot allocation scheme that satisfies network constraints to complete dynamic time slot allocation; wherein, the near-end policy optimization algorithm integrating the attention mechanism is updated according to the feedback reward; the reward is calculated based on the network performance after execution.

2. The dynamic time slot allocation method based on prediction and reinforcement learning according to claim 1, characterized in that, The method, based on historical time slot demand feature sequences, employs a pre-constructed hybrid model of variational mode decomposition and deep learning to perform multi-step predictions of future time slot demands, resulting in a predicted demand sequence, including: The variational mode decomposition method is used to decompose the historical time slot demand characteristic sequence of each node into multiple intrinsic mode function components; The original features are combined with the intrinsic mode function components, and the combined features are input into a deep learning model to obtain intermediate output results; the deep learning model adopts an improved temporal convolutional network. The intermediate output results are fed into the improved Transformer encoder module, which synchronously outputs the prediction requirement sequence for all node types through a multi-task learning architecture.

3. The dynamic time slot allocation method based on prediction and reinforcement learning according to claim 2, characterized in that, The improved temporal convolutional network integrates: Gated dilated causal convolution mechanism is used to achieve adaptive adjustment of input features; A multi-scale skip connection aggregation mechanism is used to fuse local temporal features from different receptive fields; A dual attention mechanism is used to recalibrate the importance of the feature map in both the channel and temporal dimensions. The improved Transformer encoder module includes: Temporally enhanced location coding is used to fuse absolute time, periodicity, and trend information; A local-global hybrid attention architecture, comprising local causal convolutional layers for capturing recent associations and sparse global attention layers for modeling long-term dependencies; The improved temporal convolutional network and the improved Transformer encoder module are deeply integrated through feature injection and knowledge distillation loss.

4. The dynamic time slot allocation method based on prediction and reinforcement learning according to claim 1, characterized in that, The state vector of the Markov decision process adopts a node-feature two-layer structure. For a network containing N types of nodes, the state vector... for: in, For node type The 7-dimensional feature vector includes at least: the current time slot allocation ratio, the future demand ratio based on the predicted demand sequence, the node relay capability index, the node communication capability index, the operational priority, the historical network average utilization rate, and the dynamic communication quality factor.

5. The dynamic time slot allocation method based on prediction and reinforcement learning according to claim 1, characterized in that, The policy network and value network of the proximal policy optimization algorithm with integrated attention mechanism both adopt an architecture that includes a multi-head attention mechanism. The training process of the proximal policy optimization algorithm adopts a course learning mechanism, and the training environment parameters gradually transition from simple static scenarios to complex dynamic scenarios including high interference, high mobility and burst traffic.

6. The dynamic time slot allocation method based on prediction and reinforcement learning according to claim 1, characterized in that, The reward function of the Markov decision process Weighted sum for multiple objectives Specifically as follows: in, These represent the normalized throughput reward, latency penalty, fairness reward, priority guarantee reward, and stability penalty, respectively. , , , , These are the corresponding weight coefficients, all determined using the analytic hierarchy process (AHP).

7. The dynamic time slot allocation method based on prediction and reinforcement learning according to claim 1, characterized in that, The process of transforming the time slot allocation action into an actual time slot allocation scheme that satisfies network constraints to complete dynamic time slot allocation includes: Perform node-level minimum / maximum ratio pruning on time slot allocation actions; The ratio is mapped to an integer number of time slots to generate a preliminary allocation scheme; If the total number of time slots in the initial allocation scheme exceeds the total number of time slots in the system, the weights are adjusted based on node priority, and iterative reduction is performed until the total system constraint is met. The actual time slot allocation scheme is then output, and dynamic time slot allocation is completed.

8. A dynamic time slot allocation system based on prediction and reinforcement learning, characterized in that, include: The data acquisition module is used to acquire historical time slot demand characteristic sequences based on simulated communication datasets. The time slot prediction module is used to perform multi-step prediction of future time slot demand based on the historical time slot demand feature sequence, using a pre-built variational mode decomposition and deep learning hybrid model to obtain the predicted demand sequence. A construction module is used to model the time slot allocation problem as a Markov decision process; wherein, the state space of the Markov decision process includes the current allocation information and the predicted demand sequence, and the action space is the time slot allocation ratio corresponding to each communication node; The solution module is used to solve the Markov decision process using a proximal policy optimization algorithm with an integrated attention mechanism to obtain the time slot allocation action; The output module is used to transform the time slot allocation action into an actual time slot allocation scheme that satisfies network constraints, so as to complete the dynamic time slot allocation; wherein, the near-end policy optimization algorithm with integrated attention mechanism is updated according to the feedback reward; the reward is calculated based on the network performance after execution.

9. A dynamic time slot allocation device based on prediction and reinforcement learning, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the dynamic time slot allocation method based on prediction and reinforcement learning as described in any one of claims 1-7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it is used to implement the dynamic time slot allocation method based on prediction and reinforcement learning as described in any one of claims 1-7.

Images