A method and system for resource scheduling based on LSTM and QoS perception in a multi-slice network

Abstract

This invention belongs to the field of wireless communication technology and discloses a wireless communication resource scheduling method and system based on LSTM and QoS demand awareness in multi-slice networks. This invention introduces Long Short-Term Memory (LSTM) networks and time-series prediction technology, and integrates an improved SAC reinforcement learning algorithm to construct a QoS-aware multi-slice resource scheduling framework. This framework accurately predicts users' future movement trajectories and slice service demands through LSTM, and dynamically allocates RBGs (Resource Blocks) based on the exploration-utilization balancing capability of the improved SAC algorithm, achieving coordinated optimization of multi-slice QoS demands and global resource utilization efficiency. Extensive simulation results show that the scheduling scheme combining LSTM and the improved SAC algorithm proposed in this invention outperforms traditional DDPG and PPO algorithms, demonstrating superior performance in latency control, throughput improvement, and packet loss rate optimization.

Description

Technical Field

[0001] This invention belongs to the field of wireless communication technology, and particularly relates to a wireless communication resource scheduling method and system based on LSTM and QoS demand awareness in a multi-slice network. Background Technology

[0002] With the mature deployment of 5G communication technology and the gradual advancement of 6G network research, the integration of communication networks with vertical industries such as the Industrial Internet, autonomous driving, and smart healthcare is deepening. The diverse business scenarios are exhibiting significantly differentiated demands for network service quality. Specifically, enhanced mobile broadband (eMBB) services focus on scenarios such as high-definition video transmission and virtual reality interaction, with high throughput and wide bandwidth support as core requirements; ultra-reliable low-latency communication (URLLC) services target critical scenarios such as autonomous vehicle collaboration and remote surgical control, imposing stringent requirements on communication latency and data transmission reliability; and elastic broadband (BE) services cover everyday applications such as web browsing and ordinary file transfer, aiming to flexibly adapt to network resource conditions and meet basic communication needs.

[0003] To address the diverse needs of these heterogeneous services, network slicing technology has emerged. This technology constructs multiple logically isolated virtual networks on the same physical communication infrastructure. Each virtual network can flexibly configure network resources, customize communication protocols, and service quality standards according to the needs of its corresponding service, achieving flexible resource allocation and customized service guarantees. It has become a core enabling technology supporting the efficient operation of various types of heterogeneous services. In actual network deployments, the application of network slicing technology effectively breaks the traditional "one-size-fits-all" service model, enabling different types of services to obtain dedicated service guarantees while sharing physical resources, laying a solid foundation for the deep integration of 5G / 6G networks with vertical industries.

[0004] However, in actual network operation, the contradiction between the dynamic network environment and limited wireless resources is becoming increasingly prominent, severely restricting the application effect of network slicing technology. On the one hand, the dynamic characteristics of the network environment are becoming more and more significant. The mobility of users within the network coverage area leads to frequent switching of access base stations, and service traffic exhibits dramatic dynamic fluctuations in time and space. In addition, the wireless channel itself has time-varying attenuation characteristics, making the network state constantly changing. Traditional static resource allocation strategies are based on fixed network scenario assumptions and cannot perceive the dynamic changes in network state in real time. This leads to a serious disconnect between resource allocation results and actual service needs and network carrying capacity, resulting in problems such as resource waste or deterioration of service quality. For example, during peak hours, eMBB service traffic surges, and statically allocated resources cannot be adjusted in time, causing some users to experience problems such as video stuttering and slow loading; while during off-peak hours, a large amount of resources are idle, resulting in low resource utilization.

[0005] On the other hand, the coexistence of differentiated QoS requirements across multiple slices exacerbates resource competition. Different types of slices have fundamentally different QoS requirements. For example, the low latency and high reliability requirements of URLLC slices conflict with the high throughput requirements of eMBB slices in terms of resource consumption, while the elastic requirements of BE slices need to be dynamically adjusted without affecting the service quality of high-priority slices. Existing scheduling methods often struggle to accurately balance the priority differences between different slices. While ensuring the core requirements of high-priority slices, they fail to achieve efficient utilization of global network resources, resulting in some slices failing to meet service quality standards or having low resource utilization. For instance, when URLLC slices and eMBB slices request resources simultaneously, existing scheduling methods may not prioritize the low latency requirements of URLLC slices, thus affecting the normal operation of critical services such as autonomous driving and telemedicine.

[0006] Deep reinforcement learning (DRL), as an important branch of artificial intelligence, offers a novel and effective solution to the problem of high-dimensional dynamic network resource scheduling by virtue of its core characteristic: the agent autonomously learns and optimizes decision-making strategies through continuous interaction with the environment. Unlike traditional optimization algorithms, DRL does not rely on precise mathematical models of the environment. It can automatically capture the evolutionary patterns of network states through end-to-end learning, quickly finding near-optimal decision-making solutions in complex scenarios with multiple constraints and high dynamics. Therefore, it demonstrates enormous application potential in the field of network resource optimization.

[0007] In the field of Radio Access Network (RAN) slice scheduling, various deep reinforcement learning algorithms, such as Soft Actor-Critic (SAC), Deep Deterministic Policy Gradient (DDPG), and Proximity Policy Optimization (PPO), have been attempted to be applied to resource scheduling strategy design, and have initially demonstrated performance superior to traditional algorithms. For example, some studies have optimized slice resource allocation using the DDPG algorithm, improving resource utilization to some extent; other studies have designed scheduling strategies based on the PPO algorithm, improving the stability of slice service quality. However, existing research on network slice resource scheduling based on deep reinforcement learning still has many key limitations, making it difficult to meet the needs of actual network deployment.

[0008] First, existing research generally fails to adequately incorporate the critical impacts of user mobility. User movement directly triggers a series of chain reactions, including access base station switching, fluctuations in channel transmission conditions, and dynamic changes in service access locations. These changes directly affect slice resource demands and network service quality. Existing scheduling strategies often overlook this core dynamic factor, leading to a disconnect between the trained scheduling model and the actual network operation, resulting in insufficient practicality and robustness. In scenarios with high-speed user movement, existing scheduling strategies struggle to respond quickly to changes in user location, causing significant fluctuations in slice service quality.

[0009] Secondly, the reward mechanism design lacks deep integration of slice QoS requirements and priorities. The reward function is the core guiding principle of deep reinforcement learning algorithms, directly determining the agent's learning direction. Existing reward mechanisms mostly adopt simple performance index weighting methods, failing to fully reflect the priority differences and core QoS requirements of different slices. This makes it difficult to guide the algorithm to prioritize the service quality of critical slices such as URLLC, resulting in the core requirements of high-priority slices not being effectively met. For example, the reward function simply weights and sums indicators such as latency and throughput without setting a higher weight coefficient for the latency constraints of URLLC slices, preventing the agent from fully valuing the core requirements of high-priority slices during the learning process.

[0010] Third, the unique characteristics of network slice resource management further increase the complexity of the problem. Network slice resource management needs to simultaneously consider the overall optimization of global network resources and the customized service requirements of individual slices, creating an inherent balance contradiction. Furthermore, in dynamic network environments, sudden fluctuations in service traffic and dynamic additions or removals of slices can cause QoS requirements of slices to drift. Existing research largely fails to consider this dynamic change, resulting in insufficient adaptability of scheduling strategies to complex dynamic scenarios and difficulty in continuously ensuring the stable service quality of each slice. For example, when a large number of BE slice users suddenly appear in a certain area, existing scheduling strategies may not be able to quickly adjust resource allocation schemes, leading to a decrease in throughput for eMBB slice users in that area.

[0011] Furthermore, existing research largely focuses on resource scheduling optimization under single static scenarios, neglecting the dynamic switching characteristics of multiple scenarios in real-world networks. During actual network operation, network scenarios dynamically switch due to changes in time, location, and service type. The slice resource requirements, network topology, and channel conditions vary across different scenarios. Models trained using existing scheduling strategies under a single scenario often experience significant performance degradation after scenario switching, making them ill-suited to the complex and ever-changing real-world network environment.

[0012] These limitations collectively highlight the urgency and necessity of developing high-performance deep reinforcement learning scheduling strategies adapted to RAN slicing scenarios. There is an urgent need to design a resource scheduling scheme that fully considers user mobility, deep fusion of slice priorities and QoS requirements, and possesses strong dynamic adaptability. This scheme should achieve an optimal balance between observation space design, reward mechanism optimization, and training stability improvement, thereby effectively addressing issues such as the disconnect between resource scheduling and actual network conditions, insufficient guarantee of high-priority slice requirements, and poor adaptability to dynamic scenarios in existing technologies. This will provide reliable technical support for the deep integration of 5G / 6G networks with vertical industries.

[0013] Based on the above analysis, the problems and shortcomings of the existing technology are as follows:

[0014] (1) Existing studies generally do not adequately incorporate the key impacts of user mobility.

[0015] (2) The reward mechanism design lacks deep integration of slice QoS requirements and priorities.

[0016] (3) The special nature of network slice resource management further increases the complexity of the problem.

[0017] (4) Existing research focuses on resource scheduling optimization in a single static scenario, neglecting the dynamic switching characteristics of multiple scenarios in real networks. In actual network operation, network scenarios will dynamically switch with changes in time, location, service type, etc., and the slice resource requirements, network topology, channel conditions, etc., will vary in different scenarios. Existing scheduling strategies trained in a single scenario often experience a significant performance drop after scenario switching, making it difficult to adapt to the complex and ever-changing real network environment. Summary of the Invention

[0018] To address the problems existing in the prior art, this invention provides a wireless communication resource scheduling method and system based on LSTM and QoS demand awareness in multi-slice networks.

[0019] This invention is implemented as follows: A wireless communication resource scheduling method based on LSTM and QoS demand awareness in a multi-slice network includes:

[0020] Step 1: Model the multi-base station cellular network scenario, user mobility model, multi-type network slice (eMBB, URLLC, BE) service model and channel transmission model.

[0021] Step 2: Input the user's historical location and slice requirements from multiple time slots into the LSTM model to obtain the user's predicted location and slice requirements for the next time slot; integrate the prediction results with the current base station resource status, channel quality, and real-time slice requirements to transform the optimization problem into a Markov decision problem, complete the modeling of the state space and action space, and design a differentiated reward function with priority weights for the differentiated QoS requirements of slices.

[0022] Step 3: Define the optimization objectives. Define the core objectives of maximizing the QoS compliance rate of multi-slice network and the utilization rate of global network resources. At the same time, set constraints such as total resource constraints, slice priority constraints, and hard constraints on URLLC latency and reliability.

[0023] Step 4: Construct a QoS-aware resource scheduling model based on the improved SAC algorithm and train the model; compare the trained scheduling model with benchmark algorithms such as PF scheduling, DDPG, and PPO in static scenarios, user mobility scenarios, and traffic burst scenarios to verify its superior performance.

[0024] Furthermore, step 1 specifically includes:

[0025] (1): The number and initial location of base stations deployed in the multi-slice network scenario and users in each slice are known. The LSTM prediction module and the improved SAC scheduling module work together. The input window length and hidden layer dimension of the LSTM module are pre-configured. During the scheduling process, the historical location of users and slice demand data are continuously collected and the prediction results are output.

[0026] (2): Define the channel model of the wireless network; base station and users between Clusters and The received power of the reference signal for each ray can be expressed as:

[0027]

[0028] in For path loss, For the shadow to fade, Corresponding LOS path coefficient, For the first Cluster, the first NLOS path coefficient of a ray For base stations The transmit power and spectral efficiency of this cluster are given by the following formula:

[0029]

[0030] in, For noise power, Inter-cell interference can be represented as:

[0031]

[0032] (3): Define the multi-slice model in the network: Each slice contains a group of user equipment (UEs) with similar service characteristics and consistent QoS requirements, and define the resource allocation vector:

[0033]

[0034] in, Indicates the first Step to slice The number of RBGs allocated, The total number of slices; resource allocation must meet the following constraints:

[0035]

[0036] This represents the total number of RBGs currently available, based on the number of RBGs available in the system. It is a fixed value. The RBG combination methods for slice resource allocation are limited. Define a resource combination vector:

[0037]

[0038] in, This represents one of the possible combinations of RBG allocations. This represents the total number of all possible RBG combinations in this step;

[0039] (4): Define the user QoS performance indicator model: user In the Step throughput The calculation is as follows:

[0040]

[0041] in It is the first Step by slice scheduling, from slice Assigned to user The number of RBGs, User In the The spectral efficiency of the step is given by R, where R is the total number of available RGBGs in the system, and B is the total bandwidth of the system. It refers to the data packet size; effective throughput. This refers to the UE in the first... The actual throughput transmitted over the network is defined as:

[0042]

[0043] in yes In the The theoretical maximum throughput of the step User In the The amount of cached data available for each step; simultaneously, define the cache occupancy rate. This reflects the load status of the user's cache:

[0044]

[0045] in This refers to the maximum capacity of the user cache; packet loss rate. Is the UE in length of The percentage of packets dropped within the sliding window is defined as:

[0046]

[0047] in User In the The request throughput per step For users In the The amount of data discarded per step; average cache latency This refers to the average waiting time of data packets in the UE buffer, defined as:

[0048]

[0049] in User In the The cache latency vector of the step. Indicates the first Step, cache wait latency is The number of packets per TTI; This is the maximum allowable cache latency; long-term average throughput. and quintile throughput These are the UEs with a length of The average service throughput and the quintile of throughput within the sliding window are calculated as follows:

[0050]

[0051]

[0052] in This indicates that the throughput data within the window is calculated using the quintiles.

[0053] Furthermore, step 2 specifically includes:

[0054] (1): Construct the state space in the time slot In this context, the state space of the Agent can be represented as:

[0055]

[0056] in This represents the time-series data of user location over the past 10 time slots. This represents the time-series data of service requests for each slice over the past 10 time slots. This represents the real-time QoS metrics for each slice. This indicates the RBG allocation status of each slice in the current time slot. This indicates the user's location at one future moment predicted by LSTM;

[0057] (2): Constructing the action space, the agent's action space can be represented as:

[0058]

[0059] in, Assign action factors to the three types of slices; This indicates that the reserved action factors are based on predictions, and the values ​​of all action factors range from [−1, 1]. The continuous action values ​​output by the Agent are matched to feasible RBG assignment sets through Euclidean distance. ,Right now:

[0060]

[0061] in Represents the set of feasible combinations. Candidate Combinations and Agent Output Euclidean distance; Allocate a predefined set of legal resources to reserve a combination of resources;

[0062] (3): Design the reward function. The immediate reward that the Agent can obtain can be expressed as:

[0063]

[0064] in, The penalty weight for wasting resources; Describe the degree of idleness of reserved resources; For slices The QoS requirement drift reward needs to be designed with differentiated QoS metrics for different slices:

[0065] The QoS requirement drift reward for eMBB slices is based on the satisfaction of throughput, latency, and packet loss rate, and is defined as follows:

[0066]

[0067] The sub-items are as follows:

[0068] Throughput Bonus:

[0069]

[0070] Delay bonus:

[0071]

[0072] Packet loss rate bonus:

[0073]

[0074] in, ;

[0075] The QoS requirement drift reward for URLLC slices takes into account that URLLC focuses on low latency and high reliability. The reward function is consistent with that of eMBB, but uses higher weights to reflect priority.

[0076]

[0077] The packet loss rate metric must meet strict constraints. ;

[0078] QoS requirement drift reward for BE slices; BE slices focus on long-term throughput and worst-case performance, and the reward function is designed based on long-term throughput and quintile throughput.

[0079]

[0080] The sub-items are as follows:

[0081] Long-term throughput rewards:

[0082]

[0083] 5th percentile throughput bonus:

[0084]

[0085] In each time slot t, the agent observes the state. Output action To complete resource allocation and reservation; the environment calculates rewards based on the network state after the action is executed. and transition to the state of the next time slot. .

[0086] Furthermore, step 3 specifically includes:

[0087] (1): The joint optimization problem is defined as realizing dynamic radio resource allocation of multiple network slices to maximize the long-term satisfaction of the QoS requirements of each slice. Therefore, the joint optimization problem is:

[0088]

[0089] (2): Determine the constraints;

[0090] The constraints are:

[0091]

[0092] constraint Ensure that the total amount of RBG resources does not exceed the total available resources of the system; constraints Limit the resource reservation cap for high-priority slices; constraints Limit user movement characteristics: speed not exceeding 10m / s, and probability of direction change not exceeding 0.2 / s.

[0093] Furthermore, step 4 specifically includes: constructing and solving a multi-slice resource scheduling algorithm based on LSTM prediction and QoS awareness algorithms.

[0094] (1.1) Initialize LSTM prediction network parameters Initialize the parameters of the Actor network, Critic network, target Actor network, and target Critic network. ;

[0095] (1.2) Initialize the experience playback buffer Batch size Target network update coefficients Entropy weight coefficient ;

[0096] (1.3) for each round do

[0097] (1.3.1) Reset base station resource status, user initial location and slice requirements

[0098] (1.3.2) while t ≤ T do

[0099] (1.3.2.1) Obtain the user's historical location and slice requirements for the past 10 time slots.

[0100] (1.3.2.2) Input the LSTM network and output the predicted user location and slice requirements for the next time slot.

[0101] (1.3.2.3) Combine the prediction results with the current base station resources and channel quality to construct the global environment status.

[0102] (1.3.2.4) Actor network output actions This includes the current RBG allocation and predicted resource reservation, and executes actions to update the resource allocation status.

[0103] (1.3.2.5) Obtain instant differentiated rewards Change the environment state to

[0104] (1.3.2.6) Sequence information ( , , , Stored in the experience replay buffer From which random samples are taken. Small batch samples

[0105] (1.3.2.7) Calculate the target Q value Update the current Critic network weights using mean squared error loss.

[0106] (1.3.2.8) Update the current Actor network weights using policy gradient descent.

[0107] (1.3.2.9) Update the weights of the target Critic and Actor networks using a soft update method.

[0108] (1.3.2.10) Update the LSTM network parameters with the error between the predicted and actual positions.

[0109] (1.3.3) End the while loop

[0110] (1.4) End the for loop.

[0111] Another objective of this invention is to provide a wireless communication resource scheduling system based on LSTM and QoS demand awareness in a sliced ​​network, comprising:

[0112] The modeling module is used to model multi-slice network scenarios, user mobility models, and slice service requirement models.

[0113] The target construction module is used to construct optimization targets, clarifying the goals and constraints for maximizing the QoS guarantee level of multi-slice services and the global resource utilization rate.

[0114] The function design module is used to transform problems into Markov decision problems based on scenarios and constraints, model the state space and action space, and design differentiated reward functions for optimization objectives.

[0115] The model training module is used to build a QoS-aware multi-slice resource scheduling algorithm based on LSTM prediction and combined with the improved SAC algorithm, train the model, and compare the performance of the trained model with the benchmark algorithm under different user mobility scenarios to verify its performance.

[0116] Another object of the present invention is to provide a computer device including a memory and a processor, the memory storing a computer program, which, when executed by the processor, causes the processor to perform the steps of a wireless communication resource scheduling method based on LSTM and QoS demand awareness in a multi-slice network.

[0117] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of a wireless communication resource scheduling method based on LSTM and QoS demand awareness in a multi-slice network.

[0118] Another objective of this invention is to provide an information data processing terminal for implementing a wireless communication resource scheduling system based on LSTM and QoS demand awareness in the multi-slice network.

[0119] Based on the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solution to be protected by this invention are as follows:

[0120] The core idea of ​​this invention is as follows: First, a cellular network system model containing multiple base stations and multiple types of slice users is constructed, and a user mobility model is integrated to accurately characterize the changes in user movement speed and direction. The location of users in future time slots and slice requirements are predicted by an LSTM time series prediction model. The prediction results are then fused with information such as base station resource status, channel quality, and slice QoS requirements to construct a comprehensive Markov decision process (MDP) observation space.

[0121] Based on the above observation space, a differentiated reward mechanism is designed, which is deeply integrated with slice priority and core QoS requirements: hard constraint reward weights are set for latency and reliability constraints of URLLC slices, optimization reward coefficients are set for throughput indicators of eMBB slices, and dynamic penalty items are set for resource elasticity adaptation requirements of BE slices, guiding the algorithm to prioritize the service quality of high-priority slices.

[0122] An improved deep reinforcement learning (SAC) algorithm is used to train the agent, integrating slice QoS requirement parsing, user mobility prediction, resource status awareness, and dynamic scheduling decision-making. Through continuous interaction between the agent and the network environment, it autonomously learns the optimal resource allocation strategy, enabling dynamic adjustment of multi-slice resources. Furthermore, an adaptive adjustment mechanism is introduced to address scenarios such as dynamic slice additions / reductions and sudden traffic fluctuations, improving the robustness of the scheduling strategy.

[0123] This invention effectively solves problems such as the disconnect between traditional scheduling strategies and actual network conditions, resource contention conflicts, and insufficient adaptability to dynamic scenarios by adopting an integrated framework of "QoS awareness-mobility prediction-intelligent decision-making". While ensuring the low latency and high reliability of URLLC slices, the high throughput of eMBB slices, and the elastic adaptation of BE slices, it significantly improves network resource utilization and overall service quality stability.

[0124] In 5G / 6G multi-slice networks, the conflict between the differentiated QoS requirements of heterogeneous services such as eMBB, URLLC, and BE (e.g., low latency and high reliability of URLLC, high throughput of eMBB) and dynamic scenarios such as user mobility and traffic bursts makes it difficult for traditional static scheduling to balance "high priority guarantee" and "resource efficiency". This invention discloses a multi-slice resource dynamic scheduling method based on QoS awareness and user mobility prediction. By predicting the future location and slice requirements of users through LSTM and combining it with an improved SAC algorithm, it realizes dynamic allocation and reservation of resources. This not only reduces the modeling complexity in complex scenarios, but also provides a more accurate resource scheduling solution for multi-slice networks, which is suitable for communication scenarios integrating multiple industries such as industrial internet, autonomous driving, and smart healthcare.

[0125] This invention addresses the issues of poor dynamic adaptability and insufficient high-priority guarantees in multi-slice resource scheduling by integrating LSTM prediction with a QoS-aware algorithm. First, it constructs a network model that incorporates user mobility prediction, transforming the optimization problem into a Markov decision process. Second, it designs a differentiated reward mechanism that deeply binds slice QoS requirements and priorities. Finally, it improves the QoS-aware algorithm to achieve autonomous policy learning. Compared to traditional algorithms, this invention can simultaneously meet the core requirements of multiple slices in complex dynamic scenarios, significantly improving service quality and resource efficiency.

[0126] In designing the reward function, this invention fully considers slice priority and core QoS constraints: high-weight rewards are set for latency and reliability of URLLC, optimization rewards are set for throughput of eMBB, and dynamic penalties are set for resource elasticity adaptation of BE. This incentivizes agents to prioritize critical services while avoiding resource waste, ultimately achieving collaborative optimization of service quality and resource utilization across multiple slices.

[0127] This invention effectively improves the dynamic adaptability of multi-slice networks by integrating user mobility prediction and QoS-aware resource scheduling strategies. On the one hand, based on LSTM user location and demand prediction, resources are reserved in advance to avoid base station overload caused by sudden user gatherings. On the other hand, the combination of differentiated reward mechanisms and improved SAC algorithms ensures that the core QoS requirements of high-priority slices such as URLLC are given priority.

[0128] This invention addresses the core pain point of multi-slice network resource scheduling by providing an integrated intelligent scheduling solution with the following technical effects and advantages:

[0129] High-priority slice QoS guarantee: This invention transforms the latency and reliability constraints of URLLC into the core optimization objectives of the algorithm through a QoS awareness mechanism and differentiated reward design, significantly improving its latency compliance rate, which is far higher than that of traditional algorithms, and can fully meet the stringent requirements of key businesses such as autonomous driving and telemedicine.

[0130] Efficient utilization of global resources: By predicting user movement and dynamically adjusting resources, this invention significantly improves network resource utilization, showing a marked improvement compared to the PF scheduling algorithm. It effectively avoids resource idleness or local overload and achieves balanced allocation of global resources.

[0131] Strong adaptability to dynamic scenarios: For complex scenarios such as user movement, traffic surges, and dynamic addition or reduction of slices, the adaptive scheduling strategy of this invention can quickly adjust the resource allocation scheme. In dynamic scenarios such as high-speed user movement, the slice service quality fluctuates less and its stability is significantly better than that of the basic reinforcement learning algorithm.

[0132] Low implementation complexity: This invention adopts an end-to-end reinforcement learning framework, which does not rely on complex environmental mathematical models. Compared with traditional optimization algorithms, the computational complexity is greatly reduced, making it easier to deploy in engineering and promote practical applications.

[0133] (1) The expected benefits and commercial value of the technical solution of this invention after transformation are as follows:

[0134] The technical solution of this invention addresses the core pain point of resource scheduling in 5G / 6G multi-slice networks by providing an intelligent scheduling scheme based on QoS awareness and user mobility prediction. The expected benefits and commercial value of transforming this technical solution into a specific product are mainly reflected in the following aspects:

[0135] Improving network service quality: This invention can significantly improve the QoS compliance rate of slices such as URLLC and eMBB, helping operators provide more reliable and differentiated services to vertical industry customers and effectively enhance market competitiveness.

[0136] Reduce network operating costs: By improving resource utilization, operators can reduce hardware investment in base station expansion and reduce network energy consumption, which is expected to lead to a significant decrease in long-term operating costs.

[0137] Expanding industry application scenarios: The dynamic scheduling capability of this invention can support the implementation of scenarios with stringent communication quality requirements, such as autonomous driving and the industrial internet, helping operators expand into high-value industry markets and further increase business revenue.

[0138] In summary, the technical solution of this invention has significant expected benefits and commercial value after transformation, which helps to improve network service quality, reduce operating costs, and expand industry applications, and can bring considerable economic benefits and market competitiveness to telecommunications operators and related enterprises.

[0139] (2) The technical solution of this invention fills a technical gap in the industry both domestically and internationally:

[0140] To address the resource scheduling challenge of coupling user mobility and heterogeneous QoS requirements in multi-slice networks, this invention is the first to integrate LSTM time-series prediction with improved SAC reinforcement learning to construct a QoS requirement-aware dynamic scheduling framework. This fills the technical gap in the existing technology of transitioning from passively responding to resource requirements to proactively predictive scheduling, and achieves synergistic optimization of heterogeneous slice QoS guarantee and resource utilization.

[0141] (3) The technical solution of the present invention solves a technical problem that people have long wanted to solve but have never been able to solve successfully:

[0142] Existing multi-slice resource scheduling technologies struggle to simultaneously adapt to the uncertainty of user movement trajectories and the differentiated QoS requirements of eMBB / URLLC / BE services, resulting in a long-standing contradiction between "high-priority services experiencing excessive latency" and "low-priority services facing resource starvation." This invention, through LSTM's accurate prediction of users' future locations and needs, combined with an improved SAC algorithm that leverages balancing characteristics to dynamically allocate radio resource blocks, successfully solves this long-standing technical challenge.

[0143] (4) The technical solution of the present invention overcomes technical bias:

[0144] There was a technical bias in the industry that the complexity and real-time performance of reinforcement learning scheduling algorithms could not be achieved simultaneously, believing that high-precision prediction and complex policy optimization were difficult to adapt to the low-latency scheduling requirements of multi-slice networks. This invention breaks this technical bias by simplifying the network architecture and optimizing the experience replay mechanism, while ensuring prediction accuracy and scheduling performance, and controlling the algorithm execution latency to the millisecond level. Attached Figure Description

[0145] Figure 1 This is a flowchart of a wireless communication resource scheduling method based on LSTM and QoS demand awareness in a multi-slice network provided in an embodiment of the present invention.

[0146] Figure 2 This is a block diagram of a wireless communication resource scheduling system based on LSTM and QoS demand awareness in a multi-slice network provided in an embodiment of the present invention.

[0147] Figure 3 A schematic diagram of a system scenario provided in this embodiment of the invention;

[0148] Figure 4 This is a schematic diagram of an application scenario provided by an embodiment of the present invention;

[0149] Figure 5 This is a flowchart of the solution based on LSTM prediction combined with the improved SAC algorithm provided in the embodiments of the present invention;

[0150] Figure 6 These are simulation result diagrams provided in the embodiments of the present invention;

[0151] Figure 7 This is a comparison chart of cumulative rewards provided in an embodiment of the present invention. Detailed Implementation

[0152] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0153] like Figure 1 As shown in the figure, the wireless communication resource scheduling method based on LSTM and QoS demand awareness in a multi-slice network provided by this embodiment of the invention includes the following steps:

[0154] S1 models multi-base station cellular network scenarios, user mobility models, multi-type network slice (eMBB, URLLC, BE) service models, and channel transmission models.

[0155] S2, input the user's historical location and slice requirements from multiple time slots into the LSTM model to obtain the user's predicted location and slice requirements for the next time slot; integrate the prediction results with the current base station resource status, channel quality, and real-time slice requirements to transform the optimization problem into a Markov decision problem, complete the modeling of the state space and action space, and design a differentiated reward function with priority weights for the differentiated QoS requirements of slices.

[0156] S3 defines the optimization objectives, namely, to maximize the QoS compliance rate of multi-slice network resources and the utilization rate of global network resources, while setting constraints such as total resource constraints, slice priority constraints, and hard constraints on URLLC latency and reliability.

[0157] S4. Based on the improved SAC algorithm, a QoS-aware resource scheduling model is constructed and trained. The trained scheduling model is compared with benchmark algorithms such as PF, DDPG, and PPO in static scenarios, user mobility scenarios, and traffic burst scenarios to verify its superior performance.

[0158] S1 provided in this embodiment of the invention specifically includes:

[0159] (1): The number and initial location of base stations deployed in the multi-slice network scenario and users in each slice are known. The LSTM prediction module and the improved SAC scheduling module work together. The input window length and hidden layer dimension of the LSTM module are pre-configured. During the scheduling process, the historical location of users and slice demand data are continuously collected and the prediction results are output.

[0160] (2): Define the channel model of the wireless network; base station and users between Clusters and The received power of the reference signal for each ray can be expressed as:

[0161]

[0162] in For path loss, For the shadow to fade, Corresponding LOS path coefficient, For the first Cluster, the first NLOS path coefficient of a ray For base stations The transmit power and spectral efficiency of this cluster are given by the following formula:

[0163]

[0164] in, For noise power, Inter-cell interference can be represented as:

[0165]

[0166] (3): Define the multi-slice model in the network: Each slice contains a group of user equipment (UEs) with similar service characteristics and consistent QoS requirements, and define the resource allocation vector:

[0167]

[0168] in, Indicates the first Step to slice The number of RBGs allocated, The total number of slices; resource allocation must meet the following constraints:

[0169]

[0170] This represents the total number of RBGs currently available, based on the number of RBGs available in the system. It is a fixed value. The RBG combination methods for slice resource allocation are limited. Define a resource combination vector:

[0171]

[0172] in, This represents one of the possible combinations of RBG allocations. This represents the total number of all possible RBG combinations in this step;

[0173] (4): Define the user QoS performance indicator model: user In the Step throughput The calculation is as follows:

[0174]

[0175]

[0176] in It is the first Step by slice scheduling, from slice Assigned to user The number of RBGs, User In the The spectral efficiency of the step is given by R, where R is the total number of available RBGs in the system, and B is the total bandwidth of the system. It refers to the data packet size; effective throughput. This refers to the UE in the first... The actual throughput transmitted over the network is defined as:

[0177]

[0178] in yes In the The theoretical maximum throughput of the step User In the The amount of cached data available for each step; simultaneously, define the cache occupancy rate. This reflects the load status of the user's cache:

[0179]

[0180] in This refers to the maximum capacity of the user cache; packet loss rate. Is the UE in length of The percentage of packets dropped within the sliding window is defined as:

[0181]

[0182] in User In the The request throughput per step For users In the The amount of data discarded per step; average cache latency This refers to the average waiting time of data packets in the UE buffer, defined as:

[0183]

[0184] in User In the The cache latency vector of the step. Indicates the first Step, cache wait latency is The number of packets per TTI; This is the maximum allowable cache latency; long-term average throughput. and quintile throughput These are the UEs with a length of The average service throughput and the quintile of throughput within the sliding window are calculated as follows:

[0185]

[0186]

[0187] in This indicates that the throughput data within the window is calculated using the quintiles.

[0188] S2 provided in this embodiment of the invention specifically includes:

[0189] (1): Construct the state space in the time slot In this context, the state space of the Agent can be represented as:

[0190]

[0191] in This represents the time-series data of user location over the past 10 time slots. This represents the time-series data of service requests for each slice over the past 10 time slots. This represents the real-time QoS metrics for each slice. This indicates the RBG allocation status of each slice in the current time slot. This indicates the user's location at one future moment predicted by LSTM;

[0192] (2): Constructing the action space, the agent's action space can be represented as:

[0193]

[0194] in, Assign action factors to the three types of slices; This indicates that the reserved action factors are based on predictions, and the values ​​of all action factors range from [−1, 1]. The continuous action values ​​output by the Agent are matched to feasible RBG assignment sets through Euclidean distance. ,Right now:

[0195]

[0196] in Represents the set of feasible combinations. Candidate Combinations and Agent Output Euclidean distance; Allocate a predefined set of legal resources to reserve a combination of resources;

[0197] (3): Design the reward function. The immediate reward that the Agent can obtain can be expressed as:

[0198]

[0199] in, The penalty weight for wasting resources; Describe the degree of idleness of reserved resources; For slices The QoS requirement drift reward needs to be designed with differentiated QoS metrics for different slices:

[0200] The QoS requirement drift reward for eMBB slices is based on the satisfaction of throughput, latency, and packet loss rate, and is defined as follows:

[0201]

[0202] The sub-items are as follows:

[0203] Throughput Bonus:

[0204]

[0205] Delay bonus:

[0206]

[0207] Packet loss rate bonus:

[0208]

[0209] in, ;

[0210] The QoS requirement drift reward for URLLC slices takes into account that URLLC focuses on low latency and high reliability. The reward function is consistent with that of eMBB, but uses higher weights to reflect priority.

[0211]

[0212] The packet loss rate metric must meet strict constraints. ;

[0213] QoS requirement drift reward for BE slices; BE slices focus on long-term throughput and worst-case performance, and the reward function is designed based on long-term throughput and quintile throughput.

[0214]

[0215] The sub-items are as follows:

[0216] Long-term throughput rewards:

[0217]

[0218] 5th percentile throughput bonus:

[0219]

[0220] In each time slot t, the agent observes the state. Output action To complete resource allocation and reservation; the environment calculates rewards based on the network state after the action is executed. and transition to the state of the next time slot. .

[0221] S3 provided in this embodiment of the invention specifically includes:

[0222] (1): The joint optimization problem is defined as realizing dynamic radio resource allocation of multiple network slices to maximize the long-term satisfaction of the QoS requirements of each slice. Therefore, the joint optimization problem is:

[0223]

[0224] (2): Determine the constraints;

[0225] The constraints are:

[0226]

[0227] constraint Ensure that the total amount of RBG resources does not exceed the total available resources of the system; constraints Limit the resource reservation cap for high-priority slices; constraints Limit user movement characteristics: speed not exceeding 10m / s, and probability of direction change not exceeding 0.2 / s.

[0228] The S4 provided in this embodiment of the invention specifically includes: constructing a multi-slice resource scheduling algorithm based on LSTM prediction and QoS awareness algorithm to solve the problem.

[0229] (1.1) Initialize LSTM prediction network parameters Initialize the parameters of the Actor network, Critic network, target Actor network, and target Critic network. ;

[0230] (1.2) Initialize the experience playback buffer Batch size Target network update coefficients Entropy weight coefficient ;

[0231] (1.3) for each round do

[0232] (1.3.1) Reset base station resource status, user initial location and slice requirements

[0233] (1.3.2) while t ≤ T do

[0234] (1.3.2.1) Obtain the user's historical location and slice requirements for the past 10 time slots.

[0235] (1.3.2.2) Input the LSTM network and output the predicted user location and slice requirements for the next time slot.

[0236] (1.3.2.3) Combine the prediction results with the current base station resources and channel quality to construct the global environment status.

[0237] (1.3.2.4) Actor network output actions This includes the current RBG allocation and predicted resource reservation, and executes actions to update the resource allocation status.

[0238] (1.3.2.5) Obtain instant differentiated rewards Change the environment state to

[0239] (1.3.2.6) Sequence information ( , , , Stored in the experience replay buffer From which random samples are taken. Small batch samples

[0240] (1.3.2.7) Calculate the target Q value Update the current Critic network weights using mean squared error loss.

[0241] (1.3.2.8) Update the current Actor network weights using policy gradient descent.

[0242] (1.3.2.9) Update the weights of the target Critic and Actor networks using a soft update method.

[0243] (1.3.2.10) Update the LSTM network parameters with the error between the predicted and actual positions.

[0244] (1.3.3) End the while loop

[0245] (1.4) End the for loop.

[0246] like Figure 2 As shown in the figure, an embodiment of the present invention provides a wireless communication resource scheduling system based on LSTM and QoS demand awareness in a multi-slice network, comprising:

[0247] The modeling module is used to model multi-slice network scenarios, user mobility models, and slice service requirement models.

[0248] The target construction module is used to construct optimization targets, clarifying the goals and constraints for maximizing the QoS guarantee level of multi-slice services and the global resource utilization rate.

[0249] The function design module is used to transform problems into Markov decision problems based on scenarios and constraints, model the state space and action space, and design differentiated reward functions for optimization objectives.

[0250] The model training module is used to build a QoS-aware multi-slice resource scheduling algorithm based on LSTM prediction and combined with the improved SAC algorithm, train the model, and compare the performance of the trained model with the benchmark algorithm under different user mobility scenarios to verify its performance.

[0251] Another object of the present invention is to provide a computer device including a memory and a processor, the memory storing a computer program, which, when executed by the processor, causes the processor to perform the steps of a wireless communication resource scheduling method based on LSTM and QoS demand awareness in a multi-slice network.

[0252] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of a wireless communication resource scheduling method based on LSTM and QoS demand awareness in a multi-slice network.

[0253] Another objective of this invention is to provide an information data processing terminal for implementing a wireless communication resource scheduling system based on LSTM and QoS demand awareness in the multi-slice network.

[0254] Specific implementation of the present invention:

[0255] Reference Figure 3 The following provides a more detailed description of specific implementation scenarios of the present invention. In one scenario where the present invention employs a hexagonal honeycomb structure, wherein... The analysis is performed within the target cell, considering interference from six adjacent cells. Each cell is equipped with a base station, which forms three independent sector antennas via a controlled vertical uniform linear array, totaling [number missing]. Each base station serves a cellular unit. There is no cooperation mechanism between base stations and sectors; all interference is treated as noise, including antenna backlobe interference and interference from other base stations. A hexagonal network layout was chosen to simplify the model and ensure reproducibility, and a feature-rich and expressive random channel generator was employed to produce realistic channel variations even with a hexagonal base station layout.

[0256] This invention divides the entire system into equal parts. Each time slot is defined, and the system parameters remain constant within each time slot. (Refer to...) Figure 4 The multi-slice network model of this invention is described in further detail below. Each type of slice achieves differentiated service guarantees through a dedicated Virtual Network Function (VNF) and resource pool. Base stations share a spectrum resource pool containing 17 RGB spectrums, and resources are dynamically scheduled and isolated through virtualization technology.

[0257] Various service slices are bound to specific business scenarios and QoS requirements, forming a precise mapping relationship between slices and resources. eMBB slices are designed for high-bandwidth services such as VR / AR and 4K / 8K video, with core QoS requirements being high throughput and low latency. We have defined three main quality of service objectives for eMBB slices: average service throughput... It should be equal to or higher than the specified minimum service throughput. Average delay and packet loss rate The required average delay should be maintained. and packet loss rate Below. Throughput of user equipment requests related to eMBB slicing. Follow the mean The Poisson distribution of URLLC slices is typical; however, user equipment in URLLC slices requires low latency and ultra-reliable communication, typically characterized by low packet loss. The throughput requested in URLLC is usually much lower than that required by eMBB. Some examples of URLLC applications include control signal transmission for autonomous vehicles, monitoring automated processes, and their remote control; these applications do not require high throughput but rather high communication reliability and low latency to function properly. We define slice QoS requirements similar to those for eMBB, where... , and These represent URLLC service throughput, average latency, packet loss rate, and required values, respectively. URLLC User Equipment Requested throughput Defined as mean The Poisson distribution; BE slices represent slices with lower priority than eMBB and URLLC, with no strict requirements on throughput and latency. Web browsing and email sending are examples of BE applications. BE considers two main objectives: throughput for long-running services. It should be equal to or higher than the specified minimum throughput. Furthermore, the throughput of the 5th percentile service It should be equal to or higher than the specified minimum throughput. User equipment Throughput from BE slice requests Defined as a Poisson distribution with mean 1 / 2. .

[0258] Reference Figure 5 This paper describes in detail the QoS demand-aware multi-slice resource scheduling algorithm solution based on LSTM prediction combined with the improved SAC algorithm. The specific algorithm flow is shown in Table 1. Using LSTM prediction and the improved SAC reinforcement learning algorithm as the core, dynamic scheduling of multi-slice resources is achieved through an end-to-end learning framework. The specific process is as follows:

[0259] First, the modules and parameters are initialized. Key parameters such as network parameters, buffer size, and update coefficients of the LSTM prediction module and reinforcement learning module are initialized to lay the foundation for subsequent training. In each time slot, the LSTM module predicts future location and slice requirements based on the user's historical data, and merges this prediction with the real-time network state to construct a global state, enabling the agent to perceive the user's movement trend in advance.

[0260] The agent outputs actions including resource allocation and reservation based on its current state, receives immediate rewards after interacting with the network environment, and stores the experience in a replay buffer to improve sampling efficiency. Subsequently, experience is sampled from the buffer, the target Q-value is calculated, and the Critic network is updated. Then, an entropy regularization term is used to update the Actor network to balance policy exploration and exploitation, ensuring the algorithm's stability in dynamic scenarios.

[0261] The target network employs a soft update strategy to avoid oscillations during training, while the LSTM module continuously iterates to optimize prediction accuracy. Through iterative iteration, the algorithm eventually converges to the optimal resource scheduling strategy that integrates user mobility predictions, achieving coordinated optimization of multi-slice QoS requirements and global resource efficiency.

[0262]

[0263] Table 1

[0264] The invention will be further described below with reference to simulation experiments.

[0265] 1. Simulation conditions:

[0266] The simulation experiment of this invention is conducted on a Windows platform, with the following main configurations: CPU: Intel(R) Core(TM) i5-10400F, 2.90GHz; Memory: 16G; Operating System: Windows 10; Simulation Software: PyCharm.

[0267] 2. Simulation Content and Result Analysis:

[0268] This simulation experiment compares the method of this invention with deep reinforcement learning algorithms based on DDPG and PPO to verify and evaluate the performance superiority of the method of this invention in a multi-slice network dynamic scheduling scenario. In the simulation experiment, a multi-base station cellular network scenario with uneven user distribution was constructed, covering a crossroads and surrounding roads in the core urban area. Within this scenario, 7 base stations and 10 user terminals were deployed, moving dynamically along the roads, simultaneously generating three types of heterogeneous slice service requests and initiating resource scheduling requests to the base stations. Furthermore, the system integrates an LSTM timing prediction module and an improved SAC reinforcement learning scheduling module to achieve coordinated operation of user mobility prediction and dynamic resource allocation.

[0269] During the simulation, we set the weight parameters for wireless communication and slicing QoS requirements (as shown in Tables 2 and 3), and the requirement compliance parameters. Set as non-compliant parameter The QoS requirement parameters for each network slice under medium and high load network traffic are set at 0.8 times the normal value to avoid excessive suppression of the penalty constraint by positive feedback. Simultaneously, improved SAC network hyperparameters are set (as shown in Table 5). In the model of this algorithm, the LSTM prediction network adopts a three-layer LSTM network architecture, with a 6-dimensional input feature dimension, covering the user's past 10 time slots, with 100ms of core movement parameters per time slot: including two-dimensional location coordinates. Movement speed and direction angle of motion The velocity and direction angle are calculated from the position difference between adjacent time slots. The output layer is the prediction result of the position and slice requirement of the next time slot, with an output dimension of 5. The activation function is tanh, and the optimizer is Adam. The Actor network and Critic network of the improved SAC algorithm both adopt a four-layer fully connected structure with 800, 600, 512, and 256 neurons, respectively. ReLU is used as the activation function between adjacent fully connected layers. The Actor network output layer normalizes the action vector through the tanh function to ensure that the action factor value range is within [-1,1], which is suitable for the learning requirements of continuous action space.

[0270]

[0271] Table 2

[0272]

[0273] Table 3

[0274]

[0275] Table 4

[0276]

[0277] Table 5

[0278] Figure 6 A compares the throughput performance of four algorithms—SAC, PF, DDPG, and PPO—on BE slices using two sub-graphs: The left graph focuses on the quintile throughput, reflecting the minimum performance guarantee. It shows that the SAC algorithm's quintile throughput remains consistently around 14 Mbps, while PPO only fluctuates between 3-7 Mbps, and DDPG and PF are essentially close to zero. The right graph shows the long-term throughput, reflecting average performance. The SAC algorithm maintains a stable high throughput of 14-16 Mbps, PPO and DDPG exhibit larger throughput fluctuations, while PF is close to zero for most of the time. Overall, the SAC algorithm significantly outperforms the other three algorithms in terms of performance stability and average performance on BE slices, providing more reliable QoS guarantees.

[0279] Figure 6 Table B illustrates the latency characteristics of eMBB and URLLC slices under different algorithms. The horizontal axis represents time, and the vertical axis represents the communication latency of the slice users. Curves of different colors and shapes correspond to the latency performance of different slices under different algorithms.

[0280] It can be observed that in eMBB slices, the latency of the PF algorithm initially spiked to nearly 70ms, and although it subsequently dropped, it remained above 15ms for a long period. The latency of the PPO and DDPG algorithms fluctuated but remained in the range of 5-15ms. In contrast, the latency of the SAC algorithm remained consistently around 0ms, outperforming other algorithms. For high-priority URLLC slices, the latency of the SAC algorithm remained at 0ms throughout, fully meeting the low-latency QoS requirements of URLLC. While the latency of the PPO and DDPG algorithms was generally low, it fluctuated slightly. The URLLC latency of the PF algorithm gradually increased to around 8ms, failing to guarantee the latency requirements of highly sensitive services.

[0281] This demonstrates that the SAC algorithm has significant advantages in controlling the delay of different slices, especially in stably meeting the stringent delay constraints of URLLC.

[0282] Figure 6 C illustrates the dynamic changes in packet loss rate for eMBB and URLLC slices under different resource scheduling algorithms. The horizontal axis represents time, and the vertical axis represents the packet loss rate of the slice user. Curves of different colors and shapes correspond to the packet loss rate performance of different slices under various algorithms.

[0283] It can be observed that for eMBB slices, the packet loss rate of the PF algorithm remains consistently high, ranging from 0.3 to 0.4. The packet loss rates of PPO and DDPG fluctuate but are generally above 0.3. While the SAC algorithm does not fully meet the packet loss rate requirement, it significantly reduces the rate compared to other algorithms, and its packet loss rate is below the required 0.2 most of the time. For URLLC slices, which have stringent reliability requirements, the SAC algorithm maintains a packet loss rate of 0 throughout, fully meeting the requirement of URLLC packet loss rate ≤ 10⁻. 5 The high reliability requirements of the business cannot be guaranteed, but the URLLC packet loss rate of the PF and PPO algorithms fluctuates significantly, even exceeding 0.2 at some points, which makes it impossible to guarantee the reliability requirements of the business.

[0284] This demonstrates that the SAC algorithm has significant advantages in controlling packet loss rates across different slices, and in particular, it can meet the high reliability requirements of URLLC.

[0285] Figure 6D illustrates the cumulative distribution function (CDF) characteristics of eMBB and URLLC slice service throughput under different resource scheduling algorithms. The horizontal axis represents the actual service throughput of slice users, and the vertical axis represents the CDF value, i.e., the percentage of users whose throughput does not exceed the current horizontal axis value. Curves of different colors and shapes correspond to the throughput distribution performance of the two types of slices under various algorithms.

[0286] It can be observed that for eMBB slices, the CDF curve of the PF algorithm is close to 1.0 at a throughput of about 1 Mbps, meaning that almost all users have a throughput ≤ 1 Mbps, which cannot meet the throughput requirements of eMBB. The CDF values ​​of the PPO and DDPG algorithms are about 0.8 at a throughput of 20 Mbps, meaning that 80% of users have a throughput ≤ 20 Mbps. In contrast, the CDF value of the eMBB curve corresponding to the SAC algorithm is 0.5 at a throughput of 40 Mbps, indicating that more than 50% of users have a throughput exceeding 40 Mbps, which meets the QoS requirements of eMBB. For URLLC slices, the CDF value of the SAC algorithm is less than 0.2 at a throughput of 5 Mbps, meaning that more than 80% of users have a throughput ≥ 5 Mbps, which also meets the throughput requirements of URLLC.

[0287] This indicates that the SAC algorithm can significantly improve the throughput coverage of eMBB slices while ensuring that URLLC slices are not affected.

[0288] Figure 7 This diagram illustrates the cumulative reward variation characteristics of different algorithms in both the full and partial observation spaces. The horizontal axis represents time, and the vertical axis represents the cumulative reward obtained by the algorithm. Curves of different colors and shapes correspond to the reward performance of different algorithms in the two types of observation spaces.

[0289] It can be observed that the cumulative reward of the PF algorithm has been in the negative range for a long time, indicating that it cannot effectively meet the QoS requirements of the slice. Although the cumulative rewards of the PPO and DDPG algorithms show an upward trend, the growth rate is slow and the final value is low. In contrast, the cumulative reward growth rate of the SAC algorithm is not only significantly faster than that of other algorithms, but also the cumulative reward value in both observation spaces is significantly greater than that of PPO and DDPG, reflecting its adaptability and robustness to the observation space.

[0290] This indicates that in multi-slice resource scheduling scenarios, the SAC algorithm can obtain better cumulative rewards regardless of whether the observation information is complete, and its performance is significantly better than other comparative algorithms.

[0291] Example 1: Predictive Collaborative Resource Scheduling Example in Static User Scenarios

[0292] In this embodiment, a multi-base station cellular network scenario is constructed, in which enhanced mobile broadband services, ultra-reliable low-latency services, and best-effort services are carried simultaneously. The user's location remains relatively stable over multiple consecutive time slots. The base station collects historical user location data and service request data for each slice in each time slot, and inputs the historical data of 10 consecutive time slots into the LSTM time series prediction model to output the user location prediction result and slice service demand prediction result for the next time slot.

[0293] During resource scheduling, the base station integrates the predicted user location and service demand with the current time slot's radio resource availability and slice real-time service status to construct a scheduling state. The scheduling agent simultaneously generates slice resource allocation decisions and predicted resource reservation decisions. Radio resource allocation is completed through a mapping mechanism from continuous actions to legitimate resource combinations. This allows predictive information to effectively reduce resource waste and improve the overall satisfaction of multi-slice service quality in scenarios where user locations change dynamically.

[0294] Example 2: Prediction-driven scheduling example in a low-speed user movement scenario

[0295] In this embodiment, the user moves within the network coverage area at a speed not exceeding 10 meters per second, and the probability of changing the direction of movement does not exceed 0.2 per second. The base station continuously collects the user's location information and slice service request information in multiple time slots, and uses a time-series prediction model to predict the user's location and slice demand in the next time slot, thereby anticipating potential channel quality change trends.

[0296] During the scheduling decision-making phase, the scheduling agent outputs resource allocation action factors and resource reservation action factors based on the fused system state, and selects a legitimate combination of radio resources through distance matching. This approach enables base stations to reserve resources in advance for potential channel degradation during user movement, thereby ensuring the continuity of ultra-reliable low-latency services while avoiding unnecessary resource crowding out of enhanced mobile broadband and best-effort services.

[0297] Example 3: Slicing Demand Forecasting in High-Speed ​​Business Fluctuation Scenarios

[0298] In this embodiment, a service surge scenario occurs in the multi-slice network, where the enhanced mobile broadband slice experiences a rapid increase in throughput demand, while the service load of the best-effort slice and the ultra-reliable low-latency slice remains relatively stable. The base station continuously collects historical service request data from each slice and feeds it as an input sequence into a time-series prediction model to output the slice service demand prediction result for the next time slot.

[0299] During scheduling execution, the scheduling agent makes joint decisions on resource allocation and reservation based on the predicted trends in service demand. When an increase in demand for enhanced mobile broadband slices is predicted, the agent increases the proportion of available resources for that slice in advance and reduces scheduling jitter caused by frequent adjustments to time slots through a resource reservation mechanism, thereby improving the system's stability and resource utilization efficiency under sudden service scenarios.

[0300] Example 4: An Example of Mapping Continuous Actions to Discrete Radio Resource Combinations

[0301] In this embodiment, the system pre-generates multiple combinations of wireless resources that satisfy the total wireless resource constraint. Each resource combination corresponds to the wireless resource allocation ratio between different slices. The scheduling agent outputs continuously valued resource allocation action factors and resource reservation action factors in each time slot. These continuous actions do not directly correspond to a specific number of wireless resources.

[0302] During scheduling, the system calculates the distance between the agent's output action and each legal combination of wireless resources, and selects the resource combination with the smallest distance as the final scheduling result. This approach unifies the continuous decision-making model with discrete wireless resource constraints, ensuring that the scheduling strategy maintains the continuity of the search space while always satisfying the system's physical resource limitations.

[0303] Example 5: Quality of Service Driven Scheduling Example for Enhanced Mobile Broadband Slices

[0304] In this embodiment, the quality of service (QoS) evaluation of enhanced mobile broadband slices is based on a comprehensive assessment of actual user throughput, buffer latency, and packet loss rate. The system calculates the deviation between the actual service performance of the slice and the preset QoS requirements in each time slot, and generates corresponding reward or penalty signals accordingly.

[0305] During training, the scheduling agent uses the reward signal to adjust the resource allocation strategy. When the actual throughput of the enhanced mobile broadband slice is lower than the demand target or the buffer latency exceeds the target, the system reduces the value evaluation of the strategy in this state. When the service performance reaches or exceeds the target, the value evaluation of the corresponding strategy is increased, thereby guiding the scheduling strategy to gradually evolve towards high throughput, low latency and low packet loss rate.

[0306] Example 6: Strictly Constrained Scheduling of Ultra-Reliable Low-Latency Slices

[0307] In this embodiment, the ultra-reliable low-latency slice is set to have the highest scheduling priority, and its quality of service evaluation focuses on latency and reliability indicators. The system monitors the buffer latency and packet loss rate of this slice in each time slot, and uses whether it meets the preset strict constraints as an important basis for scheduling feedback.

[0308] In scheduling decisions, once the predictive model or real-time status indicates that an ultra-reliable low-latency slice poses a risk of service quality degradation, the scheduling agent prioritizes the radio resource needs of that slice through resource reservation and immediate resource allocation. This implementation ensures that, even with multiple slices coexisting, ultra-reliable low-latency services consistently meet the core service objectives of low latency and high reliability.

[0309] Example 7: Long-term performance assurance of best-effort slices

[0310] In this embodiment, the best-effort slice does not primarily optimize instantaneous performance, but rather focuses on long-term average throughput and low-quantile throughput performance within a sliding time window. The system statistically analyzes the service throughput data of the slice over multiple consecutive time slots to characterize its overall service capacity and worst-case service level.

[0311] During the reward calculation phase, the scheduling agent comprehensively considers the long-term average throughput and low-quantile throughput results of the best-effort slices. When the long-term performance is lower than expected, the resource allocation ratio of the slice is gradually increased. When the overall network load is low, the system allows the best-effort slices to utilize the remaining resources to improve service performance, thereby achieving full utilization of resources without affecting high-priority slices.

[0312] Example 8: An Example of Predictive Resource Reservation and Resource Waste Suppression

[0313] In this embodiment, the system reserves a portion of radio resources in each time slot based on the predicted user locations and slice requirements to cope with potential traffic growth or channel degradation in the next time slot. The scheduling system counts the actual amount of reserved resources used in subsequent time slots and calculates the proportion of unused reserved resources.

[0314] During the reward feedback phase, the system generates a resource waste penalty signal based on the proportion of unused reserved resources, guiding the scheduling agent to gradually reduce unnecessary resource reservations while ensuring service quality. This mechanism achieves an adaptive balance between prediction reliability and resource utilization, avoiding resource idleness caused by long-term over-reservation.

[0315] Example 9: Implementation of a Scheduling Model for Multi-Scenario Joint Training

[0316] In this embodiment, the scheduling model incorporates both user mobility scenarios and sudden service disruptions during the training phase, enhancing the model's generalization ability through alternating training across multiple scenarios. In each training round, the system randomly initializes user locations and sliced ​​service loads, and continuously updates the parameters of the prediction and scheduling models.

[0317] Through multi-scenario joint training, the scheduling agent can learn stable and effective resource allocation strategies under different network load conditions, so that there is no need to retrain the model for a single scenario in actual deployment, thereby improving the adaptability and robustness of the scheduling scheme in complex network environments.

[0318] Example 10: End-to-End Predictive Scheduling Closed-Loop Example

[0319] In this embodiment, the system forms a closed-loop process of prediction, decision-making, execution, and feedback in each scheduling time slot. The base station collects historical status and completes prediction at the beginning of the time slot, performs resource allocation and reservation operations in the middle of the time slot, and counts actual service quality indicators and resource usage at the end of the time slot.

[0320] The system uses prediction errors, service quality deviations, and resource utilization as feedback signals to synchronously update the parameters of the time-series prediction model and the resource scheduling model. Through this end-to-end closed-loop implementation, the prediction model and the scheduling model co-evolve during long-term operation, thereby continuously improving the service quality satisfaction and wireless resource utilization efficiency of the multi-slice network.

[0321] Evidence related to the technical effects obtained by the embodiments of the present invention.

[0322] This invention verifies the performance advantages of the QoS demand-aware multi-slice resource scheduling scheme based on LSTM prediction combined with the improved SAC algorithm in multiple scenarios through multiple sets of comparative experiments.

[0323] like Figure 6 As shown in Figure A, the long-term throughput and quartile throughput of the BE slice are evaluated. The results show that the improved SAC algorithm of this invention maintains the quartile throughput stably at 12-14 Mbps throughout the entire time period, which is significantly higher than the DDPG and PPO algorithms, proving that this scheme can provide more stable worst-case service guarantees for the BE slice; Figure 6 As shown in Figure B, the end-to-end latency of eMBB and URLLC slices is evaluated. The results show that the improved SAC algorithm of this invention stabilizes the URLLC latency at 2-5ms, which is much lower than that of the DDPG and PF algorithms, proving that this solution fully meets the low latency requirements of URLLC services. Figure 6 As shown in Figure C, the packet loss rate of eMBB and URLLC slices is evaluated. The results show that the improved SAC algorithm of this invention stabilizes the packet loss rate of URLLC slices below 0.1, while the packet loss rate of DDPG and PPO algorithms fluctuates between 0.2 and 0.4, proving that this solution has excellent guarantee capability for highly reliable services; Figure 6As shown in Figure D, the cumulative throughput distribution (CDF) of eMBB and URLLC slices is evaluated. The results show that the CDF of eMBB slice throughput corresponding to the improved SAC algorithm of this invention approaches 1 at 40 Mbps, while the PF algorithm approaches 1 at 10 Mbps, proving that this scheme can provide higher throughput for more users; Figure 7 As shown, the long-term cumulative reward of the algorithm is evaluated under both full observation and limited observation spaces. The results show that the cumulative reward of the improved SAC algorithm in the full observation space continues to increase over time, significantly higher than that of the DDPG and PPO algorithms, proving that the long-term benefits of this scheme are superior.

[0324] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.

[0325] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A predictive cooperative wireless communication resource scheduling method in a multi-slice network, characterized in that, In multi-base station cellular networks, for multi-slice service scenarios including enhanced mobile broadband slices, ultra-reliable low-latency slices, and best-effort slices, the following collaborative mechanism is used to complete radio resource scheduling: Input the user's location change information and slice service request information from multiple historical time slots into the time series prediction model to obtain the user location prediction result and slice service demand prediction result for the next time slot. The prediction results are fused with the available radio resource status of the base station in the current time slot, the channel quality status, and the real-time service status of the slice to construct a system state for resource scheduling decisions. Based on the system state, the scheduling agent simultaneously generates slice resource allocation decisions and predictive resource reservation decisions, enabling a collaborative scheduling relationship between current time slot allocation and future time slot demand for wireless resources, thereby achieving dynamic wireless resource scheduling in multi-slice scenarios.

2. The method as described in claim 1, characterized in that, The time-series prediction model is based on a recurrent neural network structure. It models user location data and slice service request data for 10 consecutive historical time slots, and outputs the user location prediction result and slice service demand prediction result for the next time slot.

3. The method as described in claim 1, characterized in that, The system status includes at least the user's historical location time-series information, slice service request time-series information, real-time service quality indicators for each slice, slice radio resource allocation status for the current time slot, and predicted user location information.

4. A joint decision-making method for wireless resource allocation and reservation in a multi-slice network, characterized in that, In a multi-slice wireless network, the resource scheduling problem is represented as a decision problem in a continuous action space. The scheduling agent outputs continuously valued resource allocation action factors and resource reservation action factors. These continuous actions are then mapped to a predefined set of legal wireless resource combinations through distance matching. The wireless resource combination with the smallest distance to the agent's output action is selected as the slice resource allocation result and resource reservation result for the current time slot, thereby achieving an effective connection between continuous decision-making and discrete wireless resource constraints.

5. The method as described in claim 4, characterized in that, The continuous actions include at least resource allocation action factors for enhanced mobile broadband slices, ultra-reliable low-latency slices, and best-effort slices, as well as resource reservation action factors based on predicted demand, and the value range of each action factor is limited to between -1 and +1.

6. The method as described in claim 4, characterized in that, The set of legal wireless resource combinations satisfies the constraint of the total amount of available wireless resources in the system, such that the sum of the number of wireless resources allocated to each slice in any resource combination does not exceed the total amount of currently available wireless resources in the system.

7. A differentiated quality-of-service-driven reward method for resource scheduling in multi-slice wireless networks, characterized in that, In each scheduling time slot, a reward item corresponding to the service quality requirements of each slice is constructed according to the slice type. The rewards of each slice are weighted and combined with the resource waste penalty to form an instant reward for training the resource scheduling agent. The reward for enhanced mobile broadband slices is constructed based on the degree of satisfaction of throughput, buffer latency and packet loss rate. The reward for ultra-reliable low latency slices is constructed based on strict constraints of low latency and high reliability. The reward for best-effort slices is constructed based on long-term average throughput and low percentile throughput. This allows the scheduling agent to meet the differentiated service quality requirements of different slices while optimizing the overall resource utilization.

8. The method as described in claim 7, characterized in that, The resource waste penalty is determined based on the number of wireless resources that are not actually used after the predicted resource reservation, and is used to characterize the degree of idleness of the predicted reserved resources.

9. The method as described in claim 7, characterized in that, The rewards for enhanced mobile broadband slices consist of throughput rewards, latency rewards, and packet loss rate rewards. A penalty weight is applied when the actual service performance is lower than the preset service quality requirements, and a positive reward weight is applied when the actual service performance meets or exceeds the preset service quality requirements.

10. The method as described in claim 7, characterized in that, The reward for best-effort slices is constructed based on the long-term average throughput and low-quantile throughput statistics within the sliding window, which is used to simultaneously improve the overall service performance of best-effort slices and the service performance in the worst case.

Images