Predictive enhanced QoS (Quality of Service) sensing millimeter wave beam and power combined scheduling method

Abstract

The invention provides a combined scheduling method for predicting and enhancing QoS (Quality of Service) sensing millimeter wave beams and power, and belongs to the technical field of wireless communication and resource management. In a millimeter wave system, user equipment periodically measures downlink reference signal receiving power and reports the downlink reference signal receiving power to a base station; and the BS maintains a historical RSRP sequence for each UE, and performs multi-step prediction by using an AI model to obtain a Top-K candidate beam at a future moment and the quality of the Top-K candidate beam. And the base station fuses the prediction result with the queue state, the service type and the QoS constraint, constructs a prediction enhancement state, and jointly optimizes beam selection and power distribution based on a deep reinforcement learning model. According to the method, a neural network does not need to be deployed on the UE side, the problem of channel aging is effectively relieved, the spectrum efficiency is remarkably improved in a high-mobility scene, meanwhile, time delay is reduced, the QoS default probability is reduced, and the method is suitable for a millimeter wave communication system of eMBB, URLLC and mMTC mixed services.

Description

Technical Field

[0001] This invention relates to the field of wireless communication technology, and in particular to a predictive enhanced QoS-aware millimeter-wave beam and power joint scheduling method. Background Technology

[0002] In millimeter-wave communication systems, beam mismatch and channel aging severely impact system performance due to high propagation loss, strong beam directivity, and rapid channel changes caused by high-speed user movement. Existing resource scheduling methods often rely on instantaneous or delayed channel information for decision-making, making it difficult to balance high spectral efficiency with stringent latency and reliability requirements. On the other hand, while deep reinforcement learning can handle dynamic and complex environments, its performance remains constrained by incomplete observations and delayed decisions if it lacks a forward-looking understanding of channel evolution trends. Therefore, a joint design scheme that can introduce predictive information and improve QoS-aware scheduling performance under low-complexity conditions is urgently needed. Summary of the Invention

[0003] The present invention aims to at least partially solve one of the technical problems in the related art.

[0004] Therefore, the first objective of this invention is to propose a predictive enhanced QoS-aware millimeter-wave beam and power joint scheduling method.

[0005] Another objective of this invention is to provide a predictive enhanced QoS-aware millimeter-wave beam and power joint scheduling device.

[0006] The third objective of this invention is to provide a computer device.

[0007] A fourth objective of this invention is to provide a non-transitory computer-readable storage medium.

[0008] To achieve the above objectives, a first aspect of the present invention proposes a predictive enhanced QoS-aware millimeter-wave beam and power joint scheduling method, comprising: S1, obtain the historical RSRP measurement sequence reported by the user equipment, and generate a Top-K candidate beam set and corresponding prediction quality for multiple future time slots based on the historical sequence through a time-series prediction model; S2, the Top-K candidate beam set is fused with the queue status, service type identifier and QoS constraint parameters of each user equipment to construct a prediction-enhanced global state vector containing time-series prediction information; S3, based on the predicted enhanced global state vector, a deep reinforcement learning model is used to perform beam selection and power allocation decisions, wherein the decision-making process balances weighted spectral efficiency with URLLC delay violation penalty and mMTC service interruption penalty through a custom reward function; S4. Based on the decision results, configure the downlink beamforming parameters and transmission power of the base station, and update the queue status and QoS satisfaction of user equipment to form training samples for model iterative optimization.

[0009] In one embodiment of the present invention, S1 includes: Let X be the set of RSRP measurements reported by the UE. After j beam management cycles, the signal received at the BS can be represented as: ; BS predicts the beam quality vector and Top-K beam index for the next h time slots, with the objective function being: ; in For the beam prediction model, the optimization objective is to minimize the weighted sum of the prediction error and the Top-K classification loss: .

[0010] In one embodiment of the present invention, S2 includes: The method for constructing the prediction-enhanced global state vector is as follows: for each UE k User feature vectors are constructed as :

[0011] in, Let t be the initial queue length for time slot t. The measured RSRP vector for time slot t. For future h-slot Top-K beam indexing, To correspond to the predicted RSRP value, One-hot encoding for service type; Stack the user feature vectors of all K UEs to form a global state vector. .

[0012] In one embodiment of the present invention, S3 includes: The objective function and training method of the DRL scheduler are as follows: The scheduler adopts the DQN architecture, and the objective is to maximize the sum of long-term discounted rewards.

[0013] Among them, instant rewards Defined as:

[0014] For service type weights, For UE kThe actual data rate, For the number of URLLC delay violations, For mMTCoutage times, , This is the penalty coefficient.

[0015] In one embodiment of the present invention, S4 includes: The scheduler training employs an ε-greedy strategy, storing empirical samples through a replay buffer and stabilizing the training process by separating the target network from the main network. Power allocation decisions satisfy... .

[0016] In one embodiment of the present invention, the method further includes: The number of antennas for BS and UE are respectively and The downlink channel matrix is ​​denoted as ,in For static multipath components, Dynamic factors caused by user movement.

[0017] In one embodiment of the present invention, the method further includes: The weight vector for beamforming can be expressed as:

[0018] in, and These are the weights of the antenna elements in the vertical and horizontal directions of the UPA, respectively. It is the size of the beamforming codebook; The beam pair (m,n) signal received by the UE can be represented as:

[0019] in, The average transmit power, , Beamforming vectors for BS and UE; To transmit pilot signals, For additive noise, the RSRP calculation formula for beam pair (m,n) is as follows: .

[0020] To achieve the above objectives, a second aspect of the present invention provides a predictive enhanced QoS-aware millimeter-wave beam and power joint scheduling device, comprising: The historical RSRP acquisition and beam prediction module is used to acquire the historical RSRP measurement sequences reported by user equipment, and generate a Top-K candidate beam set and corresponding prediction quality for multiple future time slots based on the historical sequences through a time-series prediction model. The state vector construction module is used to fuse the Top-K candidate beam set with the queue status, service type identifier and QoS constraint parameters of each user equipment to construct a prediction-enhanced global state vector containing time-series prediction information. The beam and power decision module is used to perform beam selection and power allocation decisions based on the predicted enhanced global state vector using a deep reinforcement learning model. The decision process balances weighted spectral efficiency with URLLC latency violation penalty and mMTC service interruption penalty through a custom reward function. The parameter configuration and model optimization module is used to configure the downlink beamforming parameters and transmission power of the base station according to the decision results, and update the queue status and QoS satisfaction of user equipment to form training samples for model iterative optimization.

[0021] The present invention provides a predictive enhanced QoS-aware millimeter-wave beam and power joint scheduling method and apparatus, which can effectively alleviate channel aging problems in dynamic millimeter-wave channel environments, improve the foresight and QoS guarantee capabilities of beam and power scheduling, thereby significantly improving system spectrum efficiency and reducing URLLC delay violation probability and mMTC outage probability.

[0022] To achieve the above objectives, a third aspect of this application provides a computer device comprising a processor and a memory; wherein the processor runs a program corresponding to the executable program code stored in the memory, for implementing a predictive enhanced QoS-aware millimeter-wave beam and power joint scheduling method as described in the first aspect embodiment.

[0023] To achieve the above objectives, a fourth aspect of this application provides a non-transitory computer-readable storage medium storing a computer program that, when executed by a processor, implements a predictive enhanced QoS-aware millimeter-wave beam and power joint scheduling method as described in the first aspect embodiment.

[0024] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0025] Figure 1 This is a flowchart of a predictive enhanced QoS-aware millimeter-wave beam and power joint scheduling method according to an embodiment of the present invention; Figure 2This is a block diagram of 5G / 6G millimeter-wave communication according to an embodiment of the present invention; Figure 3 This is a diagram of a prediction-enhanced QoS-aware joint scheduling framework according to an embodiment of the present invention; Figure 4 This is a performance comparison chart of scheduling methods according to embodiments of the present invention; Figure 5 This is a structural diagram of a predictive enhanced QoS-aware millimeter-wave beam and power joint scheduling device according to an embodiment of the present invention; Figure 6 It is a computer device according to an embodiment of the present invention. Detailed Implementation

[0026] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] The following description, with reference to the accompanying drawings, describes a predictive enhanced QoS-aware millimeter-wave beam and power joint scheduling method and apparatus according to an embodiment of the present invention.

[0029] Figure 1 This is a flowchart of a predictive enhanced QoS-aware millimeter-wave beam and power joint scheduling method according to an embodiment of the present invention, such as... Figure 1 As shown, it includes: S1, obtain the historical RSRP measurement sequence reported by the user equipment, and generate a Top-K candidate beam set and corresponding prediction quality for multiple future time slots based on the historical sequence through a time-series prediction model; S2, the Top-K candidate beam set is fused with the queue status, service type identifier and QoS constraint parameters of each user equipment to construct a prediction-enhanced global state vector containing time-series prediction information; S3, based on the predicted enhanced global state vector, a deep reinforcement learning model is used to perform beam selection and power allocation decisions, wherein the decision-making process balances weighted spectral efficiency with URLLC delay violation penalty and mMTC service interruption penalty through a custom reward function; S4. Based on the decision results, configure the downlink beamforming parameters and transmission power of the base station, and update the queue status and QoS satisfaction of user equipment to form training samples for model iterative optimization.

[0030] Specifically: Step 1: Consider a millimeter-wave communication system. The base station (BS) uses analog beamforming and is equipped with a uniform planar array (UPA) antenna; the user equipment (UE) also uses a UPA antenna and supports analog beam reception. The system simultaneously carries three heterogeneous services: eMBB, URLLC, and mMTC, each with different QoS constraint parameters.

[0031] Step 2: In each beam measurement time slot, the UE measures the RSRP data of all BS transmitted beams without requiring additional beam scanning attempts. The RSRP reported by the UE is the instantaneous measurement value of the corresponding beam pair. The beamforming weight vector between the BS and the UE is the Kronecker product of the vertical and horizontal steering vectors.

[0032] Step 3: Deploy a centralized beam prediction module on the BS side to generate Top-K beam indices and corresponding predicted RSRP values ​​for multiple future time slots based on recent historical RSRP sequences.

[0033] Step 4: The BS collects all reported data from all UEs, including queue length, service type identifier, and QoS constraint parameters, and constructs a predictive enhanced global state vector.

[0034] Step 5: Evaluation metrics: The BS-side scheduler based on DRL processes the global state vector and outputs beam selection, user scheduling, and power allocation decisions, which must meet the total power constraint.

[0035] Step 6: The BS configures downlink beamforming parameters and transmission power according to the decision. The UE completes beam alignment and receives data based on the received configuration information. The BS updates the UE queue status and QoS satisfaction status. Evaluation metrics: Weighted spectral efficiency, URLLC delay violation probability, mMTCoutage probability.

[0036] Furthermore, in the communication system constructed in step 1, the number of antennas for the BS and the UE are respectively and The downlink channel matrix is ​​denoted as .in For static multipath components, Dynamic factors caused by user movement.

[0037] The weight vector for beamforming can be expressed as:

[0038] in, and These are the weights of the antenna elements in the vertical and horizontal directions of the UPA, respectively. It is the size of the beamforming codebook.

[0039] The beam pair (m,n) signal received by the UE can be represented as:

[0040] in, The average transmit power, , For the beamforming vectors of BS and UE. To transmit pilot signals, For additive noise, the RSRP calculation formula for beam pair (m,n) is as follows:

[0041] 10. Further, in step 3, an optimization objective function is constructed. Let X be a set of RSRP measurements reported by the UE. After j beam management cycles, the signal received at the BS can be expressed as:

[0042] Based on this, BS predicts the beam quality vector and Top-K beam index for the next h time slots, with the objective function being:

[0043] in For the beam prediction model, the optimization objective is to minimize the weighted sum of the prediction error and the Top-K classification loss:

[0044] Furthermore, the method for constructing the global state vector in step 4 is as follows: for each UE k User feature vectors are constructed as :

[0045] in, Let t be the initial queue length for time slot t. The measured RSRP vector for time slot t. For future h-slot Top-K beam indexing, To correspond to the predicted RSRP value, One-hot encoding for service type.

[0046] Stack the user feature vectors of all K UEs to form a global state vector. Furthermore, the objective function and training method of the DRL scheduler in step 5 are as follows: The scheduler adopts a DQN architecture, and the objective is to maximize the sum of long-term discounted rewards:

[0047] Among them, instant rewards Defined as:

[0048] For service type weights, For UE k The actual data rate, For the number of URLLC delay violations, For mMTCoutage times, , This is the penalty coefficient.

[0049] The scheduler training employs an ε-greedy strategy, storing empirical samples through a replay buffer and stabilizing the training process by separating the target network from the main network. Power allocation decisions satisfy... .

[0050] The present invention provides a prediction-enhanced QoS-aware scheduling method that, through deep coupling of beam prediction and DRL scheduling, improves weighted spectrum efficiency by 20%-30% compared to traditional methods, reduces URLLC latency violation probability by 40%-60%, and reduces feedback overhead by more than 60%, making it suitable for 6G millimeter wave heterogeneous service scenarios.

[0051] like Figure 2 The millimeter-wave communication system to which this invention is applicable is demonstrated. The base station (BS) is located at the center of the cell (radius 200m) and equipped with an 8×8 UPA antenna array (64 antennas in total), configured with a 64-beam DFT-based analog beamforming codebook. The user equipment (UE) is equipped with a 2×2 UPA antenna array (4 antennas in total) and supports analog beam reception. The system operates in the 28GHz frequency band with a bandwidth of 400MHz, and the BS has a maximum transmit power of 50dBm. The UE moves within a city road network generated by SUMO.

[0052] Figure 3 The diagram shown illustrates the predictive enhanced QoS-aware scheduling block diagram provided by this invention. The UE side includes a beam measurement module and a beam reporting module; the BS side includes a beam prediction module, a state construction module, a DRL scheduling module, and a transmission configuration module. These modules work together to complete the entire process from data acquisition and predictive reporting to scheduling execution.

[0053] The BS-side beam prediction module uses an LSTM network (LSTM-S). The input is a historical RSRP sequence window of length j=6, and the output is the RSRP prediction vector for the next h=4 time slots and the Top-K=4 beam indices. The BS collects the queue length for each UE. Current measured RSRP vector Top-4 beam index and predicted RSRP values ​​for the next 4 time slots, and one-hot coding of service type. Concatenate them into a user feature vector Then, stack the feature vectors of all UEs to form a global state vector. The BS-side decision model uses a DRL scheduler (DQN) architecture. Both the main network and the target network contain three fully connected layers (hidden layer dimensions of 1024, 512, and 256 respectively), with ReLU activation function. The action space includes scheduling decisions (whether to schedule the UE), beam selection (selecting from the Top-4 beams), and power allocation (discrete into 8 power levels). During training, experience samples are stored in the replay buffer, and network parameters are updated through batch sampling. The scheduling process is as follows: the UE collects and reports the current time slot RSRP data; the BS-side beam prediction module generates the Top-4 beam information for the next 4 time slots and reports it to the BS; the BS constructs a global state vector, and the DQN scheduler outputs optimization decisions; the BS configures downlink transmission parameters, and the UE completes beam alignment and data reception; the BS updates the UE queue status and QoS satisfaction, and... Stored in the replay buffer for model training.

[0054] The performance comparison of the scheduling method proposed in this invention is as follows: Figure 4 As shown in the figure. Under the same simulation conditions, compared with baseline methods such as CSI-only short-sighted RL and prediction-assisted greedy scheduling, our method outperforms other methods in terms of return rate, thus verifying the effectiveness of the method.

[0055] The embodiments of the present invention also have the following technical effects: Centralized beam prediction module: deployed on the BS side, used to receive beam measurement results reported by the UE, and maintain a length of [unclear] for each UE. A sliding window of historical RSRP sequences; based on the historical sequences, multi-step beam quality prediction is performed, and future beam quality is output. Predicted RSRP vectors and Top of each time slot A set of K candidate beam indices is used to provide forward-looking beam information for subsequent scheduling. Predictive-enhanced state construction: Beam prediction data is deeply integrated with queue state, service type, and QoS constraints to provide forward-looking information to the DRL scheduler, mitigating performance losses caused by channel aging. Adaptive scheduling strategy: Multi-service optimized scheduling decisions are made based on deep reinforcement learning, balancing throughput, latency, and reliability through a custom reward function to achieve QoS collaborative protection for heterogeneous services.

[0056] To achieve the above embodiments, such as Figure 5 As shown, this embodiment also provides a predictive enhanced QoS-aware millimeter-wave beam and power joint scheduling device 10, including: The historical RSRP acquisition and beam prediction module is used to acquire the historical RSRP measurement sequences reported by user equipment, and generate a Top-K candidate beam set and corresponding prediction quality for multiple future time slots based on the historical sequences through a time-series prediction model. The state vector construction module is used to fuse the Top-K candidate beam set with the queue status, service type identifier and QoS constraint parameters of each user equipment to construct a prediction-enhanced global state vector containing time-series prediction information. The beam and power decision module is used to perform beam selection and power allocation decisions based on the predicted enhanced global state vector using a deep reinforcement learning model. The decision process balances weighted spectral efficiency with URLLC latency violation penalty and mMTC service interruption penalty through a custom reward function. The parameter configuration and model optimization module is used to configure the downlink beamforming parameters and transmission power of the base station according to the decision results, and update the queue status and QoS satisfaction of user equipment to form training samples for model iterative optimization.

[0057] The present invention provides a predictive enhanced QoS-aware millimeter-wave beam and power joint scheduling method and apparatus, which can effectively alleviate channel aging problems in dynamic millimeter-wave channel environments, improve the foresight and QoS guarantee capabilities of beam and power scheduling, thereby significantly improving system spectrum efficiency and reducing URLLC delay violation probability and mMTC outage probability.

[0058] To implement the methods of the above embodiments, the present invention also provides a computer device, such as... Figure 6 As shown, the computer device 600 includes a memory 601 and a processor 602; wherein, the processor 602 reads the executable program code stored in the memory 601 to run a program corresponding to the executable program code, so as to implement the various steps of the predictive enhanced QoS-aware millimeter-wave beam and power joint scheduling method described above.

[0059] To implement the above embodiments, this application also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a predictive enhanced QoS-aware millimeter-wave beam and power joint scheduling method as described in the foregoing embodiments.

[0060] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0061] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A predictive enhanced QoS-aware millimeter-wave beam and power joint scheduling method, characterized in that, include: S1, obtain the historical RSRP measurement sequence reported by the user equipment, and generate a Top-K candidate beam set and corresponding prediction quality for multiple future time slots based on the historical sequence through a time-series prediction model; S2, the Top-K candidate beam set is fused with the queue status, service type identifier and QoS constraint parameters of each user equipment to construct a prediction-enhanced global state vector containing time-series prediction information; S3, based on the predicted enhanced global state vector, a deep reinforcement learning model is used to perform beam selection and power allocation decisions, wherein the decision-making process balances weighted spectral efficiency with URLLC delay violation penalty and mMTC service interruption penalty through a custom reward function; S4. Based on the decision results, configure the downlink beamforming parameters and transmission power of the base station, and update the queue status and QoS satisfaction of user equipment to form training samples for model iterative optimization.

2. The method as described in claim 1, characterized in that, S1 includes: Let X be the set of RSRP measurements reported by the UE. After j beam management cycles, the signal received at the BS can be represented as: ； BS predicts the beam quality vector and Top-K beam index for the next h time slots, with the objective function being: ； in For the beam prediction model, the optimization objective is to minimize the weighted sum of the prediction error and the Top-K classification loss: 。 3. The method as described in claim 1, characterized in that, The S2 includes: The method for constructing the prediction-enhanced global state vector is as follows: for each UE k User feature vectors are constructed as : in, Let t be the initial queue length for time slot t. The measured RSRP vector for time slot t. For future h-slot Top-K beam indexing, To correspond to the predicted RSRP value, One-hot encoding for service type; Stack the user feature vectors of all K UEs to form a global state vector. .

4. The method as described in claim 1, characterized in that, The S3 includes: The objective function and training method of the DRL scheduler are as follows: The scheduler adopts the DQN architecture, and the objective is to maximize the sum of long-term discounted rewards. Among them, instant rewards Defined as: For service type weights, For UE k The actual data rate, For the number of URLLC delay violations, For mMTCoutage times, , This is the penalty coefficient.

5. The method as described in claim 1, characterized in that, The S4 includes: The scheduler training employs an ε-greedy strategy, storing empirical samples through a replay buffer and stabilizing the training process by separating the target network from the main network. Power allocation decisions satisfy... 。 6. The method as described in claim 2, characterized in that, The method further includes: The number of antennas for BS and UE are respectively and The downlink channel matrix is ​​denoted as ,in For static multipath components, Dynamic factors caused by user movement.

7. The method as described in claim 1, characterized in that, The method further includes: The weight vector for beamforming can be expressed as: in, and These are the weights of the antenna elements in the vertical and horizontal directions of the UPA, respectively. It is the size of the beamforming codebook; The beam pair (m,n) signal received by the UE can be represented as: in, The average transmit power, , Beamforming vectors for BS and UE; To transmit pilot signals, For additive noise, the RSRP calculation formula for beam pair (m,n) is as follows: 。 8. A predictive enhanced QoS-aware millimeter-wave beam and power joint scheduling device, characterized in that, include: The historical RSRP acquisition and beam prediction module is used to acquire the historical RSRP measurement sequences reported by user equipment, and generate a Top-K candidate beam set and corresponding prediction quality for multiple future time slots based on the historical sequences through a time-series prediction model. The state vector construction module is used to fuse the Top-K candidate beam set with the queue status, service type identifier and QoS constraint parameters of each user equipment to construct a prediction-enhanced global state vector containing time-series prediction information. The beam and power decision module is used to perform beam selection and power allocation decisions based on the predicted enhanced global state vector using a deep reinforcement learning model. The decision process balances weighted spectral efficiency with URLLC latency violation penalty and mMTC service interruption penalty through a custom reward function. The parameter configuration and model optimization module is used to configure the downlink beamforming parameters and transmission power of the base station according to the decision results, and update the queue status and QoS satisfaction of user equipment to form training samples for model iterative optimization.

9. A computer device, characterized in that, Including processor and memory; The processor reads executable program code stored in the memory to run a program corresponding to the executable program code, so as to implement a predictive enhanced QoS-aware millimeter-wave beam and power joint scheduling method as described in any one of claims 1-7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements a predictive enhanced QoS-aware millimeter-wave beam and power joint scheduling method as described in any one of claims 1-7.

Images