A wireless resource allocation optimization apparatus and method

Abstract

The application discloses a wireless resource allocation optimization device and method, the device comprises: a data acquisition module for collecting channel data uploaded by a UE and realizing positioning of the UE; a data prediction module for realizing training of a neural network model and prediction of collected data; and a strategy optimization module for realizing allocation of subcarrier resources and selection of transmission rates. In the wireless resource allocation optimization device and method, inter-cell interference is introduced, different schemes are used to calculate channel capacity for edge UEs and center UEs, and the schemes are more in line with real systems; prediction technology is used for interference information and channel information, more accurate channel and interference information can be obtained, the problem that channel states in feedback are different from channel states at data transmission time is solved, subcarrier allocation and transmission rate selection are realized, and greater throughput can be obtained in the whole transmission time period.

Description

A wireless resource allocation optimization device and method Technical Field

[0001] This invention belongs to the field of mobile communication technology, and specifically relates to a wireless resource allocation optimization device and method. Background Technology

[0002] Wireless resource management refers to providing quality of service (QoS) guarantees for wireless user equipment (UE) under limited bandwidth conditions, flexibly allocating and dynamically adjusting available resources in the wireless transmission section and network, and maximizing wireless spectrum utilization. Conventional channel modeling for resource allocation problems often considers channel models for both useful and noisy signals, but rarely models interference.

[0003] Chinese invention patent, publication number CN112566253A, entitled "A Joint Optimization Method and Apparatus for Wireless Resource Allocation," discloses: modeling the application scenario of the MIMO network architecture to be optimized for resource allocation, obtaining the corresponding channel model of the MIMO network architecture; wherein, NOMA communication technology is used between user pairs, and zero-forcing beamforming technology is used at the transmitting end to encode the transmitted signal; based on the channel model, a deep reinforcement learning network is used for learning to realize the allocation of inter-cluster interference and intra-cluster interference in the MIMO network architecture; wherein, the goal of deep reinforcement learning is to maximize the sum rate of the second group of users while ensuring the minimum target data rate of the first group of users in the network architecture. However, this patent has the following problems: 1) The interference considered in this patent is the interference between different antennas, which can be understood as interference within the system, but it does not consider the interference of other cells to the serving cell, i.e., inter-cell interference. Inter-cell interference will affect the system throughput; 2) Rate selection. The patent considers the rate selection as continuous, but in reality, the rate selection is often discrete, that is, there are multiple rate alternatives, from which a suitable rate is selected for transmission.

[0004] Chinese invention patent, publication number CN114666900A, entitled "A Method and Apparatus for Wireless Resource Allocation," discloses a method applied to blockchain nodes. The method includes: receiving a service name and service characteristics sent by a terminal; performing demand mapping on the service based on the service name and service characteristics to obtain demand mapping information; and sending the demand mapping information to a base station so that the base station can allocate wireless resources according to the demand mapping information. However, this patent has the following problems: 1) It does not mention a method for rate selection, so how the rate is selected after resource allocation is unclear; 2) It requires the UE to report service information, and the base station to allocate corresponding downlink resources after receiving the relevant information. This requires the terminal to have corresponding upload attributes, thus placing requirements on the type and capabilities of the terminal. It cannot be directly used in existing terminals, networks, and protocols, and its application has certain limitations.

[0005] Chinese invention patent, publication number CN115052244A, entitled "Method and Apparatus for Wireless Resource Allocation of Base Station", discloses: the base station obtains the dynamic location information of user terminals.

[0006] The base station processes the dynamic location information to generate the user terminal's location factor, velocity factor, channel quality factor for data transmission between the user terminal and the base station, and data transmission delay factor at the first moment. Based on these parameters, the base station determines whether the triggering conditions for a radio resource adjustment strategy are met. When the triggering conditions are met, the resource adjustment strategy is updated. The base station then allocates radio resources to the user terminal according to the updated strategy. However, this patent has the following problems: 1) The channel and location information does not consider the impact of interference on parameters such as channel and delay, therefore it may not be applicable to locations or cells with strong interference; 2) It does not consider the time difference between measuring the channel and transmitting actual data. Since there is a time difference between the channel measurement time and the data transmission time, the channel capacity will differ within this time difference, potentially leading to further optimization of resource allocation.

[0007] For digital communication systems, throughput is one of the indicators for measuring their communication capabilities. It refers to the amount of data successfully transmitted per unit time to a network, device, port, virtual circuit, or other facility. In cellular network transmission, when the data transmission rate is less than the maximum allowable transmission rate of the transmission channel, the transmission is considered successful, and the throughput increases; otherwise, the transmission fails. Therefore, selecting an appropriate data transmission rate based on channel conditions is a method to increase throughput.

[0008] Data transmission rates are often dynamically changing, and due to communication protocol requirements, they are discrete. Therefore, it is necessary to select the maximum rate that satisfies the channel conditions as the transmission rate based on changes in the channel.

[0009] In the downlink resource allocation scheme, the downlink channel measurement method is as follows: the base station sends a reference signal to the terminal; the terminal receives the reference signal and solves for the channel state; the terminal sends the channel state back to the base station. From the above channel measurement process, it can be seen that because signal solving and state return take time, the returned channel state is the channel state at the measurement time rather than the arrival time. Therefore, it is necessary to predict the channel state at the current time. The channel state often refers to the channel capacity (C), i.e., the useful signal (S), interference signal (I), and noise (N), as well as the transmission channel bandwidth (B), which is obtained using the following Shannon theorem:

[0010]

[0011] Inter-cell interference refers to the disruption of communication for users at the cell edge in a cellular network caused by the higher power of neighboring cells on the same frequency. The main factors affecting its magnitude are the load and power of the neighboring cells on the same frequency. Summary of the Invention

[0012] To address the technical problems existing in the prior art, the present invention aims to provide a wireless resource allocation optimization device and method.

[0013] To achieve the above objectives and technical effects, the technical solution adopted by this invention is as follows:

[0014] A wireless resource allocation optimization device, comprising:

[0015] The data acquisition module is used to collect channel data uploaded by the UE and to locate the UE.

[0016] The data prediction module is used to perform training on neural network models and to predict the collected data.

[0017] The strategy optimization module is used to allocate subcarrier resources and select transmission rates.

[0018] Furthermore, the data acquisition module includes a UE location module, an interference data acquisition module, and a channel data acquisition module. The UE location module determines the UE's location (cell edge / cell center) by the difference between the serving cell reference signal received power (RSRP_S) and the neighboring cell reference signal received power (RSRP_N) reported by the UE. This result determines whether it is necessary to collect inter-cell interference data. The interference data acquisition module is responsible for collecting the intensity of inter-cell interference, that is, calculating the average power of the signals received by the UE from neighboring cells as the interference value, and recording the interference data in the interference database. The channel data acquisition module is responsible for collecting the average power value of the serving cell signal as the power value of the useful signal, and recording it in the channel database.

[0019] Furthermore, the data prediction module includes an interference data prediction module, a channel data prediction module, and a neural network model. The interference data prediction module is responsible for using Kalman filtering to predict interference in the time dimension from the database and the real-time collected interference data, and using the filtered data as the prediction data output. The channel data prediction module is used to generate a dataset in the time dimension from the channel data in the database using the sliding window method. The neural network model is responsible for training the model using the training set in the dataset, and then using the test set in the dataset to check the prediction accuracy. The model is considered to have been successfully trained when the error is less than the calibration threshold.

[0020] Furthermore, the neural network model has a three-layer structure: the first layer is a fully connected layer as the input layer, the second layer is a long short-term memory layer, and the third layer is a single neuron as the output layer. Each of the first and second layers has 64 neurons, and the activation function is a linear rectified function.

[0021] Furthermore, the strategy optimization module uses deep reinforcement learning to allocate subcarrier resources and select transmission rates, thereby enabling the training and use of reinforcement learning models. The strategy optimization module includes a transmission rate candidate set module and a transmission rate selection module.

[0022] Furthermore, the strategy optimization module uses deep reinforcement learning to allocate subcarrier resources and select transmission rates. The steps for training and using the reinforcement learning model include:

[0023] 1) Initialize the model's state space, action space, and deep reinforcement learning network parameters. The state space consists of the channel state, interference state, and average number of subcarriers already allocated for each subcarrier to be allocated. The action space consists of the number of subcarriers allocated at the current time and the transmission rate.

[0024] 2) Calculate the channel capacity based on the current state and select an action;

[0025] 3) Obtain the reward function, the next state, and store the experience;

[0026] 4) Determine whether to proceed to deep reinforcement learning based on the number of training iterations; if the number of training iterations does not meet the requirements, proceed to step 2).

[0027] Furthermore, the reward function is:

[0028]

[0029] Where R(t) is the transmission rate, C(t) is the channel capacity, and I(*) is the indicator function. I(R(t)≤C(t)) means that when the transmission rate is less than the transmission capacity, the indicator function is 1 and the transmission rate is included in the throughput; otherwise, it is not included. For the penalty term, Ω SC (τ) represents the number of subcarriers allocated at time t, σ sc The maximum number of subcarriers allowed to be transmitted.

[0030] This invention also discloses a method for optimizing wireless resource allocation, comprising the following steps:

[0031] Offline data acquisition and online resource allocation;

[0032] The steps for offline data acquisition include:

[0033] Step 1: Based on the RSRP_S and RSRP_N parameter information reported by the UE, calculate the difference and obtain its relative position relationship with the base station. Compare it with the threshold to obtain the location of the UE, whether it belongs to the cell edge UE or the cell center UE, and realize UE positioning.

[0034] Step 2: Obtain the UE's location from the previous step. If the UE is located at the cell edge, interference data is collected through the interference data acquisition module. If the UE is located at the cell center, channel data is collected through the channel data acquisition module.

[0035] Step 3: If interference data acquisition was performed in Step 2, this step proceeds to channel data acquisition; if channel data acquisition was performed in Step 2, this step is skipped.

[0036] Step 4: Store the data collected in Steps 2 and 3 for subsequent model training;

[0037] Step 5: The neural network model extracts the stored data and uses the sliding window method to generate the dataset in the time dimension. The dataset is divided into a training set and a test set. The divided dataset is then input into the neural network model for model training.

[0038] Step 6: The policy optimization module extracts the stored data and uses deep reinforcement learning methods to train the reinforcement learning model;

[0039] Step 7: Store the models trained in Steps 5 and 6 for later retrieval;

[0040] The steps for online resource allocation include:

[0041] Step 1: Based on the RSRP_S and RSRP_N parameter information reported by the UE, calculate the difference and obtain its relative position relationship with the base station. Compare it with the threshold to obtain the location of the UE, whether it belongs to the cell edge UE or the cell center UE, and realize UE positioning.

[0042] Step 2: Obtain the UE's location from the previous step. If the UE is located at the cell edge, interference data is collected through the interference data acquisition module. If the UE is located at the cell center, channel data is collected through the channel data acquisition module.

[0043] Step 3: If interference data acquisition was performed in Step 2, this step proceeds to channel data acquisition; if channel data acquisition was performed in Step 2, this step is skipped.

[0044] Step 4: Store the data collected in Steps 2 and 3 for subsequent model training;

[0045] Step 5: Input the collected interference data and channel data into the Kalman filter and neural network model respectively to obtain the predicted interference data and predicted data. Use the predicted data and Shannon's theorem to calculate the channel capacity and obtain the predicted channel capacity.

[0046] Step 6: Input the initial channel capacity into the reinforcement learning model, and use the model to allocate subcarriers and select the transmission rate to realize wireless resource allocation.

[0047] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0048] 1. This invention introduces inter-cell interference and incorporates inter-cell interference information into capacity calculation. By introducing inter-cell interference, different schemes are used to calculate channel capacity for UEs at the cell edge and UEs at the cell center, which is more consistent with the real system. Prediction technology is used for interference and channel information to obtain more accurate channel and interference information. This solves the problem that the channel state fed back by the UE changes with the channel state at the time of data transmission. In contrast, the existing technology uses the channel and interference information fed back by the UE directly as the channel and interference at the time of data transmission, which has a reporting time difference with the real information.

[0049] 3. This invention employs a deep reinforcement learning method to achieve subcarrier allocation and transmission rate selection, which enables greater throughput throughout the entire transmission period. Attached Figure Description

[0050] Figure 1 is a working block diagram of the present invention;

[0051] Figure 2 is a flowchart of the present invention;

[0052] Figure 3 is a schematic diagram of the strategy optimization module of the present invention;

[0053] Figure 4 is a result diagram of Embodiment 1 of the present invention. Detailed Implementation

[0054] The present invention will now be described in detail so that its advantages and features can be more easily understood by those skilled in the art, thereby providing a clearer and more explicit definition of the scope of protection of the present invention.

[0055] The following provides a brief overview of one or more aspects to offer a basic understanding of them. This overview is not an exhaustive summary of all conceived aspects, nor is it intended to identify key or decisive elements of all aspects, nor to define the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form to prepare for the more detailed descriptions that follow.

[0056] As shown in Figures 1-4, a wireless resource allocation optimization device includes:

[0057] The data acquisition module is primarily responsible for collecting channel data uploaded by the UE, including the UE location module, interference data acquisition module, and channel data acquisition module. The UE location module determines the UE's location (cell edge / cell center) by the difference between the serving cell's reference signal received power (RSRP_S) and the neighboring cell's reference signal received power (RSRP_N). This result determines whether inter-cell interference data needs to be collected subsequently. The interference data acquisition module is responsible for collecting the intensity of inter-cell interference, i.e., calculating the average power of the signals received by the UE from neighboring cells as the interference value, and recording the interference data in the interference database. The channel data acquisition module is responsible for collecting the average power value of the serving cell signal as the power value of the useful signal and recording it in the channel database.

[0058] The data prediction module is used to train the neural network model and predict acquired data. It includes an interference data prediction module, a channel data prediction module, and the neural network model itself. The neural network model is the prediction model. Specifically, the interference data prediction module uses Kalman filtering to predict interference in the time dimension from the database and real-time acquired interference data, outputting the filtered data as the prediction data. The channel data prediction module uses a sliding window method to generate a dataset from the channel data in the database in the time dimension. The dataset is divided into training and test sets. The neural network model is trained using the divided training set and then tested for prediction accuracy using the test set. If the error is less than a calibrated threshold, the model is considered successfully trained. The trained model is then used to predict channel data. The neural network model has a three-layer structure: the first layer is a fully connected layer as the input layer; the second layer is a Long Short-Term Memory (LSTM) layer, each with 64 neurons and the activation function is ReLU; the third layer is a single neuron as the output layer.

[0059] The strategy optimization module utilizes deep reinforcement learning to allocate subcarrier resources and select transmission rates. It implements the training and use of the reinforcement learning model, and its main steps include:

[0060] 1) Initialize the model's state space, action space, and deep reinforcement learning network parameters. The state space consists of the channel state, interference state, and average number of subcarriers already allocated for each subcarrier to be assigned. The action space consists of the number of subcarriers allocated at the current time and the transmission rate.

[0061] 2) Calculate the channel capacity based on the current state and select an action;

[0062] 3) Obtain the reward function, the next state, and store the experience;

[0063] 4) Determine whether to proceed to deep reinforcement learning based on the number of experiences. If the number of experiences meets the requirement, proceed to deep reinforcement learning; otherwise, proceed to step 3). If the number of training sessions does not meet the requirement, proceed to step 2); otherwise, the entire process ends.

[0064] The reward function is:

[0065]

[0066] Where R(t) is the transmission rate, C(t) is the channel capacity, and I(*) is the indicator function. I(R(t)≤C(t)) means that when the transmission rate is less than the transmission capacity, the indicator function is 1 and the transmission rate is included in the throughput; otherwise, it is not included. For the penalty term, Ω SC (τ) represents the number of subcarriers allocated at time t, σsc The maximum number of subcarriers allowed to be transmitted;

[0067] By training a deep reinforcement learning model using historical data and obtaining better model parameters, it can be applied to the actual subcarrier allocation and transmission rate selection decision scheme.

[0068] Specifically, the strategy optimization module includes a transmission rate candidate set module and a transmission rate selection module. The transmission rate candidate set module contains a set of transmission rates allowed by various protocols, meaning that the final transmission rate can only be selected from the rates specified in the set. The transmission rate selection module contains a rate optimization algorithm based on deep reinforcement learning.

[0069] A method for optimizing wireless resource allocation includes the following steps:

[0070] It is mainly divided into an offline data acquisition process (the dashed flow in Figure 2) and an online resource allocation process. The offline data acquisition steps include:

[0071] Step 1: Based on the RSRP_S and RSRP_N parameter information reported by the UE, calculate the difference and obtain its relative position relationship with the base station. Compare it with the threshold to obtain the location of the UE, whether it belongs to the cell edge UE or the cell center UE, and realize UE positioning.

[0072] Step 2: Obtain the UE's location from the previous step. If the UE is located at the cell edge, interference data is collected through the interference data acquisition module. If the UE is located at the cell center, channel data is collected through the channel data acquisition module.

[0073] Step 3: If interference data acquisition was performed in Step 2, this step proceeds to channel data acquisition; if channel data acquisition was performed in Step 2, this step is skipped.

[0074] Step 4: Store the data collected in Steps 2 and 3 for subsequent model training;

[0075] Step 5: The neural network model extracts the stored data and uses the sliding window method to generate the dataset in the time dimension. The dataset is divided into a training set and a test set. The divided dataset is then input into the neural network model for model training.

[0076] Step 6: The policy optimization module extracts the stored data and uses deep reinforcement learning methods to train the reinforcement learning model;

[0077] Step 7: Store the models trained in Steps 5 and 6 for later retrieval;

[0078] The online resource allocation process includes:

[0079] Step 1: Based on the RSRP_S and RSRP_N parameter information reported by the UE, calculate the difference and obtain its relative position relationship with the base station. Compare it with the threshold to obtain the location of the UE, whether it belongs to the cell edge UE or the cell center UE, and realize UE positioning.

[0080] Step 2: Obtain the UE's location from the previous step. If the UE is located at the cell edge, interference data is collected through the interference data acquisition module. If the UE is located at the cell center, channel data is collected through the channel data acquisition module.

[0081] Step 3: If interference data acquisition was performed in Step 2, this step proceeds to channel data acquisition; if channel data acquisition was performed in Step 2, this step is skipped.

[0082] Step 4: Store the data collected in Steps 2 and 3 for subsequent model training;

[0083] Step 5: Input the collected interference data and channel data into the Kalman filter and neural network model respectively to obtain the predicted interference data and predicted data. Use the predicted data and Shannon's theorem to calculate the channel capacity and obtain the predicted channel capacity.

[0084] Step 6: Input the initial channel capacity into the reinforcement learning model, and use the model to allocate subcarriers and select the transmission rate to realize wireless resource allocation.

[0085] Example 1

[0086] 1. Data: Channel data was generated using a Markov model, with 100 channel states set up, and their numerical distribution following the Rayleigh channel distribution law. For interference data, two interference states were defined: interference (represented by 1) and no interference (represented by 0). When interference is present, the interference magnitude is 0.8, and data was also generated using a Markov model. 10,000 time points were generated for each type of data for model training.

[0087] 2. Neural Network Model Training: For prediction with interfering data, a sliding window of length 5 in the time domain is used to generate the training and test sets. The training epochs are set to 500, and the batch size for each training iteration is 64.

[0088] 3. Reinforcement learning parameter settings: Set the communication duration to 100 time points, the maximum number of subcarriers allowed to be allocated to 20, and the number of training rounds for reinforcement learning to 1000 rounds. Use the generated data and the set parameters to conduct simulation experiments.

[0089] 4. Comparative Experiments. Two comparative experiments were set up. Method 1: Data prediction was performed, but the resource allocation scheme was not used, and a general maximum allocation scheme was adopted. Method 2: Data prediction was not performed, and the channel capacity was directly calculated using the collected data, and the resource allocation was performed using a reinforcement learning method.

[0090] 5. Experimental Results. Using the same data and configuration parameters, the two comparative experiments described above were conducted. The results are shown in Figure 4. Clearly, the reward value of the wireless resource allocation optimization method proposed in this invention is the best among the three schemes, followed by Method 1, and Scheme 2 is the worst. The scheme proposed in this invention can obtain a better reward in the reinforcement learning method, while Method 1 does not use reinforcement learning. Therefore, when comparing rewards, the penalty term of the reward function is set to 0 to make them comparable.

[0091] Table 1 below shows the average throughput and average number of subcarriers occupied for each of the three methods at each time point.

[0092] Table 1

[0093]

[0094] As shown in Table 1, the method proposed in this invention has a higher average throughput per moment, improving by 6.43% compared to Method 1 and by 139.26% compared to Method 2. The method proposed in this invention has an average subcarrier usage of 19.79, meeting the requirement of less than 20, and is also higher than the usage of Method 2. In summary, the method proposed in this invention has the best performance and can achieve optimized resource allocation.

[0095] Any parts or structures not specifically described in this invention can be made using existing technologies or products, and will not be elaborated upon here.

[0096] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A wireless resource allocation optimization device, characterized in that, include: The data acquisition module is used to collect channel data uploaded by the UE and to locate the UE. The data prediction module is used to perform training on neural network models and to predict the collected data. The strategy optimization module is used to allocate subcarrier resources and select transmission rates. The data acquisition module includes a UE location module, an interference data acquisition module, and a channel data acquisition module. The UE location module determines the UE's location by the difference between the serving cell reference signal received power (RSRP_S) reported by the UE and the reference signal received power (RSRP_N) of neighboring cells, classifying it as either a cell edge UE or a cell center UE. This result determines whether inter-cell interference data needs to be collected. The interference data acquisition module is responsible for collecting the intensity of inter-cell interference, i.e., calculating the average power of the signals received by the UE from neighboring cells as the interference value, and recording the interference data in the interference database. The channel data acquisition module is responsible for collecting the average power value of the serving cell signal as the power value of the useful signal, and recording it in the channel database. The data prediction module includes an interference data prediction module, a channel data prediction module, and a neural network model. The interference data prediction module is responsible for using Kalman filtering to predict interference in the time dimension from the database and real-time collected interference data, and outputting the filtered data as the prediction data. The channel data prediction module generates a dataset in the time dimension from the channel data in the database using a sliding window method. The neural network model is responsible for training the model using the training set in the dataset, and then verifying the prediction accuracy using the test set in the dataset. The model is considered successfully trained when the error is less than a calibration threshold. The policy optimization module uses deep reinforcement learning to allocate subcarrier resources and select transmission rates, enabling the training and use of the reinforcement learning model. The policy optimization module includes a transmission rate candidate set module and a transmission rate selection module. The strategy optimization module uses deep reinforcement learning to allocate subcarrier resources and select transmission rates. The steps for training and using the reinforcement learning model include: 1) Initializing the model's state space, action space, and deep reinforcement learning network parameters. The state space contains the channel state, interference state, and average number of allocated subcarriers for each subcarrier to be allocated. The action space contains the number of subcarriers allocated at the current time and the transmission rate; 2) Calculating the channel capacity and selecting actions based on the current state; 3) Obtaining the reward function, the next state, and storing the experience; 4) Determining whether to proceed to deep reinforcement learning based on the amount of experience. If the number of training iterations is insufficient, proceed to step 2). The reward function is: ;in, For transmission rate, For channel capacity, For indicator functions, This indicates that when the sending rate is less than the transmission capacity, the indicator function is 1, and the transmission rate is included in the throughput; otherwise, it is not. For the penalty item, among which The number of subcarriers allocated at time t. The maximum number of subcarriers allowed to be transmitted.

2. The wireless resource allocation optimization device according to claim 1, characterized in that, The neural network model has a three-layer structure: the first layer is a fully connected layer as the input layer, the second layer is a long short-term memory layer, and the third layer is a single neuron as the output layer. Each of the first and second layers has 64 neurons, and the activation function is a linear rectified function.

3. A method for optimizing wireless resource allocation, characterized in that, Includes the following steps: Offline data acquisition and online resource allocation; The offline data acquisition steps include: Step 1: Based on the RSRP_S and RSRP_N parameter information reported by the UE, calculate the difference and obtain its relative positional relationship with the base station. Compare it with a threshold to obtain the UE's location, whether it belongs to the cell edge UE or the cell center UE, thus achieving UE positioning; Step 2: Based on the UE's location obtained in the previous step, if it is located as a cell edge UE, then perform interference data acquisition through the interference data acquisition module; if it is located as a cell center UE, then perform channel data acquisition through the channel data acquisition module; Step 3: If interference data acquisition was performed in Step 2, then this step proceeds to channel data acquisition; if channel data acquisition was performed in Step 2, this step is skipped; Step 4: Store the data acquired in Steps 2 and 3 for subsequent model training; Step 5: The neural network model extracts the stored data and utilizes it... The sliding window method is used to generate the dataset in the time dimension. The dataset is divided into a training set and a test set. The divided dataset is input into the neural network model for model training. Step 6: The policy optimization module extracts the stored data and uses deep reinforcement learning to train the reinforcement learning model. Specifically, it includes the following steps: 1) Initialize the model's state space, action space, and deep reinforcement learning network parameters. The state space consists of the channel state, interference state, and average number of subcarriers already allocated for each subcarrier to be allocated. The action space consists of the number of subcarriers allocated at the current time and the transmission rate. 2) Calculate the channel capacity and select actions based on the current state. 3) Obtain the reward function, the next state, and store the experience. 4) Determine whether to enter deep reinforcement learning based on the number of experience points. If the number of training iterations does not meet the requirements, proceed to step 2). The reward function is: ;in, For transmission rate, For channel capacity, For indicator functions, This indicates that when the sending rate is less than the transmission capacity, the indicator function is 1, and the transmission rate is included in the throughput; otherwise, it is not. For the penalty item, among which The number of subcarriers allocated at time t. The maximum number of subcarriers allowed for transmission; Step 7: Store the models trained in Steps 5 and 6 for later retrieval; The online resource allocation steps include: Step a: Based on the RSRP_S and RSRP_N parameter information reported by the UE, calculate the difference and obtain its relative positional relationship with the base station, compare it with the threshold to obtain the UE's location, whether it belongs to the cell edge UE or the cell center UE, and realize UE positioning; Step b: Based on the UE's location obtained in the previous step, if it is located as a cell edge UE, then interference data is collected through the interference data acquisition module; if it is located as a cell center UE, then channel data is collected through the channel data acquisition module. Step c: If interference data acquisition was performed in step b, then this step proceeds to channel data acquisition; if channel data acquisition was performed in step b, this step is skipped. Step d: Store the data acquired in steps b and c for subsequent model training. Step e: Input the acquired interference data and channel data into the Kalman filter and neural network model respectively to obtain the predicted interference data and predicted data. Use the predicted data and Shannon's theorem to calculate the channel capacity and obtain the predicted channel capacity. Step f: Input the initial channel capacity into the reinforcement learning model and use the model to allocate subcarriers and select the transmission rate to achieve wireless resource allocation.

Images