A Hybrid Beamforming and Resource Allocation Method and System for MIMO-OFDM Networks

Abstract

This invention discloses a hybrid beamforming and resource allocation method and system for MIMO-OFDM networks, belonging to the technical field of resource allocation for multi-user MIMO-OFDM networks. The method includes: establishing a single-cell downlink multi-user large-scale MIMO-OFDM system using a hybrid beamforming architecture; jointly optimizing analog precoding, digital precoding, subcarrier allocation, and power control, proposing a hierarchical deep reinforcement learning framework for solution; modeling analog precoding as MMDP, using a GRPO-based algorithm for decision-making, and introducing a fine-tuning step after training to further optimize the analog precoding; given the analog precoding, using a VDN-based method for subcarrier allocation; under the analog precoding and subcarrier allocation strategies given in the framework, first performing zero-forcing digital precoding, then using the Lagrange multiplier method to optimize power control, and calculating the reward accordingly; finally, obtaining the optimal strategy for joint hybrid beamforming and resource allocation. This solves the practical hybrid precoding and resource allocation problem, maximizes resource utilization, and improves system performance and speed.

Description

Technical Field

[0001] This invention belongs to the field of resource allocation technology for multi-user MIMO-OFDM networks, specifically relating to a hybrid beamforming and resource allocation method and system for MIMO-OFDM networks. Background Technology

[0002] To meet the ever-increasing demand for data rates, massive MIMO networks have attracted widespread attention due to their ability to improve system throughput through beamforming. To overcome hardware limitations, a hybrid analog-digital beamforming architecture has been proposed, enabling multiple antennas to share a limited number of radio frequency chains. Given that beamforming is typically designed for flat fading channels, while real-world wireless channels are often wideband and frequency-selective, OFDM technology is introduced to decompose wideband channels into approximately flat subcarriers. Numerous hybrid beamforming schemes have been proposed for massive MIMO-OFDM systems. To further improve system performance and resource utilization efficiency, the joint optimization problem of resource allocation and hybrid beamforming in multi-user massive MIMO-OFDM systems has also been studied.

[0003] Despite advancements in hybrid beamforming and resource allocation schemes for multi-user massive MIMO-OFDM systems, several key issues remain unresolved. First, existing subcarrier allocation and power control typically rely on the assumption that each subcarrier serves a fixed number of users and that base station transmit power is evenly distributed across all subcarriers. This assumption reduces the flexibility of resource utilization, thereby weakening system performance. Second, existing solutions largely depend on optimization algorithms, requiring repeated solutions under different channel conditions, resulting in high complexity and limiting practical deployment. Summary of the Invention

[0004] To address the problems existing in the prior art, this invention provides a hybrid beamforming and resource allocation method for MIMO-OFDM networks to improve system performance and speed.

[0005] To achieve the above objectives, in a first aspect, the present invention provides: 1. A method for hybrid beamforming and resource allocation in a MIMO-OFDM network, comprising the following steps: A single-cell downlink multi-user massive MIMO-OFDM system model is constructed, wherein the base station is configured with a hybrid beamforming architecture, the number of users served by each subcarrier is flexibly variable, and the transmit power of the base station is flexibly allocated among the subcarriers; Based on the system model, a joint optimization of the analog precoding matrix is ​​constructed with the goal of maximizing system performance and rate. Digital precoding matrix Subcarrier allocation matrix and power control matrix The optimization problem; A hierarchical deep reinforcement learning framework is constructed to solve the optimization problem, resulting in a hybrid beamforming and resource allocation strategy. When constructing a hierarchical deep reinforcement learning framework to solve the optimization problem, the simulated precoding problem is placed at the upper layer, modeled as a Multimarker process, and an algorithm based on group-related policy optimization is used for decision-making. Given analog precoding, the lower layer performs subcarrier allocation using a value decomposition network-based method; The training phase adopts a bottom-up process, while the execution phase adopts a top-down process.

[0006] Furthermore, the optimization problem specifically refers to:

[0007] in, The maximum transmit power of the base station is represented by the first constraint in the problem, which reflects the hardware limitations of the fully connected, single-phase shifter architecture; the third constraint limits the number of users that can be served on each subcarrier to the number of radio chains; and the fourth constraint ensures that the total power allocated to the base station does not exceed the maximum transmit power. Represents the simulated beamforming matrix, where and These represent the number of antennas and the number of radio frequency chains at the transmitting end, respectively. Indicates subcarrier The digital beamforming matrix on, where Indicates the use of subcarriers Number of users Denotes the channel selection matrix, where Indicates user Does it occupy a subcarrier? Indicator variables, Indicates subcarrier The power allocation matrix on, Indicates allocation to user In subcarrier The power level on This is the user calculated according to Shannon's formula. In subcarrier The transmission rate on Indicates user In subcarrier Digital beamforming vector on This represents a predefined discrete Fourier transform codebook.

[0008] Furthermore, a hierarchical deep reinforcement learning framework is constructed to solve the optimization problem, resulting in hybrid beamforming and resource allocation strategies, including: The simulation precoding problem is placed at a higher level, modeled as a Multimarker process, and a decision is made using an algorithm based on group correlation strategy optimization. Given analog precoding, the lower layer performs subcarrier allocation using a value decomposition network-based method; Under the analog precoding and subcarrier allocation strategy given in the hierarchical deep reinforcement learning framework, zero-forcing digital precoding is first performed, followed by power control optimization using the Lagrange multiplier method, and the reward used for training is calculated accordingly. The training phase adopts a bottom-up process: first, the lower-layer subcarrier allocation strategy is trained under zero-forcing digital precoding and power control, which provides a reward for the training of the upper-layer analog precoding strategy; The execution phase employs a top-down process: first, analog precoding is determined, then subcarrier allocation is performed, and finally, zero-forcing digital precoding and power control are used.

[0009] Furthermore, the lower-layer subcarrier allocation is modeled as a multi-agent MDP, where each agent corresponds to one subcarrier and selects the user it serves, as follows: State: Subcarriers are constructed based on the singular values ​​of the equivalent channel after simulated precoding and the correlation between users. state, Specifically, let Indicates user In subcarrier The equivalent channel after analog precoding, user In subcarrier The singular values ​​on are denoted as ,user and ( In subcarrier The correlation coefficient on is defined as:

[0010] subcarrier The status is:

[0011] The global state can be represented as ; Action: Subcarrier The action is defined as a binary indicator variable selected by the user. The joint action is defined as the subcarrier allocation matrix. ; Reward: After an action is taken, all agents receive an immediate reward, which is shared among all agents. The reward is defined as follows:

[0012] The training process for subcarrier allocation based on value decomposition networks is conducted in a centralized manner, while execution remains decentralized.

[0013] Furthermore, the simulation precoding problem is modeled as a Multimarker process as follows: The state of a DoMARK process includes a shared state and There are several local states, and a shared state acquired by all agents. This shared state characterizes each user's preference for each analog beam in the DFT codebook, denoted as . ,user For analog beam Preference value Defined as using analog beams For users The reference signal received power obtained when beamforming the channel is expressed as:

[0014] in, Indicates base station and user In subcarrier Channel gain on The first character in the Discrete Fourier Transform codebook represents the... Each simulated beam weight; intelligent agent The local state is defined as the state of the agent. In the action steps, the current joint simulated beam selection vector of all radio frequency chains, and the joint state of all agents can be represented as: ; Action: Each agent takes an action sequentially under shared parameters. Action defined as radio frequency chain Selected analog beam index The joint action of all agents is represented as , Reward: For the constructed Multimarker process, the reward is defined as the achieved system and rate, expressed as:

[0015] Using neural networks Parameterized sharing strategy ,in Represents network parameters for the agent. Neural Networks by state As input, and output the corresponding action probability distribution, in a given state In this case, intelligent agent The actions are sampled from the learned distribution, and a group-related policy optimization algorithm is used to train the policy network. The final policy is obtained through iteration. In a given shared state Based on strategy The probability distribution is obtained, and each agent sequentially samples the actions to obtain the simulated beam selection strategy and the simulated precoding matrix.

[0016] Furthermore, the fine-tuning step introduced after policy training based on group-related policy optimization includes: For radio frequency chain Keep all other analog beams selected by the RF chain fixed, and then evaluate all the unselected analog beams; For candidate simulated beams to form candidate solutions And the corresponding rewards The computation can be performed in parallel; the simulated beam selection strategy with the highest reward is selected to obtain the final simulated beam selection strategy and simulated precoding matrix based on fine-tuning.

[0017] In a second aspect, the present invention provides a communication system comprising a base station and several terminals, wherein the base station is a multi-transmit antenna base station, and each terminal is equipped with a receiving antenna. During communication, the system performs hybrid beamforming and resource allocation based on the hybrid beamforming and resource allocation method described in any one of claims 1 to 7.

[0018] Thirdly, the present invention also provides a hybrid beamforming and resource allocation system for MIMO-OFDM networks, including a system model construction module, an optimization problem construction module, and a solution module; The system model building module is used to build a single-cell downlink multi-user massive MIMO-OFDM system model, in which the base station is configured with a hybrid beamforming architecture, the number of users served by each subcarrier is flexibly variable, and the transmit power of the base station is flexibly allocated among the subcarriers; The optimization problem construction module is used to construct a joint optimization simulation precoding matrix based on the system model, with the objective of maximizing system performance and rate. Digital precoding matrix Subcarrier allocation matrix and power control matrix The optimization problem; The solution module is used to construct a hierarchical deep reinforcement learning framework to solve the optimization problem and obtain a hybrid beamforming and resource allocation strategy. In the solution module, the analog precoding problem is placed at the upper layer, modeled as a multi-Markov process, and a decision is made using an algorithm based on group correlation strategy optimization. Under a given analog precoding, the lower layer performs subcarrier allocation through a value decomposition network-based method. The training phase adopts a bottom-up process, while the execution phase adopts a top-down process.

[0019] Fourthly, the present invention can provide a computer device including a processor and a memory, the memory being used to store a computer executable program, the processor reading the computer executable program from the memory and executing it, and the processor executing the computer executable program can realize the above-mentioned hybrid beamforming and resource allocation method of MIMO-OFDM network.

[0020] Simultaneously, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, it can implement the above-described hybrid beamforming and resource allocation method for MIMO-OFDM networks.

[0021] Compared with the prior art, the present invention has at least the following beneficial effects: The present invention considers a more practical hybrid beamforming and resource allocation scenario, that is, the number of users served by each subcarrier is flexibly adjustable, and the base station transmit power is flexibly allocated among the subcarriers; it jointly optimizes hybrid beamforming and resource allocation to maximize system throughput; the present invention proposes an HDRL architecture to solve the optimization problem, demonstrating strong adaptability to dynamic channel conditions; it uses GRPO and VDN methods to solve the optimization problem and introduces fine-tuning to avoid getting trapped in poor local optima. Attached Figure Description

[0022] Figure 1 The curves showing the system and rate of the present invention versus the signal-to-noise ratio under the method of the present invention and the comparative scheme are shown.

[0023] Figure 2 The curves showing the system and rate of the present invention and the comparative scheme as a function of the number of users are shown.

[0024] Figure 3 The curves showing the system and rate variations with the number of subcarriers under the method of this invention and the comparative scheme are shown. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] Consider a single-cell downlink multi-user massive MIMO-OFDM system, serving A single-antenna user, represented as The base station is equipped with Root transmitting antenna and The root RF chain, each RF chain connects to all transmit antennas, and has The transmitting antennas are arranged in a matrix. Assume the base station includes... Subcarriers, denoted as The subcarrier allocation matrix is ​​represented as follows: , Specifically, Indicates user Subcarrier occupied ,otherwise, Considering the constraints of the radio frequency chain, there are ,in Indicates subcarrier The number of users of the service.

[0027] Considering the hybrid beamforming architecture in a multi-user massive MIMO-OFDM system, for subcarriers , data vector First, through a power module Then through a digital baseband pre-encoder After performing the inverse fast Fourier transform and adding a cyclic prefix, an analog RF beamforming matrix is ​​used. Furthermore, this matrix is ​​identical for all subcarriers. Additionally, the analog radio frequency beamforming matrix... Each column in the code is derived from a predefined DFT codebook. The selection is made from the subcarriers. Therefore, after precoding, the subcarriers are selected from the subcarriers. The uplink and downlink transmission signals can be represented as .

[0028] To characterize the scattering characteristics of the wireless channel, the Saleh–Valenzuela channel model is adopted. The base station and the... The user in the first The channel gain on each subcarrier is defined as:

[0029] in, and These represent the number of clusters and the number of paths within each cluster, respectively. Indicates user In cluster Middle path Gain on, and These represent the departure azimuth angle and the departure pitch angle, respectively. Therefore, the user In subcarrier The received signal can be represented as:

[0030] in, yes Corresponding user in China column vectors, For vectors The Each component represents a component on a subcarrier. Send to user symbols, Indicates user In subcarrier Additive white Gaussian noise. Then, the user... In subcarrier The reachable rate on can be calculated as:

[0031] in, Indicates the subcarrier channel bandwidth. Indicates user In subcarrier The interference received can be represented as:

[0032] Construct optimization problem The goal is to jointly optimize the analog precoding matrix. Digital precoding matrix Subcarrier allocation matrix and power control matrix To maximize the system and speed. Therefore, the optimization problem can be mathematically formulated as:

[0033] in, This represents the maximum transmit power of the base station. In the problem, the first constraint reflects the hardware limitations of the fully connected, single-phase shifter architecture; the third constraint limits the number of users that can be served on each subcarrier to the number of radio chains; and the fourth constraint ensures that the total power allocated to the base station does not exceed the maximum transmit power. Represents the simulated beamforming matrix, where and These represent the number of antennas and the number of radio frequency chains at the transmitting end, respectively. Indicates subcarrier The digital beamforming matrix on, where Indicates the use of subcarriers Number of users Denotes the channel selection matrix, where Indicates user Does it occupy a subcarrier? Indicator variables, Indicates subcarrier The power allocation matrix on, Indicates allocation to user In subcarrier The power level on This is the user calculated according to Shannon's formula. In subcarrier The transmission rate on Indicates user In subcarrier Digital beamforming vector on This represents a predefined discrete Fourier transform codebook.

[0034] Hybrid Beamforming and Resource Allocation Scheme Based on Hierarchical Deep Reinforcement Learning Because the problem is a high-dimensional mixed-integer nonlinear programming problem, finding the optimal solution is difficult. In the joint hybrid beamforming and resource allocation architecture, analog precoding determines the performance ceiling by directing the beam to the served users; given the analog precoding, digital precoding and resource allocation further improve the system and rate by managing inter-user interference. To address this, a hierarchical deep reinforcement learning (HDRL) framework is proposed: the analog precoding problem is placed at the upper layer, modeled as a multi-Markov decision process (MMDP), and a group relative policy optimization (GRPO)-based algorithm is used for decision-making; given the analog precoding, the lower layer performs subcarrier allocation using a value decomposition network (VDN)-based method. Furthermore, given the analog precoding and subcarrier allocation strategies within the hierarchical deep reinforcement learning framework, zero-forcing (ZF) digital precoding is first performed, followed by power control optimization using the Lagrange multiplier method, and the reward for training is calculated accordingly. The framework employs a bottom-up process during the training phase: first, it trains the lower-layer subcarrier allocation strategy under ZF digital precoding and power control to provide rewards for training the upper-layer analog precoding strategy; while in the execution phase, it adopts a top-down process: first, it determines the analog precoding, then performs subcarrier allocation, and finally employs ZF digital precoding and power control. This phased design ensures both the stability of the training process and improves the effectiveness of problem solving.

[0035] Upper-level analog precoding scheme based on GRPO: Given that the action space of the analog precoding problem is... Due to its high dimensionality, it is modeled as a sequentially executed MMDP to handle this complexity. Each radio frequency chain acts as a homogeneous agent and shares policy parameters. The MMDP is defined as follows: (1) State: The state of MMDP includes a shared state and A local state. Specifically, this shared state can be acquired by all agents and is used to characterize each user's preference for each analog beam in the DFT codebook, denoted as . , .user For analog beam Preference value Defined as using analog beams For users The reference signal received power obtained when beamforming the channel can be expressed as:

[0036] in, Indicates base station and user In subcarrier Channel gain on The first character in the Discrete Fourier Transform codebook represents the... Each simulated beam weight.

[0037] In addition to sharing the state, each agent can also observe its local state. ( The local state of an agent is defined as the state of the agent within the agent's local state. In the action steps, the current joint analog beam selection vector of all RF chains, i.e. .in Indicates radio frequency chain The selected beam number. Because the agents execute actions sequentially, therefore... , Therefore, intelligent agents The observable state can be represented as The joint state of all agents can be represented as .

[0038] (2) Actions: Each agent takes actions sequentially under shared parameters. Action defined as radio frequency chain Selected analog beam index It is selected from the set of DFT codebook candidate beams that have not yet been selected. Accordingly, the joint action of all agents can be represented as... .

[0039] (3) Reward: For the constructed MMDP, the reward needs to be calculated after all agents have determined their actions. The reward is defined as the implemented system and rate, and can be expressed as:

[0040] in, Indicates user Does it occupy a subcarrier? Indicator variables, This is the user calculated according to Shannon's formula. In subcarrier The transmission rate on the device.

[0041] It can be observed that this return depends not only on the upper-layer analog beam selection action, but also on the lower-layer subcarrier allocation and optimized digital precoding and power control.

[0042] Using neural networks Parameterized sharing strategy ,in This represents network parameters. For the agent... , by state As input, and output the corresponding action probability distribution, i.e. In a given state In this case, intelligent agent The action is to sample from the learned distribution.

[0043] To reduce training costs and enhance learning stability, this invention employs the GRPO algorithm to train the policy network. Specifically, in each training round, firstly, random samples are drawn... 1 experience. Each experience Depend on The decision-making process can be represented as follows: Among them, regarding the state According to the current strategy Collect a set of actions Then you can get a set of corresponding rewards. Based on the collected experience, the policy network is optimized by maximizing the following formula: in and It's a hyperparameter. This indicates that entropy-based regularization terms are used to encourage exploration. and Representing intelligent agents respectively In the group and experience Importance weights and advantage functions in the context of [the topic] Indicates the number of experience points. Indicates the number of decision groups. This indicates the number of radio frequency chains. Specifically, and They can be represented as follows:

[0044] and

[0045] go through Through multiple iterations, the final strategy can be obtained. In a given shared state Based on strategy The probability distribution is such that each agent sequentially samples actions. Therefore, the final simulated beam selection strategy... It can be obtained, and the final analog precoding matrix can be represented as .

[0046] While modeling the analog precoding problem as MMDP significantly reduces complexity, it also introduces a limitation: actions taken by previous radio frequency chains do not consider potential choices for subsequent chains, potentially leading the algorithm to converge to a suboptimal solution. To further improve performance with acceptable computational overhead, a fine-tuning step is introduced after training the GRPO-based policy. Specifically, for the radio frequency chain... Keep the analog beams selected by all other RF chains fixed, i.e. Then evaluate all the unselected analog beams, i.e. For candidate simulated beams Candidate solutions can be formed. And the corresponding rewards The computation can be performed in parallel. Then, the simulated beam selection strategy with the highest reward is selected, which can be expressed as: Therefore, a final analog beam selection strategy based on fine-tuning can be obtained. And the final analog precoding matrix can be represented as ( ).

[0047] The lower-layer VDN-based subcarrier allocation scheme: Given the analog precoding matrix determined by the upper layer, the lower layer is responsible for handling the subcarrier allocation problem. This problem is modeled as a multi-agent MDP, where each agent corresponds to one subcarrier and selects the user it serves. Specifically: (1) State: In order to reduce dimensionality, subcarriers ( The state is constructed based on the singular values ​​of the equivalent channel after simulated precoding and the correlation between users. Specifically, let Indicates user In subcarrier The equivalent channel after simulated precoding. User In subcarrier The singular values ​​on are denoted as .user and ( In subcarrier The correlation coefficient on is defined as:

[0048] Therefore, subcarrier The state can be defined as:

[0049] Therefore, the global state can be represented as .

[0050] (2) Action: Subcarrier The action is defined as a binary indicator variable selected by the user. The joint action is defined as the subcarrier allocation matrix. .

[0051] (3) Reward: After taking an action, all agents receive an immediate reward. To maximize system performance and speed, the reward is shared among all agents, and the reward is defined as follows:

[0052] In addition, the strategy Defined in state Lower subcarrier Take action The probability of [the outcome]. Based on this, the joint strategy [is determined]. It can be represented as In the constructed MDP framework, the joint state action value is defined as the expected reward, denoted as . .

[0053] To improve the efficiency of learning the joint state-action value function, the VDN method is adopted to decompose the joint state-action value function into individual state-action value functions for each subcarrier. This decomposition can be expressed as:

[0054] in Indicates subcarrier The state-action value function. Therefore, the optimal action. ( ) can be represented as:

[0055] Corresponding optimal strategy This can be expressed as, if ,but ,otherwise .

[0056] The proposed VDN-based subcarrier allocation training process is centralized, while execution remains decentralized. Before training begins, multiple tuples are collected in the form of an off-policy. The experience is formally represented and stored in a buffer. In each round of training, random sampling is performed. A piece of experience can be represented as ( Based on sampling experience, the loss function can be calculated using the following formula:

[0057] in, Indicates the discount factor. Indicated by state and actions For input, parameters are The output of the neural network, and and The parameters are respectively The evaluation network and parameters are as follows The target network is determined by the loss function. In each iteration, stochastic gradient descent is used to update the parameters of the evaluation network. Parameters of the target network Every The cycle is repeated. Therefore, the post-training state-action value... It is obtainable. Given the state of each subcarrier, the final subcarrier scheduling scheme can be obtained, and can be expressed as: , Indicates subcarrier The final user scheduling scheme.

[0058] Zero-forcing-based digital precoding schemes: Before applying the zero-forcing method, it is necessary to base the precoding on a given radio frequency precoding matrix. Subcarrier allocation matrix Determine the equivalent channel for all serving users on each subcarrier. Specifically, we assume the subcarrier... The set of users served is denoted as ,in Therefore, subcarriers The equivalent channel matrix of the service user can be represented as: ,in It is the index of the user in the set. Based on the equivalent channel and zero-forcing beamforming scheme, a zero-forcing-based digital precoding matrix can be obtained. ,in , Represents the trace of the matrix. To satisfy the constraints... The final zero-forcing-based digital precoding matrix can be calculated as follows: ,in .

[0059] Power allocation scheme based on Lagrange multiplier method: In a given RF precoding matrix Subcarrier allocation matrix and zero-forced baseband precoding matrix In this case, the original problem can be restated as:

[0060] in, Indicates user Does it occupy a subcarrier? Indicator variables, Indicates subcarrier The power allocation matrix on, Indicates allocation to user In subcarrier The power level on This is the user calculated according to Shannon's formula. In subcarrier The transmission rate on This indicates the maximum transmit power of the base station.

[0061] Because zero-forced baseband precoding is used to reduce inter-user interference, users In subcarrier The data rate on can be restated as ,in It can be easily proven that this optimization problem is a convex problem, and therefore can be solved using the Lagrange multiplier method. Specifically, the Lagrange function can be expressed as: and Let represent a non-convex Lagrange multiplier.

[0062] Given the hybrid precoding and subcarrier allocation, the optimal power control strategy can be obtained, which can be specifically expressed as:

[0063] in Optimal Lagrange multiplier This can be obtained using a binary search method. Therefore, the optimal power control matrix is... It can be determined.

[0064] The present invention can also provide a hybrid beamforming and resource allocation module for MIMO-OFDM networks, including a system model construction module, an optimization problem construction module, and a solution module; The system model building module is used to build a single-cell downlink multi-user massive MIMO-OFDM system model, in which the base station is configured with a hybrid beamforming architecture, the number of users served by each subcarrier is flexibly variable, and the transmit power of the base station is flexibly allocated among the subcarriers; The optimization problem construction module is used to construct a joint optimization simulation precoding matrix based on the system model, with the objective of maximizing system performance and rate. Digital precoding matrix Subcarrier allocation matrix and power control matrix The optimization problem; The solution module is used to construct a hierarchical deep reinforcement learning framework to solve the optimization problem and obtain a hybrid beamforming and resource allocation strategy. In the solution module, the analog precoding problem is placed at the upper layer, modeled as a multi-Markov process, and a decision is made using an algorithm based on group correlation strategy optimization. Under a given analog precoding, the lower layer performs subcarrier allocation through a value decomposition network-based method. The training phase adopts a bottom-up process, while the execution phase adopts a top-down process.

[0065] The solution module includes an analog precoding solution unit and a subcarrier allocation solution unit. The analog precoding solution unit is used to model the analog precoding problem as a Multimarker process and makes decisions using a GRPO-based algorithm. A fine-tuning step is introduced after the GRPO-based policy training to further optimize the analog precoding. The subcarrier allocation solution unit is used to perform subcarrier allocation in the lower layer using a VDN-based method under a given analog precoding.

[0066] In addition, the present invention can also provide a computer device, including a processor and a memory, wherein the memory is used to store a computer executable program, the processor reads part or all of the computer executable program from the memory and executes it, and when the processor executes part or all of the computer executable program, it can realize the hybrid beamforming and resource allocation method of the MIMO-OFDM network described in the present invention.

[0067] On the other hand, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the hybrid beamforming and resource allocation method for MIMO-OFDM networks described in the present invention.

[0068] The computer device may be a laptop, a desktop computer, or a workstation.

[0069] The processor can be a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or an off-the-shelf programmable gate array (FPGA).

[0070] The memory described in this invention can be an internal storage unit of a laptop, desktop computer, or workstation, such as memory or hard disk; or it can be an external storage unit, such as a portable hard disk or flash memory card.

[0071] Computer-readable storage media can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media can include: read-only memory (ROM), random access memory (RAM), solid-state drives (SSDs), or optical discs, etc. Random access memory can include resistive random access memory (ReRAM) and dynamic random access memory (DRAM).

[0072] Numerical simulation and results analysis Simulation parameter settings: This invention evaluates the system performance of the proposed scheme through simulation. The system parameter settings are shown in Tables 1, 2, and 3. Table 1 System Scenario Simulation Parameters

[0073] Table 2 Parameters of the Upper Neural Network

[0074] Table 3 Parameters of the lower-level neural network

[0075] To demonstrate the hybrid beamforming and resource allocation method for MIMO-OFDM networks proposed in this invention, the following schemes are compared: (1) the proposed HDRL scheme; (2) the proposed FT-HDRL scheme; (3) the GA-RAHB scheme based on genetic algorithm for joint resource allocation; (4) the GA-PCHB scheme based on genetic algorithm for joint power control and hybrid beamforming; and (5) the random RF precoding scheme, wherein RF precoding is determined by randomly selecting beams from the DFT codebook, and BB precoding, subcarrier allocation, and power control are still determined by the proposed algorithm.

[0076] Figure 1 , Figure 2 and Figure 3 The relationships between the total system rate and signal-to-noise ratio (SNR), number of users, and number of subcarriers for the proposed FT-HDRL and HDRL schemes, GA-RAHB scheme, GA-PCHB scheme, and random RF precoding scheme are presented respectively. Figure 1 As can be seen, the overall system rate of all schemes improves with the increase of SNR. However, the proposed FT-HDRL and HDRL schemes still outperform the other comparative schemes. Specifically, the FT-HDRL scheme improves the speed by 12.37% compared to the HDRL scheme, and achieves average gains of 39.30%, 133.03%, and 89.86% compared to the GA-RAHB, GA-PCHB, and random RF precoding schemes, respectively. Figure 2 The results show that the total system rate of the proposed FT-HDRL and HDRL schemes, as well as the random RF precoding scheme, increases with the number of users, and the proposed FT-HDRL and HDRL schemes still outperform other comparative schemes. Furthermore, as... Figure 3 As shown, although the overall system rate of all schemes increases with the number of subcarriers, the proposed scheme still exhibits the best performance, especially when the number of subcarriers is large. The advantages of the proposed scheme mainly come from the following two aspects: (1) Under the constraints of the radio frequency chain, each subcarrier can flexibly adjust the set and number of serving users, and can adaptively allocate transmit power among subcarriers. (2) The proposed HDRL framework can efficiently utilize limited resources. In addition, the fine-tuning step in the upper layer further improves the overall system rate.

[0077] Table 4 shows the execution times of the proposed FT-HDRL and HDRL schemes, the GA-RAHB scheme, the GA-PCHB scheme, and the random RF precoding scheme. We can observe that the proposed scheme significantly reduces the execution time compared to the GA-RAHB and GA-PCHB schemes. Furthermore, although the execution time of the random RF precoding scheme is slightly lower, its overall system speed is significantly lower than our proposed scheme. Moreover, compared to the HDRL scheme, although the FT-HDRL scheme increases execution time by 5.83%, its overall system speed is improved by 13.42%, demonstrating the effectiveness of the fine-tuning steps.

[0078] Table 4 Execution time of different algorithms

[0079] The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A hybrid beamforming and resource allocation method for MIMO-OFDM networks, characterized in that, Includes the following steps: A single-cell downlink multi-user massive MIMO-OFDM system model is constructed, wherein the base station is configured with a hybrid beamforming architecture, the number of users served by each subcarrier is flexibly variable, and the transmit power of the base station is flexibly allocated among the subcarriers; Based on the system model, a joint optimization of the analog precoding matrix is ​​constructed with the goal of maximizing system performance and rate. Digital precoding matrix Subcarrier allocation matrix and power control matrix The optimization problem; A hierarchical deep reinforcement learning framework is constructed to solve the optimization problem, resulting in a hybrid beamforming and resource allocation strategy. When constructing a hierarchical deep reinforcement learning framework to solve the optimization problem, the simulated precoding problem is placed at the upper layer, modeled as a Multimarker process, and an algorithm based on group-related policy optimization is used for decision-making. Given analog precoding, the lower layer performs subcarrier allocation using a value decomposition network-based method; The training phase adopts a bottom-up process, while the execution phase adopts a top-down process.

2. The hybrid beamforming and resource allocation method for MIMO-OFDM networks according to claim 1, characterized in that, The optimization problem is specifically: in, The maximum transmit power of the base station is represented by the first constraint in the problem, which reflects the hardware limitations of the fully connected, single-phase shifter architecture; the third constraint limits the number of users that can be served on each subcarrier to the number of radio chains; and the fourth constraint ensures that the total power allocated to the base station does not exceed the maximum transmit power. Represents the simulated beamforming matrix, where and These represent the number of antennas and the number of radio frequency chains at the transmitting end, respectively. Indicates subcarrier The digital beamforming matrix on, where Indicates the use of subcarriers The number of users, Denotes the channel selection matrix, where Indicates user Does it occupy a subcarrier? Indicator variables, Indicates subcarrier The power allocation matrix on, Indicates allocation to user In subcarrier The power level on This is the user calculated according to Shannon's formula. In subcarrier The transmission rate on Indicates user In subcarrier Digital beamforming vector on This represents a predefined discrete Fourier transform codebook.

3. The hybrid beamforming and resource allocation method for MIMO-OFDM networks according to claim 1, characterized in that, The construction of a hierarchical deep reinforcement learning framework to solve the optimization problem includes: Under the analog precoding and subcarrier allocation strategy given in the hierarchical deep reinforcement learning framework, zero-forcing digital precoding is first performed, followed by power control optimization using the Lagrange multiplier method, and the reward used for training is calculated accordingly. During the training phase: First, the lower-layer subcarrier allocation strategy is trained under zero-forcing digital precoding and power control, providing a reward for the training of the upper-layer analog precoding strategy; During the execution phase: first, analog precoding is determined, then subcarrier allocation is performed, and finally zero-forcing digital precoding and power control are adopted.

4. The hybrid beamforming and resource allocation method for MIMO-OFDM networks according to claim 3, characterized in that, The lower-layer subcarrier allocation is modeled as a multi-agent MDP, where each agent corresponds to one subcarrier and selects the user it serves, as follows: State: Subcarriers are constructed based on the singular values ​​of the equivalent channel after simulated precoding and the correlation between users. state, Specifically, let Indicates user In subcarrier The equivalent channel after analog precoding, user In subcarrier The singular values ​​on are denoted as ,user and ( In subcarrier The correlation coefficient is defined as follows: subcarrier The status is: The global state can be represented as ; Action: Subcarrier The action is defined as a binary indicator variable selected by the user. The joint action is defined as the subcarrier allocation matrix. ; Reward: After an action is taken, all agents receive an immediate reward, which is shared among all agents. The reward is defined as follows: The training process for subcarrier allocation based on value decomposition networks is conducted in a centralized manner, while execution remains decentralized.

5. The hybrid beamforming and resource allocation method for MIMO-OFDM networks according to claim 1, characterized in that, The simulation precoding problem is modeled as a Multi Markov process as follows: The state of a DoMARK process includes a shared state and There are several local states, and a shared state acquired by all agents. This shared state characterizes each user's preference for each analog beam in the DFT codebook, denoted as . ,user For analog beam Preference value Defined as using analog beam For users The reference signal received power obtained when beamforming the channel is expressed as: in, Indicates base station and user In subcarrier Channel gain on The first character in the Discrete Fourier Transform codebook represents the... Each simulated beam weight; intelligent agent The local state is defined as the state of the agent. In the action steps, the current joint simulated beam selection vector of all radio frequency chains, and the joint state of all agents can be represented as: ; Action: Each agent takes an action sequentially under shared parameters. Action defined as radio frequency chain Selected analog beam index The joint action of all agents is represented as , Reward: For the constructed Multimarker process, the reward is defined as the achieved system and rate, expressed as: Using neural networks Parameterized sharing strategy ,in Represents network parameters for the agent. Neural Networks by state As input, and output the corresponding action probability distribution, in a given state In this case, intelligent agent The actions are sampled from the learned distribution, and a group-related policy optimization algorithm is used to train the policy network. The final policy is obtained through iteration. In a given shared state Based on strategy The probability distribution is obtained, and each agent sequentially samples the actions to obtain the simulated beam selection strategy and the simulated precoding matrix.

6. The hybrid beamforming and resource allocation method for MIMO-OFDM networks according to claim 5, characterized in that, The fine-tuning step introduced after policy training based on group-related policy optimization includes: For radio frequency chain Keep all other analog beams selected by the RF chain fixed, and then evaluate all the unselected analog beams; For candidate simulated beams to form candidate solutions And the corresponding rewards The computation can be performed in parallel; the simulated beam selection strategy with the highest reward is selected to obtain the final simulated beam selection strategy and simulated precoding matrix based on fine-tuning.

7. A communication system, characterized in that, The system includes a base station and several terminals. The base station is a multi-transmit antenna base station, and each terminal has a receiving antenna. During communication, the system performs hybrid beamforming and resource allocation based on the hybrid beamforming and resource allocation method described in any one of claims 1 to 7.

8. A hybrid beamforming and resource allocation system for a MIMO-OFDM network, characterized in that, It includes a system model building module, an optimization problem building module, and a solution module; The system model building module is used to build a single-cell downlink multi-user massive MIMO-OFDM system model, in which the base station is configured with a hybrid beamforming architecture, the number of users served by each subcarrier is flexibly variable, and the transmit power of the base station is flexibly allocated among the subcarriers; The optimization problem construction module is used to construct a joint optimization simulation precoding matrix based on the system model, with the objective of maximizing system performance and rate. Digital precoding matrix Subcarrier allocation matrix and power control matrix The optimization problem; The solution module is used to construct a hierarchical deep reinforcement learning framework to solve the optimization problem and obtain a hybrid beamforming and resource allocation strategy. In the solution module, the analog precoding problem is placed at the upper layer, modeled as a multi-Markov process, and a decision is made using an algorithm based on group correlation strategy optimization. Under a given analog precoding, the lower layer performs subcarrier allocation through a value decomposition network-based method. The training phase adopts a bottom-up process, while the execution phase adopts a top-down process.

9. A computer device, characterized in that, It includes a processor and a memory, the memory being used to store a computer-executable program, the processor reading the computer-executable program from the memory and executing it, and the processor executing the computer-executable program being able to implement the hybrid beamforming and resource allocation method of the MIMO-OFDM network according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, A computer-readable storage medium stores a computer program that, when executed by a processor, enables the implementation of the hybrid beamforming and resource allocation method for MIMO-OFDM networks as described in any one of claims 1 to 6.

Images