A MIMO full-duplex power allocation method based on smart antenna
By using smart antenna user management and DDPG model optimization, the self-interference problem in large-scale MIMO full-duplex systems is solved, improving spectral efficiency and reducing algorithm complexity, making it suitable for wireless communication systems in multi-user scenarios.
Patent Information
- Application Number
- CN202210851953.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-07-19
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2042-07-19
AI Technical Summary
Existing technologies are unable to effectively eliminate self-interference in large-scale MIMO full-duplex systems, especially in multi-user scenarios, which leads to limited system performance and fails to meet the requirements for high spectral efficiency.
A MIMO full-duplex power allocation method based on smart antennas is adopted. Real-time enable and silent management are performed through the user's globally unique identifier. The system spectral efficiency objective function is designed by combining the transmit and receive status of the smart antenna and subcarrier allocation. The DDPG model is used to optimize power allocation and reduce algorithm complexity.
It improves the spectral efficiency of full-duplex systems, reduces algorithm learning time, is suitable for deployment in large-scale MIMO full-duplex wireless communication systems, and reduces computational complexity.
Smart Images

Figure CN115278896B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of wireless communication technology, and in particular to a MIMO full-duplex power allocation method based on smart antenna. Background Art
[0002] In recent years, with the rapid development of multimedia Internet, AR (Augmented Reality) and other technologies, the services derived from these technologies have also been integrated into and enriched people's lives. However, traditional communication technology has approached the Shannon capacity limit and it is difficult to meet people's ever-changing business needs. Therefore, future wireless communication technologies have put forward higher requirements for performance indicators such as high throughput and high spectrum efficiency.
[0003] Full-duplex technology, that is, full-duplex in the same frequency and at the same time in the band, can transmit information on the same time domain and frequency domain resources. Compared with traditional duplex methods such as TDD (Time Division Duplex) and FDD (Frequency Division Duplex), it can effectively improve the freedom of the communication system. In view of this, this technology is regarded as one of the candidate enhanced wireless air interface technologies for 6G and has a very important research prospect. In theory, according to the definition of Shannon's capacity formula, full-duplex technology can double the freedom of the communication system under the existing conditions. However, this technology faces serious self-interference problems, which to some extent restricts its performance advantage of higher system freedom. The so-called self-interference is the interference caused by the transmitting antenna of the self to the receiving antenna under the condition of simultaneous frequency. If effective measures are not taken to deal with the interference, when the base station transmit power is too high, the receiving signal of the self will be completely buried in the transmitting signal, which is not conducive to the decoding of the user signal by the base station and seriously affects the user experience. Therefore, this phenomenon has become a bottleneck restricting the development of full-duplex technology. How to effectively eliminate self-interference is an issue that needs to be solved urgently.
[0004] Existing research mainly eliminates self-interference from two aspects: passive elimination and active elimination. Passive elimination includes physical isolation of antennas, directional isolation, absorption shielding, etc. Active elimination is based on the signal domain, including estimating a copy of the self-interference signal from the digital domain or analog domain, and removing the estimated signal copy from the mixed signal. Although the existing self-interference elimination technology has been able to achieve an elimination margin of 110dB, some self-interference still remains, especially for situations with a large number of antennas and high transmission power. The impact cannot be ignored. At present, residual self-interference elimination is mostly achieved by adjusting the power through beamforming on the base station side, thereby regulating the residual self-interference, and combining the current channel environment to ensure that the ratio of degrees of freedom to residual self-interference is as large as possible, thereby achieving a higher spectrum efficiency. However, existing research is mostly aimed at the situation where the base station transmit / receive antenna is fixed, which limits the improvement of antenna diversity gain to a certain extent.
[0005] With the in-depth research of m-MIMO (massive multiple-input multiple-output) technology, the system capacity can be greatly improved and QoS (quality of service) can be guaranteed by deploying a large-scale antenna array on the base station side. However, the large number of antennas will increase the complexity of the algorithm and the degree of residual self-interference. Especially when there are multiple users accessing the cell, the base station will not only be interfered by itself, but also receive interference from the users, and users will also interfere with each other. This will cause the full-duplex communication system, which already has strong residual self-interference, to face more severe challenges.
[0006] Therefore, a reasonable power allocation scheme needs to be designed for massive MIMO full-duplex multi-user scenarios to ensure the inherent performance advantages of the full-duplex system. Summary of the invention
[0007] Purpose of the invention: In view of the defects in the prior art, the present invention discloses a MIMO full-duplex power allocation method based on smart antennas, which adopts a globally unique user identifier to enable and silently manage users in real time, so as to avoid abnormal situations such as some users in the cell causing significant base station residual self-interference or interference between cell users in the current time slot, and designs an effective system spectrum efficiency objective function in combination with the sending and receiving states and subcarrier allocation of each smart antenna. By solving the optimization problem of the objective function, the system spectrum efficiency is ensured to be maximized, so as to maximize the degree of freedom of the full-duplex system.
[0008] Technical solution: In order to achieve the above technical objectives, the present invention adopts the following technical solution.
[0009] A MIMO full-duplex power allocation method based on smart antenna comprises the following steps:
[0010] S1. Establishing a full-duplex massive MIMO network model; the full-duplex massive MIMO network model includes a MIMO network, a base station equipped with N antennas, and Z users that can communicate with the base station, the base station operates in full-duplex mode, each user is equipped with one antenna, and the user operates in half-duplex mode;
[0011] S2. Define the three elements of smart antenna, user state and subcarrier allocation in the full-duplex massive MIMO network model; the smart antenna intelligently selects one of four working modes for each antenna of the base station, and the four working modes include for sending, for receiving, sending and receiving coexisting, and neither sending nor receiving; the user state is defined as using a global unique identifier to define the current state for all uplink and downlink users, and construct an uplink user set and a downlink user set; the subcarrier allocation includes: defining a subcarrier allocation matrix for redistribution according to the uplink user set and the downlink user set The reallocated subcarrier allocation matrix Indicates the subcarrier allocation status of uplink and downlink enabled users;
[0012] S3, constructing an optimized spectrum efficiency objective function for a full-duplex massive MIMO network model;
[0013] S4. Establish a DDPG model to solve the optimized spectrum efficiency objective function: establish a DDPG model mapped to a full-duplex massive MIMO network model, wherein the DDPG model includes an agent, a state, an action, and a reward, and is provided with a DDPG network, wherein the DDPG network includes two neural networks with different structures, namely, Actor and Critic;
[0014] S5. Optimize the states and actions in the DDPG model: Based on the principle of fairness, find the lower limit of the uplink user transmit power and the lower limit of the base station downlink precoding matrix to obtain the reduced action and state space;
[0015] S6. Run the DDPG model to output the optimal action and state to achieve MIMO full-duplex power allocation.
[0016] Beneficial effects: The present invention adopts a globally unique user identifier to enable and silently manage users in real time, avoiding abnormal situations such as some users in the cell causing significant base station residual self-interference or interference between cell users in the current time slot, and combining the sending and receiving states and subcarrier allocation of each smart antenna to design an effective system spectrum efficiency objective function. By solving the objective function optimization problem, the system spectrum efficiency is maximized to maximize the degree of freedom of the full-duplex system. At the same time, on this basis, the coupling variables in the objective function are theoretically analyzed, the variables are transformed, and the number of variables is effectively reduced. Ultimately, in order to realize the deep reinforcement learning algorithm, the goal of reducing the number of algorithm actions and state values, shortening the algorithm learning time, and reducing the algorithm complexity is achieved, which is suitable for deployment in large-scale MIMO full-duplex wireless communication systems. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 is a flow chart of the method of the present invention;
[0018] Figure 2 This is a schematic diagram of a full-duplex massive MIMO network model in Example 2;
[0019] Figure 3 This is a schematic diagram of the architecture of the traditional transceiver antenna and the intelligent transceiver antenna of the base station in Example 2;
[0020] Figure 4 This is a flow chart of the deep reinforcement learning algorithm in Example 2. DETAILED DESCRIPTION
[0021] The following further describes and explains a MIMO full-duplex power allocation method based on smart antennas according to the present invention in conjunction with the accompanying drawings and embodiments.
[0022] The present invention proposes a MIMO full-duplex power allocation method based on smart antennas, which is suitable for application scenarios where a base station is equipped with a large number of antennas and there are multiple users in a cell. Each antenna of the base station is connected through analog devices such as a circulator to achieve uplink and downlink separation. Therefore, each antenna can intelligently select four working modes for reception, transmission, coexistence of reception and transmission, or neither reception nor transmission. Compared with the traditional base station transceiver antenna, the antenna physical resources are fully utilized and the diversity gain of the antenna is improved. The numerous antennas configured by the base station are uniformly managed through the antenna state matrix to ensure the working mode of each antenna. The downlink power allocation of the base station is determined by the base station precoding matrix, and the uplink power is determined by the transmission power of each uplink user. At the same time, in order to avoid abnormal situations such as significant base station residual self-interference or interference between users in the cell caused by some users in the current time slot, such abnormal users are muted in the current time slot, and the silence is released in the next time slot for pre-calling. The enable / silence status of each user service is maintained by the user service status matrix. Finally, based on the enabled users in the current time slot, a subcarrier allocation matrix is designed to allocate subcarriers to each enabled user in order to reasonably utilize bandwidth resources and maximize the overall spectrum efficiency of the system.
[0023] The present invention mainly solves the following technical problems:
[0024] Embodiment 1:
[0025] As attached Figure 1 As shown, the present invention discloses a MIMO full-duplex power allocation method based on smart antenna, comprising the following steps:
[0026] S1. Establish a full-duplex massive MIMO network model; the full-duplex massive MIMO network model includes a MIMO network, a base station equipped with N antennas, and Z users that can communicate with the base station. The users can communicate with the base station or not. The base station works in full-duplex mode, but any antenna of the base station can work in full-duplex or half-duplex, but the base station has the ability to work in full-duplex; the Z users include J uplink users, K downlink users, and O dormant users, J+K+O=Z; each user is equipped with an antenna, and the user works in half-duplex mode. The MIMO network includes M mutually orthogonal subcarriers, and the number of subcarriers is less than the number of users.
[0027] S2. Define the three elements of smart antenna, user status, and subcarrier allocation in the full-duplex massive MIMO network model;
[0028] The smart antenna means that each antenna of the base station is connected through a circulator, and each antenna intelligently selects one of four working modes, including for sending, for receiving, sending and receiving at the same time, and neither sending nor receiving;
[0029] Each antenna of the base station includes a transmitting / receiving antenna, a circulator, a receiver and a transmitter. The transmitting / receiving antenna is connected to the receiver and the transmitter respectively through the circulator, and the transmitter is connected to the receiver through self-interference elimination; each antenna of the base station is connected through a circulator to realize smart antenna and uplink and downlink separation, so each antenna can intelligently choose to be used for sending / receiving, or sending and receiving coexisting, or neither sending nor receiving working mode, which improves the diversity gain compared with the traditional base station transceiver antenna, such as the attached Figure 3 As shown, self-interference elimination is a prior art method, including passive elimination and active elimination, and is not specifically limited in the present invention.
[0030] The user status is defined as using a global unique identifier to define the current state of all uplink and downlink users, and constructing an uplink user set and a downlink user set; the uplink user set and the downlink user set are:
[0031] At the current moment, the uplink user meets Downlink user satisfaction α 2β-1 (t) is the uplink service identifier of the βth user at the current moment, α 2β (t) is the downlink service identifier of the βth user at the current moment; the global unique identifier A = {α1(t), α2(t), ..., α 2Z (t)}, where α 2Z (t) is the service identifier of the Zth user at time t;
[0032]
[0033]
[0034] α 2β-1 (t) = 1, indicating that user β is currently enabling uplink services, α 2β-1 (t) = 0, indicating that user β does not enable uplink services at the current moment, α 2β-1 (t) = 2, indicating that user β is currently invoking the uplink service and will not be muted; α 2β (t) = 1, indicating that user β is currently enabling downlink services, α 2β (t) = 0, indicating that user β does not enable downlink services at the current moment, α 2β (t) = 2, indicating that user β is currently invoking the downlink service and will not be muted; Sic u (t-1) = 1 or 0 indicates that the user's uplink service was muted or not muted at the previous moment respectively; Sic d (t-1) = 1 or 0 respectively indicates that the user's downlink service was muted or not muted at the previous moment; Req u (t) = 1 or 0 means that the user has received or not received the uplink service request at the current moment; Req d (t)=1 or 0 indicates that the user has received or has not received a downlink service demand at the current moment.
[0035] Define the reallocated subcarrier allocation matrix based on the uplink user set and the downlink user set include:
[0036] Define the uplink user service state matrix e at the current moment according to the uplink user set and the downlink user set u and downlink user service state matrix e d ; Uplink user service state matrix e u and downlink user service state matrix e d After determining, define the reallocation subcarrier allocation matrix
[0037] Define the uplink user service state matrix e at the current moment u and downlink user service state matrix e d :
[0038]
[0039]
[0040] in Defined as:
[0041]
[0042] Defined as:
[0043]
[0044] Among them, 1≤j≤J, 1≤k≤K;
[0045] Define the subcarrier allocation matrix represents the subcarrier allocation status of (J+KS) uplink and downlink capable users, where There are two subscripts, the first subscript refers to the user, and the second subscript refers to the subcarrier, 1≤z≤Z; satisfy:
[0046]
[0047]
[0048] and
[0049] Subcarrier set M is the number of subcarriers; uplink user set J is the number of uplink users; the downlink user set K is the number of downlink users; the set of muted uplink and downlink users S is the number of muted uplink and downlink users;
[0050] S3. Construct the optimized spectrum efficiency objective function of the full-duplex massive MIMO network model:
[0051]
[0052]
[0053]
[0054]
[0055] Among them, R represents the total spectrum efficiency of uplink and downlink users, G is the comprehensive matrix, The synthesis matrix with zeroed elements for antenna and user mismatches, is the uplink user service state matrix e u and downlink user service state matrix e d Determine the subcarrier allocation matrix after redistribution, x u Indicates the symbol sent by the uplink user to the base station, x d Indicates the symbol sent by the base station to the downlink user, q u and q dis the base station antenna state matrix, which represents the antennas used for uplink reception and downlink transmission of the base station, respectively, and w k is the precoding matrix of the kth downlink user, satisfying w k ∈W={w1,w2,...,w K}, W is the precoding matrix of all downlink users, that is, the base station downlink precoding matrix, p j is the transmission power of the jth uplink user, satisfying p j ∈P={p1,p2,...,p J}, P is the transmission power of all uplink users.
[0056] The optimized spectrum efficiency objective function has multiple mutually coupled variables. The present invention analyzes the nested relationship between the variables and effectively reduces the number of variables.
[0057] S4. Establish a DDPG model to solve the optimized spectrum efficiency objective function: establish a DDPG model mapped to a full-duplex massive MIMO network model, wherein the DDPG model includes an agent, a state, an action, and a reward, and is provided with a DDPG network, wherein the DDPG network includes two neural networks with different structures, namely, Actor and Critic;
[0058] The intelligent agents are smart antennas and uplink and downlink users in a full-duplex massive MIMO network model;
[0059] The state is: signal to interference and noise ratio of uplink user and the signal-to-interference-noise ratio of downlink users Matching between the base station antenna and the jth uplink user And the matching situation between the base station antenna and the kth downlink user And subcarrier allocation for each uplink and downlink user Recorded as status
[0060] The actions are: base station downlink precoding matrix W, uplink user transmission power p j ∈P, the above actions are collectively recorded as action a={W;p j ∈P};
[0061] The reward is: Spectral efficiency of uplink users and spectrum efficiency for downlink users Recorded as
[0062] in:
[0063]
[0064]
[0065] The DDPG network has two neural networks with different structures, Actor and Critic. Each neural network contains a replica network with the same structure but different parameters. The target network is denoted as target_net and the replica network is denoted as eval_net. The Actor and Critic contain four neural networks, respectively: critic_target_net: Q′(s1, s2, s3, a|θ Q′ ), critic_eval_net: Q(s1, s2, s3, a|θ G ), actor_target_net: μ′(s1, s2, s3|θ Q′ ), actor_eval_net: μ(s1, s2, s3|θ G );
[0066] S5. Optimize the states and actions in the DDPG model: Based on the principle of fairness, find the lower limit of the uplink user transmit power and the lower limit of the base station downlink precoding matrix to obtain the reduced action and state space;
[0067] The element w in action a k and p j Respectively limited to:
[0068]
[0069] p min ≤p j ≤p max , p j ∈P
[0070] Reduced action state
[0071] S6. Run the DDPG model to output the optimal action and state to achieve MIMO full-duplex power allocation.
[0072] The present invention uses a large-scale MIMO full-duplex system as the input environment, designs a deep reinforcement learning algorithm, combines three key elements: smart antenna, user state, and subcarrier allocation, and designs a reasonable system spectrum efficiency objective function. The objective function variables include base station antenna state matrix, user service state matrix, base station downlink precoding matrix, user uplink power set, and subcarrier allocation matrix. At the same time, the objective function variables are converted to effectively reduce the number of variables, reduce the number of actions and states that can be selected in the deep reinforcement learning algorithm, and effectively shorten the algorithm learning time.
[0073] S61, for critic_eval_net: Q(s1, s2, s3, a|θ G ) and actor_eval_net: μ(s1, s2, s3|θ μ ) G and θ μ Randomly assign initial values;
[0074] S62, respectively, the θ in the eval network G and θ μ Assign to θ in the target network Q′ and θ μ′ ;
[0075] S63, initialize the experience pool R, the experience pool R stores a number of {s t , a t , r t};
[0076] S64, for action Assign initial values, and adopt equal power allocation strategy for uplink and downlink users;
[0077] S65, iterate, the iteration includes two layers: outer loop and inner loop. The outer loop iteration resets the state, and the inner loop iteration traverses the state steps until the maximum number of outer loop iterations is reached, and the optimal action is output. And the state corresponding to the action Realize MIMO full-duplex power allocation.
[0078] The outer loop and inner loop process in step S65 include:
[0079] S651, enter the outer loop iteration, the outer loop iteration assigns the current value to the state, where the initial state is recorded as s0, the subscript 0 represents the state step number, and each state includes s1, s2, s3;
[0080] S652, enter the inner loop iteration, each time the inner loop iteration selects action a from actor_eval_net t , and the action a t In full-duplex environment, the state changes from s t Migrate to t+1 And get reward t ,Right now:
[0081]
[0082]
[0083]
[0084] S653, the current state s t , next state s t+1 and the selected action a t and the reward r t They are stored in the experience pool R respectively;
[0085] S654, randomly select several groups (s t , α t , r t ,s t+1 ) to train the DDPG network;
[0086] S653, calculate action α through eval critic network t curQ under: curQ=Q(s1,s2,s3,a|θ G );
[0087] S654, action α output by target actor network t+1 , targetQ is calculated using the target critic network as the eval critic network. The calculation formula of targetQ is:
[0088] targetQ=r t +gamma·Q′(s1,s2,s3,μ′(s1,s2,s3|θ μ′ )|θ Q′ )
[0089] Where gamma represents the discount factor, gamma∈[0,1];
[0090] S655. According to the obtained curQ and targetQ, the gradient of the mean square error of the loss function is calculated. The gradient calculation formula of the mean square error is:
[0091] Loss=mean(diff(targetQ-curQ))
[0092] S656. Update parameter θ in curQ G ;
[0093] S657, Update the Actor's parameters θ using the Monte Carlo estimation method of the gradient μ ; Monte Carlo estimation method updates the Actor's parameters θ μ The calculation formula is:
[0094]
[0095] S658, using parameter θ G and θ μ The sliding average method is used to update the parameters θ of the target critic network and the target actor network respectively. Q′ and θ μ′ ; The calculation formula is:
[0096] θ Q′ ←ρθ G +(1-ρ)θ Q′
[0097] θ μ′ ←ρθ μ +(1-ρ)θ μ′
[0098] Where ρ∈(0,1) is a hyperparameter;
[0099] S659: Determine whether the maximum number of inner loop iterations has been reached. If not, return to S652; if yes, return to S651; Determine whether the maximum number of outer loop iterations has been reached. If not, return to S651; if yes, output the optimal action. And the state corresponding to the action Realize MIMO full-duplex power allocation.
[0100] The present invention discloses a MIMO full-duplex power allocation method based on smart antennas. It adopts a global unique identification of users to enable and silently manage users in real time, so as to avoid abnormal situations such as some users in the cell causing significant base station residual self-interference or interference between users in the cell at the current moment, and combines the sending and receiving states and subcarrier allocation of each smart antenna to design an optimized system spectrum efficiency objective function. By solving the optimization problem of the objective function, the system spectrum efficiency is ensured to be maximized, so as to maximize the degree of freedom of the full-duplex system. At the same time, on this basis, the coupling variables in the objective function are theoretically analyzed, the variables are transformed, and the number of variables is effectively reduced. Finally, the algorithm complexity is reduced by reducing the algorithm action and state selection space, and the network training time is shortened, thereby realizing an efficient deep reinforcement learning scheme, which is suitable for deployment in large-scale MIMO full-duplex wireless communication systems.
[0101] Embodiment 2:
[0102] As attached Figure 1 As shown, the present invention proposes a MIMO full-duplex power allocation method based on smart antenna, which includes the following steps:
[0103] Process 1: Establish a full-duplex massive MIMO network model;
[0104] As attached Figure 2 As shown in the figure, there is a BS (Base Station) in a massive MIMO cell, equipped with a large number of N antennas, working in full-duplex mode. There are Z users in the cell, divided into uplink users, downlink users and dormant users according to user type. Each user is equipped with only one antenna and works in half-duplex mode. The network system is divided into M mutually orthogonal subcarriers, and the number of subcarriers is less than the number of users. Subcarriers can be reused between users.
[0105] Each antenna of the base station is connected through a circulator analog device to achieve uplink and downlink separation. Therefore, each antenna can intelligently choose to be used for sending / receiving, or sending and receiving at the same time, or neither sending nor receiving. Compared with traditional base station transceiver antennas, the diversity gain is improved. Figure 3 shown.
[0106] Process 2: Define the three elements of smart antenna, user status, and subcarrier allocation;
[0107] Assume that the Z users include J uplink users, K downlink users and O dormant users at the current moment. Considering that some downlink users may significantly increase the residual self-interference of the base station, or some uplink users may significantly interfere with other users, the best contribution state of such users to the system performance is that their receiving or transmitting power is zero, that is, in a silent state.
[0108] Considering the fading characteristics of the channel at different times and the mobility characteristics of the user, these factors will aggravate the time-varying characteristics of the channel. Therefore, in order to take into account the faster time-varying characteristics of the channel, the time slot is usually divided into shorter times, which are less than the coherence time of the time-varying channel, so that the channel gain in a time slot is considered to be a constant. In fact, for users, the duration of conventional uplink and downlink service requirements is usually in the order of hundreds of milliseconds, which does not require a particularly high service delay. For users who require a large throughput for a period of time, the impact of the short silence state of the previous divided time slot on QoS will be much lower than the negative impact caused by insufficient throughput for a period of time. In view of this, the present invention proposes a service-driven user status strategy, that is, the service status of the uplink and downlink users in the next time slot depends on the real-time uplink and downlink service requirements and the silence of the uplink and downlink services in the previous time slot. Therefore, for any uplink and downlink users in the current time slot, there are uplink and downlink service switching or maintenance, deactivation and other states. Therefore, in order to facilitate the description of the respective states of the uplink and downlink users in the current time slot, a globally unique identifier A = {α1(t), α2(t), ..., α 2Z (t)}, where α 2Z(t) is the service identifier of the Zth user at time t. Meanwhile, for simplicity of expression, the present invention defines the time slots of time t and (t-1) as the current time and the previous time, respectively.
[0109] Since different types of service users cause different types of interference to the massive MIMO full-duplex system, the present invention uniformly divides the service users into two categories: uplink users and downlink users. Taking into account the time-varying characteristics of the uplink and downlink user service types, the present invention uses user services as the driver and defines a globally unique identifier to effectively enable and silently manage users in real time.
[0110] Taking the βth user at the current moment as an example, the uplink service identifier is α 2β-1 (t), the downlink service identifier is denoted as α 2β (t), respectively expressed as follows
[0111]
[0112]
[0113] where α 2β-1 (t) = 1, indicating that user β is currently enabling uplink services, α 2β-1 (t) = 0, indicating that user β does not enable uplink services at the current moment, α 2β-1 (t) = 2, indicating that user β is currently invoking the uplink service and will not be muted; similarly, α 2β (t) = 1, indicating that user β is currently enabling downlink services, α 2β (t) = 0, indicating that user β does not enable downlink services at the current moment, α 2β (t) = 2, indicating that user β is currently invoking the downlink service and will not be muted. u (t-1) = 1 or 0 respectively indicates that the user's uplink service was muted or not muted at the previous moment; similarly, Sic d (t-1) = 1 or 0 respectively indicates that the downlink service of the user was muted or not muted at the previous moment. u (t) = 1 or 0 means that the user has received or not received the uplink service request at the current moment; similarly, Req d (t) = 1 or 0 means that the user has received or not received the downlink service demand at the current moment. In the present invention, β takes a value from 1 to Z. If β takes a value of z, it means the zth user. If the zth user represents an uplink user, α 2z-1 (t) = 1, 2 and α 2z (t) = 0; if the zth user represents a downlink user, α 2z (t) = 1, 2 and α 2z-1 (t)=0.
[0114] In summary, at the current moment, the uplink user meets Downlink user satisfaction
[0115] In the present invention, the unique identifier A determines the set of uplink users and downlink users at the current moment, which respectively satisfy and That is to say, the collection and It is not certain at every moment, and the constraint condition needs to be satisfied. In the subsequent steps of the present invention, the set and The selection is based on the unique identifier A. Therefore, the present invention adopts the user's global unique identifier to enable and silently manage the user in real time to avoid abnormal situations in actual application scenarios, such as some users in the cell may cause significant base station residual self-interference or interference between cell users at the current moment.
[0116] To simplify the description of the problem, the present invention temporarily does not consider the base station antenna working mode, the enabling / silent status of uplink and downlink users, and the user subcarrier allocation.
[0117] Then the transmission signal of the jth uplink user received by the base station side is:
[0118]
[0119] in represents the uplink channel vector from the jth uplink user to the base station, represents the transmission symbol of the jth uplink user, It refers to the interference caused to the base station by all uplink users j′ except the jth uplink user. is the residual self-interference of the base station after the self-interference elimination technology is applied, H SI is the residual self-interference matrix of the base station, represents the symbol sent by the base station to the kth downlink user, is the Gaussian white noise generated by the base station side receiving the jth uplink user. j is the transmission power of the jth uplink user, satisfying p j ∈P={p1,p2,...,p J}. k is the precoding matrix of the kth downlink user, satisfying w k ∈W={w1,w2,...,w K}.
[0120] Similarly, the signal received by the kth downlink user from the base station is:
[0121]
[0122] in represents the downlink channel vector from the base station to the kth downlink user, It refers to the interference caused by the signals of all downlink users k′ except the one sent by the base station to the kth downlink user to the reception of the kth downlink user. k,j represents the channel gain from the jth uplink user to the kth downlink user, represents the interference caused by the transmission of all uplink users to the reception of the kth downlink user, is the Gaussian white noise generated by the kth downlink user.
[0123] Based on the above problem model, the limiting factors of the base station antenna working mode and the enabling / silent status of the uplink and downlink users are considered. Therefore, the signal sent by the jth uplink user and the signal received by the kth downlink user from the base station are rewritten as:
[0124]
[0125]
[0126] where q u and q d is the base station antenna state matrix, which represents the antennas used for uplink reception and downlink transmission of the base station, satisfying
[0127] The definition is as follows:
[0128]
[0129] The definition is as follows:
[0130]
[0131] The uplink and downlink user sets at the current moment have been defined. and However, in fact, among these uplink and downlink users, except for the uplink and downlink service identifier 2, which will not be muted at the current moment, the other identifiers 1 may be muted. Therefore, the uplink and downlink user service state matrix e is defined at the current moment respectively. u and e d as follows:
[0132]
[0133]
[0134] in The definition is as follows:
[0135]
[0136] The definition is as follows:
[0137]
[0138] It is also known that the silent state of the uplink and downlink users at the current moment will affect the uplink and downlink user states at the next moment, so it is necessary to mark the silent state of the uplink and downlink users at the current moment at the same time, as follows:
[0139]
[0140]
[0141] At the same time, the number of uplink and downlink users that are muted at the current moment is marked as S, and this parameter is related to subcarrier allocation.
[0142] Then the uplink signal to interference plus noise ratio of the jth uplink user and the downlink signal to interference plus noise ratio of the kth downlink user are defined as follows:
[0143]
[0144]
[0145] in I is the identity matrix, is the Gaussian white noise variance generated by the base station side receiving the jth uplink user, is the Gaussian white noise variance generated by the kth downlink user,
[0146] Considering the subcarrier allocation for enabled uplink and downlink users, the subcarrier set Define the subcarrier allocation matrix B = [b1, b2, ..., b J+K-S ] T , represents the subcarrier allocation status of (J+KS) uplink and downlink capable users, where b m =[b 1,m , b 2,m , ..., b Z,m ], b z,m ∈{0,1} is defined as follows:
[0147]
[0148] It is known that the subcarrier allocation matrix B is the user service state matrix e u and e dRe-allocate when determined, so redefine the subcarrier allocation matrix make satisfy:
[0149]
[0150] and
[0151] Process 3: Establish the system spectrum efficiency objective function optimization problem, where the system is the full-duplex massive MIMO network model constructed above.
[0152] In summary, the uplink spectrum efficiency of the jth uplink user and the downlink spectrum efficiency of the kth downlink user are defined as follows:
[0153]
[0154]
[0155] in
[0156]
[0157]
[0158]
[0159]
[0160] Where Φ and φ are the interference powers received by uplink user j and downlink user k respectively.
[0161] The total spectrum efficiency of uplink and downlink users is defined as:
[0162]
[0163] Therefore, the maximum total spectrum efficiency of uplink and downlink users is defined as follows:
[0164]
[0165] It can be seen that the maximization problem involves the coupling of multiple variables and is non-convex, so the conventional solution is complex. Considering that the base station antenna is used for uplink reception and matches the uplink user, and similarly, the base station antenna is used for downlink transmission and matches the downlink user, the base station antenna state matrix and the user service state matrix can be described by a comprehensive matrix G, as follows
[0166]
[0167] According to the matching principle between base station antennas and users, let the elements in the comprehensive matrix G that do not match the antennas and users be zero, and get the matrix as follows
[0168]
[0169] make
[0170]
[0171]
[0172] have
[0173]
[0174] Therefore, the objective function of maximizing the total spectrum efficiency of uplink and downlink users is redefined as
[0175]
[0176] Process 4: Building a DDPG model mapped to a full-duplex massive MIMO network
[0177] The variables of the optimization problem in process 3 are reduced, but the problem is still an NP-hard problem in polynomial time. The present invention uses DDPG (Deep Deterministic Policy Gradient, deep reinforcement learning) to solve the non-convex problem in process 3.
[0178] The DDPG model of the present invention consists of an agent, actions, states, and rewards.
[0179] Intelligent agent: full-duplex base station antenna (i.e. smart antenna) and uplink and downlink users;
[0180] Status: Signal-to-interference-noise ratio of uplink and downlink users and Matching between the base station antenna and the jth uplink user And the matching situation between the base station antenna and the kth downlink user and subcarrier allocation for each user Recorded as status s t ={s1, s2, s3}.
[0181] Action: Base station downlink precoding matrix W, uplink user transmit power p j ∈P, the above actions are collectively recorded as action a={W;p j ∈P}.
[0182] award:
[0183] The spectrum efficiency of uplink and downlink users is defined as follows:
[0184]
[0185]
[0186] Recorded as
[0187] The DDPG network of the present invention is composed of two neural networks with different structures, Actor and Critic, where each neural network contains a replica network with the same structure but different parameters. The own network is recorded as target_net and the replica network is recorded as eval_net. Therefore, Actor and Critic contain four neural networks in total, which are recorded as:
[0188] critic_target_net: Q′(s1, s2, s3, a|θ Q′ );
[0189] critic_eval_net: Q(s1, s2, s3, a|θ G );
[0190] actor_target_net:μ′(s1,s2,s3|θ μ′ );
[0191] actor_eval_net:μ(s1,s2,s3|θ μ );
[0192] Step 5: Optimizing the number of actions and states of the DDPG algorithm
[0193] Since the Actor in the DDPG model exhibits forward propagation characteristics and needs to output specific action values, it is known that reducing the number of optional actions will reduce the time complexity of the deep reinforcement learning algorithm.
[0194] In order to make the uplink and downlink users reach the silent state, the present invention usually searches the user's transmit / receive power from zero, that is, the user power of zero means that the user is in the silent state. However, in order to meet the minimum QoS throughput requirements of uplink and downlink users, it is not very meaningful to search the power value from zero in the actual communication system, which will introduce a large amount of calculation overhead. For this reason, the present invention sets a minimum power threshold so that the transmit / receive power is limited between the minimum and maximum thresholds, thereby reducing unnecessary action space. However, introducing the minimum power threshold is equivalent to putting all users in the working state by default, which conflicts with putting some users in the silent state. To resolve this contradiction, the present invention introduces the uplink and downlink user silent flag through process 2 to ensure that the uplink and downlink users can still be put in the silent state as needed while reducing the action space.
[0195] Assume that there is no residual self-interference in a full-duplex base station, and there is no inter-user interference between uplink and downlink users. Since uplink and downlink users are randomly distributed around the base station, and the interference between users is not considered, the performance expectations of each uplink and downlink user should be the same from a statistical point of view, satisfying:
[0196]
[0197] Based on the principle of fairness (i.e., in an ideal state, there is no interference, that is, there is no residual self-interference in a full-duplex base station, and there is no inter-user interference between uplink and downlink users), let
[0198]
[0199]
[0200] Solve to get p j =p min , w k =diag(0,…,0,w min , 0, …0)
[0201] From this we can see that the element w in action a k and p j Respectively limited to:
[0202]
[0203] p min ≤p j ≤p max , p j ∈P
[0204] Among them, the upper limit P max Determined by the hardware conditions of the base station, the upper limit p max Determined by the user's hardware conditions; k is the precoding matrix of the kth downlink user, satisfying w k ∈W={w1,w2,...,w K}. ||w k || 2 It is used to indicate the downlink user transmission power. Therefore, the present invention sets the lower limit of uplink and downlink power, which is reduced to a certain extent compared with the previous action value range. At the same time, the subcarrier allocation matrix used in the present invention is The number of original states is reduced, so that the number of users to which subcarriers can be allocated is only (J+KS).
[0205] The reduced action state
[0206] In summary, the present invention improves the learning efficiency of the algorithm from two perspectives: action and state.
[0207] Step 6: Execute the DDPG algorithm to output the optimal action and state
[0208] The DDPG algorithm process is as follows Figure 4 As shown:
[0209] Step 1:
[0210] For critic_eval_net: Q(s1, s2, s3, a|θ G ) and actor_eval_net: μ(s1, s2, s3|θ μ ) G and θ μ Randomly assign an initial value in the range of [0, 1].
[0211] Step 2:
[0212] Respectively change the θ in the eval network G and θ μ Assign to θ in the target network Q′ and θ μ′ ,have
[0213] θ Q′ ←θ G
[0214] θ μ′ ←θ μ
[0215] Step 3:
[0216] Initialize the experience pool R, which stores several {s t , a t , r t}, that is, the current state s t , selected action a t and the reward r t , each state consists of three elements, namely s t ={s t,1 ,s t,2 ,s t,3}, abbreviated as s in the present invention t ={s1, s2, s3}.
[0217] Step 4:
[0218] For Action Assign initial values and adopt equal power allocation strategy for uplink and downlink users.
[0219] Step 5:
[0220] Iteration is performed, and the iteration includes two layers: an outer loop and an inner loop. The outer loop iteration resets the state, that is, step 6 to step 16, and the inner loop iteration traverses the state steps, that is, step 7 to step 16, where the state steps are the superposition of moments, that is, reflected in t=t+1.
[0221] Step 6:
[0222] The outer loop iteration assigns the current value to state s0. State s0 is the initial state, that is, the initial step 0.
[0223] Step 7:
[0224] Each iteration of the inner loop selects action a from actor_eval_net t , and the action a t In full-duplex environment, the state changes from s t Migrate to t+1 And get reward t ,have
[0225]
[0226]
[0227]
[0228] Step 8:
[0229] The current state s t , next state s t+1 , and the action a selected t and the reward r t They are stored in the experience pool R respectively.
[0230] Step 9:
[0231] Randomly select a number of groups (s t , a t , r t ,s t+1 ) to train the DDPG network.
[0232] Step 10:
[0233] Calculate action a through the eval critic network t The following curQ:
[0234] curQ=Q(s1,s2,s3,a|θ G )
[0235] Step 11:
[0236] According to the action α output by the target actor network t+1 , use the target critic network to calculate targetQ for the evalcritic network as follows:
[0237] targetQ=r t +gamma·Q′(s1,s2,s3,μ′(s1,s2,s3|θ μ′ )|θ Q′ )
[0238] Where gamma represents the discount factor, gamma∈[0,1].
[0239] Step 12:
[0240] According to curQ and targetQ obtained in step 10 and step 11 respectively, calculate the gradient of the mean square error of the loss function:
[0241] Loss=mean(diff(targetQ-curQ))
[0242] Step 13:
[0243] Update the parameter θ in curQ G .
[0244] Step 14:
[0245] Actor's parameters θ μ Updated by Monte Carlo estimation of the gradient, as follows
[0246]
[0247] Step 15:
[0248] Using the parameter θ G and θ μ The sliding average method is used to update the parameters θ of the target critic network and the target actor network respectively. Q′ and θ μ′ .
[0249] θ Q′ ←ρθ G +(1-ρ)θ Q′
[0250] θ μ′ ←ρθ μ +(1-ρ)θ μ′
[0251] where ρ∈(0, 1) is a hyperparameter.
[0252] Step 16:
[0253] The maximum number of inner loop iterations, i.e. the maximum number of state steps t, has not been reached max ; Return to step 7, otherwise return to step 6; if the maximum number of outer loop iterations is reached, output the optimal action And the state corresponding to the action
[0254] The present invention allocates power through full-duplex intelligent antenna selection of base stations and silent identification of uplink and downlink users, effectively improving the total spectrum efficiency of uplink and downlink users in the cell. At the same time, the number of actions and states selected is further reduced on the basis of deep reinforcement learning, effectively shortening the algorithm learning time. Therefore, the present invention reduces the computational complexity while meeting the QoS of communication system users, and is suitable for deployment in full-duplex large-scale MIMO wireless communication systems.
[0255] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A MIMO full-duplex power allocation method based on smart antenna, characterized in that: The following steps are involved: S1. Establishing a full-duplex massive MIMO network model; the full-duplex massive MIMO network model includes a MIMO network, a base station equipped with N antennas, and Z users communicating with the base station, the base station operates in full-duplex mode, each user is equipped with one antenna, and the user operates in half-duplex mode; S2. Define the three elements of smart antenna, user state and subcarrier allocation in the full-duplex massive MIMO network model; the smart antenna intelligently selects one of four working modes for each antenna of the base station, and the four working modes include for sending, for receiving, sending and receiving coexisting, and neither sending nor receiving; the user state is defined as using a global unique identifier to define the current state for all uplink and downlink users, and construct an uplink user set and a downlink user set; the subcarrier allocation includes: defining a subcarrier allocation matrix for redistribution according to the uplink user set and the downlink user set The reallocated subcarrier allocation matrix Indicates the subcarrier allocation status of uplink and downlink capable users; S3, constructing an optimized spectrum efficiency objective function for a full-duplex massive MIMO network model; S4. Establish a DDPG model to solve the optimized spectrum efficiency objective function: establish a DDPG model mapped to a full-duplex massive MIMO network model, wherein the DDPG model includes an agent, a state, an action, and a reward, and is provided with a DDPG network, wherein the DDPG network includes two neural networks with different structures, namely, Actor and Critic; S5. Optimize the states and actions in the DDPG model: Based on the principle of fairness, find the lower limit of the uplink user transmit power and the lower limit of the base station downlink precoding matrix to obtain the reduced action and state space; S6. Run the DDPG model to output the optimal action and state to achieve MIMO full-duplex power allocation; In step S2, the uplink user set and the downlink user set are: At the current moment, the uplink user meets Downlink user satisfaction α 2β-1 (t) is the uplink service identifier of the βth user at the current moment, α 2β (t) is the downlink service identifier of the βth user at the current moment; the global unique identifier A = {α1(t), α2(t), ..., α 2Z (t)}, where α 2Z (t) is the service identifier of the Zth user at time t; α 2β-1 (t) = 1, indicating that user β is currently enabling uplink services, α 2β-1 (t) = 0, indicating that user β does not enable uplink services at the current moment, α 2β-1 (t) = 2, indicating that user β is currently invoking the uplink service and will not be muted; α 2β (t) = 1, indicating that user β is currently enabling downlink services, α 2β (t) = 0, indicating that user β does not enable downlink services at the current moment, α 2β (t) = 2, indicating that user β is currently invoking the downlink service and will not be muted; Sic u (t-1) = 1 or 0 indicates that the user's uplink service was muted or not muted at the previous moment respectively; Sic d (t-1) = 1 or 0 respectively indicates that the user's downlink service was muted or not muted at the previous moment; Req u (t) = 1 or 0 means that the user has received or not received the uplink service request at the current moment; Req d (t) = 1 or 0 indicates that the user has received or has not received a downlink service request at the current moment; In step S2, the redistributed subcarrier allocation matrix is defined according to the uplink user set and the downlink user set. include: Define the uplink user service state matrix e at the current moment according to the uplink user set and the downlink user set u and downlink user service state matrix e d ; Uplink user service state matrix e u and downlink user service state matrix e d After determining, define the reallocated subcarrier allocation matrix The uplink user service status matrix e at the current moment u and downlink user service state matrix e d for: in Defined as: Defined as: Among them, 1≤j≤J, 1≤k≤K; represents the subcarrier allocation status of (J+KS) uplink and downlink active users, Z users include J uplink users, K downlink users and O dormant users, J+K+O=Z; where 1≤z≤Z; 1≤m≤M; satisfy: and Subcarrier set M is the number of subcarriers; uplink user set J is the number of uplink users; the downlink user set K is the number of downlink users; the set of muted uplink and downlink users S is the number of muted uplink and downlink users; The spectrum efficiency objective function optimized in step S3 is: Among them, R represents the total spectrum efficiency of uplink and downlink users, G is the comprehensive matrix, The synthesis matrix with zeroed elements for antenna and user mismatches, is the uplink user service state matrix e u and downlink user service state matrix e d Determine the subcarrier allocation matrix after redistribution, x u Indicates the symbol sent by the uplink user to the base station, x d Indicates the symbol sent by the base station to the downlink user, q u and q d is the base station antenna state matrix, which represents the antennas used for uplink reception and downlink transmission of the base station, respectively, and w k is the precoding matrix of the kth downlink user, satisfying w k ∈W={w1,w2,...,w K }, W is the precoding matrix of all downlink users, that is, the base station downlink precoding matrix, p j is the transmission power of the jth uplink user, satisfying p j ∈P={p1,p2,...,p J }, P is the transmission power of all uplink users; Z users include J uplink users, K downlink users and O dormant users, J+K+O=Z; subcarrier set M is the number of subcarriers; uplink user set J is the number of uplink users; the downlink user set K is the number of downlink users; the set of muted uplink and downlink users S is the number of muted uplink and downlink users; In step S4, the intelligent agent is a smart antenna and uplink and downlink users in a full-duplex massive MIMO network model; The state is: signal to interference and noise ratio of uplink user and the signal-to-interference-noise ratio of downlink users Matching between the base station antenna and the jth uplink user And the matching situation between the base station antenna and the kth downlink user And subcarrier allocation for each uplink and downlink user Recorded as status s = {s1, s2, s3}; The actions are: base station downlink precoding matrix W, uplink user transmission power p j ∈P, denoted as action a={W; p j ∈P}; The reward is: Spectral efficiency of uplink users and spectrum efficiency for downlink users in: Recorded as In step S4, the DDPG network is provided with two neural networks with different structures, Actor and Critic, wherein each neural network contains a replica network with the same structure as itself but different parameters, the own network is recorded as target_net, the replica network is recorded as eval_net, and the Actor and Critic contain a total of four neural networks, which are recorded as: critic_target_net: Q′(s1, s2, s3, a|θ Q′ ), critic_eval_net: Q(s1, s2, s3, a|θ G ), actor_target_net: μ′(s1, s2, s3|θ Q′ ), actor_eval_net: μ(s1, s2, s3|θ G ); In step S5, the lower limit of the uplink user transmit power and the lower limit of the base station downlink precoding matrix are calculated based on the principle of fairness, and the calculation formula is: Solve to get p j =p min , w k =diag(0,…,0,w min , 0, …0); Reduced action state 2. The MIMO full-duplex power allocation method based on smart antenna according to claim 1, characterized in that: The step S6 comprises: S61, for critic_eval_net: Q(s1, s2, s3, a|θ G ) and actor_eval_net: μ(s1, s2, s3|θ μ ) G and θ μ Randomly assign initial values; S62, respectively, the θ in the eval network G and θ μ Assign to θ in the target network Q′ and θ μ′ ; S63, initialize the experience pool R, the experience pool R stores a number of {s t , a t , r t }; S64, for action Assign initial values, and adopt equal power allocation strategy for uplink and downlink users; S65, iterate, the iteration includes two layers: outer loop and inner loop. The outer loop iteration resets the state, and the inner loop iteration traverses the state steps until the maximum number of outer loop iterations is reached, and the optimal action is output. And the state corresponding to the action Realize MIMO full-duplex power allocation.
3. The MIMO full-duplex power allocation method based on smart antenna according to claim 2, characterized in that: The outer loop and inner loop process in step S65 include: S651, enter the outer loop iteration, the outer loop iteration assigns the current value to the state s0; S652, enter the inner loop iteration, each time the inner loop iteration selects action a from actor_eval_net t , and the action a t In full-duplex environment, the state changes from s t Migrate to t+1 And get reward t ; S653, the current state s t , next state s t+1 and the selected action a t and the reward r t They are stored in the experience pool R respectively; S654, randomly select several groups (s t , a t , r t ,s t+1 ) to train the DDPG network; S653, calculate action a through eval critic network t curQ under: curQ=Q(s1,s2,s3,a|θ G ); S654, action a output by target actor network t+1 , targetQ is calculated using the target critic network as the eval critic network. The calculation formula of targetQ is: <h2 style=";text-align:left;direction:ltr">targetQ=r<h2 style=";text-align:left;direction:ltr"> t <h2 style=";text-align:left;direction:ltr"> +gamma·Q′(s1,s2,s3,μ′(s1,s2,s3|θ<h2 style=";text-align:left;direction:ltr"> μ′ <h2 style=";text-align:left;direction:ltr"> )|θ<h2 style=";text-align:left;direction:ltr"> Q′ <h2 style=";text-align:left;direction:ltr"> ) Where gamma represents the discount factor, gamma∈[0,1]; S655. According to the obtained curQ and targetQ, the gradient of the mean square error of the loss function is calculated. The gradient calculation formula of the mean square error is: Loss=mean(diff(targetQ-curQ)) S656. Update parameter θ in curQ G ; S657, Update the Actor's parameters θ using the Monte Carlo estimation method of the gradient μ ; Monte Carlo estimation method updates the Actor's parameters θ μ The calculation formula is: S658, using parameter θ G and θ μ The sliding average method is used to update the parameters θ of the target critic network and the target actor network respectively. Q′ and θ μ′ ; The calculation formula is: i Q′ ←ρθ G +(1-p)θ Q′ i μ′ ←ρθ μ +(1-p)θ μ′ Where ρ∈(0,1) is a hyperparameter; S659: Determine whether the maximum number of inner loop iterations has been reached. If not, return to S652; if yes, return to S651; Determine whether the maximum number of outer loop iterations has been reached. If not, return to S651; if yes, output the optimal action. And the state corresponding to the action Realize MIMO full-duplex power allocation.
4. The MIMO full-duplex power allocation method based on smart antenna according to claim 3, characterized in that: In step S652, the state changes from s t Migrate to t+1 And get reward t , the calculation formula includes: Among them, time t and (t+1) are defined as the current time and the next time respectively.
