A multi-user beam prediction method based on artificial intelligence algorithm

Abstract

The present invention relates to a multi-user beam prediction method based on an artificial intelligence algorithm, the method comprising the following steps: constructing a total target optimization function; dividing a time slot into T micro-slots; in each micro-slot, using a deep deterministic policy gradient algorithm to predict the beam, and obtaining a global optimal solution for beam selection, wherein the state space of the deep deterministic policy gradient algorithm is established as the beam selected by each user in the t-1 time slot, the action space is established as the probability matrix of each user selecting each precoding variation amplitude in the t time slot, and the reward function is established as the sum of the modulus values ​​of all user received signals at time t. The present invention reduces the dimension of the action space, accelerates the convergence rate of the network, and reduces the computing power overhead required for algorithm training by transforming the complex precoding matrix selection of multiple users into a unified precoding matrix variation amplitude selection, thereby improving the versatility of the prediction method.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technology, and in particular to a multi-user beam prediction method based on an artificial intelligence algorithm. Background Art

[0002] With the development of communication technology, especially the application of MIMO technology, the quality of communication has been greatly improved. At the same time, the signaling overhead required for beam selection has also increased. The traditional beam matrix scheme needs to traverse all beam pairs, which is inefficient. Other beam matrix selection schemes are generally only solved for a single user. For example, the patent with publication number CN113411110A provides a millimeter wave communication beam training method based on deep reinforcement learning, which proposes a millimeter wave MIMO communication system for a single user, and realizes beam selection for a single user communication scenario by designing a deep reinforcement learning algorithm. However, this scheme cannot meet the beam prediction in a multi-user scenario, and lacks portability and efficiency. In addition, when there are multiple users in the scene, multiple agents are required to perform separate calculations, which increases the computing power loss. At the same time, the scheme selects the optimal action for beam selection, the network training lacks exploration, and it is easy to fall into a local optimal solution. Summary of the invention

[0003] In view of the problems of poor versatility, low communication efficiency and high signaling overhead in existing beam prediction methods when there are a large number of users in the scene, the present invention proposes a multi-user beam prediction method based on an artificial intelligence algorithm. The method models and solves the selection of a multi-user precoding matrix, reduces the signaling overhead required for beam selection, and improves the efficiency of beam selection.

[0004] To solve the above problems, the present invention adopts the following technical solutions:

[0005] A multi-user beam prediction method based on an artificial intelligence algorithm comprises the following steps:

[0006] Step 1: Construct the overall target optimization function;

[0007] Step 2: Divide a time slot into T micro-slots;

[0008] Step 3: In each micro-slot, a deep deterministic policy gradient algorithm is used to predict the beam to obtain the global optimal solution for beam selection. The state space of the deep deterministic policy gradient algorithm is established as the beam selected by each user in the t-1 time slot, the action space is established as the probability matrix of each user selecting each precoding change amplitude in the t time slot, and the reward function is established as the sum of the modulus values ​​of the received signals of all users at time t.

[0009] Compared with the prior art, the present invention has the following beneficial effects:

[0010] (1) The present invention uses an artificial intelligence algorithm to perform beam prediction and precoding matrix selection, and reduces signaling overhead through neural network learning;

[0011] (2) Through the design of the action space, the complex precoding matrix selection of multiple users is transformed into a unified precoding matrix change amplitude selection, which reduces the dimension of the action space and accelerates the convergence rate of the network;

[0012] (3) When there are user changes (such as adding or removing users or user changes), this algorithm does not need to retrain the network. It only needs to add the changed data to the training set and perform simple training. Compared with other solutions targeting a single user, this reduces the computing power required for algorithm training and improves the versatility of the prediction method.

[0013] (4) The action space output designed by the present invention is not a certain action, but the probability of each action selection. The probability of the optimal action being selected is high, and other actions also have the probability of being selected. This scheme increases the exploration of the algorithm, reduces the probability of the algorithm falling into the local optimal solution, and improves the stability of the algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A flowchart of a multi-user beam prediction method based on an artificial intelligence algorithm according to an embodiment of the present invention;

[0015] Figure 2 This is a schematic diagram of a multi-user downlink communication scenario;

[0016] Figure 3 Schematic diagram of time slot division. DETAILED DESCRIPTION

[0017] The technical solution of the present invention will be described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0018] The embodiment of the present invention proposes a multi-user beam prediction method based on an artificial intelligence algorithm, the purpose of which is to obtain a global optimal solution for beam selection. Figure 1 As shown, the multi-user beam prediction method of this embodiment includes the following steps:

[0019] Step 1: Construct the overall objective optimization function.

[0020] like Figure 1 As shown, it is a single-base-station multi-user downlink communication scenario, in which the channel state of each user is relatively similar.

[0021] Assume that the user list is {0,1,…,m,…,M}, and the codebook of user m, i.e. the precoding matrix, is set to w m, then the communication channel model H m It can be modeled as follows:

[0022]

[0023] Among them, α m is the large-scale shadow fading, K is the Rice K factor, is the scattering component, which is an independent and identically distributed complex Gaussian random variable with a mean and variance of 0. L and N are the number of antennas at the transmitting end and the mobile user side, respectively. m (φ r )and are the antenna array response vectors of the user and transmitter, respectively, r and θ t They are angle of arrival (AOA) and angle of departure (AOD) respectively.

[0024] Then the received signal of user m can be expressed as follows:

[0025]

[0026] in, is the channel equalization matrix. m is the signal sent by the mth user, x u is the transmission signal of the u-th user, where u ranges from {0,1,…,M} and u≠m. m and w u are the precoding matrices for the mth user and the uth user respectively. n is the noise vector. By orthogonal design of the channel equalization matrix, the signal interference between users can be eliminated, that is, the above formula (2) is transformed into the following form:

[0027]

[0028] Then the final goal to be achieved, that is, the overall target optimization function, can be modeled as:

[0029]

[0030] Where D represents the DFT codebook.

[0031] Step 2: Divide a time slot into T micro-slots;

[0032] In order to ensure the accuracy of beam selection, this embodiment divides a time slot into T micro-slots, such as Figure 3 As shown, a beam prediction and selection is performed once in each mini-slot.

[0033] Step 3: In each micro-slot, a deep deterministic policy gradient algorithm is used to predict the beam to obtain the global optimal solution for beam selection. The state space of the deep deterministic policy gradient algorithm is established as the beam selected by each user in the t-1 time slot, the action space is established as the probability matrix of each user selecting each precoding change amplitude in the t time slot, and the reward function is established as the sum of the modulus values ​​of the received signals of all users at time t.

[0034] In real mobile communication scenarios, the user's moving speed is not high. Therefore, for each user in the system, the precoding selected each time the beam is selected will only be within the i-column range around the precoding selected in the previous time slot, where i is a small integer. That is, the range of precoding change is a m ={-i,-(i-1),…,0,…,i-1,i}. Through this method, the complex precoding selection scheme in the multi-user scenario can be transformed into a simple and unified precoding change range selection scheme, which greatly reduces the complexity of the action space and improves the efficiency of the algorithm.

[0035] In this embodiment, an artificial intelligence algorithm is introduced to solve the beam prediction problem. Since the action space dimension is high in a multi-user scenario, the present invention considers using a deep deterministic policy gradient (DDPG) algorithm to predict and select beams, thereby obtaining a global optimal solution for beam selection.

[0036] In the DDPG algorithm of this embodiment, the state s t The input state space is constructed as the beam w selected by each user in time slot t-1 m (t-1). The action space is established as the probability matrix of each user selecting each precoding change amplitude in time slot t, that is, a t ={p0,p1,…,p m ,…,p M}, where p m ={p m,-i ,p m,-(i-1) ,…,p m,0 ,p m,i-1 ,p m,i} represents the probability of the mth user selecting each precoding change amplitude. Each time an action is selected, the probability of the optimal action being selected is the highest, and there is also a probability of selecting other actions. This method increases the number of explorations of the algorithm, prevents the algorithm from falling into a local optimal solution, and improves the stability of the algorithm. The reward function is established as the sum of the modulus values ​​of the signals received by all users at time t, that is,

[0037] In order to achieve the goal of obtaining the selection scheme of the precoding matrix with only simple training when the number of users changes, this embodiment also fixes the number of rows in the action space to a larger value U. When the number of users M is less than U, only the first M rows of data are selected for beam selection, and the remaining UM rows are set to a fixed value. When the number of users changes within the range of [0, U], the algorithm obtains the number of connected users in advance, and after training, uses the neural network to obtain the output action and uses it for beam selection.

[0038] Furthermore, the deep deterministic policy gradient algorithm in this embodiment includes the following steps:

[0039] S1: Initialization: online policy network parameters θ, target policy network parameters θ - , online value network parameter w, target value network parameter w - , Timing difference error δ t , discount factors γ, α, β;

[0040] S2: Determine whether the number of iterations episode is less than or equal to the maximum number of training iterations. If so, execute step S3; otherwise, execute step S8;

[0041] S3: Determine whether the time slot t is less than or equal to T, if so, execute step S4;

[0042] S4: According to the state s t Select action a t ;

[0043] S5: Observation reward function r(a t ,s t ) and the next state s t+1 ;

[0044] S6: Store a transition(s t ,a t ,r t ,s t+1 ) into the buffer;

[0045] S7: Extract D transitions from the buffer (s t ,a t ,r t ,s t+1 ) for batch training;

[0046] S8: The algorithm ends.

[0047] Furthermore, when performing batch training, extract a transition(s t ,a t ,r t ,st+1 )The process of training and updating network parameters is as follows:

[0048] Make decisions based on an online policy networka t ;

[0049] Update online policy network parameters Where q(s t ,a t ,w) is the Q value;

[0050] Calculate q based on the online value network t =q(s t ,a t ,w);

[0051] According to the target strategy network and target value network μ(s t |θ - ) and q(s t ,a t ,w - ) Calculate q t+1 ;

[0052] Calculate the timing difference error: δ t =q t -(r t +γ·q t+1 );

[0053] Update the online value network using the temporal difference error:

[0054] Update target value network: w - ←τ·w+(1-τ)·w - ; where τ is the discount factor;

[0055] Update the target policy network: θ - ←τ·θ+(1-τ)·θ - .

[0056] The multi-user beam prediction method based on artificial intelligence algorithm proposed in this embodiment has the following advantages:

[0057] Artificial intelligence algorithms are used for beam prediction and precoding matrix selection, and signaling overhead is reduced through neural network learning;

[0058] Through the design of action space, the complex precoding matrix selection of multiple users is transformed into a unified precoding matrix change range selection, which reduces the dimension of action space and accelerates the convergence rate of the network.

[0059] This embodiment proposes a beam selection method in a multi-user scenario. When there is a user change (such as adding or reducing a user or changing a user), the algorithm does not need to retrain the network. It only needs to add the changed data to the training set and perform simple training. Compared with other solutions for a single user, the computing power required for algorithm training is reduced, and the versatility of the prediction method is improved.

[0060] The action space output designed in this embodiment is not a certain action, but the probability of each action selection. The probability of the optimal action being selected is high, and other actions also have the probability of being selected. This solution increases the exploration of the algorithm, reduces the probability of the algorithm falling into a local optimal solution, and improves the stability of the algorithm.

[0061] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0062] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A multi-user beam prediction method based on artificial intelligence algorithm, characterized in that: The following steps are involved: Step 1: Construct the overall target optimization function; Step 2: Divide a time slot into T micro-slots; Step 3: In each micro-slot, a deep deterministic policy gradient algorithm is used to predict the beam to obtain the global optimal solution for beam selection. The state space of the deep deterministic policy gradient algorithm is established as the beam selected by each user in the t-1 time slot, the action space is established as the probability matrix of each user selecting each precoding change amplitude in the t time slot, and the reward function is established as the sum of the modulus values ​​of the received signals of all users at time t; The deep deterministic policy gradient algorithm includes the following steps: S1: Initialization: online policy network parameters θ, target policy network parameters θ - , online value network parameter w, target value network parameter w - , timing difference error δ t , discount factors γ, α, β; S2: Determine whether the number of iterations episode is less than or equal to the maximum number of training iterations. If so, execute step S3; otherwise, execute step S8; S3: Determine whether the time slot t is less than or equal to T, if so, execute step S4; S4: According to the state s t Select action a t ; S5: Observation reward function r(a t ,s t ) and the next state s t+1 ; S6: Store a transition(s t ,a t ,r t ,s t+1 ) into the buffer, r t is the reward function value; S7: Extract D transitions from the buffer (s t ,a t ,r t ,s t+1 ) for batch training; S8: The algorithm ends; When performing batch training, extract a transition(s t ,a t ,r t ,s t+1 )The process of training and updating network parameters is as follows: Make decisions based on an online policy networka t ; Update online policy network parameters Calculate q based on the online value network t =q(s t ,a t ,w); According to the target strategy network and target value network μ(s t |θ - ) and q(s t ,a t ,w - ) Calculate q t+1 ; Calculate the timing difference error: δ t =q t -(r t +γ·q t+1 ); Update the online value network using the temporal difference error: Update target value network: w - ←τ·w+(1-τ)·w - ; where τ is the discount factor; Update the target policy network: θ - ←τ·θ+(1-τ)·θ - .

2. The multi-user beam prediction method based on artificial intelligence algorithm according to claim 1, characterized in that: The overall objective optimization function is: Among them, y m represents the received signal of user m, w m represents the codebook of user m, D represents the DFT codebook, and M represents the total number of users.

3. The multi-user beam prediction method based on artificial intelligence algorithm according to claim 2, characterized in that: The user's communication channel model H m Modeled as follows: Among them, α m is the large-scale shadow fading, K is the Rice K factor, is the scattering component, which is an independent and identically distributed complex Gaussian random variable with a mean and variance of 0. L and N are the number of antennas at the transmitter and mobile user side, respectively. m (φ r )and are the antenna array response vectors of the user and transmitter, respectively, r and θ t are the arrival angle and departure angle respectively; Then the received signal of user m can be expressed as follows: in, is the channel equalization matrix, x m is the signal sent by the mth user, x u is the signal sent by the u-th user, the value range of u is {0,1,…,M} and u≠m, w m and w u are the precoding matrices for the mth user and the uth user respectively, and n is the noise vector. By orthogonal design of the channel equalization matrix, the signal interference between users is eliminated, and the above formula (2) is transformed into the following form:

4. The multi-user beam prediction method based on artificial intelligence algorithm according to claim 1, characterized in that: The number of rows in the action space is fixed to U. When the number of users M is less than U, only the first M rows of data are selected for beam selection, and the remaining UM rows are set to a fixed value. When the number of users changes in the range [0, U], the algorithm obtains the number of connected users in advance and uses appropriate actions for beam selection after training.

Images