Millimeter wave cooperative beamforming method assisted by microwave channel

Abstract

The invention relates to a microwave channel assisted millimeter wave cooperative beam forming method, and belongs to the field of unmanned aerial vehicle communication. Comprising the following steps: establishing a distributed mapping relation from historical microwave channel information to millimeter wave cooperative beamforming, and in the distributed mapping relation, predicting a millimeter wave cooperative beamforming vector according to the historical microwave channel information by taking optimization of spectrum efficiency of an interference network as a target; establishing a plurality of long-short-term memory network models; the model takes historical microwave channel information of multiple links as input and outputs a predicted millimeter wave cooperative beamforming vector; using a centralized training distributed execution framework in multi-agent deep reinforcement learning to carry out self-supervised centralized training on the distributed complex long-short term memory network model; and carrying out testing and distributed deployment on the trained distributed complex long-short-term memory network model. Real-time and accurate millimeter wave channel information does not need to be collected, and the application range of an unmanned aerial vehicle communication system is widened.

Description

Technical Field

[0001] The present invention belongs to the field of unmanned aerial vehicle (UAV) communications, and in particular relates to a microwave channel-assisted millimeter wave collaborative beamforming method. Background Art

[0002] Drone communication is an emerging communication method with the advantages of good communication environment (most of which are line-of-sight communication environments), flexibility, agility, and strong environmental adaptability. It will serve as an effective supplement to future ground wireless communication networks and play a huge role in scenarios such as the lack of communication infrastructure and emergency rescue.

[0003] Drone communication systems face the following challenges: Because drones offer a robust communication environment, when multiple drones serve as access points and provide communication services to multiple end users, multi-link interference can occur between them. When using multi-antenna communication systems, beamforming can be used to coordinate interference and optimize overall network performance. However, millimeter-wave communication systems typically utilize large-scale antenna arrays, making beamforming computationally extremely complex. Furthermore, accurate millimeter-wave channel information is difficult to obtain. Therefore, a method is needed that is computationally simple and can quickly derive collaborative beamforming vectors, independent of millimeter-wave channel information. Summary of the Invention

[0004] In view of the above analysis, the present invention aims to provide a microwave channel-assisted millimeter-wave collaborative beamforming method to solve the problems of high computational complexity and dependence on millimeter-wave channel information in the existing technology.

[0005] The present invention discloses a microwave channel-assisted millimeter wave collaborative beamforming method, which is characterized by comprising:

[0006] Step S1: Establishing a distributed mapping relationship from historical microwave channel information to millimeter wave collaborative beamforming;

[0007] In the distributed mapping relationship, with the goal of optimizing the spectral efficiency of the interfering network, the millimeter-wave collaborative beamforming vector is predicted based on historical microwave channel information;

[0008] Step S2: Establishing a complex long short-term memory network model; the model implements the distributed mapping relationship in step S1, takes the historical microwave channel information of multiple links as input, and outputs a predicted millimeter wave collaborative beamforming vector;

[0009] Step S3: Using the centralized training distributed execution framework in multi-agent deep reinforcement learning, perform self-supervised centralized training on the distributed complex long short-term memory network model;

[0010] Step S4: testing and distributed deployment of the trained complex long short-term memory network model;

[0011] In actual deployment, each link makes decisions based on local microwave channel information, and then centrally collects the millimeter wave channel information of each link, and periodically updates the distributed complex long short-term memory network model.

[0012] Furthermore, in step S1, the process of establishing a distributed mapping relationship from historical microwave channel information to millimeter wave collaborative beamforming includes:

[0013] Step S101: Obtaining the signal-to-interference-and-noise ratio (SINR) and spectral efficiency of the millimeter-wave link received by the user based on a user received millimeter-wave signal model that considers normal communication signals, intra-cell interference, inter-cell interference, and link Gaussian noise;

[0014] Step S102: Establish a millimeter wave collaborative beamforming problem; the optimization goal of the problem is to maximize the sum of the millimeter wave spectrum efficiencies of all users in the interfering network by designing beamforming vectors for all users in the interfering network, thereby achieving optimal global network performance;

[0015] Step S103: In the microwave-millimeter wave dual-connection network, a mapping relationship is established from microwave channels to millimeter wave channels and then to millimeter wave collaborative beamforming vectors by using the channel similarity between microwave and millimeter waves.

[0016] Step S104: Under the conditions of the mobility of the UAV base station and the user, the microwave channel information collection delay, and the time-varying characteristics of the channel, a predictive mapping relationship from the historical microwave channel information to the millimeter wave collaborative beamforming vector is established.

[0017] Furthermore, in an area with M cells, each cell is provided with a dual-connection base station, and each base station serves K users. Let represents the set of all base stations;

[0018] The millimeter wave signal received by user k in cell m at time t is represented by y m,k (t) is:

[0019]

[0020] in, is the millimeter wave channel for normal communication between base station m and user k in the same cell at time t, is the millimeter wave channel between base station i and user k in cell m at time t; “H” represents the conjugate transpose operation; f m,k (t), f m,j (t) and f i,j (t) denotes the beamforming vectors of cell m serving user k, cell m serving user j, and cell i serving user j at time t, respectively;

[0021] xm,k (t), x m,j (t) and x i,j (t) are the signals sent by cell m to user k, cell m to user j, and cell i to user j at time t, respectively. m,k Represents the link Gaussian white noise.

[0022] Furthermore, the signal-to-interference-and-noise ratio of the millimeter wave link from cell m to user k at time t is and spectrum efficiency They are:

[0023]

[0024] and

[0025]

[0026] Furthermore, the millimeter wave collaborative beamforming problem is:

[0027]

[0028] Among them, P max Indicates the maximum transmit power of the millimeter wave transmitter.

[0029] Furthermore, in the complex long short-term memory network model,

[0030] 1) Using complex normalization operations to normalize the overall characteristics of the input time-series multi-link microwave channel information at each moment;

[0031] 2) The standardized time-series multi-link microwave channel information is passed to the complex fully connected neural network module for processing;

[0032] 3) The output of the complex fully connected neural network module is passed to the complex rectified linear unit for nonlinear activation;

[0033] 4) The output of the complex rectified linear unit is passed to the complex layer normalization module for complex feature normalization;

[0034] 5) The output of the complex layer normalization module is passed to the CVLSTM module to extract time-varying patterns;

[0035] 6) The output of the CVLSTM module is passed to the decoder CVFCN and the complex tanh function to generate the complex beamforming vector for a single link.

[0036] Furthermore, in the complex long short-term memory network model, a complex normalization operation is used to normalize the overall characteristics of the input time series multi-link microwave channel information at each moment. The mathematical expression is:

[0037]

[0038] Where, is the initial input t ′ Multi-link microwave channel information at each moment, where: is the microwave inter-cell interfered link channel information of cell m, is the microwave inter-cell interference link channel information of cell m; represents the standardized microwave channel information, represents the mean of the input channel information calculated along the feature dimension, and V is the covariance matrix of the input complex channel information.

[0039] Furthermore, the complex long short-term memory network model is trained in a self-supervised centralized manner, and the loss function at time t is Expressed as:

[0040]

[0041] Among them, the function The independent variables include the millimeter wave channel information h of all links mmW (t) and the beamforming vector {f1(t),f2(t),…,f MK (t)}Network global information.

[0042] Furthermore, in step S3, a centralized training distributed execution framework is used in multi-agent deep reinforcement learning, and a centralized judge-distributed actor architecture is adopted:

[0043] The distributed actor is a complex long short-term memory network model distributed in each agent, which outputs a local collaborative beamforming vector to the centralized judge;

[0044] A centralized evaluator is used to evaluate the global value of the multi-agent distributed decision-making based on the sum of the spectral efficiencies of the interfering networks and serves as a training target for the multi-agent decision-making network;

[0045] During training, the centralized evaluator collects millimeter-wave channel information and millimeter-wave beamforming vectors from all links and then calculates the training target for the complex long-short-term memory network model based on this information. The millimeter-wave channel information comes from the external environment, while the millimeter-wave beamforming vectors are inferred by the complex long-short-term memory network model of each link based on local microwave channel information.

[0046] After calculating the training target of the complex long short-term memory network model, the network parameters of the complex long short-term memory network model are updated using the stochastic gradient descent method and passed to each distributed complex long short-term memory network model for deployment or the next round of training.

[0047] Furthermore, in the stochastic gradient descent method, according to the chain rule of gradient calculation, the training gradient for:

[0048]

[0049] in, Indicates microwave channel information The beamforming vector of link n obtained by and neural network parameters θ;

[0050] is the gradient of the beamforming vector with respect to the neural network parameter θ;

[0051] is the training objective for updating the neural network parameters θ;

[0052] is the gradient of the training target with respect to the beamforming vector;

[0053] Represents the averaging operation of all link gradients in a single batch.

[0054] The present invention can achieve at least one of the following beneficial effects:

[0055] The present invention can generate a globally optimized millimeter-wave collaborative beamforming vector using only local historical microwave channel information, achieving multi-link interference coordination while significantly reducing computational complexity. It also eliminates the need to collect real-time and accurate millimeter-wave channel information, thereby increasing the application scope of the UAV communication system. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The accompanying drawings are only used for the purpose of illustrating specific embodiments and are not to be considered as limiting the present invention. Throughout the drawings, the same reference symbols denote the same components.

[0057] Figure 1 This is a flow chart of a microwave channel-assisted millimeter-wave collaborative beamforming method in an embodiment of the present invention;

[0058] Figure 2 Schematic diagram of the microwave channel-assisted CVLSTM millimeter-wave collaborative beamforming technology in an embodiment of the present invention;

[0059] Figure 3 Schematic diagram of the CVLSTM operation structure in an embodiment of the present invention;

[0060] Figure 4 Schematic diagram of a distributed deployment architecture for self-supervised multi-agent centralized training of collaborative beamforming tasks in an embodiment of the present invention;

[0061] Figure 5This is a flow chart of a distributed intelligent collaborative beamforming algorithm based on a multi-agent CVLSTM architecture in an embodiment of the present invention;

[0062] Figure 6 1 is a performance comparison diagram of different collaborative beamforming methods in an embodiment of the present invention;

[0063] Figure 7 FIG. 4 is a diagram comparing the computational complexity of different collaborative beamforming methods in an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The preferred embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, and are not used to limit the scope of the present invention.

[0065] A specific embodiment of the present invention discloses a microwave channel assisted millimeter wave collaborative beamforming method, such as Figure 1 Shown, including:

[0066] Step S1: Establishing a distributed mapping relationship from historical microwave channel information to millimeter wave collaborative beamforming;

[0067] In the distributed mapping relationship, with the goal of optimizing the spectral efficiency of the interfering network, the millimeter-wave collaborative beamforming vector is predicted based on historical microwave channel information;

[0068] Step S2: Establishing a complex long short-term memory network model; the model implements the distributed mapping relationship in step S1, takes the historical microwave channel information of multiple links as input, and outputs a predicted millimeter wave collaborative beamforming vector;

[0069] Step S3: Using the centralized training distributed execution framework in multi-agent deep reinforcement learning, perform self-supervised centralized training on the distributed complex long short-term memory network model;

[0070] Step S4: testing and distributed deployment of the trained complex long short-term memory network model;

[0071] In actual deployment, each link makes decisions based on local microwave channel information, and the central background server collects millimeter wave channel information of each link to periodically update the distributed complex long short-term memory network model.

[0072] In step S1, the process of establishing a distributed mapping relationship from historical microwave channel information to millimeter wave collaborative beamforming includes:

[0073] Step S101: Obtaining the signal-to-interference-and-noise ratio (SINR) and spectral efficiency of the millimeter-wave link received by the user based on a user received millimeter-wave signal model that considers normal communication signals, intra-cell interference, inter-cell interference, and link Gaussian noise;

[0074] Step S102: Establish a millimeter wave collaborative beamforming problem; the optimization goal of the problem is to maximize the sum of the millimeter wave spectrum efficiencies of all users in the interfering network by designing beamforming vectors for all users in the interfering network, thereby achieving optimal global network performance;

[0075] Step S103: In the microwave-millimeter wave dual-connection network, a mapping relationship is established from microwave channels to millimeter wave channels and then to millimeter wave collaborative beamforming vectors by using the channel similarity between microwave and millimeter waves.

[0076] Step S104: Under the conditions of the mobility of the UAV base station and the user, the microwave channel information collection delay, and the time-varying characteristics of the channel, a predictive mapping relationship from the historical microwave channel information to the millimeter wave collaborative beamforming vector is established.

[0077] Specifically, in an area with M cells, each cell has a dual-connection base station providing services, and each base station serves K users. Let represents the set of all base stations;

[0078] Then, in the user receiving millimeter wave signal model established in step S101, the millimeter wave signal received by user k in cell m at time t is represented by y m,k (t) is:

[0079]

[0080] in, is the millimeter wave channel for normal communication between base station m and user k in the same cell at time t, is the millimeter wave channel between base station i and user k in cell m at time t; “H” represents the conjugate transpose operation; f m,k (t), f m,j (t) and f i,j (t) denotes the beamforming vectors of cell m serving user k, cell m serving user j, and cell i serving user j at time t, respectively;

[0081] x m,k (t), x m,j (t) and x i,j (t) are the signals sent by cell m to user k, cell m to user j, and cell i to user j at time t, respectively. m,k represents the link Gaussian white noise.

[0082] According to the user receiving millimeter wave signal model, the millimeter wave link signal to noise ratio of cell m to user k at time t is obtained and spectrum efficiency They are:

[0083]

[0084] and

[0085]

[0086] Then, in step S102, the collaborative beamforming aims to design beam vectors for all users in the interfering network to achieve optimal global network performance, in order to maximize the spectrum efficiency of all users and as the optimization goal, the established millimeter wave collaborative beamforming problem is:

[0087]

[0088] Among them, P max Indicates the maximum transmit power of the millimeter wave transmitter.

[0089] The calculation of the global collaborative beamforming vector can be simplified to the local normal communication channel information Interfered link channel information within the cell Channel information of the inter-cell interfered link and intercellular interference link channel information Perform distributed computing.

[0090] In step S103,

[0091] In a microwave-millimeter-wave dual-connection network, the channel similarity between microwave and millimeter waves can be used to establish a mapping relationship from microwave channels to millimeter-wave channels and then to millimeter-wave collaborative beamforming vectors:

[0092]

[0093] in, Contains normal communication channel information of the microwave local area Interfered link channel information within the cell and the channel information of the intercellular interfered link Indicates microwave intercellular interference link channel information; is the millimeter-wave collaborative beamforming vector obtained by leveraging the channel similarity between microwave and millimeter waves.

[0094] In step S104, a predictive mapping relationship is established from historical microwave channel information to millimeter wave collaborative beamforming vectors under the conditions of the mobility of the UAV base station and the user, the delay in collecting microwave channel information, and the time-varying characteristics of the channel:

[0095]

[0096] Where T and t d represent the length of historical microwave channel information sequence and input delay respectively.

[0097] The overall network structure of the Complex-Valued Long Short-Term Memory (CVLSTM) model established in step S2 is shown in the attached figure. Figure 2 As shown in Figure 2, the system receives historical microwave channel information sequences from its own normal communication link, interference link, and interfered link as input, and the time series data is input along the batch dimension.

[0098] The processing of the complex long short-term memory network model CVLSTM includes:

[0099] Step S201, using a complex normalization operation to normalize the overall characteristics of the input time-series multi-link microwave channel information at each moment;

[0100] Step S202: The normalized time-series multi-link microwave channel information is transferred to a complex fully connected neural network module for processing;

[0101] Step S203: The output of the complex fully connected neural network module is passed to a complex rectified linear unit for nonlinear activation;

[0102] Step S204: The output of the complex rectified linear unit is passed to the complex layer normalization module for complex feature normalization;

[0103] Step S205: The output of the complex layer normalization module is passed to the CVLSTM module to extract the time-varying pattern;

[0104] In step S206 , the output of the CVLSTM module is passed to the decoder CVFCN and the complex tanh function to generate a complex beamforming vector for a single link.

[0105] Specifically, in practical applications, the channel information of these different links and at different times usually spans multiple orders of magnitude and has random distribution characteristics, and its probability distribution is difficult to determine. However, deep learning algorithms usually assume that all features have the same effect on inference and prediction. If random features of multiple orders of magnitude are directly input into the model, the following problems may occur: features with extremely small orders of magnitude may not be effectively recognized, while features with larger orders of magnitude will continue to dominate the model inference process. Therefore, the present invention standardizes the input channel information so that its amplitude range and distribution characteristics are more suitable for neural network reasoning.

[0106] Then, in step S201, a complex normalization operation is used to normalize the overall characteristics of the input time-series multi-link microwave channel information at each moment. The mathematical expression is:

[0107]

[0108] Where, is the initial input t ′ Multi-link microwave channel information at each moment, where: is the microwave inter-cell interfered link channel information of cell m, is the microwave inter-cell interference link channel information of cell m; represents the standardized microwave channel information, represents the mean of the input channel information calculated along the feature dimension, and V is the covariance matrix of the input complex channel information.

[0109] The expression for V is:

[0110]

[0111] Where, V rr and V ii Respectively represent the autocovariance of the real and imaginary parts of the input complex channel information, V ri and V ir are the cross-covariances between the real and imaginary parts, Cov(·) represents the covariance calculation function, and They are used to extract the real and imaginary parts of the complex number, respectively. This normalization method can convert the input complex channel information into a complex standard normal distribution with a mean of 0, a covariance of 1, and a pseudo-covariance of 0, so that the normalized complex data exhibits circular symmetry on the real-imaginary part plane.

[0112] In step S202, the standardized multi-link multi-antenna channel information is input into a complex-valued fully connected neural network (CVFCN) module for processing.

[0113] The core idea of ​​the CVFC module is to extend the internal operations of the neural network to the complex domain.

[0114] The complex weights and inputs of the CVFC module are W = W R +jW I and The product between weight and input can be expressed as:

[0115]

[0116] This formula is equivalent to

[0117]

[0118] like Figure 2 The CVFCN module structure shown in FIG, which uses two real FCNs as the real part of the complex weight W R and the imaginary part W I To realize complex number processing, the above formula is used to process the input complex channel information and output the complex result.

[0119] In step S203, the output of the CVFCN module is passed to the Complex Rectified Linear Unit (CRLU). ) performs nonlinear activation, which implements nonlinear transformation by applying two real ReLUs to the real and imaginary parts of the complex features respectively. Its specific form is as follows:

[0120]

[0121] In step S204, the output of the complex rectified linear unit is passed to the complex layer normalization module for complex feature normalization;

[0122] The complex layer normalization module uses the same normalization process as in step S201 to normalize complex features, adjusting the feature distribution to a standard normal distribution. After layer normalization, the input data dimensions are adjusted to the form (batch size, time series length, feature dimension) and input into the CVLSTM module for further processing.

[0123] In step S205, the output of the complex layer normalization module is passed to the CVLSTM module to extract the time-varying pattern;

[0124] The core of the CVLSTM module is to extend the internal operations of LSTM to the complex domain. The weight and feature product operations within LSTM are consistent with those of CVFCN. The calculation method of the gate activation function is also adjusted to the complex form: the sigmoid and tanh functions operate on the real and imaginary parts respectively, similar to Implementation method.

[0125] Specifically, the element of the input complex sequence at time t is s(t), which is decomposed into the real part s R (t) and the imaginary part s I (t). The complex hidden state h(t) and cell state c(t) in CVLSTM are respectively composed of their real parts h R (t), c R (t) and the imaginary part h I (t), c I (t) composition. The definitions of each gate are as follows:

[0126] The complex input gate i(t) is decomposed into the real part i R (t) and the imaginary part i I (t);

[0127] The complex forget gate f(t) is decomposed into the real part f R (t) and the imaginary part f I (t);

[0128] The complex unit gate g(t) is decomposed into the real part g R (t) and the imaginary part g I (t);

[0129] The complex output gate o(t) is decomposed into the real part o R (t) and the imaginary part o I (t).

[0130] The weight matrix of each gate is also expressed in complex form:

[0131] Complex weight of input gate: real part is and The imaginary part is and

[0132] Complex weight of forget gate: real part is and The imaginary part is and

[0133] Complex weight of unit gate: real part is and The imaginary part is and

[0134] The complex weight of the output gate: the real part is and The imaginary part is and

[0135] Based on the above definition, the output calculation process of CVLSTM is as follows:

[0136]

[0137] c(t)=f R (t)⊙c R (t-1)-f I (t)⊙c I (t-1)+i R (t)⊙g R (t)-i I (t)⊙g I (t)+j(f I (t)⊙c R (t-1)+f R (t)⊙c I (t-1)+i I (t)⊙g R (t)+i R (t)⊙g I (t)),

[0138] h(t)=o R (t)⊙tanh(c R (t))-o I (t)⊙tanh(c I (t))+j(o I (t)⊙tanh(c R (t))+o R (t)⊙tanh(c I (t))),

[0139] Here, σ(·) is the sigmoid function. CVLSTM also uses two parallel real-valued LSTM networks to process the real and imaginary parts of a complex sequence, respectively, and then merges them through complex operations.

[0140] Attachment Figure 3 Shows the overall operation structure of CVLSTM, where δ R and δ I Represents two different real-valued LSTM networks for processing the real part and the imaginary part, respectively.

[0141] In step S206, the output of the CVLSTM module is passed to the decoder CVFCN and the complex tanh function to generate a complex beamforming vector for a single link.

[0142] It should be noted that within the same cell, all generated complex beamforming vectors need to be scaled to meet the collaborative beamforming problem. The power constraints defined in .

[0143] Specifically, in step S3, the distributed CVLSTM model is trained in a self-supervised centralized manner.

[0144] Cooperative beamforming problem The goal is to maximize the sum of the interference network spectrum efficiency, and the training goal is to minimize the loss function. Therefore, the negative value of the sum of the interference network spectrum efficiency can be taken as the loss function. The loss function at time t is Expressed as:

[0145]

[0146] Among them, the function The independent variables include the millimeter wave channel information h of all links mmW (t) and the beamforming vector {f1(t),f2(t),…,f MK (t)}Network global information.

[0147] However, the CVLSTM models proposed in the embodiments of the present invention are all deployed locally on each link and can only obtain local information, making it difficult to achieve effective training locally.

[0148] To solve this problem, in step S3, a centralized training distributed execution framework is used in multi-agent deep reinforcement learning, and a centralized judge-distributed actor architecture is adopted:

[0149] The distributed actor is a complex long short-term memory network model distributed in each agent, responsible for executing specific decisions and outputting local collaborative beamforming vectors to the centralized judge;

[0150] A centralized judge is used to evaluate the global value of multi-agent distributed decision-making based on the sum of the spectral efficiencies of the interfering networks, and serves as the training target for the multi-agent decision-making network.

[0151] The distributed deployment architecture of self-supervised multi-agent centralized training for collaborative beamforming tasks is shown in the attached figure. Figure 4 As shown, for simplicity, the time index t is omitted in the annotations in the figure. In addition, the overall structure of the distributed CVLSTM collaborative beamforming model considered in the present invention is consistent (that is, the dimension of the multi-antenna microwave channel information input to each link is consistent and the dimension of the output beamforming vector is consistent), and the internal structure of the deep neural network is also the same. Therefore, the distributed CVLSTM model can share the neural network parameters θ, and the experience of multiple agents can be combined into one training batch to improve learning efficiency. The distributed intelligent collaborative beamforming algorithm process based on the multi-agent CVLSTM architecture is shown in the attached figure. Figure 5 As shown in Figure 1, it can be divided into the training phase, the testing phase, and the distribution deployment phase. Multiple rounds of iterative optimization are required during training.

[0152] During training, the centralized evaluator collects millimeter-wave channel information and millimeter-wave beamforming vectors from all links and then calculates the training target for the complex long-short-term memory network model based on this information. The millimeter-wave channel information comes from the external environment, while the millimeter-wave beamforming vectors are inferred by the complex long-short-term memory network model of each link based on local microwave channel information.

[0153] After calculating the training target of the complex long short-term memory network model, the network parameters of the complex long short-term memory network model are updated using the stochastic gradient descent method and passed to each distributed complex long short-term memory network model for deployment or the next round of training.

[0154] Specifically, the centralized evaluator can be deployed in the cloud or on a central server at the edge to collect millimeter-wave channel information and millimeter-wave beamforming vectors for all links. The millimeter-wave channel information comes from the external environment, while the millimeter-wave beamforming vectors are inferred by each link's CVLSTM (denoted as μ(·; θ)) based on local microwave channel information.

[0155] The beamforming vector of link n can be expressed as in It is an abbreviation for local multi-link microwave channel timing sequence.

[0156] Therefore, the training target for updating θ can be calculated

[0157] Then, the stochastic gradient descent method is used based on the training gradient Update θ, and the new θ will be passed to each distributed CVLSTM model for deployment or the next round of training.

[0158] According to the chain rule of gradient calculation, in the stochastic gradient descent method, according to the chain rule of gradient calculation, the training gradient for:

[0159]

[0160] in, Indicates microwave channel information The beamforming vector of link n obtained by and neural network parameters θ;

[0161] is the gradient of the beamforming vector with respect to the neural network parameter θ;

[0162] is the training objective for updating the neural network parameters θ;

[0163] is the gradient of the training target with respect to the beamforming vector;

[0164] Represents the averaging operation of all link gradients in a single batch.

[0165] It's worth noting that the training loss function (i.e., the negative of the global sum rate) varies in magnitude under different communication environments. A high sum rate can produce high gradient values, potentially leading to exploding gradients. To address this issue, the gradient norm is clipped to keep the gradient within a certain range to ensure effective training.

[0166] Step S4: Test and distribute the trained distributed CVLSTM model.

[0167] As attached Figure 5 As shown in the figure, in actual deployment, each link only needs local microwave channel information to make decisions. Meanwhile, the central backend server can collect millimeter-wave channel information from each link and periodically update the CVLSTM model. It is worth noting that millimeter-wave channel information is only used during the training phase, and model training and decision execution can be performed asynchronously. The collection of millimeter-wave channel information does not affect beamforming or normal communication. After beamforming is complete, millimeter-wave channel information can be estimated based on normal communication data.

[0168] The method of this embodiment has good scalability and low complexity by establishing a distributed mapping model from local microwave channel information to millimeter wave collaborative beamforming vectors. A CVLSTM model is designed to effectively process the input complex historical channel information, and a self-supervised multi-agent centralized training and distributed execution architecture is further adopted to realize the intelligent collaboration of the distributed CVLSTM model. Figure 6 The comparison shows that this method is close to or even exceeds the suboptimal solution performance of the iterative optimization algorithm WMMSE, but it only requires microwave channel information; the performance is significantly better than the millimeter wave beam tracking method that can also be implemented using microwave channels, proving that it not only focuses on the gain optimization of its own communication link, but also can suppress interference with other links in the network. In addition, Figure 7 It shows that the operation time of this method is only in milliseconds, which is significantly shorter than the hundreds of milliseconds of operation time of the WMMSE iterative optimization algorithm.

[0169] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with this technical field within the technical scope disclosed by the present invention should be covered by the scope of protection of the present invention.

Claims

1. A microwave channel-assisted millimeter wave collaborative beamforming method, characterized in that: include: Step S1: Establishing a distributed mapping relationship from historical microwave channel information to millimeter wave collaborative beamforming; In the distributed mapping relationship, with the goal of optimizing the spectral efficiency of the interfering network, the millimeter-wave collaborative beamforming vector is predicted based on historical microwave channel information; Step S2: Establishing a complex long short-term memory network model; the model implements the distributed mapping relationship in step S1, takes the historical microwave channel information of multiple links as input, and outputs a predicted millimeter wave collaborative beamforming vector; Step S3: Using the centralized training distributed execution framework in multi-agent deep reinforcement learning, perform self-supervised centralized training on the distributed complex long short-term memory network model; Step S4: testing and distributed deployment of the trained complex long short-term memory network model; In actual deployment, each link makes decisions based on local microwave channel information, and then centrally collects the millimeter wave channel information of each link, and periodically updates the distributed complex long short-term memory network model.

2. The microwave channel-assisted millimeter wave collaborative beamforming method according to claim 1, characterized in that: In step S1, the process of establishing a distributed mapping relationship from historical microwave channel information to millimeter wave collaborative beamforming includes: Step S101: Obtaining the signal-to-interference-and-noise ratio (SINR) and spectral efficiency of the millimeter-wave link received by the user based on a user received millimeter-wave signal model that considers normal communication signals, intra-cell interference, inter-cell interference, and link Gaussian noise; Step S102: Establish a millimeter wave collaborative beamforming problem; the optimization goal of the problem is to maximize the sum of the millimeter wave spectrum efficiencies of all users in the interfering network by designing beamforming vectors for all users in the interfering network, thereby achieving optimal global network performance; Step S103: In the microwave-millimeter wave dual-connection network, a mapping relationship is established from microwave channels to millimeter wave channels and then to millimeter wave collaborative beamforming vectors by using the channel similarity between microwave and millimeter waves. Step S104: Under the conditions of the mobility of the UAV base station and the user, the microwave channel information collection delay, and the time-varying characteristics of the channel, a predictive mapping relationship from the historical microwave channel information to the millimeter wave collaborative beamforming vector is established.

3. The microwave channel-assisted millimeter wave collaborative beamforming method according to claim 2, characterized in that: In an area with M cells, each cell has a dual-connection base station providing services, and each base station serves K users. Let represents the set of all base stations; The millimeter wave signal received by user k in cell m at time t is represented by y m,k (t) is: in, is the millimeter wave channel for normal communication between base station m and user k in the same cell at time t, is the millimeter wave channel between base station i and user k in cell m at time t; "H" represents the conjugate transpose operation; f m,k (t), f m,j (t) and f i,j (t) denotes the beamforming vectors of cell m serving user k, cell m serving user j, and cell i serving user j at time t, respectively; x m,k (t), x m,j (t) and x i,j (t) are the signals sent by cell m to user k, cell m to user j, and cell i to user j at time t, respectively. m,k Represents the link Gaussian white noise.

4. The microwave channel-assisted millimeter wave collaborative beamforming method according to claim 3, characterized in that: The signal-to-interference-and-noise ratio of the millimeter wave link from cell m to user k at time t and spectrum efficiency They are: and 5. The microwave channel-assisted millimeter wave collaborative beamforming method according to claim 4, characterized in that: The millimeter wave collaborative beamforming problem is: Among them, P max Indicates the maximum transmit power of the millimeter wave transmitter.

6. The microwave channel-assisted millimeter wave collaborative beamforming method according to claim 5, characterized in that: In the complex long short-term memory network model, 1) Using complex normalization operations to normalize the overall characteristics of the input time-series multi-link microwave channel information at each moment; 2) The standardized time-series multi-link microwave channel information is passed to the complex fully connected neural network module for processing; 3) The output of the complex fully connected neural network module is passed to the complex rectified linear unit for nonlinear activation; 4) The output of the complex rectified linear unit is passed to the complex layer normalization module for complex feature normalization; 5) The output of the complex layer normalization module is passed to the CVLSTM module to extract time-varying patterns; 6) The output of the CVLSTM module is passed to the decoder CVFCN and the complex tanh function to generate the complex beamforming vector for a single link.

7. The microwave channel-assisted millimeter wave collaborative beamforming method according to claim 6, characterized in that: In the complex long short-term memory network model, the complex normalization operation is used to normalize the overall characteristics of the input time series multi-link microwave channel information at each moment. The mathematical expression is: Where, is the initial input t ′ Multi-link microwave channel information at each moment, where: is the microwave inter-cell interfered link channel information of cell m, is the microwave inter-cell interference link channel information of cell m; represents the standardized microwave channel information, represents the mean of the input channel information calculated along the feature dimension, and V is the covariance matrix of the input complex channel information.

8. The microwave channel-assisted millimeter wave collaborative beamforming method according to claim 6, characterized in that: The complex long short-term memory network model is trained in a self-supervised centralized manner, and the loss function at time t is Expressed as: Among them, the function The independent variables include the millimeter wave channel information h of all links mmW (t) and the beamforming vector {f1(t),f2(t),…,f MK (t)}Network global information.

9. The microwave channel-assisted millimeter wave collaborative beamforming method according to claim 8, characterized in that: In step S3, a centralized training distributed execution framework is used in multi-agent deep reinforcement learning, and a centralized judge-distributed actor architecture is adopted: The distributed actor is a complex long short-term memory network model distributed in each agent, which outputs a local collaborative beamforming vector to the centralized judge; A centralized evaluator is used to evaluate the global value of the multi-agent distributed decision-making based on the sum of the spectral efficiencies of the interfering networks and serves as a training target for the multi-agent decision-making network; During training, the centralized evaluator collects millimeter-wave channel information and millimeter-wave beamforming vectors from all links and then calculates the training target for the complex long-short-term memory network model based on this information. The millimeter-wave channel information comes from the external environment, while the millimeter-wave beamforming vectors are inferred by the complex long-short-term memory network model of each link based on local microwave channel information. After calculating the training target of the complex long short-term memory network model, the network parameters of the complex long short-term memory network model are updated using the stochastic gradient descent method and passed to each distributed complex long short-term memory network model for deployment or the next round of training.

10. The microwave channel-assisted millimeter wave collaborative beamforming method according to claim 9, characterized in that: In the stochastic gradient descent method, according to the chain rule of gradient calculation, the training gradient for: in, Indicates microwave channel information The beamforming vector of link n obtained by and neural network parameters θ; is the gradient of the beamforming vector with respect to the neural network parameter θ; is the training objective for updating the neural network parameters θ; is the gradient of the training target with respect to the beamforming vector; Represents the averaging operation of all link gradients in a single batch.

Images