Path scheduling and beam forming design system and method for distributed RIS-assisted millimeter wave multi-hop system

Abstract

The invention discloses a path scheduling and beam forming design system and method for a distributed RIS auxiliary millimeter wave multi-hop system, and the system comprises a base station which comprises N antennas; a plurality of RISs, wherein each RIS comprises M reflection units; the plurality of terminal devices share the same multi-hop path to receive own signals; wherein joint optimization is carried out on multi-hop path scheduling, base station active beam forming and RIS passive beam forming, so that the total achievable rate of the system is maximized. According to the distributed RIS-assisted millimeter wave multi-hop communication system framework, the base station transmits signals to the remote user through a multi-RIS cascade multi-hop path, the problems that the transmission distance of millimeter wave communication is limited and the signals are blocked are solved, and multi-RIS cooperative transmission is achieved. The method can be widely applied to the technical field of wireless communication.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technology, and in particular to a path scheduling and beamforming design system and method for a distributed RIS-assisted millimeter wave multi-hop system. Background Art

[0002] In order to overcome the problems of short transmission distance of millimeter-wave signals and easy obstruction by obstacles, the academic community has conducted extensive research on RIS-assisted millimeter-wave systems.

[0003] The first prior art scheme considers a RIS-assisted millimeter wave system and proposes an algorithm based on block coordinate descent to jointly optimize the beamforming of the base station, RIS discrete phase shift and user scheduling strategy to maximize the total achievable rate of the system. The second prior art scheme considers a distributed RIS-assisted large-scale multiple-input multiple-output (MIMO) system, which improves the performance of millimeter wave communication in areas with weak coverage by jointly optimizing RIS phase shift and base station beamforming. The third prior art scheme studies the beamforming design of multi-RIS-assisted millimeter wave multi-antenna orthogonal frequency division multiplexing (OFDM) systems, and proposes a delay matching-based method to obtain a closed-form solution for RIS phase shift. The fourth prior art scheme studies the multi-hop path scheduling problem of multi-RIS-assisted single-user communication systems, and proposes a graph theory-based algorithm to optimize multi-hop path scheduling. The fifth prior art scheme studies a multi-RIS-assisted MIMO system in which the base station uses orthogonal beams to transmit information to remote single-antenna users, and proposes an efficient recursive algorithm to solve the multi-hop reflection path of the beam. The sixth prior art solution further considers the scattering interference between RISs in multi-hop path scheduling, and proposes to eliminate the interference by dividing users into different activation groups.

[0004] However, the existing technical solutions still have the following defects: 1) At present, most of the research on RIS-assisted millimeter wave communication systems focuses on using a single RIS to assist communication. However, the ability of a single RIS deployment to regulate wireless channels is limited, because in complex environments (such as urban environments), there may be no unblocked reflection link between the base station and the remote user through only a single RIS. 2) Although there are studies on multi-hop path scheduling of distributed RIS-assisted communication systems, they all use traditional graph-based optimization methods. However, these methods are not suitable for RIS-assisted millimeter wave multi-hop communication, mainly for the following two reasons. First, RIS-assisted millimeter wave multi-hop communication faces complex channel environments and multi-path propagation, resulting in a sharp increase in the dimension of the solution space, while traditional graph-based optimization methods are inefficient in dealing with high-dimensional problems and it is difficult to effectively explore the global optimal solution. Second, the transmission distance of millimeter wave signals is limited, so a large number of RIS need to be deployed to improve coverage performance, which makes the multi-hop path scheduling of millimeter wave systems involve the connection and path selection of multiple nodes. Traditional graph-based optimization methods need to traverse a large number of possible path combinations to find the optimal solution, which may lead to insufficient real-time performance in actual communication. 3) In multi-RIS-assisted millimeter wave communication systems, joint optimization of multiple strongly coupled parameters is usually involved. Although traditional iterative optimization methods can obtain suboptimal solutions, in strongly time-varying millimeter wave channels, iterative optimization methods will bring huge computational overhead, limiting their application in time-sensitive information transmission. Summary of the invention

[0005] In order to solve at least one of the technical problems existing in the prior art to a certain extent, the object of the present invention is to provide a path scheduling and beamforming design system and method for a distributed RIS-assisted millimeter wave multi-hop system.

[0006] The first technical solution adopted by the present invention is:

[0007] A distributed RIS-assisted millimeter wave multi-hop system path scheduling and beamforming design system, comprising:

[0008] A base station including N antennas;

[0009] A plurality of RISs, each RIS comprising M reflection units;

[0010] Multiple terminal devices share the same multi-hop path to receive their own signals;

[0011] Among them, multi-hop path scheduling, base station active beamforming and RIS passive beamforming are jointly optimized to maximize the total achievable rate of the system.

[0012] Furthermore, the joint optimization step includes two stages:

[0013] In the first stage, a multi-hop path scheduling algorithm based on graph neural network (GNN) is proposed. The system is modeled as a graph topology structure, where base stations, RIS and terminal devices are nodes in the graph topology. The multi-hop path scheduling problem is transformed into the shortest path problem in graph theory, and the optimal multi-hop path is determined.

[0014] In the second stage, based on the obtained optimal multi-hop path, the joint optimization of the active beamforming of the base station and the passive beamforming of the selected RIS is modeled as a Markov decision process, and a deep reinforcement learning algorithm based on deep deterministic policy gradient (DDPG) is used to obtain the optimal beamforming design.

[0015] The second technical solution adopted by the present invention is:

[0016] A method for designing path scheduling and beamforming of a distributed RIS-assisted millimeter wave multi-hop system, comprising the following steps:

[0017] Construct a distributed RIS-assisted downlink millimeter wave communication system model; the system model includes a base station, RIS, and multiple terminal devices, and multiple terminal devices share the same multi-hop path to receive their own signals;

[0018] Construct channel model and data transmission model according to the system model;

[0019] Determine the optimization problem based on the constructed model;

[0020] Based on the optimization problem, multi-hop path scheduling, base station active beamforming and RIS passive beamforming are jointly optimized to maximize the total achievable rate of the system.

[0021] Further, the base station is equipped with N antennas, and the RIS is equipped with M reflection units;

[0022] The RIS can only perform half-space reflection, that is, only the beam incident on the reflection surface can be reflected;

[0023] set up and Respectively represent RIS set and terminal device set, define node set Nodes 0 and J+1 represent the base station and typical user, respectively; d i,j Represents the distance between node i and node j; defines a binary channel status indicator l(i,j)∈{0,1} to describe the communication status between two nodes; l(i,j)=1 means that node i and node j can communicate with each other, otherwise l(i,j)=0.

[0024] Furthermore, the channel model is constructed as follows:

[0025] set up represents the channel matrix between the base station and the jth RIS, represents the channel matrix between the i-th RIS and the j-th RIS, represents the channel vector between the Jth RIS and the kth terminal device; for each RIS, use represents the phase shift matrix;

[0026] In RIS reflection, only the main line-of-sight (LoS) channel component is considered, and all non-line-of-sight (NLoS) components are considered as part of the ambient scattering; if l(0,j)=1, the channel matrix between the base station and the jth RIS is expressed as:

[0027]

[0028] Among them, PL(d o,j ) represents the path loss of the signal during propagation, d 0,j is the distance between the base station and RIS j, represents the receiving array response of the jth RIS, and are the direction angle and elevation angle from the base station to the jth RIS, respectively. represents the transmit array response of the base station, is the departure angle from the base station to the RIS; and Respectively expressed as:

[0029]

[0030] Among them, d B and d R are the spacing between antennas and the spacing between RIS reflector units, u is the steering vector, N is the number of antennas; M is the number of reflector units in RIS, M x and M z Respectively represent the number of reflection units of RIS in the horizontal and vertical directions; λ is the wavelength, is the Kronecker product;

[0031] If l(i,j)=1, it means that the i-th RIS and the j-th RIS can communicate with each other. At this time, the channel matrix between them is expressed as:

[0032]

[0033] in represents the receiving array response of the jth RIS, and are the direction angle and elevation angle from the i-th RIS to the j-th RIS, respectively. represents the transmit array response of the i-th RIS, and are the departure angles from the i-th RIS to the j-th RIS in the horizontal and vertical directions, respectively;

[0034] If l(J,J+1)=1, the channel vector between the Jth RIS and the kth terminal device is expressed as:

[0035]

[0036] in represents the send array response of the Jth RIS, and are the departure angles from the Jth RIS to the kth terminal device in the horizontal and vertical directions, respectively.

[0037] Furthermore, the data transmission model is constructed as follows:

[0038] Let Ω={a1,a2,…,a I} represents the reflection path from the base station to the terminal device, where a i and I represent the pulling sequence number of RIS and the number of selected RIS respectively;

[0039] Ω is a feasible beam path if and only if the following conditions hold, expressed as:

[0040]

[0041] l(0,a1)=l(a I ,J+1)=1 . (3)

[0042] Formula (1) ensures that each RIS can reflect the signal at most once. Formula (2) shows that there is a LoS path between the two cascaded RIS in the path Ω. Formula (3) ensures that the base station can reach the a1th IRS and the a1th IRS. I There is a LoS path between each RIS and the user set;

[0043] Given a multi-hop reflection path Ω={a1,a2,…,a I}, then the equivalent channel between the base station and the kth terminal device is expressed as:

[0044]

[0045] The signal transmitted from the base station reaches the kth terminal device after passing through the multi-hop reflection path Ω as follows:

[0046] y k =h0,k (Ω)x+n k ,

[0047] in represents the transmission signal of the base station, w k is the active beamforming vector of the kth terminal device, x k is the transmission signal of the kth terminal device, is the noise received by the kth terminal device; then, the signal-to-noise ratio (SINR) corresponding to the kth terminal device is expressed as:

[0048]

[0049] Therefore, the achievable rate for the kth terminal device is:

[0050] R k =log(1+γ k )

[0051] Finally, the total achievable rate of the system is expressed as:

[0052]

[0053] Where K is the number of terminal devices.

[0054] Furthermore, determining the optimization problem according to the constructed model includes:

[0055] By jointly optimizing the multi-hop reflection path Ω of the beam and the active beamforming w of the base station k Passive beamforming of RIS contained in the multi-hop reflection path Ω To maximize the total achievable rate of all users in the considered system, the mathematical expression of the optimization problem is (P1):

[0056]

[0057] C5:l(0,a1)=l(a I ,J+1)=1.

[0058] Where P max is the maximum transmit power of the base station; in the optimization problem (P1), constraint C1 is the constant modulus constraint of the RIS phase shift, constraint C2 is the maximum transmit power constraint of the base station, and constraints C3-C5 are the feasibility constraints of the multi-hop reflection path Ω.

[0059] Furthermore, the joint optimization of multi-hop path scheduling, base station active beamforming and RIS passive beamforming includes:

[0060] In the first stage, a multi-hop path scheduling algorithm based on graph neural network (GNN) is proposed. The system is modeled as a graph topology structure, where base stations, RIS and terminal devices are nodes in the graph topology. The multi-hop path scheduling problem is transformed into the shortest path problem in graph theory, and the optimal multi-hop path is determined.

[0061] In the second stage, based on the obtained optimal multi-hop path, the joint optimization of the active beamforming of the base station and the passive beamforming of the selected RIS is modeled as a Markov decision process, and a deep reinforcement learning algorithm based on deep deterministic policy gradient (DDPG) is used to obtain the optimal beamforming design.

[0062] Furthermore, the first stage specifically includes:

[0063] A1. Determine the expression of RIS-assisted cascade equivalent channel;

[0064] A2. For multi-hop reflection path optimization, the optimization goal is to maximize the gain of the equivalent channel; the maximization of the equivalent channel gain is transformed into the shortest distance problem in graph theory, and the channel weights between nodes are defined;

[0065] A3. Model the communication system under consideration as a directed weighted graph;

[0066] A4. Use the trained graph neural network model to determine the optimal beam reflection path; the graph neural network model includes an encoder and a decoder. The encoder converts all nodes and edges into embeddings (i.e., high-dimensional vector representations), which contain the feature information of the nodes and edges; the decoder generates a node at each time step until the signal reaches the user end and obtains the optimal beam reflection path.

[0067] Furthermore, the second stage specifically includes:

[0068] B1. Build a DRL model and define the state space, action space and reward function;

[0069] B2. Execute the DRL model:

[0070] B21, the agent obtains channel state information and multi-hop reflection path Ω by observing the environment, forming state s (n) ;

[0071] B22, change the state s (n) Input the actor network to get action a (n) , i.e., the active beamforming of the base station and the passive beamforming of the RIS involved in Ω;

[0072] B23, change the state s (n) and action a (n) As the input of the critic network, it evaluates the action and outputs the Q value;

[0073] B24. Take action in the communication environment a (n) After that, the state changes to s (n+1) , and get an immediate reward r based on the total achievable rate (n) ;

[0074] B25, will {s (n) ,a (n) ,r (n) ,s (n+1)}Record into memory buffer for network training;

[0075] B3. Use back propagation to train the DRL model.

[0076] The beneficial effects of the present invention are as follows: the present invention proposes a system framework for distributed RIS-assisted millimeter wave multi-hop communication, in which the base station transmits signals to remote users through multi-RIS cascade multi-hop paths, overcoming the problems of limited transmission distance and signal shielding in millimeter wave communication, and realizing multi-RIS collaborative transmission. Based on this framework, a joint optimization algorithm for multi-hop path scheduling optimization, base station active beamforming and RIS passive beamforming is designed to realize the joint optimization of multi-hop path scheduling and beamforming, and maximize the total achievable rate of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the embodiments of the present invention or the drawings of related technical solutions in the prior art are introduced below. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0078] Figure 1 A system block diagram of a distributed RIS-assisted millimeter wave system considered by the present invention;

[0079] Figure 2 A graph showing the relationship between the total achievable rate of the system and the number of each RIS reflection unit when the two-stage algorithm proposed by the present invention, the traditional iterative optimization scheme, the traditional graph optimization scheme, and the single RIS scheme are respectively adopted in a distributed RIS-assisted millimeter wave system in an embodiment of the present invention;

[0080] Figure 3 Graph showing the relationship between the total achievable rate of the system and the base station transmit power when the two-stage algorithm proposed in the present invention, the traditional iterative optimization scheme, the traditional graph optimization scheme, and the single RIS scheme are respectively adopted in a distributed RIS-assisted millimeter wave system in an embodiment of the present invention. DETAILED DESCRIPTION

[0081] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limitations of the present invention. For the step numbers in the following embodiments, they are only provided for the convenience of explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0082] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0083] In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0084] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention in combination with the specific content of the technical solution.

[0085] Terminology explanation:

[0086] RIS: abbreviation of reconfigurable intelligent surface, intelligent super surface.

[0087] DRL: abbreviation of Deep Reinforcement Learning, deep reinforcement learning.

[0088] GNN: abbreviation of Graph Neural Network, graph neural network.

[0089] With the rapid development of wireless communication technology, millimeter wave communication is regarded as a key technology to meet the future high-speed and large-capacity requirements due to its rich bandwidth resources. However, millimeter wave communication faces serious challenges such as path loss, signal shielding and limited propagation distance in practical applications, which greatly limit its application scope. The RIS technology proposed in recent years provides an effective solution to the problem of limited millimeter wave propagation distance. RIS is a metasurface that can dynamically adjust the propagation characteristics of electromagnetic waves. It is composed of a large number of passive, low-cost reflection units. Each reflection unit adjusts the amplitude and / or phase of the signal through digital control, intelligently adjusts the incident signal, and thus synergistically changes the physical transmission environment. Compared with traditional methods, RIS has the advantages of low cost, low energy consumption and flexible deployment. Therefore, in complex communication environments, such as urban environments, RIS can be deployed on the exterior walls of multiple buildings to achieve long-distance millimeter wave communication through multi-hop. However, multi-hop path scheduling is a complex combinatorial optimization problem because it involves the connection and path selection of multiple nodes, resulting in a rapid increase in the dimension of the solution space. Traditional optimization methods need to traverse a large number of possible path combinations to find the optimal solution, so the computational overhead is large. In recent years, the application of machine learning (ML) in wireless communications has made significant progress. Its powerful learning ability and adaptability enable it to show excellent performance in dealing with complex optimization problems. Therefore, this paper will explore how to use ML algorithms to efficiently handle the path scheduling and beamforming design of distributed RIS-assisted millimeter-wave multi-hop systems.

[0090] In order to overcome the problems of limited transmission distance and signal shielding in millimeter wave communication and realize multi-RIS cooperative transmission, the present invention proposes a system framework of millimeter wave multi-hop communication assisted by distributed RIS, in which the base station transmits signals to remote users through multi-RIS cascade multi-hop paths. Different from the work based on pre-determined transmission paths, the present invention explores the characteristics of multi-RIS cascade equivalent channels and determines an optimal multi-hop path that maximizes the equivalent channel gain to realize long-distance millimeter wave communication. In order to improve the total achievable rate of the system, the present invention proposes a joint optimization method for beam reflection path scheduling, base station active beamforming and RIS passive beamforming. The joint optimization method is a machine learning-based method divided into two stages. In stage one, a multi-hop path scheduling algorithm based on graph neural network (GNN) is proposed. Specifically, the system under consideration is first modeled as a graph topology structure, in which the base station, RIS and user are nodes in the graph topology. Secondly, the expression of the equivalent channel assisted by multi-RIS is derived, and the composition characteristics of the expression are analyzed. Then, the weights between adjacent nodes are defined, which are related to the path loss and the number of RIS reflection units. Finally, the multi-hop path scheduling problem is transformed into the shortest path problem in graph theory, and an efficient GNN-based method is proposed to determine the optimal multi-hop path. In stage two, based on the obtained optimal multi-hop path, the present invention models the joint optimization of the active beamforming of the base station and the passive beamforming of the selected RIS as a Markov decision process, in which the state space covers all channel state information and the optimal multi-hop path, and the action space consists of active and passive beamforming vectors. Since the optimization variable is a continuous variable, the present invention adopts a deep reinforcement learning algorithm based on deep deterministic policy gradient (DDPG) to obtain the optimal beamforming design. The simulation results show that compared with other optimization schemes, the proposed optimization allocation scheme can effectively improve the total achievable rate of the system.

[0091] The technical solution of the present invention is explained in detail below in conjunction with the accompanying drawings and specific embodiments.

[0092] (1) System model

[0093] See also Figure 1 , the embodiment of the present invention considers a distributed RIS-assisted downlink millimeter wave communication system, in which the direct link between the base station and the remote single-antenna user is blocked by obstacles. Therefore, the base station can only communicate with the remote user through a reflection path composed of multiple RIS. Assume that the base station is equipped with N antennas and each RIS is equipped with M reflection units. Moreover, RIS can only perform half-space reflection, that is, only the beam incident on the reflection surface can be reflected. With M x and M z Respectively represent the number of reflection units of RIS in the horizontal and vertical directions, so M = M x×M z Since the present invention realizes the long-distance transmission of millimeter wave signals, it is assumed that each user can share the same multi-hop path to receive its own signal, and focuses on designing the optimal multi-hop path between the base station and the typical user. and Respectively represent the RIS set and the user set. In order to facilitate the subsequent graph representation, the present invention defines the node set Nodes 0 and J+1 represent the base station and typical user, respectively. i,j Indicates the distance between node i and node j. The present invention defines a binary channel state indicator l(i,j)∈{0,1} to describe the communication state between two nodes. l(i,j)=1 indicates that node i and node j can communicate with each other, otherwise l(i,j)=0.

[0094] a) Channel Model

[0095] set up represents the channel matrix between the base station and RIS j, represents the channel matrix between RIS i and RIS j, represents the channel vector between RIS J and user k. For each RIS j, represents the phase shift matrix, where the present invention sets the reflection amplitude to 1 to maximize the reflection power, φ j,m ∈[0,2π) represents the phase shift of the reflection unit.

[0096] The embodiment of the present invention only considers the main line-of-sight (LoS) channel component in RIS reflection, and regards all non-line-of-sight (NLoS) components as part of the environmental scattering, because NLoS has little impact on user performance. Considering that the millimeter wave channel has limited scattering characteristics, the present invention uses the three-dimensional Saleh-Valenzuela geometric channel model to characterize the relevant LoS channel. Specifically, if l(0,j)=1, the channel matrix from the base station to RIS j is expressed as

[0097]

[0098] Among them, PL(d o,j ) represents the path loss of the signal during propagation, d 0,j is the distance between the base station and RIS j, represents the receive array response of RISj, and are the direction angle and elevation angle from the base station to RISj, represents the transmit array response of the base station, is the departure angle from the base station to the RIS. and Respectively expressed as:

[0099]

[0100] Among them, d B and d R are the spacing between antennas and the spacing between RIS reflector units, u(ζ,U)=[1,e -jπζ ,…,e -j(U-1)πζ ] is the steering vector, where ζ is the phase difference between two adjacent elements and U is the number of elements. If l(i,j)=1, it means that RIS i and RIS j can communicate with each other, and the channel matrix between them is expressed as:

[0101]

[0102] in represents the receive array response of RIS j, and are the azimuth and elevation angles from RIS i to RIS j, represents the transmit array response of RIS i, and are the departure angles from RIS i to RIS j in the horizontal and vertical directions, respectively. Similarly, if l(J,J+1)=1, the channel vector between RIS J and user k is expressed as:

[0103]

[0104] in represents the transmit array response of RIS J, and are the departure angles from RIS J to user k in the horizontal and vertical directions, respectively.

[0105] b) Data transmission model

[0106] Let Ω={a1,a2,…,a I} represents the reflected path from the base station to the user, where a i and I represent the pulling sequence number of RIS and the number of selected RIS respectively. Then, Ω is a feasible beam path if and only if the following conditions are met, expressed as:

[0107]

[0108] l(0,a1)=l(a I ,J+1)=1. (8)

[0109] In the above formula, formula (6) ensures that each RIS can only reflect the signal once at most, formula (7) shows that there is a LoS path between the two cascaded RIS in the path Ω, and formula (8) ensures that the base station to IRS a1 and RIS a I There is a LoS path between the user set and the user set. In fact, in formula (8) there is a I =J, because the present invention assumes that the signal first reaches the last RIS and then is reflected to the user set by the RIS.

[0110] According to equations (6)-(8), given a multi-hop reflection path Ω = {a1, a2, …, a I}, then the equivalent channel from the base station to user k can be expressed as:

[0111]

[0112] Therefore, the signal transmitted from the base station to the user k after passing through the multi-hop reflection path Ω can be expressed as:

[0113] y k =h 0,k (Ω)x+n k , (10)

[0114] in represents the transmission signal of the base station, w k is the active beamforming vector of the kth user, x k is the transmitted signal of user k, is the noise received by the kth user. Then, the signal-to-noise ratio (SINR) corresponding to user k can be expressed as:

[0115]

[0116] Therefore, the achievable rate for user k is R k =log(1+γ k ). Finally, the total achievable rate of the system is expressed as:

[0117]

[0118] (2) Optimization Problem

[0119] To complete the above definition, the present invention aims to optimize the multi-hop reflection path Ω of the beam, the active beamforming of the base station w k Passive beamforming of RIS contained in the multi-hop reflection path Ω To maximize the total achievable rate of all users in the system under consideration. The mathematical expression of the optimization problem is (P1):

[0120]

[0121] Where P max is the maximum transmit power of the base station. In the optimization problem (P1), constraint C1 is the constant modulus constraint of the RIS phase shift. Constraint C2 is the maximum transmit power constraint of the base station. Constraints C3-C5 are the feasibility constraints of the multi-hop reflection path Ω.

[0122] (3) Two-stage optimization design method

[0123] Solving the optimization problem (P1) faces three major challenges. First, due to the existence of multiple multi-hop paths, it is necessary to select an optimal multi-hop path that not only minimizes the path loss suffered by the signal during transmission, but also ensures that the total achievable rate of the system is maximized. This is a typical combinatorial optimization problem and is difficult to solve using traditional convex optimization methods. Second, the traditional graph-based path optimization method performs well when the number of users and RIS is small, but the computational complexity increases sharply in a large-scale communication environment and is not suitable for real-time communication needs. In response to these two challenges, the present invention uses a method based on a graph neural network (GNN) for multi-hop path scheduling. GNN can effectively capture the complex dependencies between nodes in a multi-hop path and improve the accuracy of path selection through efficient feature representation learning. In addition, once the GNN model is trained, it can be quickly inferred on the graph to select the optimal multi-hop path with a nearly constant time complexity, which has high real-time performance. Third, the strong coupling of the optimization variables in problem (P1) and the fractional expression of SINR turn problem (P1) into a thorny non-convex optimization problem. When solving it using traditional iterative optimization algorithms, the computational overhead is large and it is easy to fall into local optimality. In response to this challenge, the present invention adopts a method based on deep reinforcement learning (DRL) for multi-parameter optimization. DRL models non-convex optimization problems as Markov decision processes (MDPs), and combined with policy iteration and exploration mechanisms, it can effectively solve the global optimal solution of parameters. Therefore, the present invention proposes a two-stage optimization design method combining GNN and DRL. Specifically, in stage one, a multi-hop path scheduling algorithm based on GNN is proposed. Specifically, the system under consideration is first modeled as a graph topology structure, in which the base station, RIS and users are all nodes in the graph topology; then, the expression of the RIS-assisted equivalent channel is derived, and the weights between adjacent nodes are defined, which are related to the path loss and the number of RIS reflection units; finally, a multi-hop path scheduling algorithm based on GNN is proposed to determine the optimal beam reflection path. In stage two, according to the optimal multi-hop path obtained, the DRL algorithm based on deep deterministic policy gradient (DDPG) is used to jointly optimize the active beamforming of the base station and the passive beamforming of the selected RIS. The specific process is:

[0124] a) Phase 1: GNN-based multi-hop path scheduling optimization

[0125] In this stage, the goal is to find an optimal beam reflection path that not only minimizes the path loss suffered by the signal during transmission, but also ensures that the total achievable rate of the system is maximized. To this end, this stage first derives the expression of the RIS-assisted cascade equivalent channel; then, the weight between any two nodes is constructed, which is related to the path loss and the number of RIS reflection units; finally, a GNN-based multi-hop path optimization method is proposed to determine the optimal multi-hop path. The specific steps are as follows:

[0126] Step 1: Derive the expression of the RIS-assisted cascade equivalent channel. As mentioned above, after determining the multi-hop reflection path Ω = {a1, a2, …, a I}, the equivalent channel from the base station to user k is shown in equation (9). Substituting equations (1), (4) and (5) into equation (9) yields a new expression for the cascade equivalent channel, which is:

[0127]

[0128] in

[0129]

[0130] Step 2: Define the weights between nodes. For multi-hop reflection path optimization, this stage takes maximizing the gain of the equivalent channel as the optimization goal, hoping to find an optimal beam reflection path to maximize the user's equivalent channel gain, that is, |h 0,k (Ω)| 2 Maximum. Observing formula (14), we can see that |h0,k(Ω)| 2 Maximum, that is, G i The modulus of is the largest. Below, 2≤i≤a I -1 is used as an example to illustrate the solution of G i Specifically, |G| i 2 It can be expressed as:

[0131]

[0132] For the sake of explanation, assume that:

[0133]

[0134] where vec(·) represents vectorization, ζ t and They represent the transmission phase shift and the reception phase shift of the channel respectively. Substituting equations (17)-(19) into equation (16), we can obtain:

[0135]

[0136] in Represents the phase difference term. When the phase shift satisfies Δδ1=Δδ2=…=Δδ M =0 time |G i | 2 Reach the maximum value Similarly, when i=1 and i=a I The same conclusion can be obtained.

[0137] From the above analysis, we can see that node a i with a i+1 The individual channel gain between |G i | 2 To determine, and |G i | 2 The optimal value of is proportional to the number of reflection units of RIS and the path loss between nodes. In order to transform the maximization of equivalent channel gain into the shortest distance problem in graph theory, the embodiment of the present invention defines node a i with a i+1 The channel weight between is expressed as:

[0138]

[0139] in

[0140] Step 3: Model the communication system under consideration as a directed weighted graph. The present invention defines a directed weighted graph G = (V, E, W), where the vertex set V consists of all nodes in the system, i.e. The edge set is represented as:

[0141]

[0142] The weight function is W i,j =Γ i,j ,if l(i,j)=1. In addition, use n i ,i∈V represents the feature of node i, that is, the three-dimensional coordinates of node i.

[0143] Step 4: Multi-hop path scheduling optimization based on GNN. The present invention proposes a GNN-based method to determine the optimal beam reflection path Ω = {a1, a2, ..., a I}∪{0,J+1}, the method consists of an encoder and a decoder. The encoder converts all nodes and edges into embeddings (i.e., high-dimensional vector representations), which contain the feature information of nodes and edges; while the decoder generates a node at each time step. After generating a node, the decoder will cover the node to prevent the model from accessing this node again in the next step. For convenience, define a0=0, aI+1 = J + 1. Next, the specific structure and algorithm execution process of the encoder and decoder will be introduced in detail.

[0144] Encoder: The encoder of the present invention is designed based on the graph attention network (GAT), which is a neural network architecture that transmits node information through the attention mechanism. However, GAT only updates the features of each node by assigning new weights to the neighborhood, and does not consider the information of the edges in the graph structure, which may cause the loss of some neighborhood information in the graph structure, making the network unable to represent the graph structure well. Therefore, the present invention proposes a residual edge-graph attention network (RE-GAT), which not only considers the information of nodes and edges in the graph structure at the same time, but also adds a residual connection in each RE-GAT sublayer to avoid gradient disappearance. The execution process of the encoder is as follows:

[0145] First, the node feature n i and edge feature Γ i,j ,i,j∈V input fully connected layer and are embedded into d n and d Γ In the dimension feature, it is expressed as:

[0146]

[0147] in BN(·) and ω(·) represent the batch normalization operation and the neural network fitting function, respectively. and is the training parameter of the neural network, and the superscript 0 indicates that the feature is before entering RE-GAT.

[0148] Then, the embedded features are input into RE-GAT to fuse node information and edge information and update node information. The layer index l∈{1,2,…,L} is used to represent the node embedding of the lth layer obtained in RE-GAT. Specifically, the input of RE-GAT is the node features and edge features Output is a new node feature The edge feature Here, we take the first layer as an example to illustrate the calculation process of RE-GAT:

[0149] A1. Calculate the attention coefficient α between node i and its neighboring node j ij , expressed as:

[0150]

[0151] in is the set of neighborhood nodes of node i, g and F are the learnable parameter vector and parameter matrix respectively, σ(·) is the LeakyReLU activation function, and ·||· represents the concatenation operation.

[0152] A2. Update the information of each node, expressed as:

[0153]

[0154] Where F′ is a learnable parameter matrix.

[0155] A3. Use residual connection to add the input node features and the updated node features to obtain the final new node features, which are expressed as:

[0156]

[0157] The encoder consists of L RE-GAT layers, and each layer performs the same process as described above. These layers are connected in a cascade manner, where the output of the previous layer serves as the input of the next layer. The output node embedding of the Lth layer RE-GAT is represented as

[0158] Finally, using Compute average graph embedding It is expressed as:

[0159]

[0160] At this point, the fusion of nodes and edges in the graph structure and the update of node information are completed.

[0161] Decoder: The decoder of the embodiment of the present invention adopts an attention mechanism and uses a softmax function to select an output node from the input sequence in each decoding step. The decoder consists of a multi-head attention layer and a single-head attention layer, where the multi-head attention layer is used to calculate context information, while the single-head attention layer generates the probability distribution of the selected node. During the decoding process, the decoder selects the output node from the previous node a at each time step t. t′ , t<t′ and all nodes are embedded to generate a node a t The detailed decoding process of the decoder is as follows:

[0162] First, at time step t, the output of the encoder is used The last selected node a t-1 Calculate the context vector with the first selected node a1 It is expressed as:

[0163]

[0164] Among them, F nis a learnable parameter matrix, and the superscript 0 indicates the context vector before entering the multi-head attention layer.

[0165] Secondly, Input to the multi-head attention layer to generate a new context information The calculation process of the multi-head attention layer is as follows:

[0166] B1. Define three ds v dimensional vectors, respectively, the query vector Key Vector Sum value vector It is expressed as:

[0167]

[0168] in and is the learnable parameter matrix, d v =d n / P, P means there are P heads.

[0169] B2. Using query vector q and key vector k i Calculate the attention coefficient of the multi-head attention layer at time step t, expressed as:

[0170]

[0171] In the above formula, the coefficient of the node that has been selected is set to -∞ to prevent it from being visited again.

[0172] B3. Use the softmax function to normalize the attention coefficient, that is,

[0173] B4, P heads perform independent attention calculations and connect them in sequence, and then use a fully connected layer to obtain the final context vector It is expressed as:

[0174]

[0175] where F c is the learnable parameter matrix, and are the attention coefficient and value vector of the p-th head respectively.

[0176] Then, the obtained context vector Enter the single-head attention layer and calculate the selection probability of each node. The execution process of the single-head attention layer is:

[0177] C1. Calculate the attention coefficient of the single-head attention layer at time step t And use the tanh function to cut it within the range of [-C, C], expressed as:

[0178]

[0179] C2, use the softmax function to calculate the selection probability of each node, expressed as

[0180] Finally, using the random sampling strategy, according to p i,t Predict the next node to visit.

[0181] Step 5: Train the GNN model. The proposed GNN model is trained using the REINFORCE framework. The REINFORCE framework is an Actor-Critic reinforcement learning framework. The present invention first defines the length of the multi-hop reflection path Ω, expressed as:

[0182]

[0183] Secondly, a random strategy p(Ω) is defined to represent the selection probability of the multi-hop reflection path Ω, which is expressed as:

[0184]

[0185] Where θ is the parameter of GNN, For node a i The selection probability is calculated by the decoder. Finally, the loss function of the REINFORCE framework is defined as The present invention uses the policy gradient theorem to update the parameter θ, which is expressed as:

[0186]

[0187] Where b represents the baseline, which is estimated by the policy network. In the REINFORCE framework, the state space is defined as the set of nodes that have been visited; the action space is defined as adding a node to the current path set; the reward function is the change in the length of the new path after each addition of a node; the stopping condition is that node J+1 is visited, that is, the signal reaches the user end.

[0188] After completing the above definitions, the proposed GNN model can be trained using the REINFORCE framework.

[0189] b) Phase II: DRL-based active and passive beamforming design

[0190] In this stage, the goal is to optimize the active beamforming of the base station and the passive beamforming of the RIS contained in the beam reflection path Ω according to the optimal beam reflection path obtained in stage 1. Due to the mutual coupling of the optimization variables in problem (P1) and the fractional expression of SINR, the problem becomes a difficult non-convex problem and is difficult to solve using traditional iterative optimization. Therefore, the present invention proposes a beamforming optimization algorithm based on DRL.

[0191] The DRL algorithm can model the joint optimization problem of active and passive beamforming as a Markov decision process. Specifically, the changes in the active beamforming of the base station, the passive beamforming of the selected RIS, and the total achievable rate of the system caused by the decision of the agent can be abstracted as a Markov transition process, which consists of the state action Reward (n) and the next state The DRL algorithm is designed to provide each state s (n) Find an optimal action a (n) , and introduce the Q-value function to evaluate the long-term value of each action. The Q-value function satisfies the Bellman equation and is expressed as:

[0192] Q(s (n) ,a (n) )=r (n) (s (n) ,a (n) )+γmaxQ(s (n) ,a (n) ), (37)

[0193] Where γ is the discount rate between [0,1], the optimal action is defined as:

[0194]

[0195] Since the variables involved in the optimization problem (P1) are continuous variables, the deep deterministic policy gradient (DDPG) algorithm is used to solve the optimization problem. The DDPG algorithm consists of two pairs of deep neural networks, namely the actor network and the critic network, and their corresponding target networks. The actor network and the critic network are responsible for selecting actions and evaluating the value of actions, respectively, while their corresponding target networks are used to stabilize the training process. The execution process of the active and passive beamforming algorithms based on DDPG is as follows:

[0196] Step 1: Define state space, action space and reward function. In order to solve the optimization problem (P1), the present invention defines the following state space, action space and reward function:

[0197] 1) State space: The channel state information and the multi-hop reflection path Ω are defined as the state space. Since the data type input of the neural network needs to be unified, the state space is expressed as:

[0198]

[0199] Where real{·} and imag{·} represent the real and imaginary parts of a complex number.

[0200] 2) Action space: The action space consists of two optimization variables, namely the active beamforming of the base station and the passive beamforming of the RIS involved in Ω, which can be expressed as:

[0201]

[0202] 3) Reward function: Since the optimization goal of the present invention is to maximize the total achievable rate of the system, the reward function of DRL at the nth moment is set to the total achievable rate of the system, expressed as r (n) =R sum .

[0203] Step 2: Execute the DRL model. The specific execution process is as follows:

[0204] D1, the agent obtains channel state information and multi-hop reflection path Ω by observing the environment, forming state s (n) .

[0205] D2, the state s (n) Input the actor network to get action a (n) , namely the active beamforming of the base station and the passive beamforming of the RIS involved in Ω.

[0206] D3, the state s (n) and action a (n) As input to the critic network, action evaluation is performed and Q-value is output.

[0207] D4. Take action in the communication environment (n) After that, the state changes to s (n+1) , and get an immediate reward r based on the total achievable rate (n) .

[0208] D5. (n) ,a (n) ,r (n) ,s (n+1)}Record into memory buffer for network training.

[0209] Step 3: Train the model. Use back propagation to train the DRL model.

[0210] (4) Simulation Experiment Results

[0211] In this embodiment, simulation experiments verify the effectiveness of the proposed two-stage resource optimization configuration method in improving the total achievable rate of the system, and compare it with the traditional iterative optimization scheme, the traditional graph optimization scheme and the single RIS deployment scheme. In the traditional iterative optimization scheme, the GNN model proposed in the present invention is used in stage one to determine the optimal reflection path of the beam, and in stage two, the iterative optimization algorithm proposed in document [7] is used to perform active beamforming and passive beamforming optimization. In the traditional graph optimization scheme, the traditional Dijkstra algorithm is used in stage one to determine the optimal reflection path of the beam, and in stage two, the DRL model proposed in the present invention is used to perform active beamforming and passive beamforming optimization. In the single RIS deployment scheme, a single RIS is used to provide services to users, and the DRL model proposed in the present invention is used for active and passive beamforming design.

[0212] The relevant parameters of the simulation experiment are set as follows: The embodiment of the present invention considers a three-dimensional scene, in which the coordinates of the base station are (2m, 0m, 10m), the users are randomly distributed in a circular area with a center of (12m, 105m, 2m) and a radius of 10m, and multiple RIS are deployed between the base station and the user. In addition, it is assumed that the coordinates of the last RIS are (11m, 100m, 10m), and the signal is reflected to the user after reaching the last RIS. For the millimeter wave channels from the base station to the RIS, between the RIS, and from the RIS to the user, the path loss model is Where β0 is the signal power attenuation at the reference distance d0 = 1m, α P =2.1 is the path loss index. The carrier frequency is 28GHz and the bandwidth is 251MHz. Therefore, the corresponding wavelength is λ=0.0107m and the signal power attenuation is β0=(λ / 4π) 2 =-61.4dB, noise power σ 2 =-174+10log 10 B = -90dBm. The antenna spacing and the reflection unit spacing of each RIS are set to d B =d R =λ / 2.

[0213] Figure 2The total achievable rate of the system when the resource optimization allocation scheme proposed in the present invention is adopted in the distributed RIS-assisted millimeter wave system under consideration is demonstrated, and is compared with the scheme based on traditional iterative optimization, traditional graph optimization and deployment of a single RIS. It can be seen from the simulation diagram that with the increase in the number of RIS reflection units, the total achievable rate of all schemes is improved, and the algorithm proposed in the present invention is always better than the other three algorithms. It can also be found that the performance of the millimeter wave system assisted by distributed multi-RIS is significantly better than that of the millimeter wave system assisted by a single RIS. It can be seen that the proposed two-stage resource optimization configuration scheme is effective in improving the total achievable rate of the system, and that distributed multi-RIS plays an important role in improving the performance of millimeter wave transmission.

[0214] Figure 3 The total achievable rate of the system when the resource optimization allocation scheme proposed in the present invention is adopted in the considered distributed RIS-assisted millimeter wave system is demonstrated, and is compared with the scheme based on traditional iterative optimization, traditional graph optimization and deployment of a single RIS. It can be seen from the simulation diagram that the total achievable rate of the above four schemes continues to increase with the increase of the maximum transmission power of the base station, among which the algorithm proposed in the present invention is significantly better than the other three schemes. This shows that under high power conditions, the two-stage algorithm proposed in the present invention can more efficiently utilize the base station transmission power, thereby improving the total achievable rate of the system. It can be seen that the proposed two-stage resource optimization configuration scheme is crucial to improving the total achievable rate of the system.

[0215] (5) Advantages and beneficial effects

[0216] At present, most of the research on RIS-assisted millimeter wave communication systems focuses on the use of a single RIS-assisted communication, but fewer studies explore the feasibility and potential advantages of distributed RIS-assisted millimeter wave communication systems, and few studies involve the scheduling of beam reflection paths and the performance optimization of the entire system. At the same time, the traditional graph-based path scheduling optimization algorithm is not suitable for distributed RIS-assisted millimeter wave multi-hop communication due to its high complexity and poor real-time performance. In addition, in distributed RIS-assisted millimeter wave communication systems, the joint optimization of multiple strongly coupled parameters is usually involved. Although traditional iterative optimization methods can obtain suboptimal solutions, in strongly time-varying millimeter wave channels, iterative optimization methods will bring huge computational overhead, limiting their application in time-sensitive information transmission. Taking into account the research status of the prior art, the present invention proposes a system framework for distributed RIS-assisted millimeter wave multi-hop communication, as well as corresponding path scheduling and beamforming design methods. The present invention mainly has the following advantages:

[0217] 1) This invention proposes a distributed RIS-assisted millimeter wave multi-hop communication system framework, in which the base station transmits signals to remote users through multi-RIS cascade multi-hop paths. Different from the work based on pre-determined transmission paths, this invention explores the characteristics of multi-RIS cascade equivalent channels and determines an optimal multi-hop path that maximizes the equivalent channel gain to achieve long-distance millimeter wave communication.

[0218] 2) In order to overcome the high complexity problem of traditional graph-based optimization algorithms in path scheduling optimization, the present invention proposes a multi-hop path scheduling method based on graph neural network (GNN) to determine the optimal beam reflection path with extremely low complexity and high real-time performance.

[0219] 3) On the basis of multi-hop path scheduling, in order to improve the total achievable rate of the system, the present invention further proposes a beamforming design method based on deep reinforcement learning (DRL), which can effectively improve the total achievable rate of the system.

[0220] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0221] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable ordinary technicians in the field to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made based on the essence of the content of the present invention should be included in the protection scope of the present invention.

Claims

1. A distributed RIS-assisted millimeter wave multi-hop system path scheduling and beamforming design system, characterized in that: include: A base station including N antennas; A plurality of RISs, each RIS comprising M reflection units; Multiple terminal devices share the same multi-hop path to receive their own signals; Among them, multi-hop path scheduling, base station active beamforming and RIS passive beamforming are jointly optimized to maximize the total achievable rate of the system.

2. According to claim 1, a distributed RIS-assisted millimeter wave multi-hop system path scheduling and beamforming design system is characterized in that: The joint optimization process consists of two stages: In the first stage, a multi-hop path scheduling algorithm based on graph neural network is proposed. The system is modeled as a graph topology structure, where base stations, RIS and terminal devices are nodes in the graph topology. The multi-hop path scheduling problem is transformed into the shortest path problem in graph theory, and the optimal multi-hop path is determined. In the second stage, based on the obtained optimal multi-hop path, the joint optimization of the active beamforming of the base station and the passive beamforming of the selected RIS is modeled as a Markov decision process, and a deep reinforcement learning algorithm based on deep deterministic policy gradient is used to obtain the optimal beamforming design.

3. A method for path scheduling and beamforming design of a distributed RIS-assisted millimeter wave multi-hop system, characterized in that: The following steps are involved: Construct a distributed RIS-assisted downlink millimeter wave communication system model; The system model includes a base station, a RIS, and multiple terminal devices, where multiple terminal devices share the same multi-hop path to receive their own signals; Construct channel model and data transmission model according to the system model; Determine the optimization problem based on the constructed model; Based on the optimization problem, multi-hop path scheduling, base station active beamforming and RIS passive beamforming are jointly optimized to maximize the total achievable rate of the system.

4. The method for path scheduling and beamforming design of a distributed RIS-assisted millimeter wave multi-hop system according to claim 3, characterized in that: The base station is equipped with N antennas, and the RIS is equipped with M reflection units; The RIS can only perform half-space reflection, that is, only the beam incident on the reflection surface can be reflected; set up and Respectively represent RIS set and terminal device set, define node set Nodes 0 and J+1 represent the base station and typical user, respectively; d i,j Represents the distance between node i and node j; defines a binary channel status indicator l(i,j)∈{0,1} to describe the communication status between two nodes; l(i,j)=1 means that node i and node j can communicate with each other, otherwise l(i,j)=0.

5. The method for path scheduling and beamforming design of a distributed RIS-assisted millimeter wave multi-hop system according to claim 3, characterized in that: The channel model is constructed as follows: set up represents the channel matrix between the base station and the jth RIS, represents the channel matrix between the i-th RIS and the j-th RIS, represents the channel vector between the Jth RIS and the kth terminal device; for each RIS, use represents the phase shift matrix; In RIS reflection, only the main line-of-sight channel components are considered, and all non-line-of-sight components are considered as part of the environmental scattering; if l(0,j)=1, the channel matrix between the base station and the jth RIS is expressed as: Among them, PL(d o,j ) represents the path loss of the signal during propagation, d0,j is the distance between the base station and RIS j, represents the receiving array response of the jth RIS, and are the azimuth angle and elevation angle from the base station to the jth RIS, h 0,j,t =α B (θ 0,j ,N) represents the transmitting array response of the base station, θ0,j is the departure angle from the base station to the RIS; α B (θ 0,j ,N) and Respectively expressed as: Among them, d B and d R are the spacing between antennas and the spacing between RIS reflector units, u is the steering vector, N is the number of antennas; M is the number of reflector units in RIS, M x and M z Respectively represent the number of reflection units of RIS in the horizontal and vertical directions; λ is the wavelength, is the Kronecker product; If l(i,j)=1, it means that the i-th RIS and the j-th RIS can communicate with each other. At this time, the channel matrix between them is expressed as: in represents the receiving array response of the jth RIS, and are the direction angle and elevation angle from the i-th RIS to the j-th RIS, respectively. represents the transmit array response of the i-th RIS, and are the departure angles from the i-th RIS to the j-th RIS in the horizontal and vertical directions, respectively; If l(J,J+1)=1, the channel vector between the Jth RIS and the kth terminal device is expressed as: in represents the send array response of the Jth RIS, and are the departure angles from the Jth RIS to the kth terminal device in the horizontal and vertical directions, respectively.

6. The method for path scheduling and beamforming design of a distributed RIS-assisted millimeter wave multi-hop system according to claim 3, characterized in that: The data transmission model is constructed as follows: Let Ω={a1,a2,…,a I } represents the reflection path from the base station to the terminal device, where a i and I represent the pulling sequence number of RIS and the number of selected RIS respectively; Ω is a feasible beam path if and only if the following conditions hold, expressed as: l(0,a1)=l(a I ,J+1)=1 (3) Formula (1) ensures that each RIS can reflect the signal at most once. Formula (2) shows that there is a LoS path between the two cascaded RIS in the path Ω. Formula (3) ensures that the base station can reach the a1th IRS and the a1th IRS. I There is a LoS path between each RIS and the user set; Given a multi-hop reflection path Ω={a1,a2,…,a I }, then the equivalent channel between the base station and the kth terminal device is expressed as: The signal transmitted from the base station reaches the kth terminal device after passing through the multi-hop reflection path Ω as follows: y k =h 0,k (Ω)x+n k , in represents the transmission signal of the base station, w k is the active beamforming vector of the kth terminal device, x k is the transmission signal of the kth terminal device, is the noise received by the kth terminal device; then, the signal-to-noise ratio corresponding to the kth terminal device is expressed as: Therefore, the achievable rate for the kth terminal device is: R k =log(1+γ k ) Finally, the total achievable rate of the system is expressed as: Wherein, K is the number of terminal devices.

7. The method for path scheduling and beamforming design of a distributed RIS-assisted millimeter wave multi-hop system according to claim 3, characterized in that: Determining the optimization problem according to the constructed model includes: By jointly optimizing the multi-hop reflection path Ω of the beam and the active beamforming w of the base station k Passive beamforming of RIS contained in the multi-hop reflection path Ω To maximize the total achievable rate of all users in the considered system, the mathematical expression of the optimization problem is (P1): C5:l(0,a1)=l(a I ,J+1)=1. Where P max is the maximum transmit power of the base station; in the optimization problem (P1), constraint C1 is the constant modulus constraint of the RIS phase shift, constraint C2 is the maximum transmit power constraint of the base station, and constraints C3-C5 are the feasibility constraints of the multi-hop reflection path Ω.

8. The method for path scheduling and beamforming design of a distributed RIS-assisted millimeter wave multi-hop system according to claim 3, characterized in that: The joint optimization of multi-hop path scheduling, base station active beamforming and RIS passive beamforming includes: In the first stage, a multi-hop path scheduling algorithm based on graph neural network is proposed. The system is modeled as a graph topology structure, where base stations, RIS and terminal devices are nodes in the graph topology. The multi-hop path scheduling problem is transformed into the shortest path problem in graph theory, and the optimal multi-hop path is determined. In the second stage, based on the obtained optimal multi-hop path, the joint optimization of the active beamforming of the base station and the passive beamforming of the selected RIS is modeled as a Markov decision process, and a deep reinforcement learning algorithm based on deep deterministic policy gradient is used to obtain the optimal beamforming design.

9. The method for path scheduling and beamforming design of a distributed RIS-assisted millimeter wave multi-hop system according to claim 8, characterized in that: The first stage specifically includes: A1. Determine the expression of RIS-assisted cascade equivalent channel; A2. For multi-hop reflection path optimization, the optimization goal is to maximize the gain of the equivalent channel; the maximization of the equivalent channel gain is transformed into the shortest distance problem in graph theory, and the channel weights between nodes are defined; A3. Model the communication system under consideration as a directed weighted graph; A4. Use the trained graph neural network model to determine the optimal beam reflection path; the graph neural network model includes an encoder and a decoder. The encoder converts all nodes and edges into embeddings, which contain the feature information of the nodes and edges; the decoder generates a node at each time step until the signal reaches the user end and obtains the optimal beam reflection path.

10. The method for path scheduling and beamforming design of a distributed RIS-assisted millimeter wave multi-hop system according to claim 8, characterized in that: The second stage specifically includes: B1. Build a DRL model and define the state space, action space and reward function; B2. Execute the DRL model: B21, the agent obtains channel state information and multi-hop reflection path Ω by observing the environment, forming a state s (n) ; B22, change the state s (n) Input the actor network to get action a (n) , i.e., the active beamforming of the base station and the passive beamforming of the RIS involved in Ω; B23, change the state s (n) and action a (n) As the input of the critic network, it evaluates the action and outputs the Q value; B24. Take action in the communication environment a (n) After that, the state changes to s (n+1) , and get an immediate reward r based on the total achievable rate (n) ; B25, will {s (n) ,a (n) ,r (n) ,s (n+1) }Record into memory buffer for network training; B3. Use back propagation to train the DRL model.

Images