RIS division and resource optimization method for RIS-assisted millimeter wave NOMA system

Abstract

The invention discloses an RIS division and resource optimization method for an RIS-assisted millimeter wave NOMA system. The method comprises the following steps: constructing a downlink millimeter wave NOMA multi-user system model; wherein the system model comprises a base station, an RIS and K terminal devices, and the K terminal devices are divided into I clusters; constructing an RIS model, a channel model and a transmission model according to the system model; determining an optimization problem according to the constructed model; and on the basis of the optimization problem, performing joint optimization on terminal equipment grouping, RIS division, active beam forming, passive beam forming and power distribution so as to maximize the total achievable rate of the terminal equipment in downlink transmission. The method overcomes the problems of limited transmission distance and signal shielding of millimeter wave communication, meets the large-scale connection requirement, improves the system performance, and can be widely applied to the technical field of wireless communication.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technologies, and in particular to a RIS partitioning and resource optimization method for a RIS-assisted millimeter wave NOMA system. Background Art

[0002] Millimeter wave communication has the advantages of high bandwidth and low latency, and can support large-scale data transmission and high-density device connection, making it one of the key technologies of 6G. However, millimeter waves are susceptible to path loss and obstruction, resulting in limited transmission distance. RIS effectively enhances signal coverage and communication quality by dynamically adjusting the signal reflection path, thereby making up for the limitations of millimeter wave communication. At the same time, NOMA technology has also received widespread attention in the 6G field because of its ability to improve spectrum efficiency and user access capabilities. Industry and academia have conducted a lot of research on the combination of RIS / NOMA technology and millimeter wave communication.

[0003] The first prior art scheme considers a RIS-assisted millimeter-wave multiple-input multiple-output (MIMO) system, and proposes an accelerated Riemann gradient algorithm using a principalized minimization technique to jointly optimize analog beamforming, digital beamforming, and RIS reflection matrix to minimize the sum and mean square errors of the considered system. The second prior art scheme studies the ergodic achievable rate maximization problem of RIS-assisted millimeter-wave MIMO systems under finite scattering conditions, and optimizes the transmit covariance matrix and passive beamforming using a water filling algorithm and a Riemann conjugate gradient algorithm, respectively. The third prior art scheme considers a millimeter-wave NOMA system with imperfect successive interference cancellation (SIC), and proposes a cross-entropy-based machine learning algorithm to jointly optimize user clustering, beamforming, and power allocation. The third prior art scheme studies the beamforming and antenna selection problems in millimeter-wave massive MIMO NOMA systems, and proposes a joint optimization algorithm based on the maximum-minimum (MM) method. The algorithm maximizes the total achievable rate of the system under the premise of considering SIC constraints, base station transmit power limitations, and antenna selection limitations.

[0004] However, the existing technical solutions have the following defects: 1) At present, the research in the relevant field is mainly focused on the millimeter wave system assisted by RIS or the millimeter wave system supported by NOMA technology. Few studies explore the feasibility and potential advantages of combining millimeter wave, RIS and NOMA, and few studies involve the performance optimization of the entire system. Considering the potential advantages of millimeter wave and NOMA in solving spectrum resource limitations and improving system capacity, as well as the outstanding performance of RIS in dealing with obstacle occlusion and enhancing signal coverage, the study of the communication system combining millimeter wave, RIS and NOMA is of great significance to the construction of future wireless networks. 2) At present, most of the research on RIS-assisted millimeter wave systems uses RIS as a whole for auxiliary communication. However, when RIS provides services to multiple users as a whole, multiple beams will be generated at the same time. If the control of RIS is not precise enough, some beams may generate side lobes, resulting in serious mutual interference. In addition, the beam gain away from the main beam direction is usually low, which also affects the availability of RIS. 3) At present, most of the research on resource optimization for RIS-assisted millimeter wave systems is based on traditional iterative optimization algorithms, but this method is not applicable to the RIS partitioning and resource optimization problems in the RIS-assisted millimeter wave NOMA system discussed in the present invention. The reason is that RIS partitioning involves a large number of possible combination choices, and as the number of RIS reflection units and users increases, the complexity of the problem grows exponentially. Traditional iterative algorithms need to search and iterate for each possible partitioning situation, resulting in excessive computational overhead and difficulty in finding the global optimal solution within a limited time. In addition, traditional methods are often based on the convexity assumption of the problem, while RIS partitioning and resource optimization are usually non-convex problems, which further increases the difficulty of solving them. In summary, traditional iterative optimization algorithms cannot effectively deal with such problems. Therefore, it is necessary to combine advanced machine learning techniques to propose a resource optimization solution for RIS-assisted millimeter-wave NOMA systems. Summary of the invention

[0005] In order to solve at least one of the technical problems existing in the prior art to a certain extent, the object of the present invention is to provide a RIS partitioning and resource optimization method and system for a RIS-assisted millimeter wave NOMA system.

[0006] The first technical solution adopted by the present invention is:

[0007] A RIS partitioning and resource optimization method for a RIS-assisted millimeter wave NOMA system comprises the following steps:

[0008] Construct a downlink millimeter wave NOMA multi-user system model; the system model includes a base station, a RIS, and K terminal devices, and the K terminal devices are divided into I clusters;

[0009] Construct RIS model, channel model and transmission model according to the system model;

[0010] Determine the optimization problem based on the constructed model;

[0011] Based on the optimization problem, terminal device grouping, RIS division, active beamforming, passive beamforming and power allocation are jointly optimized to maximize the total achievable rate of terminal devices in downlink transmission.

[0012] Further, the base station is equipped with N uniform linear array antennas, and the RIS is composed of M uniform planar array reflection units;

[0013] Assume that the base station forms at most I′ millimeter-wave beamforming vectors;

[0014] A group of terminal devices served by a single beam vector is defined as a user cluster, and it is assumed that the number of user clusters does not exceed the maximum number of beams that can be formed by the base station, that is, I≤I′;

[0015] use represents the user set of the i-th cluster, Θ = diag(φ 1 ,φ 2 ,…,φ M ) represents the phase shift matrix of RIS, where β m and θ m Represent the amplitude and phase of the reflection unit respectively.

[0016] Furthermore, the RIS model is constructed as follows:

[0017] The M reflection units of RIS are divided into I groups. The surface composed of multiple reflection units is called a sub-RIS, denoted by

[0018] For the grouping of RIS, the core issue is to determine the number of reflection units in each sub-RIS, that is, Where |·| represents the number of elements in the set;

[0019] Definition {Γ i |i=1,2,…I} is a grouping set, where is a grouping matrix, each of which is a diagonal matrix; when the mth element in RIS belongs to the i-th sub-RIS, there is Γ i (m,m)=1, otherwise Γ i (m,m) = 0; In addition, for the convenience of configuration, assuming that the reflection coefficients in the same sub-RIS are the same, the phase shift matrix after grouping is expressed as Θ′ = diag(φ 1 ,φ 2 ,…,φ I ); the final phase shift matrix Θ can be re-expressed as:

[0020]

[0021] Therefore, the RIS partitioning problem is finally transformed into solving the grouping set {Γ i |i=1,2,…I}. In order to ensure performance and reduce complexity, it is assumed that one sub-RIS serves one user cluster.

[0022] Furthermore, the channel model is constructed as follows:

[0023] Modeling channel information using the three-dimensional Saleh-Valenzuela channel model;

[0024] Let M = M x ×M z , where M x and M z are the number of reflection units of RIS in the horizontal and vertical directions respectively. The channel from the base station to the i-th sub-RIS is modeled as:

[0025]

[0026] in is the number of scattering paths from the base station to the i-th sub-RIS, and They represent the azimuth and elevation angles on the lth scattering path, represents the departure angle on the lth scattering path, is the complex channel gain, and are the array response vectors on the lth scattering path at the RIS and the base station, respectively, and are expressed as:

[0027]

[0028] in d is the distance between two elements, λ is the carrier wavelength, T represents transposition; i,k represents the kth terminal device in the ith cluster, then the ith sub-RIS and U i,k The channel between is modeled as:

[0029]

[0030] Among them, l = 0 represents the line-of-sight path (LoS), and l> 0 represents the non-line-of-sight path (NLoS); therefore, starting from the base station, the signal is reflected by the RIS to the terminal device U i,k The cascade channels are:

[0031]

[0032] Here, H represents the conjugate transpose.

[0033] Furthermore, the transmission model is constructed as follows:

[0034] The signal sent by the base station is expressed as:

[0035]

[0036] where w i and i are the beamforming vector and superposition signal of the i-th user cluster respectively; x i,k Indicates that it is sent to U i,k information symbol, and satisfies p i,k For terminal device U i,k The transmission power of

[0037] At the receiving end, the terminal device U i,k The received signal can be expressed as:

[0038]

[0039] Where n i,k Indicates terminal device U i,k Received additive white Gaussian noise;

[0040] At the receiving end, the downlink NOMA scheme is used for decoding, that is, the decoding order is determined according to the ascending order of the user's effective channel gain; it is assumed that the users are indexed in descending order of the effective channel gain, that is, ||g i,k w i || 2 ≥||g i,k+1 w i || 2 ; According to the order of channels considered, after successive interference cancellation (SIC), user U i,k The received signal to interference plus noise ratio (SINR) is:

[0041]

[0042] Therefore, the terminal device U i,k The obtained rate is:

[0043] R i,k =log 2 (1+γ i,k )

[0044] Finally, the total achievable rate of the proposed system is:

[0045] .

[0047] Furthermore, determining the optimization problem according to the constructed model includes:

[0048] By jointly optimizing the terminal device grouping, RIS partitioning, active beamforming, passive beamforming and power allocation, the total achievable rate of the considered system is maximized under the premise of satisfying the user's quality of service (QoS) requirements and the maximum transmission power constraint of the base station. The mathematical expression of the optimization problem is (P1):

[0049]

[0050] C 4 :|θ i |=1,1≤i≤I,

[0051]

[0052] In the optimization problem (P1), the constraint C 1 is the minimum rate constraint for each terminal device; constraint C 2 Ensure that the power allocation of each terminal device is non-negative; Constraint C 3 is the total transmit power constraint; constraint C 4 is the constant modulus constraint of the phase shift vector; constraint C 5 Indicates that the sum of terminal devices in all user clusters should be equal to the total number of users, that is, each terminal device is assigned to only one cluster; constraint C 6 Indicates that the sum of the reflection units in all RIS groups should be equal to the total number of reflection units, that is, each reflection unit is only assigned to one RIS group.

[0053] Furthermore, the joint optimization of terminal device grouping, RIS division, active beamforming, passive beamforming and power allocation includes:

[0054] In the first stage, a terminal device grouping method combining K-means algorithm and Gaussian mixture model (GMM) is proposed to cluster multiple terminal devices to reduce the interference between terminal devices;

[0055] In the second stage, the RIS is adaptively virtually partitioned through the adaptive RIS partitioning model based on deep learning;

[0056] In stage three, a deep reinforcement learning algorithm based on Markov decision making is used to jointly optimize active beamforming, passive beamforming, and power allocation to maximize the total achievable rate of the system.

[0057] Furthermore, the first stage specifically includes:

[0058] A1. Obtain channel state information: The base station obtains the channel state information CSI of all terminal devices by transmitting pilot signals to form a CSI set

[0059] A2. Define the normalized channel correlation coefficient. The normalized channel correlation coefficient between users a and b is defined as:

[0060]

[0061] A3. K-Means algorithm preliminary clustering: C a,b As a metric, the K-means algorithm is used to perform rough clustering of users;

[0062] A4. GMM fine clustering: Based on the coarse cluster centers obtained by the K-means algorithm, GMM is used to perform more refined user clustering.

[0063] Furthermore, the deep learning-based adaptive RIS partitioning model is composed of two cascaded deep neural networks, namely network A and network B; network A is used to output the number of reflection units of each sub-RIS in the horizontal direction and the vertical direction, and network B is used to calculate the total achievable rate of the system;

[0064] The second stage specifically includes:

[0065] B1. Build a cascade network;

[0066] B2. Define the loss function; take the total achievable rate as the loss function of the model, expressed as Loss = -R sum ;

[0067] B3. Execute cascade network:

[0068] B31. Input CSI into network A, and network A outputs the number of reflection units in each sub-RIS in the horizontal and vertical directions, that is, {N x ,N z}.

[0069] B32, according to the output {N x ,N z}, calculate the horizontal and vertical boundaries of each sub-RIS and

[0070] B33, according to the boundary and Calculate the grouping matrix Γ for each sub-RIS i , we get the grouping set {Γ i |i=1,2,…I};

[0071] B34, according to the obtained grouping set {Γ i |i=1,2,…I}, calculate the phase shift matrix Θ of RIS;

[0072] B35, CSI and group set {Γ i |i=1,2,…I} inputs network B, and network B outputs the total achievable rate of the system;

[0073] B4. Use back propagation to train network A and network B.

[0074] Furthermore, the deep reinforcement learning algorithm is composed of two pairs of deep neural networks, namely an actor network and an evaluator network, and their corresponding target networks; the actor network is used to generate an action space, and its input is the state and its output is the action; the evaluator network is used to evaluate the action output by the actor network, and its input is the state and the action output by the actor network, and its output is a Q-value function;

[0075] The third stage specifically includes:

[0076] C1. Construct a DRL model; the actor network and the evaluator network have the same structure, both of which are fully connected deep neural networks;

[0077] C2. Define state space, action space and reward function;

[0078] C3. Execute the DRL model:

[0079] C31, the agent obtains the system's CSI and RIS grouping matrix by observing the environment, forming a state set s (t) ;

[0080] C32, the state s (t) Input the actor network for action selection and output action a (t) ;

[0081] C33, binding state (t) and action a (t) As the input of the evaluator network, it performs action evaluation and outputs the Q-value function;

[0082] C34. Apply actions to the environment and get immediate rewards r (t) , current state s (t) Transition to next state s (t+1) , and {s (t) ,a (t) ,r (t) ,s (t+1)}stored in storage buffer for model training;

[0083] C35, copy the data of the actor network and the evaluator network to their corresponding target networks respectively as the parameters of the target network;

[0084] C4. Use back-propagation to train the actor network and the critic network.

[0085] The second technical solution adopted by the present invention is:

[0086] A RIS partitioning and resource optimization system for a RIS-assisted millimeter wave NOMA system, comprising:

[0087] Base station, equipped with N uniform linear array antennas;

[0088] RIS, consisting of M uniform planar array reflection units, is used for auxiliary communication;

[0089] K terminal devices are divided into I clusters, and the number of clusters does not exceed the maximum number of beams that can be formed by the base station;

[0090] Among them, terminal device grouping, RIS division, active beamforming, passive beamforming and power allocation are jointly optimized to maximize the total achievable rate of terminal devices in downlink transmission.

[0091] Furthermore, the joint optimization procedure includes three stages:

[0092] In the first stage, a terminal device grouping method combining K-means algorithm and Gaussian mixture model (GMM) is proposed to cluster multiple terminal devices to reduce the interference between terminal devices;

[0093] In the second stage, the RIS is adaptively virtually partitioned through the adaptive RIS partitioning model based on deep learning;

[0094] In stage three, a deep reinforcement learning algorithm based on Markov decision making is used to jointly optimize active beamforming, passive beamforming, and power allocation to maximize the total achievable rate of the system.

[0095] The beneficial effects of the present invention are as follows: the present invention proposes a RIS-assisted millimeter wave NOMA system framework, in which users are divided into different clusters, and the RIS is virtually divided into multiple sub-RISs, each of which serves a specific user cluster and transmits information at the same time. Based on this framework, an efficient ML-based resource configuration method is designed to maximize the total achievable rate of the system by jointly optimizing user grouping, RIS partitioning, active beamforming, passive beamforming and power allocation. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the embodiments of the present invention or the drawings of related technical solutions in the prior art are introduced below. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0097] Figure 1 is a system model diagram considered in the embodiment of the present invention;

[0098] Figure 2 It is a schematic diagram of the total achievable rate of the system when the RIS partitioning scheme, the RIS non-partitioning scheme and the RIS average partitioning scheme proposed by the present invention are respectively adopted in the RIS-assisted millimeter wave NOMA system in an embodiment of the present invention;

[0099] Figure 3 It is a schematic diagram of the total achievable rate of the system when the DRL model, Q learning, deep Q network and dual deep Q network proposed in the present invention are respectively adopted in the RIS-assisted millimeter wave NOMA system in an embodiment of the present invention;

[0100] Figure 4 It is a schematic diagram of the total achievable rate of the system when the three-stage algorithm based on machine learning proposed by the present invention and the traditional iterative optimization are respectively adopted in the RIS-assisted millimeter-wave NOMA system in an embodiment of the present invention. DETAILED DESCRIPTION

[0101] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limitations of the present invention. For the step numbers in the following embodiments, they are only provided for the convenience of explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0102] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0103] In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0104] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0105] Terminology explanation:

[0106] RIS: abbreviation of reconfigurable intelligent surface, intelligent super surface.

[0107] In view of the existing technical problems, the present invention proposes a RIS-assisted millimeter wave NOMA communication system framework, in which users are divided into different clusters, and the RIS is virtually divided into multiple sub-RISs, each of which serves a user cluster and assists the base station in completing information transmission. In order to maximize the total achievable rate of the system in downlink transmission, the present invention proposes a joint optimization method for user grouping, RIS virtual division, active beamforming, passive beamforming and power allocation. Specifically, the joint optimization method is a machine learning-based method, which is divided into three stages. In stage one, a user grouping method combining K-means algorithm and Gaussian mixture model (GMM) is proposed to cluster multiple users to reduce interference between users. In stage two, a RIS division method based on deep learning is proposed to adaptively virtually divide RIS. In stage three, a deep reinforcement learning algorithm based on Markov decision is used to jointly optimize active beamforming, passive beamforming and power allocation to maximize the total achievable rate of the system. Simulation results show that compared with other optimization schemes, the proposed optimization allocation scheme can effectively improve the total achievable rate of the system.

[0108] The technical solution of the present invention is explained in detail below in conjunction with the accompanying drawings and specific embodiments.

[0109] (1) System model

[0110] See also Figure 1, this embodiment considers a downlink millimeter-wave NOMA multi-user system, which consists of a base station equipped with N uniform linear array antennas and K single-antenna users (ie, terminal devices, hereinafter referred to as users). Since obstacles block the direct link between the base station and the user, RIS assisted signal transmission consisting of M uniform planar array reflection units is adopted. Assuming that the base station can form a maximum of I′ millimeter-wave beamforming vectors, it can provide service to at least one user with the help of RIS. In practical applications, the number of users usually far exceeds the number of beams that can be formed by the base station. Therefore, multiple users need to be divided into multiple clusters so that the base station can provide information transmission for multiple users at the same time. The present invention defines a group of users served by a single beam vector as a user cluster, and divides K users into I clusters, and assumes that the number of user clusters does not exceed the maximum number of beams that can be formed by the base station, that is, I≤I′. Use represents the user set of the i-th cluster, Θ = diag(φ 1 ,φ 2 ,…,φ M ) represents the phase shift matrix of RIS, where β m and θ m Respectively represent the amplitude and phase of the reflection unit. The present invention focuses on the upper bound of the system performance, so let β m =1.

[0111] a)RIS model

[0112] The present invention divides the M reflection units of RIS into I groups, and a surface composed of multiple reflection units is called a sub-RIS, which is denoted as For the RIS grouping model, the core issue is to determine the number of reflection units in each sub-RIS, that is, where |·| represents the number of elements in the set. i |i=1,2,…I} is a grouping set, where is a grouping matrix, each of which is a diagonal matrix. When the mth element in RIS belongs to the i-th sub-RIS, there is Γ i (m,m)=1, otherwise Γ i (m,m) = 0. In addition, for the convenience of configuration, assuming that the reflection coefficients in the same sub-RIS are the same, the phase shift matrix after grouping is expressed as Θ′ = diag(φ 1 ,φ 2 ,…,φ I ). Through the above analysis, the final phase shift matrix Θ can be re-expressed as:

[0113]

[0114] Therefore, the RIS partitioning problem is finally transformed into solving the grouping set {Γi |i=1,2,…I}. In order to ensure performance and reduce complexity, it is assumed that one sub-RIS serves one user cluster.

[0115] b) Channel Model

[0116] Since the millimeter wave channel has limited scattering characteristics, the three-dimensional Saleh-Valenzuela channel model is used to model the channel information. Let M = M x ×M z , where M x and M z are the number of reflection units in the horizontal and vertical directions of the RIS respectively. The channel model from the base station to the i-th sub-RIS is

[0117]

[0118] in is the number of scattering paths from the base station to the i-th sub-RIS, and They represent the azimuth and elevation angles on the lth scattering path, represents the departure angle on the lth scattering path, is the complex channel gain, and are the array response vectors on the lth scattering path at the RIS and the base station, respectively, and are expressed as:

[0119]

[0120] in d is the distance between two elements, and λ is the carrier wavelength. i,k represents the kth user in the ith cluster, then the ith sub-RIS and U i,k The channel between is modeled as:

[0121]

[0122] In equations (2) and (5), l = 0 represents a line-of-sight path (LoS), and l > 0 represents a non-line-of-sight path (NLoS). i,k The cascade channels are:

[0123]

[0124] c) Transmission Model

[0125] The signal sent by the base station is expressed as where w i and i are the beamforming vector and superimposed signal of the i-th user cluster respectively. where x i,k Indicates that it is sent to U i,k information symbol, and satisfies p i,k For U i,k At the receiving end, user U i,k The received signal can be expressed as:

[0126]

[0127] Where n i,k Indicates U i,k The received additive white Gaussian noise. In equation (7), the first term is the desired signal, the second term is the intra-cluster interference, and the third term is the inter-cluster interference.

[0128] At the receiving end, the downlink NOMA scheme is used for decoding, that is, the decoding order is determined according to the ascending order of the user's effective channel gain. Assume that the users are indexed in descending order of their effective channel gain, that is, ||g i,k w i || 2 ≥||g i,k+1 w i || 2 According to the order of channels considered, after successive interference cancellation (SIC), user U i,k The received signal to interference plus noise ratio (SINR) is:

[0129]

[0130] Therefore, user U i,k The obtained rate is:

[0131] R i,k =log 2 (1+γ i,k ). (9)

[0132] Finally, the total achievable rate of the proposed system is:

[0133]

[0134] (2) Optimization Problem

[0135] Based on the above definition, the present invention aims to maximize the total achievable rate of the system under consideration by jointly optimizing user grouping, RIS partitioning, active beamforming, passive beamforming and power allocation, while satisfying the user quality of service requirements (QoS) and the maximum transmit power constraint of the base station. The mathematical expression of the optimization problem is (P1):

[0136]

[0137] In the optimization problem (P1), the constraint C 1 is the minimum rate constraint for each user. Constraint C 2 Ensure that the power allocation for each user is non-negative. Constraint C 3 is the total transmit power constraint. Constraint C 4 is the constant modulus constraint of the phase shift vector. Constraint C 5 Indicates that the sum of users in all user clusters should be equal to the total number of users, that is, each user is assigned to only one cluster. Constraint C 6 Indicates that the sum of the reflection units in all RIS groups should be equal to the total number of reflection units, that is, each reflection unit is only assigned to one RIS group.

[0138] (3) Joint resource optimization allocation method

[0139] In the optimization problem (P1) under consideration, there are four variables to be optimized: RIS grouping matrix, active beamforming, passive beamforming, and power allocation. Among them, the optimization of the RIS grouping matrix is ​​a complex integer selection problem, C 1 and C 3 It is a non-convex constraint, and the various optimization variables are highly coupled. In order to effectively optimize these variables, the present invention proposes a three-stage optimization method based on machine learning. In stage one, a user grouping method (called K-GMM) combining the K-means algorithm and the Gaussian mixture model (GMM) is proposed to cluster multiple users to reduce interference between users. In stage two, a RIS partitioning method based on deep learning is proposed to virtually partition the RIS. In stage three, a deep reinforcement learning algorithm based on Markov decision is used to jointly optimize active beamforming, passive beamforming and power allocation to maximize the total achievable rate of the system. The specific process is:

[0140] a) Phase 1: User clustering based on K-GMM

[0141] At this stage, the goal is to divide users into different clusters based on channel correlation. In order to enhance the coverage of millimeter wave signals, channel correlation can be used to divide users into several clusters, each cluster being served by a millimeter wave beam. Although the K-means algorithm can achieve user clustering, it is only applicable to spherical distribution and it is difficult to meet the needs of real communication environments. In fact, user distribution is usually random and does not present a spherical structure. Therefore, for real communication environments, GMM can be used for user clustering. The algorithm approximates the actual distribution of users through multiple Gaussian distribution functions. However, GMM is more sensitive to initial values, so the present invention proposes to use K-means in combination with GMM to optimize the clustering results. Specifically, the users are first preliminarily clustered by the K-means algorithm to obtain the initial clustering center, and then the users are finely clustered by GMM. The specific steps are as follows:

[0142] Step 1: Obtain channel state information. The base station obtains the channel state information (CSI) of all users by transmitting pilot signals to form a CSI set.

[0143] Step 2: Define the normalized channel correlation coefficient. The normalized channel correlation coefficient between users a and b is defined as:

[0144]

[0145] Step 3: K-Means algorithm preliminary clustering. a,b As a metric, the K-means algorithm is used to perform rough clustering of users. The specific steps are as follows:

[0146] A1. Randomly select I users as the initial cluster center, represented by {c 1 ,c 2 ,…,c I}.

[0147] A2. Using C a,b Perform coarse clustering on users, that is, users with larger normalized channel correlation coefficients are classified into one category, and users with smaller correlations are classified into different clusters. For example, user k should be classified into the i-th cluster if and only if:

[0148]

[0149] A3. Update the mean of all user channel vectors assigned to each cluster center:

[0150]

[0151] in Represents the set of users assigned to the i-th cluster center.

[0152] A4. Repeat steps A2 and A3 until the cluster centers converge or the predetermined number of iterations is reached.

[0153] Step 4: GMM fine clustering. Based on the coarse cluster centers obtained by the K-means algorithm, GMM is used to perform finer user clustering. GMM assumes that the channel vector g of each user is k Generated by multiple Gaussian distributions, expressed as:

[0154]

[0155] where π i is the mixture weight of the i-th Gaussian distribution, satisfying N(g k |μ i ,Σ i ) is the probability density function of the i-th Gaussian distribution, μ i and Σ i are the mean and covariance of the i-th Gaussian distribution respectively. The specific steps of GMM fine clustering users are as follows:

[0156] B1. The center {c 1 ,c 2 ,…,c I} as the mean μ of GMM i , initialize the covariance matrix Σ i is the unit matrix, the mixing coefficient

[0157] B2. Calculate the user channel vector g k The posterior probability of belonging to each cluster:

[0158]

[0159] where γ k,i represents the probability that user k belongs to the i-th Gaussian distribution.

[0160] B3. Update the mixing coefficient, mean and covariance matrix using the following formula:

[0161]

[0162] B4. Repeat steps B2 and B3 until the log-likelihood function converges. The log-likelihood function is defined as:

[0163]

[0164] After multiple iterations, GMM will output the posterior probability γ of each user belonging to each Gaussian distribution k,i, users can be assigned to different clusters based on these probabilities. Usually, users are assigned to the cluster with the largest posterior probability, that is:

[0165]

[0166] b) Phase 2: RIS Partitioning Based on Deep Learning

[0167] In this stage, the goal is to adaptively divide the RIS into multiple sub-RISs according to the communication needs of each user cluster, each sub-RIS serves a user cluster, so as to improve the availability of the RIS. As mentioned above, the RIS partition problem is ultimately transformed into solving the grouping set {Γ i}. Since this problem is a combinatorial optimization problem, it is difficult to solve it using traditional convex optimization methods. Therefore, the embodiment of the present invention proposes a deep learning model to solve it. However, directly solving {Γ i} is somewhat challenging for the following reasons. First, the set {Γ i} for each Γ i The size of is M×M. If they are directly used as output, the output dimension of the model is IM 2 As the number of reflection units increases, the computational overhead increases. Secondly, the one-dimensional grouping matrix Γ i It does not necessarily correspond completely to the two-dimensional plane RIS in space. For example, in the one-dimensional grouping matrix, elements i and j are divided into the same sub-RIS, but they are far away in the two-dimensional plane RIS. In order to solve the above problem, the present invention does not directly calculate the grouping matrix {Γ i}, but let the deep learning model indirectly output the number of reflection units in each sub-RIS in the horizontal and vertical directions. Specifically, suppose a RIS is virtually divided into I = I x ×I z RIS, of which I x and I z Represents the number of divisions in the horizontal and vertical directions respectively, then the output of the model is {N x ,N z},in Represents a vector containing the number of reflection units in the horizontal direction, is the number of reflection units in the horizontal direction of the i-th sub-RIS, Represents a vector containing the number of vertical reflection units, is the number of reflection units of the ith sub-RIS in the vertical direction. x ,N z The horizontal and vertical boundaries of each sub-RIS can be calculated and expressed as:

[0168]

[0169] where X∈{x,z}. Therefore, the final grouping matrix Γ i (x,z) can be calculated according to the following formula:

[0170]

[0171] Based on the above analysis, the embodiment of the present invention proposes an adaptive RIS partitioning model based on deep learning, which is composed of two cascaded deep neural networks, namely network A and network B. Network A is used to output the number of reflection units in the horizontal and vertical directions of each sub-RIS, and network B is used to calculate the total achievable rate of the system. The specific steps are as follows:

[0172] Step 1: Construct a cascade network. The cascade network proposed in the present invention consists of networks A and B with the same structure, both of which are fully connected deep neural networks, including an input layer, an output layer, a hidden layer and a residual block. In order to overcome the influence of the change of the parameters of the previous layer on the subsequent input distribution, the present invention adopts batch normalization processing in the hidden layer. The model uses the ReLU function as the activation function and uses the Adam optimizer to train the network.

[0173] Step 2: Define the loss function. The purpose of adaptive partitioning of RIS is to improve its availability and maximize the total achievable rate of the system. Therefore, the present invention uses the total achievable rate as the loss function of the model, expressed as Loss = -R sum .

[0174] Step 3: Execute the cascade network. The proposed cascade network execution process is as follows:

[0175] C1. Input CSI into network A, and network A outputs the number of reflection units of each sub-RIS in the horizontal and vertical directions, that is, {N x ,N z}.

[0176] C2, according to the output {N x ,N z}, use formula (22) to calculate the boundaries Eix and

[0177] C3. According to the boundary and Use formula (23) to calculate the grouping matrix Γ of each sub-RIS i , we get the grouping set {Γ i |i=1,2,…I}.

[0178] C4, according to the obtained grouping set {Γ i|i=1,2,…I}, and use formula (1) to calculate the phase shift matrix Θ of RIS.

[0179] C5, CSI and group set {Γ i |i=1,2,…I} inputs network B, and network B outputs the total achievable rate of the system.

[0180] Step 4: Train the network. Use back propagation to train Network A and Network B.

[0181] c) Phase 3: Multi-dimensional resource optimization based on deep reinforcement learning

[0182] In this stage, the goal is to maximize the total achievable rate of the system while satisfying the constraints of base station transmit power and user QoS by jointly optimizing active beamforming, passive beamforming and power allocation. Since this problem is a complex non-convex optimization problem, the traditional optimization algorithm has high computational overhead when solving it and it is difficult to obtain a global optimal solution. Therefore, the present invention proposes an algorithm based on deep reinforcement learning (DRL) to jointly optimize active beamforming, passive beamforming and power allocation. In the DRL algorithm, the agent interacts with the environment through trial and error, and applies actions to the environment, and continuously optimizes the actions through immediate rewards and state transition information fed back by the environment. This process can be abstracted as a Markov decision process. In the millimeter wave NOMA system considered in the present invention, the changes in beamforming, power allocation and the total achievable rate of the system caused by the decision of the agent are abstracted as a Markov transition, and the current state s (t) ∈S, action a (t) ∈A, reward r (t) and the next state s (t+1) ∈S. The goal of the DRL algorithm is to find the action that maximizes the long-term total achievable rate of the system under a given state. Therefore, the agent introduces the Q-value function to quantify the potential reward of each action. The Q-value function satisfies the following Bellman equation:

[0183] Q(s (t) ,a (t) )=r (t) (s (t) ,a (t) )+γmaxQ(s (t) ,a (t) ), (twenty four)

[0184] Where γ∈[0,1] is the discount rate, then the optimal action is defined as:

[0185]

[0186] The proposed DRL algorithm consists of two pairs of deep neural networks, namely the actor network and the evaluator network, and their corresponding target networks. The actor network is used to generate the action space, with the input being the state and the output being the action. The evaluator network is used to evaluate the action output by the actor network, with the input being the state and the action output by the actor network, and the output being the Q-value function. The specific execution steps of the DRL-based resource optimization algorithm proposed in the present invention are as follows:

[0187] Step 1: Build the network model. The actor network and the evaluator network have the same structure, both are fully connected deep neural networks, consisting of an input layer, an output layer and two hidden layers. In order to overcome the influence of the parameter change of the previous layer on the subsequent input distribution, batch normalization is used in the hidden layer. In order to meet the power constraint and the constant modulus constraint of the RIS reflection unit in problem (P1), a normalization layer is introduced at the end of the actor network to adjust the output Θ′ and {p i,k} for normalization. The tanh function is used as the activation function to process non-negative inputs, and the Adam optimizer is used to train the network.

[0188] Step 2: Define the state space, action space, and reward function. The specific definitions are as follows:

[0189] 1) State space: The state is a set of observations that describe the characteristics of the environment. The present invention converts the grouping matrix {Γ i} and CSI matrix Defined as state space. Since neural networks only accept real number inputs, when constructing the state space, if complex number inputs are involved, they need to be divided into real and imaginary parts and input into the network separately. The constructed state space is expressed as:

[0190]

[0191] Where real{·} and imag{·} represent the real and imaginary parts of a complex number.

[0192] 2) Action space: Action is a set of choices. The agent selects one action from the action set to execute during the learning process. The action space of the present invention is composed of three optimization variables, namely active beamforming {w i}, passive beamforming Θ′ and power {p i,k}. The constructed action space is:

[0193] a (t) ={real{w i ,Θ′},imag{w i ,Θ′},{p i,k}}. (27)

[0194] 3) Reward function: The reward is the agent in state s (t) Next, perform action a (t) The goal of this invention is to maximize the total achievable rate of the system while meeting the QoS requirements of the user. Therefore, the reward function is defined as:

[0195]

[0196] in

[0197]

[0198] In formula (28), the first part is the total achievable rate of the system, the second part is the cost function of not meeting the user QoS, and β∈[0,1] is the QoS control coefficient.

[0199] Step 3: Execute the DRL model. The proposed DRL model execution process is as follows:

[0200] D1. The agent obtains the system's CSI and RIS grouping matrices by observing the environment, forming a state set s (t) , where the environment is the entire communication system considered by the present invention.

[0201] D2, the state s (t) Input the actor network for action selection and output action a (t) .

[0202] D3, binding state s (t) and action a (t) As the input of the evaluator network, action evaluation is performed and the Q value function is output.

[0203] D4. Apply the action to the environment and get immediate reward r (t) , current state s (t) Transition to next state s (t+1) , and {s (t) ,a (t) ,r (t) ,s (t+1)}stored in the storage buffer for model training.

[0204] D5. Copy the data of the actor network and the evaluator network to their corresponding target networks respectively as the parameters of the target network.

[0205] Step 4: Train the network. Use back-propagation to train the actor network and the critic network.

[0206] (4) Simulation Experiment Results

[0207] In this embodiment, simulation analysis verifies the effectiveness of the proposed three-stage resource optimization method based on machine learning in improving the total achievable rate of the system. This embodiment analyzes from three levels: performance verification of RIS partitioning, performance verification of DRL algorithm, and overall performance verification of the three-stage algorithm.

[0208] The relevant parameters of the simulation experiment are set as follows: the base station coordinates are (0m, 0m, 15m), the RIS coordinates are (20m, 20m, 15m), and the users are randomly distributed in a circular area with a center of (30m, 30m, 0m) and a radius of 10m. In the millimeter wave channel shown in equations (2) and (5), the complex channel gain and Satisfies Gaussian distribution PL(d TR ) is the path loss, and its calculation formula is:

[0209] PL(d TR ) = η a +10η b log 10 (d TR )+ψ, (30)

[0210] in Indicates the change of shadow, d TR represents the distance between the transmitter and the receiver. Set η a =61.4,η b =2,σ ψ =5.8dB. The system operates at 28GHz and the noise power is σ 2 =-90dBm. The user's QoS requirement is r i,k =1bps / Hz. The discount rate and QoS control coefficient are set to γ=0.99 and β=0.6 respectively.

[0211] First, the performance of the proposed deep learning-based RIS partitioning method is verified. The following two comparison schemes are considered: (1) RIS unpartitioned scheme, which uses RIS as a whole for reflected signals. (2) RIS average partitioning scheme, which divides a RIS into I sub-RISs, each of which serves a user cluster and transmits information at the same time. The above schemes use the K-GMM algorithm for user clustering and use the proposed DRL algorithm to jointly optimize active beamforming, passive beamforming and power allocation.

[0212] Figure 2The total achievable rate of the system when the deep learning-based RIS partitioning scheme proposed in the embodiment of the present invention is adopted in the RIS-assisted millimeter-wave NOMA system under consideration is shown, and compared with the RIS unpartitioned scheme and the RIS average partitioning scheme. It can be seen from the simulation diagram that the performance of the RIS partitioning scheme proposed in the present invention is significantly better than the other two schemes, indicating that the RIS partitioning method proposed in the present invention can better tap the utility of RIS, thereby improving its utilization rate, which further illustrates that RIS partitioning is crucial to improving the total achievable rate of the system.

[0213] Secondly, the performance of the proposed DRL-based multi-dimensional resource optimization scheme is verified. The proposed method is compared with the following three DRL models, namely Q-learning, deep Q-network and dual deep Q-network. Among them, Q-learning is the most basic reinforcement learning algorithm, which uses a table to store the Q function; the deep Q-network introduces a neural network to approximate the Q function and can handle the problem of high-dimensional state space; the dual deep Q-network is an improved version of the deep Q-network, which uses two neural networks to alleviate the over-estimation problem of the deep Q-network. When optimizing variables using the above three models, the K-GMM algorithm is used for user clustering, and the proposed deep learning-based RIS partitioning method is used for RIS partitioning.

[0214] Figure 3 The total achievable rate of the system when the DRL-based multi-dimensional resource optimization scheme proposed in the present invention is adopted in the considered RIS-assisted millimeter-wave NOMA system is shown, and compared with Q learning, deep Q network and dual deep Q network. It can be seen from the simulation diagram that the performance of the DRL-based multi-dimensional resource optimization algorithm proposed in the present invention is significantly better than the other three models, which shows that the DRL model proposed in the present invention can obtain a better parameter solution, which further shows that beamforming optimization design and power allocation are crucial to improving the total achievable rate of the system.

[0215] Finally, the overall performance of the proposed three-stage algorithm based on machine learning is verified. The proposed three-stage algorithm is compared with the following three schemes: (1) Scheme 1 uses the K-means algorithm to cluster users and the iterative optimization algorithm to perform active beamforming and power allocation. RIS-assisted communication is not introduced in this scheme. (2) Scheme 2 uses the K-means algorithm to cluster users and the iterative optimization algorithm to perform active beamforming and power allocation. This scheme does not divide the RIS, and the passive beamforming is randomly generated. (3) Scheme 3 uses the K-means algorithm to cluster users and the iterative optimization algorithm to perform active beamforming, passive beamforming and power allocation. This scheme does not divide the RIS.

[0216] Figure 4The total achievable rate of the system when the three-stage resource optimization allocation scheme based on machine learning proposed in the present invention is adopted in the considered RIS-assisted millimeter-wave NOMA system is shown, and compared with the traditional iterative optimization algorithm. It can be seen from the simulation diagram that the performance of the three-stage algorithm based on machine learning proposed in the present invention is significantly better than the other three algorithms, which shows that resource optimization allocation is crucial to improving the total achievable rate of the system.

[0217] (5) Advantages and beneficial effects

[0218] At present, the research in related fields mainly focuses on RIS-assisted millimeter wave systems or millimeter wave systems supported by NOMA technology, but few studies explore the feasibility and potential advantages of combining millimeter waves, RIS and NOMA, and few studies involve the performance optimization of the entire system. At the same time, most of the current research on RIS-assisted millimeter wave systems uses RIS as a whole for auxiliary communication. However, when RIS provides services to multiple users as a whole, multiple beams will be generated at the same time. If the control of RIS is not precise enough, some beams may generate side lobes, resulting in serious mutual interference. Moreover, the beam gain away from the main beam direction is usually low, which also affects the availability of RIS. Virtually dividing RIS into multiple small sub-RISs, each of which serves a specific user, can avoid these problems to a certain extent. However, few studies involve the virtual division of RIS. In addition, most of the current research on resource optimization for RIS-assisted millimeter wave systems is based on traditional iterative optimization algorithms. Such methods have high computational overhead and are difficult to obtain global optimal solutions, and cannot be well adapted to the RIS-assisted millimeter wave NOMA system considered in the present invention. Considering the research status of the prior art, the present invention proposes a RIS-assisted millimeter wave NOMA system framework, as well as a corresponding RIS division and resource configuration method. The present invention mainly has the following advantages:

[0219] 1) The present invention proposes a RIS-assisted millimeter-wave NOMA system, which provides a new solution for future wireless network construction and conducts research on the performance optimization of the system.

[0220] 2) In order to improve the coverage of millimeter waves and alleviate interference between users, the present invention proposes a user clustering method combining the K-means algorithm and the Gaussian mixture model (GMM) to divide users into several clusters.

[0221] 3) Based on user clustering, the present invention further proposes a RIS partitioning method based on deep learning, which virtually divides the RIS into multiple sub-RISs, each of which serves a user cluster and transmits information at the same time. This method can adaptively optimize the number of reflection units in each sub-RIS according to the actual communication needs of the user cluster to improve the availability of the RIS.

[0222] 4) Based on the proposed system, in order to improve the total achievable rate, the present invention proposes a joint optimization algorithm of active beamforming, passive beamforming and power allocation based on deep reinforcement learning (DRL), which can effectively improve the total achievable rate of the system.

[0223] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0224] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable ordinary technicians in the field to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made based on the essence of the content of the present invention should be included in the protection scope of the present invention.

Claims

1. A RIS partitioning and resource optimization method for a RIS-assisted millimeter wave NOMA system, characterized in that: The following steps are involved: Construct a downlink millimeter wave NOMA multi-user system model; The system model includes a base station, a RIS and K terminal devices, and the K terminal devices are divided into I clusters; Construct RIS model, channel model and transmission model according to the system model; Determine the optimization problem based on the constructed model; Based on the optimization problem, terminal device grouping, RIS division, active beamforming, passive beamforming and power allocation are jointly optimized to maximize the total achievable rate of terminal devices in downlink transmission.

2. The RIS partitioning and resource optimization method of a RIS-assisted millimeter wave NOMA system according to claim 1, characterized in that: The base station is equipped with N uniform linear array antennas, and the RIS is composed of M uniform planar array reflection units; Assume that the base station forms at most I′ millimeter-wave beamforming vectors; A group of terminal devices served by a single beam vector is defined as a user cluster, and it is assumed that the number of user clusters does not exceed the maximum number of beams that can be formed by the base station, that is, I≤I′; use represents the user set of the i-th cluster, Θ = diag(φ1, φ2, …, φ M ) represents the phase shift matrix of RIS, where β m and θ m Represent the amplitude and phase of the reflection unit respectively.

3. The RIS partitioning and resource optimization method of a RIS-assisted millimeter wave NOMA system according to claim 1, characterized in that: The RIS model is constructed as follows: The M reflection units of RIS are divided into I groups. The surface composed of multiple reflection units is called a sub-RIS, denoted by For the grouping of RIS, the core issue is to determine the number of reflection units in each sub-RIS, that is, Where |·| represents the number of elements in the set; Definition {Γ i |i=1,2,…I} is a grouping set, where is a grouping matrix, each of which is a diagonal matrix; when the mth element in RIS belongs to the i-th sub-RIS, there is Γ i (m,m)=1, otherwise Γ i (m,m) = 0; In addition, for the convenience of configuration, assuming that the reflection coefficients in the same sub-RIS are the same, the phase shift matrix after grouping is expressed as Θ′ = diag(φ1, φ2, …, φ I ); the final phase shift matrix Θ can be re-expressed as: Therefore, the RIS partitioning problem is finally transformed into solving the grouping set {Γ i |i=1,2,…I}.

4. The RIS partitioning and resource optimization method of a RIS-assisted millimeter wave NOMA system according to claim 1, characterized in that: The channel model is constructed as follows: Modeling channel information using the three-dimensional Saleh-Valenzuela channel model; Let M = M x ×M z , where M x and M z are the number of reflection units of RIS in the horizontal and vertical directions respectively. The channel from the base station to the i-th sub-RIS is modeled as: in is the number of scattering paths from the base station to the i-th sub-RIS, and They represent the azimuth and elevation angles on the lth scattering path, represents the departure angle on the lth scattering path, is the complex channel gain, and are the array response vectors on the lth scattering path at the RIS and the base station, respectively, and are expressed as: in d is the distance between two elements, λ is the carrier wavelength, T represents transposition; i,k represents the kth terminal device in the ith cluster, then the ith sub-RIS and U i,k The channel between is modeled as: Among them, l = 0 represents the line-of-sight path, and l> 0 represents the non-line-of-sight path; therefore, starting from the base station, it is reflected by the RIS to reach the terminal device U i,k The cascade channels are: Here, H represents the conjugate transpose.

5. The RIS partitioning and resource optimization method of a RIS-assisted millimeter wave NOMA system according to claim 1, characterized in that: The transmission model is constructed as follows: The signal sent by the base station is expressed as: where w i and i are the beamforming vector and superposition signal of the i-th user cluster respectively; x i,k Indicates that it is sent to U i,k information symbol, and satisfies p i,k For terminal device U i,k The transmission power of At the receiving end, the terminal device U i,k The received signal can be expressed as: where n i,k Indicates terminal device U i,k Received additive white Gaussian noise; At the receiving end, the downlink NOMA scheme is used for decoding, that is, the decoding is determined in ascending order of the user's effective channel gain. Order; assume that users are indexed in descending order of effective channel gain, i.e. ||g i,k w i || 2 ≥||g i,k+1 w i || 2 ; According to the order of channels considered, after interference elimination, user U i,k The received signal to interference plus noise ratio is: Therefore, the terminal device U i,k The obtained rate is: R i,k =log2(1+γ i,k ) Finally, the total achievable rate of the proposed system is:

6. The RIS partitioning and resource optimization method of a RIS-assisted millimeter wave NOMA system according to claim 1, characterized in that: Determining the optimization problem according to the constructed model includes: By jointly optimizing the terminal device grouping, RIS partitioning, active beamforming, passive beamforming and power allocation, the total achievable rate of the considered system is maximized under the premise of meeting the user service quality requirements and the maximum transmission power constraint of the base station. The mathematical expression of the optimization problem is (P1): C4:|θ i |=1,1≤i≤I, In the optimization problem (P1), constraint C1 is the minimum rate constraint for each terminal device; constraint C2 ensures that the power allocation of each terminal device is a non-negative value; constraint C3 is the total transmission power constraint; constraint C4 is the constant modulus constraint of the phase shift vector; constraint C5 indicates that the sum of terminal devices in all user clusters should be equal to the total number of users, that is, each terminal device is only designated as one cluster; constraint C6 indicates that the sum of reflection units in all RIS groups should be equal to the total number of reflection units, that is, each reflection unit is only designated as one RIS group.

7. The RIS partitioning and resource optimization method of a RIS-assisted millimeter wave NOMA system according to claim 1, characterized in that: The joint optimization of terminal device grouping, RIS division, active beamforming, passive beamforming and power allocation includes: In the first stage, a terminal device grouping method combining K-means algorithm and Gaussian mixture model is proposed to cluster multiple terminal devices to reduce interference between terminal devices; In the second stage, the RIS is adaptively virtually partitioned through the adaptive RIS partitioning model based on deep learning; In stage three, a deep reinforcement learning algorithm based on Markov decision making is used to jointly optimize active beamforming, passive beamforming, and power allocation to maximize the total achievable rate of the system.

8. The RIS partitioning and resource optimization method of a RIS-assisted millimeter wave NOMA system according to claim 7, characterized in that: The first stage specifically includes: A1. Obtain channel state information: The base station obtains the channel state information CSI of all terminal devices by transmitting pilot signals to form a CSI set A2. Define the normalized channel correlation coefficient. The normalized channel correlation coefficient between users a and b is defined as: A3. K-Means algorithm preliminary clustering: C a,b As a metric, the K-means algorithm is used to perform rough clustering of users; A4. GMM fine clustering: Based on the coarse cluster centers obtained by the K-means algorithm, GMM is used to perform more refined user clustering.

9. The RIS partitioning and resource optimization method of a RIS-assisted millimeter wave NOMA system according to claim 7, characterized in that: The deep learning-based adaptive RIS partitioning model consists of two cascaded deep neural networks, namely network A and network B; network A is used to output the number of reflection units of each sub-RIS in the horizontal and vertical directions, and network B is used to calculate the total achievable rate of the system; The second stage specifically includes: B1. Build a cascade network; B2. Define the loss function; take the total achievable rate as the loss function of the model, expressed as Loss = -R sum ; B3. Execute cascade network: B31. Input CSI into network A, and network A outputs the number of reflection units in each sub-RIS in the horizontal and vertical directions, that is, {N x ,N z }; B32, according to the output {N x ,N z }, calculate the horizontal and vertical boundaries of each sub-RIS and B33, according to the boundary and Calculate the grouping matrix Γ for each sub-RIS i , we get the grouping set {Γ i |i=1,2,…I}; B34, according to the obtained grouping set {Γ i |i=1,2,…I}, calculate the phase shift matrix Θ of RIS; B35, CSI and group set {Γ i |i=1,2,…I} inputs network B, and network B outputs the total achievable rate of the system; B4. Use back propagation to train network A and network B.

10. The RIS partitioning and resource optimization method of a RIS-assisted millimeter wave NOMA system according to claim 7, characterized in that: The deep reinforcement learning algorithm is composed of two pairs of deep neural networks, namely the actor network and the evaluator network, and their corresponding target networks; the actor network is used to generate the action space, and its input is the state and its output is the action; the evaluator network is used to evaluate the action output by the actor network, and its input is the state and the action output by the actor network, and its output is the Q value function; The third stage specifically includes: C1. Construct a DRL model; the actor network and the evaluator network have the same structure, both of which are fully connected deep neural networks; C2. Define state space, action space and reward function; C3. Execute the DRL model: C31, the agent obtains the system's CSI and RIS grouping matrix by observing the environment, forming a state set s (t) ; C32, the state s (t) Input the actor network for action selection and output action a (t) ; C33, binding state (t) and action a (t) As the input of the evaluator network, it performs action evaluation and outputs the Q-value function; C34. Apply actions to the environment and get immediate rewards r (t) , current state s (t) Transition to next state s (t+1) , and {s (t) ,a (t) ,r (t) ,s (t+1) }stored in storage buffer for model training; C35, copy the data of the actor network and the evaluator network to their corresponding target networks respectively as the parameters of the target network; C4. Use back-propagation to train the actor network and the critic network.

Images