A Distributed RIS-Assisted ISAC System and a Joint Resource Optimization Method

By combining distributed RIS and NOMA technologies and employing a two-stage optimization method using graph neural networks and multi-agent deep reinforcement learning, the problems of signal obstruction and small coverage of the ISAC system in the mmWave band were solved, thereby maximizing the total system throughput and reducing computational complexity.

CN120676378BActive Publication Date: 2026-11-17SOUTH CHINA UNIV OF TECH
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202510736044.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-06-04
Publication Date
2026-11-17
Estimated Expiration
2045-06-04

AI Technical Summary

Technical Problem

In existing technologies, when ISAC systems are combined with NOMA technology in the mmWave band, they face problems such as signal obstruction and small coverage area. A single RIS-assisted system cannot effectively cover all communication users and sensing targets. Traditional algorithms have high computational complexity in scenarios with large-scale users and sensing targets, and lack optimization for RIS and user pairing.

Method used

A system framework combining distributed RIS, NOMA, and ISAC technologies is adopted. Through a two-stage optimization method based on graph neural networks and multi-agent deep reinforcement learning, the system optimizes RIS and user pairing, base station active beamforming, and RIS passive beamforming to maximize the total system throughput.

Benefits of technology

It effectively improved the system's total throughput, achieved a balance between communication and sensing performance, reduced computational complexity, and improved spectrum utilization.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120676378B_ABST
    Figure CN120676378B_ABST
Patent Text Reader

Abstract

This invention discloses a distributed RIS-assisted ISAC system and a joint resource optimization method. The system includes a base station, a distributed RIS, and user terminals. The distributed RIS assists user terminals in mmWave communication. Each user terminal can only be served by one RIS, and each RIS can serve user terminals in one NOMA group. In each NOMA group served by a RIS, target information of each user is decoded using SIC technology. Sensing targets are directly detected by the base station, and sensing beam gain is used as an indicator of sensing performance. Joint optimization is performed on RIS and user pairing, base station active beamforming, and RIS passive beamforming to maximize the total system throughput while minimizing the beam gain for each sensing target. This invention proposes a framework that combines distributed RIS, NOMA, mmWave communication, and ISAC technology, and studies the application of machine learning methods within it, effectively improving the total system throughput.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of communication technology, and in particular to a distributed RIS-assisted ISAC system and a resource joint optimization method. Background Technology

[0002] Over the past few years, mmWave communication, with its extensive bandwidth capabilities, has become a key technology for meeting the demands of high-speed data transmission and expanded capacity. However, millimeter-wave signals face significant challenges due to their susceptibility to congestion and limited ability to penetrate obstacles. To overcome this inherent limitation, Relay Interruptions (RIS) have emerged as a particularly innovative technology. Composed of planar arrays with passive reflective elements (REs), RIS can intelligently manipulate electromagnetic signals to overcome channel limitations and provide superior performance compared to traditional relay methods. In complex wireless environments, such as urban areas, multiple RIS can be deployed to assist communication between base stations and users, thereby establishing virtual line-of-sight (VLoS) paths between them. Furthermore, Non-Orthogonal Multiple Access (NOMA) has emerged as a promising technology that further improves spectral efficiency by multiplexing communication users in the power domain. Unlike traditional Orthogonal Multiple Access (OMA) schemes, NOMA can significantly reduce inter-user interference using Serial Interference Cancellation (SIC), making it particularly suitable for the large-scale connectivity needs of future communication and sensing networks. Therefore, the integration of NOMA with millimeter-wave communication and RIS technologies can overcome the inherent limitations of millimeter-wave propagation while maximizing spectrum utilization.

[0003] On the other hand, since wireless spectrum resources are limited, the ideal solution for next-generation wireless communication systems to achieve sensing is to reuse existing communication frequency bands, sharing the spectrum for sensing and communication to improve spectrum utilization. Therefore, ISAC technology has emerged, integrating the hardware infrastructure for communication and sensing functions into the base station, enabling communication and sensing to operate on the same time-frequency resources. To effectively improve sensing and communication performance in a distributed RIS-assisted mmWave band ISAC system, an effective method is needed to solve the multivariate joint optimization problem involving RIS-user pairing, active beamforming, and passive beamforming. Summary of the Invention

[0004] In order to at least partially solve one of the technical problems existing in the prior art, the purpose of this invention is to provide a distributed RIS-assisted ISAC system and a resource joint optimization method.

[0005] The first technical solution adopted in this invention is:

[0006] A distributed RIS-assisted ISAC system includes a base station, multiple distributed RIS, and multiple user terminals;

[0007] Among them, the distributed RIS assists user terminals in mmWave communication. Each user terminal can only be served by one RIS, and each RIS can serve user terminals in one NOMA group. In each NOMA group served by the RIS, the target information of each user is decoded through SIC technology. The sensed target is directly detected by the base station, and the sensed beam gain is used as the indicator of sensed performance.

[0008] Joint optimization of RIS and user pairing, base station active beamforming and RIS passive beamforming is performed to maximize the total system throughput while minimizing the beam gain for each sensing target.

[0009] Furthermore, the joint optimization process includes two phases:

[0010] In the first stage, a graph neural network-based approach is used to optimize the pairing between RIS and users. Attention mechanisms and residual connection techniques are employed to better extract key graph information to obtain a better pairing strategy and effectively avoid gradient vanishing.

[0011] In the second stage, based on the pairing strategy obtained in the first stage, the active beamforming of the base station and the passive beamforming of the RIS are jointly optimized using the MADDPG method based on multi-agent deep reinforcement learning. In this stage, the base station and each RIS are constructed as an independent agent, and each agent consists of an actor network, a critic network, a target actor network, and a target critic network.

[0012] The second technical solution adopted in this invention is:

[0013] A resource joint optimization method, applied to the aforementioned distributed RIS-assisted ISAC system, includes:

[0014] Construct a distributed RIS-assisted ISAC system model in the mmWave band;

[0015] Based on the system model, mmWave channel modeling, ISAC system signal model, and communication and sensing model are constructed.

[0016] The optimization problem is determined based on the constructed model;

[0017] Based on the optimization problem, joint optimization is performed on RIS and user pairing, base station active beamforming and RIS passive beamforming to maximize the total throughput of the system while minimizing the beam gain of each sensing target.

[0018] Furthermore, the distributed RIS-assisted ISAC system model in the mmWave band includes:

[0019] K single-antenna communication users complete communication with the base station with the assistance of J RIS; each RIS with N reflective elements is modeled as a uniform planar array (UPA), and adjacent reflective elements of the RIS are spaced d apart. x and d y Separate horizontally and vertically; the phase shift matrix of the j-th RIS is set as Θ. j Furthermore, considering the constraints of actual hardware, discrete phase shifts are used to characterize the phase shift matrix, meaning that the phase shift of each reflection unit of the RIS can only take values ​​from a finite number of phase shifts.

[0020] The base station is equipped with M antennas forming a uniform linear array (ULA); in order to achieve integrated sensing, T sensing targets are directly sensed through the base station.

[0021] Define the sets of user terminals, RIS, and sensing targets as follows: and

[0022] Furthermore, the mmWave channel model is constructed as follows:

[0023] The mmWave channel is characterized by one line-of-sight (LoS) link and multiple non-line-of-sight (NLoS) links; the channel between the base station BS and the j-th RIS is represented by G. 0,j This indicates that the channel between the j-th RIS and the k-th user terminal uses h. j,k This indicates that all mmWave channels are modeled using the Saleh-Valenzuela channel model;

[0024] Furthermore, considering the passive nature of RIS, obtaining channel state information (CSI) becomes extremely difficult, introducing CSI error, and considering that only imperfect CSI can be obtained.

[0025] Furthermore, the ISAC system signal model is constructed as follows:

[0026] To simultaneously serve K communication users and T sensing targets, the base station needs to transmit superimposed NOMA and sensing signals, as shown in the following expression:

[0027] x = W c s c +W r s r =Ws

[0028] In the formula, W c Let W represent the active communication beamforming matrix of the base station. r This represents the active sensing beamforming matrix of the base station; s c It means, sr Let represent the transmission signals of K user terminals and T sensing targets; W and s represent the total active beamforming matrix and the total transmission signal, respectively.

[0029] Assume the k-th user terminal matches the μ-th... k If there are RIS, then the signal received by the k-th user terminal is represented as:

[0030]

[0031] In the formula, Represents the μth k The channel from the RIS to the k-th user, H represents the matrix conjugate transpose operation, w k,c s represents the communication beamforming vector of the k-th user. k,c w represents the communication signal that the k-th user expects to receive. d,c s represents the communication beamforming vector of the d-th user. d,c w represents the communication signal that the d-th user expects to receive. t,r Let s represent the sensing beamforming vector of the t-th target. t,r Let n represent the communication signal that the t-th target expects to receive. k This represents Gaussian white noise.

[0032] Furthermore, the communication and sensing model is constructed as follows:

[0033] Define the effective channel vector of the k-th user terminal as: Assuming communication users are ordered in descending order of channel gain; the k-th user terminal first detects and removes interference from weak users within the same RIS service, while treating the signals of strong users within the same RIS service and the signals of all other users within the same RIS service as interference; then the received signal-to-interference-plus-noise ratio (SINR) of the k-th user terminal is expressed as:

[0034]

[0035] In the formula, Indicates noise power;

[0036] For SIC decoding to proceed smoothly, the signal of the k-th user terminal needs to be decoded at the weak user n within the same RIS service, yielding the corresponding SINR:

[0037]

[0038] Then, based on the SIC principle, the achievable rate of user k is:

[0039]

[0040] Where R n→k =log2(1+SINR) n→k ) and R k→k =log2(1+SINR) k→k Let represent the data rate obtained by weak user n decoding user k and the data rate obtained by strong user n decoding its own target signal, respectively. Then, the total throughput of the system is obtained by summing the rates of all communication users:

[0041]

[0042] In terms of sensing, the active beamforming matrix W of the base station is used to sense the target; the angle θ received by the base station from the target is defined as... t The beam gain is:

[0043]

[0044] In the formula, α L (·) represents the array response of a uniform linear array.

[0045] Furthermore, the expression for the optimization problem is:

[0046]

[0047] In the optimization problem (P0), μ and Θ represent the set of paired variables between the RIS and the user, and the set of phase shift matrices of the RIS, respectively; P max R represents the maximum transmit power of the base station. k R represents the reachable rate of the k-th user. min This represents the minimum reachable rate constraint for each user. This indicates that the base station receives data from the target at an angle of θ. t beam gain, φ represents the minimum beam gain received by the base station for each sensed target. j,n This represents the phase shift of the nth reflecting unit of the j-th RIS. The set of possible values ​​for the discrete phase shift of the RIS reflector unit is represented; constraint C1 restricts each user to match only one RIS; C2 represents the maximum transmit power limit of the system; C3 guarantees the minimum rate for each communication user; C4 guarantees the minimum beam gain for each sensing target received by the base station; C5 indicates that the RIS phase shift under consideration is a discrete phase shift.

[0048] Furthermore, the joint optimization of RIS and user pairing, base station active beamforming, and RIS passive beamforming includes:

[0049] In the first stage, a graph neural network-based approach is used to optimize the pairing between RIS and users. Attention mechanisms and residual connection techniques are employed to better extract key graph information to obtain a better pairing strategy and effectively avoid gradient vanishing.

[0050] In the second stage, based on the pairing strategy obtained in the first stage, the active beamforming of the base station and the passive beamforming of the RIS are jointly optimized using the MADDPG method based on multi-agent deep reinforcement learning. In this stage, the base station and each RIS are constructed as an independent agent, and each agent consists of an actor network, a critic network, a target actor network, and a target critic network.

[0051] Furthermore, during training, each agent inputs the observed state into the actor network to obtain the action to be performed. Then, the environment evaluates and updates the critic network and actor network based on the actions of each agent. Finally, soft updates are performed on the target actor network and the target critic network.

[0052] The beneficial effects of this invention are: This invention proposes a system framework that combines distributed RIS, mmWave, NOMA and ISAC technologies, and performs two-stage optimization based on machine learning on the optimization problems of RIS and user pairing, base station active beamforming and RIS passive beamforming, which effectively improves the total throughput of the system and can better achieve a balance between communication and sensing performance. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following description is provided with accompanying drawings of the relevant technical solutions in the embodiments of the present invention or the prior art. It should be understood that the accompanying drawings described below are only for the purpose of clearly illustrating some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0054] Figure 1 This is a schematic diagram of the structure (system transmission model) of the ISAC system in the mmWave band with distributed RIS assistance in an embodiment of the present invention;

[0055] Figure 2 A schematic diagram illustrating how the total throughput of a distributed RIS-assisted mmWave band ISAC system, considered in an embodiment of the present invention, is affected by the number of base station antennas under different schemes.

[0056] Figure 3A schematic diagram illustrating the impact of the base station's maximum transmit power on the total throughput of a distributed RIS-assisted mmWave band ISAC system, considered in an embodiment of the present invention, under different schemes;

[0057] Figure 4 A schematic diagram illustrating the impact of different schemes on the total throughput of the system, the number of sensing targets, and the minimum beam gain of the sensing targets in a distributed RIS-assisted mmWave band ISAC system considered in an embodiment of the present invention.

[0058] Figure 5 This diagram illustrates the beam gain at various angles under different schemes in a distributed RIS-assisted mmWave band ISAC system considered for embodiments of the present invention. Detailed Implementation

[0059] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application. The step numbers in the following embodiments are set only for ease of explanation, and there is no limitation on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0060] The terminology used in the embodiments of this application is for the purpose of describing specific embodiments only and is not intended to limit the embodiments of this application. The singular forms "a," "described," and "the" used in the embodiments of this application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. Furthermore, unless otherwise expressly limited, terms such as "set," "install," and "connect" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this invention in conjunction with the specific content of the technical solution.

[0061] In the description of this application, it should be understood that the orientation descriptions, such as up, down, front, back, left, right, etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0062] In the description of this application, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0063] In the description of this application, "and / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the related objects before and after it are in an "or" relationship.

[0064] Terminology Explanation:

[0065] ISAC: An abbreviation for Integrated Sensing and Communication.

[0066] RIS: An abbreviation for Reconfigurable Intelligence Surface.

[0067] Due to the limited availability of wireless spectrum resources and the increasingly urgent need for sensing capabilities within wireless communication networks, integrated mmWave communication and sensing systems have been extensively studied. To address the issue of mmWave signals being easily blocked by buildings, academia has also conducted extensive research on distributed RIS-assisted millimeter-wave ISAC communication systems. However, existing technical solutions have the following drawbacks:

[0068] 1) Current research on ISAC systems in the mmWave band mostly focuses on the application of NOMA technology in mmWave ISAC systems, or the application of RIS to mmWave-ISAC systems. However, the combination of ISAC systems and NOMA technology in the mmWave band faces the challenge of mmWave signals being easily blocked and having a small coverage area, while RIS-assisted ISAC systems in the mmWave band are limited by traditional OMA technology, resulting in limited spectrum utilization efficiency.

[0069] 2) Current research on RIS-assisted mmWave-ISAC systems mostly focuses on single RIS-assisted mmWave-ISAC systems. However, in practical applications, mmWave signals face the challenge of limited coverage due to obstructions such as buildings. This makes it impossible for a single RIS to effectively cover all communication users and sensing targets, resulting in significant performance losses.

[0070] 3) Current performance optimization research on distributed RIS-assisted mmWave-ISAC systems mainly focuses on optimizing active and passive beamforming, with little attention paid to optimizing the pairing between RIS and users. However, in practical applications where multiple RIS assist multiple users in communication and sensing, the number of users each RIS can serve is limited, and each user's channel conditions differ. Therefore, it is necessary to match the optimal RIS for each user based on the actual situation to maximize the system's total throughput.

[0071] 4) Current research on performance optimization of distributed RIS-assisted mmWave-ISAC systems mainly focuses on iterative optimization using traditional algorithms, such as Alternating Iterative (AO) and BCD methods. These traditional methods have acceptable complexity when applied to RIS systems with a small number of users. However, in real-world applications, systems often involve a large number of users and sensing targets, requiring the deployment of large-scale distributed RIS systems to assist communication and sensing. Traditional algorithms face enormous computational complexity when performing multi-parameter joint optimization in such systems, which is highly detrimental to achieving real-time communication and sensing.

[0072] Considering the shortcomings of single RIS in practical applications, and to address the limited propagation range of ISAC systems in the mmWave band assisted by a single RIS, this invention proposes a framework combining distributed RIS, NOMA, mmWave communication, and ISAC technology, and investigates the application of machine learning methods within it. In this framework, communication users achieve high-quality communication with the base station through the assistance of multiple RIS, while the sensing target is directly detected through the base station. Based on this framework, this invention also proposes an efficient multi-dimensional resource allocation method based on machine learning, realizing the pairing of RIS and users, and the joint optimization of active beamforming and passive beamforming. This maximizes the total throughput of communication users in the system while ensuring the minimum sensing performance of the sensing target.

[0073] Specifically, this invention considers a distributed RIS-assisted ISAC system in the mmWave band, where distributed RIS assists communication users in mmWave communication. Each user can only be served by one RIS, and each RIS can serve users in one NOMA group. In each NOMA group served by a RIS, the target information of each user is decoded using SIC technology. The sensed target is directly detected by the base station, and the sensed beam gain is used as an indicator of sensed performance. To maximize the total throughput of the system while ensuring the minimum beam gain of each sensed target, this invention also proposes a multi-dimensional resource joint optimization method based on machine learning for RIS and user pairing, base station active beamforming, and RIS passive beamforming. Specifically, this joint optimization method can be divided into two stages. In the first stage, this invention proposes a graph neural network-based method to optimize the pairing between RIS and users, and adopts advanced attention mechanisms and residual connection technology, enabling the method to better extract graph key information to obtain a better pairing strategy and effectively avoid gradient vanishing. Based on the pairing strategy obtained in the first stage, this invention proposes a joint optimization method for active beamforming of the base station and passive beamforming of the RIS (Resource Identifier) ​​in the second stage, using a multi-agent deep reinforcement learning (MADDPG) approach. In this method, the base station and each RIS are constructed as an independent agent, each consisting of an actor network, a critic network, a target actor network, and a target critic network. During training, each agent inputs the observed state into the actor network to obtain the action to be performed. The environment then evaluates and updates the critic and actor networks based on each agent's actions, and finally, soft updates are performed on the target actor and target critic networks. Simulation results show that the proposed two-stage multi-dimensional resource joint optimization method based on machine learning significantly improves the performance of RIS-user pairing and base station and RIS beamforming in an ISAC system with distributed RIS assistance in the mmWave band.

[0074] The technical solution of the present invention will be explained in detail below with reference to the accompanying drawings and specific embodiments.

[0075] (1) System Model

[0076] See Figure 1 In the distributed RIS-assisted mmWave band ISAC system model considered in this embodiment, K single-antenna communication users complete communication with the base station with the assistance of J RIS. Each user has N=N x N y A Reflective Array (RIS) with multiple reflective elements can be modeled as a uniform planar array (UPA), with adjacent reflective elements of the RIS spaced at intervals of d.x and d y Separate horizontally and vertically. The phase shift matrix of the j-th RIS is set as... Furthermore, considering the constraints of actual hardware, discrete phase shifts are used to characterize the phase shift matrix. That is, the phase shift of each reflection unit in the RIS can only take values ​​from a finite number of phase shifts, i.e.:

[0077]

[0078] The base station is equipped with M antennas forming a uniform linear array (ULA). To achieve integrated sensing, T sensing targets are directly sensed through the base station. For ease of representation, this embodiment defines the sets of users, RIS, and sensing targets as follows: and To better reflect real-world application scenarios, the system considers K>J, meaning the number of users is greater than the number of RIS. This further illustrates that the pairing of RIS and users is very important and meaningful.

[0079] (2) mmWave channel modeling

[0080] An mmWave channel can be characterized by one line-of-sight (LoS) link and multiple non-line-of-sight (NLoS) links. The channel between the base station (BS) and the j-th RIS is represented by G. 0,j This indicates that the channel between j-th RIS and d-th users is represented by h. j,k This indicates that all mmWave channels are modeled using the Saleh-Valenzuela channel model as follows:

[0081]

[0082]

[0083] Among them, L G and L h Representing G respectively 0,j and h j,k The number of paths. τ l , and G 0,j The complex Gaussian gain of the l-th path, the azimuth angle of arrival, the elevation angle of arrival, and the elevation angle of departure, ε l , and They represent h respectively j,k The complex Gaussian gain of the l-th path, the azimuth angle, and the elevation angle. Where:

[0084]

[0085]

[0086] Where λ represents the wavelength of the millimeter-wave signal, d represents the spacing between base station antennas, and adjacent reflective elements of the RIS are spaced by d... x and d y Separate horizontally and vertically.

[0087] Furthermore, considering the passive nature of RIS, obtaining Channel State Information (CSI) becomes extremely difficult, so we introduce CSI error. Therefore, this embodiment considers that only imperfect CSI can be obtained, i.e.:

[0088]

[0089] h j,k =h j,k +Δh j,k (7)

[0090] Wherein, ΔG 0,j and Δh j,k These represent the errors generated when acquiring CSI. and h j,k This indicates an imperfect CSI obtained.

[0091] (3) Signal model of ISAC system in mmWave band with distributed RIS assistance

[0092] To simultaneously serve K communication users and T sensing targets, the base station needs to transmit superimposed NOMA and sensing signals, as follows:

[0093] x = W c s c +W r s r =Ws, (8)

[0094] in, This represents the active communication beamforming matrix of the base station. This represents the active sensing beamforming matrix of the base station. and Let W represent the active beamforming vectors for the k-th user and the t-th sensing target, respectively. c W r ] and s = [s c ,s r ] T These represent the total active beamforming matrix and the total transmitted signal, respectively. and Let represent the transmitted signals of K communication users and T sensing targets, respectively. Furthermore, for the convenience of subsequent formulas, we assume that the k-th users are matched with μ. k -th RIS, where The signal received by the k-th user can then be represented as:

[0095]

[0096] In the above formula, the first part represents the target desired signal received by the k-th user, the second and third parts represent the communication interference signal and the perception interference signal received by the user, respectively, and the fourth part represents the received additive white Gaussian noise (AWGN). σ 2 This represents noise power.

[0097] (4) Communication and Sensing Model

[0098] For ease of representation, the effective channel vector for a k-th user is defined as follows: According to NOMA theory, within each NOMA packet, strong users decode weak users' signals via SIC before decoding their own target signal. Without loss of generality, we assume that communication users are ordered in descending order of channel gain, i.e. Furthermore, in order to eliminate interference, it is assumed that the sensed signal is prior knowledge for the communication user, that is, the communication user will not be interfered with by the sensed signal.

[0099] Therefore, k-th communication users first detect and remove interference from weak users within the same RIS service, while treating signals from strong users within the same RIS service, as well as signals from all other RIS service users, as interference. The received signal-to-noise ratio (SINR) for a k-th user can then be expressed as:

[0100]

[0101] Furthermore, in order for SIC decoding to proceed smoothly, the signals of k-th users need to be decoded at the weak user n of the same RIS service, i.e. The corresponding SINR is obtained as follows:

[0102]

[0103] Then, based on the SIC principle, the achievable rate of user k is:

[0104]

[0105] Where R n→k =log2(1+SINR) n→k ) and R k→k =log2(1+SINR) k→kLet represent the data rate obtained by weak user n decoding user k and the data rate obtained by strong user n decoding its own target signal, respectively. The total throughput of the system can then be obtained by summing the rates of all communicating users, i.e.:

[0106]

[0107] In terms of perception, the active beamforming matrix of the base station It is used to sense targets. The angle θ received by the base station from the sensed target is defined as... t The beam gain is:

[0108]

[0109] (5) The problem of maximizing the total system throughput

[0110] Having completed the above definition, this invention aims to maximize total throughput while satisfying the minimum sensing performance of each sensing target by jointly optimizing the pairing relationship between RIS and users, the base station active beamforming matrix, and the RIS passive beamforming. The mathematical expression of the optimization problem is:

[0111]

[0112] In the optimization problem (P0), and Let C1 represent the set of pairing variables between the RIS and the user, and C2 represent the set of phase shift matrices of the RIS, respectively. Constraint C1 restricts each user to be paired with only one RIS. C2 represents the maximum transmit power limit of the system. C3 guarantees the minimum rate for each communication user. C4 guarantees the minimum beam gain received by the base station for each sensed target. C5 indicates that the phase shift of the RIS under consideration is a discrete phase shift.

[0113] (6) Two-stage resource optimization method

[0114] For the aforementioned optimization problem, constraints C2, C3, C4, and C5 are all non-convex constraints. Furthermore, the phase shift matrices of the J distributed RIS are highly correlated, and the optimization parameters μ, W, and Θ are interconnected and highly coupled, posing a challenge to optimizing the problem. In summary, the optimization problem (P1) is a highly complex non-convex optimization problem. Traditional alternating iterative methods require enormous computational resources to solve such problems, and the optimization results are often not optimal. To more effectively optimize this multi-parameter, multi-dimensional resource allocation problem, this invention proposes a two-stage resource optimization method based on machine learning. Specifically, in stage one, a method based on a graph neural network algorithm is proposed, and an advanced graph attention network framework is adopted for RIS and user pairing. In the second stage, based on the RIS and user pairing results obtained in stage one, a deep reinforcement learning-based method is used to optimize the design of base station active beamforming W and RIS passive beamforming Θ. Specifically:

[0115] 5.1) Phase 1: Optimizing the pairing of RIS and users

[0116] Based on graph theory, we first define the base station, RIS, and communication users as nodes in a graph, represented as follows: In this system, node 0 represents the base station, nodes j∈{1,2,…,J} represent J distributed RIS (Real-Integrated Service Providers), and nodes k+J∈{1+J,…,K+J} represent K communication users. It's important to note that with this definition, node j represents the j-th RIS, and node k+J represents the k-th user. In practical deployment scenarios, a user may be located in the service areas of multiple RISs, so matching each user with an optimal RIS is essential. For ease of description, we introduce χ². j,k+J ∈{0,1} to describe the possible pairing relationships between RIS and users, where χ j,k+J =1 indicates a possible pairing relationship between j-th RIS and k-th users, χ j,k+J =0 indicates that j-th RIS and k-th users cannot be paired. Furthermore, node characteristics are defined. These are the three-dimensional coordinates of the corresponding node.

[0117] Based on the above definitions, and given the base station active beamforming and RIS passive beamforming, the optimization problem (P0) is transformed into:

[0118]

[0119] Where the objective function The beamforming matrix W of the base station and the passive beamforming matrix Θ of the RIS are calculated by substituting them into formula (13). Constraint C6 ensures that pairing can only occur between RIS and users where pairing is possible.

[0120] Note that, due to the calculation of the objective function This involves calculating NOMA inter-group interference and NOMA intra-group interference. Since the calculation of these two parts is closely related to the pairing strategy, and prior information cannot be obtained before determining the pairing strategy, this invention considers calculating the objective function... For now, we can ignore the effects of interference. Therefore, the problem of maximizing the total system throughput can be equivalently transformed into the problem of maximizing the total effective channel gain of the system, i.e., maximizing:

[0121]

[0122] Based on the given active beamforming matrix W of the base station and the passive beamforming Θ of the RIS, the above expression depends only on the channel between the base station and the RIS. and the channel between RIS and the user Therefore, in order to maximize the total effective channel gain of the system, we introduce the edge feature ε between the base station and the RIS in the graph definition. 0,j The edge feature ε between RIS and the user j,k+J This corresponds to the channel gain, i.e.:

[0123]

[0124] In summary, the defined graph structure includes nodes, node features, edges, and edge features, and can be represented as follows: in, The set representing the edges in the graph is denoted as:

[0125]

[0126]

[0127] also, Based on node features f n and edge features ε a,b Composition. After completing the above definition, the pairing relationship between RIS and users can be solved by a graph neural network (GNN) method that employs an attention mechanism. Specifically, the graph neural network model proposed in this embodiment includes an encoder and a decoder. The encoder's role is to extract and integrate node features and edge features, while the decoder's role is to calculate the probability distribution of pairings between RIS and users, thereby selecting an optimal RIS for each user.

[0128] Encoder Design: The encoder proposed in this embodiment adopts a residual-connected graph attention network (RE-GAT) structure. Its function is to integrate edge features into node features. The advantage is that it can more effectively extract and integrate key information from the graph structure to obtain a superior pairing scheme, while also avoiding the gradient vanishing problem. The encoder will then integrate the graph as defined above... As input, node features and edge features are first processed through a fully connected layer and a batch normalization layer, as follows:

[0129]

[0130]

[0131] Where Bn(·) and Fc(·) represent the batch normalization layer and the fully connected layer, respectively. After the above processing, the node features... Sum of edge features The updated node features are input into the first layer of the RE-GAT network and output as updated node features. The edge features remain unchanged. Then, and The data is input into the second layer of the RE-GAT network, and so on. After processing through L layers of the RE-GAT network, the node features are updated, and the output f is obtained. (L) .

[0132] To describe the above process more specifically, we will use the l-th layer RE-GAT network as an example. First, to describe the importance weights between node a and the connected node b, we introduce attention coefficients calculated as follows:

[0133]

[0134] Where z l and Let represent the learnable weight vector and matrix of the l-th layer network, respectively. δ(·) represents the LeakyReLU activation layer in the RE-GAT network, and (·||·||·) represents the concatenation operation. Furthermore, residual connections are used between adjacent layers to aggregate the updated node features and the input node features, resulting in the following output node features:

[0135]

[0136] in This represents the updated node features. After L RE-GAT layers complete the above update and aggregation process, the final output node features are obtained. This allows us to obtain the final graph embedding features. in:

[0137]

[0138] Decoder Design: Based on the graph features extracted and integrated by the encoder, the decoder also employs an attention mechanism, including an H-head attention layer and a single-head attention layer. The H-head attention layer calculates the context vector based on the graph features, while the single-head attention layer generates a probability distribution for pairing the user with the RIS, thereby helping the user select the optimal RIS. Taking the optimal RIS matching user node k+J as an example, its corresponding context vector is obtained by the following formula:

[0139]

[0140] in This represents the learnable weight matrix. After inputting the aforementioned context vector into the H-head attention layer, the output is... This invention employs the (q,k,v) attention mechanism from the Transformer model. Specifically, These represent the query vector, key vector, and value vector, respectively. This represents the corresponding learnable weight matrix. During the algorithm execution, the query vector q and the key vector k are used to calculate the attention coefficients between RIS node i and user node k+J, as shown below:

[0141]

[0142] Where d t This represents the dimensions of the query vector, key vector, and value vector. Subsequently, this invention applies the softmax activation function to normalize the attention coefficients calculated above.

[0143] The above calculation operation is repeated in each head of the H-head attention layer. For ease of representation, we define... These are the normalized attention coefficients calculated in the h-th head attention layer. Through a series of concatenation operations, the updated context vector can be obtained.

[0144]

[0145] After obtaining the updated context vector, it is input into the single-head attention layer of the decoder, and the pairing probability of each user with each RIS is calculated using the tangent activation function as follows:

[0146]

[0147] Based on the above probabilities, this embodiment uses the softmax function to calculate the pairing probability distribution between user node k+J and all RIS, that is:

[0148]

[0149] After obtaining the pairing probability distribution of each user for each RIS, this embodiment also employs a sampling strategy to determine the optimal RIS for each user node k+J. Based on the encoder and decoder structures defined above, the detailed graph neural network training process will be given next.

[0150] This embodiment employs the REINFORCE algorithm for training the graph neural network and applies a baseline rollback strategy to reduce the estimation variance of the policy gradient. The REINFORCE algorithm, based on the traditional actor-critic network, replaces the critic network with a baseline actor network, forming a dual-actor network structure. This structure improves both the stability and convergence speed of the algorithm. In each training epoch, the trainable parameters of the baseline actor network remain constant, while the trainable parameters of the main actor network are continuously updated in real time. After each epoch of training, a greedy decoding algorithm is used to evaluate the training effect. Then, a t-test is used to determine whether to update the baseline policy; that is, the baseline policy is only updated to the current policy if the current policy brings a sufficiently significant improvement. This improves both training stability and convergence speed.

[0151] First, to maximize the total effective channel gain of the system, we define:

[0152]

[0153] Then, P is introduced. ξ Let (μ) represent the probability of adopting the pairing strategy μ, which is calculated as follows:

[0154]

[0155] Where ξ represents the learnable parameters of the graph neural network, p ξ (i,k+J) represents the pairing probability distribution between the decoder output RIS and the user. Based on the above definition, the loss function during training is further defined as follows: During training, the learnable parameter ξ is continuously updated through gradient descent of the policy of rolling back the baseline Γ, that is:

[0156]

[0157] The rollback baseline Γ is evaluated and updated by the baseline actor network. During training, the system state represents the currently established pairings of RIS and users, the action space is the process by which each user selects the optimal RIS, and the reward function is obtained by quantifying the total effective channel gain of the system.

[0158] The GNN network proposed in this embodiment, after effective training, can select an optimal RIS for each user to maximize the total channel gain of the system, thereby improving system performance and obtaining an approximately optimal pairing scheme in the first stage.

[0159] 5.2) Phase Two: Jointly Optimize Base Station Active Beamforming and RIS Passive Beamforming Using Deep Reinforcement Learning Algorithms

[0160] Based on the near-optimal pairing scheme of RIS and users obtained in Phase 1, the overall optimization problem can be further transformed into jointly optimizing base station active beamforming and RIS passive beamforming to maximize the total system throughput under various constraints, i.e.:

[0161]

[0162] To address the aforementioned optimization problem, this embodiment proposes a joint optimization method for active base station beamforming and passive RIS beamforming based on the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm. Leveraging its powerful exploration and learning capabilities, the MADDPG algorithm can obtain near-optimal active base station beamforming and passive RIS beamforming designs through extensive training.

[0163] MADDPG Framework: In this embodiment, the base station and J RISs are modeled as DDPG agents. Each agent learns how to specify a better policy to maximize the overall system reward. Specifically, each agent can perform two functions: first, implement actions through an actor network; and second, evaluate and update policies through a critic network. This allows multiple agents to perform their respective policy selection and network evaluation in parallel. This framework promotes collaborative design of effective active and passive beamforming between the base station agent and the RIS agents. During training, this embodiment employs a centralized training, distributed execution (CTDE) framework, which improves the training stability of MADDPG and enhances the training efficiency of the agents.

[0164] Based on the above analysis, this invention defines the base station intelligent agent as Ξ0, and the RIS intelligent agent as... For each intelligent agent All are composed of an actor network A target actor network A critic network and a target critic network Composition, in which and These represent the corresponding learnable parameters of the network, s i and a m , Representing intelligent agents Ξ i The states and actions of each agent, S = {s0, s1, ..., s...} J} represents the global state of the system.

[0165] In addition, this embodiment defines an experience playback buffer. This is used to store the experience gained by each agent during training. Since the goal of the optimization problem is to maximize the total system throughput, each agent collaborates to design its own beamforming to maximize a common reward function r, which is the total system throughput.

[0166] Algorithm flow: At the nth time step, each agent Ξ i First, observe your own state. Then, the agent's actor network uses the observed states to determine the actions the agent should take, i.e.:

[0167]

[0168] After each agent determines its own action based on the above formula, the entire system executes a global action. Receive the current reward r (n) and the global state S after the action is performed. (n+1) Subsequently, the experience of each agent {S} (n) ,S (n+1) ,a (n) ,r (n) All of these will be stored in the experience replay buffer. This provides information for subsequent training. During the experience replay buffer... Once sufficient samples are available, at each training time step, a mini-batch of samples will be randomly sampled for subsequent policy evaluation and necessary updates.

[0169] First, when updating the network parameters, the objective Q function is updated as follows:

[0170]

[0171] Where γ represents the discount factor. This represents the action obtained through the target actor network. Based on the above formula, the critic network is updated by minimizing the loss function, which is defined as follows:

[0172]

[0173] After updating the critic network for each agent, the actor network for each agent also needs to be updated. The actor network update uses the policy gradient sampling method, with the goal of maximizing the expected reward, as shown below:

[0174]

[0175] Subsequently, at the end of each time step, this embodiment uses a soft update method to update the target actor network and the target critic network, as follows:

[0176]

[0177]

[0178] in and These represent the soft update learning rates of the target actor network and the target critic network, respectively.

[0179] Definitions of the agent, state, action, and reward function:

[0180] Base station active beamforming agent Ξ0: This agent is responsible for optimizing the active beamforming matrix W of the base station. At the nth time step, the observation state of Ξ0 is determined by the RIS obtained in the first stage and the user's pairing strategy μ. * The system's channel state information is composed of the following:

[0181]

[0182] The actions determined by the actor network of agent Ξ0 are composed of the perception beamforming matrix and the communication beamforming matrix, that is:

[0183]

[0184] RIS Passive Beamforming Agent At time step n, the state observed by each RIS agent is:

[0185]

[0186] The actions obtained by applying the actor network are composed of the phase shift matrices corresponding to each RIS, that is:

[0187]

[0188] reward function r (n) Since each agent has the same goal during training—namely, maximizing the total system throughput—the reward function set in this invention is the total system throughput, i.e.:

[0189] r (n) =R sum (45)

[0190] (6) Simulation Experiment Results

[0191] In this embodiment, simulation analysis verifies the effectiveness of the proposed two-stage resource optimization method based on machine learning in improving the overall system throughput. The simulation was implemented using the PyTorch platform in Python, and the superiority of the proposed optimization scheme is further illustrated by comparing its performance with other comparative schemes.

[0192] The simulation experiment parameters are set as follows: The mmWave simulation channel is configured with one Loss path and five NLoS paths. Maximum transmit power P... max Set to 30dBm. Noise power is set to σ. 2 = -90dBm. The number of base station antennas is M=16, and the number of each RIS reflection unit is N=16. Furthermore, the carrier frequency involved in this embodiment is set to 30GHz. The number of layers L in the RE-GAT in the graph neural network is set to 3. The number of heads H of the attention layer in the decoder of the RE-GAT is set to 8. The number of training rounds of MADDPG is set to 100, and the training time step per round is set to 50000. The capacity of the experience replay buffer is set to 10000, the mini-batch sampling size is set to 128, and the discount factor γ is set to 0.99. The learning rate of the actor network updating the agent is set to 1e. -4 Update the learning rate of the critic network to 1e. -3 The learning rate for both the target actor network and the target critic network is soft-updated and set to 1e. -4 .

[0193] Comparison scheme settings:

[0194] Option 1: In the first stage, the graph neural network method proposed in this invention is used to complete the pairing between RIS and users. In the second stage, the traditional single-agent DDPG method is used to optimize the active beamforming of the base station and the passive beamforming of the RIS.

[0195] Option 2: In the first stage, the graph neural network method proposed in this invention is used to complete the pairing between RIS and users. In the second stage, the traditional BCD method is used to iteratively optimize the active beamforming of the base station and the passive beamforming of the RIS.

[0196] Option 3: In the first stage, the classic pairing algorithm, the Gael-Shapley algorithm, is used to pair RIS and users. In the second stage, the MADDPG method proposed in this invention is used to optimize active beamforming of the base station and passive beamforming of the RIS.

[0197] Option 4: In the first stage, a random pairing method is used to randomly generate pairings between RIS and users. In the second stage, the MADDPG method proposed in this embodiment is used to optimize active beamforming of the base station and passive beamforming of the RIS.

[0198] Figure 2 This paper demonstrates how the total system throughput is affected by the number of base station antennas when using the proposed machine learning-based two-stage resource optimization method in an ISAC system with distributed RIS-assisted mmWave band, as considered in this embodiment. Four different comparison schemes are set up for comparison with the proposed scheme. Simulation results show that the proposed scheme consistently outperforms the other comparison schemes in optimization performance, and this superiority becomes increasingly significant as the number of base station antennas increases. This is because the proposed MADDPG method in this embodiment demonstrates its advantage in flexibly controlling the states and actions of various agents when optimizing complex beamforming problems involving base stations and multiple RISs. Compared to the traditional DDPG and BCD methods, the proposed MADDPG method, due to its centralized learning and distributed execution framework, exhibits more stable training performance and faster convergence. Furthermore, the proposed GNN method also shows superior performance in optimizing pairing between RIS and users compared to the traditional Gell-Shapley algorithm and the baseline random pairing method. This is mainly attributed to the large-scale training of the graph neural network, which effectively extracts key pairing-related information from the system, better serving the pairing process and thus yielding superior pairing results. Simulation results further illustrate that the optimized pairing of RIS and users has a significant impact on the overall system throughput.

[0199] Figure 3 This demonstrates that in an ISAC system considering distributed RIS-assisted mmWave band, the total system throughput is affected by the base station's maximum transmit power. A comparison is made between the configuration of the proposed scheme and... Figure 2The results are similar. As the curves in the figure show, the total system throughput of all schemes increases with the increase of the maximum transmission power of the base station. The proposed scheme still achieves the highest total system throughput compared to all other schemes. Furthermore, when the maximum transmission power of the base station increases to a relatively high level, the performance gap between the proposed scheme and schemes 3 and 4 gradually widens. This further demonstrates that the proposed graph neural network method significantly outperforms the Gell-Shapley method and the random pairing method in optimizing the pairing between RIS and users. It also proves that a better pairing scheme is helpful for the subsequent design of active beamforming for base stations and passive beamforming for RIS based on the MADDPG algorithm.

[0200] Figure 4 This paper demonstrates the impact of the number of sensing targets and the minimum beam gain of the sensing targets on the total system throughput in an ISAC system under the consideration of distributed RIS-assisted mmWave band. As shown in the figure, the total system throughput of all schemes decreases with the increase of the number of sensing targets. This is because the increased number of sensing targets requires the system to allocate more resources to meet the minimum sensing performance of each sensing target. Among all schemes, the proposed scheme achieves the optimal total system throughput under different numbers of sensing targets and different minimum sensing beam gains, further illustrating the superiority of the proposed scheme. With the increase of the minimum sensing beam gain, the total system throughput of all schemes shows a decreasing trend, mainly because the sensing performance requirements of the sensing targets increase, requiring the system to allocate more resources to sensing, resulting in a decrease in communication performance. The proposed scheme maintains a significant performance gap with the comparative schemes, further demonstrating that the two-stage resource optimization scheme proposed in this invention can better balance sensing and communication in the ISAC system, thereby achieving better overall system performance.

[0201] Figure 5 This paper demonstrates the sensing performance of an ISAC system in the mmWave band under a distributed RIS-assisted architecture. The simulation included three sensing targets located at angles of -45°, 0°, and 45°, centered on a base station. Sensing performance was described by the beam gain at each angle. The figures show that, compared to Scheme 1, the proposed scheme achieves higher beam gain at the angles of the sensing targets and lower beam gain at other non-sensing target angles. This indicates that the proposed scheme achieves better sensing performance and further demonstrates that the scheme in this embodiment better balances sensing and communication, resulting in superior system performance.

[0202] (7) Beneficial effects

[0203] Current research on RIS-assisted ISAC systems in the mmWave band largely focuses on application scenarios of ISAC systems with single RIS assistance in the mmWave band, with relatively little research on distributed RIS. In system performance optimization research, existing studies mainly focus on optimizing active beamforming at the base station and passive beamforming at the RIS, with little optimization of the pairing between RIS and users. Furthermore, in ISAC system performance optimization research, existing studies mainly employ traditional optimization methods, such as alternating iterative methods and block descent methods, with few studies considering more advanced machine learning-based optimization methods. Considering that existing research on the joint optimization of RIS and user pairing and active / passive beamforming in distributed RIS-assisted mmWave band ISAC systems is insufficient, this invention proposes a distributed RIS-assisted mmWave band ISAC system, performing a two-stage optimization based on machine learning for the optimization problems involving RIS and user pairing, active beamforming at the base station, and passive beamforming at the RIS. This invention has the following main advantages:

[0204] 1) This invention considers a system framework combining distributed RIS, mmWave, NOMA, and ISAC technologies. Introducing RIS into the ISAC system in the mmWave band can solve the problems of millimeter-wave signals being easily blocked and having insufficient coverage; introducing NOMA technology into the RIS-assisted mmWave system can effectively improve the utilization rate of spectrum resources; at the same time, the application of ISAC technology enables the system to simultaneously realize sensing and communication functions, which is very much in line with the growing communication and sensing needs in real life.

[0205] 2) In practical applications, since each RIS can only serve a limited number of users, introducing a distributed RIS can effectively increase the coverage of mmWave signals, providing high-quality services to more users and resulting in higher overall system throughput. However, the introduction of a distributed RIS also introduces a new pairing problem between RIS and users, and this invention investigates the optimization of this problem.

[0206] 3) Based on the proposed system, in order to maximize the total system throughput while ensuring the minimum sensing beam gain of the sensing target, this invention proposes a two-stage multi-dimensional resource joint optimization method based on machine learning to solve the pairing between RIS and users, base station active beamforming, and RIS passive beamforming. Simulation results demonstrate that compared with other comparative schemes, the proposed scheme can more effectively improve the total system throughput and achieve a better balance between communication and sensing performance.

[0207] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0208] The above embodiments are merely illustrative of the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made based on the essence of the content of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A distributed RIS-assisted ISAC system, characterized in that, This includes base stations, multiple distributed RIS systems, and multiple user terminals; Among them, the distributed RIS assists user terminals in mmWave communication. Each user terminal can only be served by one RIS, and each RIS can serve user terminals in one NOMA group. In each NOMA group served by the RIS, the target information of each user is decoded through SIC technology. The sensed target is directly detected by the base station, and the sensed beam gain is used as the indicator of sensed performance. Joint optimization of RIS and user pairing, base station active beamforming and RIS passive beamforming is performed to maximize the total system throughput while minimizing the beam gain for each sensing target. The joint optimization process includes two stages: In the first stage, a graph neural network-based approach is used to optimize the pairing between RIS and users, and attention mechanisms and residual connection techniques are employed to extract key graph information to obtain a better pairing strategy. In the second stage, based on the pairing strategy obtained in the first stage, the active beamforming of the base station and the passive beamforming of the RIS are jointly optimized using the MADDPG method based on multi-agent deep reinforcement learning. In this stage, the base station and each RIS are constructed as an independent agent, and each agent consists of an actor network, a critic network, a target actor network, and a target critic network.

2. A resource joint optimization method, applied to the distributed RIS-assisted ISAC system as described in claim 1, characterized in that, include: Construct a distributed RIS-assisted ISAC system model in the mmWave band; Based on the system model, mmWave channel modeling, ISAC system signal model, and communication and sensing model are constructed. The optimization problem is determined based on the constructed model; Based on the optimization problem, joint optimization is performed on RIS and user pairing, base station active beamforming and RIS passive beamforming to maximize the total system throughput while minimizing the beam gain of each sensing target. The joint optimization of RIS and user pairing, base station active beamforming, and RIS passive beamforming includes: In the first stage, a graph neural network-based approach is used to optimize the pairing between RIS and users, and attention mechanisms and residual connection techniques are employed to extract key graph information to obtain a better pairing strategy. In the second stage, based on the pairing strategy obtained in the first stage, the active beamforming of the base station and the passive beamforming of the RIS are jointly optimized using the MADDPG method based on multi-agent deep reinforcement learning. In this stage, the base station and each RIS are constructed as an independent agent, and each agent consists of an actor network, a critic network, a target actor network, and a target critic network.

3. The resource joint optimization method according to claim 2, characterized in that, The distributed RIS-assisted ISAC system model in the mmWave band includes: K A single-antenna communication user J Communication between the system and the base station is completed with the assistance of RIS; each of them carries N The RIS of reflective elements is modeled as a uniform planar array, and adjacent reflective elements of the RIS are respectively spaced at... and Separated by horizontal and vertical intervals; the first j The phase shift matrix of each RIS is set as follows Furthermore, considering the constraints of actual hardware, discrete phase shifts are used to characterize the phase shift matrix, meaning that the phase shift of each reflection unit of the RIS can only take values ​​from a finite number of phase shifts. The base station is equipped with M The antennas form a uniform linear array; in order to achieve integrated sensing and communication. T Each sensing target is directly sensed through the base station; Define the sets of user terminals, RIS, and sensing targets as follows: , and .

4. The resource joint optimization method according to claim 2, characterized in that, The mmWave channel model is constructed as follows: A mmWave channel is characterized by one line-of-sight link and multiple non-line-of-sight links; the base station and the... j Channel between RIS It indicates, and the first j RIS and k Channel between user terminals This indicates that all mmWave channels are modeled using the Saleh-Valenzuela channel model; Furthermore, considering the passive nature of RIS, obtaining channel state information (CSI) becomes extremely difficult, introducing CSI error, and considering that only imperfect CSI can be obtained.

5. The resource joint optimization method according to claim 2, characterized in that, The ISAC system signal model is constructed as follows: In order to serve simultaneously K individual communication users and T For a given sensing target, the base station needs to transmit superimposed NOMA and sensing signals, as shown in the following expression: In the formula, This represents the active communication beamforming matrix of the base station. This represents the active sensing beamforming matrix of the base station; express, Indicate K individual user terminals and T The transmission signal of a sensing target; These represent the total active beamforming matrix and the total transmitted signal, respectively. Assume the first k The user terminal matched the first The first RIS, then the second k The signal received by a user terminal is represented as follows: In the formula, Indicates the first RIS to the first k Channels for individual users This represents the conjugate transpose operation on a matrix. Indicates the first k Communication beamforming vector for each user Indicates the first k The communication signals that a user expects to receive. Indicates the first d Communication beamforming vector for each user Indicates the first d The communication signals that a user expects to receive. Indicates the first t The sensing beamforming vector of a target, Indicates the first t The communication signals that a target expects to receive. This represents Gaussian white noise.

6. The resource joint optimization method according to claim 2, characterized in that, The communication and sensing model is constructed as follows: Definition of the first k The effective channel vector of each user terminal is Assume that the communication users are ordered in descending order of channel gain; the... k Each user terminal first detects and removes interference from weak users of the same RIS service, while treating the signals of strong users of the same RIS service and the signals of all other users of the same RIS service as interference; then the... k The interference-to-noise ratio of the received signal for each user terminal is expressed as: In the formula, Indicates noise power; To ensure the smooth progress of SIC decoding, the first k The signals from each user terminal need to be in the same weak user of the RIS service. n Decoding is performed at this point to obtain the corresponding SINR: Then, based on the SIC principle, the user k The achievable rate is: in, and These represent weak users. n Decoding User k The total system throughput is obtained by summing the data rates of all user terminals, based on the data rates obtained from decoding their target signals and the data rates of the strong users. In terms of perception, the active beamforming matrix of the base station Used to sense targets; the angle received by the base station from the sensed target is defined as... The beam gain is: In the formula, This represents the array response of a uniform linear array.

7. The resource joint optimization method according to claim 2, characterized in that, The expression for the optimization problem is: In the optimization problem (P0), and These represent the set of paired variables between the RIS and the user, and the set of phase shift matrices of the RIS, respectively. Indicates the maximum transmission power of the base station. Indicates the first k The achievable rate for each user This represents the minimum reachable rate constraint for each user. This indicates that the base station receives data from the sensing target at an angle of [angle value missing]. beam gain, This represents the minimum beam gain received by the base station for each sensed target. Indicates the first j The first RIS n Phase shift of each reflecting unit, The set of possible values ​​for the discrete phase shift of the RIS reflector unit is represented; constraint C1 restricts each user to match only one RIS; C2 represents the maximum transmit power limit of the system; C3 guarantees the minimum rate for each communication user; C4 guarantees the minimum beam gain for each sensing target received by the base station; C5 indicates that the RIS phase shift under consideration is a discrete phase shift.

8. The resource joint optimization method according to claim 7, characterized in that, During training, each agent inputs the observed state into the actor network to obtain the action to be performed. Then, the environment evaluates the actions of each agent and updates the critic network and actor network. Finally, the target actor network and target critic network are softly updated.

Citation Information

Patent Citations

  • RIS division and resource optimization method for RIS-assisted millimeter wave NOMA system

    CN119628689A

  • Path scheduling and beam forming design system and method for distributed RIS-assisted millimeter wave multi-hop system

    CN119628699A