A ris-assisted multi-user communication beamforming optimization method based on kan-sac

Abstract

The application discloses a RIS-assisted multi-user communication beamforming optimization method based on KAN-SAC and belongs to the field of reconfigurable intelligent surface-assisted communication.The application considers the scenario of RIS-assisted multi-user multi-input single-output communication, optimizes active beamforming of a base station and passive beamforming of RIS by using a KAN-SAC algorithm, and then maximizes system performance under implicit channel state information (CSI), thereby avoiding complex and expensive explicit channel estimation.The method better expresses complex nonlinear mapping through two layers of KAN modules, and integrates a softmax approximate quantization mechanism in action output, effectively alleviates the blocking problem caused by discrete actions on gradient propagation, enables the model to simultaneously process continuous and discrete mixed control, and enables the model to simultaneously adapt to continuous active beam control of the base station and discrete phase selection of RIS.The application does not need to rely on perfect or imperfect explicit channel estimation, and can effectively solve the joint optimization problem of active beamforming and passive beamforming.

Description

Technical Field

[0001] This invention relates to the field of reconfigurable intelligent surface (RIS) assisted communication, and particularly to a joint beamforming optimization method and system for RIS-assisted multi-user multiple-input single-output (MU-MISO) systems based on Kolmogorov-Arnold network (KAN)-Soft Actor-Critic (SAC). The method achieves joint optimization of active beamforming by the base station and passive beamforming by RIS through non-explicit channel state information (CSI). Background Technology

[0002] The RIS consists of multiple low-cost passive reflectors that can intelligently adjust the incident signal through a controller, thereby reconfiguring the wireless propagation environment and enhancing communication performance, especially in obstructed scenarios. By placing reconfigurable smart metasurfaces, the transmission rate of the communication system can be improved.

[0003] In recent years, RIS-assisted MU-MISO systems have attracted widespread attention. Researchers have proposed maximizing system transmission rates by optimizing active beamforming at the base station and passive beamforming at the RIS. However, most existing studies assume perfect channel state information and rarely consider imperfect channel state information. Traditional methods rely on explicit channel estimation, leading to high computational overhead and resource consumption. While existing deep reinforcement learning-based schemes avoid explicit CSI, they still suffer from insufficient policy expressiveness and poor adaptability to hardware constraints. For example, the existing SAC algorithm uses a multilayer perceptron (MLP) as the policy network, which struggles to efficiently approximate complex beamforming strategies and does not fully consider the hardware limitations of RIS phase discretization. Therefore, there is an urgent need for a joint beamforming optimization method that does not require explicit channel estimation, adapts to hardware constraints, and possesses high expressiveness. Summary of the Invention

[0004] Purpose of the invention: The purpose of this invention is to provide a RIS-assisted multi-user communication beamforming method based on KAN-SAC. For RIS-assisted MU-MISO systems, considering the lack of perfect channel state information, a balance between complexity and performance is achieved by jointly optimizing the active beamforming of the base station and the passive beamforming of the RIS. It also considers directly utilizing the uplink pilot of the transmission to jointly optimize the downlink beamforming, which can maximize the downlink transmission rate of the RIS-assisted MU-MISO system.

[0005] Technical solution: To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0006] In a first aspect, the present invention provides a RIS-assisted multi-user communication beamforming method based on KAN-SAC, comprising the following steps:

[0007] A RIS-assisted MU-MISO system model is constructed, and an optimization problem is established with the active beamforming of the base station and the passive beamforming of the RIS as optimization variables, and the downlink system rate as the objective.

[0008] A non-periodic Markov decision is constructed. The state space includes the decoupled pilot signal, the active beamforming matrix of the base station in the previous time step, and the passive beamforming vector of the RIS. The action space includes the real and imaginary parts of the active beamforming matrix of the base station and the passive beamforming vector of the RIS. The reward function is defined as the total downlink rate of the system at each time step minus the average reward of the previous time steps.

[0009] Design and train a deep reinforcement learning network architecture based on KAN-SAC. The policy network consists of two KAN modules, a mean output layer, and a log-standard output layer. The input dimension of the first KAN module is the dimension of the state space, and its output is fed to the second KAN module. The output of the second KAN module is fed to the mean output layer and the log-standard output layer. The input and output of both KAN modules are connected by a nonlinear mapping that combines K-nearest neighbor selection and learnable spline fitting. The mean output layer is based on a fully connected layer, and its output dimension is the dimension of the action space. The structure of the log-standard output layer is the same as that of the mean output layer, but a truncation operation is added to restrict the output of the log-standard layer to between a pre-set minimum and maximum value. The mean output layer and the log-standard deviation output layer are used to generate the mean and log-standard deviation of the Gaussian action distribution, respectively. A softmax-weighted quantization method is used to map the RIS passive phase shift output by the policy network to a discrete set, achieving approximate discretization of the phase shift while maintaining differentiability.

[0010] Based on the optimized deep reinforcement learning model, the optimal active beamforming and RIS phase shift are obtained based on the uplink pilot signal of the transmission.

[0011] Furthermore, at time step t, the system state s (t) Including the decoupled pilot signal The active beamforming matrix W of the base station at the previous time step t-1 (t-1) , and the RIS passive beamforming vector θ from the previous time step (t-1) , represented as:

[0012]

[0013] Where vec(·) represents the vectorization operation of the stacked matrix's column vectors. This represents the operation of taking the real part of a complex matrix. This represents the operation of taking the imaginary part of a complex matrix;

[0014] Action a (t) Includes base station active beamforming matrix W (t) The real and imaginary parts of the RIS passive beamforming vector θ (t) , represented as:

[0015]

[0016] The system's instantaneous reward at each time step This is expressed as the total reachable rate of the downlink:

[0017]

[0018] Where R k This represents the downlink transmission rate of the k-th user, where K is the number of users. An average reward mechanism is introduced, and the reward used during training is denoted by r. (t) It is represented and defined as:

[0019]

[0020] in The average reward is estimated based on all time steps up to time t, used to measure the improvement of the current policy.

[0021] Furthermore, in order to meet the total transmission power of the base station Constraints are applied, and the generated actions are normalized before being used; where P d It is a downlink transmit power constraint, w k It is the k-th column vector of the active beamforming matrix.

[0022] Furthermore, the decoupled pilot signal Where L is the pilot sequence length and Y is the pilot signal received by the base station. The pilot sequences are organized into a pilot matrix by rows. For an effective uplink channel matrix, P u This indicates the transmit power of the uplink pilot. This represents the equivalent noise.

[0023] Furthermore, in the policy network, the first-layer KAN module receives the input state. Output vector Where A in =4MK+N, A out =2(A in-1), M represents the number of base station antennas, N is the number of RIS reflector units, and K is the number of users; the nonlinear mapping relationship between input and output is:

[0024]

[0025] Where β c (s i () represents the c-th learnable spline basis function corresponding to the i-th state variable, where C is the number of learnable spline basis functions. These are the trainable weights of the first layer of the KAN;

[0026] The input to the second-layer KAN module is z. (1) Dimension A out The output is The dimension is h, and the mapping method is also as follows:

[0027]

[0028] in These are the trainable weights for the second-layer KAN; After the mean output layer and the log-standard deviation output layer generate the mean μ and log-standard deviation logσ of the Gaussian action distribution, the log-standard deviation is transformed into the standard deviation using an exponential function to construct a continuous Gaussian distribution. Then, the distribution is sampled using a reparameterization method.

[0029]

[0030] This yields a differentiable action output, where ⊙ represents element-wise multiplication. This indicates that from the standard normal distribution The sampled random noise, I is the identity matrix, and the final action output of the policy network includes the real and imaginary parts of the unnormalized base station active beamforming matrix and the unquantized RIS passive beamforming vector.

[0031] Furthermore, the KAN-SAC-based deep reinforcement learning network architecture also includes two Q-networks, which are used for... and This indicates that the parameters are φ1 and φ2, and each Q-value network is equipped with a target network, respectively using... and The parameters are φ′1 and φ′2, and each Q network is a three-layer fully connected structure. The input dimension is the vector dimension after concatenating the state and action, and the output dimension is 1. The output is two scalar Q-value estimates.

[0032] Furthermore, the quantization method is as follows: for each unquantized phase shift value output by the policy network... With discrete phase set For each value in the set, calculate the absolute difference, discrete phase set. The CCP has 2 F A discrete phase value, For the f-th quantized phase, these differences are then multiplied by a pre-given scaling factor η, and the negative value is used as the softmax input to calculate the corresponding weight distribution. The quantized phase shift value is expressed as... Where w n,f Unquantized phase shift value For discrete phase set The weight of the f-th quantization phase.

[0033] Secondly, the present invention provides a RIS-assisted multi-user communication beamforming optimization system based on KAN-SAC, comprising:

[0034] The problem modeling module is used to construct a RIS-assisted MU-MISO system model and establish an optimization problem with the active beamforming of the base station and the passive beamforming of the RIS as optimization variables and the downlink system rate as the objective. It also constructs an aperiodic Markov decision, with the state space including the decoupled pilot signal, the active beamforming matrix of the base station in the previous time step, and the passive beamforming vector of the RIS; the action space including the real and imaginary parts of the active beamforming matrix of the base station and the passive beamforming vector of the RIS; and the reward function is defined as the total downlink rate of the system at each time step minus the average reward of the previous time steps.

[0035] The network construction and training module is used to design and train a deep reinforcement learning network architecture based on KAN-SAC. The policy network consists of two KAN modules, a mean output layer, and a log-standard output layer. The input dimension of the first KAN module is the dimension of the state space, and its output is fed to the second KAN module. The output of the second KAN module is fed to the mean output layer and the log-standard output layer. A nonlinear mapping combining K-nearest neighbor selection and learnable spline fitting is used between the input and output of both KAN modules. The mean output layer is based on a fully connected layer, and its output dimension is the action space dimension. The log-standard output layer has the same structure as the mean output layer, but adds a truncation operation to restrict the output of the log-standard layer to between a pre-set minimum and maximum value. The mean output layer and the log-standard deviation output layer are used to generate the mean and log-standard deviation of the Gaussian action distribution, respectively. A softmax-weighted quantization method is used to map the RIS passive phase shift output by the policy network to a discrete set, achieving approximate discretization of the phase shift while maintaining differentiability.

[0036] The beamforming optimization module is used to obtain the optimal active beamforming and RIS phase shift based on the uplink pilot of the transmission, according to the optimized deep reinforcement learning model.

[0037] Thirdly, the present invention provides a computer system including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the RIS-assisted multi-user communication beamforming optimization method based on KAN-SAC.

[0038] Fourthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the KAN-SAC-based RIS-assisted multi-user communication beamforming optimization method.

[0039] Beneficial Effects: Compared with existing technologies, this invention proposes a KAN-SAC-based RIS-assisted multi-user communication beamforming optimization method. It introduces a KAN-SAC model, whose policy network has better expressive power for complex nonlinear mappings. A softmax approximation quantization mechanism is incorporated into the action output, effectively alleviating the blocking problem caused by discrete actions (such as RIS phase selection) on gradient propagation. This allows the model to simultaneously handle continuous and discrete hybrid control, adapting to both continuous active beamforming control of base stations and discrete phase selection of RIS. This invention bypasses explicit channel estimation, directly using pilot signals as input, reducing pilot overhead and computational complexity. It adaptively optimizes RIS-assisted multi-user communication beamforming, maximizing the total downlink transmission rate. This provides a key technical path for the large-scale deployment of RIS in future 6G networks, combining theoretical innovation with engineering practicality, and is of great significance for improving the performance and optimizing the cost of intelligent wireless communication systems. Attached Figure Description

[0040] Figure 1 This is a flowchart of the optimization method according to an embodiment of the present invention;

[0041] Figure 2 A schematic diagram of the RIS-assisted MU-MISO model provided in an embodiment of the present invention;

[0042] Figure 3 The graph shows the variation of the total downlink transmission rate with the base station transmit power under different algorithms. Detailed Implementation

[0043] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The embodiments described with reference to the accompanying drawings are exemplary and intended to explain the present invention, but should not be construed as limiting the present invention.

[0044] In the field of wireless communication, beamforming optimization of reconfigurable smart metasurface (RIS)-assisted multi-user multiple-input single-output (MU-MISO) systems is crucial for improving communication performance. This invention proposes a RIS-assisted multi-user communication beamforming optimization method based on KAN-SAC, aiming to achieve efficient beamforming optimization and improve the overall downlink transmission rate. Unlike traditional methods, this scheme does not require precise channel estimation, reducing complexity and enhancing system adaptability through innovative algorithm design. The specific implementation steps are detailed below in four main parts: system model construction and optimization problem establishment, problem transformation and environment construction, algorithm design and training integrating KAN-SAC and approximate quantization strategies, and the execution phase.

[0045] Step 1: Construct a MU-MISO system model including a base station, RIS, and multiple users. Define the formulas related to the channel, reflection matrix, and transmission rate. Utilize the uplink and downlink channel reciprocity of the RIS-assisted multi-user multiple-input single-output MU-MISO system to establish pilot signals based on uplink transmission. Solve an optimization problem with the active beamforming of the base station and the passive beamforming of the RIS as optimization variables, and maximizing the downlink system rate as the objective. Specifically:

[0046] (1a) Constructing a system model, such as Figure 2 As shown, consider a base station equipped with M antennas, and a RIS containing N controllable reflective elements, serving K single-antenna users. M represents the number of base station antennas, N is the number of RIS reflective elements, and K is the number of users. For the downlink, define... G represents the direct link channel from the base station to user k. r This indicates the channel from the base station to the RIS. Let represent the channel from RIS to user k. Considering the reciprocity of uplink and downlink channels, that is, the uplink channel matrix is ​​the transpose of the downlink matrix.

[0047] RIS passive beamforming vector This indicates that the RIS passive beamforming matrix is ​​composed of... The notation `diag` represents the construction of a diagonal matrix by placing a set of complex elements sequentially along the main diagonal of the matrix and setting the remaining elements to zero. This represents the phase offset of the nth RIS reflector unit, controlling the phase direction of the reflected signal, and Quantization to a set:

[0048]

[0049] Where F is the number of quantization bits, representing the number of bits for phase quantization, i.e., each phase unit can be represented as 2^32 bits. F A discrete phase value, It represents the set of all possible discrete phase offsets.

[0050] The effective downlink channel from the base station to user k It indicates that it can be done through The formula for calculating the downlink transmission rate of user k is:

[0051]

[0052] Where w k It is the active beamforming vector of the base station for user k. It is the variance of zero-mean complex Gaussian noise.

[0053] (1b) Constructing the optimization problem: A joint beamforming optimization problem is constructed with the objective of maximizing the total downlink transmission rate. Based on uplink pilot transmission, it skips channel estimation, does not require perfect channel state information, and uses the base station's active beamforming matrix and the RIS's passive beamforming vector as joint optimization variables. Specifically, in the uplink phase, each user sends a pilot sequence of length L to the base station. To achieve effective user separation at the base station, the pilot sequences assigned to different users are orthogonal to each other. These pilot sequences are organized into a pilot matrix row-wise. Satisfy XX H =LP u I K , where P u This represents the transmit power of the uplink pilot. Specifically, the pilot sequence of the k-th user is denoted as... Corresponding to the k-th row of matrix X, the pilot signal arrives at the base station via the direct link and the RIS auxiliary link. Therefore, the pilot signal received by the base station can be expressed as:

[0054]

[0055] in Represents the equivalent uplink channel matrix. Representing additive white Gaussian noise, under the assumption of no precise channel state information, considering that the only known information available to the system is Y, and secondly, in the downlink phase, the system needs to jointly design: the active beamforming matrix of the base station. The passive beamforming vector of RIS is used and Based on the variables mentioned above, the objective is to maximize the total rate of the system in the downlink. Therefore, the optimization problem can be formally expressed as:

[0056]

[0057] Where w k P is the k-th column vector of the active beamforming matrix.d This is the maximum downlink total transmit power constraint for the base station.

[0058] Step 2: Transform the beamforming joint optimization problem into a non-periodic Markov decision process. Treat the base station as an intelligent agent, and at each time step t, the base station receives the state of the RIS-assisted MU-MISO system. Based on this state, the base station and the RIS jointly execute an action through the controller. After the action is executed, the system will provide a reward corresponding to the optimization objective. Specifically, the state, action, and reward are set as follows:

[0059] (2a) State Construction: The base station uses the orthogonality of the user pilot sequence to decouple the received pilot signal. The decoupled signal is:

[0060]

[0061] Where Y is the pilot signal received by the base station. For an effective uplink channel matrix, Indicates equivalent noise;

[0062] At time step t, the system state s (t) Including the decoupled pilot signal The active beamforming matrix W of the base station at the previous time step t-1 (t-1) , and the RIS passive beamforming vector θ from the previous time step (t-1) ,Right now:

[0063]

[0064] Where vec(·) represents the vectorization operation of the stacked matrix's column vectors. This represents the operation of taking the real part of a complex matrix. This represents the operation of taking the imaginary part of a complex matrix.

[0065] (2b) Action construction: Action a (t) Includes base station active beamforming matrix W (t) The real and imaginary parts of the RIS passive beamforming vector θ (t) , represented as:

[0066]

[0067] To meet the total transmission power of the base station (P d (This is a downlink transmit power constraint) constraint, and the generated actions need to be normalized before application.

[0068] (2c) Reward Construction: The instantaneous reward of the system at each time step This is expressed as the total reachable rate of the downlink:

[0069]

[0070] Where R k Let r represent the downlink transmission rate of the k-th user. To enable the agent to formulate long-term strategies for aperiodic tasks, the influence of historical behavior outcomes must be considered; instantaneous rewards alone are insufficient to guide training. Therefore, an average reward mechanism is introduced, with the reward used during training denoted as r. (t) It is represented and defined as:

[0071]

[0072] in The average reward is estimated based on all time steps up to time t, used to measure the improvement of the current policy.

[0073] Step 3: To optimize the above aperiodic Markov decision, a deep reinforcement learning network architecture based on KAN-SAC is designed, specifically including one policy network and two Q networks, wherein the policy network uses π θ (a|s) represents a parameter θ, which consists of two KAN modules: a mean output layer and a logarithmic standard output layer. The input dimension A of the first KAN module is... in The dimension of the state function, i.e., A in =4MK+N, output dimension A out It is twice the input dimension minus one, i.e., A out =2(A in -1), a nonlinear mapping combining K-nearest neighbor selection and learnable spline fitting is used between the input and output.

[0074] The first-layer KAN module specifically represents the input state. Output vector The nonlinear mapping relationship between input and output is as follows:

[0075]

[0076] Where β c (s i () represents the c-th learnable spline basis function corresponding to the i-th state variable. In this embodiment, the learnable spline basis function adopts a uniformly distributed cubic B-spline, and C is the number of learnable spline basis functions. These are the trainable weights of the first-layer KAN; similarly, the specific architecture of the second-layer KAN module is as follows: input z (1) Dimension A out The output is The dimension is h, here we take h = 256, and the mapping method is also the same:

[0077]

[0078] in The trainable weights of the second-layer KAN are given; the mean output layer is based on a fully connected layer, with the input being the output dimension h of the second-layer KAN and the output dimension being the action space dimension N+2MK; the structure of the log-standard output layer is the same as that of the mean output layer, and is represented as follows:

[0079] u = θ u z (2) +b u

[0080] logσ = clip(θ) σ z (2) +b σ ,logσ min ,logσ max )

[0081] Where clip indicates the truncation operation, limiting the output of the logarithmic standard to between the minimum and maximum values ​​set in the pre-set parameters. Let represent the trainable linear coefficients of the mean output layer and the log-standard deviation output layer, respectively. Let represent the bias trainable parameters of the mean output layer and the log-standard deviation output layer, respectively. After converting the log-standard deviation to standard deviation using an exponential function, a continuous Gaussian distribution is constructed, and this distribution is sampled using a reparameterization method.

[0082]

[0083] This yields differentiable action outputs, ensuring effective backpropagation of the policy gradient. The final action output of the policy network includes the real and imaginary parts of the unnormalized base station active beamforming matrix and the unquantized RIS passive beamforming vector, which can be expressed as:

[0084]

[0085] in This represents the unnormalized active beamforming matrix of the base station. This represents the passive beamforming vector of the unquantized RIS.

[0086] In addition, the two Q networks use and This indicates that the parameters are φ1 and φ2, and each Q-value network is equipped with a target network, respectively using... and The parameters are φ′1 and φ′2, and each Q network is a three-layer fully connected structure. The input dimension is the vector dimension after concatenating the state and action, i.e., 2N+6MK, and the output dimension is 1. In this embodiment, the intermediate dimensions are 256, 128, and 64, and the output is two scalar Q-value estimates.

[0087] Step 4: Train the deep reinforcement learning network described above and update the model parameters.

[0088] (4a) First, the unnormalized base station active beamforming matrix is ​​processed. Normalization, and quantization of the unquantized RIS passive beamforming vector, specifically, the active beamforming matrix satisfying the transmit power is expressed as:

[0089]

[0090] A softmax-weighted quantization method is employed to map the unquantized RIS passive beamforming vector output by the policy network to a discrete set, achieving approximate discretization of the phase shift while maintaining differentiability. Specifically, for each unquantized phase shift value output by the policy network... With discrete phase set Calculate the absolute difference for each value in the discrete phase set. There are a total of 2 F A discrete phase value, For the f-th quantization phase, these differences are then multiplied by a pre-given scaling factor η, and the negative value is used as the softmax input to calculate the corresponding weight distribution, expressed by the following formula:

[0091]

[0092] Where w n,f Unquantized phase shift value For discrete phase set The weight of the f-th quantized phase is given by the following formula: Therefore, the quantized phase shift value is expressed as:

[0093]

[0094] (4b) The deep reinforcement learning model training is based on the maximum entropy framework to find the optimal policy π. * , represented as:

[0095]

[0096] Where π is the strategy, π(a) t |s t ) represents the policy function, given state s t Take action a at that time tThe probability, r(s) t ,a t ) is the immediate reward function, representing the reward in state s. t Take action a t The immediate benefits obtained Indicates the policy in state s t The entropy under the condition reflects the randomness of the strategy, α is the entropy coefficient, used to adjust the trade-off between the reward term and the entropy term, ρ π This indicates that under policy π, the state-action pair (s) t ,a t The distribution of )

[0097] The policy network is updated by maximizing the expected value function of entropy regularization, as follows:

[0098]

[0099] in, This represents the experience replay buffer, where s represents the experience replay buffer. The batch state extracted from the data. Representation of the policy network π θ Batch actions generated for each state in s.

[0100] (4c) Q-value networks are used to evaluate the performance of a given state s. t and action a t The expected cumulative return that the agent can obtain in the future is expressed by the loss function as follows:

[0101]

[0102] in, This represents the experience replay buffer, where s, a, r, and s' represent the data from the experience replay buffer. The batched states, actions, rewards, and the state for the next time slot are extracted from the data. The target Q value is defined as:

[0103]

[0104] Where γ is the discount factor, used to measure the decay of future rewards, and a′~π θ (·|s′) represents the action of sampling the next state from the current policy. The parameters of the Q-value network are iteratively optimized by backpropagating the loss function and using the gradient descent algorithm.

[0105] (4d) To ensure stable training, the target network adopts a soft update method, i.e.

[0106] φ′ i ←τφ i+(1-τ)φ′ i i = 1, 2

[0107] Where τ∈(0,1) is the soft update rate.

[0108] Step 5: During the execution phase, the trained model is loaded, and beamforming is adjusted based on the pilot signals to improve system performance. Specifically:

[0109] (5a) Loading Model: Loading a trained deep reinforcement learning model that learns during training the ability to select the optimal action (beamforming strategy) based on the system state.

[0110] (5b) Output beamforming results: Based on the system state constructed from the acquired uplink pilot signals, the model outputs the unnormalized active beamforming matrix of the base station. passive beamforming vectors of unquantized RIS This enables optimized beamforming adjustments, improving downlink transmission rates and enhancing communication performance.

[0111] (5c) Active beamforming matrix of normalized and unnormalized base stations And the passive beamforming vector of the unquantized RIS Each unquantized phase shift value Direct quantization to discrete phase set The discrete value that is closest to it.

[0112] exist Figure 1 The document describes a flowchart of a RIS-assisted multi-user communication beamforming method based on the enhanced SAC algorithm. First, a deep reinforcement learning model for joint beamforming is trained. Then, the base station optimizes the joint beamforming based on the deep reinforcement learning model trained by the enhanced SAC algorithm.

[0113] exist Figure 2 The system model of RIS-assisted MU-MISO is described, and it can be seen that the uplink pilot is used to optimize the downlink joint beamforming based on the reciprocity of the uplink and downlink.

[0114] exist Figure 3The paper describes how the total downlink rate of different algorithms varies with the base station transmit power. As the downlink transmit power increases, the performance gap between the proposed algorithm and the theoretical upper limit widens. However, since it does not require explicit channel estimation, it greatly reduces system overhead. Furthermore, the sum rate under the proposed algorithm is better than the total transmission rate under other algorithms. The use of the KAN network in this invention significantly improves the policy network's ability to express complex nonlinear mappings, thus outperforming the traditional SAC network. Compared to the improved DDPG, the SAC algorithm used in this invention is more suitable for integration with the softmax approximation quantization mechanism and is more adaptable to the continuous active beam control of the base station and the discrete phase selection of RIS, which are mixed control scenarios of continuous and discrete actions, giving it a greater advantage in practical applications.

[0115] Based on the same inventive concept, this invention provides a KAN-SAC-based RIS-assisted multi-user communication beamforming optimization system, comprising: a problem modeling module for constructing a RIS-assisted MU-MISO system model and establishing an optimization problem with the active beamforming of the base station and the passive beamforming of the RIS as optimization variables, and maximizing the downlink system rate as the objective; and constructing a non-periodic Markov decision, wherein the state space includes the decoupled pilot signal, the active beamforming matrix of the base station in the previous time step, and the passive beamforming vector of the RIS; the action space includes the real and imaginary parts of the active beamforming matrix of the base station and the passive beamforming vector of the RIS, and the reward function is defined as the total downlink rate of the system at each time step minus the average reward of previous time steps; and a network construction and training module for designing and training a KAN-SAC-based deep reinforcement learning network architecture, wherein the policy network includes two KAN modules, a mean output layer, and a... The log-standard output layer takes the state space dimension as input to the first-layer KAN module and outputs to the second-layer KAN module. The second-layer KAN module then outputs to the mean output layer and the log-standard output layer. Both KAN modules employ a nonlinear mapping combining K-nearest neighbor selection and learnable spline fitting between their inputs and outputs. The mean output layer is based on a fully connected layer, with the action space dimension as its output. The log-standard output layer has the same structure as the mean output layer, but adds a truncation operation to restrict the log-standard output to between a pre-set minimum and maximum value. The mean output layer and the log-standard deviation output layer are used to generate the mean and log-standard deviation of the Gaussian action distribution, respectively. A softmax-weighted quantization method is used to map the passive RIS phase shift output by the policy network to a discrete set, achieving approximate discretization of the phase shift while maintaining differentiability. The beamforming optimization module, based on the optimized deep reinforcement learning model and the transmitted uplink pilot, obtains the optimal active beamforming and RIS phase shift. Detailed implementation details of each module are provided in the aforementioned method implementation examples and will not be repeated here.

[0116] This invention also provides a computer system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the KAN-SAC-based RIS-assisted multi-user communication beamforming optimization method.

[0117] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the KAN-SAC-based RIS-assisted multi-user communication beamforming optimization method.

[0118] The program code used to implement the method of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the steps of the method of the present invention to be performed. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0119] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes. Furthermore, the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved, as will be understood by those skilled in the art to which embodiments of the invention pertain. Content not described in detail in this application is prior art known to those skilled in the art.

Claims

1. A RIS-assisted multi-user communication beamforming optimization method based on KAN-SAC, characterized in that, include: A RIS-assisted MU-MISO system model is constructed, and an optimization problem is established with the active beamforming of the base station and the passive beamforming of the RIS as optimization variables, and the downlink system rate as the objective. A non-periodic Markov decision is constructed. The state space includes the decoupled pilot signal, the active beamforming matrix of the base station in the previous time step, and the passive beamforming vector of the RIS. The action space includes the real and imaginary parts of the active beamforming matrix of the base station and the passive beamforming vector of the RIS. The reward function is defined as the total downlink rate of the system at each time step minus the average reward of the previous time steps. Design and train a deep reinforcement learning network architecture based on KAN-SAC. The policy network consists of two KAN modules, a mean output layer, and a log-standard output layer. The input dimension of the first KAN module is the dimension of the state space, and its output is fed to the second KAN module. The output of the second KAN module is fed to the mean output layer and the log-standard output layer. The input and output of both KAN modules are connected by a nonlinear mapping that combines K-nearest neighbor selection and learnable spline fitting. The mean output layer is based on a fully connected layer, and its output dimension is the dimension of the action space. The structure of the log-standard output layer is the same as that of the mean output layer, but a truncation operation is added to restrict the output of the log-standard layer to between a pre-set minimum and maximum value. The mean output layer and the log-standard deviation output layer are used to generate the mean and log-standard deviation of the Gaussian action distribution, respectively. A softmax-weighted quantization method is used to map the RIS passive phase shift output by the policy network to a discrete set, achieving approximate discretization of the phase shift while maintaining differentiability. Based on the optimized deep reinforcement learning model and the uplink pilot signal of the transmission, the optimal active beamforming and RIS phase shift are obtained. In the policy network, the first-layer KAN module receives the state as input. Output vector ,in , M represents the number of base station antennas, N is the number of RIS reflector units, and K is the number of users; the nonlinear mapping relationship between input and output is: ； in Indicates the first The state variable corresponding to the first A learnable spline basis function The number of learnable spline basis functions, These are the trainable weights of the first layer of the KAN; The input to the second-layer KAN module is... , dimension The output is , dimension The mapping method is also as follows: ； in These are the trainable weights for the second layer of the KAN; The mean output layer and the log-standard deviation output layer generate the mean of the Gaussian action distribution. With log standard deviation Then, the logarithmic standard deviation is transformed into standard deviation using an exponential function, a continuous Gaussian distribution is constructed, and the distribution is sampled using a reparameterization method: ； Thus, differentiable action output is obtained, where This represents element-wise multiplication. This indicates that from the standard normal distribution Random noise from sampling The identity matrix is ​​used; the final action output of the policy network includes the real and imaginary parts of the unnormalized base station active beamforming matrix and the unquantized RIS passive beamforming vector.

2. The RIS-assisted multi-user communication beamforming optimization method based on KAN-SAC according to claim 1, characterized in that, At time step The state of the system Including the decoupled pilot signal The previous step active beamforming matrix for base stations And the RIS passive beamforming vector of the previous time step , represented as: ； in The column vectors representing stacked matrices enable matrix vectorization operations. This represents the operation of taking the real part of a complex matrix. This represents the operation of taking the imaginary part of a complex matrix; action Includes base station active beamforming matrix The real and imaginary parts and the RIS passive beamforming vector , represented as: ； The system's instantaneous reward at each time step This is expressed as the total reachable rate of the downlink: ； in Indicates the first The downlink transmission rate for each user is used to introduce an average reward mechanism, with the reward used during training being... It is represented and defined as: ； in The average reward is estimated based on all time steps up to time t, and is used to measure the improvement of the current policy.

3. The RIS-assisted multi-user communication beamforming optimization method based on KAN-SAC according to claim 2, characterized in that, To meet the total transmission power of the base station Constraints are applied, and the generated actions are normalized before being used. in It is a downlink transmit power constraint. It is the first active beamforming matrix Column vector.

4. The RIS-assisted multi-user communication beamforming optimization method based on KAN-SAC according to claim 2, characterized in that, Decoupled pilot signal in Y is the pilot sequence length, and Y is the pilot signal received by the base station. The pilot sequences are organized into a pilot matrix by rows. For an effective uplink channel matrix, This indicates the transmit power of the uplink pilot. This represents the equivalent noise.

5. The RIS-assisted multi-user communication beamforming optimization method based on KAN-SAC according to claim 1, characterized in that, The KAN-SAC-based deep reinforcement learning network architecture also includes two Q-networks, which are used for... and It indicates that the parameters are respectively and Each Q-value network is equipped with a target network, and each Q-value network is used with a target network. and It indicates that the parameters are respectively and Each Q-network is a three-layer fully connected structure. The input dimension is the vector dimension after concatenating the state and action, and the output dimension is 1. The output is two scalar Q-value estimates.

6. The RIS-assisted multi-user communication beamforming optimization method based on KAN-SAC according to claim 1, characterized in that, The quantization method is as follows: for each unquantized phase shift value output by the policy network... , and discrete phase set For each value in the set, calculate the absolute difference, discrete phase set. The CCP A discrete phase value, For the first Each phase is quantized, and then these differences are multiplied by a pre-defined scaling factor. The negative value is then used as the softmax input to calculate the corresponding weight distribution. The quantized phase shift value is expressed as... in Unquantized phase shift value For discrete phase set The Middle The weights of each quantized phase.

7. A KAN-SAC-based RIS-assisted multi-user communication beamforming optimization system, used to implement the KAN-SAC-based RIS-assisted multi-user communication beamforming optimization method according to any one of claims 1-6, characterized in that, include: The problem modeling module is used to construct a RIS-assisted MU-MISO system model and establish an optimization problem with the active beamforming of the base station and the passive beamforming of the RIS as optimization variables and the downlink system rate as the objective. And constructing aperiodic Markov decision, the state space includes the decoupled pilot signal, the active beamforming matrix of the base station in the previous time step and the passive beamforming vector of the RIS; the action space includes the real and imaginary parts of the active beamforming matrix of the base station and the passive beamforming vector of the RIS, and the reward function is defined as the total downlink rate of the system in each time step minus the average reward of the past time steps. The network construction and training module is used to design and train a deep reinforcement learning network architecture based on KAN-SAC. The policy network consists of two KAN modules, a mean output layer, and a log-standard output layer. The input dimension of the first KAN module is the dimension of the state space, and its output is fed to the second KAN module. The output of the second KAN module is fed to the mean output layer and the log-standard output layer. A nonlinear mapping combining K-nearest neighbor selection and learnable spline fitting is used between the input and output of both KAN modules. The mean output layer is based on a fully connected layer, and its output dimension is the action space dimension. The log-standard output layer has the same structure as the mean output layer, but adds a truncation operation to restrict the output of the log-standard layer to between a pre-set minimum and maximum value. The mean output layer and the log-standard deviation output layer are used to generate the mean and log-standard deviation of the Gaussian action distribution, respectively. A softmax-weighted quantization method is used to map the RIS passive phase shift output by the policy network to a discrete set, achieving approximate discretization of the phase shift while maintaining differentiability. The beamforming optimization module is used to obtain the optimal active beamforming and RIS phase shift based on the uplink pilot of the transmission, according to the optimized deep reinforcement learning model.

8. A computer system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is executed by the processor, it implements the steps of the RIS-assisted multi-user communication beamforming optimization method based on KAN-SAC according to any one of claims 1-6.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the RIS-assisted multi-user communication beamforming optimization method based on KAN-SAC according to any one of claims 1-6.

Images