Antenna and RIS joint deployment method and system for 6G complex and strong shielding environment

Abstract

The invention relates to an antenna and RIS joint deployment method and system for a 6G complex and strong shielding environment, and the method comprises the steps: constructing a communication system model under the 6G complex multipath and strong shielding environment, modeling an antenna and RIS joint deployment optimization problem into a Markov decision process with the maximization of uplink throughput as a target, and defining the Markov decision process; constructing a deep reinforcement learning model, and initializing the deep reinforcement learning model; the deep reinforcement learning model is trained through historical data, the uplink throughput is maximized, model parameters are iteratively updated, and a trained antenna and RIS deployment strategy is obtained; and verifying the trained antenna and RIS deployment strategy in a communication system model in a 6G complex multipath and strong shielding environment, and outputting and applying the optimal antenna and RIS deployment strategy. The problems of high calculation cost and easy local optimum can be avoided, the convergence speed can be increased, and the total reward can be improved.

Description

Technical Field

[0001] The present application relates to the field of wireless communications, and in particular to a method and system for jointly deploying antennas and RIS for 6G complex and strongly obstructed environments. Background Art

[0002] With the commercialization of 5G and the rapid development of 6G, wireless communication systems are gradually moving towards higher frequency bands, larger bandwidths, and more complex application scenarios. In these scenarios, complex multipath and heavily obstructed environments (such as ports, industrial parks, and urban canyons) pose severe challenges to communication network performance. Metal structures, large equipment, and densely populated buildings in these environments cause multiple reflections, refractions, and obstructions in signals, significantly reducing the reliability and throughput of communication links.

[0003] Reconfigurable smart surfaces (RIS) are an emerging technology for improving communication network performance in complex multipath and heavily obstructed environments. By dynamically manipulating the propagation paths of electromagnetic waves, they can significantly improve channel conditions and enhance communication performance. However, in complex and dynamic environments, RIS deployment parameters (such as location, azimuth, and elevation) and dynamic beam adjustment are crucial to communication performance. In particular, the joint optimization of these parameters is crucial to address multidimensional, nonlinear, and time-varying scenarios.

[0004] To address the application challenges of RIS in complex multipath and strong obstruction environments, existing technologies use mathematical modeling and algorithmic derivation to determine RIS deployment parameters. However, this approach is computationally expensive in high-dimensional, non-convex environments and is prone to local optimality, making it difficult to achieve joint optimization of RIS and base station (BS) deployment parameters. Existing RIS deployment parameter optimization approaches using deep reinforcement learning (DRL) typically focus on optimizing RIS phase shift parameters. However, when addressing the joint deployment of BS and RIS, these approaches lack the ability to model the complex relationships between states (such as channel parameters and deployment locations) and actions (such as phase shift matrices and beam directions). This makes it difficult to effectively optimize the throughput between users and base stations, especially in uplink scenarios, resulting in incomplete system performance optimization.

[0005] Therefore, it is necessary to improve one or more problems existing in the above-mentioned related technical solutions.

[0006] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute prior art known to ordinary technicians in the field. Summary of the Invention

[0007] The purpose of the embodiments of the present disclosure is to provide a method and system for jointly deploying antennas and RIS for 6G complex and strongly obstructed environments, thereby overcoming one or more problems caused by the limitations and defects of related technologies to at least a certain extent.

[0008] In a first aspect, the present application provides a method for jointly deploying antennas and RIS for 6G complex and strongly obstructed environments, including:

[0009] Build a 6G communication system model for complex multipath and strong obstruction environments. Model the joint antenna and RIS deployment optimization problem as a Markov decision process with the goal of maximizing uplink throughput. Define the state space, action space, reward function, and state transition probability.

[0010] Build a deep reinforcement learning model that includes an actor network and a critic network integrated with a transformer, and initialize the deep reinforcement learning model;

[0011] The deep reinforcement learning model is trained using historical data to maximize uplink throughput, and the model parameters are iteratively updated to obtain a trained antenna and RIS deployment strategy. The antenna and RIS deployment strategy includes: antenna and RIS deployment coordinates, RIS phase shift matrix, and antenna beamforming vector.

[0012] The trained antenna and RIS deployment strategies are verified in a 6G communication system model under complex multipath and strong obstruction environments, and the optimal antenna and RIS deployment strategies are output and applied.

[0013] In one possible implementation, the steps of constructing a 6G communication system model in a complex multipath and strong obstruction environment, modeling the antenna and RIS joint deployment optimization problem as a Markov decision process with the goal of maximizing uplink throughput, and defining a state space, an action space, a reward function, and a state transition probability include:

[0014] By defining the coordinate parameters of the base station, reconfigurable smart surface and user, a receiving signal model is established;

[0015] To maximize the total uplink throughput of all users, a signal-to-interference-and-noise ratio formula is established.

[0016] Define the state space, action space and state transition probability, and define the reward function of the Markov decision process based on the received signal model and the signal-to-interference-noise ratio formula.

[0017] In a possible implementation, the received signal model is:

[0018] y k =(h D,J,K +h R,J,KfG)x+z k ;

[0019] Among them, y k is the received signal of the kth user, h D,J,K is the channel between the base station and the user, h R,J,K is the channel between RS and user, f is the diagonal phase shift matrix of RIS, G is the channel from BS to RIS, x is the signal vector transmitted from the base station, z k is additive complex Gaussian white noise;

[0020] The signal to interference and noise ratio formula is:

[0021]

[0022] Among them, γ k is the signal-to-interference-and-noise ratio of the kth user, is the interference power received by the kth user from other users, w i is the beamforming vector of the i-th user, w k is the beamforming vector of the kth user, is the noise power of the kth user.

[0023] In one possible implementation, the steps of constructing a deep reinforcement learning model including an Actor network and a Critic network integrated with a Transformer, and initializing the deep reinforcement learning model include:

[0024] Build a learning model that includes the Actor network and the Transformer Critic network, and create a target network for the Actor network and the Critic network;

[0025] Initialize the trainable parameters of the Actor network, Critic network, and target network;

[0026] Build an experience replay pool with a capacity of N to store the experience tuples generated during the training process;

[0027] Integrate the Actor network, Critic network and experience replay pool to obtain an initialized deep reinforcement learning model.

[0028] In one possible implementation, the Actor network is used to output continuous actions based on the input system state through a multi-layer perceptron or a convolutional network; the Critic network is used to convert the state and action into a high-dimensional embedding through a linear layer and concatenate them into a sequence, input the sequence into a Transformer encoder, and output the action value through a fully connected layer.

[0029] In one possible implementation, the steps of training the deep reinforcement learning model using historical data to maximize uplink throughput, iteratively updating model parameters, and obtaining a trained antenna and RIS deployment strategy include:

[0030] Randomly extract small batches of historical experience samples from the experience replay pool;

[0031] The states and actions in the historical experience samples are input into the Critic network through the linear layer to obtain the action value;

[0032] Calculate the target action value and update the Critic network by minimizing the loss based on the action value;

[0033] The action value output by the Critic network is used to update the Actor network parameters using the gradient ascent method;

[0034] Slowly synchronize the target network parameters with the master network parameters through soft updates.

[0035] In a possible implementation, the target action value is expressed as:

[0036] y j =r j +γ·Q s (s j+1 ,a j+1 );

[0037] Among them, y j is the target action value, r j is the immediate reward, γ is the signal to interference and noise ratio, Q s is the target Critic network output, s j+1 is the next state, a j+1 For the next action;

[0038] The expression for minimizing the loss is:

[0039]

[0040] Among them, L Q To minimize the loss function, M is the number of batch samples, Q now is the current action value.

[0041] In one possible implementation, the steps of verifying the trained antenna and RIS deployment strategy in a 6G communication system model under complex multipath and strong obstruction environments, and outputting and applying the optimal antenna and RIS deployment strategy, include:

[0042] Deploy the trained antenna and RIS deployment strategy into the 6G communication system model under complex multipath and strong obstruction environments;

[0043] Adjust the deployed antennas and RIS deployment strategy to maximize uplink throughput and obtain the optimal antenna and RIS deployment strategy;

[0044] Based on the deployment coordinates of the antenna and RIS and the RIS phase shift matrix, base stations and RIS are deployed in actual scenarios, and the beamforming direction is adjusted.

[0045] In a possible implementation, after the steps of deploying base stations and RIS in an actual scenario according to the deployment coordinates of the antennas and RIS and the RIS phase shift matrix, and adjusting the beamforming direction, the method further includes:

[0046] Monitor network performance in real time and re-model and train after actual scenario changes.

[0047] In a second aspect, the present application provides a system for jointly deploying antennas and RIS for 6G complex and strongly obstructed environments. The system is used to execute the above-mentioned method for jointly deploying antennas and RIS for 6G complex and strongly obstructed environments. The system includes:

[0048] The scenario construction module is used to build a 6G communication system model in complex multipath and strong obstruction environments. It models the joint antenna and RIS deployment optimization problem as a Markov decision process with the goal of maximizing uplink throughput, and defines the state space, action space, reward function, and state transition probability.

[0049] The model building module is used to build a deep reinforcement learning model that includes an actor network and a critic network integrated with a transformer, and to initialize the deep reinforcement learning model;

[0050] The model training module is used to train the deep reinforcement learning model using historical data to maximize uplink throughput, iteratively update model parameters, and obtain a trained antenna and RIS deployment strategy. The antenna and RIS deployment strategy includes: antenna and RIS deployment coordinates, RIS phase shift matrix, and antenna beamforming vector.

[0051] The strategy deployment module is used to verify the trained antenna and RIS deployment strategies in the 6G communication system model under complex multipath and strong obstruction environments, and output and apply the optimal antenna and RIS deployment strategies.

[0052] The technical solution provided by this application may have the following beneficial effects:

[0053] The present invention discloses a method and system for the joint deployment of antennas and RIS for 6G complex and strongly occluded environments. By introducing the Transformer self-attention mechanism, the Critic network in the DDPG algorithm can enhance its ability to model the complex relationship between state and action. This method achieves joint optimization of the antenna and RIS deployment positions, phase shift matrix, and antenna beam direction for 6G complex multipath and strongly occluded environments. This significantly improves the uplink throughput between users and base stations, avoids the problems of high computational cost and easy trapping in local optimality, and enhances the convergence speed and total reward, effectively enhancing the robustness and optimization efficiency of 6G communication networks in dynamic and complex scenarios.

[0054] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The accompanying drawings are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0056] Figure 1 A flowchart illustrating a method for jointly deploying antennas and RIS in a complex and strongly obstructed 6G environment in an exemplary embodiment of the present disclosure is provided;

[0057] Figure 2 A detailed flow chart of step S100 of the method for jointly deploying antennas and RIS in a 6G complex and strongly obstructed environment in an exemplary embodiment of the present disclosure is shown;

[0058] Figure 3 A detailed flow chart of step S200 of the method for jointly deploying antennas and RIS in a 6G complex and strongly obstructed environment in an exemplary embodiment of the present disclosure is shown;

[0059] Figure 4 A detailed flow chart of step S300 of the method for jointly deploying antennas and RIS in a complex and strongly obstructed environment for 6G in an exemplary embodiment of the present disclosure is shown;

[0060] Figure 5 A detailed flow chart of step S400 of the method for jointly deploying antennas and RIS in a 6G complex and strongly obstructed environment in an exemplary embodiment of the present disclosure is shown;

[0061] Figure 6 A schematic diagram of an intelligent network in a multipath complex scenario based on throughput maximization is shown for a method for jointly deploying antennas and RIS for 6G complex and strongly obstructed environments in an exemplary embodiment of the present disclosure;

[0062] Figure 7 A schematic diagram of the network optimization principle of the Transformer-enhanced DDPG algorithm for the joint deployment method of antennas and RIS for 6G complex and strongly obstructed environments in an exemplary embodiment of the present disclosure is provided;

[0063] Figure 8 A schematic diagram comparing the number of cycles and total rewards of different algorithms for the antenna and RIS joint deployment method for 6G complex and strongly obstructed environments in an exemplary embodiment of the present disclosure is provided;

[0064] Figure 9 A schematic diagram of the structure of an antenna and RIS joint deployment system for 6G complex and strongly obstructed environments in an exemplary embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0065] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0066] In addition, the accompanying drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. Identical reference numerals in the figures denote identical or similar parts, and thus repetitive descriptions thereof will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0067] This application addresses the challenges faced by wireless communication networks in complex multipath and strong obstruction environments, such as ports, industrial parks, and urban canyons. This application defines the joint deployment optimization of antennas and RIS as maximizing the uplink throughput between users and base stations, and establishes a corresponding system model and objective function. Furthermore, with the goal of maximizing uplink throughput, the deployment positions, phase shift matrices, and antenna beam directions of antennas and RIS are designed and optimized to achieve efficient and robust wireless communication network optimization.

[0068] This example embodiment first provides a method for jointly deploying antennas and RIS for 6G complex and strongly obstructed environments. This method can be applied to a terminal device, such as a mobile phone, desktop computer, personal digital assistant, laptop computer, tablet computer, smart watch, or other mobile terminal. Figure 1As shown in , the method may include the following steps:

[0069] Step S100: Construct a 6G communication system model in a complex multipath and strong obstruction environment, model the antenna and RIS joint deployment optimization problem as a Markov decision process with the goal of maximizing uplink throughput, and define the state space, action space, reward function, and state transition probability.

[0070] Step S200: Construct a deep reinforcement learning model including an Actor network and a Critic network integrated with a Transformer, and initialize the deep reinforcement learning model.

[0071] Step S300: Train the deep reinforcement learning model using historical data to maximize uplink throughput, iteratively update model parameters, and obtain a trained antenna and RIS deployment strategy; the antenna and RIS deployment strategy includes: antenna and RIS deployment coordinates, RIS phase shift matrix, and antenna beamforming vector.

[0072] Step S400: Verify the trained antenna and RIS deployment strategy in a 6G communication system model under complex multipath and strong obstruction environments, and output and apply the optimal antenna and RIS deployment strategy.

[0073] The above method.

[0074] Below, we will refer to Figures 1 to 6 Each step of the above method in this exemplary embodiment is described in more detail.

[0075] In step S100, a 6G communication system model under complex multipath and strong obstruction environments is constructed, and the antenna and RIS joint deployment optimization problem is modeled as a Markov decision process with the goal of maximizing uplink throughput. The state space, action space, reward function, and state transition probability are defined.

[0076] It is understandable that if Figure 6 As shown in the figure, a communication system model is constructed in a complex multipath and strong obstruction environment. The next state of the system is determined only by the action and the current state. Therefore, the process of optimal deployment of antennas and RIS can be simplified to a Markov decision process.

[0077] In one embodiment, Figure 2 As shown, step S100 may include the following sub-steps:

[0078] In step S110 , a received signal model is established by defining coordinate parameters of the base station, the reconfigurable smart surface, and the user.

[0079] Optionally, the received signal model is:

[0080] yk =(h D,J,K +h R,J,K fG)x+z k ;

[0081] Among them, y k For, h D,J,K is the channel between the base station and the user, h R,J,K is the channel between RS and user, f is the diagonal phase shift matrix of RIS, G is the channel from BS to RIS, x is the signal vector transmitted from the base station, z k is additive complex Gaussian white noise.

[0082] Furthermore, step S110 may include the following sub-steps:

[0083] In step S111, the area is divided into a plurality of cells of equal size, and the center coordinates of each cell are Among them, x b 、y b and z b Represents the distances between adjacent cells along the x-axis, y-axis, and z-axis, respectively.

[0084] In step S112, the coordinates of BS and RIS are represented as B a =[[x a ,y a ,z a ] T ∈D c and R a =[x a ,y a ,z a ] T ∈D c , where the sets of BSs and UEs are represented as J = {1, 2, …, J} and K = {1, 2, …, K}, respectively.

[0085] As you can understand, each BS is equipped with N > K transmit antennas, and each UE is equipped with only one receive antenna. RIS is deployed on buildings and has M reflective elements. When the direct link quality between the BS and the user is poor, RIS is used to assist the user.

[0086] In step S113, define is the channel matrix from BS to RIS, and the channel between the J-th base station and the K-th user is expressed as

[0087] In step S114, define is the diagonal phase shift matrix of RIS, where represents the phase shift of the mth element, j represents the imaginary unit, λ∈Λ.

[0088] In step S115, represents the signal transmitted at the BS, where s k is the data symbol transmitted by the kth user with unit power, so the received signal model is represented by y k =(h D,J,K +h R,J,K fG)x+z k .

[0089] It can be understood that it can be expressed as E{|s k | 2}=1 and is the corresponding beamforming vector.

[0090] In step S120, a signal-to-interference-and-noise ratio formula is established with the goal of maximizing the sum of uplink throughputs of all users.

[0091] Optionally, the signal to interference and noise ratio formula is:

[0092]

[0093] Among them, γ k is the signal-to-interference-and-noise ratio of the kth user, w i is the beamforming vector of the i-th user, w k is the beamforming vector of the kth user, is the noise power of the kth user.

[0094] It is understandable that is the interference power received by the kth user from other users, so the received signal-to-interference-and-noise ratio formula is obtained according to the received signal model.

[0095] In step S130 , the state space, action space, and state transition probability are defined, and the reward function of the Markov decision process is defined according to the received signal model and the signal-to-interference-noise ratio formula to obtain the objective function.

[0096] It should be noted that the throughput of all users can be expressed as T K =Blog(1+γ K ), B is the bandwidth.

[0097] Furthermore, step S130 may include the following sub-steps:

[0098] In step S131, a state space is defined including base station positions, smart reflective surface positions, beamforming matrices, smart reflective surface phase shift matrices, a channel matrix set, and a user position set.

[0099] It can be understood that the formula is expressed as S = (Ba ,R a ,W,f); where B a is the base station location, R a is the position of the smart reflector, W is the beamforming matrix, and f is the phase shift matrix of the smart reflector.

[0100] In step S132 , the action space is defined as the adjustment of the deployment positions of the RIS and the base station within a given action limit, as well as the offset angle of the phase shift matrix.

[0101] In step S133, the state transition probability is defined as the probability that the system transitions to the next state after executing an action in the current state.

[0102] In step S134, the reward function is defined as So the objective function is expressed as

[0103] Among them, r j is the reward function, B k is the bandwidth of the kth user, log2(1+γ k ) is the spectrum efficiency, γ k is the signal-to-interference-and-noise ratio, K is the user set, and J is the base station set.

[0104] It should be noted that the location of BS and RIS and their location parameter information are used to maximize the uplink throughput between users and BS to achieve location deployment in complex multipath environments. and Represents the concatenated channel and beamforming matrix of the base station, and uses the power constraint satisfied at the jth BS Zero forcing (ZF) is used to break the zero precoding, so that the objective function can be expressed as

[0105] The objective function and

[0106] In step S200, a deep reinforcement learning model including an Actor network and a Critic network integrated with a Transformer is constructed, and the deep reinforcement learning model is initialized.

[0107] It is understandable that if Figure 7 As shown in Figure 1, a learning model is designed that includes an Actor network and a Critic network integrated with a Transformer, as well as a corresponding target network. The network parameters and the experience replay pool are initialized. The Actor network is responsible for selecting actions, and the Critic network evaluates the value of actions. The experience replay pool is used to store and reuse experience.

[0108] In one embodiment, Figure 3 As shown, step S200 may include the following sub-steps:

[0109] In step S210, a learning model of a critic network including an actor network and a transformer is constructed, and a target network is created for the actor network and the critic network.

[0110] The Actor network is used to output continuous actions based on the input system state through a multi-layer perceptron or convolutional network; the Critic network is used to convert the state and action into high-dimensional embeddings through a linear layer and splice them into sequences, input them into a Transformer encoder, and output the action value through a fully connected layer; the initial parameters of the main Actor and Critic networks are copied to create a target network.

[0111] It should be noted that the system state input to the actor network can be from the state space, such as base station position, RIS position, and channel matrix; the continuous actions output by the actor network can be from the action space, such as base station position adjustment, RIS phase shift, and beamforming vector. The actor network uses a multi-layer perceptron (MLP) or convolutional neural network (CNN) architecture to directly map state to action, enabling policy generation. The state and action input to the critic network are converted into state and action embeddings through a linear layer, with the state embeddings being consistent with the action embeddings. The state and action embeddings are concatenated into a sequence and fed into a Transformer encoder with a multi-head self-attention layer and a feedforward network. This captures global state-action dependencies, such as the synergistic impact of multiple RIS phase shifts on the signal-to-interference-noise ratio of multiple users. The action value, or Q-value, is output through a fully connected layer to evaluate the long-term reward value of the current action and guide policy optimization. The target network then synchronizes the main network parameters through soft updates to avoid training oscillations.

[0112] In step S220 , the trainable parameters of the Actor network, the Critic network, and the target network are initialized.

[0113] It can be understood that the trainable parameters can be linear layer weights, Transformer attention matrix, bias terms, etc., to ensure that the initial strategy is exploratory; at the same time, the training parameters and Transformer parameters are set. The training parameters are learning rate, discount factor and soft update coefficient, and the Transformer parameters are embedding dimension, number of attention heads and feedforward network dimension.

[0114] In step S230, an experience replay pool with a capacity of N is constructed to store experience tuples generated during the training process.

[0115] It should be noted that the storage format of the experience tuple is (s t ,a t ,r t ,s t+1 ), where s t is the current state, a t For the action to be performed, r t For immediate rewards, t+1 At the same time, the experience replay pool is initialized to ensure that the buffer can write and read experience data normally and support batch sampling in subsequent training.

[0116] In step S240, the Actor network, the Critic network, and the experience replay pool are integrated to obtain an initialized deep reinforcement learning model.

[0117] It is understandable that after integrating the initialized deep reinforcement learning model, verification initialization is performed to check whether the network structure, parameter dimensions, and experience replay pool capacity are in line with the design to ensure that the model can operate normally. The obtained initialized deep reinforcement learning model provides an algorithm framework and parameter basis for subsequent strategy learning, realizing dynamic optimization of base station / RIS joint deployment.

[0118] In step S300, the deep reinforcement learning model is trained using historical data to maximize uplink throughput, and the model parameters are iteratively updated to obtain a trained antenna and RIS deployment strategy. The antenna and RIS deployment strategy includes: antenna and RIS deployment coordinates, RIS phase shift matrix, and antenna beamforming vector.

[0119] It can be understood that the uplink throughput, i.e. the reward function, is determined by the signal-to-noise ratio.

[0120] In one embodiment, Figure 4 As shown, step S300 may include the following sub-steps:

[0121] In step S310, a small batch of historical experience samples is randomly extracted from the experience replay pool.

[0122] It is understandable that historical experience samples include the current state, the executed action, the immediate reward, and the next state.

[0123] In step S320, the states and actions in the historical experience samples are input into the Critic network through the linear layer to obtain the action value.

[0124] It should be noted that, optionally, the state s and action a are respectively embedded into the same feature space through two linear layers to obtain the state embedding state_embed and the action embedding action_embed; then sequence splicing is performed to splice the state embedding and the action embedding into a sequence x = [state_embed, action_embed], which serves as the input of the Transformer encoder; then the Transformer encoder composed of a multi-head self-attention mechanism and a feedforward neural network can capture the complex relationship between state and action, and the output of the encoder is mapped to the action value, i.e., Q value, through a fully connected layer, and finally outputs Q now =Q(s t ,a t ), used to evaluate the value of an action.

[0125] In step S330, the target action value is calculated, and the critic network is updated by minimizing the loss according to the action value.

[0126] It should be noted that, optionally, the expression of the target action value is:

[0127] y j =r j +γ·Q s (s j+1 ,a j+1 );

[0128] Among them, y j is the target action value, r j is the immediate reward, γ is the signal to interference and noise ratio, Q s is the target Critic network output, s j+1 is the next state, a j+1 For the next action.

[0129] The expression for minimizing the loss is:

[0130]

[0131] Among them, L Q To minimize the loss function, M is the number of batch samples, Q now is the current action value.

[0132] In step S340, the Actor network parameters are updated using the gradient ascent method based on the action value output by the Critic network.

[0133] It can be understood that the Q-value gradient output by the Critic is used to update the Actor parameters through a policy gradient algorithm, such as Deterministic Policy Gradient (DPG), so that the generated action can obtain higher rewards.

[0134] In step S350, the target network parameters are slowly synchronized with the main network parameters through soft update.

[0135] It should be noted that the soft update target network parameters are optional:

[0136] ζ targ-act ←αζ act +(1-τ)ζ targ-act ,ζ targ-crit ←τζ crit +(1-τ)ζ targ-crit ; Iterate the optimization strategy until the termination condition is met.

[0137] Furthermore, the training process of this application is further explained in the form of an exemplary algorithm to assist in understanding the training process of this application, as shown in Algorithm 1:

[0138]

[0139] In step S400, the trained antenna and RIS deployment strategies are verified in a 6G communication system model under complex multipath and strong obstruction environments, and the optimal antenna and RIS deployment strategies are output and applied.

[0140] It can be understood that in the simulated 6G complex multipath and strong occlusion environment, the performance of the optimal strategy obtained through training is evaluated, and the optimal strategy, including parameters such as RIS and the optimal deployment location of base stations, is output and applied to achieve optimization of the communication network.

[0141] In one embodiment, Figure 5 As shown, step S400 may include the following sub-steps:

[0142] In step S410, the trained antenna and RIS deployment strategy are deployed to the communication system model in the 6G complex multipath and strong obstruction environment.

[0143] It is understandable that the trained strategy is deployed in a simulated 6G environment that reproduces complex multipath and strong occlusion scenarios.

[0144] In step S420, the deployed antennas and RIS deployment strategy are adjusted to maximize the uplink throughput, thereby obtaining the optimal antenna and RIS deployment strategy.

[0145] It can be understood that the uplink throughput can be maximized by adjusting the deployment coordinates of the antenna and RIS, the RIS phase shift matrix, and the antenna beamforming vector in the communication system model under the complex multipath and strong obstruction environment of 6G.

[0146] In step S430, base stations and RIS are deployed in the actual scenario according to the deployment coordinates of the antennas and RIS and the RIS phase shift matrix, and the beamforming direction is adjusted.

[0147] In one embodiment, after step S430, the method further includes:

[0148] In step S440 , the network performance is monitored in real time, and modeling and training are performed again after the actual scenario changes.

[0149] It is understandable that actual scene changes include user movement and changes in obstructions. When modeling and training are performed, the training process is restarted from step S100.

[0150] Further, such as Figure 8 The figure below shows how the total reward per round (Total Reward) changes with the number of training episodes (Episodes) for three different algorithms (DDPG, TD3, and TD-DDPG) during training. The horizontal axis represents the number of training episodes, and the vertical axis represents the total reward per round. This graph allows you to intuitively compare the performance of different algorithms during training.

[0151] It can be clearly seen from the figure that with the increase in the number of training rounds, the total rewards of the three algorithms all show an upward trend, indicating that the algorithms gradually learn and optimize the strategies during the training process. Among them, TD-DDPG is the joint deployment method of antennas and RIS for 6G complex and strongly occluded environments used in this application. Its reward curve is always above DDPG and TD3, especially in the middle and late stages of training, that is, after 500 rounds, its reward growth is more significant, indicating that the deployment method of this application has stronger optimization capabilities and stability in long-term training. In contrast, the reward curve of the DDPG algorithm grows more slowly, and its performance in the later stages of training is relatively poor, indicating that its optimization capabilities in complex tasks are limited. The performance of the TD3 algorithm is between DDPG and the deployment method of this application. Although it performs well in the early stages of training, it is still inferior to the deployment method of this application in the later stages of training.

[0152] Furthermore, in this exemplary embodiment, a system for jointly deploying antennas and RIS for 6G complex and strongly obstructed environments is also provided. Figure 9 As shown in , the system may include:

[0153] The scenario construction module is used to build a 6G communication system model in complex multipath and strong obstruction environments. It models the joint antenna and RIS deployment optimization problem as a Markov decision process with the goal of maximizing uplink throughput, and defines the state space, action space, reward function, and state transition probability.

[0154] The model building module is used to build a deep reinforcement learning model that includes an actor network and a critic network integrated with a transformer, and to initialize the deep reinforcement learning model;

[0155] The model training module is used to train the deep reinforcement learning model using historical data to maximize uplink throughput, iteratively update model parameters, and obtain a trained antenna and RIS deployment strategy. The antenna and RIS deployment strategy includes: antenna and RIS deployment coordinates, RIS phase shift matrix, and antenna beamforming vector.

[0156] The strategy deployment module is used to verify the trained antenna and RIS deployment strategies in the 6G communication system model under complex multipath and strong obstruction environments, and output and apply the optimal antenna and RIS deployment strategies.

[0157] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0158] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiment of the present disclosure, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized. The components displayed as modules or units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the disclosed scheme. Those of ordinary skill in the art can understand and implement it without paying any creative work.

[0159] In an exemplary embodiment of the present disclosure, an electronic device is further provided, which may include a processor and a memory for storing executable instructions of the processor. The processor is configured to execute the executable instructions to perform the steps of the method for jointly deploying antennas and RIS for 6G complex and strongly obstructed environments described in any of the above embodiments.

[0160] Those skilled in the art will appreciate that various aspects of the present invention may be implemented as systems, methods, or program products. Therefore, various aspects of the present invention may be implemented in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits," "modules," or "systems."

[0161] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, server, or network device, etc.) to execute the above-mentioned antenna and RIS joint deployment method for 6G complex and strongly obstructed environments according to the embodiments of the present disclosure.

[0162] In an exemplary embodiment of the present disclosure, a computer storage medium is also provided, on which a computer program is stored. When the program is executed by, for example, a processor, it can implement the steps of the antenna and RIS joint deployment method for 6G complex and strongly obstructed environments described in any of the above embodiments.

[0163] In some possible implementations, various aspects of the present invention may also be implemented in the form of a computer program product, which includes a computer program or instructions. When the computer program product is run on a terminal device, the computer program code or instructions are used to enable the terminal device to execute the steps according to various exemplary embodiments of the present invention described in the above section of the method for jointly deploying antennas and RIS for 6G complex and strongly obstructed environments of this specification.

[0164] The program product described above may be written in any combination of one or more programming languages to implement the program code of the present invention, including object-oriented programming languages such as Java, C++, and the like, as well as conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0165] The computer software product may be stored in a computer storage medium, including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0166] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the appended claims.

Claims

1. A method for jointly deploying antennas and RIS for 6G complex and strongly obstructed environments, characterized in that: include: Build a 6G communication system model for complex multipath and strong obstruction environments. Model the joint antenna and RIS deployment optimization problem as a Markov decision process with the goal of maximizing uplink throughput. Define the state space, action space, reward function, and state transition probability. Build a deep reinforcement learning model that includes an actor network and a critic network integrated with a transformer, and initialize the deep reinforcement learning model; The deep reinforcement learning model is trained using historical data to maximize uplink throughput, and the model parameters are iteratively updated to obtain the trained antenna and RIS deployment strategy. The antenna and RIS deployment strategy includes: deployment coordinates of the antenna and RIS, RIS phase shift matrix and antenna beamforming vector; The trained antenna and RIS deployment strategies are verified in a 6G communication system model under complex multipath and strong obstruction environments, and the optimal antenna and RIS deployment strategies are output and applied.

2. The method for jointly deploying antennas and RIS for 6G complex and strongly obstructed environments according to claim 1, characterized in that: The steps of constructing a 6G communication system model in a complex multipath and strong obstruction environment, modeling the antenna and RIS joint deployment optimization problem as a Markov decision process with the goal of maximizing uplink throughput, and defining the state space, action space, reward function, and state transition probability include: By defining the coordinate parameters of the base station, reconfigurable smart surface and user, a receiving signal model is established; To maximize the total uplink throughput of all users, a signal-to-interference-and-noise ratio formula is established. Define the state space, action space and state transition probability, and define the reward function of the Markov decision process based on the received signal model and the signal-to-interference-noise ratio formula.

3. The method for jointly deploying antennas and RIS for 6G complex and strongly obstructed environments according to claim 2, characterized in that: The received signal model is: y k =(h D,J,K +h R,J,K fG)x+z k ; Among them, y k is the received signal of the kth user, h D,J,K is the channel between the base station and the user, h R,J,K is the channel between RS and user, f is the diagonal phase shift matrix of RIS, G is the channel from BS to RIS, x is the signal vector transmitted from the base station, z k is additive complex Gaussian white noise; The signal to interference and noise ratio formula is: Among them, γ k is the signal-to-interference-and-noise ratio of the kth user, is the interference power received by the kth user from other users, w i is the beamforming vector of the i-th user, w k is the beamforming vector of the kth user, is the noise power of the kth user.

4. The method for jointly deploying antennas and RIS for 6G complex and strongly obstructed environments according to claim 1, characterized in that: The steps of constructing a deep reinforcement learning model including an Actor network and a Critic network integrated with a Transformer, and initializing the deep reinforcement learning model include: Build a learning model that includes the Actor network and the Transformer Critic network, and create a target network for the Actor network and the Critic network; Initialize the trainable parameters of the Actor network, Critic network, and target network; Build an experience replay pool with a capacity of N to store the experience tuples generated during the training process; Integrate the Actor network, Critic network and experience replay pool to obtain an initialized deep reinforcement learning model.

5. The method for jointly deploying antennas and RIS for 6G complex and strongly obstructed environments according to claim 4, characterized in that: The Actor network is used to output continuous actions based on the input system state through a multi-layer perceptron or convolutional network; the Critic network is used to convert the state and action into high-dimensional embeddings through a linear layer and splice them into sequences, input them into a Transformer encoder, and output the action value through a fully connected layer.

6. The method for jointly deploying antennas and RIS for 6G complex and strongly obstructed environments according to claim 1, characterized in that: The steps of training the deep reinforcement learning model using historical data to maximize uplink throughput, iteratively updating model parameters, and obtaining a trained antenna and RIS deployment strategy include: Randomly extract small batches of historical experience samples from the experience replay pool; The states and actions in the historical experience samples are input into the Critic network through the linear layer to obtain the action value; Calculate the target action value and update the Critic network by minimizing the loss based on the action value; The action value output by the Critic network is used to update the Actor network parameters using the gradient ascent method; Slowly synchronize the target network parameters with the master network parameters through soft updates.

7. The method for jointly deploying antennas and RIS for 6G complex and strongly obstructed environments according to claim 6, characterized in that: The expression of the target action value is: y j =r j +γ·Q s (s j+1 ,a j+1 ); Among them, y j is the target action value, r j is the immediate reward, γ is the signal to interference and noise ratio, Q s is the target Critic network output, s j+1 is the next state, a j+1 For the next action; The expression for minimizing the loss is: Among them, L Q To minimize the loss function, M is the number of batch samples, Q now is the current action value.

8. The method for jointly deploying antennas and RIS for 6G complex and strongly obstructed environments according to claim 1, characterized in that: The steps of verifying the trained antenna and RIS deployment strategy in the 6G communication system model under complex multipath and strong obstruction environment, and outputting and applying the optimal antenna and RIS deployment strategy, include: Deploy the trained antenna and RIS deployment strategy into the 6G communication system model under complex multipath and strong obstruction environments; Adjust the deployed antennas and RIS deployment strategy to maximize uplink throughput and obtain the optimal antenna and RIS deployment strategy; Based on the deployment coordinates of the antenna and RIS and the RIS phase shift matrix, base stations and RIS are deployed in actual scenarios, and the beamforming direction is adjusted.

9. The method for jointly deploying antennas and RIS for 6G complex and strongly obstructed environments according to claim 8, characterized in that: After the steps of deploying the base station and the RIS in an actual scenario according to the deployment coordinates of the antenna and the RIS and the RIS phase shift matrix, and adjusting the beamforming direction, the method further includes: Monitor network performance in real time and re-model and train after actual scenario changes.

10. An antenna and RIS joint deployment system for 6G complex and strongly obstructed environments, characterized by: The system is used to perform the method according to any one of claims 1 to 9, and the system includes: The scenario construction module is used to build a 6G communication system model in complex multipath and strong obstruction environments. It models the joint antenna and RIS deployment optimization problem as a Markov decision process with the goal of maximizing uplink throughput, and defines the state space, action space, reward function, and state transition probability. The model building module is used to build a deep reinforcement learning model that includes an actor network and a critic network integrated with a transformer, and to initialize the deep reinforcement learning model; The model training module is used to train the deep reinforcement learning model using historical data to maximize uplink throughput, iteratively update model parameters, and obtain a trained antenna and RIS deployment strategy. The antenna and RIS deployment strategy includes: antenna and RIS deployment coordinates, RIS phase shift matrix, and antenna beamforming vector. The strategy deployment module is used to verify the trained antenna and RIS deployment strategies in the 6G communication system model under complex multipath and strong obstruction environments, and output and apply the optimal antenna and RIS deployment strategies.

Images