Broadband capacity optimization method and device

Abstract

The present invention discloses a broadband capacity optimization method and device. The method includes: replacing some base stations in a cell-free network with an intelligent reconfigurable surface (RIS) to construct a RIS-assisted cell-free network system model; designing a joint precoding problem in the RIS-assisted cell-free network system to maximize the user sum rate by jointly optimizing the active precoding at the base station and the passive precoding at the RIS; introducing auxiliary variables: the RIS phase shift matrix Θ and the precoding vector matrix W, to transform the joint precoding problem into an optimization problem of Θ and W; and using a deep reinforcement learning algorithm to optimize the phase shift matrix Θ and the precoding vector matrix W, respectively, to achieve optimal solutions for both Θ and W, thereby maximizing the user sum rate performance and achieving broadband capacity optimization. The broadband capacity optimization method provided by the present invention can effectively solve the cost and power consumption problems of cell-free networks.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technology, and in particular to a broadband capacity optimization method and device. Background Art

[0002] In recent years, reconfigurable smart surfaces (RIS) have attracted widespread attention as a promising technology. RIS is a revolutionary new technology that can significantly improve the performance of wireless communication networks by intelligently reconfiguring the wireless propagation environment by integrating a large number of low-cost passive reflective elements on a planar surface. Specifically, different elements of a RIS can independently reflect the incident signal by controlling its amplitude and / or phase, thereby collaboratively achieving sophisticated three-dimensional passive beamforming for directional signal enhancement or nulling.

[0003] As a new user-centric network, cell-free networks leverage collaboration between base stations to effectively address the inter-cell interference problem common in traditional networks. In RIS-assisted cell-free networks, improving the sum rate of users within a cell is a key research topic, but currently no effective solution exists. Summary of the Invention

[0004] The present invention provides a broadband capacity optimization method and device, which are used to improve the sum rate of users in a cell and optimize broadband capacity in a RIS-assisted non-cell network scenario.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] In one aspect, the present invention provides a method for optimizing broadband capacity, comprising:

[0007] A RIS-assisted no-cell network system model is constructed by replacing some base stations in a no-cell network with an intelligent reconfigurable surface (RIS). RIS communication is used between user pairs in the system model.

[0008] The joint precoding problem in a RIS-assisted cell-free network system is designed. By jointly optimizing the active precoding at the base station and the passive precoding at the RIS, the user sum rate is maximized. Auxiliary variables are introduced: the RIS phase shift matrix Θ and the precoding vector matrix W, transforming the joint precoding problem into an optimization problem of Θ and W.

[0009] The phase shift matrix Θ and the precoding vector matrix W are optimized using a deep reinforcement learning algorithm, respectively, so that both Θ and W take the optimal solution to maximize the user's sum rate performance and achieve broadband capacity optimization.

[0010] Furthermore, the RIS-assisted non-cell network system is a discrete time slot system, and the RIS-assisted non-cell network system model is modeled as a Markov decision model.

[0011] Furthermore, the deep reinforcement learning algorithm is a proximal policy optimization PPO algorithm in deep reinforcement learning DRL.

[0012] Furthermore, the phase shift matrix Θ and the precoding vector matrix W are optimized using a deep reinforcement learning algorithm to optimize both Θ and W to maximize the user sum rate performance and achieve broadband capacity optimization, including:

[0013] Step 1: Initialize wireless device information, user information, and agent environment information, including: base station maximum transmission power P max , user weight η k , as well as agent actions and states;

[0014] Step 2, repeatedly perform the following process: set the current state s t Input actor-new network and get action a t , and then input it into the environment to get the reward r t and the next state s t′ ; Until the preset number of {s is stored t ,a t ,r t};

[0015] The equivalent channel from base station b to user k on subcarrier p is expressed as:

[0016]

[0017] Where, G b,r,p 、 They represent the frequency domain channels from base station b to user k, from base station b to RISr, and from RISr to user k on subcarrier p respectively; represents the phase shift matrix of RISr, R represents the number of RIS;

[0018] Consider a Gaussian channel in the current state, let z k,p represents additive white Gaussian noise, then the received signal is:

[0019]

[0020] Where y b,k,p represents the baseband frequency domain signal from base station b to user k on subcarrier p, w b,p,j represents the precoding vector of base station b, s p,j Indicated by w b,p,jPrecoded frequency domain signal; B represents the number of base stations; K represents the number of users;

[0021] Step 3: Design the objective function for finding the number of users and the rate;

[0022] Among them, the signal s of subcarrier p on user k is p,k The signal-to-noise ratio is expressed as:

[0023]

[0024] Where, Ξ k,p represents the variance of additive white Gaussian noise;

[0025] From this, the user's weighted sum rate expression is derived, that is, the objective function is:

[0026]

[0027] Where, Θ=diag(Θ1,…,Θ R ); P represents the number of subcarriers; η k Indicates user weight;

[0028] Two constraints are required to make the objective function valid:

[0029]

[0030] in, represents the feasible set of RIS reflection coefficients; θ r,n represents the reflection coefficient of RIS;

[0031] Step 4: All stored s t ,a t ,r t Combine the inputs into the critic network and calculate the advantage function:

[0032] The advantage function is calculated by the following formula:

[0033]

[0034] Where γ represents the discount factor, s t Indicates the current state, s t′ Indicates the next state, a t Indicates the current action, r t′ represents the objective function, γ t′-t represents the difference between the discount factor at the next moment and the current moment, f′(Θ,W) represents the objective function at the next moment; t′ represents the next moment, t represents the current moment, π(s t ,a t ) represents the current environment and the strategy generated by the action, P(st′ |s t ,a t ) represents the probability of generating new environmental information under the current environment and action; V Φ (s t ) represents the value function, which is calculated by the following formula:

[0035]

[0036] Step 5: Obtain instant rewards based on the Bellman equation for learning, and use the PPO algorithm to implement alternating optimization learning of Θ and W to obtain the optimal solutions of Θ and W; to maximize the user's sum rate performance.

[0037] Furthermore, the PPO algorithm is used to implement alternating optimization learning of Θ and W to obtain the optimal solution of Θ and W, including:

[0038] The precoding vector matrix W is optimized and the calculation formula is as follows:

[0039]

[0040] Where y(W) represents the actual value of the objective function, represents the constraint function; P W (a t |s t ) indicates that when W is not optimized, the environment information s t Next action a t The probability of Indicates that W is optimized to W k Later in the environmental information t Next action a t The probability of Indicates that when optimized to W k The advantage function after ε represents a parameter that limits the difference between the new and old strategies;

[0041] When the algorithm converges, terminate the optimization process and record the optimized solution W opt ;

[0042] Optimize the phase shift matrix Θ and solve the optimal solution W opt Substitute the objective function f(Θ,W) and then optimize Θ. When the algorithm converges, terminate the optimization process and obtain the optimal solution Θ opt .

[0043] On the other hand, the present invention also provides a broadband capacity optimization device, comprising:

[0044] A RIS-assisted no-cell network system model construction module is used to replace some base stations in the no-cell network with an intelligent reconfigurable surface (RIS) to construct a RIS-assisted no-cell network system model; wherein RIS communication is adopted between user pairs in the system model;

[0045] The joint precoding problem design module is used to design the joint precoding problem in RIS-assisted cell-free networks. By jointly optimizing the active precoding at the base station and the passive precoding at the RIS, the user sum rate is maximized. Auxiliary variables are introduced: the RIS phase shift matrix Θ and the precoding vector matrix W, transforming the joint precoding problem into an optimization problem involving Θ and W.

[0046] The deep reinforcement learning optimization module is used to optimize the phase shift matrix Θ and the precoding vector matrix W using the deep reinforcement learning algorithm, so that both Θ and W are optimal solutions to maximize the user's sum rate performance and achieve broadband capacity optimization.

[0047] Furthermore, the RIS-assisted non-cell network system is a discrete time slot system, and the RIS-assisted non-cell network system model is modeled as a Markov decision model.

[0048] Furthermore, the deep reinforcement learning algorithm is a proximal policy optimization PPO algorithm in deep reinforcement learning DRL.

[0049] Furthermore, the deep reinforcement learning optimization module is specifically used to:

[0050] Step 1: Initialize wireless device information, user information, and agent environment information, including: base station maximum transmission power P max , user weight η k , as well as agent actions and states;

[0051] Step 2, repeatedly perform the following process: set the current state s t Input actor-new network and get action a t , and then input it into the environment to get the reward r t and the next state s t′ ; Until the preset number of {s is stored t ,a t ,r t};

[0052] The equivalent channel from base station b to user k on subcarrier p is expressed as:

[0053]

[0054] Where, G b,r,p 、 They represent the frequency domain channels from base station b to user k, from base station b to RISr, and from RISr to user k on subcarrier p respectively; represents the phase shift matrix of RISr, R represents the number of RIS;

[0055] Consider a Gaussian channel in the current state, let z k,p represents additive white Gaussian noise, then the received signal is:

[0056]

[0057] Where y b,k,p represents the baseband frequency domain signal from base station b to user k on subcarrier p, w b,p,j represents the precoding vector of base station b, s p,j Indicated by w b,p,j Precoded frequency domain signal; B represents the number of base stations; K represents the number of users;

[0058] Step 3: Design the objective function for finding the number of users and the rate;

[0059] Among them, the signal s of subcarrier p on user k is p,k The signal-to-noise ratio is expressed as:

[0060]

[0061] Where, Ξ k,p represents the variance of additive white Gaussian noise;

[0062] From this, the user's weighted sum rate expression is derived, that is, the objective function is:

[0063]

[0064] Where, Θ=diag(Θ1,…,Θ R ); P represents the number of subcarriers; η k Indicates user weight;

[0065] Two constraints are required to make the objective function valid:

[0066]

[0067] in, represents the feasible set of RIS reflection coefficients; θ r,n represents the reflection coefficient of RIS;

[0068] Step 4: All stored s t ,a t ,r t Combine the inputs into the critic network and calculate the advantage function:

[0069] The advantage function is calculated by the following formula:

[0070]

[0071] Where γ represents the discount factor, s t Indicates the current state, s t′ Indicates the next state, a t Indicates the current action, r t′ represents the objective function, γ t′-t represents the difference between the discount factor at the next moment and the current moment, f′(Θ,W) represents the objective function at the next moment; t′ represents the next moment, t represents the current moment, π(s t ,a t ) represents the current environment and the strategy generated by the action, P(s t′ |s t ,a t ) represents the probability of generating new environmental information under the current environment and action; V Φ (s t ) represents the value function, which is calculated by the following formula:

[0072]

[0073] Step 5: Obtain instant rewards based on the Bellman equation for learning, and use the PPO algorithm to implement alternating optimization learning of Θ and W to obtain the optimal solutions of Θ and W; to maximize the user's sum rate performance.

[0074] Furthermore, the PPO algorithm is used to implement alternating optimization learning of Θ and W to obtain the optimal solution of Θ and W, including:

[0075] The precoding vector matrix W is optimized and the calculation formula is as follows:

[0076]

[0077] Where y(W) represents the actual value of the objective function, represents the constraint function; P W (a t |s t ) indicates that when W is not optimized, the environment information s t Next action a t The probability of Indicates that W is optimized to W k Later in the environmental information t Next action a t The probability of Indicates that when optimized to W kThe advantage function after ε represents a parameter that limits the difference between the new and old strategies;

[0078] When the algorithm converges, terminate the optimization process and record the optimized solution W opt ;

[0079] Optimize the phase shift matrix Θ and solve the optimal solution W opt Substitute the objective function f(Θ,W) and then optimize Θ. When the algorithm converges, terminate the optimization process and obtain the optimal solution Θ opt .

[0080] On the other hand, the present invention further provides an electronic device, comprising a processor and a memory; wherein the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the above method.

[0081] In yet another aspect, the present invention further provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, and the instruction is loaded and executed by a processor to implement the above method.

[0082] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0083] With the goal of maximizing users and rates, the present invention proposes a broadband capacity optimization solution for cell-free networks. RIS is used to replace some base stations in the cell-free network, while more RIS are deployed in the system to further increase network capacity. This invention can effectively solve the cost and power consumption issues of cell-free networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0085] Figure 1 1 is a schematic diagram of a RIS-assisted cell-free network system model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0086] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0087] First embodiment

[0088] This embodiment provides a broadband capacity optimization method for improving the sum and rate of users within a cell and optimizing broadband capacity in a RIS-assisted cell-free network scenario. This method considers utilizing intelligent reconfigurable surface (RIS) technology to address the problem of increasing capacity under cost and power consumption constraints. By applying RIS to a cell-free network, replacing some base stations in the cell-free network with low-cost, low-power RIS, and deploying more RIS in the system to further increase network capacity, the cost and power consumption issues of the cell-free network can be effectively addressed. Furthermore, this method designs and models the joint precoding problem in a RIS-assisted cell-free network system, introducing auxiliary variables: the phase shift matrix Θ and the precoding vector matrix W of the RIS, transforming the joint precoding problem into an optimization problem of Θ and W. The agent is trained using the Proximal Phase Optimization (PPO) algorithm in deep reinforcement learning to obtain corresponding actions and policies. The policy is updated using an on-policy method, and the clipped method is used to limit the difference between the new and old policies. By implementing alternating learning optimization of the RIS phase shift matrix and precoding vector matrix, the convergence speed of user sum and rate can be greatly improved, thereby achieving the goal of improving communication system performance. The feasibility of the optimization method of broadband capacity of cell-free network assisted by RIS based on PPO will be demonstrated through loop, maximizing the sum of user rates.

[0089] The key idea of ​​this method is to design a joint precoding problem for a cell-free network system model, jointly optimizing the active precoding at the base station and the passive precoding at the RIS. Deep reinforcement learning is then used to alternately optimize the auxiliary variable phase shift matrix Θ and the precoding vector matrix W, thereby maximizing the user sum rate. The system is a discrete time-slot system, modeled as a Markov decision model. PPO is used for decision optimization, aiming to maximize the system's objective function.

[0090] Specifically, the broadband capacity optimization method of this embodiment includes the following steps:

[0091] S1, replacing some base stations in a cell-free network with an intelligent reconfigurable surface (RIS) to construct a RIS-assisted cell-free network system model; wherein RIS communication is used between user pairs in the system model;

[0092] Figure 1 The figure shows a broadband scenario for a cell-free network, which includes M base station antennas, U user antennas, N RIS elements, B base stations, K users, R RIS elements, and P subcarriers, forming a RIS-assisted cell-free network model. The system is a discrete time-slot system, modeled as a Markov decision model.

[0093] S2: Design the joint precoding problem in a RIS-assisted cell-free network system. By jointly optimizing the active precoding at the base station and the passive precoding at the RIS, the user sum rate is maximized. Auxiliary variables are introduced: the RIS phase shift matrix Θ and the precoding vector matrix W, transforming the joint precoding problem into an optimization problem of Θ and W.

[0094] S3 uses a deep reinforcement learning algorithm to optimize the phase shift matrix Θ and the precoding vector matrix W, respectively, so that both Θ and W take the optimal solution to maximize the user's sum rate performance and achieve broadband capacity optimization.

[0095] The algorithm used in S3 is the PPO algorithm. Specifically, S3 includes the following steps:

[0096] S31, initialize wireless devices, users and RIS (intelligent agent) environment information, including: base station maximum transmission power P max , user weight η k , as well as agent actions and states;

[0097] The user equipment, RIS, and channel states are modeled as a finite-state Markov model. If the system state does not change at one moment, the system transitions at the next moment based on the state transition probability.

[0098] S32, the current environment information s t Input actor-new network and get action a t , and then input it into the environment to get the reward r t and the next state s t′ ; Repeat the above steps until a certain number of {s t ,a t ,r t};

[0099] The equivalent channel from base station b to user k on subcarrier p is expressed as:

[0100]

[0101] Where, G b,r,p 、 denote the frequency domain channels from base station b to user k, from base station b to RISr, and from RISr to user k on subcarrier p, respectively; Θ r represents the phase shift matrix of RISr, Θ r =diag(θ r,1 ,…,θ r,N ), represents the feasible set of RIS reflection coefficients,

[0102] Considering a Gaussian channel for the current environment information, let z k,p represents additive white Gaussian noise (with zero expectation and variance Ξ k,p =σ 2 I U ), the received signal is expressed as:

[0103]

[0104] Where y b,k,p represents the baseband frequency domain signal from base station b to user k on subcarrier p, w b,p,j represents the precoding vector of base station b, s p,j Indicated by w b,p,j precoded frequency domain signal;

[0105] S33, design the precoding problem of the model and obtain the objective function for finding the user sum rate;

[0106] Among them, the signal s of subcarrier p on user k is p,k The signal-to-noise ratio (SINR) is expressed as:

[0107]

[0108] Where,

[0109] From this, the user's weighted sum rate expression is derived, that is, the objective function is:

[0110]

[0111] Where, Θ=diag(Θ1,…,Θ R );η k represents the user weight, γ k,p The signal s of subcarrier p at user k is p,k signal-to-noise ratio (SINR).

[0112] Two constraints are required to make the objective function valid:

[0113] 1) 2)

[0114] Among them, θ r,n Represents the reflection coefficient of RIS

[0115] S34, all stored s t ,a t ,r t Combine the inputs into the critic network and calculate the advantage function:

[0116] The advantage function is calculated by the following formula:

[0117]

[0118] Where γ represents the discount factor, r t′ represents the objective function, s t Indicates the current state, s t′ Indicates the next state, a t represents the current action, γ t′-t represents the difference between the discount factor at the next moment and the current moment, f′(Θ,W) represents the objective function at the next moment; t′ represents the next moment, t represents the current moment, π(s t ,a t ) represents the current environment and the strategy generated by the action, P(s t′ |s t ,a t ) represents the probability of generating new environmental information under the current environment and action; V Φ (s t ) represents the value function, which is calculated by the following formula:

[0119]

[0120] S35 uses the Bellman equation to obtain immediate rewards for reinforcement learning, with the goal of maximizing the reward function, that is, achieving the maximum value of the user sum and rate. The PPO algorithm is used, which updates the objective function in small batches over multiple training steps, solving the problem of difficult step size determination in policy gradient algorithms. The goal of deep reinforcement learning is to maximize the reward function, that is, to achieve the maximum value of the user sum and rate.

[0121] Since there are two variables in the objective function, PPO is used to perform alternating optimization learning to obtain the optimal solution of the two variables - W opt and Θ opt , as follows:

[0122] First, the precoding vector matrix W is optimized, and the calculation formula is as follows:

[0123]

[0124] Where y(W) represents the actual value of the objective function, represents the constraint function; P W (a t |s t ) indicates that when W is not optimized, the environment information s t Next action a t The probability of Indicates that W is optimized to W k Later in the environmental informationt Next action a t The probability of Indicates that when optimized to W k The advantage function after ε represents a parameter that limits the difference between the new and old strategies.

[0125] When the algorithm converges, that is, the reward function remains within a certain range for a long time and no longer increases, the sum rate of users in the cell-free network reaches the maximum, the optimization process is terminated, and the optimized solution W is recorded. opt ;

[0126] Secondly, optimize the phase shift matrix Θ and solve the optimized solution W opt Substitute the objective function f(Θ,W) and repeat the above steps. The objective function now becomes f(Θ,W opt ), when the reward function remains within a certain range for a long time and no longer increases, the optimization process is terminated and the optimal solution Θ is obtained. opt .

[0127] In summary, this embodiment optimizes the sum rate of users in a cell-free network. Using RIS to assist the cell-free network, a joint precoding problem is designed for this system. By jointly optimizing the active precoding on the base station side and the passive precoding on the RIS side, the sum rate of the user is maximized. Furthermore, two auxiliary variables—the phase shift matrix Θ and the precoding vector matrix W—are introduced. Alternating optimization is employed, and each auxiliary variable is optimized using a deep reinforcement learning (PPO) algorithm. This method focuses on the active precoding on the base station side and the passive precoding on the RIS side, using PPO for learning. This optimizes both the phase shift matrix Θ and the precoding vector matrix W, maximizing the sum rate performance of the user.

[0128] Second embodiment

[0129] This embodiment provides a broadband capacity optimization device, which includes the following modules:

[0130] A RIS-assisted no-cell network system model construction module is used to replace some base stations in the no-cell network with an intelligent reconfigurable surface (RIS) to construct a RIS-assisted no-cell network system model; wherein RIS communication is adopted between user pairs in the system model;

[0131] The joint precoding problem design module is used to design the joint precoding problem in RIS-assisted cell-free networks. By jointly optimizing the active precoding at the base station and the passive precoding at the RIS, the user sum rate is maximized. Auxiliary variables are introduced: the RIS phase shift matrix Θ and the precoding vector matrix W, transforming the joint precoding problem into an optimization problem involving Θ and W.

[0132] The deep reinforcement learning optimization module is used to optimize the phase shift matrix Θ and the precoding vector matrix W using the deep reinforcement learning algorithm, so that both Θ and W are optimal solutions to maximize the user's sum rate performance and achieve broadband capacity optimization.

[0133] The broadband capacity optimization device of this embodiment corresponds to the broadband capacity optimization method of the first embodiment mentioned above; wherein, the functions implemented by each functional module in the broadband capacity optimization device of this embodiment correspond one-to-one to each process step in the broadband capacity optimization method of the first embodiment mentioned above; therefore, they will not be repeated here.

[0134] Third embodiment

[0135] This embodiment provides an electronic device, which includes a processor and a memory; wherein the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the method of the first embodiment.

[0136] The electronic device may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) and one or more memories, wherein the memory stores at least one instruction, which is loaded by the processor to execute the above method.

[0137] Fourth embodiment

[0138] This embodiment provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the method of the first embodiment described above. The computer-readable storage medium may be a ROM, random access memory, CD-ROM, magnetic tape, floppy disk, or optical data storage device. The instructions stored therein can be loaded by a processor in a terminal to execute the method described above.

[0139] Furthermore, it should be noted that the present invention may be provided as a method, apparatus, or computer program product. Thus, embodiments of the present invention may take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media containing computer-usable program code.

[0140] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the process in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0141] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for implementing the process in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0142] It should also be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or terminal device comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or terminal device comprising the element.

[0143] Finally, it should be noted that the above is a preferred embodiment of the present invention. It should be noted that although the preferred embodiment of the present invention has been described, it is clear that those skilled in the art, once they understand the basic inventive concept of the present invention, can make various improvements and modifications without departing from the principles of the present invention. Such improvements and modifications should also be considered as within the scope of protection of the present invention. Therefore, the appended claims are intended to be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the embodiments of the present invention.

Claims

1. A broadband capacity optimization method, characterized in that: include: The intelligent reconfigurable surface (RIS) is used to replace some base stations in a cell-free network, and a RIS-assisted cell-free network system model is constructed. In the system model, RIS communication is used between user pairs. The joint precoding problem in a RIS-assisted cell-free network system is designed. By jointly optimizing the active precoding at the base station and the passive precoding at the RIS, the user sum rate is maximized. Auxiliary variables are introduced: the RIS phase shift matrix Θ and the precoding vector matrix W, transforming the joint precoding problem into an optimization problem of Θ and W. The proximal policy optimization (PPO) algorithm in deep reinforcement learning is used to optimize the phase shift matrix Θ and the precoding vector matrix W, respectively, so that both Θ and W are optimal, thereby maximizing user sum rate performance and achieving broadband capacity optimization, including: Step 1: Initialize wireless device information, user information, and agent environment information, including: base station maximum transmission power P max , user weight η k , as well as agent actions and states; Step 2, repeatedly perform the following process: the current state s t Input actor-new network and get action a t , and then input it into the environment to get the reward r t and the next state s t′ ; Until the preset number of {s t ,a t ,r t }; The equivalent channel from base station b to user k on subcarrier p is expressed as: Where, G b,r,k 、 are the frequency domain channels from base station b to user k, from base station b to RISr, and from RISr to user k on subcarrier p, respectively; is the phase shift matrix of RISr, R is the number of RIS; Consider a Gaussian channel in the current state, let z k,p represents additive white Gaussian noise, then the received signal is: Where y b,k,p is the baseband frequency domain signal from base station b to user k on subcarrier p, W b,p,j is the precoding vector of base station b, s p,j For w b,p,j Precoded frequency domain signal; B is the number of base stations; K is the number of users; Step 3: Design the objective function for finding the number of users and the rate; Among them, the signal s of subcarrier p on user k is p,k The signal-to-noise ratio is expressed as: Where, Ξ k,p is the variance of additive white Gaussian noise; From this, the user's weighted sum rate expression is derived, that is, the objective function is: Where P is the number of subcarriers; η k is the user weight; γ k,p is the signal s of subcarrier p on user k p,k signal-to-noise ratio; Two constraints are required to make the objective function valid: in, is the feasible set of RIS reflection coefficients; θ r,n is the reflection coefficient of RIS; Step 4: All stored s t ,a t ,r t Combine the inputs into the critic network and calculate the advantage function: The advantage function is calculated by the following formula: Where γ is the discount factor, s t is the current state, s t′ is the next state, a t is the current action, r t′ is the objective function, γ t′-t is the difference between the discount factor at the next moment and the current moment, f′(Θ,W) is the objective function at the next moment; t′ is the next moment, t is the current moment, π(s t ,a t ) is the strategy generated by the current environment and action, P(s t′ |s t ,a t ) is the probability of generating new environmental information under the current environment and action; V Φ (s t ) is the value function, which is calculated by the following formula: Step 5: Obtain instant rewards based on the Bellman equation for learning, and use the PPO algorithm to implement alternating optimization learning of Θ and W to obtain the optimal solutions of Θ and W; to maximize the user's sum rate performance.

2. The broadband capacity optimization method according to claim 1, wherein: The RIS-assisted cell-free network system is a discrete time slot system, and the RIS-assisted cell-free network system model is a Markov decision model.

3. The broadband capacity optimization method according to claim 1, wherein: The PPO algorithm is used to implement alternating optimization learning of Θ and W to obtain the optimal solutions of Θ and W, including: The precoding vector matrix W is optimized and the calculation formula is as follows: Where y(W) represents the actual value of the objective function, represents the constraint function; P W (a t |s t ) indicates that when W is not optimized, the environment information s t Next action a t The probability of Indicates that W is optimized to W k Later in the environmental information t Next action a t The probability of Indicates that when optimized to W k The advantage function after ε represents a parameter that limits the difference between the new and old strategies; When the algorithm converges, terminate the optimization process and record the optimized solution W opt ; Optimize the phase shift matrix Θ and solve the optimal solution W opt Substitute the objective function f(Θ,W) and then optimize Θ. When the algorithm converges, terminate the optimization process and obtain the optimal solution Θ opt .

4. A broadband capacity optimization device, characterized in that: include: A RIS-assisted no-cell network system model construction module is used to replace some base stations in the no-cell network with an intelligent reconfigurable surface (RIS) to construct a RIS-assisted no-cell network system model; wherein RIS communication is adopted between user pairs in the system model; The joint precoding problem design module is used to design the joint precoding problem in RIS-assisted cell-free networks. By jointly optimizing the active precoding at the base station and the passive precoding at the RIS, the user sum rate is maximized. Auxiliary variables are introduced: the RIS phase shift matrix Θ and the precoding vector matrix W, transforming the joint precoding problem into an optimization problem involving Θ and W. The deep reinforcement learning optimization module is used to optimize the phase shift matrix Θ and the precoding vector matrix W using the proximal policy optimization (PPO) algorithm in deep reinforcement learning, respectively, to optimize both Θ and W to maximize user sum and rate performance and achieve broadband capacity optimization. This includes: Step 1: Initialize wireless device information, user information, and agent environment information, including: base station maximum transmission power P max , user weight η k , as well as agent actions and states; Step 2, repeatedly perform the following process: the current state s t Input actor-new network and get action a t , and then input it into the environment to get the reward r t and the next state s t′ ; Until the preset number of {s t ,a t ,r t }; The equivalent channel from base station b to user k on subcarrier p is expressed as: Where, G b,r,p 、 are the frequency domain channels from base station b to user k, from base station b to RISr, and from RISr to user k on subcarrier p, respectively; is the phase shift matrix of RISr, R is the number of RIS; Consider a Gaussian channel in the current state, let z k,p represents additive white Gaussian noise, then the received signal is: Where y b,k,p is the baseband frequency domain signal from base station b to user k on subcarrier p, w b,p,j is the precoding vector of base station b, s p,j For w b,p,j Precoded frequency domain signal; B is the number of base stations; K is the number of users; Step 3: Design the objective function for finding the number of users and the rate; Among them, the signal s of subcarrier p on user k is p,k The signal-to-noise ratio is expressed as: Where, Ξ k,p is the variance of additive white Gaussian noise; From this, the user's weighted sum rate expression is derived, that is, the objective function is: Where P is the number of subcarriers; η k is the user weight; γ k,p is the signal s of subcarrier p on user k p,k signal-to-noise ratio; Two constraints are required to make the objective function valid: in, is the feasible set of RIS reflection coefficients; θ r,n is the reflection coefficient of RIS; Step 4: All stored s t ,a t ,r t Combine the inputs into the critic network and calculate the advantage function: The advantage function is calculated by the following formula: Where γ is the discount factor, s t is the current state, s t′ is the next state, a t is the current action, r t′ is the objective function, γ t′-t is the difference between the discount factor at the next moment and the current moment, f′(Θ,W) is the objective function at the next moment; t′ is the next moment, t is the current moment, π(s t ,a t ) is the strategy generated by the current environment and action, P(s t′ |s t ,a t ) is the probability of generating new environmental information under the current environment and action; V Φ (s t ) is the value function, which is calculated by the following formula: Step 5: Obtain instant rewards based on the Bellman equation for learning, and use the PPO algorithm to implement alternating optimization learning of Θ and W to obtain the optimal solutions of Θ and W; to maximize the user's sum rate performance.

5. The broadband capacity optimization device according to claim 4, characterized in that: The RIS-assisted cell-free network system is a discrete time slot system, and the RIS-assisted cell-free network system model is a Markov decision model.

6. The broadband capacity optimization device according to claim 4, wherein: The PPO algorithm is used to implement alternating optimization learning of Θ and W to obtain the optimal solutions of Θ and W, including: The precoding vector matrix W is optimized and the calculation formula is as follows: Where y(W) represents the actual value of the objective function, represents the constraint function; P W (a t |s t ) indicates that when W is not optimized, the environment information s t Next action a t The probability of Indicates that W is optimized to W k Later in the environmental information t Next action a t The probability of Indicates that when optimized to W k The advantage function after ε represents a parameter that limits the difference between the new and old strategies; When the algorithm converges, terminate the optimization process and record the optimized solution W opt ; Optimize the phase shift matrix Θ and solve the optimal solution W opt Substitute the objective function f(Θ,W) and then optimize Θ. When the algorithm converges, terminate the optimization process and obtain the optimal solution Θ opt .

Images