Information sending method of wireless communication system and related equipment

Abstract

The embodiment of the invention provides an information sending method of a wireless communication system and related equipment, the wireless communication system comprises an information transmitting end, an intelligent reflecting surface and at least one information receiving end, and the method comprises the following steps: firstly, obtaining preorder signal-to-noise ratios of all information receiving ends at the last moment, and obtaining a reflecting surface phase model; then, all the preorder signal-to-noise ratios are input into the reflecting surface phase model, an output optimized phase shift sequence is obtained, and the optimized phase shift sequence is used for optimizing the information receiving error rate of an information receiving end; and finally, after at least one reflection unit of the intelligent reflection surface is adjusted according to the optimized phase shift sequence, the target information is sent to the intelligent reflection surface, so that the intelligent reflection surface reflects the target information to an information receiving end. The reliability of information transmission between a transmitting end and a receiving end can be effectively improved in a wireless communication system provided with an intelligent reflecting surface.

Description

Technical Field

[0001] The present application relates to the field of wireless communication technology, and in particular to an information sending method and related equipment of a wireless communication system. Background Art

[0002] In recent years, with the development of wireless communication technology, the information transmission requirements in wireless communication systems have been further improved, such as transmission reliability. However, due to the instability of wireless communication channels, it is difficult for information transmission from the transmitter to the receiver to meet the high information transmission requirements. Smart reflective surfaces usually use a large number of reconfigurable reflective elements to control the propagation path of electromagnetic waves, thereby effectively improving the quality of information transmission between the transmitter and the receiver to meet the increasingly high information transmission requirements.

[0003] In the related art, for information transmission in a wireless communication system provided with an intelligent reflective surface, instantaneous channel state information between a transmitting end, an intelligent reflective surface and a receiving end is usually first obtained, and then a phase decision of a reflection unit of a suitable intelligent reflective surface is planned and adjusted according to the instantaneous channel state information, and then the adjusted intelligent reflective surface is used to reflect the information sent by the transmitting end to the receiving end, so as to improve the reliability of the information received by the receiving end. However, due to the time-varying nature of the channel in the wireless communication system and the large number of reflection units of the intelligent reflective surface, it is usually not practical to obtain an accurate instantaneous channel state at each moment of channel change in the wireless communication system, and because an error will inevitably occur between the estimated channel state and the actual channel state, the estimated channel state is used as the basis for generating the phase decision of the intelligent reflective surface, and when the intelligent reflective surface is scheduled according to the phase decision for communication between the information transmitting end and the information receiving end, the error between the actual channel state and the estimated channel state may cause the information receiving end to be unable to completely receive the data information sent by the information transmitting end, thereby resulting in insufficient reliability of information transmission in the wireless communication system provided with an intelligent reflective surface. Summary of the invention

[0004] The embodiments of the present application provide an information sending method and related equipment of a wireless communication system, which can improve the reliability of information transmission between a transmitting end and a receiving end in a wireless communication system provided with a smart reflective surface.

[0005] To achieve the above-mentioned object, a first aspect of an embodiment of the present application provides an information sending method of a wireless communication system, wherein the wireless communication system includes an information transmitting end, a smart reflecting surface, and at least one information receiving end, and the method includes:

[0006] Obtaining the preamble signal-to-noise ratio of all the information receiving ends at the previous moment, and obtaining the reflection surface phase model;

[0007] Inputting all the preamble signal-to-noise ratios into the reflection surface phase model to obtain an output optimized phase shift sequence, wherein the optimized phase shift sequence is used to optimize the information reception bit error rate of the information receiving end;

[0008] After at least one reflection unit of the smart reflection surface is adjusted according to the optimized phase shift sequence, target information is sent to the smart reflection surface, so that the smart reflection surface reflects the target information to the information receiving end.

[0009] In some embodiments, obtaining the reflection surface phase model includes:

[0010] Acquire the number of code elements, modulation order, bit error parameters at a given moment, and acquire the phase shift parameters of the reflection unit;

[0011] Obtaining an estimated bit error rate of each of the information receiving ends based on the number of code elements at the time, the modulation order, and all the bit error parameters;

[0012] Generating a bit error rate reward model based on the estimated bit error rate, wherein the bit error rate reward model is used to reduce the receiving bit error rate of the information receiving end;

[0013] An initial reflection surface phase model is generated based on the phase shift parameter and the bit error rate reward model, and model training is performed on the initial reflection surface phase model to obtain the reflection surface phase model.

[0014] In some embodiments, generating a bit error rate reward model based on the estimated bit error rate includes:

[0015] Acquire a bit error rate weighting factor of each of the information receiving ends, and accumulate the bit error rate weighting factor and the estimated bit error rate to obtain a weighted bit error rate of each of the information receiving ends;

[0016] All the weighted bit error rates are accumulated, and the logarithm is taken, and then the negative number is taken to obtain the bit error rate reward model.

[0017] In some embodiments, generating an initial reflection surface phase model based on the phase shift parameter and the bit error rate reward model includes:

[0018] Obtain the initial action parameters of the actor network and the initial comment parameters of the commentator network;

[0019] The initial reflection surface phase model is generated based on the initial action parameter, the initial comment parameter, the phase shift parameter and the preceding signal-to-noise ratio parameter.

[0020] In some embodiments, the performing model training on the initial reflection surface phase model to obtain the reflection surface phase model includes:

[0021] Inputting the previous training signal-to-noise ratio at the current training moment into the actor network to obtain a training action, and obtaining a current training signal-to-noise ratio according to the training action;

[0022] Inputting the current training signal-to-noise ratio into the bit error rate reward model to obtain a current training reward;

[0023] Generate the training experience information of the current training moment based on the previous training signal-to-noise ratio, the training action, the current training signal-to-noise ratio and the current training reward, and store the training experience information in an experience replay pool;

[0024] The initial action parameters and the initial comment parameters are updated using the training experience information of the batch training moments in the experience replay pool to obtain an updated actor network and commentator network, and the updated actor network is used as the reflector phase model.

[0025] In some embodiments, the updating of the initial action parameter and the initial comment parameter by using the training experience information of the batch training moments in the experience replay pool includes:

[0026] Based on a state value model, obtaining a training state value corresponding to the training experience information, wherein the state value model is generated based on the bit error rate reward model;

[0027] Obtaining a training time series difference residual and a training state value based on the training experience information and the training state value;

[0028] The initial action parameter is updated based on the training temporal difference residual, and the initial comment parameter is updated based on the training state value.

[0029] In some embodiments, updating the initial action parameter based on the training temporal difference residual, and updating the initial comment parameter based on the training state value, includes:

[0030] Obtaining an update strategy importance weight, and truncating the update strategy importance weight based on a preset truncation parameter to obtain a truncated importance weight;

[0031] performing gradient descent updating on the initial action parameters based on the truncated importance weights;

[0032] The initial comment parameter is updated by gradient descent based on the training state value.

[0033] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides an information sending device of a wireless communication system, wherein the wireless communication system includes an information transmitting end, a smart reflecting surface, and at least one information receiving end, and the device includes:

[0034] An acquisition module, used to acquire the preamble signal-to-noise ratio of all the information receiving ends at the previous moment, and acquire a reflection surface phase model;

[0035] An adjustment module, used for inputting all the preamble signal-to-noise ratios into the reflection surface phase model to obtain an output optimized phase shift sequence, wherein the optimized phase shift sequence is used for optimizing the information receiving bit error rate of the information receiving end;

[0036] The information sending module is used to send target information to the smart reflecting surface after at least one reflecting unit of the smart reflecting surface is adjusted according to the optimized phase shift sequence, so that the smart reflecting surface reflects the target information to the information receiving end.

[0037] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements the information sending method of the wireless communication system as described in the first aspect.

[0038] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium, and the storage medium stores a computer program. When the computer program is executed by a processor, it implements the information sending method of the wireless communication system described in the first aspect above.

[0039] The information sending method and related equipment of a wireless communication system proposed in an embodiment of the present application, the wireless communication system includes an information transmitting end, an intelligent reflecting surface and at least one information receiving end, and the method includes: first, obtaining the preamble signal-to-noise ratio of all information receiving ends at the previous moment, and obtaining the reflecting surface phase model; then, all the preamble signal-to-noise ratios are input into the reflecting surface phase model to obtain an output optimized phase shift sequence, and the optimized phase shift sequence is used to optimize the information receiving bit error rate of the information receiving end; finally, when at least one reflecting unit of the intelligent reflecting surface is adjusted according to the optimized phase shift sequence, the target information is sent to the intelligent reflecting surface, so that the intelligent reflecting surface reflects the target information to the information receiving end. The embodiment of the present application utilizes the preamble signal-to-noise ratio at the information receiving end at the previous moment to replace the instantaneous channel state as the decision basis, and there is no need to perform complex instantaneous channel state calculation or estimation, so as to reduce the calculation complexity and setting cost; and further utilizes the reflection surface phase model generated in advance according to the information reception bit error rate of the optimized information receiving end, and generates an optimized phase shift sequence of multiple reflection units of the intelligent reflection surface that can effectively reduce the information reception bit error rate of the information receiving end at the current moment based on the preamble signal-to-noise ratio and the reflection surface phase model, so as to utilize the adjusted intelligent reflection surface to forward the target information, which can effectively improve the information transmission reliability between the transmitting end and the receiving end in the wireless communication system provided with the intelligent reflection surface.

[0040] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a structural diagram of a wireless communication system provided in one embodiment of the present application.

[0042] Figure 2 This is a flowchart of an information sending method of a wireless communication system provided by another embodiment of the present application.

[0043] Figure 3 It is a schematic diagram of an operating framework of an algorithm 1 provided in yet another embodiment of the present application.

[0044] Figure 4 It is a structural diagram of an information sending device of a wireless communication system provided in another embodiment of the present application.

[0045] Figure 5 It is a schematic diagram of the hardware structure of an electronic device provided in yet another embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0047] It should be noted that although the functional modules are divided in the device schematic and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0049] First, some nouns involved in this application are analyzed:

[0050] Intelligent reflecting surfaces (IRS) can significantly improve communication quality at a low cost and are considered to be one of the most promising key technologies in the next generation of wireless communication systems. Specifically, IRS uses a surface composed of a large number of reconfigurable reflective elements to control the propagation path of electromagnetic waves. These reconfigurable reflective elements can adjust their phase and amplitude in real time, thereby accurately controlling the signals passing through the IRS, which can effectively improve the performance of communication systems with obstacles between the information transmitter and the information receiver.

[0051] Deep reinforcement learning is a method that combines the power of deep learning with the goal-oriented mechanism of reinforcement learning. It enables computer programs to learn how to complete specific tasks by interacting with the environment. In this process, the program called "agent" observes the state of its "environment" and performs actions based on the current state, and then adjusts its behavior strategy based on the reward signal given by the environment in order to obtain the maximum cumulative reward in the future. The key to deep reinforcement learning is to use deep neural networks to approximate the value function or direct strategy in complex environments, thereby overcoming the obstacles encountered by traditional reinforcement learning methods in high-dimensional or continuous state spaces. This method has been successfully applied to many fields such as board games (such as chess and go), video games, robot navigation, and complex control problems, where the agent not only needs to understand and interpret complex sensory inputs, but also make optimal decisions in dynamically changing environments.

[0052] In recent years, with the development of wireless communication technology, the information transmission requirements in wireless communication systems have been further improved, such as transmission reliability. However, due to the instability of wireless communication channels, it is difficult for information transmission from the transmitter to the receiver to meet the high information transmission requirements. Smart reflective surfaces usually use a large number of reconfigurable reflective elements to control the propagation path of electromagnetic waves, thereby effectively improving the quality of information transmission between the transmitter and the receiver to meet the increasingly high information transmission requirements.

[0053] In the related art, for information transmission in a wireless communication system provided with an intelligent reflective surface, instantaneous channel state information between a transmitting end, an intelligent reflective surface and a receiving end is usually first obtained, and then a phase decision of a reflection unit of a suitable intelligent reflective surface is planned and adjusted according to the instantaneous channel state information, and then the adjusted intelligent reflective surface is used to reflect the information sent by the transmitting end to the receiving end, so as to improve the reliability of the information received by the receiving end. However, due to the time-varying nature of the channel in the wireless communication system and the large number of reflection units of the intelligent reflective surface, it is usually not practical to obtain an accurate instantaneous channel state at each moment of channel change in the wireless communication system, and because an error will inevitably occur between the estimated channel state and the actual channel state, the estimated channel state is used as the basis for generating the phase decision of the intelligent reflective surface, and when the intelligent reflective surface is scheduled according to the phase decision for communication between the information transmitting end and the information receiving end, the error between the actual channel state and the estimated channel state may cause the information receiving end to be unable to completely receive the data information sent by the information transmitting end, thereby resulting in insufficient reliability of information transmission in the wireless communication system provided with an intelligent reflective surface.

[0054] In order to improve the reliability of information transmission between a transmitting end and a receiving end in a wireless communication system provided with an intelligent reflecting surface, an embodiment of the present application uses a preamble signal-to-noise ratio at an information receiving end at a previous moment instead of an instantaneous channel state as a decision basis, without the need to perform complex instantaneous channel state calculations or estimates, so as to reduce the computational complexity and setting costs; and further uses a reflecting surface phase model generated in advance based on an information reception bit error rate of an optimized information receiving end, and generates an optimized phase shift sequence of multiple reflecting units of an intelligent reflecting surface that can effectively reduce the information reception bit error rate of an information receiving end at a current moment based on the preamble signal-to-noise ratio and the reflecting surface phase model, thereby using the adjusted intelligent reflecting surface to forward target information, which can effectively improve the reliability of information transmission between a transmitting end and a receiving end in a wireless communication system provided with an intelligent reflecting surface.

[0055] In order to better describe the information transmission method of the wireless communication system provided by the present application, the following first describes the perception area of ​​the information transmission method applied to the wireless communication system. Figure 1 , is a schematic diagram of the structure of a wireless communication system provided in an embodiment of the present application. Figure 1 As shown, the wireless communication system includes an information transmitting end (i.e., a transmitter), multiple information receiving ends (i.e., receivers), and an intelligent reflecting surface (IRS). The information transmitting end is used to send data information to multiple information receiving ends, and there are obstacles in the direct link between the information transmitting end and the information receiving end, that is, the quality of the direct link between the information transmitting end and the information receiving end is poor. In addition, the information transmitting end is also used to send data information to the intelligent reflecting surface so that the intelligent reflecting surface reflects the data information to the information receiving end; and the information transmitting end is connected to the intelligent reflecting surface through an intelligent reflecting surface controller (i.e., an IRS controller), and the phases of multiple reflecting units on the intelligent reflecting surface are adjusted in real time through the intelligent reflecting surface controller.

[0056] Based on Figure 1 The wireless communication system shown in the figure, the information sending method of the wireless communication system in the embodiment of the present application will be described in detail below. Figure 2 , which is an optional flowchart of the information sending method of the wireless communication system provided in the embodiment of the present application, Figure 2 The method may include but is not limited to steps 100 to 300. It can also be understood that this embodiment Figure 2 The order of steps 100 to 300 is not specifically limited, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs. The information sending method of the wireless communication system can be applied to an information sending end or a processor or controller connected to the wireless communication system.

[0057] Step 100: Obtain the preamble signal-to-noise ratio of all information receiving ends at the previous moment, and obtain the reflection surface phase model.

[0058] The following is a detailed description of step 100.

[0059] In some embodiments, since the channel between the information transmitting end and the information receiving end is complex, and since during the process of data information reflection by the smart reflective surface, each reflective unit and the information transmitting end, as well as each reflective unit and the information receiving end are considered to have a communication link, therefore, in real time, if the instantaneous channel state between the information receiving end passing through the smart reflective surface and reaching the information receiving end is to be obtained, the hardware requirements are particularly high and it is difficult to achieve.

[0060] Therefore, in this embodiment, the ratio of the signal power to the noise power of the information receiving terminal at the previous moment (i.e., the preamble signal-to-noise ratio) is used to replace the instantaneous channel state at the current moment to make the basis for the phase decision of the reflection unit of the smart reflection surface at the current moment.

[0061] In addition, in order to further obtain the appropriate optimized phase decision at the current moment, a reflection surface phase model is constructed in advance based on the relevant data information of the wireless communication system and the deep reinforcement learning framework to determine the appropriate optimized phase shift sequence of the intelligent reflection surface according to different preamble signal-to-noise ratios and the phase decision of the intelligent reflection surface at the previous moment (i.e., the optimized phase shift sequence at the previous moment) to reduce the information reception bit error rate at the information receiving end. The following will further describe how to construct the reflection surface phase model.

[0062] The step of obtaining the reflection surface phase model includes the following steps 110 to 140.

[0063] Step 110: Obtain the number of code elements at a given time, the modulation order, the bit error parameter, and the phase shift parameter of the reflection unit.

[0064] Step 120: Obtain an estimated bit error rate of each information receiving end based on the number of code elements at a given moment, the modulation order, and all bit error parameters.

[0065] Steps 110 to 120 are described in detail below.

[0066] In some embodiments, the information sending end in the wireless communication system is defined as The smart reflective surface has N = N x ×N y Passive reflection units, where N x and N y They represent the number of rows and columns of reflective units in the smart reflective surface, and there are K information receiving ends. Each information receiving end is equipped with an antenna. and They respectively represent a collection of reflection units of an information receiving end and a smart reflection surface.

[0067] In some embodiments, from the information sending end The information data transmission to each information receiving end is carried out at multiple consecutive moments, and the length of each moment is τ seconds. At each moment, the channel of the system remains unchanged, but the channel at different moments is changing. Assuming that the channels at each moment are correlated, their correlation can be described by the Gaussian Markov block fading autoregressive model, that is, the channel coefficient vector h(t) at the tth moment and the channel coefficient vector h(t-1) at the t-1th moment satisfy the following relationship:

[0068]

[0069] Where ρ is the normalized correlation coefficient between the channel coefficient vector h(t) at the t-th time and the channel coefficient vector h(t-1) at the t-1th time, represents the error vector and I is the identity matrix.

[0070] Furthermore, let γ k (t) represents the received signal-to-noise ratio of the kth information receiving end at the tth moment, and let At the same time, represents the phase shift matrix of the reflection unit of the smart reflection surface. Where θ=[θ1,K,θ N ] represents the phase shift parameter, which is a vector, is the amplitude reflection coefficient of the nth reflection unit; without loss of generality, for any In this embodiment, it is set to α n =1. Therefore, the channel coefficient of the link from the information transmitter to the kth information receiver can be expressed as in Information transmitter The channel coefficient to the kth information receiving end, is the channel coefficient vector between multiple reflection units of the intelligent reflection surface and the kth information receiving end, is the information transmitter at the tth moment A link channel coefficient vector between multiple reflection units in the smart reflection surface.

[0071] Considering that in practical applications, the phase shift of the reflection unit of the smart reflection surface can usually only take discrete values, the phase shift of the reflection unit of the smart reflection surface is quantized with D bits, and the discrete phase shift value set of each reflection unit can be expressed as Ω = {0, Δθ, K(2 D -1)Δθ}, where Δθ=π / 2 (D-1) Represents discrete angle values.

[0072] Then, in this embodiment, the Gray coding and M-order modulation are adopted, and the modulation order is M. And the time length of each code element in the parameter data is τ s , then the information sender The number of code elements sent at each time is

[0073]

[0074] Let S i is the i-th code element transmitted at a time, then the corresponding code element received by the k-th information receiving end can be expressed as

[0075]

[0076] in is the additive white Gaussian noise at the kth information receiving end, N0 is the power spectral density of the additive white Gaussian noise, and B is the system bandwidth.

[0077] Assume that the information sender The transmission power is a constant, denoted by P T Watt. At the information receiving end k, each codeword S is decoded i The received signal-to-noise ratio can be expressed as Assume that the energy of each code element is a constant, denoted by E s . Due to P T =E s / τ s , so the received signal-to-noise ratio can also be expressed as Assume that the information sender When sending a symbol, the pulse shaping uses a raised cosine pulse with a roll-off factor β = 1, so the pulse shaping time τ of one symbol is s Satisfy τ s =1 / B, so we can get At this time, γ k That is, the symbol signal-to-noise ratio of the kth information receiving end, i.e., γ s,k In other words, Furthermore, let the bit signal-to-noise ratio at the information receiving end k be γ b,k , then we can get Where E b It represents the energy consumed when transmitting each bit of information.

[0078] Therefore, there is γ b,k =γ k / log2M. At the same time, since the bit error rate is determined by the bit signal-to-noise ratio, the bit error rate at the information receiving end k can be expressed as P b,k (γ b,k ). In other words, the bit error rate at the information receiving end k can also be expressed as P b,k (γ k ), that is, the bit error rate at the information receiving end k can also be calculated by the received signal-to-noise ratio γ k For example, when QPSK is used, the estimated bit error rate is or in is the complementary cumulative distribution function of the standard normal distribution.

[0079] Based on the estimated bit error rate P obtained above b,k (γ k ), at each moment, the phase shift of each reflective unit of the smart reflective surface is optimized according to the received signal-to-noise ratio at each information receiving end to minimize the sum of the weighted bit error rates of all information receiving ends. At the tth moment, the corresponding optimization problem can be expressed as

[0080]

[0081] where λk is the bit error rate weighting factor of the kth information receiving end, and satisfies is the received signal-to-noise ratio of the kth information receiving end at the tth moment, is the phase shift of the nth reflection unit of the smart reflection surface at the tth moment.

[0082] According to the traditional existing research methods, in order to solve problem (P1), it is usually necessary to use the statistical information of the channel state information to derive the explicit expression of the bit error rate, and then use the traditional optimization theory (such as convex optimization method) to solve the optimization problem based on the explicit expression obtained. However, in this application, since the CSI is unknown, it is impossible to obtain , so it is impossible to use traditional research methods to solve problem (P1). To address this problem, the present invention proposes an optimization algorithm for solving problem (P1) based on a deep reinforcement learning method, and analyzes the applicability of the optimization algorithm.

[0083] Based on this, in an embodiment of the present application, deep reinforcement learning is used to solve problem (P1). First, an MDP model is constructed based on problem (P1), as shown below.

[0084] An MDP consists of four tuples Among them, is the state space, i.e., the set of environmental states; is the action space, that is, the set of agent actions; r is the reward, whose value is determined by the state s and the action a; γ∈[0,1] is the discount factor that reflects the impact of the reward obtained by the agent at future moments on the reward of the current state.

[0085] Since the optimization variable of problem (P1) is defined as the phase shift of the reflection unit of the smart reflection surface, the optional action at the tth moment is defined as in represents the phase shift of the nth reflection unit at the tth moment. By calculation, it is easy to get the action space The dimension is 2 DN .

[0086] In the MDP model, the state describes the characteristic state of the wireless communication system and the relevant information that the information transmitter can observe in this state. In this embodiment, since the received signal-to-noise ratio can only be obtained by measurement at the information receiving end, the time-varying received signal-to-noise ratio at the information receiving end reflects the information transmitting end. The channel quality of the communication link between the receiver and the information receiving end. In practical applications, measuring the signal-to-noise ratio will produce a certain delay, and the wireless channel state is often very time-varying, and the time (i.e., a moment) that the channel remains unchanged is very short. Therefore, if the phase shift of the reflective unit of the smart reflective surface is optimized after the receiver measures the received signal-to-noise ratio at the current moment, and information is transmitted based on the phase shift optimization result, it will not be completed within a very short moment, and the system performance consistent with the theoretical expected results cannot be obtained.

[0087] Therefore, in this embodiment, the received signal-to-noise ratio (i.e., the preamble signal-to-noise ratio) of the information receiving end at the previous moment and the phase decision of the smart reflector at the previous moment (i.e., the optimized phase shift sequence at the previous moment) are used as the state space of the MDP model, which can reflect the channel state at the previous moment; and since the channels at each moment are related in the actual wireless communication system, the preamble signal-to-noise ratio at the previous moment also reflects the current channel state. Therefore, in the training process of the reflection surface phase model under consideration, at the tth moment, its state is composed of the preamble signal-to-noise ratio of all information receiving ends at the t-1th moment and the phase shift vector of the smart reflector at the t-1th moment, i.e., s t ={Γ t-1 ,a t-1}.in, represents the received SNR of the receiver at time t-1, a t-1 represents the action taken by the agent at the t-1th moment (a random phase shift vector can be used initially). After calculation, the state space is easily obtained The dimension is K+N.

[0088] In this MDP model, the definition of the reward function is determined by the objective function of problem (P1). Since the goal of problem (P1) is to minimize the sum of the weighted bit error rates of all information receivers, the reward function at the tth moment is defined as

[0089]

[0090] The appropriate constant factor ξ and lg(·) function are used to expand the bit error rate performance gap to improve the learning efficiency and stability of the algorithm and avoid falling into the local optimum. As mentioned above, since the channel state information at the current moment is unknown, that is, the bit error rate About is an implicit function of , so its analytical expression cannot be obtained.

[0091] However, in the embodiment of the present application, it is noted that in addition to calculating by using the channel state information, the bit error rate of the wireless communication system can also be obtained by experimental measurement based on the Monte Carlo method. Specifically, in the present embodiment, in the wireless communication system, at the tth moment, in order to measure its bit error rate through experiments, the information transmitting end can be made Broadcast L bits of information to multiple information receiving terminals. According to the above formula (2), the information transmitting terminal I = L / (log2M) symbols need to be broadcast. Let the i-th symbol received at the k-th information receiving end at the t-th time be So according to the Whether it can be correctly decoded or not, the error parameter of each symbol can be obtained as shown in the following formula (6).

[0092]

[0093] Then at the tth time, the symbol error rate can be estimated as

[0094]

[0095] At the same time, due to the use of Gray coding, a symbol decoding error is equivalent to a bit error, and the estimated bit error rate can be expressed as Therefore, based on the number of symbols I at the time, the modulation order M and all bit error parameters The estimated bit error rate of each information receiving end is obtained as shown in the following formula (8).

[0096] Step 130: Generate a bit error rate reward model based on the estimated bit error rate.

[0097] Step 130 is described in detail below.

[0098] In some embodiments, based on this formula (8), the estimated bit error rate can be used Replace the bit error rate part of the above reward model formula Thus, a suitable bit error rate reward model is obtained, which is described as follows.

[0099] The bit error rate reward model is generated based on the estimated bit error rate, including the following steps 131 to 132.

[0100] Step 131: Obtain a bit error rate weighting factor of each information receiving end, and cumulatively multiply the product of the bit error rate weighting factor and the estimated bit error rate to obtain a weighted bit error rate of each information receiving end.

[0101] Step 132: All weighted bit error rates are accumulated, and the logarithm is taken, and then the negative number is taken to obtain a bit error rate reward model.

[0102] Next, step 131 to step 132 are described.

[0103] In some embodiments, the bit error rate weighting factor λ corresponding to each information receiving end is first determined. k , and then multiply the bit error rate weighting factor λ of each information receiving end k and estimated bit error rate Get the weighted bit error rate of each information receiving end

[0104]

[0105] Then, all weighted bit error rates are accumulated, their logarithms are taken, and then the negative numbers are taken to obtain the bit error rate reward model as shown in the following formula (9).

[0106]

[0107] Step 140: Generate an initial reflection surface phase model based on the phase shift parameter and the bit error rate reward model, perform model training on the initial reflection surface phase model, and obtain a reflection surface phase model.

[0108] Step 140 is described in detail below.

[0109] In some embodiments, after determining a suitable bit error rate reward model, further combining the action space shown above That is, the phase shift parameter of the smart reflector, and the state space s t ={Γ t-1 ,a t-1}, the preamble signal-to-noise ratio of all information receiving ends at the t-1th moment, the phase shift vector of the intelligent reflection surface at the t-1th moment, and the initial reflection surface phase model are generated based on the deep reinforcement learning architecture, which will be further described in detail below.

[0110] The initial reflection surface phase model is generated based on the phase shift parameter and the bit error rate reward model, including the following steps 141 to 142.

[0111] Step 141: Obtain initial action parameters of the actor network and initial comment parameters of the commentator network.

[0112] Step 142: Generate an initial reflection surface phase model based on the initial action parameter, the initial comment parameter, the phase shift parameter, the bit error rate reward model, and the preceding signal-to-noise ratio parameter.

[0113] Steps 141 to 142 are described in detail below.

[0114] In some embodiments, since the action space dimension in the above MDP model is 2 DN, which increases exponentially with the increase in the number of reflective units of the intelligent reflective surface. Traditional deep reinforcement learning algorithms (such as DQN, i.e. DeepQNetwork algorithm) cannot handle MDP problems with large action space dimensions.

[0115] It can be understood that, unlike the DQN method based on value function, the PPO algorithm is a policy-based method, which consists of an actor network for outputting policies and a critic network for estimating value functions (using action parameters and the critic parameter ω parameterizes the corresponding actor and critic networks).

[0116] Therefore, after obtaining the action space, state space, and bit error rate reward model, it is also necessary to obtain the initial action parameters of the actor network. and the initial comment parameter ω0 of the critic network, and then based on the action space (i.e., the phase shift parameter of the intelligent reflector), the state space (i.e., the preamble signal-to-noise ratio of all information receiving ends at the previous moment, the phase decision of the intelligent reflector at the previous moment) (10), the initial action parameters and the initial comment parameters, the initial reflector phase model is generated based on the PPO algorithm framework.

[0117] Then, a large number of training samples are used to train the initial reflection surface phase model to adjust the relevant parameters in the initial reflection surface phase model (such as initial action parameters, initial comment parameters, action strategies, etc.) so as to obtain a reflection surface phase model that can accurately obtain a suitable optimized phase shift sequence based on different previous signal-to-noise ratios and the phase decision of the intelligent reflection surface at the previous moment in practical applications. The following further describes how to train the initial reflection surface phase model.

[0118] The initial reflection surface phase model is trained to obtain the reflection surface phase model, including the following steps 150 to 180.

[0119] Step 150: Input the previous training signal-to-noise ratio at the current training moment into the actor network to obtain a training action, and obtain the current training signal-to-noise ratio based on the training action.

[0120] Step 160: Input the current training signal-to-noise ratio into the bit error rate reward model to obtain the current training reward.

[0121] Step 170: Generate training experience information of the current training moment based on the previous training signal-to-noise ratio, training actions, current training signal-to-noise ratio, and current training rewards, and store the training experience information in an experience replay pool.

[0122] Step 180: Use the training experience information of the batch training moments in the experience replay pool to update the initial action parameters and the initial comment parameters to obtain an updated actor network and commentator network, and use the updated actor network as the reflection surface phase model.

[0123] Steps 150 to 180 are described in detail below.

[0124] In some embodiments, after obtaining the initial reflection surface phase model, the model is trained using a large number of training samples at consecutive moments, and at each training moment, the previous state s of the current training moment t is converted into t (including the previous training signal-to-noise ratio and reflection unit phase of the previous moment) is input into the actor network; the actor network uses the current action strategy The output corresponds to the previous state s t The training action a at the current training moment t , to determine the training action a t The corresponding current training signal-to-noise ratio.

[0125] Then, the updated state s at the next training moment is calculated based on the current training signal-to-noise ratio t+1 (which includes the current training signal-to-noise ratio at the current training moment and the phase decision of the intelligent reflector), and inputs the current training signal-to-noise ratio into the bit error rate reward model (9) through Monte Carlo to obtain the corresponding current training reward r t .

[0126] Next, the training experience information {s t ,a t ,s t+1 ,r t}, and store the training experience information in the experience replay pool, so that the initial action parameters and the initial comment parameters can be updated later using the training experience information of the batch training moments in the experience replay pool to obtain the updated actor network and commentator network, as described below.

[0127] The initial action parameters and initial comment parameters are updated using the training experience information of the batch training moments in the experience replay pool, including the following steps 181 to 183.

[0128] Step 181: Based on the state value model, obtain the training state value corresponding to the training experience information.

[0129] Step 182: Obtain training time difference residuals and training state values ​​based on the training experience information and the training state values.

[0130] Step 183: Update the initial action parameters based on the training temporal difference residuals, and update the initial comment parameters based on the training state values.

[0131] Steps 181 to 183 are described in detail below.

[0132] In some embodiments, in the adopted PPO algorithm framework, the strategy is updated by a gradient-based method Then, the state value function model is updated. The value function model is generated based on the bit error rate reward model, as shown in the following formula (10).

[0133]

[0134] Its meaning is from the current state s t Get started, follow the strategy The expected value of the reward received.

[0135] Therefore, at the current training moment, based on the state value model, the training experience information {s t ,a t ,s t+1 ,r t}, which includes the corresponding training experience information s obtained by the actor network under the new decision t The first training state value The critic network obtains s in the corresponding training experience information under the old decision t The second training state value The critic network obtains s in the corresponding training experience information under the old decision t+1 The third training state value The corresponding training experience information s obtained by the critic network under the new comment parameters t The fourth training state value V(s t ; ω), and the corresponding training experience information s obtained by the critic network under the old comment parameters t+1 The fifth training state value V(s t+1 ;ω old ).

[0136] Then, based on the training experience information {s t ,a t ,s t+1 ,r t The current training reward r in} t , the second training state value The third training state value The training time difference residual is obtained as shown in the following formula (11).

[0137]

[0138] Among them, γ is the discount factor. Next, generate the action parameters corresponding to the initial action parameters Further based on action parameters Update strategy importance weights and training temporal difference residual The product of generates the action parameter optimization model under the PPO framework as shown in the following formula (12).

[0139]

[0140] Represents the parameters before each round of training (can be regarded as initial action parameters), it can be understood that, in this embodiment, the initial action parameters and the initial comment parameters can represent the network parameters of the actor network and the critic network initialized before the initial reflection surface phase model is trained, and can also be represented as the network parameters of the actor network and the critic network before each round of the initial reflection surface phase model training process.

[0141] Then, based on the training experience information {s t ,a t ,s t+1 ,r t The current training reward r in} t 、The fifth training state value V(s t+1 ;ω old )The training state value is obtained as shown in the following formula (13).

[0142] V tar (s t )=r t +γV(s t+1 ;ω old ) (13)

[0143] Then, the initial action parameters are updated using the training temporal difference residuals, and the initial comment parameters are updated using the training state values, as described below.

[0144] The updating of the initial action parameters based on the training time-series difference residual includes the following steps 184 to 186.

[0145] Step 184: Obtaining the update strategy importance weight, and truncating the update strategy importance weight based on a preset truncation parameter to obtain a truncated importance weight;

[0146] Step 185: performing gradient descent update on the initial action parameters based on the truncated importance weights;

[0147] Step 186: Perform gradient descent update on the initial comment parameters based on the training state value.

[0148] Steps 184 to 186 are described in detail below.

[0149] In some embodiments, based on the current training moment, the actor network is trained based on the training experience information {s t ,a t ,s t+1 ,r t}Old policy before update and the update strategy after the update So as to get the update strategy importance weight

[0150] Then, in order to improve the stability of training, this application adopts the PPO truncation method to constrain the change between the new and old strategies not to be too large, that is, Restricted to a safe region [1-ε,1+ε] to obtain truncated importance weights As shown in the following formula (14).

[0151]

[0152] The preset truncation parameters are: The definition is shown in the following formula (15).

[0153]

[0154] Among them, ε is a hyperparameter that controls the truncation range in the PPO algorithm. As the gradient is updated, regardless of the training time difference residual Positive and negative, which can ensure that the target strategy does not deviate too far from the behavioral strategy; It is the policy entropy with coefficient η, which is used to ensure sufficient exploration of the PPO algorithm.

[0155] Next, the initial action parameters are updated by gradient descent based on maximizing the truncated importance weight (14) as shown in the following formula (16).

[0156]

[0157] in, Represents the learning rate of the actor network. For the critic network, the initial critic parameter ω is updated by gradient descent by minimizing the mean square error using the training state value (13) as shown in the following formula (17).

[0158]

[0159] Among them, αω represents the learning rate of the critic network.

[0160] In addition, in order to solve the problem of too high dimension of MDP action space in the training process of the initial reflection surface phase model, the present application adopts the action combination method in the PPO algorithm, that is, by combining some smaller independent actions in the original MDP problem into one action, so as to reduce the dimension of the action space and thus reduce the number of neurons in the output layer of the actor network. Specifically, in the training process of the present application, let in That is, phase shift each reflective unit of the smart reflective surface As an action, t is the combination of actions corresponding to the phase shift of each reflective unit of the intelligent reflective surface. Then, the actor network does not need to output π(a t |s t ), only the simplified combined action needs to be output as shown in the following formula (18).

[0161]

[0162] in and Indicates that in state s t In this case, the phase shift value of the nth reflection unit of the smart reflection surface is Then, the information transmitter right By performing random sampling, the phase shift of the nth reflection unit of the smart reflection surface can be obtained, that is, Finally, by You can get

[0163] According to the combined action t The definition of There are 2 possible values D There are N such small actions in total, so the number of output layer neurons required by the actor network is 2 D N. Therefore, by using the action combination method, the number of neurons in the output layer of the actor network can be increased from 2 DN Reduced to 2 D N, thus greatly reducing the network size. Therefore, the PPO algorithm proposed in the present invention can be used to solve the MDP problem of large-scale discrete action space.

[0164] Based on the training process of the above initial reflection surface phase model, the following algorithm 1 can be further formed.

[0165] Algorithm 1. Phase shift optimization algorithm for IRS-assisted broadcast communication system based on SNR and exempted from CSI.

[0166] Input: reward discount factor γ, experience replay size C B and learning rate and α ω .

[0167] Initialization: Actor Network Parameters Critic network parameters ω and experience replay pool with capacity C

[0168] Output: Actor network parameters

[0169] 1) for espisode=1,2,...,E do

[0170] 2) Initialize the wireless communication system environment and obtain the initial state s0;

[0171] 3) for t=1,2,...,Tdo

[0172] 4) cnt+=1;

[0173] 5) According to the state s t , the Actor network outputs the strategy after the action combination Then Get action a for sampling t ;

[0174] 6) According to action a t Calculate the next state s t+1 , and calculate the reward r based on formula (9) through Monte Carlo simulation t ;

[0175] 7)Store experience information t ,a t ,s t+1 ,r t} to the experience replay pool

[0176] 8) if cnt%C==0 then

[0177] 9) Use The empirical information in is calculated according to formulas (11) and (13) respectively: and V tar (s t );

[0178] 10) for epoch = 1, 2, ..., J do

[0179] 11) Disrupt the experience replay pool Experience information in

[0180] 12) for k = 1, 2, ..., C / C B do

[0181] 13) From the experience replay pool Extracting small batches of experience information And update the parameters respectively through equations (16) and (17) and ω;

[0182] 14) end for

[0183] 15) end for

[0184] 16) Clear experience data;

[0185] 14) end for

[0186] 15) end for

[0187] Reference Figure 3 , is a schematic diagram of the operating framework of an algorithm 1 provided in an embodiment of the present application. Figure 3 As shown in , in the initial stage of training, the parameters are first generated as and ω of the actor network and critic network, and initialize the parameters. At the same time, build an experience replay pool with a capacity of C At the tth moment of each scene, the information transmitter First, based on the preamble signal-to-noise ratio (i.e., Γ t-1 ), the phase shift of each reflective unit of the smart reflective surface (i.e. a t-1 ), get the current state s t Then, the current state s t Input the actor network, and after combining actions and sampling the probability distribution, we can get action a t (i.e., the phase shift of the reflective unit of the intelligent reflective surface at the tth moment). According to the action a output by the neural network t , information transmitter The intelligent reflector controller adjusts the phase shift of the reflector unit of the intelligent reflector to change the signal transmission environment. L information bits are broadcast to all information receiving ends, and each information receiving end calculates the bit error rate (i.e. ) and measure the received signal-to-noise ratio (i.e. ),

[0188] And the obtained bit error rate and signal-to-noise ratio are fed back to the information transmitter Information transmitter According to the bit error rate and signal-to-noise ratio fed back by each information receiving end, the reward r is calculated using formula (9): t , and according to a t The next state s can be obtained by the received signal-to-noise ratio fed back by each information receiving end. t+1 Therefore, the information transmitter A record of interaction with the wireless communication system can be obtained. t ,a t ,r t ,s t+1} and store it in the experience replay pool It is used to train the initial reflection surface phase model.

[0189] In order to improve the reliability of the information transmission method of the wireless communication system provided by the present application, the computational complexity in the implementation process of the above-mentioned algorithm 1 is further analyzed below. Based on algorithm 1, it is assumed that the actor network and the critic network in the initial reflection surface phase model have and I ω The number of neurons in each layer is and In addition, the number of neurons in the activation layer 'tanh' in the actor network and the critic network are and

[0190] For each individual action prediction (i.e., given the previous signal-to-noise ratio and the phase decision of the smart reflective surface at the previous moment to obtain the reflection phase shift of the smart reflective surface at the current moment), the complexity is only caused by the actor network in the reflective surface phase model. The prediction complexity in practical applications can be calculated as shown in the following formula (19).

[0191]

[0192] The computational effort required to train the initial reflector phase model is caused by backpropagation, where a single 'tanh' neuron requires 6 floating point operations. Therefore, the contribution of a single backpropagation to the complexity is Both the actor network and the critic network need to be trained. It is worth noting that the results need to be predicted during the training process, and the complexity is calculated as In addition, when executing step 6 in Algorithm 1, the information transmitter It is necessary to broadcast L bits of information to multiple information receiving ends, and its complexity is recorded as C delay , so the total training complexity during the training process is shown in the following formula (20).

[0193]

[0194] When training DNN in a real environment, C delay It can be measured by the maximum delay in transmitting bits, which can be calculated as Where R and c represent the transmission rate of the system and the speed of light respectively.

[0195] For example, L = 10 5 , R = 1Gbps (corresponding to 802.11ax (WIFI 6) standard), C delay ≈0.1ms. On the other hand, for numerical simulation using computer, C delay The complexity can be reduced by multiplication through the wireless channel from The signal symbol transmission to the user is expressed as

[0196] Step 200: input all preamble signal-to-noise ratios into the reflection surface phase model to obtain an output optimized phase shift sequence.

[0197] The following is a detailed description of step 200.

[0198] In some embodiments, after obtaining a trained reflection surface phase model, the preamble signal-to-noise ratio at all information receiving ends at the previous moment and the reflection unit sequence of the intelligent reflection surface at the previous moment are input into the reflection surface phase model to obtain the optimal optimized phase shift sequence corresponding to the situation, and then the intelligent reflection surface controller is used to control the reflection unit of the intelligent reflection surface to adjust according to the optimized phase shift sequence to reduce the information reception bit error rate of the information receiving end.

[0199] Step 300: After at least one reflection unit of the smart reflection surface is adjusted according to the optimized phase shift sequence, the target information is sent to the smart reflection surface, so that the smart reflection surface reflects the target information to the information receiving end.

[0200] The following is a detailed description of step 300.

[0201] In some embodiments, when at least one reflection unit of the smart reflection surface is adjusted according to the optimized phase shift sequence, the target information is sent to the smart reflection surface so that the smart reflection surface reflects the target information to the information receiving end, thereby effectively reducing the information receiving bit error rate of the information receiving end.

[0202] The information transmission method and related equipment of the wireless communication system proposed in the embodiment of the present application, the wireless communication system includes an information transmitting end, an intelligent reflecting surface and at least one information receiving end, and the method includes: first, obtaining the preamble signal-to-noise ratio of all information receiving ends at the previous moment, and obtaining the number of code elements, the modulation order, and the code element decoding rate at the moment, and obtaining the phase shift parameter of the reflection unit, obtaining the bit error parameter of each code element based on the code element decoding rate, and obtaining the estimated bit error rate of each information receiving end based on the number of code elements, the modulation order and all the bit error parameters at the moment, obtaining the bit error rate weighting factor of each information receiving end, and accumulating the bit error rate weighting factor and the estimated bit error rate. The bit error rate obtains the weighted bit error rate of each information receiving end, accumulates all weighted bit error rates, takes the logarithm, and then takes the negative number to obtain the bit error rate reward model. The bit error rate reward model is used to reduce the receiving bit error rate of the information receiving end, obtains the initial action parameters of the actor network, the initial comment parameters of the critic network, and generates the initial reflection surface phase model based on the initial action parameters, initial comment parameters, phase shift parameters, bit error rate reward model and previous signal-to-noise ratio parameters. The previous training signal-to-noise ratio at the current training moment is input into the actor network to obtain the training action, and the current training signal-to-noise ratio is obtained according to the training action. The current training signal-to-noise ratio is input into the bit error rate reward. The model obtains the current training reward, generates the training experience information of the current training moment based on the previous training signal-to-noise ratio, training action, current training signal-to-noise ratio and current training reward, and stores the training experience information in the experience replay pool. Based on the state value model, the training state value and training state strategy value corresponding to the training experience information are obtained. The state value model is generated based on the bit error rate reward model, and the training time difference residual is obtained based on the training experience information and the training state value, and the training state value is obtained based on the training experience information and the training state strategy value. The importance weight of the update strategy is obtained, and the importance of the update strategy is updated based on the preset truncation parameter. The importance weights are truncated to obtain truncated importance weights, the initial action parameters are updated by gradient descent based on the truncated importance weights, the initial comment parameters are updated by gradient descent based on the training state values, and the updated actor network is used as the reflection surface phase model; then, all the preceding signal-to-noise ratios are input into the reflection surface phase model to obtain an output optimized phase shift sequence, and the optimized phase shift sequence is used to optimize the information reception bit error rate of the information receiving end; finally, when at least one reflection unit of the intelligent reflection surface is adjusted according to the optimized phase shift sequence, the target information is sent to the intelligent reflection surface, so that the intelligent reflection surface reflects the target information to the information receiving end.

[0203] The embodiment of the present application utilizes the preamble signal-to-noise ratio at the information receiving end at the previous moment to replace the instantaneous channel state as the decision basis, and there is no need to perform complex instantaneous channel state calculation or estimation, so as to reduce the calculation complexity and setting cost; and further utilizes the reflection surface phase model generated in advance according to the information reception bit error rate of the optimized information receiving end, and generates an optimized phase shift sequence of multiple reflection units of the intelligent reflection surface that can effectively reduce the information reception bit error rate of the information receiving end at the current moment based on the preamble signal-to-noise ratio and the reflection surface phase model, so as to use the adjusted intelligent reflection surface to forward the target information, which can effectively improve the information transmission reliability between the transmitting end and the receiving end in the wireless communication system provided with the intelligent reflection surface; in addition, the PPO algorithm is used to effectively process the MDP problem with a large action space dimension, so as to improve the reliability of the information sending method.

[0204] The embodiment of the present application also provides an information sending device of a wireless communication system, which can implement the information sending method of the wireless communication system. Figure 4 , the device 400 comprises:

[0205] The acquisition module 410 is used to acquire the preamble signal-to-noise ratio of all information receiving ends at the last moment, and acquire the reflection surface phase model;

[0206] An adjustment module 420 is used to input all preamble signal-to-noise ratios into a reflection surface phase model to obtain an output optimized phase shift sequence, where the optimized phase shift sequence is used to optimize an information receiving bit error rate at an information receiving end;

[0207] The information sending module 430 is used to send target information to the smart reflecting surface after at least one reflecting unit of the smart reflecting surface is adjusted according to the optimized phase shift sequence, so that the smart reflecting surface reflects the target information to the information receiving end.

[0208] In some embodiments, the acquisition module 410 is further configured to:

[0209] Obtaining the number of symbols, modulation order, symbol decoding rate at a given moment, and obtaining the phase shift parameters of the reflection unit;

[0210] The bit error parameter of each symbol is obtained based on the symbol decoding rate, and the estimated bit error rate of each information receiving end is obtained based on the number of symbols at the time, the modulation order and all the bit error parameters;

[0211] A bit error rate reward model is generated based on the estimated bit error rate, and the bit error rate reward model is used to reduce the receiving bit error rate of the information receiving end;

[0212] An initial reflection surface phase model is generated based on the phase shift parameter and the bit error rate reward model, and the initial reflection surface phase model is trained to obtain a reflection surface phase model.

[0213] In some embodiments, the acquisition module 410 is further configured to:

[0214] Obtaining a bit error rate weighting factor of each information receiving end, and accumulating and multiplying the bit error rate weighting factor and the estimated bit error rate to obtain a weighted bit error rate of each information receiving end;

[0215] All weighted bit error rates are accumulated, and their logarithms are taken, and then the negative values ​​are taken to obtain the bit error rate reward model.

[0216] In some embodiments, the acquisition module 410 is further configured to:

[0217] Obtain the initial action parameters of the actor network and the initial comment parameters of the commentator network;

[0218] An initial reflection surface phase model is generated based on the initial action parameter, the initial comment parameter, the phase shift parameter, the bit error rate reward model and the preceding signal-to-noise ratio parameter.

[0219] In some embodiments, the acquisition module 410 is further configured to:

[0220] Input the previous training signal-to-noise ratio of the current training moment into the actor network to obtain the training action, and obtain the current training signal-to-noise ratio according to the training action;

[0221] Input the current training signal-to-noise ratio into the bit error rate reward model to obtain the current training reward;

[0222] Generate training experience information at the current training moment based on the previous training signal-to-noise ratio, training action, current training signal-to-noise ratio, and current training reward, and store the training experience information in the experience replay pool;

[0223] The initial action parameters and the initial comment parameters are updated using the training experience information of at least one training moment in the experience replay pool to obtain an updated actor network and commentator network, and the updated actor network is used as the reflection surface phase model.

[0224] In some embodiments, the acquisition module 410 is further configured to:

[0225] Based on the state value model, the training state value and training state strategy value corresponding to the training experience information are obtained. The state value model is generated based on the bit error rate reward model.

[0226] Obtaining a training time series difference residual based on the training experience information and the training state value, and obtaining a training state value based on the training experience information and the training state strategy value;

[0227] The initial action parameters are updated based on the training temporal difference residuals, and the initial comment parameters are updated based on the training state values.

[0228] In some embodiments, the acquisition module 410 is further configured to:

[0229] Obtaining the update strategy importance weight, and truncating the update strategy importance weight based on a preset truncation parameter to obtain the truncated importance weight;

[0230] Perform gradient descent updates on the initial action parameters based on the truncated importance weights;

[0231] Perform gradient descent updates on the initial review parameters based on the training state values.

[0232] In the above embodiments, the description of each embodiment has its own emphasis. For the part not described in detail in a certain embodiment, the specific implementation of the information sending device of the wireless communication system is basically the same as the specific implementation of the information sending method of the above wireless communication system, and will not be repeated here.

[0233] In an embodiment of the present application, an information sending device of a wireless communication system uses a preamble signal-to-noise ratio at an information receiving end at a previous moment instead of an instantaneous channel state as a decision basis, without the need to perform complex instantaneous channel state calculations or estimates, so as to reduce computational complexity and setup costs; and further uses a reflection surface phase model generated in advance based on an information reception bit error rate of an optimized information receiving end, and generates an optimized phase shift sequence of multiple reflection units of an intelligent reflection surface that can effectively reduce the information reception bit error rate of the information receiving end at the current moment based on the preamble signal-to-noise ratio and the reflection surface phase model, thereby using the adjusted intelligent reflection surface to forward target information, which can effectively improve the reliability of information transmission between a transmitting end and a receiving end in a wireless communication system provided with an intelligent reflection surface; in addition, a PPO algorithm is used to effectively process MDP problems with large action space dimensions, thereby improving the reliability of the information sending method.

[0234] The present application also provides an electronic device, including:

[0235] at least one memory;

[0236] at least one processor;

[0237] at least one program;

[0238] The program is stored in the memory, and the processor executes the at least one program to implement the information sending method of the wireless communication system implemented in the present application. The electronic device can be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a car computer, etc.

[0239] See also Figure 5 , Figure 5The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0240] The processor 501 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0241] The memory 502 can be implemented in the form of ROM (Read Only Memory), static storage device, dynamic storage device or RAM (Random Access Memory). The memory 502 can store operating systems and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 502, and the processor 501 calls and executes the information sending method of the wireless communication system of the embodiment of the present application;

[0242] Input / output interface 503, used to implement information input and output;

[0243] Communication interface 504, used to realize communication interaction between the device and other devices, which can be realized by wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0244] A bus 505 that transmits information between the various components of the device (e.g., the processor 501, the memory 502, the input / output interface 503, and the communication interface 504);

[0245] The processor 501 , the memory 502 , the input / output interface 503 and the communication interface 504 are connected to each other in communication within the device via the bus 505 .

[0246] An embodiment of the present application further provides a storage medium, which is a computer-readable storage medium and stores a computer program. When the computer program is executed by a processor, the information sending method of the wireless communication system described above is implemented.

[0247] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0248] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0249] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0250] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0251] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0252] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0253] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0254] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0255] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0256] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0257] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0258] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A method for sending information in a wireless communication system, characterized in that: The wireless communication system comprises an information transmitting end, a smart reflecting surface and at least one information receiving end, and the method comprises: Obtaining the preamble signal-to-noise ratio of all the information receiving ends at the previous moment, and obtaining the reflection surface phase model; Inputting all the preamble signal-to-noise ratios into the reflection surface phase model to obtain an output optimized phase shift sequence, wherein the optimized phase shift sequence is used to optimize the information reception bit error rate of the information receiving end; After at least one reflection unit of the smart reflection surface is adjusted according to the optimized phase shift sequence, target information is sent to the smart reflection surface, so that the smart reflection surface reflects the target information to the information receiving end.

2. The information transmission method of the wireless communication system according to claim 1, characterized in that: The obtaining of the reflection surface phase model comprises: Acquire the number of code elements, modulation order, bit error parameters at a given moment, and acquire the phase shift parameters of the reflection unit; Obtaining an estimated bit error rate of each of the information receiving ends based on the number of code elements at the time, the modulation order, and all the bit error parameters; Generating a bit error rate reward model based on the estimated bit error rate, wherein the bit error rate reward model is used to reduce the receiving bit error rate of the information receiving end; An initial reflection surface phase model is generated based on the phase shift parameter and the bit error rate reward model, and model training is performed on the initial reflection surface phase model to obtain the reflection surface phase model.

3. The information transmission method of the wireless communication system according to claim 2, characterized in that: The generating a bit error rate reward model based on the estimated bit error rate comprises: Acquire a bit error rate weighting factor of each of the information receiving ends, and multiply the bit error rate weighting factor and the estimated bit error rate to obtain a weighted bit error rate of each of the information receiving ends; All the weighted bit error rates are accumulated, and the logarithm is taken, and then the negative number is taken to obtain the bit error rate reward model.

4. The information transmission method of the wireless communication system according to claim 2, characterized in that: The generating an initial reflection surface phase model based on the phase shift parameter and the bit error rate reward model comprises: Obtain the initial action parameters of the actor network and the initial comment parameters of the commentator network; The initial reflection surface phase model is generated based on the initial action parameter, the initial comment parameter, the phase shift parameter and the preceding signal-to-noise ratio parameter.

5. The information sending method of the wireless communication system according to claim 4, characterized in that: The performing model training on the initial reflection surface phase model to obtain the reflection surface phase model includes: Inputting the previous training signal-to-noise ratio at the current training moment into the actor network to obtain a training action, and obtaining a current training signal-to-noise ratio according to the training action; Inputting the current training signal-to-noise ratio into the bit error rate reward model to obtain a current training reward; Generate the training experience information of the current training moment based on the previous training signal-to-noise ratio, the training action, the current training signal-to-noise ratio and the current training reward, and store the training experience information in an experience replay pool; The initial action parameters and the initial comment parameters are updated using the training experience information of the batch training moments in the experience replay pool to obtain an updated actor network and commentator network, and the updated actor network is used as the reflector phase model.

6. The information transmission method of the wireless communication system according to claim 5, characterized in that: The updating of the initial action parameter and the initial comment parameter by using the training experience information of the batch training moments in the experience replay pool includes: Based on a state value model, obtaining a training state value corresponding to the training experience information, wherein the state value model is generated based on the bit error rate reward model; Obtaining a training time series difference residual and a training state value based on the training experience information and the training state value; The initial action parameter is updated based on the training temporal difference residual, and the initial comment parameter is updated based on the training state value.

7. The information sending method of the wireless communication system according to claim 6, characterized in that: The updating of the initial action parameter based on the training time-series difference residual and the updating of the initial comment parameter based on the training state value include: Obtaining an update strategy importance weight, and truncating the update strategy importance weight based on a preset truncation parameter to obtain a truncated importance weight; performing gradient descent updating on the initial action parameters based on the truncated importance weights; The initial comment parameter is updated by gradient descent based on the training state value.

8. An information sending device for a wireless communication system, characterized in that: The wireless communication system comprises an information transmitting end, a smart reflecting surface and at least one information receiving end, and the device comprises: An acquisition module, used to acquire the preamble signal-to-noise ratio of all the information receiving ends at the previous moment, and to acquire a reflection surface phase model; An adjustment module, used for inputting all the preamble signal-to-noise ratios into the reflection surface phase model to obtain an output optimized phase shift sequence, wherein the optimized phase shift sequence is used for optimizing the information receiving bit error rate of the information receiving end; The information sending module is used to send target information to the smart reflecting surface after at least one reflecting unit of the smart reflecting surface is adjusted according to the optimized phase shift sequence, so that the smart reflecting surface reflects the target information to the information receiving end.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the information sending method of the wireless communication system according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the information sending method of the wireless communication system according to any one of claims 1 to 7 is implemented.

Images