A data transmission optimization method and device of a communication system, a terminal and a medium

Abstract

The application discloses a data transmission optimization method and device of a communication system, a terminal and a medium, and comprises the following steps: modeling an IRS-aided channel according to a cascade channel model; obtaining the AoI violation probability of each device according to the definition of AoI; constructing an effective capacity maximization problem of information timeliness guarantee; converting a long-term random optimization problem into a per-slot deterministic optimization problem according to a Lyapunov optimization algorithm; converting the original effective capacity optimization problem into a Markov decision process; and jointly optimizing the phase shift of the IRS and the transmission power of the device according to an SAC algorithm to obtain an optimized data transmission strategy. The application solves the effective capacity maximization problem of information timeliness guarantee in an IRS-aided NOMA communication system with imperfect CSI.

Description

Technical Field

[0001] This invention relates to the field of NOMA communication technology, and in particular to a method, apparatus, terminal and medium for optimizing data transmission in a communication system. Background Technology

[0002] Compared to orthogonal multiple access (OMA) technologies, NOMA (Non-Orthogonal Multiple Access) is a key radio access method for improving spectrum efficiency. Furthermore, by utilizing the Intelligent Reflecting Surface (IRS), which can alter the radio channel environment, novel IRS-assisted NOMA communication systems can be constructed. However, due to the passive nature of the IRS, obtaining accurate Channel State Information (CSI) is challenging.

[0003] For the traditional problem of maximizing effective capacity, existing schemes only consider channel capacity optimization under communication constraints. However, channel capacity optimization also needs to consider information timeliness, which is a performance indicator different from communication delay, measured using Age of Information (AoI). Therefore, the impact of information timeliness constraints on effective capacity is a worthy research topic. Finally, considering the interference between NOMA systems and the large number of sub-cells in the IRS, existing channel capacity optimization schemes cannot maximize effective capacity while ensuring information timeliness.

[0004] Therefore, existing technologies still need improvement. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a data transmission optimization method, device, terminal and medium for a communication system, in order to solve the problem of maximizing the effective capacity for ensuring information timeliness in a NOMA communication system with IRS assistance and imperfect CSI, in order to address the deficiencies of the prior art.

[0006] The technical solution adopted by this invention to solve the technical problem is as follows:

[0007] In a first aspect, the present invention provides a data transmission optimization method for a communication system, comprising:

[0008] Model the IRS-assisted channel based on the cascaded channel model;

[0009] The probability of AoI violation for each device is obtained based on the definition of AoI.

[0010] The problem of maximizing the effective capacity to ensure information timeliness;

[0011] The long-run stochastic optimization problem is transformed into a per-slot deterministic optimization problem based on the Lyapunov optimization algorithm.

[0012] The original effective capacity optimization problem is transformed into a Markov decision process.

[0013] The optimized data transmission strategy is obtained by jointly optimizing the IRS phase shift and device transmission power using the SAC algorithm.

[0014] In one implementation, the step of modeling the IRS-assisted channel according to the cascaded channel model includes, prior to:

[0015] Time is divided into several time slots;

[0016] In the time slot At that time, obtain the signal from device k:

[0017]

[0018] in, This represents the channel state matrix between the AP and the IRS at time slot t;

[0019] This represents the channel state matrix between the IRS and device k at time slot t;

[0020] This represents the signal of device k in time slot t;

[0021] This represents the power of device k in time slot t;

[0022] This represents the noise level of device k at the base station.

[0023] This represents the phase matrix of all sub-cells of the IRS at time slot t. .

[0024] In one implementation, modeling the IRS-assisted channel according to the cascaded channel model includes:

[0025] The cascaded channel model is used to model the IRS-assisted communication channel:

[0026]

[0027] in, Represented as the estimated cascaded channel, This is the cascaded channel matrix from the AP through the IRS to the device;

[0028] This represents the channel estimation error corresponding to the k-cascaded channels of the device at time slot t.

[0029] In one implementation, obtaining the AoI violation probability for each device according to the definition of AoI includes:

[0030] Determine the AoI value of device k in time slot t:

[0031]

[0032] in, This represents the AoI value of device k at time slot t. This indicates the time when the u-th data packet arrives at device k. Indicates that device k transmits the first... Data packet time;

[0033] Calculate the probability of AoI violation for each device k:

[0034]

[0035] in, This represents the threshold value of device kAoI. Indicates an event The probability of it being true. This represents the maximum value of the probability of a device violating the kAoI rule.

[0036] In one implementation, obtaining the AoI violation probability for each device based on the definition of AoI further includes:

[0037] To obtain the relationship between the probability of device AoI violation for any task reaching a certain pattern and parameters such as the real-time task queue and the number of tasks processed:

[0038]

[0039] in, Indicates time The number of data packets has been reached at internal device k. Indicates the number of data packets used to update the transmission status;

[0040] Based on the relationship between the device AoI violation probability and parameters such as the real-time task queue and the number of processing tasks, calculate the AoI violation probability for each device k:

[0041]

[0042] in, This indicates the total number of time slots.

[0043] In one implementation, the problem of maximizing the effective capacity to ensure information timeliness includes:

[0044] Determine the delay-dependent effective capacity of device k:

[0045]

[0046] in, Indicates the service quality factor. Indicates the time slot length. Indicates the rate of device k;

[0047] The problem of maximizing the effective capacity for ensuring information timeliness can be expressed as:

[0048]

[0049] Information timeliness limitations:

[0050] Instantaneous power limit:

[0051] Average power limit:

[0052] Phase decision constraints:

[0053] in, This represents the power values ​​of K devices within the entire time slot T. This represents the phase shift value of all sub-cells of the IRS within the entire T time slot. This represents the threshold value for the instantaneous power of device k. This represents the threshold value for the average power of device k.

[0054] In one implementation, the transformation of the long-run stochastic optimization problem into a per-slot deterministic optimization problem based on the Lyapunov optimization algorithm includes:

[0055] The information timeliness and average power constraints are transformed into a queue stability problem using a virtual queue model:

[0056]

[0057]

[0058] Determine the Lyapunov offset penalty function:

[0059]

[0060] in, Represented as variable values ​​that are irrelevant to the decision variable;

[0061] The problem of maximizing the effective capacity to ensure information timeliness is transformed into a deterministic optimization problem per time slot:

[0062]

[0063] instantaneous power limit conditions:

[0064] Phase decision constraints: .

[0065] In one implementation, the transformation of the original effective capacity optimization problem into a Markov decision process includes:

[0066] Define the action space, decision space, and reward function of the Markov decision process.

[0067] Construct a discrete Markov decision process based on the action space, decision space, and reward function.

[0068] In one implementation, defining the action space, decision space, and reward function of the Markov decision process includes:

[0069] State space: The action space of an IRS-assisted NOMA communication system with imperfect CSI is defined as follows:

[0070]

[0071] in, Indicates the length of all queues;

[0072] Action space: The action space is defined as:

[0073]

[0074] in, This represents the power and phase values ​​of all devices and IRS sub-units at time slot t;

[0075] Reward function: The objective function of the deterministic optimization problem in each time slot is defined as the reward obtained after making a decision in each time slot t.

[0076] .

[0077] In one implementation, the step of jointly optimizing the IRS phase shift and device transmission power according to the SAC algorithm to obtain an optimized data transmission strategy includes:

[0078] The output of the decision distribution provides information about the current state. To obtain the current decision ;

[0079] Rewards are awarded based on current status information and actions. and the next status information and will Save it to experience replay;

[0080] Update the parameters of the Q1 and Q2 networks based on minimizing the gradient of the loss function and the mean squared error. and The minimum of the two Q-function values ​​is used as the updated Q-function value for each Q-function.

[0081] Update the parameters of the policy network by minimizing the KL divergence function and the gradient of the KL divergence. ;

[0082] We use gradient descent to minimize the weighted sum of the reward function and the entropy function, and then update the weights of the entropy function. ;

[0083] The parameters of objective Q1 and objective Q2 functions are updated using a soft update method. and .

[0084] In a second aspect, the present invention provides a data transmission optimization device for a communication system, comprising:

[0085] The channel modeling module is used to model the IRS-assisted channel based on the cascaded channel model;

[0086] The violation probability calculation module is used to obtain the AoI violation probability for each device based on the definition of AoI;

[0087] The effective capacity module is used to solve the problem of maximizing effective capacity to ensure information timeliness.

[0088] The first optimization module is used to transform the long-term stochastic optimization problem into a per-slot deterministic optimization problem based on the Lyapunov optimization algorithm.

[0089] The second optimization module is used to transform the original effective capacity optimization problem into a Markov decision process.

[0090] The joint optimization module is used to jointly optimize the IRS phase shift and device transmission power according to the SAC algorithm to obtain the optimized data transmission strategy.

[0091] Thirdly, the present invention provides a terminal, comprising: a processor and a memory, wherein the memory stores a data transmission optimization program for a communication system, and the data transmission optimization program for the communication system, when executed by the processor, is used to implement the operation of the data transmission optimization method for the communication system as described in the first aspect.

[0092] Fourthly, the present invention also provides a medium, which is a computer-readable storage medium, storing a data transmission optimization program for a communication system, which, when executed by a processor, is used to implement the operation of the data transmission optimization method for the communication system as described in the first aspect.

[0093] The present invention, by employing the above technical solution, has the following effects:

[0094] This invention addresses the problem of maximizing the effective capacity for ensuring information timeliness in IRS-assisted NOMA communication systems with imperfect CSI. First, it models the channel using an imperfect cascaded channel model and derives the relationship between the AoI violation probability and parameters such as the state update packet queue and the number of transmitted packets. This relationship is used to determine AoI violations using the packet queue length and the number of transmitted packets, and based on this, a problem for maximizing the effective capacity for ensuring timeliness in IRS-assisted NOMA systems is formulated. Second, it transforms the proposed long-term stochastic optimization problem into a per-slot deterministic optimization problem using Lyapunov optimization methods. Finally, considering that the per-slot optimization problem is non-convex, this invention proposes a joint optimization algorithm for IRS phase shift and device transmission power based on deep reinforcement learning to obtain an efficient joint optimization scheme. Attached Figure Description

[0095] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0096] Figure 1 This is a flowchart of a data transmission optimization method for a communication system in one implementation of the present invention.

[0097] Figure 2 This is a schematic diagram of an IRS-assisted NOMA communication system in one implementation of the present invention.

[0098] Figure 3 This is a schematic diagram of device AoI in one implementation of the present invention.

[0099] Figure 4 This is a schematic diagram illustrating the joint optimization of IRS phase shift and device transmission power based on soft actor critic in one implementation of the present invention.

[0100] Figure 5This is a functional schematic diagram of the terminal in one implementation of the present invention.

[0101] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0102] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0103] Exemplary methods

[0104] For the traditional problem of maximizing effective capacity, existing schemes only consider channel capacity optimization under communication constraints. However, channel capacity optimization also needs to consider information timeliness, which is a performance indicator different from communication delay, measured using Age of Information (AoI). Therefore, the impact of information timeliness constraints on effective capacity is a worthy research topic. Finally, considering the interference between NOMA systems and the large number of sub-cells in the IRS, existing channel capacity optimization schemes cannot maximize effective capacity while ensuring information timeliness.

[0105] To address the aforementioned technical problems, this embodiment provides a data transmission optimization method for communication systems. Leveraging the high frequency of Non-Orthogonal Multiple Access (NOMA) technology and the high spectral efficiency of Intelligent Reflecting Surfaces (IRS), a novel IRS-assisted NOMA communication system is constructed. For novel communication systems with imperfect Channel State Information (CSI), a joint optimization scheme for IRS phase shift and device transmission power is designed using Lyapunov optimization techniques and deep reinforcement learning methods to maximize effective capacity while ensuring information timeliness.

[0106] like Figure 1 As shown, this embodiment of the invention provides a data transmission optimization method for a communication system, comprising the following steps:

[0107] Step S100: Model the IRS-assisted channel according to the cascaded channel model.

[0108] In this embodiment, the data transmission optimization method of the communication system is applied to a terminal, which includes, but is not limited to, devices such as computers.

[0109] In this embodiment, addressing the problem of maximizing the effective capacity for ensuring information timeliness in an IRS-assisted NOMA communication system with imperfect CSI, the following steps are taken: First, a non-perfect cascaded channel model is used to model the channel, and the relationship between the AoI violation probability and parameters such as the state update packet queue and the number of transmitted packets is derived. This relationship is then used to determine AoI violations using the packet queue length and the number of transmitted packets, and based on this, the effective capacity maximization problem for ensuring timeliness in the IRS-assisted NOMA system is formulated. Second, the proposed long-term stochastic optimization problem is transformed into a per-slot deterministic optimization problem using the Lyapunov optimization method. Finally, considering that the per-slot optimization problem is non-convex, this embodiment proposes a joint optimization algorithm for IRS phase shift and device transmission power based on deep reinforcement learning to obtain an efficient joint optimization scheme.

[0110] This embodiment considers an IRS-assisted NOMA communication system, such as Figure 2 As shown, the system includes one access point (AP) and K terminal devices. The terminal devices periodically collect ambient environmental data and send the data packets to the AP via the IRS (In-Relational Data System) as status update packets. This updates the status information of the observation points on the K terminal devices at the AP (each terminal device uses a thermometer to measure the temperature at an observation point, and the AP continuously updates temperature-related data packets through the terminal devices to monitor temperature changes). These terminal devices share the same frequency band to send status update packets. Each device has a buffer; status update packets are randomly generated and then queued in the buffer in a first-come, first-served queue. The IRS has… Sub-reflection units. These sub-reflection units are used to modify the channel phase to improve the wireless channel environment and enhance communication performance. The controller controls the phase of the sub-reflection units without changing the signal magnitude. Additionally, there is no direct link between the AP and the devices due to obstructions such as buildings. Since multiple devices share the same frequency band, the devices need to control power to reduce inter-device interference and improve the wireless channel environment by modifying the phase of the IRS sub-reflection units. Therefore, a joint optimization method for IRS phase and device transmission power needs to be designed to improve IRS-assisted NOMA communication systems.

[0111] To address the problem of maximizing effective capacity to ensure information timeliness in IRS-assisted NOMA communication systems with imperfect CSI, a joint optimization method for IRS phase and transmission power based on deep reinforcement learning is proposed. First, the IRS-assisted channel is modeled using a cascaded channel model. Before modeling, the signal of device k needs to be represented.

[0112] Specifically, in one implementation of this embodiment, the following steps are included before step S100:

[0113] Step S001: Divide the time into several time slots according to the time slots;

[0114] Step S002, in time slot At that time, the signal from device k is acquired.

[0115] In this embodiment, the communication system under consideration divides time into time slots. At that time, the signal received by the base station from device k can be expressed as:

[0116] (1)

[0117] in, This represents the channel state matrix between the AP and the IRS at time slot t; This represents the channel state matrix between the IRS and device k at time slot t; This represents the signal of device k in time slot t; This represents the power of device k in time slot t; This represents the noise level of device k at the base station. This represents the phase matrix of all sub-cells of the IRS at time slot t. .

[0118] In this embodiment, considering the passive nature of the IRS, accurate CSI cannot be obtained. A cascaded channel model is used to model the IRS-assisted communication channel.

[0119] Specifically, in one implementation of this embodiment, step S100 includes the following steps:

[0120] Step S101: Model the IRS-assisted communication channel using the cascaded channel model.

[0121] During the modeling process, the definition will be... This is the cascaded channel matrix from the AP through the IRS to the device. And it will... Represented as the estimated cascaded channel, thus obtaining ,in, Therefore, the cascaded channel from the AP through the IRS to the device can be represented as:

[0122] (2)

[0123] in, This represents the channel estimation error corresponding to the k-cascaded channels of device k at time slot t, and... It follows a cyclically symmetric complex Gaussian distribution, i.e. , express The covariance matrix.

[0124] In addition, each device needs to To determine the decoding order of multiple devices, among which, It is unknown at the beginning of each time slot t. Therefore, it utilizes... Determine the decoding order of multiple devices, and assume... Therefore, at time slot t, the signal-to-interference plus-noise ratio (SINR) of device k can be expressed as:

[0125] (3)

[0126] in, This represents the power value of device k at time slot t. This represents the power value of device i at time slot t. Indicates noise power, when , Therefore, according to Shannon's formula, the speed of device k can be obtained, which is expressed as:

[0127] (4)

[0128] in, Indicates the channel bandwidth. This indicates the number of transmission status update packets, where, Indicates the time slot length. This indicates the size of the status update data packet.

[0129] like Figure 1 As shown, in one implementation of this invention, the data transmission optimization method for a communication system further includes the following steps:

[0130] Step S200: Obtain the AoI violation probability for each device according to the definition of AoI.

[0131] In this embodiment, after modeling the IRS-assisted channel, the AoI violation probability of each device is obtained according to the definition of AoI.

[0132] like Figure 3 As shown, Figure 3The diagram illustrates the AoI changes of three devices. For example, device 2 has six status update data packets. Each box represents the generation time of the data packet; for instance, box 4 is 2, indicating that this data packet was generated in time slot 2. In the seventh time slot, the device transmits three data packets. Therefore, at the AP end, device 2's AoI becomes 7-2=5. Furthermore, device 2's AoI threshold is 4, meaning that in the seventh time slot, device 2's AoI does not meet the information timeliness requirement.

[0133] Specifically, in one implementation of this embodiment, step S200 includes the following steps:

[0134] Step S201: Determine the AoI value of device k in time slot t;

[0135] Step S202: Calculate the AoI violation probability for each device k.

[0136] In this embodiment, the AoI of device k in time slot t is represented as:

[0137] (5)

[0138] in, This represents the AoI of device k at time slot t. This indicates the time when the u-th data packet arrives at device k. Indicates that device k transmits the first... Data packet time.

[0139] The probability of AoI violation for each device k is expressed as:

[0140] (6)

[0141] here, This represents the threshold value of device kAoI. Indicates an event The probability of it being true. This represents the maximum value of the probability of a device violating the kAoI rule.

[0142] In one implementation of this embodiment, step S200 further includes the following steps:

[0143] Step S203: Obtain the relationship between the probability of device AoI violation for any task reaching the mode and parameters such as real-time task queue and number of tasks processed;

[0144] Step S204: Calculate the AoI violation probability of each device k based on the relationship between the device AoI violation probability and parameters such as the real-time task queue and the number of processing tasks.

[0145] In this embodiment, after calculating the AoI violation probability for each device k, the relationship between the device AoI violation probability for any task reaching a certain mode and parameters such as the real-time task queue and the number of tasks processed is obtained, which can be expressed as:

[0146] (7)

[0147] in, Indicates time The number of data packets has been reached at internal device k. This represents the number of transmission status update packets. Therefore, formula (6) is equivalent to:

[0148] (8)

[0149] here, This indicates the total number of time slots.

[0150] like Figure 1 As shown, in one implementation of this invention, the data transmission optimization method for a communication system further includes the following steps:

[0151] Step S300: Construct the problem of maximizing the effective capacity to ensure information timeliness.

[0152] In this embodiment, after obtaining the AoI violation probability of each device, an effective capacity maximization problem for ensuring information timeliness is formulated based on the AoI violation probability.

[0153] Specifically, in one implementation of this embodiment, step S300 includes the following steps:

[0154] Step S301: Determine the latency-related effective capacity of device k;

[0155] Step S302: Determine the problem of maximizing the effective capacity to ensure information timeliness.

[0156] In this embodiment, based on the definition of effective capacity, the latency-related effective capacity of device k can be expressed as:

[0157] (9)

[0158] in, This represents a Quality of Service (QoS) factor. Indicates the time slot length. Let k represent the rate of device k. Considering the limitations of the device's instantaneous and average power, the problem of maximizing the effective capacity to ensure information timeliness is expressed as:

[0159] (P1)

[0160] Information timeliness limitations:

[0161] Instantaneous power limit:

[0162] Average power limit:

[0163] Phase decision constraints:

[0164] in, This represents the power values ​​of K devices within the entire time slot T. This represents the phase shift value of all sub-cells of the IRS within the entire T time slot. This represents the threshold value for the instantaneous power of device k. This represents the threshold value for the average power of device k.

[0165] like Figure 1 As shown, in one implementation of this invention, the data transmission optimization method for a communication system further includes the following steps:

[0166] Step S400: The long-term stochastic optimization problem is transformed into a time-slot-specific deterministic optimization problem according to the Lyapunov optimization algorithm.

[0167] In this embodiment, after formulating the effective capacity maximization problem to ensure information timeliness, the effective capacity maximization problem is solved, that is, the long-term stochastic optimization problem is transformed into a per-slot deterministic optimization problem based on Lyapunov optimization techniques.

[0168] Specifically, in one implementation of this embodiment, step S400 includes the following steps:

[0169] Step S401: Use a virtual queue model to transform the information timeliness constraint and average power constraint into a queue stability problem;

[0170] Step S402: Determine the Lyapunov offset penalty function;

[0171] Step S403 transforms the problem of maximizing the effective capacity to ensure information timeliness into a deterministic optimization problem per time slot.

[0172] In this embodiment, firstly, the information timeliness constraint and average power constraint are transformed into a queue stability problem using a virtual queue model, which is expressed as:

[0173] (10)

[0174] (11)

[0175] Secondly, according to the Lyapunov optimization technique, the Lyapunov offset penalty function can be expressed as:

[0176] (12)

[0177] in, This is represented by variable values ​​that are independent of the decision variables. Therefore, problem (P1) can be transformed into the following problem:

[0178] (P2)

[0179] instantaneous power limit conditions:

[0180] Phase decision constraints:

[0181] For problem (P2), firstly, considering that the objective function has a unit step function, this problem is non-convex. Secondly, since... and The problem involves coupling and must be decomposed into multiple subproblems, which are then solved alternately until convergence. However, when the channel state, packet queue length, and virtual queue length change, each subproblem requires resolving for its optimal solution, and frequent subproblem solving can negatively impact convergence. Therefore, in this embodiment, considering the ability of deep reinforcement learning to obtain near-optimal solutions in real time, a deep reinforcement learning-based algorithm is proposed to obtain the optimal solution for problem (P2). Furthermore, considering the drawback of overestimating Q-values ​​in deterministic policy gradient (DDPG), this embodiment utilizes the soft actorcritic deep reinforcement learning method (i.e., the SAC algorithm) to solve problem P2.

[0182] like Figure 1 As shown, in one implementation of this invention, the data transmission optimization method for a communication system further includes the following steps:

[0183] Step S500 transforms the original effective capacity optimization problem into a Markov decision process.

[0184] In this embodiment, the IRS-assisted NOMA communication system with imperfect CSI is treated as an agent. The agent's decision-making process is modeled as a discrete Markov Decision Process (MDP).

[0185] Specifically, in one implementation of this embodiment, step S500 includes the following steps:

[0186] Step S501: Define the action space, decision space, and reward function of the Markov decision process;

[0187] Step S502: Construct a discrete Markov decision process based on the action space, decision space, and reward function.

[0188] The action space, decision space, and reward function of this MDP are defined below.

[0189] 1) State Space: The action space of an IRS-assisted NOMA communication system with imperfect CSI is defined as... ,in, This indicates the length of all queues.

[0190] Specifically, This represents the packet queue of device k at time slot t. This represents the length of the virtual queue at time slot t, which is related to the timeliness constraints of the information. This represents the virtual queue length related to the average power constraint at time slot t. Indicates time The number of data packets reached at internal device k; Indicates the channel state, specifically, This indicates the channel state between the AP and the IRS at time slot t. This indicates the channel state between the IRS and device k at time slot t. This represents the error of the concatenated channel at time slot t; This represents the power and phase values ​​of all devices and IRS subunits at time slot (t-1).

[0191] 2) Action Space: In this embodiment, the transmission power and IRS phase of the joint control equipment are mainly considered to maximize effective capacity while ensuring information timeliness. Therefore, the action space is defined as follows: ,in, This represents the power and phase values ​​of all devices and IRS subunits at time slot t.

[0192] 3) Reward Function: Problem (P1) is equivalent to problem (P2). Therefore, the objective function of problem (P2) is defined as the reward obtained after making a decision in each time slot t, i.e. .

[0193] like Figure 1 As shown, in one implementation of this invention, the data transmission optimization method for a communication system further includes the following steps:

[0194] Step S600: The IRS phase shift and device transmission power are jointly optimized according to the SAC algorithm to obtain the optimized data transmission strategy.

[0195] In this embodiment, a joint optimization method based on soft actor critic for IRS phase shift and device transmission power is implemented to optimize the data transmission of the communication system.

[0196] Specifically, in one implementation of this embodiment, step S600 includes the following steps:

[0197] Step S601: The current state information is obtained through the output of the decision distribution. To obtain the current decision ;

[0198] Step S602: Obtain a reward based on the current state information and action. and the next status information and will Save it to experience replay;

[0199] Step S603: Update the parameters of Q1 network and Q2 network according to minimizing the gradient of the loss function and mean squared error. and The minimum of the two Q-function values ​​is used as the updated Q-function value for each Q-function.

[0200] Step S604: Update the parameters of the policy network based on minimizing the gradient of the KL divergence function and the KL divergence. ;

[0201] Step S605: Minimize the weighted sum of the reward function and the entropy function using gradient descent, and update the weight values ​​of the entropy function. ;

[0202] Step S606: Update the parameters of the target Q1 function and the target Q2 function using the soft update method. and .

[0203] In this embodiment, as Figure 4 As shown, steps S601 to S606 above are executed based on the soft actor critic algorithm to optimize the effective capacity while ensuring information timeliness.

[0204] In this embodiment, considering the difficulty of obtaining the channel state of CSI in the IRS-assisted NOMA system, a non-perfect concatenated channel model is used to model the channel, and on this basis, the effective capacity maximization problem of ensuring timeliness in the IRS-assisted NOMA system is formulated. Using the Lyapunov optimization method and deep reinforcement learning method, a joint optimization scheme of IRS phase shift and device transmission power based on Soft ActorCritic is proposed to optimize the effective capacity under the guarantee of information timeliness.

[0205] This embodiment achieves the following technical effects through the above technical solution:

[0206] This embodiment addresses the problem of maximizing the effective capacity for ensuring information timeliness in an IRS-assisted NOMA communication system with imperfect CSI. First, it models the channel using an imperfect cascaded channel model and derives the relationship between the AoI violation probability and parameters such as the state update packet queue and the number of transmitted packets. This relationship is used to determine AoI violations using the packet queue length and the number of transmitted packets, and based on this, a problem for maximizing the effective capacity for ensuring timeliness in the IRS-assisted NOMA system is formulated. Second, the proposed long-term stochastic optimization problem is transformed into a per-slot deterministic optimization problem using the Lyapunov optimization method. Finally, considering that the per-slot optimization problem is non-convex, this embodiment proposes a joint optimization algorithm for IRS phase shift and device transmission power based on deep reinforcement learning to obtain an efficient joint optimization scheme.

[0207] Exemplary device

[0208] Based on the above embodiments, the present invention also provides a data transmission optimization device for a communication system, comprising:

[0209] The channel modeling module is used to model the IRS-assisted channel based on the cascaded channel model;

[0210] The violation probability calculation module is used to obtain the AoI violation probability for each device based on the definition of AoI;

[0211] The effective capacity module is used to solve the problem of maximizing effective capacity to ensure information timeliness.

[0212] The first optimization module is used to transform the long-term stochastic optimization problem into a per-slot deterministic optimization problem based on the Lyapunov optimization algorithm.

[0213] The second optimization module is used to transform the original effective capacity optimization problem into a Markov decision process.

[0214] The joint optimization module is used to jointly optimize the IRS phase shift and device transmission power according to the SAC algorithm to obtain the optimized data transmission strategy.

[0215] Based on the above embodiments, the present invention also provides a terminal, the principle block diagram of which can be as follows: Figure 5 As shown.

[0216] The terminal includes: a processor, a memory, an interface, a display screen, and a communication module connected via a system bus; wherein, the processor of the terminal provides computing and control capabilities; the memory of the terminal includes a storage medium and internal memory; the storage medium stores the operating system and computer programs; the internal memory provides an environment for the operation of the operating system and computer programs in the storage medium; the interface is used to connect to external devices, such as mobile terminals and computers; the display screen is used to display relevant information; and the communication module is used to communicate with a cloud server or mobile terminal.

[0217] When executed by a processor, this computer program is used to implement a data transmission optimization method for a communication system.

[0218] It will be understood by those skilled in the art that Figure 5 The schematic diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the terminal to which the present invention is applied. A specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0219] In one embodiment, a terminal is provided, comprising: a processor and a memory, the memory storing a data transmission optimization program for a communication system, which, when executed by the processor, is used to implement the data transmission optimization method of the communication system as described above.

[0220] In one embodiment, a storage medium is provided, wherein the storage medium stores a data transmission optimization program for a communication system, which, when executed by a processor, is used to implement the operation of the data transmission optimization method for the communication system as described above.

[0221] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, database, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory.

[0222] In summary, this invention provides a data transmission optimization method, apparatus, terminal, and medium for a communication system. The method includes: modeling the IRS-assisted channel based on a cascaded channel model; obtaining the AoI violation probability for each device according to the definition of AoI; constructing an effective capacity maximization problem to ensure information timeliness; transforming the long-term stochastic optimization problem into a per-slot deterministic optimization problem using the Lyapunov optimization algorithm; transforming the original effective capacity optimization problem into a Markov decision process; and jointly optimizing the IRS phase shift and device transmission power using the SAC algorithm to obtain the optimized data transmission strategy. This invention solves the problem of maximizing the effective capacity to ensure information timeliness in an IRS-assisted NOMA communication system with imperfect CSI.

[0223] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A method for optimizing data transmission in a communication system, characterized in that, include: Model the IRS-assisted channel based on the cascaded channel model; The probability of AoI violation for each device is obtained based on the definition of AoI. The problem of maximizing the effective capacity for ensuring information timeliness includes: Determine the delay-dependent effective capacity of device k: in, Indicates the service quality factor. Indicates the time slot length. Indicates the rate of device k; The problem of maximizing the effective capacity for ensuring information timeliness can be expressed as: Information timeliness limitations: Instantaneous power limit: Average power limit: Phase decision constraints: in, This represents the power values ​​of K devices within the entire time slot T. This represents the phase shift value of all sub-cells of the IRS within the entire T time slot. This represents the threshold value for the instantaneous power of device k. This represents the threshold value for the average power of device k; Based on the Lyapunov optimization algorithm, the long-run stochastic optimization problem is transformed into a per-slot deterministic optimization problem, including: The information timeliness and average power constraints are transformed into a queue stability problem using a virtual queue model: Determine the Lyapunov offset penalty function: in, Represented as variable values ​​that are irrelevant to the decision variable; The problem of maximizing the effective capacity to ensure information timeliness is transformed into a deterministic optimization problem per time slot: instantaneous power limit conditions: Phase decision constraints: ; The original effective capacity optimization problem is transformed into a Markov decision process. The optimized data transmission strategy is obtained by jointly optimizing the IRS phase shift and device transmission power using the SAC algorithm.

2. The data transmission optimization method for a communication system according to claim 1, characterized in that, The process of modeling the IRS-assisted channel based on the cascaded channel model includes, prior to: Time is divided into several time slots; In the time slot At that time, obtain the signal from device k: in, This represents the channel state matrix between the AP and the IRS at time slot t; This represents the channel state matrix between the IRS and device k at time slot t; This represents the signal of device k in time slot t; This represents the power of device k in time slot t; This represents the noise level of device k at the base station. This represents the phase matrix of all sub-cells of the IRS at time slot t. .

3. The data transmission optimization method for a communication system according to claim 2, characterized in that, The process of modeling the IRS-assisted channel based on the cascaded channel model includes: The cascaded channel model is used to model the IRS-assisted communication channel: in, Represented as the estimated cascaded channel, This is the cascaded channel matrix from the AP through the IRS to the device; This represents the channel estimation error corresponding to the k-cascaded channels of the device at time slot t.

4. The data transmission optimization method for a communication system according to claim 3, characterized in that, The process of obtaining the AoI violation probability for each device based on the definition of AoI includes: Determine the AoI value of device k in time slot t: in, This represents the AoI value of device k at time slot t. This indicates the time when the u-th data packet arrives at device k. Indicates that device k transmits the first... Data packet time; Calculate the probability of AoI violation for each device k: in, This represents the threshold value of device kAoI. Indicates an event The probability of it being true. This represents the maximum value of the probability of a device violating the kAoI rule.

5. The data transmission optimization method for a communication system according to claim 4, characterized in that, The process of obtaining the AoI violation probability for each device based on the definition of AoI also includes: To obtain the relationship between the probability of device AoI violation for any task reaching a certain pattern and parameters such as the real-time task queue and the number of tasks processed: in, Indicates time The number of data packets has been reached at internal device k. Indicates the number of data packets used to update the transmission status; Based on the relationship between the device AoI violation probability and parameters such as the real-time task queue and the number of processing tasks, calculate the AoI violation probability for each device k: in, This indicates the total number of time slots.

6. The data transmission optimization method for a communication system according to claim 5, characterized in that, The process of transforming the original effective capacity optimization problem into a Markov decision process includes: Define the action space, decision space, and reward function of the Markov decision process. Construct a discrete Markov decision process based on the action space, decision space, and reward function.

7. The data transmission optimization method for a communication system according to claim 6, characterized in that, The definition of the action space, decision space, and reward function of the Markov decision process includes: State space: The action space of an IRS-assisted NOMA communication system with imperfect CSI is defined as follows: in, Indicates the length of all queues; Action space: The action space is defined as: in, , These represent the power and phase values ​​of all devices and IRS sub-units at time slot t, respectively. Reward function: The objective function of the deterministic optimization problem in each time slot is defined as the reward obtained after making a decision in each time slot t. 。 8. The data transmission optimization method for a communication system according to claim 7, characterized in that, The optimized data transmission strategy is obtained by jointly optimizing the IRS phase shift and device transmission power according to the SAC algorithm, including: The output of the decision distribution provides information about the current state. To obtain the current decision ; Rewards are awarded based on current status information and actions. and the next status information and will Save it to experience replay; Update the parameters of the Q1 and Q2 networks based on minimizing the gradient of the loss function and the mean squared error. and The minimum of the two Q-function values ​​is used as the updated Q-function value for each Q-function. Update the parameters of the policy network by minimizing the KL divergence function and the gradient of the KL divergence. ; We use gradient descent to minimize the weighted sum of the reward function and the entropy function, and then update the weights of the entropy function. ; The parameters of objective Q1 and objective Q2 functions are updated using a soft update method. and .

9. A data transmission optimization apparatus for a communication system, used to implement the data transmission optimization method for a communication system as described in any one of claims 1-8, characterized in that, include: The channel modeling module is used to model the IRS-assisted channel based on the cascaded channel model; The violation probability calculation module is used to obtain the AoI violation probability for each device based on the definition of AoI; The effective capacity module is used to solve the problem of maximizing effective capacity to ensure information timeliness. The first optimization module is used to transform the long-term stochastic optimization problem into a per-slot deterministic optimization problem based on the Lyapunov optimization algorithm. The second optimization module is used to transform the original effective capacity optimization problem into a Markov decision process. The joint optimization module is used to jointly optimize the IRS phase shift and device transmission power according to the SAC algorithm to obtain the optimized data transmission strategy.

10. A terminal, characterized in that, include: The processor and memory, wherein the memory stores a data transmission optimization program for a communication system, which, when executed by the processor, is used to implement the operation of the data transmission optimization method for the communication system as described in any one of claims 1-8.

11. A medium, characterized in that, The medium is a computer-readable storage medium that stores a data transmission optimization program for a communication system. When executed by a processor, the data transmission optimization program for the communication system is used to implement the operation of the data transmission optimization method for the communication system as described in any one of claims 1-8.

Images