RIS-assisted communication and computing resource allocation joint optimization method, system and program product in NOMA-MEC network
By tightly coupling the communication process with the computing process in the RIS-assisted NOMA-MEC network, decompose it into two nested solutions sub-models and adopting Lagrangian dual algorithm and deep reinforcement learning model, the neglected in the existing technology is solved. The performance reduction problems caused by the computing process achieve minimization of system energy consumption and performance improvement.
Patent Information
- Application Number
- CN202410995248.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-24
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-07-24
AI Technical Summary
The prior art ignores the computing process in the RIS-assisted NOMA-MEC network, making it difficult to obtain an optimal solution during communication and reduces system performance.
By using a method that closely couples the communication process with the calculation process, the optimization sub-model and the reflection surface phase shift optimization sub-model are combined into task offload ratio and communication process, and the optimal task is solved separately using the Lagrangian dual algorithm and the deep reinforcement learning model. The unloading ratio and reflective surface phase shift can optimize transmission power, transmission time and server computing resource allocation.
It minimizes the long-term energy consumption of users and servers, and improves the overall performance of NOMA-MEC network.
Smart Images

Figure CN119012279A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of MEC network technology, and in particular to a joint optimization method, system and program product for communication and computing resource allocation in a RIS-assisted NOMA-MEC network. Background Art
[0002] In the framework of Mobile Edge Computing (MEC), by offloading tasks to MEC servers for computing, system latency and energy consumption are significantly reduced. Non-orthogonal Multiple Access (NOMA) technology and Reconfigurable Intelligent Surface (RIS) technology further improve spectrum utilization efficiency and controllability of wireless channels, effectively improving the performance of MEC systems.
[0003] The optimization of resource allocation in RIS-assisted NOMA-MEC networks has aroused extensive discussion in academia and industry. Most current solutions only consider the communication process and ignore the computing process. The communication process and the computing process are tightly coupled. Ignoring the computing process will make it difficult to obtain the optimal solution for the relevant variables in the communication process, thereby reducing the overall performance of the system. Summary of the invention
[0004] In view of the problems existing in the prior art, the purpose of the present invention is to provide a RIS-assisted joint optimization method, system and program product for communication and computing resource allocation in a NOMA-MEC network, which tightly couples the communication process with the computing process and improves the overall performance of the NOMA-MEC network system.
[0005] To achieve the above object, the technical solution adopted by the present invention is:
[0006] A RIS-assisted joint optimization method for communication and computing resource allocation in a NOMA-MEC network, the method decomposes the original model into two nested sub-models, namely, a task offloading ratio and communication process joint optimization sub-model and a reflection surface phase shift optimization sub-model; for the task offloading ratio and communication process joint optimization sub-model, it is proved through theoretical derivation that when the reflection surface phase shift is given, the optimal transmission power is expressed as a function of the NOMA group transmission time and the task offloading ratio, and then the Lagrangian dual algorithm is used to solve the optimal task offloading ratio and the optimal NOMA group transmission time; for the reflection surface phase shift optimization sub-model, it is expressed as a deep reinforcement learning model, and the proximal strategy optimization algorithm is used to solve it, and the optimal reflection surface phase shift is obtained; finally, the optimal reflection surface phase shift is substituted into the task offloading ratio and communication process joint optimization sub-model for solving again, and the optimal offloading ratio and the optimal transmission time are obtained, and then the optimal transmission power, the optimal local computing frequency and the optimal server computing resource allocation are obtained.
[0007] The method is applicable to the NOMA-MEC network assisted by RIS, where the MEC server is connected to the BS, and a K-element RIS assists N single-antenna users in computing offload to the MEC server; the user composition set Reflective surface element assembly The time of the entire NOMA-MEC system is divided into equally spaced discrete time slots The length of each time slot is τ. In each time slot t, the user has a task to be calculated. The user adopts a partial offloading strategy to offload part of the task to the edge server for calculation, and calculates the rest locally.
[0008] The method specifically comprises the following steps:
[0009] Step 1: Build the original model;
[0010] The computation task of user n in time slot t is represented by the tuple Characterization, where L n (t) represents the task data volume of user n in time slot t, C n (t) represents the number of CPU cycles required for user n to calculate each bit task in time slot t, represents the maximum task delay allowed for user n in time slot t. The task needs to be All users’ tasks can always be completed within one time slot;
[0011] Assume that the proportion of the data volume unloaded by user n in time slot t to the total task data volume is λ n (t), the local calculation frequency of user n in time slot t is The local computing delay at this time is It is expressed as:
[0012]
[0013] Energy consumption of local computing It is expressed as:
[0014]
[0015] Among them, κ1 is the energy consumption coefficient;
[0016] The computing power of users is not unlimited, and there are computing resource constraints, so there are:
[0017]
[0018] In addition, the total latency of local computing must meet the maximum latency constraint of the user task, that is:
[0019]
[0020] use Representing the complex space of x×y dimensions, use represents the direct channel between user n and the server in time slot t, using Denote the channel from user n to RIS in time slot t, using represents the channel from RIS to the server in time slot t; using θ k (t)∈[0,2π] represents the phase shift of the kth element in time slot t, represents the phase shift matrix of RIS at time slot t, then the equivalent channel gain g between user n and the server at time slot t is n (t) is expressed as:
[0021]
[0022] All users in a NOMA group use the NOMA protocol for offloading. Suppose the transmission power of user n in time slot t is P n (t), the system bandwidth is B, and the system noise is σ 2 , then the unloading rate of user n in time slot t is expressed as:
[0023]
[0024] Assume that the transmission time of the NOMA group is d(t), then the transmission energy consumption of user n in time slot t is It is expressed as:
[0025]
[0026] When using the NOMA protocol for uninstallation, the uninstallation of all users should be completed, so the following constraints must be met:
[0027]
[0028] At the same time, there is a maximum transmission power limit for users, namely:
[0029]
[0030] Assume that the server computing resources allocated to user n in time slot t are The following constraints need to be satisfied:
[0031]
[0032] in is the total computing resource that the server can provide; the server computing delay of user n in time slot t is expressed as:
[0033]
[0034] The computational energy consumption generated by the server in time slot t for computing the offload task of user n is:
[0035]
[0036] Where κ2 is the energy consumption coefficient related to the server hardware;
[0037] The total delay of offloading calculation includes the transmission delay of NOMA group and the server calculation delay, which needs to meet the maximum delay constraint of user task, so:
[0038]
[0039] Based on the above, the optimization objective in each time slot is expressed as:
[0040]
[0041] The original model is expressed as
[0042]
[0043]
[0044]
[0045]
[0046]
[0047]
[0048]
[0049]
[0050]
[0051]
[0052] The goal of the optimization is to jointly optimize the NOMA group’s transmission time, local computing frequency, server computing resource allocation, user transmission power, task offloading ratio, and reflection surface phase shift to minimize the system’s long-term energy consumption;
[0053] Step 2: To transform;
[0054] Step 2.1, replace the local calculation frequency variable;
[0055] Lemma 1: The optimal solution for the frequency of local computation by user n at time slot t is Always meet:
[0056]
[0057] And there are:
[0058]
[0059] Bringing the result of Lemma 1 into the original model Get the model
[0060]
[0061]
[0062]
[0063]
[0064]
[0065]
[0066]
[0067]
[0068]
[0069] Step 2.2, replace the server calculation frequency variable;
[0070] Lemma 2: Model Medium Constraint The less than or equal sign can be replaced by an equal sign, and the optimal solution of the model remains unchanged; that is,
[0071]
[0072] According to Lemma 2, the server assigned to user n in time slot t calculates the frequency It is expressed as:
[0073]
[0074] Will Substitute the expression into the equation and we get:
[0075]
[0076] At the same time, in order to ensure that the calculation frequency of the assigned server is positive, a new constraint is introduced:
[0077]
[0078] Bringing the above results into the model The model can be further obtained
[0079]
[0080]
[0081]
[0082]
[0083]
[0084]
[0085]
[0086]
[0087]
[0088]
[0089] Step 3, optimizing other variables when the phase shift of the reflection surface is given;
[0090] Step 3.1: Solve the resource allocation optimization model for a given reflection surface phase shift in each time slot Consider each time slot Given the phase shift Θ of the reflection surface t Energy consumption minimization model under the condition of
[0091]
[0092]
[0093]
[0094]
[0095]
[0096]
[0097]
[0098]
[0099] Step 3.2, replace the transmit power optimization variable;
[0100] Lemma 3: Model Constraints in The less than or equal sign is replaced by an equal sign, and the optimal solution of the model remains unchanged; that is, we have:
[0101]
[0102] When the phase shift of the reflection surface in time slot t is given, the channel gain of the user is also determined accordingly; the user whose channel gain ranks qth in time slot t is denoted as s q (Θ t ), then the users are arranged in ascending order according to the channel gain as follows:
[0103]
[0104] According to Lemma 3, we can get the following according to different user numbers:
[0105]
[0106] So the transmit power is calculated by the following formula:
[0107]
[0108] From the above formula, we can get that when 2≤q≤N, It is always related to the transmission power of the user with worse channel quality. In this case, the user signal with lower channel gain is regarded as interference. When q≥2, Multiply both sides of the expression by get:
[0109]
[0110] After moving the items, we have:
[0111]
[0112] So when q ≥ 2, Calculated by continuous multiplication:
[0113]
[0114] Substituting the above derivation into In the expression, when q≥2:
[0115]
[0116] So, users q (Θ t ) is represented by d(t), The function is denoted as
[0117] In summary, users q (Θ t ) is expressed as:
[0118]
[0119] Through the above conversion, each time slot The model below can be reformulated as That is, the task offloading ratio and communication process are jointly optimized sub-model:
[0120]
[0121]
[0122]
[0123]
[0124]
[0125]
[0126]
[0127] d(t)≥0,
[0128] Step 3.3: Write the model The Langrangian function of , let:
[0129]
[0130]
[0131] Model The Lagrangian function of is expressed as:
[0132]
[0133] in,
[0134]
[0135] use Representation Model The set of Lagrange multipliers corresponding to each constraint in is: Therefore, the dual function of the original model is expressed as:
[0136]
[0137] The goal of the dual model is to maximize the dual function, namely:
[0138]
[0139] The dual model constraint is that all Lagrange multipliers are greater than or equal to 0;
[0140] Step 3.4: Find the optimal solution under the given Lagrange multiplier. At this time, d(t) and λ(t) are unconstrained for the Lagrange function. Use the coordinate descent algorithm to solve the given Lagrange multiplier. The optimal condition under and optimal Update each variable in a certain order and ensure that other variables are up to date when updating the variable. Let i be the number of iterations of the coordinate descent method. The coordinate descent algorithm is expressed as:
[0141]
[0142]
[0143] ...,
[0144]
[0145] The above process is repeated until convergence or the maximum number of iterations is reached;
[0146] Step 3.5, update the Lagrange multiplier;
[0147] get and After that, the user transmit power is calculated and recorded as Then update all Lagrange multipliers along the gradient direction of the Lagrange function with respect to the Lagrange multiplier, that is, for each And ψ5, their update directions are:
[0148]
[0149]
[0150]
[0151]
[0152] and
[0153]
[0154] Assume the update direction is ρ is the step size of each update, and the new Lalangrange multiplier is updated as:
[0155]
[0156] Step 3.6: Repeat steps 3.3 to 3.5 until the maximum number of iterations is reached;
[0157] Step 4: Express the reflection surface phase shift optimization model as a deep reinforcement learning model;
[0158] Step 4.1, define the state space and action space; the intelligent agents are based on the state to obtain better cumulative rewards, and the state is set as the channel information of the NOMA-MEC system; set is the set of all channel gains in time slot t, then the state is set to:
[0159]
[0160] The goal is to optimize the phase shift of the reflective surface, so the action space is set as follows:
[0161] a t =[θ1(t),θ2(t),...,θ k (t)],
[0162] Step 4.2: Define rewards. Based on rewards, the agent updates its strategy and establishes a mapping from state to action. The goal is to minimize the energy consumption of the system, so the reward for time slot t is set to:
[0163] r t =exp(-E total (t))+χ(t),
[0164] Among them, χ(t) represents the penalty, when at Make the model When there is no solution, χ(t)=-1, and E total The value of (t) is set to +∞, otherwise χ(t) = 0; in this way, the intelligent body will gradually abandon these invalid actions, and low system energy consumption can bring higher rewards. Through continuous training, maximizing long-term rewards can minimize long-term total energy consumption;
[0165] Step 4.3, using the proximal strategy optimization algorithm to solve the phase shift of the reflection surface;
[0166] Randomly initialize the parameters φ of the Actor network A and the Critic network parameter φ c , clear the experience pool; in each round, based on each state s t , the agent uses the strategy To select action a t , based on action a t , the RIS-assisted NOMA-MEC environment feeds back reward r to the agent t And enter the next state, and the experience <s t ,a t ,r t ,s t > Put it into the experience pool; after each round, update the PPO agent Γ times and use the Adam optimizer to update the Actor network parameter φ A and Critic network parameter φ c ; Repeat multiple rounds until the maximum number of training rounds Ω is reached, and the phase shift of the reflection surface is obtained;
[0167] Step 5: After training is completed, the near-optimal phase shift of the reflector surface can be generated in real time, and then the model in step 3 can be further used. The solution method is used to obtain the optimal offloading ratio and the optimal transmission time, and then the optimal transmission power, the optimal local computing frequency and the optimal server computing resource allocation.
[0168] A system for jointly optimizing the allocation of communication and computing resources in a RIS-assisted NOMA-MEC network, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of a method for jointly optimizing the allocation of communication and computing resources in a RIS-assisted NOMA-MEC network as described above.
[0169] A computer program product comprises a computer program / instruction, which, when executed by a processor, implements the steps of a joint optimization method for allocation of communication and computing resources in a RIS-assisted NOMA-MEC network as described above.
[0170] After adopting the above scheme, the present invention theoretically proves the relationship between the local computing frequency, server computing frequency and other optimization variables when the original model is optimal, reducing the number of variables that need to be directly optimized. Then the problem is decomposed into two nested sub-models, namely the task offloading ratio and communication process joint optimization sub-model and the reflection surface phase shift optimization sub-model, which are solved by Lagrange dual algorithm and PPO algorithm respectively, and finally the long-term energy consumption of users and servers is minimized. In this way, the present invention tightly couples the communication process with the computing process, improving the overall performance of the NOMA-MEC network system. BRIEF DESCRIPTION OF THE DRAWINGS
[0171] Figure 1 A scene graph to which the present invention is applicable;
[0172] Figure 2 The present invention is a flow chart of the method. DETAILED DESCRIPTION
[0173] like Figure 1 As shown in FIG. 1 , the present invention is applicable to a RIS-assisted NOMA-MEC network, where the MEC server is connected to the BS, and a K-element RIS assists N single-antenna users in offloading computations to the MEC server. Assuming that the user cannot communicate directly with the BS, this may be due to a service dead zone or being blocked by obstacles. Therefore, the task can only be offloaded to the BS with the assistance of the RIS, and the NOMA protocol is used for uplink transmission. The user composition set Reflective surface element assembly The time of the entire NOMA-MEC system is divided into equally spaced discrete time slots The length of each time slot is τ. In each time slot t, the user has a task to be calculated. The user adopts a partial offloading strategy to offload part of the task to the edge server for calculation, and the rest is calculated locally.
[0174] The present invention provides a joint optimization method for communication and computing resource allocation in a RIS-assisted NOMA-MEC network, which specifically includes the following steps:
[0175] Step 1: Build the original model.
[0176] The computation task of user n in time slot t is represented by the tuple Characterization, where L n (t) represents the task data volume of user n in time slot t, in bits, C n (t) represents the number of CPU cycles required by user n to calculate each bit task in time slot t, in cycles / bit. It represents the maximum task delay allowed for user n in time slot t, in ms. The task needs to be Consider the case where all users’ tasks can always complete the calculation within one time slot.
[0177] Assume that the proportion of the data volume unloaded by user n in time slot t to the total task data volume is λ n (t), which uses dynamic voltage frequency scaling technology. Assume that the local calculation frequency of user n in time slot t is The local computing delay at this time is It can be expressed as:
[0178]
[0179] Energy consumption of local computing It can be expressed as:
[0180]
[0181] Among them, κ1 is the energy consumption coefficient, which is related to the user's hardware. The user's computing power is not unlimited, and there are computing resource constraints, so there is:
[0182]
[0183] In addition, the total latency of local computing must meet the maximum latency constraint of the user task, that is:
[0184]
[0185] use Representing the complex space of x×y dimensions, use represents the direct channel between user n and the server in time slot t, using represents the channel from user n to RIS in time slot t, using represents the channel from RIS to the server in time slot t. By adjusting the phase shift of RIS, the equivalent channel between the user and the server can be dynamically adjusted. k (t)∈[0,2π] represents the phase shift of the kth element in time slot t, represents the phase shift matrix of RIS at time slot t, then the equivalent channel gain g between user n and the server at time slot t is n (t) can be expressed as:
[0186]
[0187] In the scenario of the present invention, all users in a NOMA group use the NOMA protocol for unloading. Assume that the transmission power of user n in time slot t is P n (t), the system bandwidth is B, and the system noise is σ 2 , then the unloading rate of user n in time slot t can be expressed as:
[0188]
[0189] Assume that the transmission time of the NOMA group is d(t), then the transmission energy consumption of user n in time slot t is It can be expressed as:
[0190]
[0191] When using the NOMA protocol for unloading, the unloading requirements of all users need to be considered, that is, the unloading of all users should be guaranteed to be completed, so the following constraints need to be met:
[0192]
[0193] At the same time, the user's transmission power is not unlimited, and there is a maximum transmission power limit, namely:
[0194]
[0195] For data offloaded to the server, the MEC server allocates computing resources to different users to complete the calculation. Compared with cloud servers, the computing resources of MEC servers are very limited. Suppose the server computing resources allocated to user n in time slot t are The following constraints need to be satisfied:
[0196]
[0197] in is the total computing resources that the server can provide. Similarly, the server computing delay of user n in time slot t is expressed as:
[0198]
[0199] The computational energy consumption generated by the server in time slot t for computing the offload task of user n is:
[0200]
[0201] Where κ2 is the energy consumption coefficient related to the server hardware. Since the size of the calculation result is usually much smaller than the input data, the energy consumption and delay of downloading from the MEC calculation to the user are ignored. The total delay of the offload calculation includes the transmission delay of the NOMA group and the server calculation delay, which needs to meet the maximum delay constraint of the user task, so:
[0202]
[0203] Based on the above discussion, the optimization objective for each time slot can be expressed as:
[0204]
[0205] In summary, the model is expressed as
[0206]
[0207]
[0208]
[0209]
[0210]
[0211]
[0212]
[0213]
[0214]
[0215]
[0216] The optimization goal is to jointly optimize the NOMA group's transmission time, local computing frequency, server computing resource allocation, user transmission power, task offloading ratio, and reflection surface phase shift to minimize the system's long-term energy consumption. This indicates that the user's local computing frequency is limited, and the constraint Indicates that the local computation delay cannot exceed the maximum allowed delay of the task, constraint It indicates that when using NOMA protocol transmission, the unloading of each user should be completed, and the constraints Indicates that the user's transmission power is limited, constrained This indicates that the server computing resources are limited and the constraints Indicates that the offloading delay cannot exceed the maximum allowed delay for the task.
[0217] Step 2: Transform the original model.
[0218] Step 2.1: Replace the local calculation frequency variable.
[0219] Lemma 1. The optimal solution for the local computation frequency of user n at time slot t is Always meet:
[0220]
[0221] And there are:
[0222]
[0223] Proof. By constraint and The following inequality can be obtained:
[0224]
[0225] If the original model has a solution, then the lower bound in the above formula must be less than or equal to the upper bound, that is Established, and because the objective function changes with is increasing, so the optimal solution for local calculation frequency must be The lower bound of The proof is complete.
[0226] Bringing the result of Lemma 1 into the model Models can be obtained
[0227]
[0228]
[0229]
[0230]
[0231]
[0232]
[0233]
[0234]
[0235]
[0236] Step 1.2: Replace the server calculation frequency variable.
[0237] Lemma 2. Model Medium Constraint The less than or equal sign can be replaced by an equal sign, and the optimal solution of the original model remains unchanged. That is,
[0238]
[0239] Proof. Suppose there is a user n in time slot t, and the optimal solution d(t) corresponding to it is # ,λ n (t) # and Make the inequality in the constraint strictly hold, that is:
[0240]
[0241] Then you can always find Make the constraint The equality sign in holds, and when Related constraints While being established, it also makes the total energy consumption of the system smaller. This contradicts the assumption of the optimal solution. Therefore, when obtaining the optimal solution, the constraint The equation is strictly true. Proof completed.
[0242] According to Lemma 2, the server assigned to user n in time slot t calculates the frequency It can be expressed as:
[0243]
[0244] Will By substituting the expression into , we can get:
[0245]
[0246] At the same time, in order to ensure that the calculation frequency of the assigned server is positive, new constraints need to be introduced:
[0247]
[0248] Bringing the above results into the model The model can be further obtained
[0249]
[0250]
[0251]
[0252]
[0253]
[0254]
[0255]
[0256]
[0257]
[0258]
[0259] Step 3: Optimize other variables when the phase shift of the reflection surface is given.
[0260] Step 3.1, solve the resource allocation optimization model when the phase shift of the reflection surface is given in each time slot.
[0261] Consider each time slot Given the phase shift Θ of the reflection surface t The energy consumption minimization model under the condition of
[0262]
[0263]
[0264]
[0265]
[0266]
[0267]
[0268]
[0269]
[0270]
[0271] Step 3.2: Replace the transmit power optimization variable
[0272] Lemma 3. Model Constraints in The less than or equal sign can be replaced by an equal sign, and the optimal solution of the original model remains unchanged. That is,
[0273]
[0274] Proof. Assume that for a certain user n, the optimal solution d(t) corresponding to it is # ,λ n (t) # and P n (t) # , so that the constraint The inequality sign in the string is strictly true, that is:
[0275]
[0276] We can always find d(t) * <d n (t) # , so that the constraint The equal sign holds. At this time, the constraints related to d(t) are:
[0277]
[0278] therefore Established; and because So we have:
[0279]
[0280] Therefore the constraint The local computing energy consumption remains unchanged, and the transmission energy consumption and server computing energy consumption are both smaller, so the total system energy consumption is smaller, which contradicts the assumption of the optimal solution. Therefore, when the optimal solution is obtained, the constraint The equation is strictly true. Proof completed.
[0281] When the phase shift of the reflection surface in time slot t is given, the channel gain of the user is also determined. The user whose channel gain ranks qth in time slot t is denoted as s q (Θ t ), then the users are arranged in ascending order according to the channel gain as follows:
[0282]
[0283] According to Lemma 3, we can get the following according to different user serial numbers:
[0284]
[0285] So the transmit power can be calculated by the following formula:
[0286]
[0287] From the above formula, we can get that when 2≤q≤N, It is always related to the transmission power of the user with worse channel quality. This is because the SIC decoding order based on channel gain is adopted, and the signal with higher channel gain is decoded first. At this time, the user signal with lower channel gain is regarded as interference. Further deduction is made. When q≥2, Multiply both sides of the expression by You can get:
[0288]
[0289] After moving the items, we have:
[0290]
[0291] So when q ≥ 2, It can be calculated by continuous multiplication:
[0292]
[0293] Substituting the above derivation into In the expression, we can get q≥2:
[0294]
[0295] So, users q (Θ t ) can be expressed as d(t), The function is denoted as In summary, users q (Θ t ) can be expressed as:
[0296]
[0297] Through the above conversion, each time slot The model below can be reformulated as
[0298]
[0299]
[0300]
[0301]
[0302]
[0303]
[0304]
[0305] d(t)≥0.
[0306] Step 2.3: Write the model The Langrangian function of , let:
[0307]
[0308]
[0309] Model The Lagrangian function can be expressed as:
[0310]
[0311] in,
[0312]
[0313] use Representation Model The set of Lagrange multipliers corresponding to each constraint in is: Therefore, the dual function of the original model can be expressed as:
[0314]
[0315] The goal of the dual model is to maximize the dual function, namely:
[0316]
[0317] The dual model constraint is that all Lagrange multipliers are greater than or equal to 0.
[0318] Step 3.4: Find the optimal solution given the Lagrange multiplier. At this time, d(t) and λ(t) are unconstrained for the Lagrange function. Use the coordinate descent algorithm to solve the given Lagrange multiplier. The optimal condition under and optimal Update each variable in a certain order and ensure that other variables are up to date when updating the variable. Let i be the number of iterations of the coordinate descent method. The coordinate descent algorithm can be expressed as:
[0319]
[0320]
[0321] ...,
[0322]
[0323] The above process is repeated until convergence or the maximum number of iterations is reached.
[0324] Step 3.5, update the Lagrange multiplier. Get and After that, the user transmit power can be calculated and recorded as Then update all Lagrange multipliers along the gradient direction of the Lagrange function with respect to the Lagrange multiplier, that is, for each And ψ5, their update directions are:
[0325]
[0326]
[0327]
[0328]
[0329] and
[0330]
[0331] Assume the update direction is ρ is the step size of each update, and the new Lalangrange multiplier is updated as:
[0332]
[0333] Step 3.6: Repeat steps 3.3 to 3.5 until the maximum number of iterations is reached.
[0334] Step 4: Express the reflection surface phase shift optimization model as a deep reinforcement learning model.
[0335] Step 4.1, define the state space and action space. The agents are based on the state to obtain better cumulative rewards, and the state is set as the channel information of the NOMA-MEC system. is the set of all channel gains in time slot t, then the state is set to:
[0336]
[0337] The goal is to optimize the phase shift of the reflective surface, so the action space is set as follows:
[0338] a t =[θ1(t),θ2(t),...,θ k (t)].
[0339] Step 4.2, define rewards, which reflect what behavior decisions are beneficial to the system. Based on the rewards, the agent updates the strategy and establishes a mapping from state to action. The goal is to minimize the energy consumption of the system, so the reward for time slot t is set to:
[0340] r t =exp(-E total (t))+χ(t),
[0341] Where χ(t) represents the penalty, when a t Make the model When there is no solution, χ(t)=-1, and E total The value of (t) is set to +∞, otherwise χ(t) = 0. In this way, the intelligent body will gradually abandon these invalid actions, and low system energy consumption can bring higher rewards. Through continuous training, maximizing long-term rewards can minimize long-term total energy consumption.
[0342] Step 4.3: Use the Proximal Policy Optimization (PPO) algorithm to solve the phase shift of the reflection surface.
[0343] Randomly initialize the parameters φ of the Actor network A and the Critic network parameter φ c , clear the experience pool. In each round, based on each state s t , the agent uses the strategy To select action a t , based on action a t , the RIS-assisted NOMA-MEC environment feeds back reward r to the agent t And enter the next state, and the experience <s t ,a t ,r t ,s t > Put it into the experience pool. After each round, the PPO agent is updated Γ times, and the Actor network parameters φ are updated using the Adam optimizer A and Critic network parameter φ c . Repeat multiple rounds until the maximum number of training rounds Ω is reached.
[0344] Step 5: After the training is completed, a near-optimal reflection surface phase shift can be generated in real time, and then the solution method of the model in step 3 can be further used to obtain the optimal offloading ratio and the optimal transmission time, thereby obtaining the optimal transmission power, the optimal local computing frequency, and the optimal server computing resource allocation.
[0345] The key to the present invention is that the present invention decomposes the model into two nested sub-models, namely, the task offloading ratio and communication process joint optimization sub-model and the reflection surface phase shift optimization sub-model. For the former, it is proved through theoretical derivation that when the reflection surface phase shift is given, the optimal transmission power can be expressed as a function of the NOMA group transmission time and the task offloading ratio, and then the Lagrangian dual algorithm is used to solve the optimal task offloading ratio and the optimal NOMA group transmission time; for the latter, it is expressed as a deep reinforcement learning model and solved using the PPO algorithm. In this way, the present invention tightly couples the communication process with the computing process, thereby improving the overall performance of the NOMA-MEC network system.
[0346] The present invention also provides a joint optimization system for communication and computing resource allocation in a RIS-assisted NOMA-MEC network, which includes a memory, a processor, and a computer program stored on the memory, and the processor executes the computer program to implement the steps of the joint optimization method for communication and computing resource allocation in a RIS-assisted NOMA-MEC network as described above.
[0347] The present invention further provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device.
[0348] The computer-readable medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiments.
[0349] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.
[0350] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. A joint optimization method for communication and computing resource allocation in a RIS-assisted NOMA-MEC network, characterized by: The method decomposes the original model into two nested sub-models, namely, the task offloading ratio and communication process joint optimization sub-model and the reflection surface phase shift optimization sub-model; for the task offloading ratio and communication process joint optimization sub-model, it is proved through theoretical derivation that when the reflection surface phase shift is given, the optimal transmission power is expressed as a function of the NOMA group transmission time and the task offloading ratio, and then the Lagrangian dual algorithm is used to solve the optimal task offloading ratio and the optimal NOMA group transmission time; for the reflection surface phase shift optimization sub-model, it is expressed as a deep reinforcement learning model, and the proximal strategy optimization algorithm is used to solve it to obtain the optimal reflection surface phase shift; finally, the optimal reflection surface phase shift is substituted into the task offloading ratio and communication process joint optimization sub-model for solving again, and the optimal offloading ratio and the optimal transmission time are obtained, thereby obtaining the optimal transmission power, the optimal local computing frequency and the optimal server computing resource allocation.
2. According to claim 1, a joint optimization method for communication and computing resource allocation in a RIS-assisted NOMA-MEC network is characterized in that: The method is applicable to the NOMA-MEC network assisted by RIS, where the MEC server is connected to the BS, and a K-element RIS assists N single-antenna users in computing offload to the MEC server; the user composition set Reflective surface element assembly The time of the entire NOMA-MEC system is divided into equally spaced discrete time slots The length of each time slot is τ. In each time slot t, the user has a task to be calculated. The user adopts a partial offloading strategy to offload part of the task to the edge server for calculation, and calculates the rest locally. The method specifically comprises the following steps: Step 1: Build the original model; The computation task of user n in time slot t is represented by the tuple Characterization, where L n (t) represents the task data volume of user n in time slot t, C n (t) represents the number of CPU cycles required for user n to calculate each bit task in time slot t, represents the maximum task delay allowed for user n in time slot t. The task needs to be All users’ tasks can always be completed within one time slot; Assume that the proportion of the data volume unloaded by user n in time slot t to the total task data volume is λ n (t), the local calculation frequency of user n in time slot t is The local computing delay at this time is It is expressed as: Energy consumption of local computing It is expressed as: Among them, κ1 is the energy consumption coefficient; The user's computing power is not unlimited, and there are computing resource constraints, so there are: In addition, the total latency of local computing must meet the maximum latency constraint of the user task, that is: use Representing the complex space of x×y dimensions, use represents the direct channel between user n and the server in time slot t, using Denote the channel from user n to RIS in time slot t, using represents the channel from RIS to the server in time slot t; using θ k (t)∈[0,2π] represents the phase shift of the kth element in time slot t, represents the phase shift matrix of RIS at time slot t, then the equivalent channel gain g between user n and the server at time slot t is n (t) is expressed as: All users in a NOMA group use the NOMA protocol for offloading. Suppose the transmission power of user n in time slot t is P n (t), the system bandwidth is B, and the system noise is σ 2 , then the unloading rate of user n in time slot t is expressed as: Assume that the transmission time of the NOMA group is d(t), then the transmission energy consumption of user n in time slot t is It is expressed as: When using the NOMA protocol for uninstallation, the uninstallation of all users should be completed, so the following constraints must be met: At the same time, there is a maximum transmission power limit for users, namely: Assume that the server computing resources allocated to user n in time slot t is f n BS (t), the following constraints need to be satisfied: in is the total computing resource that the server can provide; the server computing delay of user n in time slot t is expressed as: The computational energy consumption generated by the server in time slot t for computing the offload task of user n is: Where κ2 is the energy consumption coefficient related to the server hardware; The total delay of offloading calculation includes the transmission delay of NOMA group and the server calculation delay, which needs to meet the maximum delay constraint of user task, so: Based on the above, the optimization objective in each time slot is expressed as: The original model is expressed as The goal of the optimization is to jointly optimize the NOMA group’s transmission time, local computing frequency, server computing resource allocation, user transmission power, task offloading ratio, and reflection surface phase shift to minimize the system’s long-term energy consumption; Step 2: To transform; Step 2.1, replace the local calculation frequency variable; Lemma 1: The optimal solution for the frequency of local computation by user n at time slot t is Always meet: And there are: Bringing the result of Lemma 1 into the original model Get the model Step 2.2, replace the server calculation frequency variable; Lemma 2: Model Medium Constraint The less than or equal sign can be replaced by an equal sign, and the optimal solution of the model remains unchanged; that is, According to Lemma 2, the server assigned to user n in time slot t calculates the frequency f n BS (t) is expressed as: f n BS Substituting the expression of (t) into the equation, we get: At the same time, in order to ensure that the calculation frequency of the assigned server is positive, a new constraint is introduced: Bringing the above results into the model The model can be further obtained Step 3, optimizing other variables when the phase shift of the reflection surface is given; Step 3.1: Solve the resource allocation optimization model for a given reflection surface phase shift in each time slot Consider each time slot Given the phase shift Θ of the reflection surface t Energy consumption minimization model under the condition of Step 3.2, replace the transmit power optimization variable; Lemma 3: Model Constraints in The less than or equal sign is replaced by an equal sign, and the optimal solution of the model remains unchanged; that is, we have: When the phase shift of the reflection surface in time slot t is given, the channel gain of the user is also determined accordingly; the user whose channel gain ranks qth in time slot t is denoted as s q (Θ t ), then the users are arranged in ascending order according to the channel gain as follows: According to Lemma 3, we can get the following according to different user numbers: So the transmit power is calculated by the following formula: From the above formula, we can get that when 2≤q≤N, It is always related to the transmission power of the user with worse channel quality. In this case, the user signal with lower channel gain is regarded as interference. When q≥2, Multiply both sides of the expression by get: After moving the items, we have: So when q ≥ 2, Calculated by continuous multiplication: Substituting the above derivation into In the expression, when q≥2: So, users q (Θ t ) is represented by d(t), The function is denoted as In summary, users q (Θ t ) is expressed as: Through the above conversion, each time slot The model below can be reformulated as That is, the task offloading ratio and communication process are jointly optimized sub-model: d(t)≥0, Step 3.3: Write the model The Langrangian function of , let: Model The Lagrangian function is expressed as: in, use Representation Model The set of Lagrange multipliers corresponding to each constraint in is: Therefore, the dual function of the original model is expressed as: The goal of the dual model is to maximize the dual function, namely: The dual model constraint is that all Lagrange multipliers are greater than or equal to 0; Step 3.4: Find the optimal solution under the given Lagrange multiplier. At this time, d(t) and λ(t) are unconstrained for the Lagrange function. Use the coordinate descent algorithm to solve the given Lagrange multiplier. The optimal condition under and optimal Update each variable in a certain order and ensure that other variables are up to date when updating the variable. Let i be the number of iterations of the coordinate descent method. The coordinate descent algorithm is expressed as: The above process is repeated until convergence or the maximum number of iterations is reached; Step 3.5, update the Lagrange multiplier; get and After that, the user transmit power is calculated and recorded as Then update all Lagrange multipliers along the gradient direction of the Lagrange function with respect to the Lagrange multiplier, that is, for each And ψ5, their update directions are: and Assume the update direction is ρ is the step size of each update, and the new Lalangrange multiplier is updated as: Step 3.6: Repeat steps 3.3 to 3.5 until the maximum number of iterations is reached; Step 4: Express the reflection surface phase shift optimization model as a deep reinforcement learning model; Step 4.1, define the state space and action space; the intelligent agents are based on the state to obtain better cumulative rewards, and the state is set as the channel information of the NOMA-MEC system; set is the set of all channel gains in time slot t, then the state is set to: The goal is to optimize the phase shift of the reflective surface, so the action space is set as follows: a t =[θ1(t),θ2(t),...,θ k (t)], Step 4.2: Define rewards. Based on rewards, the agent updates its strategy and establishes a mapping from state to action. The goal is to minimize the energy consumption of the system, so the reward for time slot t is set to: r t =exp(-E total (t))+χ(t), Among them, χ(t) represents the penalty, when a t Make the model When there is no solution, χ(t)=-1, and E total The value of (t) is set to +∞, otherwise χ(t) = 0; in this way, the intelligent body will gradually abandon these invalid actions, and low system energy consumption can bring higher rewards. Through continuous training, maximizing long-term rewards can minimize long-term total energy consumption; Step 4.3, using the proximal strategy optimization algorithm to solve the phase shift of the reflection surface; Randomly initialize the parameters φ of the Actor network A and the Critic network parameter φ c , clear the experience pool; in each round, based on each state s t , the agent uses the strategy To select action a t , based on action a t , the RIS-assisted NOMA-MEC environment feeds back reward r to the agent t And enter the next state, and experience <s t ,a t ,r t ,s t > Put it into the experience pool; after each round, update the PPO agent Γ times and use the Adam optimizer to update the Actor network parameter φ A and Critic network parameter φ c ; Repeat multiple rounds until the maximum number of training rounds Ω is reached, and the phase shift of the reflection surface is obtained; Step 5: After training is completed, the near-optimal phase shift of the reflector surface can be generated in real time, and then the model in step 3 can be further used. The solution method is used to obtain the optimal offloading ratio and the optimal transmission time, and then the optimal transmission power, the optimal local computing frequency and the optimal server computing resource allocation.
3. A RIS-assisted joint optimization system for communication and computing resource allocation in NOMA-MEC networks, characterized in that: It includes a memory, a processor and a computer program stored in the memory, and the processor executes the computer program to implement the steps of a joint optimization method for communication and computing resource allocation in a RIS-assisted NOMA-MEC network as described in claim 1 or 2.
4. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by the processor, the steps of the joint optimization method of communication and computing resource allocation in a RIS-assisted NOMA-MEC network as described in claim 1 or 2 are implemented.
Citation Information
Patent Citations
SWIPT-assisted NOMA-MEC system resource allocation modeling method
CN114521023A
Mobile edge calculation safety rate maximization method based on TD3 reinforcement learning and intelligent reflecting surface
CN117979314A
User equipment, base station and method performed by the same in wireless communication system
US20230247416A1
Method and device for transmitting and receiving signal by using time offset in time-asynchronous non-orthogonal multiple access system
US20240396769A1
