6G computing power network-oriented computing unloading method and terminal

Abstract

The invention discloses a 6G computing power network oriented computing unloading method and a terminal in the field of computing power networks and computing unloading methods. The method comprises the following steps: constructing a computing power network model based on user local equipment, a low earth orbit satellite and a remote cloud data center; determining an unloading proportion of the user calculation task; establishing a communication model, and calculating transmission rates of different communication paths according to a Shannon formula; calculating time delay and energy consumption of task processing according to characteristics of a user calculation task and transmission rates of different communication paths; establishing a time delay and energy consumption model; the method comprises the following steps: establishing a global optimization objective function by taking minimization of user service quality perception cost in an interaction process of a user and a computing power network environment as an objective, solving an optimal solution of the global optimization objective function, and generating an optimal unloading action of each user. According to the method, optimized computing unloading is realized in the 6G computing power network, and the completion efficiency of a user computing task, the user experience and the system energy efficiency are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of computing power networks and computing offloading methods, and in particular to a computing offloading method and terminal for 6G computing power networks. Background Art

[0002] The rapid development of 6G communication technology in today's society has led to an increase in the demand for seamless communication coverage and efficient computing worldwide. As an important architecture for the future development of 6G communications, the Space-Ground Integrated Network (SGIN) can not only achieve communication coverage in remote areas (such as deserts, oceans, forests and mountains) by integrating satellites and ground networks, but also enhance communication resilience in extreme environments such as natural disasters. However, with the continuous advancement of 5G and artificial intelligence technologies, the application of computer vision, natural language processing, autonomous driving, smart wearable devices and other fields has become increasingly widespread, bringing a large number of computing-intensive tasks and placing higher requirements on resources, storage and energy consumption. When dealing with such tasks, users prefer low-latency and high-efficiency computing support, especially those in remote areas outside the coverage of ground cellular networks. Therefore, the Space-Ground Integrated Computing Power Network (SGICPN) came into being to meet users' demand for computing resources in complex scenarios.

[0003] At present, there are some research methods for computing task offloading on the market, such as methods based on heuristic algorithms and single-agent reinforcement learning methods, which have improved the efficiency of computing resource allocation to a certain extent. However, these methods have many shortcomings. First, many methods are difficult to flexibly adjust the offloading strategy in a multi-satellite dynamic collaborative environment; second, the lack of effective modeling of satellite mobility and its coverage time changes limits its effectiveness in practical applications; third, in a multi-agent environment, traditional solutions fail to fully optimize system latency and energy consumption, and it is difficult to meet the computing power network's needs for efficient resource management. In addition, the current research on collaborative computing of satellite nodes, edge computing nodes and remote cloud data centers is still relatively preliminary. Many methods only consider the computing offloading of local and single satellites in multi-satellite collaborative scenarios, and fail to effectively combine the computing power resources of remote cloud data centers, making it difficult to meet the needs of large-scale computing tasks. Summary of the invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a computing offloading method and terminal for a 6G computing power network, which can significantly improve the completion efficiency of user computing tasks and user experience.

[0005] To achieve the above object, the present invention is implemented by adopting the following technical solutions:

[0006] In a first aspect, the present invention provides a method for offloading calculations for a 6G computing network, comprising:

[0007] A computing network model is constructed based on user local devices, low-Earth orbit satellites and remote cloud data centers, and the communication method between user local devices, remote cloud data centers and low-Earth orbit satellites is defined according to the constructed computing network model;

[0008] According to the user's offloading decision, determine the offloading ratio of the user's computing tasks to the user's local device, low-Earth orbit satellite and remote cloud data center for computing;

[0009] Establish a communication model between user local devices, remote cloud data centers and low-Earth orbit satellites, and calculate the transmission rates of different communication paths based on the Shannon formula;

[0010] According to the characteristics of the user's computing tasks and the transmission rates of different communication paths, the time delay and energy consumption of the user's local device, low-Earth orbit satellite and remote cloud data center for processing the user's computing tasks are calculated in combination with the communication mode;

[0011] Based on the unloading ratio, delay and energy consumption, a delay model and an energy consumption model are established;

[0012] Based on the remaining power status of the user's local device, weights are assigned to the latency model and energy consumption model with the goal of reducing latency or reducing energy consumption;

[0013] A global optimization objective function is established with the goal of minimizing the user service quality perception cost in the process of interaction between the user and the computing network environment;

[0014] The optimal solution of the global optimization objective function is solved to generate the optimal uninstallation action for each user.

[0015] Furthermore, the computing power network model includes N user local devices, a satellite cluster network consisting of M low-Earth orbit satellites, and a remote cloud data center;

[0016] The communication method between the user local device, the remote cloud data center and the low-earth orbit satellite includes:

[0017] In the time interval T, there is a constant Low-Earth orbit satellites can communicate with user local devices, and user local devices directly use the computing resources of low-Earth orbit satellites within their communication coverage;

[0018] The low Earth orbit satellite processes user computing tasks as a computing node or forwards user computing tasks to a remote cloud data center as a relay node; where N and are both integers greater than or equal to 1, <M, the calculation formula for the time interval T is as follows:

[0019] ;

[0020] ;

[0021] ;

[0022] In the formula: represents the central angle of the Earth corresponding to the coverage area of the low Earth orbit satellite; R e represents the radius of the Earth; H represents the orbital altitude of the low Earth orbit satellite; represents the elevation angle between the user's local device and the low Earth orbit satellite; L represents the orbital arc length passed by the low Earth orbit satellite during the coverage time; represents the running speed of the low Earth orbit satellite.

[0023] Furthermore, the offloading decision of the user is defined as:

[0024] ;

[0025] In the formula: represents the proportion of the user computing task offloaded to the user's local device for computing; represents the proportion of the user computing task offloaded to the low Earth orbit satellite for computing; represents the proportion of the user computing task offloaded to the remote cloud data center for computing; , and are subject to the following relational expressions:

[0026] { a n ( t ) + b n ( t ) + c n ( t ) = 1 a n ( t ), b n ( t ), c n ( t ) ∈ [ 0 , 1 ] .

[0027] Furthermore, the communication model between the user's local device and the low Earth orbit satellite is defined as:

[0028] ;

[0029] In the formula: represents the data transmission rate of the ground-air communication between the user's local device of the nth user and the mth low Earth orbit satellite; represents the transmission bandwidth between the user's local device and the low Earth orbit satellite; represents the number of users who choose to offload their computing tasks to the mth low-Earth orbit satellite. Users offloaded to the same low-Earth orbit satellite share bandwidth resources in an equal manner; represents the transmission rate of the user local device of the nth user; represents the channel gain between the user local device of the nth user and the mth low-Earth orbit satellite at time t; represents the channel noise power;

[0030] The communication model between the low-Earth orbit satellite and the remote cloud data center is defined as:

[0031] ;

[0032] Where: represents the data transmission rate between the mth low-Earth orbit satellite and the remote cloud data center; represents the congestion factor; Represents the transmission bandwidth between low-Earth orbit satellites and remote cloud data centers; represents the number of users who choose to offload their computing tasks to remote cloud data centers; represents the transmission rate of the mth low-Earth orbit satellite; represents the channel gain between the mth low-Earth orbit satellite and the remote cloud data center at time t.

[0033] Furthermore, the characteristics of the user computing task Defined as:

[0034] ;

[0035] Where: Indicates the data size of the user's computing task, in bytes; Indicate completion The amount of computation required by the CPU to calculate the data volume, in units of CPU cycles; Indicate completion The amount of computation required by the GPU in the data volume, in units of GPU floating-point operations; Indicates the maximum delay time tolerated by the user; Indicates the maximum energy consumption tolerated by the user;

[0036] The method of calculating the time delay and energy consumption of the user's local device, the low-earth orbit satellite and the remote cloud data center for processing the user's computing task in combination with the communication mode according to the characteristics of the user's computing task and the transmission rates of different communication paths includes:

[0037] When the user computing task is offloaded to the user's local device, only the computing delay of the user computing task is considered. The calculation formula of the delay is as follows:

[0038] ;

[0039] Where: It represents the delay required to complete the user computing task when the user computing task is offloaded to the local device of the nth user at time t; Indicates the CPU computing capability of the local device of the nth user; Indicates the GPU computing capability of the local device of the nth user;

[0040] The energy consumption calculation formula is as follows:

[0041] ;

[0042] Where: It represents the energy consumption required to complete the user computing task when the user computing task is offloaded to the local device of the nth user at time t; represents the capacitance parameter of the user local device of the nth user; represents the dynamic power consumption of the GPU of the user local device of the nth user;

[0043] When the user computing task is offloaded to a low-Earth orbit satellite, it is necessary to consider the satellite-to-ground link propagation delay, data transmission delay, and the calculation delay of the user computing task. The delay calculation formula is as follows:

[0044] ;

[0045] Where: It represents the delay required to complete the user computing task when the user computing task is offloaded to the low-Earth orbit satellite computing at time t; represents the data transmission delay from the local user device of the nth user to the mth low-Earth orbit satellite; Indicates the computational latency of the user's computational task; It represents the CPU computing capacity of the mth low-Earth orbit satellite allocated to the user computing task; represents the GPU computing power allocated by the mth low-Earth orbit satellite to the user's computing task; d represents the distance between the low-Earth orbit satellite and the user's local device, ; represents the speed of light; represents the round-trip propagation delay from the Earth to the LEO satellite;

[0046] The energy consumption calculation formula is as follows:

[0047] ;

[0048] Where: It represents the energy consumption required to complete the user computing task when the user computing task is offloaded to the low-Earth orbit satellite computing at time t; represents the transmission energy consumption of the user local device of the nth user; represents the capacitance parameter of the mth low-Earth orbit satellite; represents the GPU dynamic power consumption of the mth low-Earth orbit satellite;

[0049] When the user's computing tasks are offloaded to a remote cloud data center, the data transmission delay of the data being sent to the remote cloud data center via the low-Earth orbit satellite relay needs to be considered. The delay calculation formula is as follows:

[0050] ;

[0051] Where: It represents the delay required to complete the user computing task when the user computing task is offloaded to the remote cloud data center for computing at time t; represents the data transmission delay from the mth low-Earth orbit satellite to the remote cloud data center; represents the round-trip propagation delay from the Earth via a low-Earth orbit satellite to a remote cloud data center;

[0052] The energy consumption calculation formula is as follows:

[0053] ;

[0054] Where: It represents the energy consumption required to complete the user computing task when the user computing task is offloaded to the remote cloud data center at time t; represents the transmission energy consumption of the mth low-Earth orbit satellite; combining the unloading ratio, delay and energy consumption, establish a delay model and energy consumption model , as shown below:

[0055] ; .

[0056] Furthermore, the user service quality perception cost As shown below:

[0057] ;

[0058] Where: Representation delay model The weight of Energy consumption model The weight of ω L , ω E ∈[0, 1] ,and ; According to the remaining power of different users' local devices, and The value of is set as follows:

[0059] ;

[0060] Where: Indicates the current battery power percentage of the user's local device.

[0061] Furthermore, with the goal of minimizing the user service quality perception cost in the process of interaction between the user and the computing network environment, a global optimization objective function is established, including:

[0062] By accumulating the user service quality perception cost at each moment during the interaction between the user and the computing network environment, the global optimization objective function is obtained as shown in the following formula:

[0063] ;

[0064] Where: Indicates the termination time of the user's interaction with the computing network environment; express The user's perceived cost of service quality at each moment.

[0065] Furthermore, solving the optimal solution of the global optimization objective function and generating the optimal uninstallation action for each user includes:

[0066] Define a state set to describe all possible states of each user at time t; the state includes the user's user computing task information, offloading decision, and resource allocation and offloading of low-Earth orbit satellites. The state set at time t As shown below:

[0067] ;

[0068] in,

[0069] ;

[0070] Where: Represents the user computing task information of each user at time t; Indicates the user computing task information of the nth user at time t; represents the current uninstall decision of each user at time t; represents the uninstall decision of the nth user at time t; Indicates the resource allocation of each low-Earth orbit satellite at time t; Indicates the resource allocation of the mth low-Earth orbit satellite at time t; Indicates the load conditions of each low-Earth orbit satellite at time t; Indicates the load of the mth low-Earth orbit satellite at time t;

[0071] in,

[0072] { I n ( t ) = { Task n , ε n } Ω m ( t ) = { A CPU m ( t ), A GPU m ( t ), F max LEO , TP max LEO , num n } A CPU m ( t ) = [ f 1 m , LEO , f 2 m , LEO , … , f num n m , LEO ] A GPU m ( t ) = [ TP 1 m , LEO , TP 2 m , LEO , … , TP num n m , LEO ] Ψ m ( t ) = { U CPU m ( t ), U GPU m ( t ), N queue m ( t )} ;

[0073] Where: Indicates the current battery power percentage of the local device of the nth user; and They represent the CPU computing resources and GPU computing resources allocated to the user by the mth low-Earth orbit satellite; represents the number of users unloaded to the mth low-Earth orbit satellite; Indicates that the mth low-Earth orbit satellite is assigned to The CPU computing power of the user computing tasks of each user; Indicates that the mth low-Earth orbit satellite is assigned to GPU computing power for user computing tasks of each user; Indicates the current CPU usage of the mth low-Earth orbit satellite; Indicates the current GPU usage rate of the mth low-Earth orbit satellite; represents the number of user computing tasks currently queued in the mth low-Earth orbit satellite;

[0074] Define an uninstall action set to describe the uninstall actions taken by each user at time t in the state described above. As shown below:

[0075] ;

[0076] in,

[0077] ;

[0078] Where: Indicates the uninstall action of the nth user at time t; represents a target low-Earth orbit satellite for unloading;

[0079] The unloading actions taken by each user, the resource allocation of each low-Earth orbit satellite, and the load of each low-Earth orbit satellite are subject to the following conditional constraints:

[0080] ;

[0081] ;

[0082] ;

[0083] ;

[0084] ;

[0085] ;

[0086] Where: Indicates the delay required for a single user to complete a user computing task; It represents the energy consumption required by a single user to complete the user computing task; Indicates the maximum CPU computing power of a low-Earth orbit satellite; Indicates the maximum GPU computing power of low-Earth orbit satellites;

[0087] The reward function is defined by combining the state set and the uninstall action set to describe the immediate reward obtained by each user when taking the uninstall action in the state , the reward function is defined as:

[0088]

[0089] Where: represents the penalty factor;

[0090] The deep learning model is trained using the state set, the uninstallation action set and the reward function to generate the optimal uninstallation action for each user.

[0091] Furthermore, the training of the deep learning model using the state set, the unloading action set and the reward function includes:

[0092] Initialize the state collection , Uninstall action set , Experience Replay Pool , batch size , discount factor , soft update coefficient ;

[0093] Initializing the Actor Network , so that it is consistent with the initial state set The action set of each user Guaranteed consistency; among them, the initial state set In this case, each user only chooses to offload the user computing task to the user's local device for computing, and the bandwidth resources, CPU and GPU computing resources of the low-Earth orbit satellite and remote cloud data center are not allocated;

[0094] Initialize two Critic networks and And the corresponding target network, the target network includes the target Actor network And two target critic networks ;

[0095] Among them, Actor network Used to collect data based on the current state Generate uninstall action set , ; and is the Q value calculated by the two Critic networks, indicating that in the state set Take the following uninstall action set the long-term rewards that can be obtained; Represents the target Actor network generated unloading action; and Respectively represent the Q values ​​calculated by the two target Critic networks;

[0096] Initialize the number of training rounds, the number of training steps per round, and the delay update step size Z, where the number of training steps is the total number of time steps executed in one round of training;

[0097] At the beginning of each round of training, reset the state set is the initial state set of the environment ,make ;

[0098] At each time step, each user first follows the strategy of the current Actor network. Select the initial set of uninstall actions , the initial uninstall action is given by the following formula Add Gaussian noise to the set to get the actual set of uninstall actions :

[0099] ;

[0100] Where: represents a mean of 0 and a variance of Gaussian noise;

[0101] Then the actual uninstall action set Acts on the computing power network environment and records the current state set , Uninstall action set , Instant Rewards , the next state set and Boolean variables The value of, if the current state set is the terminal state set , The value is false; otherwise, The value is true, indicating that there is another state set;

[0102] The experience generated at each time step is The experience replay pool is stored in the form of , and update the current state set, let ; Assign priority to each experience according to the time series error TD-error, select the most important experience for the Actor and Critic network update for training, and the TD-error of each experience is calculated by the current Critic network Values ​​and Goals The difference between the values ​​is calculated as follows:

[0103] ;

[0104] in, ;

[0105] Where: represents the temporal error TD-error of the jth experience; represents the target obtained by unloading action based on adding Gaussian noise value; and is the next state set of the target network calculation value, indicating future returns; min means taking the minimum value to avoid Overestimation of value; is a discount factor used to weigh the current immediate reward and future earnings, which indicates the weight of future earnings;

[0106] The sampling probability of each experience is calculated based on TD-error, as shown in the following formula:

[0107] ;

[0108] Where: represents the sampling probability of the jth experience; represents the hyperparameter of the priority experience replay, which controls the impact of TD-error on the sampling probability; k represents the experience replay pool The number of all experiences in ; Indicates the experience replay pool The absolute value of the timing error TD-error of all experiences is calculated The sum is calculated after the power operation;

[0109] After sampling B data from the experience replay pool according to priority, optimize the loss function of the Critic network through the following loss function Update the parameters of the Critic network and , so that the output of the Critic network Closer to the goal value :

[0110] ;

[0111] Where: Indicates averaging the batch size B to reduce the impact of a single piece of data on the update; Represents the importance sampling weight, which is used to correct the deviation caused by priority sampling;

[0112] If the current training step is an integer multiple of the delayed update step size Z, the following policy gradient method is used to update the parameters of the Actor network: , select the optimal offloading action to maximize the Critic network calculation value:

[0113] ;

[0114] Where: Represents the policy gradient of the Actor network; express expectations; Indicates that the experience replay pool Medium sampling; represents the gradient of the offloading action set; Represents the derivative of the Critic network to the set of unloading actions, which is used to guide the Actor network to learn better unloading actions; Represents the parameters of the Actor network The gradient of Represents the uninstall action set about parameters The gradient representation of

[0115] After each round of training, the parameters of the target network are updated based on the following soft update formula: and , so that it gradually approaches the parameters of the current Actor network and Critic network , and , to achieve smooth adjustment of target network parameters;

[0116] ;

[0117] When all training rounds are completed, a trained deep learning model is obtained.

[0118] In a second aspect, the present invention provides an electronic terminal, comprising a processor and a memory connected to the processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the above-mentioned computational offloading method for the 6G computing power network are performed.

[0119] Compared with the prior art, the present invention has the following beneficial effects:

[0120] The present invention achieves efficient computing offloading in the 6G computing network by specifically analyzing the characteristics of user computing tasks, modeling communication models, delay models, and energy consumption models. Especially in remote areas where ground base stations cannot cover, users can complete user computing tasks through the collaborative work of low-Earth orbit satellites and remote cloud data centers, thereby significantly improving the completion efficiency of user computing tasks, the flexibility of the computing network, and the utilization rate of computing resources. At the same time, the present invention takes the user service quality perceived cost as the optimization goal, comprehensively considers key indicators such as the remaining power of the user's local device, and can effectively balance the delay and energy consumption in the process of user computing task offloading, greatly improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0121] Figure 1 It is a flowchart of a calculation offloading method for a 6G computing power network provided in Embodiment 1 of the present invention;

[0122] Figure 2 This is a computing power network model diagram provided in the first embodiment of the present invention;

[0123] Figure 3 This is a satellite coverage model diagram provided by Embodiment 1 of the present invention;

[0124] Figure 4 It is an algorithm block diagram of the reinforcement learning algorithm PER-MATD3 that combines the priority experience replay mechanism and the multi-agent double-delay deep deterministic policy gradient provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0125] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0126] Embodiment 1:

[0127] Figure 1This is a flowchart of a method for offloading computations in a 6G computing network in Embodiment 1 of the present invention. This flowchart only shows the logical sequence of the method described in this embodiment. In other possible embodiments of the present invention, different methods may be used without conflict. Figure 1 The steps shown or described are performed in the order shown. Figure 1 The method of this implementation specifically includes the following steps:

[0128] A computing network model is constructed based on user local devices, low-Earth orbit satellites and remote cloud data centers, and the communication method between user local devices, remote cloud data centers and low-Earth orbit satellites is defined according to the constructed computing network model;

[0129] According to the user's offloading decision, determine the offloading ratio of the user's computing tasks to the user's local device, low-Earth orbit satellite and remote cloud data center for computing;

[0130] Establish a communication model between user local devices, remote cloud data centers and low-Earth orbit satellites, and calculate the transmission rates of different communication paths based on the Shannon formula;

[0131] According to the characteristics of the user's computing tasks and the transmission rates of different communication paths, the time delay and energy consumption of the user's local device, low-Earth orbit satellite and remote cloud data center for processing the user's computing tasks are calculated in combination with the communication mode;

[0132] Based on the unloading ratio, delay and energy consumption, a delay model and an energy consumption model are established;

[0133] Based on the remaining power status of the user's local device, weights are assigned to the latency model and energy consumption model with the goal of reducing latency or reducing energy consumption;

[0134] A global optimization objective function is established with the goal of minimizing the user service quality perception cost in the process of interaction between the user and the computing network environment;

[0135] The optimal solution of the global optimization objective function is solved to generate the optimal uninstallation action for each user.

[0136] Specifically, the computing power network model is as follows: Figure 2 As shown, it includes N user local devices, a satellite constellation network consisting of M low-Earth orbit satellites, and a remote cloud data center;

[0137] The communication method between the user local device and the low-earth orbit satellite, and between the low-earth orbit satellite and the remote cloud data center includes:

[0138] In the time interval T, there is a constant Low Earth Orbit (LEO) satellites can communicate with user local devices, and the user local devices can directly utilize the computing power resources of the LEO satellites within their communication coverage. The LEO satellites can process users' computing tasks as computing nodes or act as relay nodes to forward users' computing tasks to a remote cloud data center. Among them, N = {1, 2, …, n}, , <M. The satellite coverage model is as Figure 3 shown. In the figure, R e represents the radius of the Earth; H represents the orbital altitude of the LEO satellite; d represents the distance between the LEO satellite and the user local device; represents the elevation angle between the user local device and the LEO satellite. When > 0, the user local device can communicate with the LEO satellite; represents the central angle of the Earth corresponding to the coverage area of the LEO satellite. According to the relevant parameters provided by the satellite coverage model, the calculation time interval T is calculated as follows:

[0139] ;

[0140] ;

[0141] ;

[0142] In the formula: L represents the orbital arc length passed by the LEO satellite during the coverage time; represents the running speed of the LEO satellite. As Figure 2 shown, some users are located in remote areas such as deserts, oceans, forests or mountains, etc. These areas usually lack ground base station coverage and cannot communicate directly with the remote cloud data center. In this case, users need to rely on communicable LEO satellites for relay to offload users' computing tasks to the remote cloud data center for calculation. The LEO satellites act as bridges. Through efficient communication links, even in an environment with imperfect ground infrastructure, users can still access powerful remote computing resources to complete users' computing tasks.

[0143] Each user can choose to offload the user computing task to the user local device, the LEO satellite or the remote cloud data center for calculation. The offloading decision of the user is defined as:

[0144] ;

[0145] In the formula: represents the proportion of the user computing task offloaded to the user local device for calculation; represents the proportion of the user computing task offloaded to the LEO satellite for calculation; Indicates the proportion of user computing tasks offloaded to remote cloud data centers for computing; , and Subject to the following relationship:

[0146] { a n ( t ) + b n ( t ) + c n ( t ) = 1 a n ( t ), b n ( t ), c n ( t ) ∈ [ 0 , 1 ] .

[0147] A single user can determine the offloading ratio of his / her user computing tasks according to his / her needs. For example, if the user computing tasks are completely offloaded to the user's local device for computing, the offloading ratio of the user computing tasks is 100% allocated to the user's local device. The value is 100%. and The value is 0; if the user computing task is split into multiple parts, they can be offloaded to the user's local device, low-Earth orbit satellite, and remote cloud data center in proportion. For example, 30% of a single user computing task is allocated to the user's local device, 40% to the low-Earth orbit satellite, and 30% to the remote cloud data center. , and The values ​​of are: 30%, 40% and 30% respectively. This flexible offloading method supports multi-mode collaborative computing of user computing tasks between user local devices, low-Earth orbit satellites and remote data centers, thereby improving the overall utilization of computing resources.

[0148] According to Shannon's formula, the data transmission rate of ground-to-air communication between the user's local device and the low-Earth orbit satellite is calculated respectively. and the data transmission rate between the mth low-Earth orbit satellite and the remote cloud data center , as shown below:

[0149] ;

[0150] Where: represents the data transmission rate of the ground-to-air communication between the user local equipment of the nth user and the mth low-Earth orbit satellite; It represents the transmission bandwidth between the user's local device and the low-Earth orbit satellite; represents the number of users who choose to offload their computing tasks to the mth low-Earth orbit satellite. Users offloaded to the same low-Earth orbit satellite share bandwidth resources in an equal manner; represents the transmission rate of the user local device of the nth user; represents the channel gain between the user local device of the nth user and the mth low-Earth orbit satellite at time t; Represents the channel noise power.

[0151] The communication model between the low-Earth orbit satellite and the remote cloud data center is defined as:

[0152] ;

[0153] Where: represents the data transmission rate between the mth low-Earth orbit satellite and the remote cloud data center; represents the congestion factor, ; Represents the transmission bandwidth between low-Earth orbit satellites and remote cloud data centers; represents the number of users who choose to offload their computing tasks to remote cloud data centers; represents the transmission rate of the mth low-Earth orbit satellite; represents the channel gain between the mth low-Earth orbit satellite and the remote cloud data center at time t.

[0154] Get the transmission rate of different communication paths and Then, we analyze the characteristics of user computing tasks, which include data volume, CPU and GPU computing volume, maximum latency and energy consumption limit acceptable to users, and the characteristics of user computing tasks. Defined as:

[0155] ;

[0156] Where: Indicates the data size of the user's computing task, in bytes; Indicate completion The amount of computation required by the CPU to calculate the data volume, in units of CPU cycles; Indicate completion The amount of computation required by the GPU in the data volume, in units of GPU floating-point operations; Indicates the maximum delay time tolerated by the user; Indicates the maximum energy consumption tolerated by the user.

[0157] According to the characteristics of user computing tasks, combined with the communication method between the user's local device and the low-Earth orbit satellite, and the communication method between the low-Earth orbit satellite and the remote cloud data center, the latency and energy consumption of the user's local device, low-Earth orbit satellite and remote cloud data center in processing user computing tasks are calculated respectively.

[0158] When the user computing task is offloaded to the user's local device, the data does not need to be transmitted additionally, so the data transmission delay is not considered, and only the calculation delay of the user computing task is considered. The calculation formula of the delay is as follows:

[0159] ;

[0160] Where: It represents the delay required to complete the user computing task when the user computing task is offloaded to the local device of the nth user at time t; Indicates the CPU computing capability of the local device of the nth user; Indicates the GPU computing capability of the local device of the nth user;

[0161] The energy consumption calculation formula is as follows:

[0162] ;

[0163] Where: It represents the energy consumption required to complete the user computing task when the user computing task is offloaded to the local device of the nth user at time t; represents the capacitance parameter of the user local device of the nth user; represents the dynamic power consumption of the GPU of the user local device of the nth user;

[0164] When the user computing task is offloaded to the low-Earth orbit satellite, it is necessary to comprehensively consider the round-trip propagation delay from the earth to the low-Earth orbit satellite, the data transmission delay, and the calculation delay of the user computing task. Since the calculation result of the user computing task is much smaller than the input data, the data transmission delay only considers the delay from the user's local device of the nth user to the mth low-Earth orbit satellite, while ignoring the delay from the mth low-Earth orbit satellite to return the calculation result to the user's local device. The calculation formula for the delay is shown as follows:

[0165] ;

[0166] Where: It represents the delay required to complete the user computing task when the user computing task is offloaded to the low-Earth orbit satellite computing at time t; represents the data transmission delay from the local user device of the nth user to the mth low-Earth orbit satellite; Indicates the computational latency of the user's computational task; It represents the CPU computing capacity of the mth low-Earth orbit satellite allocated to the user computing task; represents the GPU computing power allocated by the mth low-Earth orbit satellite to the user's computing task; d represents the distance between the low-Earth orbit satellite and the user's local device;

[0167] ;

[0168] represents the speed of light; represents the round-trip propagation delay from the Earth to the LEO satellite;

[0169] The energy consumption calculation formula is as follows:

[0170] ;

[0171] Where: It represents the energy consumption required to complete the user computing task when the user computing task is offloaded to the mth low-Earth orbit satellite at time t; represents the transmission energy consumption of the user local device of the nth user; represents the capacitance parameter of the mth low-Earth orbit satellite; represents the GPU dynamic power consumption of the mth low-Earth orbit satellite;

[0172] When the user's computing task is offloaded to the remote cloud data center, due to the lack of ground base stations in the user's area, it is impossible to communicate directly with the remote cloud data center. The user needs to relay through a communicative low-earth orbit satellite to offload the user's computing task to the remote cloud data center. In this process, it is necessary to consider the data transmission delay of the user's computing task transmitted to the remote cloud data center via the low-earth orbit satellite relay. Since the remote cloud data center has abundant computing resources, the computing delay of the user's computing task can be ignored. The calculation formula for the delay is shown as follows:

[0173] ;

[0174] Where: It represents the delay required to complete the user computing task when the user computing task is offloaded to the remote cloud data center for computing at time t; represents the data transmission delay from the mth low-Earth orbit satellite to the remote cloud data center; represents the round-trip propagation delay from the Earth via a low-Earth orbit satellite to a remote cloud data center;

[0175] The energy consumption calculation formula is as follows:

[0176] ;

[0177] Where: It represents the energy consumption required to complete the user computing task when the user computing task is offloaded to the remote cloud data center at time t; represents the transmission energy consumption of the mth low-Earth orbit satellite.

[0178] Combine the unloading ratio, delay and energy consumption to establish a delay model and energy consumption model , as shown below:

[0179] ;

[0180] ;

[0181] Where: represents the total delay required to complete the user computing tasks of N users; Represents the total energy consumption required to complete the user computing tasks of N users.

[0182] Based on the remaining power status of the user's local device, weights are assigned to the delay model and energy consumption model with the goal of reducing latency or reducing energy consumption; and the user's service quality perception cost at time t is defined. As shown below:

[0183] ;

[0184] Where: Representation delay model The weight of Energy consumption model The weight of ω L , ω E ∈[0, 1] ,and ; According to the remaining power of different users' local devices, and The value of is set as follows:

[0185] ;

[0186] Where: Indicates the current battery power percentage of the user's local device.

[0187] Through this weight distribution, when the battery level of the user's local device is sufficient, the offload strategy will tend to reduce the computing delay to provide a faster response; when the battery level is low, it will prioritize reducing energy consumption to extend the usage time of the user's local device. This flexible strategy can ensure that the user's experience needs are always met according to different battery levels.

[0188] The perceived service quality cost during the interaction between the user and the computing network environment is the cumulative value of the perceived service quality cost of the user at multiple moments. The perceived service quality cost is obtained by summarizing the perceived service quality cost of the user at each moment during the interaction between the user and the environment. With the goal of minimizing the perceived service quality cost of the user, a global optimization objective function is established, and the optimal solution of the global optimization objective function is solved. The deep learning model is used to generate the optimal unloading action for each user, that is, the optimal unloading strategy. The deep learning model combines the priority experience replay mechanism and the multi-agent twin delayed deep deterministic policy gradient reinforcement learning algorithm PER-MATD3 (Prioritized Experience Replay - Multi-Agent Twin Delayed Deep Deterministic Policy Gradient).

[0189] By accumulating the user service quality perception cost at each moment during the interaction between the user and the environment, the global optimization objective function is obtained as shown in the following formula:

[0190] ;

[0191] Where: Indicates the termination time of the user's interaction with the computing network environment; express The user's perceived cost of service quality at each moment.

[0192] In the multi-user computing task offloading scenario, if the optimization goal is established only by combining the latency and energy consumption of a single user, the agents in the multi-agent dual-delay deep deterministic policy gradient reinforcement learning algorithm may tend to choose the optimal offloading strategy for a single user, such as centrally offloading user computing tasks to a remote cloud data center. Although the remote cloud data center has strong computing power and the latency and energy consumption of a single user may be reduced, as user computing tasks are centrally offloaded, the resources of the remote cloud data center are gradually overloaded, resulting in a decline in the overall performance of the computing network, and the offloading effect of user computing tasks for other users will also deteriorate. This locally optimal strategy not only reduces the utilization efficiency of computing network resources, but may also increase the perceived service quality cost of some users.

[0193] To this end, the present invention calculates the total delay of all users and total energy consumption , combined with the total latency and total energy consumption of all users, the optimization goal of the multi-agent deep reinforcement learning algorithm is defined to guide each agent to collaboratively explore a reasonable offloading strategy. Through global optimization, the agent can balance the resource allocation of user local devices, low-Earth orbit satellites, and remote cloud data centers, so that the overall user service quality perception cost of the computing network is optimized, avoiding the problem of uneven resource utilization and improving the overall computing network performance.

[0194] The solution process of the global optimization objective function is converted into a Markov decision process, and the state set, action set and reward function are defined. The optimization goal is linked to the user's unloading action through the reward function. By converting the solution process of the global optimization objective function into a Markov decision process, the present invention can dynamically adapt to changes in user computing tasks and communication conditions, realize real-time adjustment of unloading strategies, and ensure excellent performance in a changing computing power network environment.

[0195] First, a state set is defined to describe all possible states of each user at time t; the state includes the user's user computing task information, offloading decision, and resource allocation and offloading status of low-Earth orbit satellites. The state set at time t As shown below:

[0196] ;

[0197] in,

[0198] ;

[0199] Where: Represents the user computing task information of each user at time t; Indicates the user computing task information of the nth user at time t; represents the current uninstall decision of each user at time t; represents the uninstall decision of the nth user at time t; Indicates the resource allocation of each low-Earth orbit satellite at time t; Indicates the resource allocation of the mth low-Earth orbit satellite at time t; Indicates the load conditions of each low-Earth orbit satellite at time t; Indicates the load of the mth low-Earth orbit satellite at time t;

[0200] in,

[0201] { I n ( t ) = { Task n , ε n } Ω m ( t ) = { A CPU m ( t ), A GPU m ( t ), F max LEO , TP max LEO , num n } A CPU m ( t ) = [ f 1 m , LEO , f 2 m , LEO , … , f num n m , LEO ] A GPU m ( t ) = [ TP 1 m , LEO , TP 2 m , LEO , … , TP num n m , LEO ] Ψ m ( t ) = { U CPU m ( t ), U GPU m ( t ), N queue m ( t )} ;

[0202] Where: Indicates the current battery power percentage of the local device of the nth user; and They represent the CPU computing resources and GPU computing resources allocated to the user by the mth low-Earth orbit satellite; represents the number of users unloaded to the mth low-Earth orbit satellite; Indicates that the mth low-Earth orbit satellite is assigned to The CPU computing power of the user computing tasks of each user; Indicates that the mth low-Earth orbit satellite is assigned to GPU computing power for user computing tasks of each user; Indicates the current CPU usage of the mth low-Earth orbit satellite; Indicates the current GPU usage rate of the mth low-Earth orbit satellite; represents the number of user computing tasks currently queued in the mth low-Earth orbit satellite;

[0203] Next, we define an uninstall action set, which is used to describe the uninstall actions taken by each user at time t in the state described above. As shown below:

[0204] ;

[0205] in,

[0206] ;

[0207] Where: Indicates the uninstall action of the nth user at time t; represents a target low-Earth orbit satellite for unloading;

[0208] The unloading actions taken by each user, the resource allocation of each low-Earth orbit satellite, and the load of each low-Earth orbit satellite are subject to the following conditional constraints:

[0209] ;

[0210] ;

[0211] ;

[0212] ;

[0213] ;

[0214] ;

[0215] Where: Indicates the delay required for a single user to complete a user computing task; It represents the energy consumption required by a single user to complete the user computing task; Indicates the maximum CPU computing power of a low-Earth orbit satellite; Indicates the maximum GPU computing power of low-Earth orbit satellites; the conditional expression and Ensure that the latency and energy consumption of the unloading action selected by the user is less than or equal to the maximum latency tolerated by the user and maximum energy consumption ;Conditional expression and Ensure the CPU computing power of low-Earth orbit satellites allocated to user computing tasks and GPU computing power Less than or equal to the maximum CPU computing power of low-Earth orbit satellites and maximum GPU computing power ;Conditional expression and Ensure the sum of CPU computing power allocated to each user by low-Earth orbit satellites and GPU computing power Less than or equal to the sum of the maximum CPU computing capabilities of low-Earth orbit satellites and the sum of the maximum GPU computing power ;

[0216] The reward function is defined by combining the state set and the uninstall action set to describe the immediate reward obtained by each user when taking the uninstall action in the state. The reward function is defined as:

[0217] ;

[0218] Where: Represents the penalty factor. When the user's uninstall action leads to any of the following situations, <0: The user's computing task exceeds the maximum delay time tolerated by the user, exceeds the maximum energy consumption limit, or fails to successfully match the target low-Earth orbit satellite; otherwise, >0.

[0219] The deep learning model is trained using the state set, the unloading action set and the reward function, including:

[0220] A multi-agent double-delay deep deterministic policy gradient reinforcement learning algorithm is adopted. Data is sampled from the experience replay pool according to priority through the priority experience replay mechanism. The Q-value evaluation accuracy is optimized based on the dual Critic network. The target Q-value is used to guide the policy update of the Actor network. The target network parameters are synchronized with the soft update mechanism, so that the reward function gradually increases, thereby reducing the user's perceived service quality cost and generating the optimal offloading action for each user.

[0221] like Figure 4As shown in the figure, it is a block diagram of the reinforcement learning algorithm PER-MATD3, which combines the priority experience replay mechanism and the multi-agent double-delay deep deterministic policy gradient. In the figure, the agent is the decision maker in the reinforcement learning system, responsible for observing the state of the environment, selecting the unloading action, and adjusting its unloading strategy according to the environmental feedback. In this embodiment, the agent is the user, and the agent simulates the unloading decision of each user to decide whether to unload the user's computing task to the user's local device, low-Earth orbit satellite or remote cloud data center. Specifically, first initialize the state set , Uninstall action set , Experience Replay Pool , batch size , discount factor , soft update coefficient ;

[0222] Initializing the Actor Network , so that it is consistent with the initial state set The action set of each user Guaranteed consistency; among them, the initial state set In this case, each user only chooses to offload the user computing task to the user's local device for computing, and the bandwidth resources, CPU and GPU computing resources of the low-Earth orbit satellite and remote cloud data center are not allocated;

[0223] Initialize two Critic networks and And the corresponding target network, the target network includes the target Actor network And two target critic networks ;

[0224] Among them, Actor network Used to collect data based on the current state Generate uninstall action set , ; and is the Q value calculated by the two Critic networks, indicating that in the state set Take the following uninstall action set the long-term rewards that can be obtained; Represents the target Actor network generated unloading action; and Respectively represent the Q values ​​calculated by the two target Critic networks;

[0225] Initialize the number of training rounds episode, the number of training steps per round step, and the delay update step Z, where the number of training steps step is the total number of time steps executed in one round of training;

[0226] At the beginning of each round of training, reset the state set is the initial state set of the environment ,make ;

[0227] At each time step, each user first follows the strategy of the current Actor network. Select the initial set of uninstall actions , the initial set of uninstall actions is given by the following formula Add Gaussian noise to get the actual set of unloading actions :

[0228] ;

[0229] Where: represents a mean of 0 and a variance of Gaussian noise;

[0230] Then the actual uninstall action set Acts on the computing power network environment and records the current state set , Uninstall action set , Instant Rewards , the next state set and Boolean variables The value of, if the current state set is the terminal state set , The value is false; otherwise, The value is true, indicating that there is another state set;

[0231] The experience generated at each time step is The experience replay pool is stored in the form of , and update the current state set, let ; Assign priority to each experience according to the time series error TD-error, select the most important experience for the Actor and Critic network update for training, and the TD-error of each experience is calculated by the current Critic network Values ​​and Goals The difference between the values ​​is calculated as follows:

[0232] ;

[0233] in, ;

[0234] Where: represents the temporal error TD-error of the jth experience; represents the target obtained by unloading action based on adding Gaussian noise value; and is the next state set of the target network calculation value, indicating future returns; min means taking the minimum value to avoid Overestimation of value; is a discount factor used to weigh the current immediate reward and future earnings, which indicates the weight of future earnings;

[0235] The sampling probability of each experience is calculated according to TD-error, as shown in the following formula:

[0236] ;

[0237] Where: represents the sampling probability of the jth experience; represents the hyperparameter of the priority experience replay, which controls the impact of TD-error on the sampling probability; k represents the experience replay pool The number of all experiences in ; Indicates the experience replay pool The absolute value of the time series error TD-error of all experiences is calculated. Perform the power operation and then sum it up;

[0238] After sampling B data from the experience replay pool according to priority, optimize the loss function of the Critic network through the following loss function Update the parameters of the Critic network and , so that the output of the Critic network Closer to the goal value :

[0239] ;

[0240] Where: Indicates averaging the batch size B to reduce the impact of a single piece of data on the update; Represents the importance sampling weight, which is used to correct the deviation caused by priority sampling;

[0241] ;

[0242] Where: Represents the policy gradient of the Actor network; express expectations; Indicates that the experience replay pool Medium sampling; represents the gradient of the offloading action set; Represents the derivative of the Critic network to the set of unloading actions, which is used to guide the Actor network to learn better unloading actions; Represents the parameters of the Actor network The gradient of Represents the uninstall action set about parameters The gradient representation of

[0243] After each round of training, the parameters of the target network are updated based on the following soft update formula: and , so that it gradually approaches the parameters of the current Actor network and Critic network , and , to achieve smooth adjustment of target network parameters;

[0244] ;

[0245] When all training rounds are completed, a trained deep learning model is obtained. The unloading actions generated by the Actor network can make the Critic network Maximize the value, thereby gradually increasing the reward function , reducing user service quality perception costs , to minimize the perceived cost of service quality during the interaction between users and the computing network environment , and output the optimal uninstall action for each user.

[0246] The embodiments of the present invention analyze the characteristics of user computing tasks, efficiently model communication as well as latency and energy consumption models, and achieve optimized computing offloading in the 6G computing network, significantly improving the efficiency of user computing task completion, user experience, and system energy efficiency. It also achieves dynamic adaptability and global optimal strategy through multi-mode collaborative computing and multi-agent deep reinforcement learning algorithms.

[0247] Embodiment 2:

[0248] An embodiment of the present invention also provides an electronic terminal, comprising a processor and a memory connected to the processor, wherein a computer program is stored in the memory. When the computer program is executed by the processor, the steps of the computational offloading method for the 6G computing power network described in the above-mentioned embodiment 1 are performed.

[0249] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0250] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0251] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0252] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0253] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A computation offloading method for 6G computing network, characterized in that: include: Construct a computing network model based on user local devices, low-Earth orbit satellites and remote cloud data centers, and define the communication method between user local devices, remote cloud data centers and low-Earth orbit satellites according to the constructed computing network model; According to the user's offloading decision, determine the offloading ratio of the user's computing tasks to the user's local device, low-Earth orbit satellite and remote cloud data center for computing; Establish a communication model between user local devices, remote cloud data centers and low-Earth orbit satellites, and calculate the transmission rates of different communication paths based on the Shannon formula; According to the characteristics of the user's computing tasks and the transmission rates of different communication paths, the time delay and energy consumption of the user's local device, low-Earth orbit satellite and remote cloud data center for processing the user's computing tasks are calculated in combination with the communication mode; Based on the unloading ratio, delay and energy consumption, a delay model and an energy consumption model are established; Based on the remaining power status of the user's local device, weights are assigned to the latency model and energy consumption model with the goal of reducing latency or reducing energy consumption; A global optimization objective function is established with the goal of minimizing the user service quality perception cost in the process of interaction between the user and the computing network environment; The optimal solution of the global optimization objective function is solved to generate the optimal uninstallation action for each user.

2. The method for offloading calculations for a 6G computing network according to claim 1, characterized in that: The computing network model includes N user local devices, a satellite cluster network consisting of M low-Earth orbit satellites, and a remote cloud data center; The communication method between the user local device, the remote cloud data center and the low-earth orbit satellite includes: In the time interval T, there is a constant Low-Earth orbit satellites can communicate with user local devices, and user local devices directly use the computing resources of low-Earth orbit satellites within their communication coverage; The low-Earth orbit satellite acts as a computing node to process user computing tasks, or acts as a relay node to forward user computing tasks to a remote cloud data center; where N and are both integers greater than or equal to 1, <For M, the calculation formula for the time interval T is as follows: ； ； ； Where: L represents the length of the orbital arc that the low-Earth orbit satellite passes during the coverage time; Represents the speed of low-Earth orbit satellites; R e represents the radius of the earth; H represents the orbital altitude of the low-Earth orbit satellite; represents the geocentric angle corresponding to the area covered by the low-Earth orbit satellite; Represents the elevation angle between the user's local device and the low-Earth orbit satellite.

3. The method for offloading calculations for a 6G computing network according to claim 2, characterized in that: The user's uninstall decision Defined as: ； Where: Indicates the proportion of user computing tasks offloaded to the user's local device for computing; It indicates the proportion of user computing tasks offloaded to low-Earth orbit satellites for computing; Indicates the proportion of user computing tasks offloaded to remote cloud data centers for computing; , and Subject to the following relationship: 。 4. The method for offloading calculations for a 6G computing network according to claim 3, characterized in that: The communication model between the user's local device and the low-Earth orbit satellite is defined as: ； Where: represents the data transmission rate of the ground-to-air communication between the user local equipment of the nth user and the mth low-Earth orbit satellite; It represents the transmission bandwidth between the user's local device and the low-Earth orbit satellite; represents the number of users who choose to offload their computing tasks to the mth low-Earth orbit satellite. Users offloaded to the same low-Earth orbit satellite share bandwidth resources in an equal manner; represents the transmission rate of the user local device of the nth user; represents the channel gain between the user local device of the nth user and the mth low-Earth orbit satellite at time t; represents the channel noise power; The communication model between the low-Earth orbit satellite and the remote cloud data center is defined as: ； Where: represents the data transmission rate between the mth low-Earth orbit satellite and the remote cloud data center; represents the congestion factor; Represents the transmission bandwidth between low-Earth orbit satellites and remote cloud data centers; represents the number of users who choose to offload their computing tasks to remote cloud data centers; represents the transmission rate of the mth low-Earth orbit satellite; represents the channel gain between the mth low-Earth orbit satellite and the remote cloud data center at time t.

5. The method for offloading calculations for a 6G computing network according to claim 4, characterized in that: Characteristics of the user computing task Defined as: ； Where: Indicates the data size of the user's computing task, in bytes; Indicate completion The amount of computation required by the CPU to calculate the data volume, in units of CPU cycles; Indicate completion The amount of computation required by the GPU in the data volume, in units of GPU floating-point operations; Indicates the maximum delay time tolerated by the user; Indicates the maximum energy consumption tolerated by the user; The method of calculating the time delay and energy consumption of the user's local device, the low-earth orbit satellite and the remote cloud data center for processing the user's computing task in combination with the communication mode according to the characteristics of the user's computing task and the transmission rates of different communication paths includes: When the user computing task is offloaded to the user's local device, only the computing delay of the user computing task is considered. The calculation formula of the delay is as follows: ； Where: It represents the delay required to complete the user computing task when the user computing task is offloaded to the local device of the nth user at time t; Indicates the CPU computing capability of the local device of the nth user; Indicates the GPU computing capability of the local device of the nth user; The energy consumption calculation formula is as follows: ； Where: It represents the energy consumption required to complete the user computing task when the user computing task is offloaded to the local device of the nth user at time t; represents the capacitance parameter of the user local device of the nth user; represents the dynamic power consumption of the GPU of the user local device of the nth user; When the user computing task is offloaded to a low-Earth orbit satellite, it is necessary to consider the satellite-to-ground link propagation delay, data transmission delay, and the calculation delay of the user computing task. The delay calculation formula is as follows: ； Where: It represents the delay required to complete the user computing task when the user computing task is offloaded to the low-Earth orbit satellite computing at time t; represents the data transmission delay from the local user device of the nth user to the mth low-Earth orbit satellite; Indicates the computational latency of the user's computational task; It represents the CPU computing capacity of the mth low-Earth orbit satellite allocated to the user computing task; represents the GPU computing power allocated to the user computing task by the mth low-Earth orbit satellite; d represents the distance between the low-Earth orbit satellite and the user's local device, , represents the speed of light; represents the round-trip propagation delay from the Earth to the LEO satellite; The energy consumption calculation formula is as follows: ； Where: It represents the energy consumption required to complete the user computing task when the user computing task is offloaded to the mth low-Earth orbit satellite at time t; represents the transmission energy consumption of the user local device of the nth user; represents the capacitance parameter of the mth low-Earth orbit satellite; represents the GPU dynamic power consumption of the mth low-Earth orbit satellite; When the user's computing tasks are offloaded to a remote cloud data center, the data transmission delay of the data being sent to the remote cloud data center via the low-Earth orbit satellite relay needs to be considered. The delay calculation formula is as follows: ； Where: It represents the delay required to complete the user computing task when the user computing task is offloaded to the remote cloud data center for computing at time t; represents the data transmission delay from the mth low-Earth orbit satellite to the remote cloud data center; represents the round-trip propagation delay from the Earth via a low-Earth orbit satellite to a remote cloud data center; The energy consumption calculation formula is as follows: ； Where: It represents the energy consumption required to complete the user computing task when the user computing task is offloaded to the remote cloud data center at time t; represents the transmission energy consumption of the mth low-Earth orbit satellite; combining the unloading ratio, delay and energy consumption, establish a delay model and energy consumption model , as shown below: ； 。 6. The method for offloading calculations for a 6G computing network according to claim 5, characterized in that: User service quality perceived cost As shown below: ； Where: Representation delay model The weight of Energy consumption model The weight of ,and ; According to the remaining power of different users' local devices, and The value of is set as follows: ； Where: Indicates the current battery power percentage of the user's local device.

7. The method for offloading calculations for a 6G computing network according to claim 6, characterized in that: With the goal of minimizing the user service quality perception cost in the process of interaction between the user and the computing network environment, a global optimization objective function is established, including: By accumulating the user service quality perception cost at each moment during the interaction between the user and the computing power network environment, the global optimization objective function is obtained as shown in the following formula: ； Where: Indicates the termination time of the user's interaction with the computing network environment; express The user's perceived cost of service quality at each moment.

8. The method for offloading calculations for a 6G computing network according to claim 7, characterized in that: Solving the optimal solution of the global optimization objective function and generating the optimal uninstall action for each user includes: Define a state set to describe all possible states of each user at time t; the state includes the user's user computing task information, offloading decision, and resource allocation and offloading of low-Earth orbit satellites. The state set at time t As shown below: ； in, ； Where: Represents the user computing task information of each user at time t; Indicates the user computing task information of the nth user at time t; represents the current uninstall decision of each user at time t; represents the uninstall decision of the nth user at time t; Indicates the resource allocation of each low-Earth orbit satellite at time t; Indicates the resource allocation of the mth low-Earth orbit satellite at time t; Indicates the load conditions of each low-Earth orbit satellite at time t; Indicates the load of the mth low-Earth orbit satellite at time t; in, ； Where: Indicates the current battery power percentage of the local device of the nth user; and They represent the CPU computing resources and GPU computing resources allocated to the user by the mth low-Earth orbit satellite; represents the number of users unloaded to the mth low-Earth orbit satellite; Indicates that the mth low-Earth orbit satellite is assigned to The CPU computing power of the user computing tasks of each user; Indicates that the mth low-Earth orbit satellite is assigned to GPU computing power for user computing tasks of each user; Indicates the current CPU usage of the mth low-Earth orbit satellite; Indicates the current GPU usage rate of the mth low-Earth orbit satellite; represents the number of user computing tasks currently queued in the mth low-Earth orbit satellite; Define an uninstall action set to describe the uninstall actions taken by each user at time t in the state described above. As shown below: ； in, ； Where: Indicates the uninstall action of the nth user at time t; represents a target low-Earth orbit satellite for unloading; The unloading actions taken by each user, the resource allocation of each low-Earth orbit satellite, and the load of each low-Earth orbit satellite are subject to the following conditional constraints: ； ； ； ； ； ； Where: Indicates the delay required for a single user to complete a user computing task; It represents the energy consumption required by a single user to complete the user computing task; Indicates the maximum CPU computing power of a low-Earth orbit satellite; Indicates the maximum GPU computing power of low-Earth orbit satellites; The reward function is defined by combining the state set and the uninstall action set to describe the immediate reward obtained by each user when taking the uninstall action in the state , the reward function is defined as: ； Where: represents the penalty factor; The deep learning model is trained using the state set, the uninstallation action set and the reward function to generate the optimal uninstallation action for each user.

9. The method for offloading calculations for a 6G computing network according to claim 8, characterized in that: The training of the deep learning model using the state set, the unloading action set and the reward function includes: Initialize the state collection , Uninstall action set , Experience Replay Pool , batch size , discount factor , soft update coefficient ; Initializing the Actor Network , so that it is consistent with the initial state set The uninstall actions of each user are guaranteed to be consistent; among them, the initial state set In this case, each user only chooses to offload the user computing task to the user's local device for computing, and the bandwidth resources, CPU and GPU computing resources of the low-Earth orbit satellite and remote cloud data center are not allocated; Initialize two Critic networks and And the corresponding target network, the target network includes the target Actor network And two target critic networks ; Among them, Actor network Used to collect data based on the current state Generate uninstall action set , ; and is the Q value calculated by the two Critic networks, indicating that in the state set Take the following uninstall action set the long-term rewards that can be obtained; Represents the set of unloading actions generated by the target Actor network; and Respectively represent the Q values ​​calculated by the two target Critic networks; Initialize the number of training rounds, the number of training steps per round, and the delay update step size Z, where the number of training steps is the total number of time steps executed in one round of training; At the beginning of each round of training, reset the state set is the initial state set of the environment ,make ; At each time step, each user first follows the strategy of the current Actor network. Select the initial set of uninstall actions , the initial set of uninstall actions is given by the following formula Add Gaussian noise to get the actual set of unloading actions : ； Where: represents a mean of 0 and a variance of Gaussian noise; Then the actual uninstall action set Acts on the computing power network environment and records the current state set , Uninstall action set , Instant Rewards , the next state set and Boolean variables The value of, if the current state set is the terminal state set , The value is false; otherwise, The value is true, indicating that there is another state set; The experience generated at each time step is The experience replay pool is stored in the form of , and update the current state set, let ; Assign priority to each experience according to the time series error TD-error, select the most important experience for the Actor and Critic network update for training, and the TD-error of each experience is calculated by the current Critic network Values ​​and Goals The difference between the values ​​is calculated as follows: ； in, ; Where: represents the temporal error TD-error of the jth experience; represents the target obtained by unloading action based on adding Gaussian noise value; and is the next state set of the target network calculation value, indicating future returns; min means taking the minimum value to avoid Overestimation of value; is a discount factor used to weigh the current immediate reward and future earnings, which indicates the weight of future earnings; The sampling probability of each experience is calculated according to TD-error, as shown in the following formula: ； Where: represents the sampling probability of the jth experience; represents the hyperparameter of the priority experience replay, which controls the impact of TD-error on the sampling probability; k represents the experience replay pool The number of all experiences in ; Indicates the experience replay pool The absolute value of the timing error TD-error of all experiences is calculated The sum is calculated after the power operation; After sampling B data from the experience replay pool according to priority, optimize the loss function of the Critic network through the following loss function Update the parameters of the Critic network and , so that the output of the Critic network Closer to the goal value : ； Where: Indicates averaging the batch size B to reduce the impact of a single piece of data on the update; Represents the importance sampling weight, which is used to correct the deviation caused by priority sampling; If the current training step is an integer multiple of the delayed update step size Z, the following policy gradient method is used to update the parameters of the Actor network: , select the optimal offloading action to maximize the Critic network calculation value: ； Where: Represents the policy gradient of the Actor network; express expectations; Indicates that the experience replay pool Medium sampling; represents the gradient of the offloading action set; Represents the derivative of the Critic network to the set of unloading actions, which is used to guide the Actor network to learn better unloading actions; Represents the parameters of the Actor network The gradient of Represents the uninstall action set about parameters The gradient representation of After each round of training, the parameters of the target network are updated based on the following soft update formula: and , so that it gradually approaches the parameters of the current Actor network and Critic network , and , to achieve smooth adjustment of target network parameters; ； When all training rounds are completed, a trained deep learning model is obtained.

10. An electronic terminal, characterized in that: It includes a processor and a memory connected to the processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the computational offloading method for a 6G computing power network as described in any one of claims 1 to 9 are performed.

Images