An edge-computing-based satellite multi-hop task offloading optimization method and system

Abstract

The application discloses a satellite multi-hop task unloading optimization method and system based on edge computing, and the method comprises the following steps: when the access satellite computing resource of a ground user is insufficient, a on-demand routing protocol is adopted, and under the constraint condition, based on environment exploration information, a task unloading optimization problem is obtained by taking the minimum energy consumption of a system model as a target; the task unloading optimization problem is constructed into a Markov decision process, and a deep reinforcement learning algorithm is combined to solve the problem, so that a task unloading and routing selection optimization strategy is obtained. In the application, in the case that the access satellite computing resource is insufficient, the on-demand routing protocol is combined to support efficient multi-hop task transmission unloading, so that the selection of an intermediate satellite node on a routing path from a source satellite node to a destination satellite node is optimized on the basis of the minimum number of hops, the energy consumption is comprehensively considered, the multi-hop is dynamically used to reach a processing node, and the control overhead of the whole network topology in the task unloading process is maximally reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of wireless communication, and in particular to a satellite multi-hop task offloading optimization method and system based on edge computing. BACKGROUND

[0002] In recent years, with the function of cloud computing moving to the edge of the network, computing is emerging a new trend. Collecting a large amount of idle computing power and storage space distributed at the edge of the network can generate sufficient capacity, which can be used to perform computing-intensive and delay-critical tasks on mobile devices, and this paradigm is called mobile edge computing (MEC). Unlike cloud computing, the computing tasks of terminal devices in MEC are offloaded to edge servers closer to the terminal devices for running, and the edge servers can provide computing, content caching and other functions. The servers distributed at the edge of the network (also known as computing nodes, edge nodes) can reduce the computing pressure of terminal devices and reduce the frequency of centralized data interaction with cloud computing, and can also significantly reduce the waiting time in message exchange. Since the edge server has a certain storage space and computing power and is closer to the terminal device, a mobile terminal device with computing-intensive or delay-sensitive can offload the computing task to the edge server for computation.

[0003] On the one hand, the computing offloading of the task request not only relieves the communication pressure of the core network, but also reduces the delay caused by long-distance data transmission. On the other hand, emerging applications in 5 / 6G also rely on computing offloading technology to provide efficient services for users. Therefore, computing offloading is one of the key technologies in MEC. Although MEC technology has been widely used in ground base stations to provide good services for mobile terminals, ground edge computing faces many problems. Ground edge computing cannot fully cover some complex terrains such as mountains and oceans, and therefore cannot meet the global demand for "ubiquitous connectivity". At the same time, the ground network infrastructure is vulnerable to natural disasters such as earthquakes and hurricanes, resulting in communication interruption of users.

[0004] With the development of space communication networks, satellite technology has made great progress in commercial, civilian and military services, and the miniaturization and economic development of low earth orbit satellites are changing rapidly. Compared with other communication methods, satellite networks have the advantages of large coverage area and are not limited by terrain. Therefore, by learning from the idea of ground network MEC, edge computing is introduced into satellite networks. Satellites deploying MEC servers need to be as close to users as possible, so they are usually located in low earth orbit (LEO), and space is considered as the edge. Users in the station directly obtain reliable computing services from the satellite instead of communicating with the ground base station. Satellites perform on-board processing tasks instead of just acting as a relay transponder, which will be an indispensable paradigm for future integrated STN.

[0005] Satellite-based edge computing (Sa-MEC) formed by deploying edge servers on LEO satellites has many application scenarios, such as being used as a communication relay station to help improve the coverage and capacity of global communication networks, speed up data processing and forwarding, and reduce communication delays between ground stations. In some remote or inaccessible areas, LEO satellite edge servers can be used for remote monitoring and control systems. However, offloading a large number of computing tasks to low-orbit satellites also brings some problems. The computing, bandwidth and storage resources on the satellite are limited, and it is impossible to compute tasks without limit; the high mobility between satellites causes the connection to need to be constantly switched, and the supply of energy at different times all cause the satellite to be unable to meet the user's demand. The Chinese patent with the application number CN118250750A discloses a satellite edge computing task offloading and resource allocation method based on deep reinforcement learning, which uses a three-layer computing architecture to meet the service needs of the user end to cope with the problem of computing task overload, but when the access satellite computing resources are insufficient, this method fails to fully utilize the resources of the satellite system to achieve efficient task processing. SUMMARY

[0006] The present application provides a satellite multi-hop task offloading optimization method and system based on edge computing, which is used to solve the technical problem that the existing satellite edge computing task processing method fails to fully utilize the resources of the satellite system to achieve efficient task processing when the access satellite computing resources are insufficient.

[0007] The first aspect of the present application provides a satellite multi-hop task offloading optimization method based on edge computing, which comprises:

[0008] Obtaining environment exploration information based on an edge computing network system model of a ground user and a low-orbit satellite network, and selecting local computing or offloading the computing task to satellite network computing according to the environment exploration information;

[0009] When the computing task is offloaded to satellite network computing, if the access satellite computing resources of the ground user are insufficient, a on-demand routing protocol is adopted, and under the constraint conditions of satellite time and computing resources, a task offloading optimization problem is obtained based on the environment exploration information and with the minimum energy consumption of the system model as the target;

[0010] The task offloading optimization problem is constructed as a Markov decision process, and a deep reinforcement learning algorithm is combined to solve it, so as to obtain a task offloading and routing selection optimization strategy to offload part of the tasks of the ground user to a satellite node with MEC enabled and sufficient computing resources in the satellite network through multi-hop, and the routing table provides optimal path information from the current satellite to other reachable satellites.

[0011] Specifically, the low-orbit satellite network is composed of a plurality of low-orbit satellite nodes.

[0012] In the system model, the network topology of the satellite network is constructed as a directed graph ,in and Represents satellite set and intersatellite link set respectively; satellite network is based on access satellite Built around is the number of hops, where one hop represents the link distance between two adjacent satellites; ground users are randomly distributed on the access satellites Within the coverage area, the computing tasks generated by the ground user in each time slot k are , ,in Expressed as data size, Indicates the number of cycles required to calculate a unit bit, It's a task completion deadline.

[0013] Specifically, the satellite time and computing resource constraints include:

[0014] When a satellite network provides computing services to ground users, the constraints on offloading the ground users’ computing tasks to the satellite are expressed as follows:

[0015]

[0016] Where: Indicates that the computing task of user u is offloaded to the satellite A binary decision variable, Indicates that the task of user u is offloaded to the satellite network binary decision variables;

[0017] The constraint on the offloading rate of computing tasks is expressed as:

[0018]

[0019] Where: Indicates the offloading rate of computing tasks;

[0020] The computational resource constraints of each satellite are expressed as:

[0021]

[0022] Where: represents the allocated computing resources of the satellite network, Indicates satellite Available computing resources;

[0023] The input and output degree constraints of each satellite are expressed as:

[0024]

[0025] In the formula: denotes the link is used to offload the computing task of the ground user u to the satellite network The binary decision variable, and respectively represent the input link set and the output link set of the satellite ;

[0026] The input degree and the output degree of each satellite are equal, wherein when the input degree of the access satellite is 0 and the output degree of the offload satellite is 0, the path selection constraint is represented as:

[0027]

[0028] The constraint representing the total time required by the system model to complete the computing task is represented as:

[0029]

[0030] In the formula: denotes the total time required by the system model to complete the computing task, denotes the maximum service delay time of the ground user u.

[0031] Specifically, the total time required by the system model to complete the computing task includes the transmission delay of the ground user and the access satellite, the propagation delay of the ground user to the access satellite, the propagation delay of the computing task along the routing path, the total transmission time of the computing task on the path, the local computing delay of the ground user and the computing delay of the satellite node processing the computing task.

[0032] Specifically, the total energy consumption of the system model is represented as:

[0033]

[0034] In the formula: denotes the energy consumption caused by local computing, denotes the transmission energy consumption of the computing task offloaded to the satellite network, denotes the energy consumption caused by multi-hop transmission, denotes the computing energy consumption of the computing task offloaded to the satellite with sufficient computing resources;

[0035] Wherein,

[0036]

[0037] In the formula: denotes the computing delay of the satellite ​to the next hop satellite a link transmit power of the next hop satellite, denotes the total transmission time of the computing task on the path;

[0038] wherein,

[0039]

[0040] in the formula: denotes the energy consumption coefficient of the effective switch capacitor of the chip architecture.

[0041] Specifically, the task offloading optimization problem is constructed as a Markov decision process, and a deep reinforcement learning algorithm is used to solve it to obtain a task offloading and routing selection optimization strategy to realize offloading part of the ground user's task to a satellite node with MEC enabled and sufficient computing resources in the satellite network through multi-hop, and the step of recording the optimal path information from the current satellite to other reachable satellites in real time by the routing table, comprising:

[0042] The task offloading optimization problem is constructed as a Markov decision process, the environment exploration information is taken as a state, a task allocation strategy and a routing selection action are performed, the total energy consumption of the task hop number and the system model is set as a reward, and a TD3 algorithm is used to solve it to intelligently select the optimal decision under the current state, and then obtain a task offloading and routing selection optimization strategy to realize offloading part of the ground user's task to a satellite node with MEC enabled and sufficient computing resources in the satellite network through multi-hop, and the step of recording the optimal path information from the current satellite to other reachable satellites in real time by the routing table.

[0043] The second aspect of the present application provides a satellite multi-hop task offloading optimization system based on edge computing, comprising:

[0044] The selection module is used to obtain environment exploration information based on an edge computing network system model of a ground user and a low-orbit satellite network, and select local computing or offload computing tasks to satellite network computing for the computing tasks of the ground user according to the environment exploration information.

[0045] The optimization problem generation module is used to, when offloading computing tasks to satellite network computing, if the access satellite computing resources of the ground user are insufficient, obtain a task offloading optimization problem based on the environment exploration information with the minimum energy consumption of the system model as the target under the constraint conditions of satellite time and computing resources.

[0046] An optimization strategy generation module is configured to construct a task offloading optimization problem as a Markov decision process and solve the problem by combining a deep reinforcement learning algorithm to obtain a task offloading and routing optimization strategy to implement the partial tasks of the ground user to be offloaded to a satellite node with sufficient computing resources and enabled MEC in the satellite network through multi-hop, and provide optimal path information from the current satellite to other reachable satellites.

[0047] Specifically, the low-orbit satellite network is composed of a plurality of low-orbit satellite nodes.

[0048] In the system model, the network topology of the satellite network is constructed as a directed graph , wherein and respectively represent a satellite set and an inter-satellite link set; the satellite network is constructed with the access satellite as the center, and the number of hops is , wherein one hop represents the link distance between two adjacent satellites; the ground users are randomly distributed in the coverage range of the access satellite , and the ground user generates a computing task , in each time slot k, wherein represents the data size, represents the number of periods required for one computing unit bit, is the completion deadline of the task .

[0049] The third aspect of the present application provides a computer device comprising a memory and a processor, the memory stores a computer program, and the processor implements the steps of the satellite multi-hop task offloading optimization method according to any one of the above when executing the computer program.

[0050] The fourth aspect of the present application provides a computer readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the steps of the satellite multi-hop task offloading optimization method according to any one of the above.

[0051] From the above technical solutions, the present application has the following advantages:

[0052] The application provides a satellite multi-hop task offloading optimization method and system based on edge computing.

[0053] In the application, when there are a large number of computing tasks, part of the computing tasks can be offloaded to the satellite LEO satellite for collaborative processing. In the case of insufficient access satellite computing resources, a multi-hop task offloading mode is adopted, and a on-demand routing protocol is used to support efficient multi-hop transmission, so that the task is transmitted to the processing node through multi-hop in a dynamic manner on the basis of the minimum number of hops, the selection of the intermediate satellite node on the routing path from the source satellite node to the destination satellite node is optimized by comprehensively considering the energy consumption, the control overhead of the entire network topology in the task offloading process is minimized, and the technical problem that the existing satellite edge computing task processing method cannot fully utilize the resources of the satellite system to realize efficient task processing when the access satellite computing resources are insufficient is solved. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0055] Figure 1 A step flow chart of a satellite multi-hop task offloading optimization method based on edge computing provided by the embodiment of the application;

[0056] Figure 2 An architecture diagram of an edge computing network system model based on a ground user and a low-orbit satellite network provided by the embodiment of the application;

[0057] Figure 3The optimization method implementation flowchart of the edge computing network system model based on a ground user and a low earth orbit satellite network provided by the embodiment of the present application is provided.

[0058] Figure 4 The structural block diagram of the satellite multi-hop task offloading optimization method system based on edge computing provided by the embodiment of the present application is provided. DETAILED DESCRIPTION

[0059] The embodiment of the present application provides a satellite multi-hop task offloading optimization method and system based on edge computing, which is used for solving the technical problem that the existing satellite edge computing task processing method cannot fully utilize the resources of a satellite system to realize efficient task processing when satellite computing resources are insufficient.

[0060] In order to make the application purpose, features and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the following described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0061] Referring to Figures 1-3 , the first aspect of the present application provides a satellite multi-hop task offloading optimization method based on edge computing, the method comprising:

[0062] Step 101, obtaining environment exploration information of an edge computing network system model based on a ground user and a low earth orbit satellite network, and selecting local computing or offloading a computing task to a satellite network computing according to the environment exploration information.

[0063] Referring to Figure 2 , the edge computing network system model based on a ground user and a low earth orbit satellite network is composed of a ground user and a low earth orbit satellite network, the satellite network is composed of a plurality of low earth orbit satellite nodes, and the system model is built based on a quasi-static scene. Among them, the network topology of the satellite and the associated user remain fixed during the computing service, and the routing table and trajectory information of the LEO constellation are known.

[0064] When there are a large number of computing tasks, part of the computing tasks can be offloaded to the satellite LEO satellite for collaborative processing, so the computing tasks generated by the ground user can be divided into two parts, one part of the tasks is processed by the user locally, and the other part of the tasks is uploaded to the satellite network for computing.

[0065] In the system model, the running time of the entire satellite network is divided into equal length time slots, each time slot has a duration of The network topology of the satellite network is constructed as a directed graph Wherein And Respectively represent the satellite set and the inter-satellite link set; the satellite network is constructed with the access satellite as the center, and has a hop number of , wherein a hop represents the link distance between two adjacent satellites. For the satellite , , the in-degree link set and the out-degree link set thereof are respectively represented as And , and the computing resources available to the current satellite node are represented as .

[0066] Suppose there are U users, and the index set thereof is , the users are randomly distributed within the coverage range of the access satellite , and the computing task generated by each ground user in each time slot is , , wherein is represented as the data size, is represented as the number of cycles required for a computing unit bit, is the completion deadline of the task . If the computing task is not completed before , it will be discarded, and the user terminal re-sends the task request at the next appropriate time.

[0067] Wherein, the environment exploration information can include task information , a signal-to-noise ratio vector of the link around the satellite node, , and the computing resources available to the satellite network ; the environment exploration information is used to determine the state of the system model and the processing selection of the computing task. When the computing resources of the access satellite of the ground user are sufficient, the computing processing can be directly performed on the satellite accessed by the ground user.

[0068] Step 102, when the computing task is offloaded to the satellite network computing, if the computing resources of the access satellite of the ground user are insufficient, an on-demand routing protocol is adopted, and under the constraint conditions of the satellite time and the computing resources, the environment exploration information is used to obtain a task offloading optimization problem with the minimum energy consumption of the system model as the target.

[0069] In the present application, when the access satellite computing resources of the ground user are insufficient, a multi-hop offloading scheme is adopted, the task is offloaded to the satellite node enabled with MEC, and the on-demand routing protocol (AODV routing protocol) is combined to support efficient multi-hop transmission, so that the task reaches the processing node dynamically through multi-hop on the basis of minimum hop number, and the energy consumption is considered comprehensively.

[0070] In order to better illustrate the allocation of the task, in the present application, binary decision variables are defined , otherwise ; binary decision variables indicate that the task of user u is offloaded to the satellite , otherwise ; binary decision variables indicate that the link is used to offload the task of user u to the satellite network , otherwise .

[0071] Among them, the target of the system model is to minimize the energy consumption of completing the task under the constraints of LEO satellite time and computing resources, and specifically, the description of each module model, the specification of the optimization problem and the optimization algorithm in the system model is as follows:

[0072] I. Communication model between user and satellite

[0073] When the task of user u needs to be offloaded , the task will be transmitted to the satellite network through the satellite-ground link. Then, the channel gain between the user and the access satellite can be represented by the free space path loss model, which is given by equation (1):

[0074]

[0075] Among them, represents the power gain when the reference distance is 1m, represents the distance between the user and the access satellite .

[0076] During the task transmission process, it is assumed that the bandwidth is evenly distributed to each user, and the transmission rate between the user and the access satellite in time slot k is:

[0077]

[0078] In the formula: represents the transmission power of user u, represents the variance of Gaussian white noise.

[0079] Transmission delay of user to access satellite is:

[0080]

[0081] wherein: denotes the remaining task size.

[0082] Transmission energy of user task offloaded to satellite network is:

[0083]

[0084] Propagation delay of user to access satellite may be expressed as:

[0085]

[0086] wherein: denotes the distance between user and access satellite, and c denotes the speed of light.

[0087] II. Transmission model between satellites

[0088] In satellite network, tasks can be forwarded over several hops, and when the source satellite node is insufficient in computing resources, the tasks can be offloaded to the satellite nodes with MEC enabled on the routing path. Since the distance between the source satellite node and the destination satellite node where the data is finally processed can exceed the range of direct communication, each task will be transmitted through multiple hops until it reaches the destination satellite node.

[0089] Let denote the transmission path with H hops between the source satellite node and the destination satellite node, wherein denotes the path the index of the hth hop satellite on the path.

[0090] The routing decision of each satellite node depends on the real-time updated link state information, such as signal-to-noise ratio (SNR), available frequency, link interval, and buffer space of the next hop node, etc. In the present application, the signal-to-noise ratio between nodes is taken as the key metric for node routing selection. Let denote the signal-to-noise ratio vector of the links around the node, wherein denotes the satellite index around the current node, then:

[0091]

[0092] wherein: denotes the channel power gain of the current node and the neighboring nodes denotes the current node​ and the link transmit power of node , denotes the variance of the Gaussian white noise.

[0093] denotes the available bandwidth of the link between node , and node , thus the transmission rate between satellite nodes can be denoted as:

[0094]

[0095] At each time slot, the satellite node makes offloading and routing decisions, if the satellite chooses to transmit the task, the transmission time from the satellite to the next hop satellite is given by:

[0096]

[0097] The propagation delay of the computation task along the routing path can be denoted as:

[0098]

[0099] The total transmission time over the whole path can be given by the summation of all single-hop transmission times:

[0100]

[0101] Thus the energy expenditure caused by multi-hop transmission can be denoted as:

[0102]

[0103] III. Computation Model

[0104] To reduce the energy consumption while ensuring the task completion, computation offloading is performed at appropriate nodes during the transmission process, thus the total delay also includes the computation delay at the nodes.

[0105] Let denote the computation capability of the ground user, which can be different for different ground users. Then, the local computation delay of the ground user can be denoted as:

[0106]

[0107] The energy expenditure caused by local computation​​​ can be expressed as:

[0108]

[0109] where is the energy consumption coefficient of the effective switched-capacitor depending on the chip architecture.

[0110] After the task is uploaded to the processing satellite, the represents the allocated computing resources of the satellite, and the computing delay of the satellite node processing the task is is:

[0111]

[0112] When the task is offloaded to a satellite with sufficient computing resources, the energy consumption of computing the task is is:

[0113]

[0114] Therefore, the total time required to complete the task is can be expressed as:

[0115]

[0116] The total energy consumption on the entire routing path is can be expressed as:

[0117]

[0118] After the computation is completed, the satellite is responsible for sending the results back to the user. Since the size of the computation result is much smaller than the size of the task, the transmission time and energy consumption required to return the result can be ignored. In addition, after the processing of multiple time slots is completed, the task result is downloaded to the user through the shortest path from the computing satellite node to the access satellite node.

[0119] IV. Constraints

[0120] When the satellite network provides computing services for users, the user task can and can only be offloaded to the satellite, which can be expressed as:

[0121]

[0122] At the same time, when the user task is offloaded to the satellite network, it will guarantee the partial offloading rate is a number between 0 and 1, so the constraint on task offloading is:

[0123]

[0124] Considering the lack of satellite computing resources, it is necessary to ensure that the resource requirements of each user task do not exceed the current available resources of the satellite. Then, the computing resource constraints of each satellite can be expressed as:

[0125]

[0126] When tasks are offloaded between different satellites, it is necessary to ensure that a path is selected to route traffic. For data routing in a satellite network, it is necessary to first ensure that the number of input and output degrees of each satellite is not greater than 1, which can be expressed as:

[0127]

[0128] Then, it is necessary to ensure that the input and output degrees of each satellite are equal, where the input degree of the access satellite is 0 and the output degree of the offload satellite is 0, and the path selection constraint can be expressed as:

[0129]

[0130] It is also necessary to ensure that the service delay time of each user task does not exceed the acceptable maximum service delay time , which can be expressed as:

[0131]

[0132] Five, task offloading optimization problem

[0133] The energy of the entire routing path is taken as the overhead of the system model, and the energy consumption of all nodes is minimized in the long-term task processing process under the delay and resource constraints, thereby minimizing the control overhead of the entire network topology in the task offloading process:

[0134]

[0135] Step 103, the task offloading optimization problem is constructed as a Markov decision process, and a deep reinforcement learning algorithm is used to solve it, to obtain a task offloading and routing selection optimization strategy to realize the offloading of part of the tasks of the ground user to the satellite nodes with MEC enabled and sufficient computing resources in the satellite network through multi-hop, and the optimal path information from the current satellite to other reachable satellites is recorded in real time by the routing table.

[0136] In the multi-hop mode, the satellite node finds the next hop satellite and minimizes the energy consumption according to the offloading ratio. Considering that the action of the satellite may affect the state of the environment, the total energy consumption of the system is determined by the joint action of the current system environment state and the satellite. Moreover, the previous state and the previous action jointly trigger the system environment to enter the next new random state.

[0137] In this case, the task offloading optimization problem can be expressed as a Markov decision process (MDP), denoted as .in Indicates possible states. Represents a collection of actions, represents the reward function, represents the state transition matrix.

[0138] In the present invention, environmental exploration information is used as the state, task allocation strategy and routing selection actions are executed, the number of task hops and the total energy consumption of the system model are set as rewards, and the TD3 algorithm is used for solution. The optimal decision under the current state is intelligently selected, and the selection of intermediate satellite nodes in the path from the source node (access satellite) to the destination node (MEC-enabled satellite) is optimized. It can be understood that due to the different signal-to-noise ratios between satellites, the transmission energy consumption between satellites is different. Therefore, the intelligent agent selects a path with the minimum energy consumption under the multi-hop task offloading mode through training and learning.

[0139] Specifically, each element in the MDP is described as follows:

[0140] State Space ( ): In each time slot, the system model observes the current state, obtains environmental exploration information, and uses the calculation task information , the signal-to-noise ratio vector of the links around the satellite node , computing resources available on the satellite network To represent the environmental exploration information. Therefore, the system state in time slot k The definition is as follows:

[0141]

[0142] Action Space ( ):get After that, discrete continuous actions will be generated. For satellites, the action set needs to select the appropriate next-hop routing node to minimize transmission energy consumption. The specific settings can be In addition, the satellite also needs to determine the task allocation ratio , so the action in time slot k The definitions are as follows:

[0143] award( ): For each time slot k, obtain The reward for executing the selected action in the current state. Generally speaking, the reward function is associated with the objective function. This invention is aimed at the traditional AODV routing protocol, and comprehensively considers the number of hops H and the energy consumption of the task to generate the reward in time slot k. The definitions are as follows:

[0144]

[0145] in ,when The larger the value of , the higher the reward value.

[0146] To solve the above MDP problem, the TD3 method is proposed, considering the continuous action space of the task offloading optimization problem. The TD3 algorithm is a deep reinforcement learning algorithm based on a high-dimensional continuous action space and is an optimized version of DDPG. Similar to DDPG, based on the actor-critic (AC) framework and using a target network, the actor-critic algorithm consists of a policy function and a Q-value function. The policy function acts as an actor to generate actions, and the Q-value function acts as a critic to evaluate the performance of actions. The specific implementation process is as follows: Figure 3 shown.

[0147] The present invention provides a satellite multi-hop task offloading optimization method based on edge computing. When there are a large number of computing tasks, some of the computing tasks can be selected to be offloaded to the LEO satellite for collaborative processing. At the same time, considering the situation where the computing resources of the access satellite are insufficient, the computing tasks are offloaded to the satellite nodes equipped with MEC in a multi-hop manner under the constraint of meeting the computing task delay. At the same time, combined with the AODV routing protocol, the energy consumption of all nodes is minimized during the long-term task processing process subject to delay and resource constraints, thereby minimizing the control overhead of the entire network topology during the task offloading process, optimizing the selection of intermediate satellite nodes on the routing path from the source satellite node to the destination satellite node, and recording the optimal path information from the current satellite to other reachable satellites in real time.

[0148] See also Figure 4 The second aspect of the present invention provides a satellite multi-hop task offloading optimization system based on edge computing, comprising:

[0149] The selection module 201 is used to obtain environmental exploration information based on the edge computing network system model of ground users and low-orbit satellite networks, and select local computing for ground users or offload computing tasks to satellite network computing according to the environmental exploration information;

[0150] An optimization problem generating module 202 is used to generate a task offloading optimization problem based on environmental exploration information and with the goal of minimizing energy consumption of the system model, while satisfying satellite time and computing resource constraints when offloading computing tasks to satellite network computing.

[0151] The optimization strategy generation module 203 is configured to construct the task offloading optimization problem as a Markov decision process, and solve the problem by combining a deep reinforcement learning algorithm to obtain a task offloading and routing optimization strategy to implement the partial task of the ground user to the satellite network through multi-hop to the satellite node with enabled MEC and sufficient computing resources, and provide the optimal path information from the current satellite to other reachable satellites.

[0152] The third aspect of the present application provides a computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps of the satellite multi-hop task offloading optimization method according to any one of the above when executing the computer program.

[0153] The fourth aspect of the present application provides a computer readable storage medium storing a computer program, and the processor implementing the steps of the satellite multi-hop task offloading optimization method according to any one of the above when executing the computer program.

[0154] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0155] In the several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented by other manners. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and actual implementation can be in another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0156] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., they can be located in one place, or distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0157] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit.

[0158] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or the entire or part of the technical solutions that essentially contribute to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0159] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A satellite multi-hop task offloading optimization method based on edge computing, characterized in that: The method comprises: Obtaining environmental exploration information based on an edge computing network system model of ground users and a low-orbit satellite network, and selecting local computing for ground users or offloading computing tasks to satellite network computing based on the environmental exploration information; When offloading computing tasks to satellite network computing, if the ground user's access satellite computing resources are insufficient, an on-demand routing protocol is used. Under the constraints of satellite time and computing resources, the task offloading optimization problem is obtained based on the environmental exploration information and the goal of minimizing the energy consumption of the system model. The task offloading optimization problem is formulated as a Markov decision process and solved with a deep reinforcement learning algorithm. The task offloading and routing optimization strategies are obtained to offload part of the ground user's tasks to satellite nodes in the satellite network that have MEC enabled and sufficient computing resources through a multi-hop approach. At the same time, the routing table provides the optimal path information from the current satellite to other reachable satellites.

2. The satellite multi-hop task offloading optimization method according to claim 1, wherein: The low-orbit satellite network is composed of a plurality of low-orbit satellite nodes; In the system model, the network topology of the satellite network is constructed as a directed graph ,in and Represents satellite set and intersatellite link set respectively; satellite network is based on access satellite Built around is the number of hops, where one hop represents the link distance between two adjacent satellites; ground users are randomly distributed on the access satellites Within the coverage area, the computing tasks generated by the ground user in each time slot k are , ,in Expressed as data size, Indicates the number of cycles required to calculate a unit bit, It's a task completion deadline.

3. The satellite multi-hop task offloading optimization method according to claim 2, characterized in that: The satellite time and computing resource constraints specifically include: When a satellite network provides computing services to ground users, the constraints on offloading the ground users’ computing tasks to the satellite are expressed as follows: Where: Indicates that the computing task of user u is offloaded to the satellite A binary decision variable, Indicates that the task of user u is offloaded to the satellite network binary decision variables; The constraint on the offloading rate of computing tasks is expressed as: Where: Indicates the offloading rate of computing tasks; The computational resource constraints of each satellite are expressed as: Where: represents the allocated computing resources of the satellite network, Indicates satellite Available computing resources; The input and output degree constraints of each satellite are expressed as: Where: Indicates a link Used to offload computing tasks from ground user u to the satellite network A binary decision variable, and Represents satellites The in-degree link set and out-degree link set of ; The input degree and output degree of each satellite are equal, where the input degree of the connected satellite is 0 and the output degree of the unloaded satellite is 0. The path selection constraint is expressed as: The constraint on the total time required for the system model to complete the computational task is expressed as: Where: Indicates the total time required for the system model to complete the calculation task, Indicates the maximum service delay time of ground user u.

4. The satellite multi-hop task offloading optimization method according to claim 2, characterized in that: The total time required for the system model to complete the computing task includes the transmission delay between the ground user and the access satellite, the propagation delay from the ground user to the access satellite, the propagation delay of the computing task along the routing path, the total transmission time of the computing task on the path, the local computing delay of the ground user, and the computing delay of the satellite node processing the computing task.

5. The satellite multi-hop task offloading optimization method according to claim 2, characterized in that: Total energy consumption of the system model Expressed as: Where: represents the energy cost caused by local computation, represents the transmission energy consumption of offloading computing tasks to the satellite network, represents the energy overhead caused by multi-hop transmission, represents the computational energy consumption of offloading computational tasks to satellites with sufficient computational resources; in, Where: Indicates that from the satellite To the next hop satellite The link transmission power, Indicates the total transmission time of the computing task on the path; in, Where: The energy dissipation factor representing the effective switching capacitance of the chip architecture.

6. The satellite multi-hop task offloading optimization method according to claim 1, characterized in that: The task offloading optimization problem is constructed as a Markov decision process and solved in combination with a deep reinforcement learning algorithm to obtain a task offloading and routing optimization strategy to offload part of the ground user's tasks to a satellite node in the satellite network that has MEC enabled and sufficient computing resources through a multi-hop manner. At the same time, the routing table records the optimal path information from the current satellite to other reachable satellites in real time, including the following steps: The task offloading optimization problem is constructed as a Markov decision process. The environmental exploration information is used as the state, and the task allocation strategy and routing selection action are executed. The number of task hops and the total energy consumption of the system model are set as rewards. The TD3 algorithm is used to solve the problem, and the optimal decision under the current state is intelligently selected. Then, the task offloading and routing selection optimization strategy is obtained to achieve the goal of offloading part of the ground user's tasks to the satellite nodes in the satellite network that have enabled MEC and sufficient computing resources through multi-hop. At the same time, the routing table records the optimal path information from the current satellite to other reachable satellites in real time.

7. A satellite multi-hop task offloading optimization system based on edge computing, characterized in that: include: A selection module is used to obtain environmental exploration information based on an edge computing network system model of ground users and a low-orbit satellite network, and select local computing for ground users or offload computing tasks to satellite network computing according to the environmental exploration information; An optimization problem generation module is used to generate a task offloading optimization problem based on the environmental exploration information and the goal of minimizing energy consumption of the system model, while satisfying satellite time and computing resource constraints when offloading computing tasks to satellite network computing. The optimization strategy generation module is used to construct the task offloading optimization problem as a Markov decision process and solve it in combination with a deep reinforcement learning algorithm to obtain a task offloading and routing optimization strategy to offload part of the ground user's tasks to satellite nodes in the satellite network that have enabled MEC and sufficient computing resources through multiple hops, while providing the optimal path information from the current satellite to other reachable satellites.

8. The satellite multi-hop task offloading optimization system according to claim 7, characterized in that: The low-orbit satellite network is composed of a plurality of low-orbit satellite nodes; In the system model, the network topology of the satellite network is constructed as a directed graph ,in and Represents satellite set and intersatellite link set respectively; satellite network is based on access satellite Built around is the number of hops, where one hop represents the link distance between two adjacent satellites; ground users are randomly distributed on the access satellites Within the coverage area, the computing tasks generated by the ground user in each time slot k are , ,in Expressed as data size, Indicates the number of cycles required to calculate a unit bit, It's a task completion deadline.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the satellite multi-hop task offloading optimization method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the satellite multi-hop task offloading optimization method according to any one of claims 1 to 6 are implemented.

Images