Satellite multi-hop task unloading optimization method and system based on edge calculation

Abstract

The invention discloses a satellite multi-hop task offload optimization method and system based on edge computing, and the method comprises the steps: obtaining a task offload optimization problem through employing an on-demand routing protocol under a constraint condition based on environment exploration information with the minimum energy consumption of a system model as a target when the access satellite computing resources of a ground user are insufficient; and constructing a task unloading optimization problem into a Markov decision process, and solving by combining a deep reinforcement learning algorithm to obtain a task unloading and routing optimization strategy. In the invention, under the condition of insufficient computing resources of an access satellite, an on-demand routing protocol is combined to support efficient multi-hop task transmission unloading, so that energy consumption is comprehensively considered and selection of an intermediate satellite node on a routing path from a source satellite node to a target satellite node is optimized on the basis of the minimum hop count of the task; and the processing nodes are dynamically reached through multiple hops, and meanwhile, the control overhead of the whole network topology in the task unloading process is reduced to the greatest extent.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technologies, and in particular to a satellite multi-hop task offloading optimization method and system based on edge computing. Background Art

[0002] In recent years, as the functions of cloud computing are increasingly moving to the edge of the network, new trends in computing are emerging. Collecting a large amount of idle computing power and storage space distributed at the edge of the network can generate sufficient capacity to perform computationally intensive and latency-critical tasks on mobile devices. This paradigm is called mobile edge computing (MEC). Unlike cloud computing, the computing tasks of terminal devices in MEC are offloaded to edge servers that are closer to the terminal devices, and the edge servers can provide computing, content caching and other functions. Servers distributed at the edge of the network (also called computing nodes, edge nodes) can reduce the computing pressure of terminal devices, reduce the frequency of centralized data interaction with cloud computing, and significantly reduce the waiting time in message exchange. Since edge servers have certain storage space and computing power and are closer to terminal devices, computationally intensive or latency-sensitive mobile terminal devices can offload computing tasks to edge servers for computing.

[0003] On the one hand, the computational offloading of task requests not only relieves the communication pressure of the core network, but also reduces the delay caused by long-distance data transmission. On the other hand, emerging applications in 5 / 6G also rely on computational offloading technology to provide users with efficient services. Therefore, computational offloading is one of the key technologies in MEC. Although MEC technology has been widely used in ground base stations and provides good services for mobile terminals, ground edge computing faces many problems. Ground edge computing cannot meet the global demand for "ubiquitous connectivity" because it is difficult to fully cover some complex terrains such as mountains and oceans. At the same time, ground network infrastructure is easily damaged by natural disasters such as earthquakes and hurricanes, resulting in user communication interruptions.

[0004] With the development of space communication networks, satellite technology has made great progress in commercial, civil and military services, and the economy and miniaturization of low-Earth orbit satellites are developing rapidly. Compared with other communication methods, satellite networks have advantages such as large coverage area and no terrain restrictions. Therefore, edge computing is introduced into satellite networks by borrowing the idea of ​​MEC on the ground network. Satellites deploying MEC servers need to be as close to users as possible, so they are usually located in low-Earth orbit (LEO), and space is regarded as the edge. In-station users get reliable computing services directly from satellites instead of communicating with ground bases. Satellites perform on-board processing tasks, not just as relay transponders, which will be an indispensable example of integrated STN in the future.

[0005] Satellite-based edge computing (Sa-MEC) formed by deploying edge servers on LEO satellites has many application scenarios, such as being used as communication relay stations to help improve the coverage and capacity of global communication networks, speed up the processing and forwarding of data, and reduce communication delays between ground stations. In some remote or difficult-to-access areas, LEO satellite edge servers can be used for remote monitoring and control systems. However, offloading a large number of computing tasks to low-orbit satellites also brings certain problems. The computing, bandwidth and storage resources on satellites are limited, and it is impossible to compute tasks without limit; the high mobility between satellites requires constant switching of connections, and the supply of energy at different times causes the satellite to be unable to meet user needs. The Chinese patent with application number CN118250750A discloses a satellite edge computing task offloading and resource allocation method based on deep reinforcement learning, which uses a three-layer computing architecture to meet the service needs of the user end to cope with the problem of computing task overload, but when the computing resources of the access satellite are insufficient, the method fails to fully utilize the resources of the satellite system to achieve efficient task processing. Summary of the invention

[0006] The present invention provides a satellite multi-hop task offloading optimization method and system based on edge computing, which is used to solve the technical problem that the existing satellite edge computing task processing method fails to fully utilize the resources of the satellite system to achieve efficient task processing when the computing resources of the access satellite are insufficient.

[0007] A first aspect of the present invention provides a satellite multi-hop task offloading optimization method based on edge computing, the method comprising:

[0008] Obtaining environmental exploration information of an edge computing network system model based on ground users and a low-orbit satellite network, and selecting local computing for ground users' computing tasks or offloading computing tasks to satellite network computing according to the environmental exploration information;

[0009] When offloading computing tasks to satellite network computing, if the computing resources of the access satellite of the ground user are insufficient, an on-demand routing protocol is adopted. Under the constraints of satellite time and computing resources, the task offloading optimization problem is obtained based on the environmental exploration information and the goal of minimizing the energy consumption of the system model.

[0010] The task offloading optimization problem is constructed as a Markov decision process and solved with a deep reinforcement learning algorithm. The task offloading and routing selection optimization strategies are obtained to offload part of the tasks of ground users to satellite nodes in the satellite network that have MEC enabled and sufficient computing resources through multi-hop. At the same time, the routing table provides the optimal path information from the current satellite to other reachable satellites.

[0011] Specifically, the low-orbit satellite network is composed of a plurality of low-orbit satellite nodes;

[0012] In the system model, the network topology of the satellite network is constructed as a directed graph ,in and Represents satellite set and intersatellite link set respectively; satellite network is based on access satellite centered on is the number of hops, where one hop represents the link distance between two adjacent satellites; ground users are randomly distributed on the access satellites Within the coverage area, the computing tasks generated by ground users in each time slot k are , ,in Expressed as data size, It represents the number of cycles required to calculate a unit bit. It's a task deadline for completion.

[0013] Specifically, the satellite time and computing resource constraints include:

[0014] When the satellite network provides computing services to ground users, the constraints on offloading the computing tasks of ground users to the satellite are expressed as:

[0015]

[0016] Where: Indicates that the computing task of user u is offloaded to the satellite The binary decision variables are Indicates that the task of user u is offloaded to the satellite network Binary decision variables of ;

[0017] The constraint on the offloading rate of computing tasks is expressed as:

[0018]

[0019] Where: Indicates the offloading rate of computing tasks;

[0020] The computational resource constraints of each satellite are expressed as:

[0021]

[0022] Where: represents the allocated computing resources of the satellite network, Indicates satellite Available computing resources;

[0023] The input and output degree constraints of each satellite are expressed as:

[0024]

[0025] Where: Indicates link Used to offload computing tasks from ground user u to the satellite network The binary decision variables are and Satellite The in-degree link set and out-degree link set of ;

[0026] The input degree and output degree of each satellite are equal, where the input degree of the connected satellite is 0 and the output degree of the unloaded satellite is 0. The path selection constraint is expressed as:

[0027]

[0028] The constraint on the total time required for the system model to complete the computational task is expressed as:

[0029]

[0030] Where: Represents the total time required for the system model to complete the calculation task, Represents the maximum service delay time of ground user u.

[0031] Specifically, the total time required for the system model to complete a computing task includes the transmission delay between the ground user and the access satellite, the propagation delay from the ground user to the access satellite, the propagation delay of the computing task along the routing path, the total transmission time of the computing task on the path, the local computing delay of the ground user, and the computing delay of the satellite node processing the computing task.

[0032] Specifically, the total energy consumption of the system model It is expressed as:

[0033]

[0034] Where: represents the energy cost caused by local computation, represents the transmission energy consumption of offloading computing tasks to the satellite network, represents the energy overhead caused by multi-hop transmission, represents the computational energy consumption of offloading computational tasks to satellites with sufficient computational resources;

[0035] in,

[0036]

[0037] Where: From satellite To the next satellite The link transmission power, Indicates the total transmission time of the computing task on the path;

[0038] in,

[0039]

[0040] Where: An energy dissipation factor that represents the effective switching capacitance of the chip architecture.

[0041] Specifically, the task offloading optimization problem is constructed as a Markov decision process, and solved in combination with a deep reinforcement learning algorithm to obtain a task offloading and routing selection optimization strategy to offload part of the tasks of ground users to a satellite node in the satellite network that has MEC enabled and sufficient computing resources through a multi-hop manner, and at the same time, the routing table records the optimal path information from the current satellite to other reachable satellites in real time, including:

[0042] The task offloading optimization problem is constructed as a Markov decision process. The environmental exploration information is used as the state, and the task allocation strategy and routing selection action are executed. The number of task hops and the total energy consumption of the system model are set as rewards. The TD3 algorithm is used to solve the problem, and the optimal decision under the current state is intelligently selected. Then, the task offloading and routing selection optimization strategy is obtained to offload part of the tasks of ground users to satellite nodes in the satellite network that have enabled MEC and sufficient computing resources through multi-hop. At the same time, the routing table records the optimal path information from the current satellite to other reachable satellites in real time.

[0043] A second aspect of the present invention provides a satellite multi-hop task offloading optimization system based on edge computing, comprising:

[0044] A selection module is used to obtain environmental exploration information of an edge computing network system model based on ground users and a low-orbit satellite network, and select local computing for ground users' computing tasks or offload computing tasks to satellite network computing according to the environmental exploration information;

[0045] An optimization problem generating module is used to obtain a task offloading optimization problem based on the environmental exploration information and taking the minimization of energy consumption of the system model as the goal, when offloading the computing task to the satellite network computing, if the access satellite computing resources of the ground user are insufficient, under the constraints of satellite time and computing resources;

[0046] The optimization strategy generation module is used to construct the task offloading optimization problem as a Markov decision process and solve it in combination with the deep reinforcement learning algorithm to obtain the task offloading and routing selection optimization strategy to achieve the offloading of part of the tasks of ground users to the satellite nodes in the satellite network that have enabled MEC and sufficient computing resources through multi-hop, and at the same time provide the optimal path information from the current satellite to other reachable satellites.

[0047] Specifically, the low-orbit satellite network is composed of a plurality of low-orbit satellite nodes;

[0048] In the system model, the network topology of the satellite network is constructed as a directed graph ,in and Represents satellite set and intersatellite link set respectively; satellite network is based on access satellite centered on is the number of hops, where one hop represents the link distance between two adjacent satellites; ground users are randomly distributed on the access satellites Within the coverage area, the computing tasks generated by ground users in each time slot k are , ,in Expressed as data size, It represents the number of cycles required to calculate a unit bit. It's a task deadline for completion.

[0049] A third aspect of the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of any of the above-mentioned satellite multi-hop task offloading optimization methods are implemented.

[0050] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of any of the satellite multi-hop task offloading optimization methods described above are implemented.

[0051] It can be seen from the above technical solutions that the present invention has the following advantages:

[0052] The present invention provides a satellite multi-hop task offloading optimization method and system based on edge computing, wherein the method comprises: obtaining environmental exploration information of an edge computing network system model based on ground users and low-orbit satellite networks, selecting local computing for ground users' computing tasks or offloading computing tasks to satellite network computing according to the environmental exploration information; when offloading computing tasks to satellite network computing, if the computing resources of the access satellite of the ground users are insufficient, adopting an on-demand routing protocol, and obtaining a task offloading optimization problem based on environmental exploration information and taking minimizing energy consumption of a system model as a goal under the constraints of satellite time and computing resources; constructing the task offloading optimization problem as a Markov decision process, and solving it in combination with a deep reinforcement learning algorithm, obtaining a task offloading and routing selection optimization strategy to realize offloading of part of the ground users' tasks to satellite nodes in the satellite network that have enabled MEC and sufficient computing resources in a multi-hop manner, and providing optimal path information from the current satellite to other reachable satellites by a routing table.

[0053] In the present invention, when there are a large number of computing tasks, some of the computing tasks can be selected to be offloaded to the satellite LEO satellite for collaborative processing. At the same time, considering the situation where the computing resources of the access satellite are insufficient, a multi-hop task offloading method is adopted, combined with an on-demand routing protocol to support efficient multi-hop transmission, so that the task is based on the minimum number of hops, comprehensively considers energy consumption, optimizes the selection of intermediate satellite nodes on the routing path from the source satellite node to the destination satellite node, dynamically reaches the processing node through multiple hops, and minimizes the control overhead of the entire network topology during the task offloading process, thereby solving the technical problem that the existing satellite edge computing task processing method fails to fully utilize the resources of the satellite system to achieve efficient task processing when the computing resources of the access satellite are insufficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0055] Figure 1 A flowchart of a satellite multi-hop task offloading optimization method based on edge computing provided by an embodiment of the present invention;

[0056] Figure 2 An architecture diagram of an edge computing network system model based on a ground user and low-orbit satellite network provided in an embodiment of the present invention;

[0057] Figure 3A flowchart of an optimization method for implementing an edge computing network system model based on a ground user and low-orbit satellite network provided in an embodiment of the present invention;

[0058] Figure 4 A structural block diagram of a satellite multi-hop task offloading optimization method system based on edge computing provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0059] The embodiments of the present invention provide a satellite multi-hop task offloading optimization method and system based on edge computing, which are used to solve the technical problem that the existing satellite edge computing task processing method fails to fully utilize the resources of the satellite system to achieve efficient task processing when the computing resources of the access satellite are insufficient.

[0060] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] See also Figure 1-Figure 3 The first aspect of the present invention provides a satellite multi-hop task offloading optimization method based on edge computing, the method comprising:

[0062] Step 101, obtain environmental exploration information of the edge computing network system model based on ground users and low-orbit satellite networks, and select local computing for ground users' computing tasks or offload computing tasks to satellite network computing according to the environmental exploration information.

[0063] See also Figure 2 The edge computing network system model based on ground users and low-orbit satellite networks consists of ground users and a low-orbit satellite network. The satellite network consists of many low-orbit satellite nodes. The system model is built based on a quasi-static scenario. The network topology and associated users of the satellite remain fixed during the computing service, and the routing table and trajectory information of the LEO constellation are known.

[0064] When there are a large number of computing tasks, we can choose to offload part of the computing tasks to the satellite LEO satellite for collaborative processing. Therefore, the computing tasks generated by ground users can be divided into two parts, one part of the tasks is processed locally by the user, and the other part of the tasks is uploaded to the satellite network for calculation.

[0065] In the system model, the operation time of the entire satellite network is Divided into equal length Time slices, the duration of each time slice is , construct the network topology of the satellite network as a directed graph ,in and Respectively represent the satellite set and the intersatellite link set; satellite network Satellite access centered on is the number of hops, where one hop represents the link distance between two adjacent satellites. , , its in-degree link set and out-degree link set are respectively expressed as and , the computing resources currently available to the satellite node are expressed as .

[0066] Assume there are U users, whose index set is , users are randomly distributed on the access satellite The computing tasks generated by ground users in each time slot within the coverage area , ,in Expressed as data size, It represents the number of cycles required to calculate a unit bit. It's a task If the task Not in If it is completed before, it will be discarded and the user terminal will resend the task request at the next appropriate time.

[0067] Among them, environmental exploration information can include mission information , the signal-to-noise ratio vector of the links around the satellite node , computing resources available on the satellite network Environmental exploration information is used to determine the state of the system model and the processing selection of the computing task. When the computing resources of the satellite accessed by the ground user are sufficient, the computing processing can be performed directly on the satellite accessed by the ground user.

[0068] Step 102, when offloading the computing task to the satellite network computing, if the ground user's access satellite computing resources are insufficient, an on-demand routing protocol is used. Under the constraints of satellite time and computing resources, the task offloading optimization problem is obtained based on the environmental exploration information and the goal of minimizing the energy consumption of the system model.

[0069] In the present invention, when the computing resources of the access satellite of the ground user are insufficient, a multi-hop offloading scheme is adopted to offload the task to the satellite node enabled with MEC, and the on-demand routing protocol (AODV routing protocol) is combined to support efficient multi-hop transmission, so that the task can reach the processing node dynamically through multi-hops with the minimum number of hops and comprehensive consideration of energy consumption.

[0070] In order to better illustrate the assignment of tasks, in the present invention, the binary decision variables are defined as Indicates that the task of user u is offloaded to the satellite network ,otherwise ; Binary decision variables Indicates that the task of user u is offloaded to the satellite ,no ; Binary decision variables Indicates link Used to offload the tasks of user u to the satellite network ,otherwise .

[0071] The goal of the system model is to minimize the energy consumption of completing the task while meeting the time and computing resource constraints of the LEO satellite. Specifically, the model of each module in the system model, the specification of the optimization problem, and the description of the optimization algorithm are as follows:

[0072] 1. Communication model between users and satellites

[0073] When user u's task When offloading is required, the task will be transmitted to the satellite network through the satellite-to-ground link. Then, the channel gain between the user and the access satellite can be represented by the free space path loss model, which is given by equation (1):

[0074]

[0075] in It represents the power gain when the reference distance is 1m. Indicates user With access satellite The distance between.

[0076] During task sending, assuming that the bandwidth Evenly distributed to each user, in time slot k, the transmission rate between the user and the access satellite is for:

[0077]

[0078] Where: represents the transmit power of user u, represents the variance of Gaussian white noise.

[0079] Transmission delay between user and access satellite for:

[0080]

[0081] Where: Indicates the remaining task size.

[0082] Therefore, the transmission energy consumption of user tasks offloaded to the satellite network for:

[0083]

[0084] Propagation delay from user to access satellite It can be expressed as:

[0085]

[0086] Where: represents the distance between the user and the access satellite, and c represents the speed of light.

[0087] 2. Transmission Model between Satellites

[0088] In satellite networks, tasks can be forwarded several hops away. When the computing resources of the access satellite node are insufficient, the tasks can be offloaded to the satellite node with MEC enabled on the routing path. Since the distance between the source satellite node and the destination satellite node that finally processes the data may exceed the range of direct communication, each task will be transmitted through multiple hops until it reaches the destination satellite node.

[0089] set up There are H hops between the source satellite node and the destination satellite node, where Indicates the path The index of the hth satellite on the previous hop.

[0090] The routing decision of each satellite node depends on the real-time updated link status information, such as signal-to-noise ratio (SNR), available frequency, link interval, and cache space of the next-hop node. In this invention, the signal-to-noise ratio between nodes is used as the key metric for node routing selection. To represent the signal-to-noise ratio vector of the links around the node, Represents the satellite index around the current node, then:

[0091]

[0092] Where: Indicates the current node and neighboring nodes The channel power gain, Indicates the current node and nodes The link transmission power, represents the variance of Gaussian white noise.

[0093] by Representation Node and nodes The available bandwidth of the link between satellite nodes The transmission rate It can be expressed as:

[0094]

[0095] In each time slot, the satellite node makes offloading and routing decisions. Select the transmission task, then To the next satellite Transmission time As shown below:

[0096]

[0097] Calculate the propagation delay of the task along the routing path It can be expressed as:

[0098]

[0099] Total transmission time along the entire path It can be given by the sum of all single-hop times:

[0100]

[0101] Therefore, the energy overhead caused by multi-hop transmission It can be expressed as:

[0102]

[0103] 3. Computational Model

[0104] To reduce energy consumption while ensuring task completion, the task is offloaded to appropriate nodes during transmission, so the total delay also includes the computational delay on the node.

[0105] Will It is defined as the computing power of ground users, and the computing power of different ground users can be different. Local calculation delay It can be expressed as:

[0106]

[0107] Energy overhead caused by local computation It can be expressed as:

[0108]

[0109] in is the energy dissipation factor of the effective switching capacitance that depends on the chip architecture.

[0110] After the mission is uploaded to the processing satellite, Represents the satellite's allocated computing resources and the computing delay of processing tasks at the satellite node for:

[0111]

[0112] When the task is offloaded to a satellite with sufficient computing resources, calculate the energy consumption of the task for:

[0113]

[0114] Therefore, the total time required to complete the task It can be expressed as:

[0115]

[0116] Total energy consumption on the entire routing path It can be expressed as:

[0117]

[0118] After the calculation is completed, the satellite is responsible for sending the results back to the user. Since the calculation result size is much smaller than the task size, the transmission time and energy consumption required to return the result can be ignored. In addition, after completing the processing of multiple time slots, the task results are downloaded to the user via the shortest path from the calculation satellite node to the access satellite node.

[0119] 4. Constraints

[0120] When the satellite network provides computing services to users, user tasks can and can only be offloaded to the satellite, which can be expressed as:

[0121]

[0122] At the same time, when user tasks are offloaded to the satellite network, the partial offloading rate will be guaranteed. is a number between 0 and 1, so the constraint on task offloading is:

[0123]

[0124] Considering the shortage of satellite computing resources, it is necessary to ensure that the resource requirements of each user task will not exceed the current available resources of the satellite. Then, the computing resource constraints of each satellite can be expressed as:

[0125]

[0126] When tasks are offloaded between different satellites, it is necessary to ensure that a path is selected to route traffic. For data routing in a satellite network, it is first necessary to ensure that the number of input and output degrees of each satellite is not greater than 1, which can be expressed as:

[0127]

[0128] Then, the input degree and output degree of each satellite are guaranteed to be equal, where the input degree of the connected satellite is 0 and the output degree of the unloaded satellite is 0. The path selection constraint can be expressed as:

[0129]

[0130] It is also necessary to ensure that the service delay of each user task does not exceed the maximum acceptable service delay. , which can be expressed as:

[0131]

[0132] 5. Task offloading optimization issues

[0133] The energy on the entire routing path is used as the overhead of the system model to minimize the energy consumption of all nodes during long-term task processing subject to delay and resource constraints, thereby minimizing the control overhead of the entire network topology during task offloading:

[0134]

[0135] Step 103, construct the task offloading optimization problem as a Markov decision process, and solve it in combination with a deep reinforcement learning algorithm to obtain a task offloading and routing optimization strategy to offload part of the tasks of ground users to satellite nodes in the satellite network that have enabled MEC and sufficient computing resources through multi-hop. At the same time, the routing table records the optimal path information from the current satellite to other reachable satellites in real time.

[0136] In the multi-hop approach, the satellite node searches for the next hop satellite and minimizes the energy consumption according to the offloading ratio. Considering that the actions of the satellite may affect the state of the environment, the total energy consumption of the system is determined by the current system environment state and the joint actions of the satellite. Moreover, the previous state and the previous action jointly trigger the system environment to enter the next new random state.

[0137] In this case, the task offloading optimization problem can be expressed as a Markov decision process (MDP), denoted as .in Indicates possible states. Represents a collection of actions, represents the reward function, represents the state transition matrix.

[0138] In the present invention, environmental exploration information is used as the state, task allocation strategy and route selection action are executed, the number of task hops and the total energy consumption of the system model are set as rewards, and the TD3 algorithm is used for solving, the optimal decision under the current state is intelligently selected, and the selection of intermediate satellite nodes in the path from the source node (access satellite) to the destination node (MEC-enabled satellite) is optimized; it can be understood that due to the different signal-to-noise ratios between satellites, the transmission energy consumption between satellites is different, so the intelligent agent selects a path with the minimum energy consumption under the multi-hop task offloading mode through training and learning.

[0139] Specifically, each element in MDP is described as follows:

[0140] State Space ( ): In each time slot, the system model observes the current state, obtains environmental exploration information, and uses the calculation task information , the signal-to-noise ratio vector of the links around the satellite node , computing resources available on the satellite network to represent the environmental exploration information. Therefore, the system state in time slot k The definition is as follows:

[0141]

[0142] Action Space ( ):get After that, discrete continuous actions will be generated. For satellites, the action set needs to select a suitable next-hop routing node to minimize transmission energy consumption. The specific settings can be In addition, the satellite also needs to determine the task allocation ratio , so the action in time slot k The definitions are as follows:

[0143] award( ): For each time slot k, obtain The reward for executing the selected action in the current state. Generally speaking, the reward function is associated with the objective function. The present invention is aimed at the traditional AODV routing protocol, and comprehensively considers the number of hops H and the energy consumption of the task to generate, so the reward in time slot k is The definitions are as follows:

[0144]

[0145] in ,when The larger the value of , the higher the reward value.

[0146] In order to solve the above MDP problem, considering the continuous action space of the task offloading optimization problem, the TD3 method is proposed. The TD3 algorithm is a deep reinforcement learning algorithm based on a high-dimensional continuous action space and is an optimized version of DDPG. Similar to DDPG, based on the actor-critic (AC) framework and using a target network, the actor-critic algorithm consists of a policy function and a Q-value function. The policy function acts as a participant in generating actions, and the Q-value function acts as a critic to evaluate the performance of the action. The specific implementation process is as follows: Figure 3 shown.

[0147] The present invention provides a satellite multi-hop task offloading optimization method based on edge computing. When there are a large number of computing tasks, some computing tasks can be selected to be offloaded to the satellite LEO satellite for collaborative processing. At the same time, the situation of insufficient computing resources of the access satellite is taken into consideration. Under the constraint of computing task delay, the computing task is offloaded to the satellite node equipped with MEC through multi-hop. At the same time, combined with the AODV routing protocol, the energy consumption of all nodes is minimized in the long-term task processing process subject to delay and resource constraints, thereby minimizing the control overhead of the entire network topology during the task offloading process, optimizing the selection of intermediate satellite nodes on the routing path from the source satellite node to the destination satellite node, and recording the optimal path information from the current satellite to other reachable satellites in real time.

[0148] See also Figure 4 The second aspect of the present invention provides a satellite multi-hop task offloading optimization system based on edge computing, comprising:

[0149] The selection module 201 is used to obtain environmental exploration information of the edge computing network system model based on the ground user and the low-orbit satellite network, and select local computing for the computing task of the ground user or offload the computing task to the satellite network computing according to the environmental exploration information;

[0150] The optimization problem generating module 202 is used to obtain the task offloading optimization problem based on the environmental exploration information and the goal of minimizing the energy consumption of the system model, while satisfying the satellite time and computing resource constraints when offloading the computing task to the satellite network computing.

[0151] The optimization strategy generation module 203 is used to construct the task offloading optimization problem as a Markov decision process, and solve it in combination with the deep reinforcement learning algorithm to obtain the task offloading and routing selection optimization strategy to achieve the offloading of part of the tasks of ground users to the satellite nodes in the satellite network that have enabled MEC and sufficient computing resources through multi-hop, and at the same time provide the optimal path information from the current satellite to other reachable satellites.

[0152] A third aspect of the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of any of the above satellite multi-hop task offloading optimization methods are implemented.

[0153] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above satellite multi-hop task offloading optimization methods.

[0154] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0155] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0156] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0157] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0158] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0159] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A satellite multi-hop task offloading optimization method based on edge computing, characterized in that: The method comprises: Obtaining environmental exploration information of an edge computing network system model based on ground users and a low-orbit satellite network, and selecting local computing for ground users' computing tasks or offloading computing tasks to satellite network computing according to the environmental exploration information; When offloading computing tasks to satellite network computing, if the computing resources of the access satellite of the ground user are insufficient, an on-demand routing protocol is adopted. Under the constraints of satellite time and computing resources, the task offloading optimization problem is obtained based on the environmental exploration information and the goal of minimizing the energy consumption of the system model. The task offloading optimization problem is constructed as a Markov decision process and solved with a deep reinforcement learning algorithm. The task offloading and routing selection optimization strategies are obtained to offload part of the tasks of ground users to satellite nodes in the satellite network that have MEC enabled and sufficient computing resources through multi-hop. At the same time, the routing table provides the optimal path information from the current satellite to other reachable satellites.

2. The satellite multi-hop task offloading optimization method according to claim 1, characterized in that: The low-orbit satellite network is composed of a plurality of low-orbit satellite nodes; In the system model, the network topology of the satellite network is constructed as a directed graph ,in and Represents satellite set and intersatellite link set respectively; satellite network is based on access satellite centered on is the number of hops, where one hop represents the link distance between two adjacent satellites; ground users are randomly distributed on the access satellites Within the coverage area, the computing tasks generated by ground users in each time slot k are , ,in Expressed as data size, It represents the number of cycles required to calculate a unit bit. It's a task deadline for completion.

3. The satellite multi-hop task offloading optimization method according to claim 2, characterized in that: The satellite time and computing resource constraints specifically include: When the satellite network provides computing services to ground users, the constraints on offloading the computing tasks of ground users to the satellite are expressed as: Where: Indicates that the computing task of user u is offloaded to the satellite The binary decision variables are Indicates that the task of user u is offloaded to the satellite network Binary decision variables of ; The constraint on the offloading rate of computing tasks is expressed as: Where: Indicates the offloading rate of computing tasks; The computational resource constraints of each satellite are expressed as: Where: represents the allocated computing resources of the satellite network, Indicates satellite Available computing resources; The input and output degree constraints of each satellite are expressed as: Where: Indicates link Used to offload computing tasks from ground user u to the satellite network The binary decision variables are and Satellite The in-degree link set and out-degree link set of ; The input degree and output degree of each satellite are equal, where the input degree of the connected satellite is 0 and the output degree of the unloaded satellite is 0. The path selection constraint is expressed as: The constraint on the total time required for the system model to complete the computational task is expressed as: Where: Represents the total time required for the system model to complete the calculation task, Represents the maximum service delay time of ground user u.

4. The satellite multi-hop task offloading optimization method according to claim 2, characterized in that: The total time required for the system model to complete the computing task includes the transmission delay between the ground user and the access satellite, the propagation delay from the ground user to the access satellite, the propagation delay of the computing task along the routing path, the total transmission time of the computing task on the path, the local computing delay of the ground user and the computing delay of the satellite node processing the computing task.

5. The satellite multi-hop task offloading optimization method according to claim 2, characterized in that: Total energy consumption of the system model It is expressed as: Where: represents the energy cost caused by local computation, represents the transmission energy consumption of offloading computing tasks to the satellite network, represents the energy overhead caused by multi-hop transmission, represents the computational energy consumption of offloading computational tasks to satellites with sufficient computational resources; in, Where: From satellite To the next satellite The link transmission power, Indicates the total transmission time of the computing task on the path; in, Where: An energy dissipation factor that represents the effective switching capacitance of the chip architecture.

6. The satellite multi-hop task offloading optimization method according to claim 1, characterized in that: The task offloading optimization problem is constructed as a Markov decision process, and is solved in combination with a deep reinforcement learning algorithm to obtain a task offloading and routing optimization strategy to offload part of the tasks of ground users to a satellite node in a satellite network that has MEC enabled and sufficient computing resources through a multi-hop manner, and at the same time, the routing table records the optimal path information from the current satellite to other reachable satellites in real time, including: The task offloading optimization problem is constructed as a Markov decision process. The environmental exploration information is used as the state, and the task allocation strategy and routing selection action are executed. The number of task hops and the total energy consumption of the system model are set as rewards. The TD3 algorithm is used to solve the problem, and the optimal decision under the current state is intelligently selected. Then, the task offloading and routing selection optimization strategy is obtained to offload part of the tasks of ground users to satellite nodes in the satellite network that have enabled MEC and sufficient computing resources through multi-hop. At the same time, the routing table records the optimal path information from the current satellite to other reachable satellites in real time.

7. A satellite multi-hop task offloading optimization system based on edge computing, characterized in that: include: A selection module is used to obtain environmental exploration information of an edge computing network system model based on ground users and a low-orbit satellite network, and select local computing for ground users' computing tasks or offload computing tasks to satellite network computing according to the environmental exploration information; An optimization problem generating module is used to obtain a task offloading optimization problem based on the environmental exploration information and taking the minimization of energy consumption of the system model as the goal, when offloading the computing task to the satellite network computing, if the access satellite computing resources of the ground user are insufficient, under the constraints of satellite time and computing resources; The optimization strategy generation module is used to construct the task offloading optimization problem as a Markov decision process and solve it in combination with the deep reinforcement learning algorithm to obtain the task offloading and routing selection optimization strategy to achieve the offloading of part of the tasks of ground users to the satellite nodes in the satellite network that have enabled MEC and sufficient computing resources through multi-hop, and at the same time provide the optimal path information from the current satellite to other reachable satellites.

8. The satellite multi-hop task offloading optimization system according to claim 7, characterized in that: The low-orbit satellite network is composed of a plurality of low-orbit satellite nodes; In the system model, the network topology of the satellite network is constructed as a directed graph ,in and Represents satellite set and intersatellite link set respectively; satellite network is based on access satellite centered on is the number of hops, where one hop represents the link distance between two adjacent satellites; ground users are randomly distributed on the access satellites Within the coverage area, the computing tasks generated by ground users in each time slot k are , ,in Expressed as data size, It represents the number of cycles required to calculate a unit bit. It's a task deadline for completion.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the satellite multi-hop task offloading optimization method as described in any one of claims 1-6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the satellite multi-hop task offloading optimization method as described in any one of claims 1-6 are implemented.

Images