Intelligent task unloading method for satellite-ground fusion network

Abstract

The invention provides an intelligent task unloading method for a satellite-ground fusion network, and the method comprises the steps: enabling a low-orbit satellite to obtain the task characteristics and network environment information of a reasoning task, and building a mathematical model of reasoning precision, task time delay and system overhead; based on the mathematical model, constructing a task unloading optimization problem oriented to inference service; a task unloading optimization problem is expressed as a local observation Markov decision process by taking a user as an intelligent agent and taking a task unloading decision as an action; adopting a multi-agent deep reinforcement learning intelligent algorithm based on a centralized training and distributed execution framework to obtain an optimized strategy network; and according to the optimized strategy network deployed on the low-orbit satellite, solving to obtain a task unloading decision. According to the method, the task unloading problem of the satellite-ground convergence network under the ubiquitous reasoning service requirement is solved, and the communication bandwidth burden of a satellite-ground link can be relieved and the system overhead can be minimized under the condition that the user time delay and reasoning precision requirements are met.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of wireless communication, and particularly relates to an intelligent task offloading method of a satellite-ground integrated network. BACKGROUND

[0002] AI inference services are widely used due to their excellent performance, including text generation, multi-modal data analysis, summarization, and generalization. AI inference services are usually implemented using ground clouds with sufficient resources. However, cloud-based inference services have many problems, including long communication latency, high bandwidth cost, and data privacy leakage. 6G satellite-ground integrated networks are expected to use satellite edge computing to provide low-latency ubiquitous AI inference services for user terminals.

[0003] Although existing technologies have made progress in ground networks, there are still serious challenges in task offloading for inference service-oriented satellite-ground integrated networks.

[0004] Firstly, existing research mainly focuses on ground networks. The edge nodes are usually set as ground base stations and user equipment. These solutions fail to fully consider the high-speed mobility of low-orbit satellites, the time-varying network topology, and the spatiotemporal dynamics of traffic distribution.

[0005] Secondly, AI task inference involves high-dimensional tensor computation and multi-layer Transformer architecture, requiring a large amount of memory resources and computing resources. Typically, the deep learning model Resnet-110 has about 1.7 million parameters, while the large language model Deepseek-V3 has up to 671 billion parameters. This will result in a huge memory and computing burden.

[0006] Secondly, the heterogeneity of satellite-ground integrated networks increases the computational complexity of task offloading decisions. Satellites as edge computing nodes are limited by size and weight, and have limited resources. Low-orbit satellites and ground cloud servers usually deploy inference models of different parameter sizes according to the difference in capabilities. Inference models of different parameter sizes have differentiated inference accuracy and computing load. For example, low-orbit satellites deploy quantized or distilled lightweight models, and ground cloud servers deploy complete models. This heterogeneity will result in the need to consider the trade-off between resource overhead and accuracy loss when choosing on-orbit processing or ground cooperation in offloading decisions, further amplifying the load balancing problem. Most current task offloading researches mainly focus on traditional, divisible, and accuracy-insensitive computing tasks, and fail to fully consider the huge resource requirements of inference tasks, node model heterogeneity, inference accuracy constraints, and state dependence.

[0007] Moreover, the key characteristics of inference tasks, such as task dependency, inference accuracy requirement, and output data size, further exacerbate the decision-making difficulty. The task offloading work of existing satellite-ground fusion networks needs to further research the dynamic decision-making process that balances the complex trade-offs between inference accuracy, task latency, and system overhead. On the one hand, AI inference tasks often have state dependency, with temporal or logical associations between tasks for the same user. For example, in multi-turn dialogue or multi-modal inference of LLM, the inference output generation depends on the previous calculation results. On the other hand, the inference accuracy and output size of AI inference tasks mainly depend on the model architecture and user settings. Unlike traditional tasks, the amount of inference output data is usually fixed or large and cannot be ignored.

[0008] Therefore, task offloading decisions need to consider the inherent characteristics of inference tasks in high-dynamic, resource-heterogeneous satellite-ground fusion networks, optimize on-orbit processing and ground collaborative decision-making to minimize system overhead, while meeting the strict requirements of users for inference accuracy and task latency.

[0009] To address these challenges, Deep Reinforcement Learning (DRL) provides an effective approach due to its excellent ability to handle high-dimensional, dynamic, and complex decision-making problems. However, there are many problems when applying DRL to satellite-ground fusion networks, including "dimension disaster" of global state, "combination explosion" of joint action space, "delay and signaling storm" of global information collection, and poor scalability and robustness.

[0010] DRL is considered an effective approach to solving the task offloading problem in 6G satellite-ground fusion networks due to its efficient self-adaptation and learning ability in handling high-dimensional complex problems. Literature [1] focuses on AI inference tasks, considering the characteristics of AI model heterogeneity, low-orbit satellite rapid movement, and limited on-orbit resources, and proposes a centralized hybrid DRL algorithm to jointly optimize task offloading location, communication power, and computing resources to minimize system cost. Literature [2] focuses on LEO satellite-assisted vehicle networks and uses a task priority-based Proximal Policy Optimization (PPO) algorithm to minimize the average latency as the optimization objective, deciding whether to offload computation to the local vehicle or LEO satellite. The above research is based on Markov Decision Process (MDP) and relies on global information for joint resource optimization. This will bring additional communication overhead and delay, and the robustness is poor. Once some state information is lost or new state information is added, such as the addition or departure of users in the network, or when a satellite fails or a new satellite enters orbit, it may lead to performance degradation.

[0011] References: [1] W. Fan, Q. Meng, G. Wang, H. Bian, Y. Liu and Y. Liu, "Satellite Edge Intelligence: DRL-Based Resource Management for Task Inference in LEO-Based Satellite-Ground Collaborative Networks," IEEE Trans. Mob. Comput., vol. 24, no. 10, pp. 10710-10728, 2025. [2] L. Wang, J. Li, M. Dai and H. Zhang, "Low-Earth-Orbit Satellite Assisted Edge Computing for Vehicular Networks: A Task Priority-Based Delay Minimization Approach," IEEE Internet Things J., vol. 12, no. 17, pp. 35482-35496, 2025. SUMMARY

[0012] The present application aims to provide an intelligent task offloading method for a satellite-ground integrated network, which can alleviate the communication bandwidth burden of the satellite-ground link, minimize system overhead, and meet the user delay and inference accuracy requirements.

[0013] To achieve the above-mentioned purpose, the present application provides an intelligent task offloading method for a satellite-ground integrated network, which is suitable for determining task offloading decisions when the satellite-ground integrated network provides inference services to users, and the satellite-ground integrated network includes at least one user, a plurality of low-orbit satellites, and a ground cloud server, comprising: S1: The low-orbit satellite obtains the task characteristics and network environment information of the inference task, and establishes a mathematical model of inference accuracy, task delay, and system overhead according to the task characteristics, network environment information, and task offloading decisions of the inference task; S2: Based on the mathematical model of step S1, a task offloading optimization problem for inference services is constructed; S3: Taking each user as an agent and the task offloading decision as an action, the task offloading optimization problem is expressed as a local observation Markov decision process; S4: based on the local observation Markov decision process of step S3, a multi-agent deep reinforcement learning intelligent algorithm based on a centralized training and distributed execution framework is used to obtain an optimized strategy network; each strategy network belongs to an agent, and takes local observation as input and outputs task offloading decision as action; S5: the low-orbit satellite solves the task offloading decision according to the optimized strategy network deployed thereon, and the task offloading decision includes the offloading location and model type of the inference task selected by the user.

[0014] The satellite-ground integrated network provides inference services for users, specifically including: first, the user sends an inference request of an inference task to a low-orbit satellite accessed by the user through a wireless link; the low-orbit satellite determines a task offloading decision, and then sends a control signaling message for preparing inference to the user and the offloading location of the inference task; then, the user sends the content of the inference request to the specified offloading location, so that the offloading location completes inference calculation processing and feeds back the output result to the user.

[0015] The task characteristics of the inference task include task input data volume, task output data volume, state information data volume, task delay upper limit and inference accuracy threshold; the network environment information includes communication and link information, satellite topology and dynamic characteristics and node resource state; the communication and link information includes uplink transmission from the user to the satellite, downlink transmission from the satellite to the user, inter-satellite transmission, downlink transmission from the satellite to the ground cloud and uplink transmission from the ground cloud to the satellite; the satellite topology and dynamic characteristics include the satellite moving speed of the low-orbit satellite, the remaining coverage time window of the satellite serving the ground user, and the remaining connection time of the inter-satellite link of the low-orbit satellite and the adjacent satellite; the node resource state includes the calculation amount and occupied memory resources of the local, adjacent star and ground cloud servers caused by the inference task.

[0016] The task offloading decision includes the offloading location and model type of the inference task selected by the user, wherein the offloading location of the inference task selected includes satellite local processing, adjacent star on-orbit processing and ground cloud processing.

[0017] If the inference task is processed locally on the satellite, the task latency includes request waiting latency, uplink transmission latency of task input from the user to the satellite, local computation latency, and downlink transmission latency of task output from the satellite to the user. If the inference task is processed in orbit on a neighboring satellite, the task latency includes request waiting latency, uplink transmission latency of task input from the user to the satellite, inter-satellite offloading latency of task input, neighboring satellite computation latency, inter-satellite offloading latency of task output, and downlink transmission latency of task output from the satellite to the user. If the inference task is processed by a ground cloud, the task latency mainly includes request waiting latency, uplink transmission latency of task input from the user to the satellite, downlink transmission latency of task input from the satellite to the ground cloud, ground cloud computing latency, uplink transmission latency of task output from the ground cloud to the satellite, and downlink transmission latency of task output from the satellite to the user. The system overhead caused by the user's inference task in the current time slot mainly includes model deployment overhead, state synchronization overhead, computation overhead, and communication overhead caused by the task transmission process.

[0018] In step S2, based on the task characteristics and network environment information of the inference task, and based on the mathematical model, a task offloading decision is determined within the decision cycle. This task offloading decision ensures the inference accuracy and task latency requirements of each user, while minimizing the total system overhead as the optimization objective. The task offloading decision of each user is used as the optimization variable to construct the task offloading optimization problem.

[0019] In the task offloading optimization problem model, the constraints include the limited service coverage of the offloading node, the limited on-orbit resources of the satellite, the lower limit of task inference accuracy, the upper limit of task latency, and constraints on decision variables.

[0020] In the local observation Markov process, the system state of each time slot includes the task offloading decision of the previous time slot, the task characteristics of the inference task, and network environment information; the user as the intelligent agent In the current time slot Local observation This includes the task offloading decision from the previous time slot, the task characteristics of the inference task, and network environment information that the agent can directly obtain; and the user as the agent. In the current time slot The definition of an action is user In the current time slot Task unloading decision; reward function Including instantaneous return functions and cumulative return functions, users Instantaneous revenue function in the current time slot It is the negative value of the system overhead caused by the user's inference task in the current time slot; the optimization objective is to maximize the cumulative revenue function, which is obtained by summing the instantaneous revenue functions at different times using the discount rate.

[0021] In the multi-agent deep reinforcement learning intelligent algorithm based on the centralized training and distributed execution framework, a participant-critic architecture is adopted; the participant is a policy network, each policy network corresponds to a user as an agent and is deployed on a low earth orbit satellite to which the user is uniquely connected, and the policy network is set to select the action of the user as an agent in the current time slot according to the local observation of the user as an agent in the current time slot In the current time slot The local observation of the user as an agent In the current time slot The action of the user as an agent; the critic is a value network, which is deployed on the ground cloud, and the value network is set to iteratively update all policy networks to obtain the optimized policy network.

[0022] When training the multi-agent deep reinforcement learning intelligent algorithm, the policy network parameters are updated based on the policy gradient rule to maximize the cumulative revenue; and the value network parameters are trained by minimizing the loss function.

[0023] The intelligent task offloading method of the star-ground fusion network provided by the application, on the one hand, the task offloading optimization problem faces the inference task, considers the task characteristics of the inference task, and combines the characteristics of the high-speed movement of the low earth orbit satellite of the star-ground fusion network and the unbalanced wide-area load, and considers the network environment information; on the other hand, the method takes each user as an agent, minimizes the total system overhead as the optimization target, and proposes a multi-agent deep reinforcement learning algorithm under the constraints of limited on-orbit resources and dynamic service demand, realizes the task offloading decision, and solves the task offloading problem of the star-ground fusion network facing the ubiquitous inference service demand, can meet the user delay and inference accuracy demand, and relieve the communication bandwidth burden of the star-ground link, and minimize the system overhead.

[0024] In addition, in the centralized training stage, the ground cloud data center aggregates the system state, the local observation of all users, the action and the revenue for policy learning update. In the distributed execution stage, the satellite is deployed as an agent's deployment node, and each agent deployed on the satellite independently decides based on the training strategy only using the local observation, so based on the intelligent algorithm, the efficiency of the task offloading decision is improved. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 is a flowchart of the intelligent task offloading method of the star-ground fusion network provided by the application.

[0026] Figure 2 is a star-ground fusion network architecture diagram for AI inference tasks.

[0027] Figure 3is a schematic diagram of a multi-agent deep reinforcement learning algorithm based on a centralized training and distributed execution framework (CTDE framework). DETAILED DESCRIPTION

[0028] The embodiments of the present application are described in detail below, which are implemented on the premise of the technical solutions of the present application, and detailed implementation manners and specific operation processes are given, but the protection scope of the present application is not limited to the following embodiments.

[0029] The intelligent task offloading method of the star-ground fusion network of the present application is suitable for determining task offloading decisions when the star-ground fusion network provides inference services to users, for example, in the scenario of providing AI inference services for users in remote areas. Figure 2 As shown in the figure, the star-ground fusion network includes at least one user in a remote area, a plurality of low earth orbit (LEO) satellites, and a ground cloud server, and the user in the remote area cannot directly connect to the ground cloud server through the ground network.

[0030] Among them, the star-ground fusion network is a time slot system with the same time slot interval, and the time slot interval is The user accesses the low earth orbit satellite every time slot and requests an inference service of a specified service type, and each user is connected to at most one low earth orbit satellite per time slot. The low earth orbit satellite has edge computing capability and is deployed with a plurality of inference models, and can provide AI inference services for users. Among them, different inference models have different parameter sizes, inference accuracies and computing loads. Each low earth orbit satellite is connected to adjacent low earth orbit satellites through inter-satellite links. In addition, the low earth orbit satellite is connected to the ground cloud data center through a backhaul link. The ground cloud data center can process user requests that are not provided with inference services by the remaining low earth orbit satellites.

[0031] In the star-ground fusion network, the low earth orbit satellite n covers ground users in remote areas and ground cloud data centers, the number of low earth orbit satellites is N, the set of ground users is , the set of low earth orbit satellites is , the set of ground cloud data centers is , the set of time slots is , the time slot interval is , and T is the decision period.

[0032] Different computing nodes (such as low earth orbit satellites and ground clouds) can deploy different types of inference models due to node heterogeneity, which are stored in storage space by default. The set of types of inference models that can be deployed on the low earth orbit satellite and the ground cloud is represented as The set of inference models that can be deployed on the low earth orbit satellite n and the ground cloud is represented as and ,have Among them, the reasoning model The parameter size, inference accuracy, and computational load are respectively expressed as: , and Low-Earth orbit satellite n can connect with adjacent low-Earth orbit satellite n' via inter-satellite links, with a maximum of 4 inter-satellite links for each low-Earth orbit satellite n.

[0033] The space-ground converged network provides inference services to users, specifically including: First, the user sends the inference request of the inference task (such as an AI inference task) to the low-Earth orbit satellite accessed by the user via a wireless link; the low-Earth orbit satellite, acting as a service controller, after determining the task offloading decision, sends a control signaling message to the user and the offloading location of the inference task to prepare for inference. Specifically, the service controller instructs the user on the offloading location of the inference task, and also instructs the offloading location to reserve the corresponding communication, computing, and memory resources; then, the user sends the content of its inference request to the designated offloading location, so that the offloading location (i.e., the computing node) completes the inference computing processing and feeds back the output result to the user.

[0034] like Figure 1 As shown, the intelligent task offloading method for satellite-ground fusion networks of the present invention is applicable to determining task offloading decisions when providing AI inference services to users via satellite-ground fusion networks, specifically including: Step S1: The low-Earth orbit satellite acquires the mission characteristics and network environment information of the inference mission, and establishes mathematical models of inference accuracy, mission latency, and system overhead based on the mission characteristics, network environment information, and mission offloading decisions. The user request includes, but is not limited to, the user ID and the task characteristics of the inference task. The task characteristics of the inference task include, but are not limited to, the amount of task input data, the amount of task output data, the amount of state information data, the upper limit of task latency, and the inference accuracy threshold.

[0035] That is, users In the current time slot Task characteristics of reasoning tasks Represented as a quintuple .in, Indicates the amount of input data for the task. Indicates the amount of data output by the task. This indicates the amount of state information data that the task depends on. Indicates the upper limit of task latency. This represents the inference accuracy threshold.

[0036] The state information data volume reflects the logical association between the user's time series tasks, including the session data volume required to maintain the consistency of the inference, and the task offloading decision that depends on the previous time slot. For example, in a multi-round task, the state information data volume may contain the context of the historical task. Generally, the output data size of the task completion processing is generally much smaller than the input, and the output size is negligible. Unlike traditional tasks, the input data volume and the state information data volume of the inference task dynamically change over time. The task output data volume depends on, but is not limited to, the inference model type, the task complexity, and the inference optimization technique, and the task output data volume cannot be ignored, which is set to be fixed in this embodiment.

[0037] The network environment information includes, but is not limited to, communication and link information (including the link rates of the uplink transmission from the user to the satellite, the downlink transmission from the satellite to the user, the inter-satellite transmission, the downlink transmission from the satellite to the ground cloud, and the uplink transmission from the ground cloud to the satellite), satellite topology and dynamic characteristics (including the satellite moving speed of the low-orbit satellite, the remaining coverage time window of the satellite serving the ground user, and the remaining connection time of the inter-satellite link between the low-orbit satellite and the adjacent satellite), and node resource state (including the calculation amount and the occupied memory resources caused by the inference task of the local, adjacent satellite, and ground cloud server).

[0038] The communication transmission process focuses on the transmission of task data (user input and output result), and the communication and link information mainly includes the relevant parameters of the uplink transmission from the user to the satellite, the downlink transmission from the satellite to the user, the inter-satellite transmission, the downlink transmission from the satellite to the ground cloud, and the uplink transmission from the ground cloud to the satellite. These parameters are derived from the user equipment (providing requests and power through the uplink), the satellite itself measurement (channel gain, noise, orbit parameter), the adjacent satellite (exchanged through the inter-satellite link), and the ground cloud (feedback through the backhaul link), and are acquired and calculated by the low-orbit satellite in real time.

[0039] The spectrum resource, as a system configuration parameter, is used to calculate the link rate and optimize the system overhead, and the parameter is derived from the system preset or the ground control center and provided to the low-orbit satellite through the configuration module or the backhaul link of the satellite. In this embodiment, it is assumed that the spectrum resource is orthogonally allocated to the user.

[0040] Therefore, in the absence of inter-user interference, the user The uplink communication rate of the user to the satellite n is expressed as: , wherein, represents the total bandwidth of the uplink communication link, represents the bandwidth proportion allocated by the satellite n to the user , represents the uplink transmission power, represents the channel gain, It is noise power.

[0041] Sourced from system configuration (satellite storage or ground update); Assigned by satellite (based on user request); Originating from user equipment (reported via control signaling); Measurements are taken via satellite channel (pilot or CSI feedback). Noise is monitored (estimated or measured) by satellite.

[0042] Similarly, satellite n to user The downlink communication rate, the inter-satellite communication rate between satellite n and satellite n', the uplink communication rate between ground cloud g and satellite n, and the downlink communication rate between satellite n and ground cloud g are respectively expressed as: , , and .

[0043] Unlike stationary ground-based communication infrastructure, low-Earth orbit (LEO) satellites maintain continuous high-speed movement. Therefore, the high-speed movement characteristic of LEO satellites is also a part of their satellite topology and dynamic characteristics. Within these characteristics, the coverage window for LEO satellites serving ground users primarily depends on the satellite's orbit, speed, elevation range, and the user's geocentric angle relative to the satellite.

[0044] Assumption For the Earth's radius, is the Kepler constant. For the satellite's orbital altitude, is... The satellite, the satellite's orbital altitude The satellite's moving speed was calculated using data provided by low-Earth orbit satellites. . Indicates the current time slot satellite n relative to user The elevation angle (this parameter comes from uplink feedback), the geocentric angle of the satellite is expressed as... Therefore, the remaining coverage time window Represented as ,in, For the Earth's radius, For the satellite's orbital altitude, For the geocentric angle of the satellite, This represents the satellite's moving speed. If in the current time slot... Satellite n is the user If the access satellites meet the remaining coverage time window requirements. .also, Indicates the current time slot Low earth orbit satellite n inter-satellite link residual connection time with neighboring satellite n'. The inter-satellite link residual connection time is derived from satellite orbit parameters (provided by the navigation system) and neighboring satellite positions (exchanged through ISL communication) and calculated by the low earth orbit satellite based on geometric formulas, the calculation is carried out on the on-board processor.

[0045] Unlike traditional tasks, the computational load and memory resources occupied by inference tasks are determined according to the complexity of the user's inference task and the properties of the model selected in the task offloading decision of the last time slot (including the inference optimization techniques adopted). According to the following, the computational load and memory resources occupied by inference tasks are and wherein, is the computational load factor (FLOPS / byte) of the inference model m, reflecting the required computational intensity of the model, which depends on the model structure and implementation, represents the amount of task input data. The introduced inference optimization techniques include but are not limited to speculative decoding and quantization techniques. Specifically, AI inference usually generates output tokens one by one based on autoregressive mode, and each step needs to wait for the completion of the previous step, which is low in inference efficiency and high in computational resource consumption. Typically, speculative decoding is based on a "draft after verification" mechanism, which uses a draft model to quickly generate a candidate token sequence, and then uses a target model to verify it in parallel. Speculative encoding introduces a lightweight draft model to increase peak memory at the cost of significantly reducing inference computational load, achieving a trade-off between inference speed and output quality. However, the output distribution overlap rate of the draft model and the target model is positively correlated with the parameter size of the draft model. Different parameter sizes of the draft model and the number of candidate tokens generated by the draft model in parallel in each iteration will affect the occupied memory resources and task latency to varying degrees. On the one hand, the memory resources occupied by speculative decoding mainly come from loading two models and the corresponding context at the same time. In order to quantify this effect, let denote the impact of speculative decoding on the memory resources occupied by node inference. On the other hand, the inference latency under speculative decoding mainly comes from the generation of candidate sequences by the "draft model" and the parallel verification by the target model. In addition, model quantization techniques can significantly reduce the occupied memory resources and task latency by encoding the inference model weights, activations, or tensors to a floating-point precision lower than FP16.

[0046] The task offloading decision includes the offloading location and model type selected by the user's inference task. The offloading location for the inference task includes satellite local processing, neighboring satellite on-orbit processing, and ground cloud processing. Satellite local processing is defined as the immediate selection of a suitable inference model and local processing of the task after the user's inference request is received by a low-Earth orbit satellite. Neighboring satellite on-orbit processing is defined as the user's inference request being offloaded to a neighboring satellite after being received by a low-Earth orbit satellite, where the neighboring satellite selects a suitable inference model for processing. Ground cloud processing is defined as the user's inference request being offloaded to a ground cloud via a low-Earth orbit satellite, where a suitable inference model is selected for processing.

[0047] In this embodiment, It is a binary variable representing the task unloading method. These are binary variables representing whether the inference task is processed locally on satellite n, whether it is offloaded to the adjacent satellite n', and whether it is offloaded to the ground cloud server. A binary variable. If satellite n is processed locally, then ;otherwise, If unloaded to adjacent satellite n', then ;otherwise, If uninstalled to a ground-based cloud server ,but ;otherwise, Each task should be offloaded to a compute node, i.e., satisfying: .

[0048] Each user task must select an inference model for computation service on the offload node, which means it must satisfy the following: , A binary variable indicating whether model m was selected for the inference task. , The meanings are whether model m' was selected for the binary variable in the reasoning task and whether model m” was selected for the binary variable in the reasoning task, respectively. If model m” was selected... For reasoning tasks, then ;otherwise, .

[0049] The inference model selected for each user task must ensure that the inference accuracy is not less than the user's inference accuracy requirement, that is, it must meet the following: , also, These represent the current time slots. Can be assigned to users The computing resources of the low-Earth orbit satellites that can be accessed and allocated to users computing resources of the adjacent satellite n' of the user computing resources of the ground cloud g; parameters , , , the inference accuracy of the inference model , and the inference accuracy threshold of the user . The value of the computing resources of the user allocable to the user is updated in real time by the resource monitoring module of each computing node and fed back to the low-orbit satellite for outputting the task offloading decision. The relationship between the computing resources and the computing amount is the computing delay (s) = computing amount (FLOPs) / computing resources (FLOPS / s), so that the computing delay can be derived according to the computing resources and the computing amount.

[0050] If the inference task is processed locally by the satellite, the task delay mainly includes the request waiting delay, the uplink transmission delay of the task input from the user to the satellite, the local computing delay, and the downlink transmission delay of the task output from the satellite to the user. The request waiting delay includes the deployment delay of the inference model corresponding to the user task and the state synchronization delay required by the inference task.

[0051] If the inference task is processed on-orbit by the adjacent satellite, the task delay mainly includes the request waiting delay, the uplink transmission delay of the task input from the user to the satellite, the inter-satellite offloading delay of the task input, the adjacent satellite computing delay, the inter-satellite offloading delay of the task output, and the downlink transmission delay of the task output from the satellite to the user.

[0052] If the inference task is processed by the ground cloud, the task delay mainly includes the request waiting delay, the uplink transmission delay of the task input from the user to the satellite, the downlink transmission delay of the task input from the satellite to the ground cloud, the ground cloud computing delay, the uplink transmission delay of the task output from the ground cloud to the satellite, and the downlink transmission delay of the task output from the satellite to the user.

[0053] In addition, if the required inference model is not currently deployed on the computing node selected by the user, the inference model needs to be deployed from the storage space to the memory space. In addition, if the task offloading decision of the current time slot is not the same computing node as the task offloading decision of the last time slot, the user state information needs to be synchronized from the computing node of the user task of the last time slot to the computing node selected by the user of the current time slot, resulting in a state synchronization delay.

[0054] In the embodiment, the inference task of the user in the current time slot under different offloading modes is analyzed as follows: Satellite Local Processing: After the user's inference request is obtained from the low-Earth orbit satellite, a suitable inference model is immediately selected locally. The task is processed locally. At this point, there is... , In the current time slot users Task latency includes request waiting latency. Task input: Uplink transmission delay from user to satellite Local computing latency and the downlink transmission delay from satellite to user in the mission output The request waiting latency includes the deployment latency and state synchronization latency of the inference model corresponding to the user task. If the required model is not currently deployed on the compute node selected by the user, it needs to be deployed from its storage space to the memory space. If the current user state information is not on the current unloading node, the user state information needs to be synchronized from the user's unloading node in the previous time slot to the unloading node in the current time slot. Therefore, the task latency is... .

[0055] In-orbit processing by neighboring satellites: After user inference requests are acquired by the access satellites, they are offloaded to neighboring satellites. The neighboring satellites then select an appropriate inference model to process the task. At this point, , In the current time slot users Task latency includes request waiting latency. Task input: Uplink transmission delay from user to satellite Inter-satellite offloading delay in task input Neighboring satellite calculation delay Inter-satellite offloading latency of task output and the downlink transmission delay from satellite to user in the mission output Therefore, the task latency is: .

[0056] Ground-based cloud processing: User inference requests are offloaded to the ground-based cloud via satellite access, where a suitable inference model is selected for processing. At this point, , In the current time slot users Task latency includes request waiting latency. Task input: Uplink transmission delay from user to satellite The task input is the downlink transmission delay from the satellite to the ground cloud. Cloud computing latency The task output is the uplink transmission latency from the cloud to the satellite. and downlink transmission delay from satellite to user . Therefore, the task latency is: .

[0057] At this time, the inference task latency of any user in any time slot should not exceed its latency threshold , i.e. it needs to satisfy: .

[0058] The above-mentioned various types of latency are calculated according to the task offloading decision of the low-orbit satellite and the network environment information. For example, the model deployment latency represents the time required to load the model weights from storage into memory / video memory (0 if already in memory). When the "current time slot task offloading decision is not the same computing node as the last time slot task offloading decision", the state synchronization latency will be generated. The state synchronization latency is the communication transmission latency of state information data between the last time slot and the current time slot computing nodes. Similarly, the transmission latency is calculated based on the task characteristics of the user's inference task, through real-time measurement of channel state and bandwidth. The calculation latency is calculated based on the model type, task complexity and node computing power. Various types of latency are calculated by the satellite, with data sources from user uplink, rate from measurement, resources from node feedback, and obtained through link transmission.

[0059] The memory resources occupied by the inference service of low-orbit satellite n should not exceed its upper limit of constraints, i.e. it needs to satisfy: , wherein, represents the upper limit of the memory resources occupied by satellite n for inference service, represents the input data volume of the task, represents the output data volume, represents the state information data volume of the task dependency, represents the parameter size of the inference model , represents the binary variable whether the model is selected for inference task.

[0060] The system overhead incurred by a user's inference task in the current time slot mainly includes model deployment overhead, state synchronization overhead, computational overhead, and communication overhead during task transmission. If the inference task is processed locally on the satellite, the computational overhead is caused by local satellite computation, and the task transmission process includes task input from the user to the satellite and task output from the satellite to the user. If the inference task is processed in orbit on a neighboring satellite, the computational overhead is caused by neighboring satellite computation, and the task transmission process includes task input from the user to the satellite, inter-satellite offloading of task input, inter-satellite offloading of task output, and task output from the satellite to the user. If the inference task is processed on a terrestrial cloud, the computational overhead is caused by terrestrial cloud computing, and the task transmission process includes task input from the user to the satellite, task input from the satellite to the terrestrial cloud, task output from the terrestrial cloud to the satellite, and task output from the satellite to the user.

[0061] The four types of overhead—model deployment overhead, state synchronization overhead, computational overhead, and communication overhead during mission transmission—are all calculated by the satellite after decision-making, calculated as "usage × unit price." "Usage" refers to the amount of data (bytes) or computational load (FLOPs), and "unit price" is the resource price list corresponding to the "usage," provided by the resource provider. Specifically, the "usage" in model deployment overhead is the number of bytes occupied by model weight data; the "usage" in state synchronization overhead is the number of bytes occupied by user state information data; the "usage" in computational overhead is the computational load (FLOPs) consumed by the user mission, depending on the selected inference model type and user input; and the "usage" in communication overhead is the amount of data (bytes) that needs to be transmitted during mission transmission.

[0062] In this embodiment, the user In the current time slot The system overhead caused by inference tasks This mainly includes model deployment overhead. State synchronization overhead Calculation overhead and the communication overhead caused by the task transmission process. Therefore, users In the current time slot The system overhead caused by inference tasks Represented as Total system overhead Represented as .

[0063] Step S2: Based on the mathematical model in Step S1, construct a task offloading optimization problem for inference services.

[0064] In step S2, based on the acquired task characteristics and network environment information of the inference task, and based on the mathematical model, in the decision-making cycle... The task offloading decision is determined, the task offloading decision guarantees the requirements of inference accuracy and task latency of each user, and minimizes the total system overhead as an optimization target, taking the task offloading decision of each user as an optimization variable, to construct a task offloading optimization problem.

[0065] In the task offloading optimization problem model, the constraint conditions include but are not limited to limited service coverage of offloading nodes, limited on-orbit resources of satellites, lower limit of task inference accuracy, upper limit of task latency, and constraint of decision variables.

[0066] The task offloading decision of each user includes the offloading location and model type of the inference task selection of each user in the current time slot. The offloading location of the inference task selection of each user in the current time slot is represented by a set of binary variables 、 、 The model type is represented by a set of binary variables .

[0067] In the embodiment, the task offloading optimization problem is described as: , Wherein, the constraint condition indicates that the user is ensured to be located within the service coverage of the satellite. The constraint condition indicates that each user task must select a computing node. The constraint condition indicates that each user task must select an inference model on the offloading node for calculation. The constraint condition requires that the inference accuracy of each task output is not less than its lower limit. The constraint condition guarantees that the task latency of each user should not exceed its latency constraint. The constraint condition indicates that the memory resource occupied by the inference service of the low-orbit satellite n should not exceed its constraint upper limit. The constraint condition indicates that each offloading location decision is a binary variable. The constraint condition indicates that each inference model selection is a binary variable.

[0068] Step S3: taking each user as an agent and taking the task offloading decision as an action, the task offloading optimization problem is expressed as a partially observable Markov decision process.

[0069] Based on the task offloading optimization problem established in step S2, the task offloading optimization problem is expressed as a partially observable Markov decision process (POMDP).

[0070] The core elements of the locally observed Markov process established in this invention are defined as follows: state space Based on the optimization problem, the offloading location is limited by satellite coverage, network topology, network communication, user needs, and on-board resource status. Therefore, the system status of each time slot includes the task offloading decision of the previous time slot, the task characteristics of the inference task, and network environment information.

[0071] In this embodiment, the system state of each time slot Including task uninstallation decisions for all users in the previous time slot Task characteristics of inference tasks for all users in the current time slot Link speed and the allocation of computing resources between satellite and ground nodes. The system status reflects the impact of the offloading decision on model deployment and state synchronization by incorporating the user offloading decision from the previous time slot.

[0072] Therefore, the current time slot System status Represented as: , in, , , , , in, For users in the previous time slot Task unloading decision, User In the current time slot The task characteristics of the inference task are represented by a quintuple of the amount of input data, the amount of output data, the amount of state information data, the upper limit of task latency, and the inference accuracy threshold. The data is provided by the user equipment in the inference request via the uplink and directly extracted by the satellite. The amount of status information data depends on the task offloading decision of the previous time slot. They are users Uplink communication rate to satellite n, and uplink communication rate from satellite n to user The downlink communication rate, the inter-satellite communication rate from satellite n to satellite n', the uplink communication rate from ground cloud g to satellite n, and the downlink communication rate from satellite n to ground cloud g. These represent the current time slots. Can be assigned to users The computing resources of the low-Earth orbit satellites that can be accessed and allocated to users The computing resources of neighboring satellite n' can be allocated to users computing resources of the ground cloud g.

[0073] Local observation space : Each agent (i.e., user) can only observe its local environment, therefore, the user as an agent the local observation at the current time slot includes the task offloading decision of the last time slot which the agent can directly obtain, the task features of the inference task, and the network environment information. The user as an agent the local observation at the current time slot includes but is not limited to the task offloading decision of the last time slot of the user , the task features of the inference task of the user at the current time slot , the link rate related to the user , and the satellite-ground node resource situation related to the user .

[0074] Therefore, the local observation of the user as an agent at the current time slot is expressed as: , , , wherein, are the uplink communication rate of the user to satellite n, the downlink communication rate of satellite n to the user , the inter-satellite communication rate of satellite n to satellite n’, the uplink communication rate of the ground cloud g to satellite n and the downlink communication rate of satellite n to the ground cloud g, respectively represent the computing resources of the accessible low earth orbit satellite of the user at the current time slot , the computing resources of the adjacent satellite n’ of the user at the current time slot , and the computing resources of the ground cloud g of the user at the current time slot

[0075] . Action space : the definition of the action of the user as an agent at the current time slot at the current time slot ​​​​​​​​​The task offloading decision includes the user's choice of offloading location and model type for the inference task. In other words, the user, as the intelligent agent... In the current time slot action Represented as: ,in, These are binary variables representing whether the inference task is processed locally on satellite n, whether it is offloaded to the adjacent satellite n', and whether it is offloaded to the ground cloud server. binary variables, The binary variable is used to indicate whether model m was selected for the reasoning task.

[0076] Payoff function : Payoff function This includes instantaneous reward functions and cumulative reward functions. The instantaneous reward function refers to the reward function for each user as an agent. The instantaneous gains obtained in each time slot, based on the optimization problem, are for the user. Instantaneous revenue function in the current time slot It is the negative value of the system overhead caused by the user's inference task in the current time slot. Instantaneous revenue function in the current time slot Represented as , For users In the current time slot The system overhead caused by the inference task. The optimization objective is to maximize the cumulative reward function, which is obtained by summing the instantaneous reward functions at different times using a discount rate. The maximization of the cumulative reward function is expressed as: , , It's the discount rate. Let be the instantaneous revenue function for the current time slot. This is the current time slot.

[0077] Step S4: Based on the local observation Markov decision process in Step S3, a multi-agent deep reinforcement learning intelligent algorithm based on a centralized training and distributed execution framework is used to obtain the optimized policy network; each policy network belongs to an agent, takes local observation as input, and takes task offloading decision as action as output. like Figure 3 As shown, in the multi-agent deep reinforcement learning intelligent algorithm based on a centralized training and distributed execution framework, the user is the agent, and a participant-critic architecture is adopted. Here, the participant is the policy network, and each policy network corresponds to a user as an agent (i.e., corresponds to one agent) and is deployed on a low-Earth orbit satellite uniquely connected to that user. The policy network is configured to operate based on the user's... In the current time slot Local observation To select users as intelligent agents In the current time slot action The critic is the value network, deployed on a terrestrial cloud platform. The value network is configured to iteratively update all policy networks to obtain an optimized policy network. The policy networks eventually converge, at which point their parameters reach their optimum, and this optimized policy network is taken as the final policy network. It should be noted that different value networks are used to optimize different policy networks; therefore, the optimal parameters for different policy networks may differ.

[0078] During centralized training, the ground side (such as a ground cloud data center) will aggregate the system state, local observations, actions, and gains of all users as intelligent agents, and use the value network (i.e., critics) for centralized training, so that the policy networks of all participants are iteratively updated.

[0079] During distributed execution, the satellite serves as the deployment node for the user's corresponding agent. The policy network (participant) of each agent on the satellite selects actions based only on local observations and optimized task offloading decisions, without the need for global synchronization, making it suitable for real-time decision-making on the satellite.

[0080] Training multi-agent deep reinforcement learning intelligent algorithms, specifically including: Step A1: Initialize value network parameters and policy network parameters , ; Step A2: Initialize the action and experience replay pool , ; Step A3: Set the number of training rounds EP and the number of time slots T in each training round; Step A4: In the current training round, for each time slot t, each user u is based on user... In the current time slot Local observation and the current policy network Sampling to obtain user In the current time slot action ; Perform actions Afterwards, collect user data. Instantaneous revenue function in the current time slot , Acquire users Local observation in the next time slot and the experience group Store to experience replay pool .

[0081] user In the current time slot Local observation This includes the task offloading decision from the previous time slot, the task characteristics of the inference task, and network environment information, all of which are directly accessible to the agent. This action... The task unloading decision therefore includes the unloading location and model type selected by the user for the inference task.

[0082] Step A5: Repeat step A4 above for different training rounds; Step A6: From the experience replay pool Mini-batch samples are used to train the value network and policy network.

[0083] In step A6, the value network and policy network are trained, specifically including: minimizing the loss function to update the parameters of each value network; and updating the parameters of each policy network based on the policy gradient rule to maximize the cumulative reward.

[0084] When training multi-agent deep reinforcement learning algorithms, the methods for updating the policy network parameters and value network parameters can refer to MADDPG and MAPPO, but are not limited to them. On the one hand, for each user as an agent... Participants will be based on policy network parameters Policy network Local observation Mapping to Action Among them, policy network parameters The policy gradient rule is used to update the policy and maximize cumulative returns. The policy gradient rule is as follows: , in, For policy networks, For policy network parameters, This indicates that the gradient is calculated with respect to the policy network parameters. For the current time slot The system status, For the set of actions in the current time slot t, , For user U in the current time slot The action, For users In the current time slot The action, For users In the current time slot Local observations denotes the advantage function estimate obtained based on the centralized training, which can be simplified as ; denotes the policy network corresponding to the cumulative return function.

[0085] the advantage function estimate obtained based on the centralized training is obtained based on the general advantage estimation, the advantage function estimate obtained based on the centralized training is calculated as follows: , wherein, is the system state of the agent at the time slot , , is the instantaneous return of the agent at the time slot , , is a discount rate, t is the current time slot, l is the remaining time slot in the calculation period, and T is the calculation period, is a GAE coefficient, denotes a trade-off between bias and variance, denotes the cumulative return expectation starting from the system state of the current time slot under the value network parameter .

[0086] On the other hand, the value network parameter is trained by minimizing a loss function. The loss function of the value network parameter is defined as: , wherein, denotes a value learning target, , is the advantage function estimate obtained based on the centralized training, denotes the cumulative return expectation starting from the system state of the current time slot under the value network parameter .

[0087] Step S5: The low-orbit satellite obtains a task offloading decision according to the optimized policy network deployed thereon, the task offloading decision including an offloading location and a model type of the inference task selected by the user, thereby providing inference service for the user.

[0088] wherein each user As an agent, the optimized strategy network corresponding to each agent is deployed to the low-orbit satellite. The optimized strategy network deployed on the low-orbit satellite will collect local observations As input, output action That is, the task offloading decision of each user in each time slot.

[0089] The present application adopts a star-ground fusion network offloading optimization modeling oriented to the characteristics of reasoning tasks. Specifically, the present application constructs a task offloading optimization problem model, which deeply combines the inherent characteristics of reasoning tasks and the scene characteristics of star-ground fusion networks, and makes decisions more in line with actual reasoning business. For reasoning tasks, the model considers the model heterogeneity between nodes, task reasoning accuracy constraints, task state dependency, and the non-negligible reasoning output data size. At the same time, the model combines the scene characteristics of star-ground fusion networks, considers the characteristics of low-orbit satellite resource constraints and high-speed movement, star-ground time-varying topology, and wide-area load imbalance, optimizes the decision of "on-satellite processing" and "ground coordination" to minimize system overhead, and meets the user delay and reasoning accuracy requirements while relieving the bandwidth burden of the star-ground link.

[0090] In addition, the present application expresses the optimization problem as a local observation Markov process. In order to solve the problems of "dimension disaster", "combination explosion", high communication overhead and low robustness faced by the global information method in the star-ground fusion network, the present application expresses the task offloading optimization problem as a local observation Markov process, so that the strategy execution link can only rely on the obtained local observation to make stable decisions, which is in line with the time-varying star-ground topology of the network, dynamic user demand and on-orbit resource constraints. This can avoid the additional communication overhead and delay caused by relying on global information, improve the robustness in dynamic and partial state missing scenarios, and the performance does not drop sharply when new users / satellites are added or fail. Among them, the system state is composed around the reasoning task characteristics and the inherent characteristics of the star-ground fusion network, including but not limited to the task characteristics of the reasoning tasks of all users in the current time slot, the offloading decisions of all users in the last time slot, the star-ground link rate and the star-ground node resource situation; the user local observation is based on the local environment information that the agent can directly obtain, including but not limited to local business information, offloading decisions in the last time slot, current link rate and star-ground node resource situation. The benefit is a cost function representing the minimization of system overhead and the satisfaction of system constraints.

[0091] Furthermore, the present application adopts a multi-agent deep reinforcement learning intelligent algorithm based on a centralized training and distributed execution framework in view of the limited on-orbit resources and the high dynamic characteristics of the satellite-ground integrated network. The algorithm regards each user as an agent, aggregates multi-user information through centralized training to improve the strategy learning quality, and distributes the execution to deploy the satellites as the deployment nodes of the agents, so that the decision can be made with only local observation. This makes the algorithm effectively reduce the online communication and computing burden, balances the stability of strategy learning and deployment feasibility, and thus reduces the system overhead and relieves the link pressure under the premise of meeting the user delay and inference accuracy.

[0092] In summary, the intelligent task offloading method of the satellite-ground integrated network of the present application, on the one hand, the task offloading optimization problem faces the inference task, considers the task characteristics of the inference task, and combines the characteristics of the high-speed movement of the low-orbit satellite of the satellite-ground integrated network and the wide-area load imbalance, and considers the network environment information; on the other hand, the method takes each user as an agent, minimizes the total system overhead as the optimization objective, and proposes a multi-agent deep reinforcement learning algorithm under the constraints of limited on-orbit resources and dynamic service demand, realizes the task offloading decision, and thus solves the task offloading problem of the satellite-ground integrated network facing the ubiquitous inference service demand, can relieve the communication bandwidth burden of the satellite-ground link under the premise of meeting the user delay and inference accuracy demand, and minimize the system overhead. In addition, in the centralized training stage, the ground cloud data center aggregates the system state, the local observation of all users, the action and the income for strategy learning update; in the distributed execution stage, the satellite is deployed as the deployment node of the agent, and each agent deployed on the satellite makes independent decision based on the training strategy only with local observation, so that the efficiency of the task offloading decision is improved based on the intelligent algorithm.

[0093] The above is only a preferred embodiment of the present application, not to limit the scope of the present application, and the above embodiment of the present application can be variously changed. Any simple, equivalent change and modification made according to the content of the claims and description of the present application falls within the scope of protection of the present application. The present application is not described in detail.

Claims

1. A method for intelligent task offloading of a space-ground integrated network, which is suitable for determining a task offloading decision when the space-ground integrated network provides inference services to a user, the space-ground integrated network comprising at least one user, a plurality of low earth orbit satellites and a ground cloud server, characterized in that, Comprise: Step S1: Low-orbit satellite acquires task characteristics and network environment information of inference task, and establishes mathematical model of inference accuracy, task delay, and system overhead according to task characteristics, network environment information, and task offloading decision of inference task; Step S2: Based on the mathematical model of step S1, a task offloading optimization problem for inference service is constructed; Step S3: Each user is taken as an intelligent agent, and the task offloading optimization problem is expressed as a local observation Markov decision process with the task offloading decision as an action; Step S4: Based on the local observation Markov decision process of step S3, a multi-agent deep reinforcement learning intelligent algorithm based on a centralized training and distributed execution framework is used to obtain an optimized strategy network; each strategy network belongs to an intelligent agent, and outputs the task offloading decision with the local observation as input and the task offloading decision as action; Step S5: The low-orbit satellite solves the task offloading decision according to the optimized strategy network deployed thereon, and the task offloading decision includes the offloading position and model type selected by the user for the inference task. 2.The intelligent task offloading method of the satellite-terrestrial converged network according to claim 1, characterized in that, The satellite-ground integrated network provides inference service to the user, specifically including: first, the user sends an inference request of the inference task to the low-orbit satellite accessed by the user through a wireless link; the low-orbit satellite determines the task offloading decision, and then sends a control signaling message for preparing inference to the user and the offloading position of the inference task; then, the user sends the content of the inference request to the specified offloading position, so that the offloading position completes the inference calculation processing, and then feeds back the output result to the user. 3.The intelligent task offloading method of the integrated satellite-terrestrial network according to claim 1, characterized in that, The task characteristics of the inference task include task input data volume, task output data volume, state information data volume, task delay upper limit, and inference accuracy threshold; the network environment information includes communication and link information, satellite topology and dynamic characteristics, and node resource state; The communication and link information includes link rate of uplink transmission from the user to the satellite, downlink transmission from the satellite to the user, inter-satellite transmission, downlink transmission from the satellite to the ground cloud, and uplink transmission from the ground cloud to the satellite; The satellite topology and dynamic characteristics include satellite moving speed of the low-orbit satellite, residual coverage time window of the satellite serving the ground user, and residual connection time of the inter-satellite link of the low-orbit satellite and the adjacent satellite; the node resource state includes calculation amount and occupied memory resource of the inference task of the local, adjacent satellite, and ground cloud server. 4.The intelligent task offloading method of the integrated satellite-terrestrial network according to claim 1, characterized in that, The task offloading decision includes the offloading position and model type selected by the user for the inference task, wherein the offloading position selected by the user for the inference task includes satellite local processing, adjacent satellite on-orbit processing, and ground cloud processing. 5.The intelligent task offloading method of the satellite-terrestrial converged network according to claim 4, characterized in that, If the inference task is processed by the satellite locally, the task delay includes request waiting delay, uplink transmission delay of task input from the user to the satellite, local calculation delay, and downlink transmission delay of task output from the satellite to the user; If the inference task is processed by the adjacent satellite on-orbit, the task delay includes request waiting delay, uplink transmission delay of task input from the user to the satellite, inter-satellite offloading delay of task input, adjacent satellite calculation delay, inter-satellite offloading delay of task output, and downlink transmission delay of task output from the satellite to the user; If the inference task is processed by the ground cloud, the task latency mainly includes the request waiting latency, the uplink transmission latency of the task input from the user to the satellite, the downlink transmission latency of the task input from the satellite to the ground cloud, the ground cloud computing latency, the uplink transmission latency of the task output from the ground cloud to the satellite, and the downlink transmission latency of the task output from the satellite to the user; The system overhead caused by the inference task of the user in the current time slot mainly includes the model deployment overhead, the state synchronization overhead, the computing overhead, and the communication overhead caused by the task transmission process. 6.The intelligent task offloading method of the integrated satellite-terrestrial network according to claim 1, characterized in that, In the step S2, according to the task characteristics of the inference task and the network environment information, a task offloading decision is determined within a decision period based on the mathematical model, the task offloading decision guarantees the requirements of the inference accuracy and the task latency of each user, and minimizes the total system overhead as an optimization target, each user's task offloading decision is taken as an optimization variable, and a task offloading optimization problem is constructed. 7.The intelligent task offloading method of the integrated satellite-terrestrial network according to claim 1, characterized in that, In the task offloading optimization problem model, the constraint conditions include the limited service coverage of the offloading node, the limited on-orbit resources of the satellite, the lower limit of the inference accuracy of the task, the upper limit of the task latency, and the constraint of the decision variable. 8.The intelligent task offloading method of the integrated satellite-terrestrial network according to claim 1, characterized in that, In the local observation Markov process, the system state of each time slot includes the task offloading decision of the last time slot, the task characteristics of the inference task, and the network environment information. a user as an agent in the current time slot partial observation including the task offloading decision of the previous time slot that the agent can directly obtain, the task features of the reasoning task, and the network environment information a user as an agent in the current timeslot the definition of an action for the user in the current timeslot the task offloading decision; revenue function including an instantaneous revenue function and a cumulative revenue function, a user the instantaneous revenue function at the current time slot is the negative value of the system overhead caused by the inference task of the user at the current time slot; the optimization objective is to maximize the cumulative revenue function, which is obtained by summing the instantaneous revenue functions at different time instants after weighting by a discount rate. 9.The intelligent task offloading method of the integrated satellite-terrestrial network according to claim 8, characterized in that, In a multi-agent deep reinforcement learning intelligent algorithm based on a centralized training and distributed execution framework, a participant-critic architecture is adopted; the participant is a policy network, each policy network corresponds to a user as an agent and is deployed on a low earth orbit satellite to which the user is uniquely connected, and the policy network is set to select the action of the user as an agent in the current time slot according to the local observation of the user as an agent in the current time slot In the current time slot The local observation Selects the user as an agent In the current time slot The action ; the critic is a value network deployed on the ground cloud, and the value network is set to iteratively update all policy networks to obtain optimized policy networks. 10.The intelligent task offloading method of the integrated satellite-terrestrial network according to claim 8, characterized in that, When training the multi-agent deep reinforcement learning intelligent algorithm, the policy network parameters are updated based on the policy gradient rule to maximize the cumulative reward. The value network parameters are trained by minimizing the loss function.

Images