A satellite-ground fusion network multi-task unloading method, device, medium and equipment

Abstract

The present invention discloses a satellite-ground fusion network multi-task unloading method, device, medium and equipment, and relates to the field of satellite-ground fusion network technology. The present invention first determines the uplink transmission rate of each ground user when unloading to a low-orbit satellite based on the path loss between each ground user and each low-orbit satellite and the transmission power of each ground user, and then takes the task unloading ratio of each ground user as a variable to construct a communication model of the data transmission rate of the multimodal tasks of each ground user. Based on this, three multimodal learning methods are further proposed, among which, a centralized Actor-Critic algorithm is used to process computationally intensive tasks to improve the data transmission rate, a distributed multi-agent deep deterministic policy gradient is used to process delay-sensitive tasks to reduce network delay, and a quantization-based federated learning algorithm is used to process privacy-preserving tasks to protect privacy data, thereby meeting the needs of different aspects of multimodal tasks and achieving better resource scheduling.

Description

Technical Field

[0001] The present invention relates to the field of satellite-ground fusion network technology, and in particular to a satellite-ground fusion network multi-task unloading method, device, medium and equipment. Background Art

[0002] At present, satellite-ground fusion network is a promising network architecture that can help reduce the load pressure of ground network, provide giant access capability and intensive task offloading function. However, due to the hierarchical, heterogeneous and dynamic three-dimensional characteristics of satellite-ground fusion network, traditional resource management methods are difficult to apply directly. In addition, a large amount of multimodal and multitasking information hinders the improvement of satellite-ground network service quality, and the multimodal information includes time-varying channels and low-orbit satellite positions.

[0003] In the prior art, resource allocation for task offloading is usually assisted by random scheduling, greedy offloading or deep Q-learning (DQN). For the original optimization problem, random scheduling will randomly select the local execution ratio or the remote satellite offloading ratio for each ground user to optimize the data transmission rate; greedy offloading will evenly allocate CPU cycle frequency and transmission power to each ground user for multiple task types; deep Q learning has two structures, current Q network and target Q network, to adapt to dynamic low-orbit satellite orbits and time-varying channel gains, which can process CPU cycle frequency, transmission power and offloading ratio through a centralized method.

[0004] However, random scheduling, which randomly selects the offloading ratio, or greedy offloading, which evenly distributes CPU cycle frequency and transmission power, is difficult to achieve good task offloading resource scheduling. Deep Q learning uses a centralized method to train neural networks, which takes too long to train and will cause huge transmission delays for delay-sensitive tasks. Summary of the invention

[0005] Based on this, it is necessary to provide a satellite-ground fusion network multi-task unloading method, device, medium and equipment to address the above technical problems.

[0006] The present invention adopts the following technical solutions:

[0007] The present invention provides a satellite-ground fusion network multi-task unloading method, which is applied to the satellite-ground fusion network including multiple ground users, multiple low-orbit satellites and a satellite cloud platform; each ground user unloads at least part of the multi-modal tasks generated locally to the low-orbit satellite, and the satellite cloud platform is used to assist the task unloading between the ground user and the low-orbit satellite; the multi-modal tasks include computationally intensive tasks, delay-sensitive tasks and privacy-preserving tasks; the method comprises:

[0008] According to the path loss between each ground user and each low-orbit satellite and the transmission power of each ground user, the expression of the uplink transmission rate when each ground user is unloaded to the low-orbit satellite under the dynamic low-orbit satellite orbit is determined;

[0009] Taking the task offloading ratio of each ground user as a variable, and based on the CPU cycle frequency of each ground user's local processing task and the uplink transmission rate expression when offloading to the low-orbit satellite, the data transmission rate expression of the multimodal task of each ground user is determined;

[0010] For computationally intensive tasks, the satellite cloud platform uses a Markov decision process to centrally determine the actions to be taken under the corresponding network status according to the network status of the satellite-ground fusion network, and optimize the actions with the goal of maximizing the data transmission rate corresponding to the actions; the actions include the CPU cycle frequency, transmission power and task offloading ratio of the ground user;

[0011] For delay-sensitive tasks, a distributed multi-agent deep deterministic policy gradient is used based on the agents deployed on various ground users. The local network environment faced by each ground user is used as its network state, and the data transmission rate after taking the action is used to determine the reward, and the actions of each ground user are optimized in a distributed manner.

[0012] For privacy-preserving tasks, a quantitative model analysis is performed on the local models used by various ground users to handle privacy-preserving tasks, and the quantized local model parameters are uploaded to the satellite cloud platform through each low-orbit satellite. The satellite cloud platform aggregates the quantized local model parameters through federated learning and updates the local models of various ground users.

[0013] The present invention provides a satellite-ground fusion network multi-task unloading device, comprising:

[0014] A satellite-ground fusion network formed by multiple ground users, multiple low-orbit satellites and a satellite cloud platform; each ground user offloads at least part of the multimodal tasks generated locally to the low-orbit satellite, and the satellite cloud platform is used to assist in task offloading between the ground users and the low-orbit satellite; the multimodal tasks include computationally intensive tasks, delay-sensitive tasks and privacy-preserving tasks;

[0015] An unloading rate determination module is used to determine the uplink transmission rate expression when each ground user is unloaded to the low-orbit satellite under the dynamic low-orbit satellite orbit according to the path loss between each ground user and each low-orbit satellite and the transmission power of each ground user;

[0016] The transmission rate determination module is used to determine the data transmission rate expression of the multimodal tasks of each ground user by taking the task offloading ratio of each ground user as a variable and according to the CPU cycle frequency of the local processing tasks of each ground user and the uplink transmission rate expression when offloading to the low-orbit satellite;

[0017] The intensive task processing module is used for computing-intensive tasks. It uses the Markov decision process through the satellite cloud platform to centrally determine the actions to be taken under the corresponding network status according to the network status of the satellite-ground fusion network, and optimizes the actions with the goal of maximizing the data transmission rate corresponding to the actions; the actions include the CPU cycle frequency, transmission power and task offloading ratio of the ground user;

[0018] The delay task processing module is used for delay-sensitive tasks. It uses distributed multi-agent deep deterministic policy gradient based on the agents deployed on various ground users. It uses the local network environment faced by each ground user as its network state and the data transmission rate after taking the action to determine the reward, so as to optimize the actions of each ground user in a distributed manner.

[0019] The privacy task processing module is used to perform quantitative model analysis on the local models used by various ground users to process privacy-preserving tasks, and upload the quantized local model parameters to the satellite cloud platform through each low-orbit satellite, so that the satellite cloud platform aggregates the quantized local model parameters through federated learning and updates the local models of various ground users.

[0020] The present invention provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned satellite-ground fusion network multi-task unloading method is implemented.

[0021] The present invention provides a computer device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned satellite-ground fusion network multi-task unloading method when executing the program.

[0022] At least one of the above technical solutions adopted by the present invention can achieve the following beneficial effects:

[0023] The present invention first determines the time-varying channel gain and the uplink transmission rate of each ground user when unloading to the low-orbit satellite under the dynamic low-orbit satellite orbit according to the path loss between each ground user and each low-orbit satellite and the transmission power of each ground user, and then constructs the data transmission rate expression of the multimodal tasks of each ground user with the task unloading ratio as a variable, so as to optimize the data transmission rate in different ways through three multimodal learning methods based on the requirements of different aspects of the multimodal tasks, among which, the centralized Actor-Critic algorithm is used to process computationally intensive tasks to improve the data transmission rate globally, the distributed multi-agent deep deterministic policy gradient is used to process delay-sensitive tasks, the distributed local optimization method is used to reduce the amount of data processed and reduce network delay, and the quantization-based federated learning algorithm is used to process privacy-preserving tasks to protect privacy data, thereby meeting the requirements of different aspects of multimodal tasks and achieving better resource scheduling. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0025] Figure 1 A schematic flow chart of a satellite-ground fusion network multi-task offloading method provided by the present invention;

[0026] Figure 2 A schematic diagram of a satellite-ground fusion network multi-modal-multi-task scenario provided by the present invention;

[0027] Figure 3 A schematic diagram of a centralized Actor-Critic network framework provided by the present invention;

[0028] Figure 4 A schematic diagram of a distributed multi-agent deep deterministic policy gradient framework provided by the present invention;

[0029] Figure 5 A schematic diagram of a quantitative federated learning framework provided by the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0031] At present, satellite-ground fusion network is a promising network architecture, which can help reduce the load pressure of ground network, provide giant access capability and intensive task offloading function. However, due to the hierarchical, heterogeneous and dynamic three-dimensional characteristics of satellite-ground fusion network, traditional resource management methods are difficult to apply directly. In addition, a large amount of multimodal and multi-task information hinders the service quality of satellite-ground network.

[0032] Therefore, the present invention designs a multi-task fusion computational offloading model to process multimodal network information, which includes time-varying channel gain and dynamic low-orbit satellite orbit, which can effectively improve the data transmission rate and privacy protection level. Furthermore, the present invention proposes three multimodal learning methods: centralized Actor-Critic algorithm, distributed multi-agent deep deterministic policy gradient and quantization-based learning algorithm to process computationally intensive, delay-sensitive and privacy-preserving tasks, which can further optimize the local execution or LEO offloading ratio, CPU cycle frequency and transmission power.

[0033] The technical solutions provided by various embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.

[0034] Figure 1 The present invention is a schematic flow chart of a satellite-ground fusion network multi-task unloading method, which specifically includes the following steps:

[0035] S101: Determine the time-varying channel gain and the uplink transmission rate expression when each ground user is unloaded to the low-orbit satellite under the dynamic low-orbit satellite orbit according to the path loss between each ground user and each low-orbit satellite and the transmission power of each ground user.

[0036] S102: Taking the task offloading ratio of each ground user as a variable, and based on the CPU cycle frequency of each ground user's local processing task and the uplink transmission rate expression when offloading to the low-orbit satellite, determine the data transmission rate expression of the multimodal task of each ground user.

[0037] S103: For computing-intensive tasks, the Markov decision process is adopted through the satellite cloud platform to centrally determine the actions to be taken under the corresponding network status according to the network status of the satellite-ground fusion network, and optimize the actions with the goal of maximizing the data transmission rate corresponding to the actions; the actions include the CPU cycle frequency, transmission power and task offloading ratio of the ground user.

[0038] S104: For delay-sensitive tasks, a distributed multi-agent deep deterministic policy gradient is used based on the agents deployed on various ground users. The local network environment faced by each ground user is used as its network state, and the reward is determined by the data transmission rate after the action is taken, so as to optimize the actions of each ground user in a distributed manner.

[0039] S105: For privacy-preserving tasks, a quantized model analysis is performed on the local models used by various ground users to process privacy-preserving tasks, and the quantized local model parameters are uploaded to the satellite cloud platform via each low-orbit satellite, so that the satellite cloud platform aggregates the quantized local model parameters through federated learning and updates the local models of various ground users.

[0040] Figure 2 FIG. 1 is a schematic diagram of a multi-modal-multi-task scenario of a satellite-ground fusion network in the present invention, Figure 2 It can be seen that multiple low-orbit satellites, satellite cloud platforms and ground users constitute a multi-task processing system, which can achieve effective task arrival, offloading and processing. Among them, each ground user can offload at least part of the multimodal tasks generated locally to the low-orbit satellite, and the satellite cloud platform is used to assist the task offloading between the ground users and the low-orbit satellite. The multimodal tasks may include computationally intensive tasks, delay-sensitive tasks and privacy-preserving tasks.

[0041] Specifically, low-orbit satellites can provide global coverage for ground users, and their tasks are offloaded to multiple low-orbit satellites in proportion to achieve certain performance indicators. The collection of low-orbit satellites is defined as , and then, the corresponding satellite cloud platform will help process the ground user needs of the low-orbit satellite platform, and then train the neural network parameters to speed up the model convergence rate. Assuming the total duration is time slots, which is defined as At the same time, multimodal network information can be represented as channel gain between satellites and ground users, dynamic low-orbit satellite orbital positions, and complex mission types. Based on the corresponding mission requirements, the mission types are divided into computationally intensive, delay-sensitive, and privacy-preserving. The ground user set is represented as .

[0042] In the satellite-ground fusion network, assuming that , each ground user Generate computationally intensive tasks; when , each ground user Generate latency-sensitive tasks; when , each ground user Generate privacy-preserving tasks. Based on the information generated by the above tasks, and ground users The path loss between can be defined as: .

[0043] in, is the path loss between ground user i and low-orbit satellite j, is the carrier frequency, is the speed of light, is the horizontal distance between ground user i and low-orbit satellite j, is the vertical distance between ground user i and low-orbit satellite j, is the additive path loss of the line-of-sight link between ground user i and low-orbit satellite j, is the additive path loss of the non-line-of-sight link between ground user i and low-orbit satellite j, and further, Related to the line-of-sight link, it can be expressed as: .

[0044] in, and It is a constant parameter determined by the dynamic low-orbit satellite. and low-orbit satellites The uplink transmission rate between is expressed as: .

[0045] in, is the transmission power from ground user i to low-orbit satellite j, is the transmission bandwidth, is the channel noise power.

[0046] The number of bits processed locally by the terrestrial user is expressed as: .

[0047] in, The CPU cycle frequency for local processing tasks for each user on the ground, is the interval time between adjacent time slots, It is the number of CPU cycles required for each ground user to process a 1-bit task.

[0048] For ground users , the purpose of the present invention is to achieve optimal task scheduling in a satellite-ground fusion network, which can maximize the number of bits processed. Furthermore, the present invention defines is the ratio of unloaded tasks, where and Represents the corresponding local execution and remote low-orbit satellite offloading ratio. For computationally intensive tasks, due to the larger task capacity, this requires high-speed data transmission. For delay-sensitive tasks, since they are highly sensitive to network delays, this requires low-orbit satellite collaboration to offload tasks and reduce network delays. For delay-sensitive tasks, since they are unwilling to share private data, it is necessary to design an offloading method based on federated learning to protect privacy data. Therefore, the data transmission rate of the above tasks can be expressed as: .

[0049] The joint optimization problem of the above three tasks is expressed as: .

[0050] Among them, the offload ratio of local execution and remote low-orbit satellite execution should be less than 1, and the local CPU cycle frequency Should be less than , transmit power Should be less than , I is the total number of ground users.

[0051] Figure 3 A schematic diagram of a centralized Actor-Critic network framework in the present invention is shown in FIG. Figure 3 As shown, because computationally intensive tasks are insensitive to network delays, the present invention proposes a centralized closed-loop network structure to adapt to dynamic network states, such as channel gain, dynamic low-orbit orbital position, and complex task types. Since the decision set includes discrete and continuous variables, the proposed learning framework needs to handle multiple task types. However, the traditional DQN method can only handle discrete decisions, which makes it difficult to optimize continuous task strategies. Therefore, the present invention needs to introduce a policy-based offloading decision and resource scheduling method. Specifically, the present invention considers the closed-loop network structure as a Markov decision process, in which the network state includes channel gain , dynamic low-orbit satellite position and complex task sets . And, the global state set is represented as: , Generate variables for the i-th ground user's task.

[0052] Next, action set Includes various execution actions, such as CPU cycle frequency , transmit power and the ratio of local execution to LEO execution , , which is expressed as: .

[0053] in, Can be divided into uninstall ratio and Resource Scheduling Unit When the satellite cloud platform interacts with the network environment, this can generate corresponding rewards , is indicated .

[0054] Based on the above predefined states, actions and reward functions, the state-action set It is input into the critic neural network and then generates the corresponding Q function Therefore, the timing difference error is represented as .

[0055] in, is the discount factor. Next, the present invention uses the mean square loss function to update the critic network parameters, which is expressed as: .

[0056] in, is the learning rate of the critic network. And, the present invention updates the actor neural network parameters, which is expressed as: .

[0057] in, is the learning rate of the actor neural network, and m is the number of selected training samples.

[0058] Figure 4 A schematic diagram of a distributed multi-agent deep deterministic policy gradient framework in the present invention is shown in Figure 2. For delay-sensitive tasks, such as Figure 4 As shown in Figure 1, the algorithm framework includes the current actor network, the target actor network, the current critic network, and the target critic network. First, each agent Interact with dynamic network environments and collect private state , then enter to the current actor network. Assuming the current Actor network strategy is , which can generate the corresponding action The framework can then interact with the dynamic network environment and obtain rewards , and transition to the next state At the same time, based on the storage queue obtained , the present invention collects the global storage queue ,in and , is the network state of the I-th agent, is the action taken by the Ith agent. Since multiple agents have reward functions to exchange information, this makes Therefore, the present invention describes the temporal difference update of the current critic network, which is expressed as: .

[0059] in, and are the model parameters corresponding to the target critic network and the target actor network, and, and are the corresponding current critic network and current actor network model parameters. Further, is the discount factor and the policy gradient of the current critic network is expressed as: .

[0060] in, is the learning rate of the current critic network gradient update. Since the gradient update is the deterministic strategy of the current actor network, the loss gradient is expressed as: .

[0061] In order to simplify the parameter updating process, the present invention considers that a larger Q value means a smaller loss function. Therefore, the present invention only needs to add a negative number to the Q value, which is expressed as: .

[0062] Finally, based on the current actor network strategy described above and critic network parameters , whose target critic network and actor network parameters are defined as:

[0063] ,

[0064] .

[0065] in, and is the corresponding parameter update rate and is the target actor network policy.

[0066] Figure 5 Figure 1 is a schematic diagram of a quantitative federated learning framework in the present invention. For privacy-preserving tasks, since multiple ground users are unwilling to share private data, a new learning framework needs to be designed to protect user privacy. Figure 4 As shown, the present invention proposes a quantized federated learning framework to transmit model parameters and protect data privacy. When each ground user receives a corresponding task, the task set can be defined as ,in Each ground user The total number of task blocks. Therefore, the global task set for all ground users is defined as Based on the above task set, the model loss of this federated learning is: .

[0067] in, is a model parameter consisting of d dimensions and is the model calculation loss. Further, the present invention divides the learning process into T rounds and each ground user can receive the global weight parameter from the remote satellite cloud platform Therefore, for any iteration round , the transmitted weight parameter can be expressed as: .

[0068] in, is the initial satellite cloud platform remote server parameter, Is experiencing The model parameters after the step update, is the learning rate of gradient update. Therefore, each ground user transmits the updated weight parameters to the satellite cloud platform, which is expressed as: .

[0069] in, is the corresponding task weight ratio and It is in The weight parameter of the round.

[0070] When transmitting a large number of model parameters to the satellite cloud platform, this may take up a lot of storage space and energy consumption. Compared with integer operations, floating-point operations lead to long computational delays. Therefore, the present invention introduces a quantization strategy to accelerate the model convergence rate. Specifically, the weight parameter transmitted to each ground user is: .

[0071] Among them, the present invention can design a quantization function To the remote server. First, the present invention sorts the weight parameters and designs the minimum value and . Assume that the number of quantization bits is , and its quantization interval is: .

[0072] Among them, the parameter sequence Can be defined as Based on the above parameter sequence, each sequence subscript is represented as: .

[0073] in, When a weight parameter falls into the parameter sequence , the quantization function is expressed as: , .

[0074] Among them, sgn is the sign function, and wp represents the probability. Therefore, the quantization model parameters are expressed as Since each weight parameter needs to be passed The total number of quantization bits is When transmitting model parameters to a remote server, the transformation is calculated as: .

[0075] in, , .

[0076] based on Figure 1The multi-task offloading method for the satellite-ground fusion network shown in the figure, the present invention first determines the uplink transmission rate of each ground user when unloading to the low-orbit satellite according to the path loss between each ground user and each low-orbit satellite and the transmission power of each ground user, and then takes the task offloading ratio of each ground user as a variable to construct a communication model of the data transmission rate of the multimodal tasks of each ground user, based on which three multimodal learning methods are further proposed, among which, the computationally intensive tasks are processed by a centralized Actor-Critic algorithm to improve the data transmission rate, the delay-sensitive tasks are processed by a distributed multi-agent deep deterministic policy gradient to reduce the network delay, and the privacy-preserving tasks are processed by a quantization-based federated learning algorithm to protect privacy data, thereby meeting the needs of different aspects of multimodal tasks and achieving better resource scheduling.

[0077] The present invention designs a multi-task fusion computational offloading model to process multimodal network information, which includes time-varying channel gain and dynamic low-orbit satellite orbit, which can effectively improve the data transmission rate and privacy protection level. Three multimodal learning methods are proposed: centralized Actor-Critic algorithm, distributed multi-agent deep deterministic policy gradient and quantization-based learning algorithm to handle computationally intensive, delay-sensitive and privacy-preserving tasks, which can further optimize the local execution or LEO offloading ratio, CPU cycle frequency and transmission power.

[0078] When applying the satellite-ground fusion network multi-task unloading method provided by the present invention, it is not necessary to Figure 1 The steps are executed in the order shown. The specific execution order of the steps can be determined according to needs, and the present invention does not limit this.

[0079] The above is a satellite-ground fusion network multi-task unloading method provided by one or more embodiments of the present invention. Based on the same idea, the present invention also provides a corresponding satellite-ground fusion network multi-task unloading device, including:

[0080] A satellite-ground fusion network formed by multiple ground users, multiple low-orbit satellites and a satellite cloud platform; each ground user offloads at least part of the multimodal tasks generated locally to the low-orbit satellite, and the satellite cloud platform is used to assist in task offloading between the ground users and the low-orbit satellite; the multimodal tasks include computationally intensive tasks, delay-sensitive tasks and privacy-preserving tasks;

[0081] An unloading rate determination module is used to determine the uplink transmission rate expression when each ground user is unloaded to the low-orbit satellite under the dynamic low-orbit satellite orbit according to the path loss between each ground user and each low-orbit satellite and the transmission power of each ground user;

[0082] The transmission rate determination module is used to determine the data transmission rate expression of the multimodal tasks of each ground user by taking the task offloading ratio of each ground user as a variable and according to the CPU cycle frequency of the local processing tasks of each ground user and the uplink transmission rate expression when offloading to the low-orbit satellite;

[0083] The intensive task processing module is used for computing-intensive tasks. It uses the Markov decision process through the satellite cloud platform to centrally determine the actions to be taken under the corresponding network status according to the network status of the satellite-ground fusion network, and optimizes the actions with the goal of maximizing the data transmission rate corresponding to the actions; the actions include the CPU cycle frequency, transmission power and task offloading ratio of the ground user;

[0084] The delay task processing module is used for delay-sensitive tasks. It uses distributed multi-agent deep deterministic policy gradient based on the agents deployed on various ground users. It uses the local network environment faced by each ground user as its network state and the data transmission rate after taking the action to determine the reward, so as to optimize the actions of each ground user in a distributed manner.

[0085] The privacy task processing module is used to perform quantitative model analysis on the local models used by various ground users to process privacy-preserving tasks, and upload the quantized local model parameters to the satellite cloud platform through each low-orbit satellite, so that the satellite cloud platform aggregates the quantized local model parameters through federated learning and updates the local models of various ground users.

[0086] For the specific definition of the satellite-ground fusion network multi-task unloading device, please refer to the definition of the satellite-ground fusion network multi-task unloading method above, which will not be repeated here. Each module in the above-mentioned satellite-ground fusion network multi-task unloading device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0087] The present invention also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 A multi-task offloading method for satellite-ground fusion network is provided.

[0088] The present invention also provides a computer device. At the hardware level, the computer device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory. Of course, it may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 A multi-task offloading method for satellite-ground fusion network is provided.

[0089] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0090] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present invention.

Claims

1. A satellite-ground fusion network multi-task unloading method, characterized in that: The method is applied to a satellite-ground fusion network including multiple ground users, multiple low-orbit satellites and a satellite cloud platform; each ground user offloads at least part of a locally generated multimodal task to a low-orbit satellite, and the satellite cloud platform is used to assist in task offloading between the ground user and the low-orbit satellite; The multimodal tasks include computationally intensive tasks, delay-sensitive tasks, and privacy-preserving tasks; and the method includes: According to the path loss between each ground user and each low-orbit satellite and the transmission power of each ground user, the expression of the uplink transmission rate when each ground user is unloaded to the low-orbit satellite under the dynamic low-orbit satellite orbit is determined; Taking the task offloading ratio of each ground user as a variable, and based on the CPU cycle frequency of each ground user's local processing task and the uplink transmission rate expression when offloading to the low-orbit satellite, the data transmission rate expression of the multimodal task of each ground user is determined; For computing-intensive tasks, the satellite cloud platform uses a Markov decision process to centrally determine the actions to be taken under the corresponding network status according to the network status of the satellite-ground fusion network, and optimize the actions with the goal of maximizing the data transmission rate corresponding to the actions; the actions include the CPU cycle frequency, transmission power and task offloading ratio of the ground user; For delay-sensitive tasks, a distributed multi-agent deep deterministic policy gradient is used based on the agents deployed on various ground users. The local network environment faced by each ground user is used as the network state, and the reward is determined by the data transmission rate after taking the action, so as to optimize the actions of each ground user in a distributed manner. For privacy-preserving tasks, a quantized model analysis is performed on the local models of each ground user, and the quantized local model parameters are uploaded to the satellite cloud platform through each low-orbit satellite, so that the satellite cloud platform aggregates the quantized local model parameters through federated learning and updates the local models of each ground user; the local model is used to optimize the CPU cycle frequency, transmission power and task offloading ratio of the ground user when performing privacy-preserving tasks; The determining of the time-varying channel gain and the expression of the uplink transmission rate when each ground user is unloaded to the low-orbit satellite under the dynamic low-orbit satellite orbit according to the path loss between each ground user and each low-orbit satellite and the transmission power of each ground user specifically includes: The path loss between each ground user and each low-orbit satellite is determined by the following formula: The uplink transmission rate of each ground user when unloading to the low-orbit satellite is determined by the following formula based on the path loss between each ground user and each low-orbit satellite and the transmission power of each ground user: in, is the path loss between ground user i and low-orbit satellite j, f c is the carrier frequency, c is the speed of light, m i,j is the horizontal distance between ground user i and low-orbit satellite j, n i,j is the vertical distance between ground user i and low-orbit satellite j, is the additive path loss of the line-of-sight link between ground user i and low-orbit satellite j, is the additive path loss of the non-line-of-sight link between ground user i and low-orbit satellite j, x1 and x2 are constant parameters determined by the low-orbit satellite, and R i,j is the uplink transmission rate when ground user i is unloaded to low-orbit satellite j, B i,j is the transmission bandwidth between ground user i and low-orbit satellite j, r i,j is the transmission power from ground user i to low-orbit satellite j, σ 2 is the channel noise power; The data transmission rate of the multimodal tasks of each ground user is expressed as: Among them, S i,j is the data transmission rate from ground user i to low-orbit satellite j, s is the overall mission generation variable, α i is the local execution ratio of ground user i, D i is the number of bits processed by the local user, τ is the interval between adjacent time slots, is the offloading ratio of ground user i to low-orbit satellite j, f i t The CPU cycle frequency for local processing tasks for each user on the ground, The number of CPU cycles required to process a 1-bit task for a ground user.

2. The satellite-ground fusion network multi-task unloading method according to claim 1, characterized in that: The method of centrally determining the actions to be taken under the corresponding network state according to the network state of the satellite-ground fusion network, and optimizing the actions with the goal of maximizing the data transmission rate corresponding to the actions, specifically includes: The path loss between each ground user and each low-orbit satellite, the dynamic position of the low-orbit satellite and the set of computationally intensive tasks of each ground user are taken as the network status of the satellite-ground fusion network; The action to be taken is determined according to the network status of the satellite-ground fusion network through the action neural network, and the value function of the action under the network status is determined according to the network status and action through the value neural network. The time difference error used to update the value neural network is determined according to the value function through the following formula: δ=R t +γQ(S t+1 ,A t+1 )-Q(S t ,A t ), The parameters of the value neural network are updated according to the temporal difference error using the following formula: The parameters of the action neural network are updated according to the value function using the following formula: Among them, δ is the temporal difference error used to update the value neural network, γ is the discount factor, S t is the network status of the satellite-ground fusion network at time t, A t In the network state S t Action taken, S t+1 The network status S t Action A t The network status at time t+1 later, A t+1 is the network status S at time t+1 t+1 The action taken, Q(S t ,A t ) is in network state S t Action taken A t The value function, R t In the network state S t Action taken A t Rewards, i Generate variables for the task of the i-th ground user, I is the total number of ground users, α i is the local execution ratio of ground user i, D i The number of bits processed for local users, is the unloading ratio of ground user i to low-orbit satellite j, R i,j is the uplink transmission rate when ground user i is offloaded to low-orbit satellite j, w is the parameter of the value neural network, α is the learning rate of the value neural network, θ is the parameter of the action neural network, β is the learning rate of the action neural network, and m is the number of selected training samples.

3. The satellite-ground fusion network multi-task unloading method according to claim 1, characterized in that: The distributed multi-agent deep deterministic policy gradient is adopted based on the agents deployed on various ground users, and the local network environment faced by each ground user is used as its network state, and the data transmission rate after taking the action is used to determine the reward, and the actions of each ground user are optimized in a distributed manner, specifically including: Based on the agents deployed on each ground user, a distributed multi-agent deep deterministic policy gradient is adopted. For each ground user deployed agent, the agent interacts with the dynamic network environment and collects the network status S t , S t Input to the current actor network to get the corresponding action A t ; According to the agent, take action A t Determine the corresponding reward R t ; The current critic network is updated with the following temporal difference: Δ i =R t +εQ t ' arget (S t ' otal ,A t ' otal |w t ' arget )-Q current (S total ,A total w current ), The current policy gradient of the critic network is: The loss gradient of the current actor network is: The target critic network and actor network parameters are: w t ' arget =νw current +(1-ν)w t ' arget , Among them, Q t ' arget is the model parameter corresponding to the target critic network, w t ' arget is the model parameter corresponding to the target actor network, Q current is the model parameter corresponding to the current critic network, w current is the model parameter corresponding to the current actor network, ε is the discount factor, is the network state of the I-th agent, is the action taken by the I-th agent, λ is the learning rate of the current critic network gradient update, π1(A t |S t ) is the current actor network strategy, π2(A t '|S t ') is the target actor network strategy, ν is the parameter update rate corresponding to the target critic network, is the parameter update rate corresponding to the actor network.

4. A satellite-ground fusion network multi-task unloading device, characterized in that: include: A satellite-ground fusion network formed by multiple ground users, multiple low-orbit satellites and satellite cloud platforms; Each ground user offloads at least part of the locally generated multimodal tasks to the low-orbit satellite, and the satellite cloud platform is used to assist the task offloading between the ground users and the low-orbit satellite; the multimodal tasks include computationally intensive tasks, delay-sensitive tasks, and privacy-preserving tasks; The unloading rate determination module is used to determine the path loss between each ground user and each low-orbit satellite by the following formula: The uplink transmission rate of each ground user when unloading to the low-orbit satellite is determined by the following formula based on the path loss between each ground user and each low-orbit satellite and the transmission power of each ground user: The transmission rate determination module is used to determine the data transmission rate expression of the multimodal tasks of each ground user based on the task offloading ratio of each ground user as a variable and the CPU cycle frequency of the local processing tasks of each ground user and the uplink transmission rate expression when unloading to the low-orbit satellite: The intensive task processing module is used for computing-intensive tasks. It uses the Markov decision process through the satellite cloud platform to centrally determine the actions to be taken under the corresponding network status according to the network status of the satellite-ground fusion network, and optimizes the actions with the goal of maximizing the data transmission rate corresponding to the actions; the actions include the CPU cycle frequency, transmission power and task offloading ratio of the ground user; The delay task processing module is used for delay-sensitive tasks. It uses distributed multi-agent deep deterministic policy gradient based on the agents deployed on various ground users. It uses the local network environment faced by each ground user as its network state and the data transmission rate after taking the action to determine the reward, so as to optimize the actions of each ground user in a distributed manner. The privacy task processing module is used to perform quantitative model analysis on the local models used by various ground users to process privacy-preserving tasks, and upload the quantified local model parameters to the satellite cloud platform through each low-orbit satellite, so that the satellite cloud platform aggregates the quantified local model parameters through federated learning and updates the local models of various ground users; in, is the path loss between ground user i and low-orbit satellite j, f c is the carrier frequency, c is the speed of light, m i,j is the horizontal distance between ground user i and low-orbit satellite j, n i,j is the vertical distance between ground user i and low-orbit satellite j, is the additive path loss of the line-of-sight link between ground user i and low-orbit satellite j, is the additive path loss of the non-line-of-sight link between ground user i and low-orbit satellite j, x1 and x2 are constant parameters determined by the low-orbit satellite, B i,j is the transmission bandwidth between ground user i and low-orbit satellite j, r i,j is the transmission power from ground user i to low-orbit satellite j, σ 2 is the channel noise power, S i,j is the data transmission rate from ground user i to low-orbit satellite j, s is the overall mission generation variable, α i is the local execution ratio of ground user i, D i is the number of bits processed by the local user, τ is the interval between adjacent time slots, is the unloading ratio of ground user i to low-orbit satellite j, R i,j is the uplink transmission rate when ground user i is unloaded to low-orbit satellite j, f i t The CPU cycle frequency for local processing tasks for each user on the ground, The number of CPU cycles required to process a 1-bit task for a ground user.

5. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 3 is implemented.

6. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 3 is implemented.

Images