A satellite-ground collaborative unmanned aerial vehicle network computing offloading method and system

Abstract

The present invention relates to the field of UAV technology, and specifically discloses a satellite-ground coordinated UAV network computing offloading method and system, the method comprising: establishing a satellite-ground coordinated network including ground base stations, satellites, and UAVs, the satellites obtaining global information of the satellite-ground coordinated network through ground base stations, and sending computing tasks to UAVs; the UAVs segmenting the captured video data and sending them to corresponding satellites for analysis; formulating a satellite offloading information optimization strategy based on a proximal strategy optimization algorithm, and establishing a training model; training satellites according to the training model to determine the optimal strategy for satellite offloading information. The satellite-ground coordinated UAV network computing offloading method and system proposed in the present application can optimize the computing task allocation strategy of UAVs, including the selection of offloading satellites and the selection of offloading ratios, thereby achieving the effect of reducing the maximum delay of the system.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle technology, and in particular to a satellite-ground coordinated unmanned aerial vehicle network computing offloading method and system. Background Art

[0002] With the rapid development of Unmanned Aerial Vehicle (UAV) technology, drones are playing an increasingly important role in post-disaster rescue, geological exploration, agriculture, military and other fields. Due to the wide application scenarios and huge development potential of drones, it has also become a research hotspot for scholars at home and abroad. With the development of artificial intelligence, video analysis has been widely used in drone services, such as motion capture and human recognition in disaster areas. However, due to limited energy and computing resources, drones cannot perform neural network training that requires a lot of computing resources.

[0003] With the development of industrial manufacturing and launch technology, the cost of satellite network deployment has gradually decreased, becoming a hot topic in the research of the sixth generation (6G) network. Currently, many satellite network operators such as SpaceX and OneWeb deploy a large number of low-Earth orbit satellites to different orbital altitudes to achieve global coverage. Low Earth Orbit (LEO) satellites constitute the LEO network. While providing low-cost, high-speed communication services, they also have stronger computing and storage capabilities, and have significant advantages in deployment flexibility, cost-effectiveness, maintenance performance, and global coverage.

[0004] Based on the above situation, it is considered to comprehensively utilize satellites and ground collaboration to assist UAVs in communication and computing, so as to improve the computing performance of video analysis and reduce transmission delay. Some scholars have established a ground-satellite-UAV satellite-ground collaborative network, using the wide coverage of satellites to provide data offloading services for UAVs within the coverage area. However, the introduction of low-orbit satellites has greatly increased the complexity of the communication network while improving the transmission and computing performance of UAVs. To this end, this paper proposes a network communication resource scheduling method based on the multi-agent proximal gradient algorithm (MAPPO) for the satellite-ground collaborative network, which effectively reduces the data processing delay in the UAV edge computing scenario assisted by low-orbit satellites. Summary of the invention

[0005] To achieve the purpose of the present invention, the present application provides a satellite-ground collaborative UAV network computing offloading method, comprising:

[0006] Step S1: establishing a satellite-ground collaborative network including ground base stations, satellites, and unmanned aerial vehicles. The satellite obtains global information of the satellite-ground collaborative network through the ground base station and sends computing tasks to the unmanned aerial vehicle.

[0007] Step S2: The drone segments the captured video data and sends it to corresponding satellites for analysis;

[0008] Step S3: formulating a satellite unloading information optimization strategy based on a proximal strategy optimization algorithm and establishing a training model;

[0009] The establishment of the training model includes: treating each satellite as an intelligent agent that makes independent unloading decisions, and determining the satellite's state, action, and reward mechanism based on the satellite's position and channel noise power;

[0010] Step S4: training the satellite according to the training model to determine the optimal strategy for satellite unloading information.

[0011] In some specific embodiments, step S3 further includes: the training model represents the current strategy through parameters, and then uses two neural networks to update the parameters according to the following formula:

[0012] π θ (a|s)=P(a|s,θ)

[0013] In the formula, π θ (a|s) represents the current strategy, θ represents the parameter, a represents the action of the satellite, and s represents the state of the satellite.

[0014] In some specific embodiments, in step S3, the state of the satellite includes the position of the satellite and the channel noise power, and is determined according to the following formula:

[0015]

[0016] Where, X m (t) represents the position of the satellite, represents the channel noise power of the satellite, S(t) represents the state function of the satellite, N represents the number of satellites, M represents the number of drones, n represents the number of satellites, and m represents the number of drones.

[0017] In some specific embodiments, in step S3, the action of the satellite is determined according to the network status, and the action is composed of the selected drone and the corresponding data offloading ratio, and the action is expressed as:

[0018]

[0019] In the formula, a m (t) represents the action function, L m,n (t) represents the task offloading decision variable between UAV m and satellite n, ρ m,n (t) represents the ratio of unloading tasks from UAV m to satellite n;

[0020] In some specific embodiments, in step S3, the satellite's reward mechanism determines the selection data according to the action ratio of the drone, sends the data to the corresponding satellite, and then determines the reward according to the delay of the drone:

[0021] R(t)=-T m (t)

[0022] Where R(t) represents the reward function, T m (t) represents the time delay of the UAV.

[0023] In some specific embodiments, step S4 includes:

[0024] Step S41: Initialize the network state at the beginning of each training;

[0025] Step S42: In each training time slot, collect the status information of each satellite, including channel noise power and satellite position status;

[0026] Step S43: Each satellite selects an action according to the local state and current strategy to provide video analysis services to ground users;

[0027] Step S44: After the satellite selects an action, the satellite is given a current instant reward, and the next state of the satellite is observed, wherein the state transition, the selected action and the reward of each satellite are stored in the training model in the form of a tuple;

[0028] Step S45: When updating the network, randomly select a small batch of samples and update them according to the loss function;

[0029] Step S46: Repeat steps S41-S45 to allow the satellites to repeat learning and training until the optimal strategy for all satellites is determined.

[0030] To achieve the same invention purpose, the present application also provides a satellite-ground collaborative drone network computing offloading system, including:

[0031] Information acquisition module: used to establish a satellite-ground collaborative network including ground base stations, satellites, and drones. The satellite obtains the global information of the satellite-ground collaborative network through the ground base station and sends computing tasks to the drone.

[0032] Data segmentation module: used for the drone to segment the captured video data and send it to the corresponding satellite for analysis;

[0033] Strategy formulation module: used to formulate satellite unloading information optimization strategy and establish training model based on proximal strategy optimization algorithm;

[0034] The establishment of the training model includes: treating each satellite as an intelligent agent that makes independent unloading decisions, and determining the satellite's state, action, and reward mechanism based on the satellite's position and channel noise power;

[0035] Satellite training module: used to train satellites according to the training model to determine the optimal strategy for satellite unloading information.

[0036] In some specific embodiments, the training model represents the current strategy through parameters, and then uses two neural networks to update the parameters according to the following formula:

[0037] π θ (a|s)=P(a|s,θ)

[0038] In the formula, π θ (a|s) represents the current strategy, θ represents the parameter, a represents the action of the satellite, and s represents the state of the satellite.

[0039] In some specific embodiments, the state of the satellite includes the position of the satellite and the channel noise power, and is determined according to the following formula:

[0040]

[0041] Where, X m (t) represents the position of the satellite, represents the channel noise power of the satellite, S(t) represents the state function of the satellite, N represents the number of satellites, M represents the number of drones, n represents the number of satellites, and m represents the number of drones;

[0042] The action of the satellite is determined according to the network status. The action consists of the selected drone and the corresponding data offloading ratio. The action is expressed as:

[0043]

[0044] In the formula, a m (t) represents the action function, L m,n (t) represents the task offloading decision variable between UAV m and satellite n, ρ m,n (t) represents the ratio of unloading tasks from UAV m to satellite n;

[0045] The satellite's reward mechanism determines the selection data based on the drone's action ratio, sends the data to the corresponding satellite, and then determines the reward based on the drone's latency:

[0046] R(t)=-T m (t)

[0047] Where R(t) represents the reward function, T m(t) represents the time delay of the UAV.

[0048] In some specific embodiments, the satellite training module is used to perform the following training steps:

[0049] Step S41: Initialize the network state at the beginning of each training;

[0050] Step S42: In each training time slot, collect the status information of each satellite, including channel noise power and satellite position status;

[0051] Step S43: Each satellite selects an action according to the local state and current strategy to provide video analysis services to ground users;

[0052] Step S44: After the satellite selects an action, the satellite is given a current instant reward, and the next state of the satellite is observed, wherein the state transition, the selected action and the reward of each satellite are stored in the training model in the form of a tuple;

[0053] Step S45: When updating the network, randomly select a small batch of samples and update them according to the loss function;

[0054] Step S46: Repeat steps S41-S45 to allow the satellites to repeat learning and training until the optimal strategy for all satellites is determined.

[0055] Beneficial effects of the above technical solution:

[0056] The satellite-ground collaborative UAV network computing offloading method and system proposed in this application can optimize the computing task allocation strategy of the UAV, including the selection of offloading satellites and the selection of offloading ratios, thereby achieving the effect of reducing the maximum delay of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0058] Figure 1 A schematic diagram of a flow chart of an embodiment provided for an embodiment of the present invention;

[0059] Figure 2 A schematic diagram of the structure of an embodiment provided for one embodiment of the present invention;

[0060] Figure 3 A schematic diagram of a system model of an embodiment of the present invention provided for one embodiment of the present invention;

[0061] Figure 4 A schematic diagram of a PPO algorithm framework of an embodiment of the present invention provided for one embodiment of the present invention;

[0062] Figure 5 A schematic diagram of the positions of a drone and a satellite according to an embodiment of the present invention provided for one embodiment of the present invention;

[0063] Figure 6-Figure 8 A schematic diagram of the convergence performance of the algorithm in Embodiment 3 of the present invention at different learning rates provided as an embodiment of the present invention;

[0064] Fig. 9 A schematic diagram comparing the convergence performance of the algorithm in Embodiment 3 of the present invention at different learning rates provided for one embodiment of the present invention. DETAILED DESCRIPTION

[0065] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0066] Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar symbols throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0067] Embodiment 1

[0068] An embodiment of the present invention provides a satellite-ground collaborative UAV network computing offloading method, referring to Figure 1 As shown, including:

[0069] Step S1: Establish a satellite-ground collaborative network including ground base stations, satellites, and unmanned aerial vehicles. The satellite obtains global information of the satellite-ground collaborative network through the ground base station and sends computing tasks to the unmanned aerial vehicle.

[0070] In a specific embodiment of the present invention, since the drone and the satellite are always in motion, that is, the position of the node in the satellite-ground collaborative network is constantly changing, this paper uses a time slice method to process the changing network. It is divided into multiple time slots according to time, each time slot is a time slice, and it is assumed that the node position in each time slice remains fixed. Due to the limited energy and computing resources of the drone, it is impossible to complete the computationally intensive video analysis task locally, so the drone unloads the video to the satellite server for processing. Further, taking advantage of the low-Earth orbit satellite network, multiple satellites can collaboratively serve a drone to achieve faster data processing. Therefore, in the same time slot, a drone can connect to multiple satellites, while a satellite can only serve one drone at the same time. The drone can split the task and unload it to different satellites for processing. Specifically, the low-orbit satellite informs the drone of its available computing resources through the forward link; each drone generates the corresponding data block based on the received satellite information and sends it to different satellites for processing; finally, the low-orbit satellite sends the processed results to the ground base station through the reverse link.

[0071] Step S2: The drone segments the captured video data and sends it to the corresponding satellite for analysis.

[0072] Reference Figure 3 As shown in the figure, a ground-satellite-UAV satellite-ground collaborative network is established. M and N represent the set of UAVs and LEO satellites respectively. Assume that the satellites meet Drones meet

[0073] Assume that the channel transmission bandwidth of each satellite is w n Since there are no obstacles between the satellite and the drone, assuming that the link between each satellite and the drone is a line-of-sight (LOS) link, as time goes by, the satellite may gradually leave the drone's airspace, and then it is necessary to find a new satellite to supplement it. Since the amount of processed downlink data is generally much smaller than the amount of uplink data, the time required for processed data transmission is much less than the time required for calculation and unloading.

[0074] In time slot t, the link attenuation between satellite n and drone m is the free propagation loss. The free space transmission loss can be expressed as:

[0075]

[0076] Among them, d m,n represents the distance between UAV m and satellite n. At this time, the channel gain can be expressed as:

[0077]

[0078] Among them, G m , G Rn represents the transmitting antenna gain of UAV m and the receiving antenna gain of satellite n. Small-scale fading represents the noise power between the UAV and satellite channel at time slot t.

[0079] Then the channel capacity between UAV m and satellite n is:

[0080]

[0081] Among them, P m,n represents the transmission power from UAV m to satellite n. m,n} M′N is a binary matrix, called the drone-satellite association matrix, which is used to characterize the link connectivity between LEO satellite n and drone m. If L m,n =1 indicates that satellite n establishes a data link with UAV m, otherwise no data link is established.

[0082] If the channel transmission bandwidth of each satellite is w n Since there are no obstacles between the satellite and the drone, it is assumed that the link between each satellite and the drone is a line-of-sight link (LOS). As time goes by, the satellite may gradually leave the drone's airspace, and then it is necessary to find a new satellite to supplement it. Since the amount of processed downlink data is generally much smaller than the amount of uplink data, the time required for processed data transmission is much smaller than the time required for calculation and unloading, so the delay of data return is ignored in this article.

[0083] In time slot t, the link attenuation between satellite n and drone m is the free propagation loss. The free space transmission loss can be expressed as:

[0084]

[0085] Among them, d m,n represents the distance between UAV m and satellite n. At this time, the channel gain can be expressed as:

[0086]

[0087] Among them, G m , G Rn represents the transmitting antenna gain of UAV m and the receiving antenna gain of satellite n. Small-scale fading represents the noise power between the UAV and satellite channel at time slot t.

[0088] Then the channel capacity between UAV m and satellite n is:

[0089]

[0090] Where P m,n represents the transmission power from UAV m to satellite n. m,n} M′N is a binary matrix, called the drone-satellite association matrix, which is used to characterize the link connectivity between LEO satellite n and drone m. If L m,n =1 indicates that satellite n establishes a data link with UAV m, otherwise no data link is established.

[0091] Each satellite obtains global information of the network through the ground base station and then sends computing tasks to the drone. The drone divides the data and sends it to the corresponding satellite. Considering the movement of low-orbit satellites, the network is constantly changing. Considering the changes in the channel, the noise power also changes over time, and the offloading problem is modeled as a DMDP problem.

[0092] Step S3: Formulate a satellite offloading information optimization strategy based on the proximal strategy optimization algorithm and establish a training model.

[0093] In a specific embodiment of the present invention, step S3 includes: considering each satellite as an intelligent agent that makes unloading decisions independently, and determining the state, action and reward mechanism of the satellite according to the position of the satellite and the channel noise power.

[0094] In a specific embodiment of the present invention, step S3 further includes: the training model represents the current strategy through parameters, and then uses two neural networks to update the parameters according to the following formula:

[0095] π θ (a|s)=P(a|s,θ)

[0096] In the formula, π θ (a|s) represents the current strategy, θ represents the parameter, a represents the action of the satellite, and s represents the state of the satellite.

[0097] In a specific embodiment of the present invention, each satellite is an intelligent agent that makes unloading decisions independently, and its state S, action A, and reward R are respectively expressed as follows:

[0098] The state of the satellite includes the satellite's position and the channel noise power and is determined according to the following formula:

[0099]

[0100] Where, X m (t) represents the position of the satellite, represents the channel noise power of the satellite, S(t) represents the state function of the satellite, N represents the number of satellites, M represents the number of drones, n represents the number of satellites, and m represents the number of drones;

[0101] The action of the satellite is determined according to the network status. The action consists of the selected drone and the corresponding data offloading ratio. The action is expressed as:

[0102]

[0103] In the formula, a m (t) represents the action function, L m,n (t) represents the task offloading decision variable between UAV m and satellite n, ρ m,n (t) represents the ratio of unloading tasks from UAV m to satellite n;

[0104] The satellite's reward mechanism determines the selected data based on the drone's action ratio, sends the data to the corresponding satellite, and then determines the reward based on the drone's latency:

[0105] R(t)=-T m (t)

[0106] Where R(t) represents the reward function, T m (t) represents the time delay of the UAV.

[0107] Specifically, the Proximal Policy Optimization (PPO) algorithm is a deep reinforcement learning algorithm based on the policy gradient algorithm. Figure 4 As shown. The PPO algorithm is a policy optimization algorithm based on the Actor-Critic architecture, which uses a set of parameters θ to represent the current policy π θ (a|s), and then use two neural networks to update the parameter θ. The strategy updated by the algorithm is an action distribution function based on the parameter θ, which can be expressed as:

[0108] π θ (a|s)=P(a|s;θ)

[0109] In this algorithm framework, the Actor network generates a set of action samples, and then uses the advantage function A(s,a) to calculate the advantage of the current action relative to the average action, so as to help the Agent better know which actions are currently advantageous. The advantage function is usually defined by the following formula:

[0110] A(s,a)=Q(s,a)-V(s)

[0111] Among them, Q(s,a) is the expected return of taking action a in state s, and V(s) refers to the expected return of following the current strategy in state s. Using the time difference method to solve Q(s,a), Q(s,a) is expressed as: Q(s t ,a)=Rt+1 +L+γ n-1 R t+n +γ n V(S t+n )

[0112] The advantage function A(s,a) is expressed as:

[0113] A(s,a)=R t +γV π (S t+1 )-V π (S t )

[0114] If A(s,a) is positive, it means that taking action a is worse than the current strategy; if it is negative, it means that taking action a is not as good as the current strategy. The Actor network updates its own parameters based on the advantage function A(s,a).

[0115] The Critic network has no strategy to maintain. Its strategy is obtained through the current value function, and the reward function value obtained through the interaction between the value function and the current strategy is used to optimize the parameters of the Critic network. Its loss function is:

[0116]

[0117] Where γ is the discount factor, R is the reward for an action in a certain state, and V θ (S t ) means in state S t The value expectations.

[0118] For the strategy optimization algorithm, the goal is to select a strategy that allows it to take actions with the maximum expected return. Therefore, the maximization objective function J(θ) can be expressed as:

[0119]

[0120] So the parameter optimization strategy of the policy network is obtained as follows:

[0121]

[0122] Step S4: training the satellite according to the training model to determine the optimal strategy for satellite unloading information.

[0123] In a specific embodiment of the present invention, step S4 includes:

[0124] Step S41: At the beginning of each training, the network state is initialized;

[0125] Step S42: In each training time slot, collect the status information of each satellite, including channel noise power and satellite position status;

[0126] Step S43: Each satellite selects an action according to the local state and current strategy to provide video analysis services to ground users;

[0127] Step S44: After selecting the action, each satellite will obtain the current instant reward and observe the next state. Each satellite stores its state transition, selected action and reward in the training model in the form of a tuple;

[0128] Step S45: When updating the network, randomly select a small batch of samples and update them according to the loss function;

[0129] Step S46: Repeat steps S41-S45 to allow the satellites to repeat learning and training until the optimal strategy for all satellites is determined.

[0130] Specifically, a MAPPO algorithm is proposed under a centralized training and decentralized execution framework by using collaborative training of multiple agents. Global information is used to train the independent strategy of each agent. In the proposed algorithm framework, it is assumed that there are n agents in total, and all agents share a reward function. Each agent i is based on the current strategy. Input the current environment status information and output an action a i , then the environment is based on all actions a=[a1,a2,…,a n ]Output reward r.

[0131] Based on the basic framework of the PPO algorithm in the previous section, each agent obtains a new action strategy based on the reward function and the critic network update

[0132] Based on the MAPPO algorithm, extending it to the satellite-ground collaborative network can effectively solve the problem of collaborative computing offloading. Each low-orbit satellite acts as an intelligent agent, aiming to make effective decisions to increase the average reward.

[0133] From the perspective of improving network throughput, an on-board resource allocation algorithm based on the proximal strategy optimization algorithm is proposed. Through dynamic intelligent decision-making of link resources, dynamic optimization and adjustment of links are achieved.

[0134] Specifically, the training of MAPPO consists of several segments, each of which contains several steps. At the beginning of each training, the network state is initialized. Then, in each training time slot, the state information of each LEO satellite is collected, including the channel noise power and the satellite position state. Each LEO satellite selects an action according to the observed local state according to the current strategy to provide video analysis services to ground users. After selecting the action, each LEO satellite will obtain the current instant reward r(t) and can observe the next state s(t+1). Each satellite is represented by a tuple<s(t)、a(t)、r(t)、s(t+1)> The state transitions, selected actions, and rewards are stored in the experience pool in the form of . When updating the agent network, a small batch of samples is randomly selected and updated according to the loss function. In this way, through continuous learning and training, each LEO satellite can eventually learn the strategy.

[0135] The satellite-ground collaborative UAV network computing offloading method and system proposed in this application can optimize the computing task allocation strategy of the UAV, including the selection of offloading satellites and the selection of offloading ratios, thereby achieving the effect of reducing the maximum delay of the system.

[0136] Embodiment 2

[0137] An embodiment of the present invention provides a satellite-ground collaborative UAV network computing offloading system, comprising:

[0138] Information acquisition module 10: used to establish a satellite-ground collaborative network including ground base stations, satellites, and drones. The satellite obtains global information of the satellite-ground collaborative network through the ground base station and sends computing tasks to the drone.

[0139] Data segmentation module 20: used for the drone to segment the captured video data and send it to corresponding satellites for analysis;

[0140] Strategy formulation module 30: used to formulate satellite unloading information optimization strategy and establish a training model based on the proximal strategy optimization algorithm;

[0141] The establishment of the training model includes: treating each satellite as an intelligent agent that makes independent unloading decisions, and determining the satellite's state, action, and reward mechanism based on the satellite's position and channel noise power;

[0142] Satellite training module 40: used for training satellites according to the training model to determine the optimal strategy for satellite unloading information.

[0143] In some specific embodiments, the training model represents the current strategy through parameters, and then uses two neural networks to update the parameters according to the following formula:

[0144] π θ (a|s)=P(a|s,θ)

[0145] In the formula, π θ (a|s) represents the current strategy, θ represents the parameter, a represents the action of the satellite, and s represents the state of the satellite.

[0146] In some specific embodiments, the state of the satellite includes the position of the satellite and the channel noise power, and is determined according to the following formula:

[0147]

[0148] Where, X m (t) represents the position of the satellite, represents the channel noise power of the satellite, S(t) represents the state function of the satellite, N represents the number of satellites, M represents the number of drones, n represents the number of satellites, and m represents the number of drones;

[0149] The action of the satellite is determined according to the network status. The action consists of the selected drone and the corresponding data offloading ratio. The action is expressed as:

[0150]

[0151] In the formula, a m (t) represents the action function, L m,n (t) represents the task offloading decision variable between UAV m and satellite n, ρ m,n (t) represents the ratio of unloading tasks from UAV m to satellite n;

[0152] The satellite's reward mechanism determines the selection data based on the drone's action ratio, sends the data to the corresponding satellite, and then determines the reward based on the drone's latency:

[0153] R(t)=-T m (t)

[0154] Where R(t) represents the reward function, T m (t) represents the time delay of the UAV.

[0155] In some specific embodiments, the satellite training module 40 is used to perform the following training steps:

[0156] Step S41: Initialize the network state at the beginning of each training;

[0157] Step S42: In each training time slot, collect the status information of each satellite, including channel noise power and satellite position status;

[0158] Step S43: Each satellite selects an action according to the local state and current strategy to provide video analysis services to ground users;

[0159] Step S44: After the satellite selects an action, the satellite is given a current instant reward, and the next state of the satellite is observed, wherein the state transition, the selected action and the reward of each satellite are stored in the training model in the form of a tuple;

[0160] Step S45: When updating the network, randomly select a small batch of samples and update them according to the loss function;

[0161] Step S46: Repeat steps S41-S45 to allow the satellites to repeat learning and training until the optimal strategy for all satellites is determined.

[0162] Embodiment 3

[0163] This embodiment simulates the proposed satellite-ground collaborative network collaborative offloading solution. Assume that both the satellite and the drone will appear in an area of ​​100km×100km. Figure 5 As shown, and assuming that five low-orbit satellites appear over this area at a certain time slot and can provide data offloading services to two drones, the current algorithm is used to select a suitable offloading strategy for the drone. The remaining parameters are shown in Table 1 below:

[0164] Table 1 Parameters

[0165]

[0166]

[0167] In a certain time slot, the relative positions of the satellite and the drone are as follows: Figure 5 As shown, algorithms are used to optimize the optimal configuration of resources.

[0168] Depend on Figure 6-Figure 9 It can be seen that the algorithm proposed in this application can optimize the computing task allocation strategy of the UAV, including the selection of unloading satellites and the selection of unloading ratios, so as to achieve the effect of reducing the maximum delay of the system. The influence of different learning rates on the algorithm was studied, and it was found that different learning rates would affect the convergence speed of the algorithm, and the learning rate should be adjusted to a reasonable range.

[0169] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0170] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referenced to each other. The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the functions in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing terminal device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1The steps of the functions specified in one or more boxes. Although the preferred embodiments of the embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they know the basic creative concept. Therefore, the attached claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention. Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or terminal device including a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0171] The method and device provided by the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principle and implementation mode of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation mode and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

[0172] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", "one specific embodiment" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0173] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A satellite-ground collaborative UAV network computing offloading method, It is characterized in that include: Step S1: establishing a satellite-ground collaborative network including ground base stations, satellites, and unmanned aerial vehicles. The satellite obtains global information of the satellite-ground collaborative network through the ground base station and sends computing tasks to the unmanned aerial vehicle. Step S2: The drone segments the captured video data and sends it to corresponding satellites for analysis; Step S3: formulating a satellite unloading information optimization strategy based on a proximal strategy optimization algorithm and establishing a training model; wherein the establishing of the training model includes: treating each satellite as an intelligent agent that makes an independent unloading decision, and determining the state, action and reward mechanism of the satellite according to the satellite's position and channel noise power; Step S4: training the satellite according to the training model to determine the optimal strategy for satellite unloading information; In step S3, the satellite's reward mechanism determines the selected data according to the drone's action ratio, sends the data to the corresponding satellite, and then determines the reward according to the drone's delay: R(t)=-T m (t) Where R(t) represents the reward function, T m (t) represents the time delay of the UAV.

2. According to the satellite-ground collaborative UAV network computing offloading method of claim 1, It is characterized in that Step S3 also includes: the training model represents the current strategy through parameters, and then uses two neural networks to update the parameters according to the following formula: p θ (a|s)=P(a|s,θ) In the formula, π θ (a|s) represents the current strategy, θ represents the parameter, a represents the action of the satellite, and s represents the state of the satellite.

3. According to the satellite-ground collaborative UAV network computing offloading method of claim 2, It is characterized in that In step S3, the state of the satellite includes the position of the satellite and the channel noise power, and is determined according to the following formula: Where, X m (t) represents the position of the satellite, represents the channel noise power of the satellite, S(t) represents the state function of the satellite, N represents the number of satellites, M represents the number of drones, n represents the number of satellites, and m represents the number of drones.

4. According to the satellite-ground collaborative UAV network computing offloading method of claim 2, It is characterized in that In step S3, the action of the satellite is determined according to the network status. The action consists of the selected drone and the corresponding data offloading ratio. The action is expressed as: a m (t)={L m,n (t),ρ m,n (t)}, In the formula, a m (t) represents the action function, L m,n (t) represents the task offloading decision variable between UAV m and satellite n, ρ m,n (t) represents the ratio of unloading tasks from UAV m to satellite n.

5. The satellite-ground collaborative UAV network computing offloading method according to claim 1, It is characterized in that Step S4 includes: Step S41: Initialize the network state at the beginning of each training; Step S42: In each training time slot, collect the status information of each satellite, including channel noise power and satellite position status; Step S43: Each satellite selects an action according to the local state and current strategy to provide video analysis services to ground users; Step S44: After the satellite selects an action, the satellite is given a current instant reward, and the next state of the satellite is observed, wherein the state transition, the selected action and the reward of each satellite are stored in the training model in the form of a tuple; Step S45: When updating the network, randomly select a small batch of samples and update them according to the loss function; Step S46: Repeat steps S41-S45 to allow the satellites to repeat learning and training until the optimal strategy for all satellites is determined.

6. A satellite-ground collaborative UAV network computing offloading system, It is characterized in that include: Information acquisition module: used to establish a satellite-ground collaborative network including ground base stations, satellites, and drones. The satellite obtains the global information of the satellite-ground collaborative network through the ground base station and sends computing tasks to the drone. Data segmentation module: used for the drone to segment the captured video data and send it to the corresponding satellite for analysis; Strategy formulation module: used to formulate satellite unloading information optimization strategy based on proximal strategy optimization algorithm and establish training model; wherein, establishing training model includes: treating each satellite as an intelligent agent that makes unloading decisions independently, and determining the state, action and reward mechanism of the satellite according to the satellite's position and channel noise power; Satellite training module: used for training satellites according to the training model to determine the optimal strategy for satellite unloading information; The state of the satellite includes the position of the satellite and the channel noise power, and is determined according to the following formula: Where, X m (t) represents the position of the satellite, represents the channel noise power of the satellite, S(t) represents the state function of the satellite, N represents the number of satellites, M represents the number of drones, n represents the number of satellites, and m represents the number of drones; The action of the satellite is determined according to the network status. The action consists of the selected drone and the corresponding data offloading ratio. The action is expressed as: a m (t)={L m,n (t),ρ m,n (t)}, In the formula, a m (t) represents the action function, L m,n (t) represents the task offloading decision variable between UAV m and satellite n, ρ m,n (t) represents the ratio of unloading tasks from UAV m to satellite n; The satellite's reward mechanism determines the selection data based on the drone's action ratio, sends the data to the corresponding satellite, and then determines the reward based on the drone's latency: R(t)=-T m (t) Where R(t) represents the reward function, T m (t) represents the time delay of the UAV.

7. The satellite-ground coordinated UAV network computing offloading system according to claim 6, It is characterized in that The training model represents the current strategy through parameters, and then uses two neural networks to update the parameters according to the following formula: p θ (a|s)=P(a|s,θ) In the formula, π θ (a|s) represents the current strategy, θ represents the parameter, a represents the action of the satellite, and s represents the state of the satellite.

8. The satellite-ground coordinated UAV network computing offloading system according to claim 6, It is characterized in that The satellite training module is used to perform the following training steps: Step S41: Initialize the network state at the beginning of each training; Step S42: In each training time slot, collect the status information of each satellite, including channel noise power and satellite position status; Step S43: Each satellite selects an action according to the local state and current strategy to provide video analysis services to ground users; Step S44: After the satellite selects an action, the satellite is given a current instant reward, and the next state of the satellite is observed, wherein the state transition, the selected action and the reward of each satellite are stored in the training model in the form of a tuple; Step S45: When updating the network, randomly select a small batch of samples and update them according to the loss function; Step S46: Repeat steps S41-S45 to allow the satellites to repeat learning and training until the optimal strategy for all satellites is determined.

Images