Airborne user calculation unloading method in high-dynamic civil aviation scene

Abstract

The invention relates to the technical field of edge computing, in particular to an airborne user computing unloading method in a high-dynamic civil aviation scene. The method comprises the following steps: firstly, constructing a mobile edge computing system model comprising a satellite, an aviation ad hoc network and an airborne user, and designing a communication model, a computing model and a switching model; then providing an optimization problem of maximizing system time delay and energy consumption; thirdly, describing an optimization problem as a Markov decision process and solving the optimization problem by using an improved depth deterministic strategy gradient algorithm; and finally, designing a domain-based parameter transfer strategy training method. According to the method, the problems of high mobility and resource limitation of airborne users in a civil aviation scene are comprehensively considered, and the method is designed by taking low-time-delay and low-energy-consumption task unloading and resource allocation as targets. When the method is used for dealing with a high-dynamic network environment, by designing a switching evaluation function and a task pre-unloading module based on a time evolution graph, the switching frequency and network congestion are remarkably reduced, and meanwhile the resource utilization rate and the algorithm convergence speed are improved; when complex resource scheduling is dealt with, efficient execution of tasks and load balance of the system are ensured through dynamic calculation unloading and resource allocation strategies.

Description

Technical Field

[0001] The present invention mainly relates to the technical fields related to computation offloading and resource allocation, and specifically to an onboard user computation offloading method in a highly dynamic civil aviation scenario. Background Art

[0002] As a key development direction for future mobile communications, 6G technology offers significant advantages in providing high reliability, high capacity, low latency, a high-quality communication experience, high security, and low energy consumption. The rapid development of 6G technology makes the vision of ubiquitous internet access possible. Today, people expect to enjoy the same internet services on board as they do on the ground, such as real-time calls and high-definition video. However, existing civil aviation communication networks are limited by a shortage of wireless resources and degraded channel quality caused by high-speed flight, making it difficult to provide high-quality wireless services. Furthermore, the rise of intelligent applications such as fault detection, automated flight, and the aviation Internet of Things (IoT) places higher demands on computing resources. Relying solely on onboard resources is insufficient to meet the demands of intensive and latency-sensitive missions.

[0003] Mobile Edge Computing (MEC) provides a forward-looking approach to solving this problem by offloading tasks from local to edge servers. Currently, airline operational communications (AOC) and aircraft passenger communications (APC) data services are mainly provided through air-to-ground (A2G) macrocellular systems or air-to-satellite (A2S) communications. However, due to the isolation and ultra-long distance of air travel, it is almost impossible to rely on A2G communications to achieve full coverage of oceans or remote airspace; and A2S links face high costs and significant end-to-end delay problems. To this end, the present invention proposes to use civil aircraft as a potential platform for computing offloading services, and introduces an aviation self-organizing network architecture to achieve optimal allocation and efficient utilization of resources. This resource sharing model not only balances the resource allocation between different aircraft, but also improves the resource utilization of the overall system.

[0004] Despite this, the architecture still faces two major challenges. First, the high mobility of civil aircraft and low-Earth orbit (LEO) satellites leads to frequent handovers, which can cause service interruptions and network congestion. Existing handover strategies are mostly based on static network conditions and are difficult to adapt to time-varying scenarios of onboard user computing offload. New handover strategies are urgently needed to cope with highly dynamic network environments. Second, civil aircraft mobility management is complex, and resource scheduling is difficult. How to implement dynamic resource scheduling during flight to ensure efficient offloading and routing of computing tasks has become a key problem that needs to be solved urgently. Summary of the Invention

[0005] The present invention aims to study the collaborative mechanism between LEO satellite networks and aeronautical ad hoc networks, provide seamless Internet access and efficient computing offloading services for airborne users, and solve key problems such as resource scheduling, switching management and low latency requirements in highly dynamic network environments.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows: a method for offloading onboard user computing in a highly dynamic civil aviation scenario, characterized by comprising the following steps:

[0007] Step 1: Establish a computational offloading system model for collaboration between satellite networks and aeronautical ad hoc networks;

[0008] Step 2: Design the communication model, calculation model, and switching model;

[0009] Step 3: Establish an optimization problem with the goal of minimizing the delay and energy consumption of all onboard users;

[0010] Step 4: Define the interactive task offloading decision process as a Markov decision process and use the improved deep deterministic policy gradient algorithm to solve the problem;

[0011] Step 5: Design a training method for the neighborhood-based parameter transfer strategy.

[0012] Furthermore, the establishment of a satellite network and aeronautical ad hoc network collaborative computing offloading system model described in step 1 is specifically as follows:

[0013] The system model includes R civil aircraft and S LEO satellites, which are denoted as and There are N onboard users on each civil aircraft, and all onboard users are denoted as Where, U=R×N. Airborne users The resulting task τ can be described by the following terms, τ u ={D,c,T max}, where D represents the amount of data for the task, c represents the computational complexity of the task, and T max Indicates the maximum tolerable delay of a task. represents the offloading decisions of all onboard users, in

[0014]

[0015] Use a tuple Represents the resource allocation decisions of all onboard users.

[0016] Furthermore, the design communication model, calculation model and switching model described in step 2 are specifically as follows:

[0017] First, a communication model is constructed, including A2S communication link and A2A communication link.

[0018] 1) A2S link. Due to the limited bandwidth of airborne users, the present invention introduces bandwidth allocation variables Indicates satellite s k The proportion of bandwidth occupied. k The signal-to-noise ratio (SNR) received at the end can be expressed as

[0019]

[0020] in, Transmit power, are the transmitting antenna gain of user u and satellite s respectively k The receiving antenna gain. L add-A2s represents the additional losses caused by the atmosphere and environment, is the Rice fading coefficient, and n0 is the noise power spectrum density. FSPL (u,s k ) represents the free space link loss:

[0021]

[0022] in, is the signal wavelength, For airborne users u and satellite s k The distance between them.

[0023] According to the Shannon-Hartley theorem, satellite s k The achievable rate when receiving tasks from onboard user u is:

[0024]

[0025] Since the distance between the airborne user and the low-orbit satellite is long and the mobility is high, the A2S link may be unstable. Therefore, the present invention defines the link availability probability (LAP) to quantify the possibility of the link maintaining connection. k )| 2 Obey non-central chi-square Distribution. Let K(u,s k ) is user u and satellite s k The Rician factor based on the angle between. By setting the threshold rate of transmission to LAP can be expressed as:

[0026]

[0027] in, is the Marcum Q function, and is a modified Bessel function of the first kind and order 0.

[0028] Assuming that the mission is successfully launched at the xth time, the total time consumption can be expressed as:

[0029]

[0030] Since the maximum tolerable delay of the task is T max If the total launch time exceeds T max , the mission offloading fails. Therefore, the onboard user u to the satellite s k The actual transmission delay is:

[0031]

[0032] Onboard user u offloads tasks to satellite s k The transmission energy consumption is expressed as

[0033]

[0034] Since the distance between the low-orbit satellite and the airborne user is relatively far, the propagation delay cannot be ignored. The propagation delay is

[0035]

[0036] 2) A2A link. The A2A channel is a frequency-flat block-fading Rayleigh channel, and the transmission rate can be expressed as

[0037]

[0038] in, Transmit power, h A2A is the fading coefficient of the A2A channel, and N0 is the white Gaussian noise power.

[0039] Therefore, the transmission delay of the A2A link is

[0040]

[0041] Then a computing model is constructed, including local computing, satellite computing and aerial ad hoc network computing.

[0042] 1) Local computing. The processing power of local computing is f u,l , the total time for the task to be calculated locally is

[0043]

[0044] The total energy consumption is the energy consumption of local calculation, expressed as

[0045]

[0046] Among them, κ l It is a coefficient that reflects the relationship between local processing power and energy consumption, and depends on the effective switching capacitance of the chip structure.

[0047] 2) Unload to the satellite. After the data is unloaded to the satellite, the satellite processes the data. Indicates satellite s k The computing resources allocated to user u. k The computation time on

[0048]

[0049] Satellites k The computational energy consumption is expressed as

[0050]

[0051] Offload to satellites k Total delay

[0052]

[0053] Offload to satellites k Total energy consumption

[0054]

[0055] 3) Offloading to AANET: When an onboard user transfers a task to AANET, if the computing resources of the AANET node with which it directly communicates are insufficient, the number of forwardings is recorded as n. Represents node r m The computing resources allocated to user u. The computing time is

[0056]

[0057] The energy consumption is expressed as

[0058]

[0059] The total delay of offloading to AANET is

[0060]

[0061] The total energy consumption offloaded to AANET is

[0062]

[0063] Finally, build the switching model.

[0064] Due to the high mobility of airborne users and satellites, handover is triggered when an airborne user leaves the coverage area of ​​the currently connected satellite. To ensure seamless connectivity and provide high-quality computing services, the remaining visible time, available channel resources, and average received signal strength are comprehensively considered as handover criteria.

[0065] 1) Remaining visible time. To ensure service continuity, satellites with longer service time are preferred. Indicates satellite s k The remaining service time that can be provided to onboard user u, Satisfying formula (22):

[0066]

[0067] The coordinates of the satellite are The coordinates of the onboard user u are (x u ,y u ), and v u The speed of Direction of flight, is the radius of the satellite service area.

[0068] 2) Available channel resources: To prevent network congestion caused by overloading of a large number of satellites, the number of idle channels is considered as one of the switching criteria.

[0069] The available channels are represented as

[0070]

[0071] in, Candidate satellite s for airborne user u k The number of remaining idle channels, is the maximum number of idle channels owned by the visible satellites of airborne user u.

[0072] 3) Average received signal strength. Switching decisions based solely on received signal strength at a specific moment are not suitable for time-varying aviation scenarios. Therefore, the concept of average received signal strength is proposed, which is calculated as follows:

[0073]

[0074] in, is the receiving antenna gain, is the angle between the satellite service area center-satellite line and the satellite-airborne user line. The initial coordinates of the airborne user are (x u ,y u ), and with v u The speed of the aircraft is continuously flying in the direction of θ, so after time t the coordinates of the onboard user are

[0075] In view of the fact that the service seamless switching requirement is closely related to the above three indicators, the present invention defines a switching evaluation function: Used to characterize the relationship between the airborne user u and the satellite s k The switch evaluation of the function is:

[0076]

[0077] Among them, η1, η2, and η3 are weight coefficients, which are used to adjust the influence of each indicator on the switching evaluation.

[0078] Furthermore, the optimization problem established in step 3 is specifically as follows:

[0079] The goal of this invention is to reduce the time delay and energy consumption of all onboard users. The optimization problem is expressed as follows:

[0080]

[0081]

[0082] (30a) ensures that all offload tasks can be completed within their allowed delay. (30b) ensures that the transmission energy consumption of each onboard user does not exceed the maximum energy stored. (30c) takes into account the remaining visibility time of the satellite, available channel resources and average received signal strength to ensure seamless switching and load balancing of the satellite. (30d) represents the binary offload decision. (30e)(30f)(30g) ensure that the allocated computing resources are less than the total computing resources of the server.

[0083] Furthermore, the interactive task offloading decision process described in step 4 is defined as a Markov decision process, and the improved deep deterministic policy gradient algorithm is used to solve the problem, as follows:

[0084] To address the above issues, the states, actions, and rewards in the MDP model are described as follows.

[0085] ● State space: Position information P(t), including the position of the onboard user and the position of the satellite. Remaining resource information F(t), including the remaining local resources.

[0086] computing resources and remaining computing resources of satellites.

[0087] Action space: Take actions based on the current state, including uninstall decisions and resource allocation decisions

[0088] Reward function: Although Equation (31c) constrains the switching strategy, it is not a strong constraint and switching failures may still occur. Therefore, a penalty term Λ(t) is introduced:

[0089]

[0090] Therefore, the reward function is:

[0091]

[0092] Because the action space encompasses computational offloading and resource allocation decisions that each onboard user needs to jointly optimize, and contains both discrete and continuous variables, if computations are directly involved, the action space required to be explored would be extremely large. This paper designs a task pre-offloading module based on a time evolution graph (TEG) to reduce the action space.

[0093] The TEG contains storage links and A2A transmission links. A storage link represents the connection between two adjacent moments of the same node and is determined by the node's storage capacity. To ensure the stable topology of the AANET, the link duration (LET) between any two nodes is calculated as shown in Equation (28). Under the constraint of the number of antennas per node, the node with the larger LET is preferentially selected to establish the A2A transmission link.

[0094]

[0095] in, d=y i -y j .

[0096] The task pre-offloading module aims to find a target node within the AANET with sufficient computing resources to handle the source node's computational tasks. Using the bisection method and Dijkstra's algorithm, the ideal minimum latency from the onboard user to all AANET nodes is calculated. Only AANET nodes that meet the latency requirements are retained, reducing the action space to be explored and improving convergence speed. During the offloading and resource allocation decision-making process, the agent obtains training samples through interaction with the environment and uses historical samples to estimate the value function. The ultimate goal is to find the optimal strategy for offloading decisions and resource allocation.

[0097] Furthermore, the training method for designing a neighborhood-based parameter transfer strategy described in step 5 is specifically as follows:

[0098] The reward function of this invention includes three objectives: latency, energy consumption, and a penalty function. Existing algorithms typically use a linear weighting approach to transform multiple objective functions into a single objective function for analysis, thereby obtaining a comprehensive evaluation metric. However, this approach loses information about the objective function and the solution. The Pareto model can better address this issue by operating directly based on the original values.

[0099] In order to obtain the Pareto frontier of the original problem, the multi-objective optimization problem is decomposed into a series of scalar optimization sub-problems using the decomposition idea, and the model parameters of each sub-problem are optimized in a collaborative manner. Solving each scalar optimization sub-problem usually results in a Pareto optimal solution. When all sub-optimization problems are solved, the Pareto frontier of the original problem can be obtained. First, establish a set of uniformly distributed weighted vectors Where K is the number of weight vectors. Number each subproblem j, j∈{1,2,···,K}, where, M represents the number of optimization objectives. The objective function of the j-th sub-problem after decomposition is as follows:

[0100]

[0101] Specifically, the network model parameters of the j-1th sub-problem can be expressed as Definition [w * ,θ * ] represents the parameters of the network that have been optimized, and [w,θ] represents the parameters that have not been optimized. Assume that the j-1 problem has been solved, that is, its network parameters have been optimized to the optimal value. Then for the network training in the j-1 sub-problem, the best network parameters obtained in the j-1 sub-problem are used. The network parameters are sequentially transferred from the previous problem to the next subproblem. By transferring network parameters, the subproblems are solved sequentially, and the Pareto front can be approximated based on the obtained model. BRIEF DESCRIPTION OF THE DRAWINGS

[0102] Figure 1 This is a diagram of the computation offloading network architecture for the highly dynamic civil aviation scenario of the present invention;

[0103] Figure 2 A computation offloading diagram based on an improved deep deterministic gradient algorithm according to the present invention; DETAILED DESCRIPTION

[0104] The present invention will be further described with reference to the accompanying drawings.

[0105] Step 1: Establish a computational offloading system model for satellite network and aeronautical ad hoc network collaboration, as follows:

[0106] Combine Figure 1 The present invention considers a computation offloading architecture for onboard users, in which the LEO network and the Aeronautical Ad Hoc Network (AANET) collaborate to transmit content to passengers on the aircraft, aiming to provide seamless Internet access and high-quality services for onboard users. The system model includes R civil aircraft and S LEO satellites, where the civil aircraft and LEO satellites are denoted as and There are N onboard users on each civil aircraft who need computing offloading services at the same time. All onboard users are recorded as Where U = R × N. Assume that at a certain moment, each user can only generate one content request. The resulting task τ can be described by the following terms, τ u ={D,c,T max}, where D represents the amount of data for the task, c represents the computational complexity of the task, and T max Indicates the maximum tolerable delay of a task. Use a vector represents the offloading decisions of all onboard users in

[0107]

[0108] Use a tuple Represents the resource allocation decisions of all onboard users.

[0109] The design communication model, calculation model, and switching model described in step 2 are as follows:

[0110] First, a communication model is constructed, including A2S communication link and A2A communication link.

[0111] 1) A2S link. Due to the limited bandwidth of airborne users, the present invention introduces bandwidth allocation variables Indicates satellite s k The proportion of bandwidth occupied. k The signal-to-noise ratio (SNR) received at the end can be expressed as

[0112]

[0113] in, Transmit power, are the transmitting antenna gain of user u and satellite s respectively k The receiving antenna gain. L add-A2S represents the additional losses caused by the atmosphere and environment, is the Rice fading coefficient, and n0 is the noise power spectrum density. FSPL (u,s k ) represents the free space link loss:

[0114]

[0115] in, is the signal wavelength, For airborne users u and satellite s k The distance between them.

[0116] According to the Shannon-Hartley theorem, satellite s k The achievable rate when receiving tasks from onboard user u is:

[0117]

[0118] Since the distance between airborne users and low-orbit satellites is long and their mobility is high, the A2S link may be unstable. Therefore, the link availability probability (LAP) is defined to quantify the possibility of the link remaining connected. k )| 2 Obey non-central chi-square Distribution. Let K(u,s k ) is user u and satellite s k The Rician factor based on the angle between. By setting the threshold rate of transmission to LAP can be expressed as:

[0119]

[0120] in, is the Marcum Q function, and is a modified Bessel function of the first kind and order 0.

[0121] Assuming that the mission is successfully launched at the xth time, the total time consumption can be expressed as:

[0122]

[0123] Since the maximum tolerable delay of the task is T max If the total launch time exceeds T max , the mission offloading fails. Therefore, the onboard user u to the satellite s k The actual transmission delay is:

[0124]

[0125] Onboard user u offloads tasks to satellite s k The transmission energy consumption is expressed as

[0126]

[0127] Since the distance between the low-orbit satellite and the airborne user is relatively far, the propagation delay cannot be ignored. The propagation delay is

[0128]

[0129] 2) A2A link. The A2A channel is a frequency-flat block-fading Rayleigh channel, and the transmission rate can be expressed as

[0130]

[0131] in, Transmit power, h A2A is the fading coefficient of the A2A channel, and N0 is the white Gaussian noise power.

[0132] Therefore, the transmission delay of the A2A link is

[0133]

[0134] Then a computing model is constructed, including local computing, satellite computing and aerial ad hoc network computing.

[0135] 1) Local computing. The processing power of local computing is f u,l , the total time for the task to be calculated locally is

[0136]

[0137] The total energy consumption is the energy consumption of local calculation, expressed as

[0138]

[0139] Among them, κ l It is a coefficient that reflects the relationship between local processing power and energy consumption, and depends on the effective switching capacitance of the chip structure.

[0140] 2) Unload to the satellite. After the data is unloaded to the satellite, the satellite processes the data. Indicates satellite s k The computing resources allocated to user u. k The computation time on

[0141]

[0142] Satellites k The computational energy consumption is expressed as

[0143]

[0144] 3) Offloading to AANET: When an onboard user transfers a task to AANET, if the computing resources of the AANET node with which it directly communicates are insufficient, the number of forwardings is recorded as n. Represents node r m The computing resources allocated to user u. The computing time is

[0145]

[0146] The energy consumption is expressed as

[0147]

[0148] Finally, build the switching model.

[0149] Due to the high mobility of airborne users and satellites, handover is triggered when an airborne user leaves the coverage area of ​​the currently connected satellite. To ensure seamless connectivity and provide high-quality computing services, the remaining visible time, available channel resources, and average received signal strength are comprehensively considered as handover criteria.

[0150] 1) Remaining visible time. To ensure service continuity, satellites with longer service time are preferred. Indicates satellite s k The remaining service time that can be provided to onboard user u, Satisfying formula (22):

[0151]

[0152] The coordinates of the satellite are The coordinates of the onboard user u are (x u ,y u ), and v u The speed of Direction of flight, is the radius of the satellite service area.

[0153] 2) Available channel resources: To prevent network congestion caused by overloading of a large number of satellites, the number of idle channels is considered as one of the switching criteria.

[0154] The available channels are represented as

[0155]

[0156] in, Candidate satellite s for airborne user u k The number of remaining idle channels, is the maximum number of idle channels owned by the visible satellites of airborne user u.

[0157] 3) Average received signal strength. Switching decisions based solely on received signal strength at a specific moment are not suitable for time-varying aviation scenarios. Therefore, the concept of average received signal strength is proposed, which is calculated as follows:

[0158]

[0159] in, is the receiving antenna gain, is the angle between the satellite service area center-satellite line and the satellite-airborne user line. The initial coordinates of the airborne user are (x u ,y u ), and with v u The speed of The flight continues in this direction, so after time t the coordinates of the onboard user are

[0160] In view of the fact that the service seamless switching requirement is closely related to the above three indicators, the present invention defines a switching evaluation function: Used to characterize the relationship between the airborne user u and the satellite s k The switch evaluation of the function is:

[0161]

[0162] Step 3: Establish the optimization problem as follows:

[0163] The goal of this invention is to reduce the time delay and energy consumption of all onboard users. The optimization problem is expressed as follows:

[0164]

[0165] (60a) ensures that all offload tasks can be completed within their allowed delay. (60b) ensures that the transmission energy consumption of each onboard user does not exceed the maximum energy stored. (60c) takes into account the remaining visibility time of the satellite, available channel resources and average received signal strength to ensure seamless switching and load balancing of the satellite. (60d) represents the binary offload decision. (60e)(60f)(60g) ensure that the allocated computing resources are less than the total computing resources of the server.

[0166] Step 4: Define the interactive task offloading decision process as a Markov decision process and use the improved deep deterministic policy gradient algorithm to solve the problem as follows:

[0167] To address the above issues, the states, actions, and rewards in the MDP model are described as follows.

[0168] State space: Position information P(t), including the position of the onboard user and the satellite. Remaining resource information F(t), including the remaining local computing resources and the remaining satellite computing resources.

[0169] Action space: Take actions based on the current state, including uninstall decisions and resource allocation decisions

[0170] Reward function: Although Equation (61c) constrains the switching strategy, it is not a strong constraint and switching failures may still occur. Therefore, a penalty term Λ(t) is introduced:

[0171]

[0172] Therefore, the reward function is:

[0173]

[0174] Combine Figure 1 Because the action space includes computational offloading and resource allocation decisions that each onboard user needs to jointly optimize, and contains both discrete and continuous variables, if the computation is directly involved, the action space that needs to be explored will be very large. This paper designs a task pre-offloading module based on a time evolution graph (TEG) to reduce the action space.

[0175] The TEG contains storage links and A2A transmission links. A storage link represents the connection between two adjacent moments of the same node and is determined by the node's storage capacity. To ensure the stable topology of the AANET, the link duration (LET) between any two nodes is calculated as shown in Equation (63). Under the constraint of the number of antennas per node, the node with the larger LET is preferentially selected to establish the A2A transmission link.

[0176]

[0177] in, d=y i -y j .

[0178] The task pre-offloading algorithm aims to find a target node within the AANET with sufficient computing resources to handle the source node's computational tasks. It uses the bisection method and Dijkstra's algorithm to calculate the ideal minimum latency from the airborne user to all AANET nodes. Only the AANET nodes that meet the latency requirements are retained, reducing the action space that needs to be explored and improving convergence speed.

[0179] During the offloading and resource allocation decision-making process, the agent obtains training samples through interaction with the environment and uses historical samples to estimate the value function. The ultimate goal is to find the optimal strategy for offloading decisions and resource allocation. The state and action spaces are high-dimensional, dynamic, and continuous. Therefore, the TPO-DDPG algorithm is proposed to optimize the joint decision-making and resource allocation problem, as shown in Algorithm 1.

[0180]

[0181] Step 5: Design a training method for the neighborhood-based parameter transfer strategy, as follows:

[0182] Existing algorithms usually use a linear weighted approach to transform multiple objective functions into a single objective function for analysis, thereby obtaining a comprehensive evaluation index. However, this method loses information about the objective function and the solution. The Pareto model can better solve this problem. In order to obtain the Pareto frontier of the original problem, the multi-objective optimization problem is decomposed into a series of scalar optimization sub-problems using the idea of ​​decomposition, and the model parameters of each sub-problem are optimized in a collaborative manner. Solving each scalar optimization sub-problem usually results in a Pareto optimal solution. When all sub-optimization problems are solved, the Pareto frontier of the original problem can be obtained. Therefore, a training method based on a neighborhood-based parameter transfer strategy is adopted, as shown in Algorithm 2. Specifically, the network model parameters of the j-1th sub-problem can be expressed as Definition [w * ,θ * ] represents the parameters of the neural network that have been optimized, and [w,θ] represents the parameters that have not been optimized. Assume that the j-1 problem has been solved, that is, its network parameters have been optimized to the optimal value. Then for the network training in the j-1 sub-problem, the best network parameters obtained in the j-1 sub-problem are used. The network parameters are transferred from the previous problem to the next sub-problem in sequence.

[0183]

[0184] The above description describes the implementation process and advantages of the present invention. It should be understood by those skilled in the art that various changes and improvements may be made to the present invention without departing from the principles of the present invention, and such changes and improvements fall within the scope of the present invention as claimed.

Claims

1. A method for offloading onboard user computing in a highly dynamic civil aviation scenario, characterized in that: The following steps are involved: Step 1: Establish a computational offloading system model for collaboration between satellite networks and aeronautical ad hoc networks; Step 2: Design the communication model, calculation model, and switching model; Step 3: Establish an optimization problem with the goal of minimizing the delay and energy consumption of all onboard users; Step 4: Define the interactive task offloading decision process as a Markov decision process and use the improved deep deterministic policy gradient algorithm to solve the problem; Step 5: Design a training method for the neighborhood-based parameter transfer strategy.

2. The onboard user computing offloading method in a high-dynamic civil aviation scenario according to claim 1 is characterized in that: The establishment of a computational offloading system model for satellite network and aeronautical ad hoc network collaboration as described in step 1 is as follows: The present invention considers a computation offloading architecture for onboard users, in which the LEO network and the Aeronautical Ad Hoc Network (AANET) collaborate to transmit content to passengers on the aircraft, aiming to provide seamless Internet access and high-quality services for onboard users. The system model includes R civil aircraft and S LEO satellites, where the civil aircraft and LEO satellites are denoted as and There are N onboard users on each civil aircraft who need computing offloading services at the same time. All onboard users are recorded as Where U = R × N. Assume that at a certain moment, each user can only generate one content request. Onboard user u, The resulting task τ can be described by the following terms, τ u ={D,c,T max }, where D represents the amount of data for the task, c represents the computational complexity of the task, and T max Denotes the maximum tolerable delay of the task. A vector O represents the offloading decision of all onboard users. in Use a tuple Represents the resource allocation decisions of all onboard users.

3. The onboard user computing offloading method in a high-dynamic civil aviation scenario according to claim 1 is characterized in that: The design communication model, calculation model, and switching model described in step 2 are as follows: Step 1: Build a communication model, including A2S communication link and A2A communication link. (1) A2S link. Since the bandwidth of airborne users is limited, the bandwidth allocation variable is introduced. in, Indicates satellite s k The ratio of occupied bandwidth. k A2S link transmission, in satellite s k The signal-to-noise ratio (SNR) received at the end can be expressed as in, is the transmission power of the onboard user u, are the transmitting antenna gain of user u and satellite s respectively k The receiving antenna gain. L add-A2S represents the additional losses caused by the atmosphere and environment, is the Rice fading coefficient, and n0 is the noise power spectrum density. FSPL (u,s k ) represents the free space link loss: in, is the signal wavelength, For airborne users u and satellite s k According to the Shannon-Hartley theorem, satellite s k The achievable rate when receiving tasks from onboard user u is: Since the distance between the airborne user and the low-orbit satellite is long and the mobility is high, the A2S link may be unstable. The present invention defines the Link Availability Probability (LAP) to quantify the possibility of the link remaining connected. is the Rician fading coefficient, Obey non-central chi-square Distribution. Let K(u,s k ) is user u and satellite s k The Rician factor based on the angle between. By setting the threshold rate of transmission to LAP can be expressed as: in, is the Marcum Q function, and is a modified Bessel function of the first kind and order 0. During the mission launch, if the first launch fails, the onboard user u will send k Initiate a second launch attempt. Assuming the mission succeeds at the xth launch, the total time consumption can be expressed as: Since the maximum tolerable delay of the task is T max If the total launch time exceeds T max , the mission offloading fails. Therefore, the onboard user u to the satellite s k The actual transmission delay is: Onboard user u offloads the task to satellite s k The transmission energy consumption of the star is expressed as Since the distance between the low-orbit satellite and the airborne user is relatively far, the propagation delay cannot be ignored. The propagation delay is (2) A2A link. Assuming that the A2A channel is a frequency-flat block-fading Rayleigh channel, the path loss is modeled as θ represents the path loss exponent. According to Shannon’s formula, when data is transmitted in a specified wireless bandwidth B, the transmission rate can be expressed as in, is the transmission power, h A2A is the fading coefficient of the A2A channel, and N0 is the white Gaussian noise power. Therefore, the transmission delay of the A2A link is Step 2: Build a computing model, including local computing, satellite computing, and aeronautical ad hoc network computing. (1) Local computing. The processing power of local computing is f u，l , the total delay of the task in local calculation is The total energy consumption is (2) Satellite computing. Indicates satellite s k The computing resources allocated to user u. k The computational delay on Calculate the energy consumption as (3) Aeronautical ad hoc network computing. When an airborne user transmits a task to AANET, if the computing resources of the AANET node with which it directly communicates are insufficient, the task will be forwarded. The number of forwardings is recorded as n. Represents node r m The computing resources allocated to user u. The computing time is Calculate the energy consumption as Step 3: Build a switching model. Due to the high-speed mobility of airborne users and satellites, when an airborne user leaves the coverage of the currently connected satellite, a handover is triggered and the next target satellite is selected for access. In order to ensure seamless connection and provide high-quality computing services, the remaining visible time, available channel resources and average received signal strength are comprehensively considered as handover criteria, and a handover evaluation function is designed. (1) Remaining viewing time. Assume Indicates satellite s k The remaining service time that can be provided to onboard user u, satisfy The coordinates of the satellite are The coordinates of the onboard user u are (x u ,y u ), is the vertical height difference between the satellite and user u, and user u is v u The speed continues to fly in the direction of θ, For satellites k The radius of the service area. (2) Available channel resources. In order to prevent a large number of satellites from being overloaded and causing network congestion, the number of idle channels is considered as one of the switching criteria. The available channels are expressed as in, Candidate satellite s for airborne user u k The number of remaining idle channels, is the maximum number of idle channels owned by the visible satellites of airborne user u. (3) Average received signal strength. The present invention believes that it is unreasonable to make a switching decision based on the received channel strength at a certain moment. Therefore, the present invention proposes the concept of average received signal strength, which is to accumulate the channel strength during the service time and then divide it by the service time. The calculation is as follows in, is the receiving antenna gain, is the angle between the satellite service area center-satellite line and the satellite-airborne user line. The initial coordinates of the airborne user are (x u ,y u ), and with v u The speed of the aircraft is continuously flying in the direction of θ, so after time t the coordinates of the onboard user are In view of the fact that the service seamless switching requirement is closely related to the above three indicators, the present invention defines a switching evaluation function: Used to characterize the relationship between the airborne user u and the satellite s k The switch evaluation of the function is:

4. The method for offloading onboard user computing in a highly dynamic civil aviation scenario according to claim 1, characterized in that: The optimization problem described in step 3 is established as follows: Hybrid computing task offloading can be formulated as a constrained optimization problem, where the main goal is to minimize the energy consumption and latency of all onboard users. The optimization problem is formulated as follows: Constraint (26a) ensures that all offload tasks can be completed within their allowed delay. (26b) ensures that the transmission energy consumption of each onboard user does not exceed the maximum energy stored. (26c) takes into account the remaining visibility time of the satellite, available channel resources and average received signal strength to ensure seamless switching and load balancing of the satellite. (26d) represents the binary offload decision. (26e)(26f)(26g) ensure that the allocated computing resources are less than the total computing resources of the server.

5. The onboard user computing offloading method in a high-dynamic civil aviation scenario according to claim 1 is characterized in that: The interactive task offloading decision process described in step 4 is defined as a Markov decision process, and the improved deep deterministic policy gradient algorithm is used to solve the problem, as follows: Defines a tuple To model the MDP, S represents the state set, A is the action set, P is the set of state transition probabilities, R represents the reward function, and Π is a decision rule mapping from state s∈S to action a∈A. Specifically: (1) State space: Position information P(t), including the position of the onboard user and the position of the satellite. Remaining resource information F(t), including the remaining local computing resources and the remaining satellite computing resources. And related information G(t) of the aeronautical ad hoc network. (2) Action space: Take actions based on the current state, including unloading decisions and resource allocation decisions (3) Reward function: Although Equation (26c) constrains the switching strategy, it is not a strong constraint and switching failures may still occur. Consider a situation where the onboard user leaves the coverage of the current satellite and triggers a handover. The task is offloaded to one of the visible satellites. However, when the calculation is completed, the user is no longer within the service range of the satellite, and the calculation result cannot be transmitted back to the user. Although the maximum tolerable delay constraint defined in (26a) is met, the result shows that the user does not enjoy the service provided by the satellite. Therefore, the penalty term Λ(t) is introduced. in, It means that the time of unloading to the satellite exceeds the remaining computing time of the satellite, and λ is a constant penalty weight. Therefore, the reward function is expressed as Because the action space encompasses computational offloading and resource allocation decisions that each onboard user needs to jointly optimize, and contains both discrete and continuous variables, if computations are directly involved, the action space required to be explored would be extremely large. This paper designs a task pre-offloading module based on a time evolution graph (TEG) to reduce the action space. TEGs include storage links and A2A transmission links. A storage link represents the connection between two adjacent time windows on the same node, determined by the node's storage capacity. An A2A transmission link connects different nodes within the same time window. To ensure the stable topology of AANET, the link duration (LET) between any two nodes is calculated as shown in Equation (29). Under the constraint of the number of antennas of each node, the node with larger LET is preferentially selected to establish the A2A transmission link. in, a=v i ·(cosθ i )-v j ·(cosθ j ),b=x i -x j ,c=v i ·(sinθ i )-v j ·(sinθ j ),d=y i -y j 。 The task pre-offloading algorithm aims to find a target node within the AANET with sufficient computing resources to handle the source node's computational tasks. It uses the bisection method and Dijkstra's algorithm to calculate the ideal minimum latency from the airborne user to all AANET nodes. Only the AANET nodes that meet the latency requirements are retained, reducing the action space that needs to be explored and improving convergence speed. During the offloading and resource allocation decision-making process, the agent obtains training samples through interaction with the environment and uses historical samples to estimate the value function. The ultimate goal is to find the optimal strategy for offloading decisions and resource allocation. The state and action spaces are high-dimensional, dynamic, and continuous. Therefore, the TPO-DDPG algorithm is proposed to optimize the joint decision-making and resource allocation problem, as shown in Algorithm 1.

6. The method for offloading onboard user computing in a highly dynamic civil aviation scenario according to claim 1, characterized in that: The training method for designing a domain-based parameter transfer strategy described in step 5 is as follows: Existing algorithms typically use a linear weighting approach to transform multiple objective functions into a single objective function for analysis, thereby obtaining a comprehensive evaluation metric. However, this approach loses information about the objective function and the solution. The Pareto model can better address this issue by operating directly based on the original values. therefore A training method based on a neighborhood-based parameter transfer strategy is adopted, which can effectively bring out the advantages of Algorithm 1. In order to obtain the Pareto frontier of the original problem, the multi-objective optimization problem is decomposed into a series of scalar optimization sub-problems using the decomposition idea, and the model parameters of each sub-problem are optimized in a collaborative manner. Solving each scalar optimization sub-problem usually results in a Pareto optimal solution. When all sub-optimization problems are solved, the Pareto frontier of the original problem can be obtained. First, establish a set of uniformly distributed weighted vectors Where K is the number of weight vectors. Number each subproblem j, j∈{1,2,···,K}, where, M represents the number of optimization objectives. The objective function of the j-th sub-problem after decomposition is as follows: Algorithm 1 is used to model and solve each subproblem, and network parameters are transferred to achieve sequential solution of the subproblems. Therefore, the Pareto front can be approximated based on the resulting model. The overall framework of the algorithm, which combines this decomposition with a neighborhood-based parameter transfer strategy, is shown in Algorithm 2.

Images