Large-scale constellation general calculation collaborative resource allocation method facing non-uniform service requirements

Abstract

The invention discloses a large-scale constellation general calculation collaborative resource allocation method facing non-uniform service requirements, and mainly solves the problems of time delay and high energy consumption overhead when task unloading is carried out by utilizing a satellite network under the condition that ground network resources are limited in the prior art. According to the implementation scheme, a large-scale constellation network is constructed; the method comprises the following steps: dividing the ground into a plurality of position areas, taking one area as a reference, jointly optimizing task scheduling by adopting an improved DRL-DAO (Deep Reinforcement Learning Differentiated Task Offload) algorithm, realizing efficient dynamic resource allocation to complete calculation of an offload task, and calculating the offload task between adjacent areas according to channel space-time correlation and task correlation. And performing fine adjustment on the trained model through transfer learning to complete task scheduling and resource allocation of all regions. According to the method, differentiated services can be provided for different types of tasks, resources are reasonably utilized to improve the user service quality, the high task completion rate and the minimum time delay and energy consumption weighted sum are achieved, and the method can be used for scheduling processing of various types of tasks of a user terminal.

Description

Technical Field

[0001] The present invention belongs to the field of satellite communication technology, and in particular relates to a large-scale constellation computing collaborative resource allocation method, which can be used to schedule various computing type tasks generated for user terminals. Background Art

[0002] With the explosive growth of ubiquitous and high-speed communication needs, ground networks are becoming saturated, and satellite networks have attracted attention as a supplement to ground networks. Compared with medium-orbit and high-orbit satellites, the large-scale low-orbit satellite constellation network that has emerged has advantages such as low launch cost, easy maintenance, and short propagation delay. Therefore, more and more institutions have begun to study large-scale constellations consisting of thousands of low-Earth orbits to provide global services, such as SpaceX, OneWeb, Telesat, etc. For 5G and 6G mobile networks, massive data connections and large-capacity services make ultra-reliable low-latency communications and ubiquitous connections new scenario requirements and goals. The core concept of the integrated computing model that integrates communication and computing resources is to deeply integrate communication and computing capabilities and break the boundaries that separate the two in traditional networks. Under this model, data is processed and analyzed in real time while being transmitted over the network, which greatly reduces data transmission delays and computing bottlenecks and improves the overall efficiency of the system. The types of services supported by large-scale satellite constellation computing are also becoming more and more diverse, and different services require different performance guarantees. However, the existing air computing offloading scenarios are mainly aimed at uniform business flows, without considering the actual emergencies, and rarely consider differentiated services. Providing differentiated services for non-uniform business flows can reasonably utilize communication and computing resources. Therefore, after integrating the satellite network, how to provide differentiated services for non-uniform business needs to minimize latency and energy consumption is a key issue that needs to be solved.

[0003] The patent document with application number 202311591342.0 discloses a "method for secure transmission and resource allocation of air edge computing data based on deep reinforcement learning". It takes into account the random mobility characteristics of users and the 3D trajectory flight characteristics of drones. It designs dynamic resource allocation strategies and the 3D trajectory of drones to minimize the computing cost of system users, and introduces the NOMA transmission strategy to improve the spectrum utilization efficiency of the system. In the case of uncertain aerial eavesdropping locations, it considers the worst security situation through the estimated eavesdropping range, and jointly designs the 3D trajectory of drones, the user's transmission power and computing frequency to minimize the long-term average network computing cost. While ensuring the security of user data, it reduces the computing delay and energy consumption of system users as much as possible, saving users' average computing costs. Since this method does not take into account the differences in the types of data that users unload, it does not provide differentiated services for different types of computing-intensive and delay-sensitive task data, and therefore fails to minimize latency and energy consumption overhead when allocating resources, and cannot guarantee the quality of service to users.

[0004] In their paper "Aerial Edge Computing on Orbit: ATaskOffloading and Allocation Scheme", Zhang Yuru et al., in response to the problem that users' requirements for latency and bandwidth are gradually increasing and traditional ground networks are no longer sufficient to support user needs, they use the characteristics of low propagation delay and low link loss of low earth orbit satellite networks, draw on the idea of ​​ground network edge computing, and deploy multi-access edge computing servers on the LEO satellite constellation to implement orbital edge computing technology. Based on the above motivation, this paper proposes a LEO satellite network OEC task allocation OECTA algorithm based on a greedy strategy for the Walker Delta satellite constellation. For two different types of tasks, it takes into account the different requirements for latency and energy consumption to achieve reasonable allocation and minimize resource utilization. However, this paper has two shortcomings: First, it only considers a few small-scale satellites and a few tasks to achieve one-to-one allocation of tasks to satellites, and does not take into account the large-scale satellites in actual situations and the large number of user demands. This will lead to a sharp increase in computational complexity and increased overhead if a greedy algorithm is used to make the optimal choice for each task when facing a large number of task demands. Second, it only considers the uniform arrival of services under static topology, but does not consider the dynamic arrival of non-uniform services under the dynamic topology of the air network. In reality, a large number of services are non-uniformly distributed, and the needs of different types of services are inconsistent. This will make it impossible to reasonably perform inter-satellite switching in actual large-scale constellation networks, resulting in a large number of non-uniform computing tasks generated by user terminals that cannot be reasonably scheduled to the target satellite, reducing the task completion rate. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the prior art and propose a large-scale constellation computing collaborative resource allocation method for non-uniform business needs, so as to effectively reduce the communication and computing resource overhead and improve the task completion rate by deeply integrating communication and computing capabilities.

[0006] To achieve the above object, the technical solution of the present invention comprises the following steps:

[0007] (1) Build a large-scale constellation network including low-orbit satellites, ground stations, and ground cloud management centers;

[0008] (2) The ground is divided into multiple mobility management areas, and a ground station is set up in each mobility management area to be responsible for the control of the area in which it is located; a low-orbit satellite coverage area is set up above the area to receive the service data generated by user terminals in the area;

[0009] (3) Taking T as the time slot length, the operation time of the large-scale constellation network is divided into N time slots, N ≥ 1;

[0010] (4) According to the definition of task offloading, bit is used as the minimum unit to measure the task amount, and the task completion rate is expressed as R = P / Q, where Q is the total number of tasks sent and P is the total number of tasks successfully received and executed;

[0011] (5) Divide the world into B location areas, and determine the central longitude and latitude information C of the area based on the global longitude and latitude division and the location area distribution. b = {lat b ,lon b}, b∈{1,2,......,B}, let the center longitude and latitude be the location of the ground station, and the longitude and latitude range of each location area is R b = {lat1 b ,lat2 b lon1 b ,lon2 b};

[0012] (6) Taking one of the location areas b as the reference, according to the orbital parameters such as the satellite's orbital period and inclination angle and the longitude and latitude C of the ground station b , derive the K visible time windows between its ground station and the visible M low-orbit satellites in is the start time of the jth visible time window between the ground station and the mth visible satellite, is the end time of the jth visible time window between the ground station and the mth visible satellite;

[0013] (7) The user terminal randomly generates data tasks, which are uploaded to the low-orbit satellite through the ground station or directly. The differentiated air task offloading DRL-DAO algorithm based on deep reinforcement learning is used to jointly optimize the dynamic resource allocation strategy for task offloading and data processing:

[0014] (7a) Assume that the task arrival time follows a Poisson distribution: λ is the arrival rate;

[0015] (7b) The ground cloud management center traverses all visible time windows of M low-orbit satellites according to the time slot i of the mission arrival:

[0016] If the time slot i is within the kth visible time window of the mth satellite, Then the subsequent time windows are no longer traversed and (7c) is executed;

[0017] Otherwise, continue to traverse the remaining visible time window of the satellite;

[0018] (7c) marking the satellite m as a visible satellite of the current time slot i and adding it to the current visible satellite list, and adding the time window k to the visible time window list of the current time slot;

[0019] (7d) The differentiated air task offloading (DRL-DAO) algorithm based on deep reinforcement learning is used to assist task scheduling and select the target satellite for offloading;

[0020] (7e) Query the remaining storage resources Stor of the unloading destination satellite m curr And the remaining visible time window resources T curr , observe whether the remaining storage resources are sufficient for task caching and whether the remaining visible time window resources are sufficient to execute the current task and return:

[0021] If the above two conditions are met, the current task is accessed and (7f) is executed;

[0022] Otherwise, discard the current task;

[0023] (7f) The destination satellite allocates bandwidth resources corresponding to the task according to the channel status, adds the task to the selected satellite queue, allocates the time window resources and storage resources required for execution to the task, and updates the satellite storage and queue status;

[0024] (7g) The tasks added to the satellite queue are divided into two categories: low-latency tasks and large-bandwidth tasks. Different task completion delay limits T are set according to the different delay tolerances of the task types. req , determine whether the task is successfully executed based on the upper limit of the delay:

[0025] If the task is completed within the upper limit of the delay, the task is successfully executed and a reward is given;

[0026] Otherwise, the task execution fails and a penalty is imposed;

[0027] (7h) releasing the storage resources and computing resources occupied by the task in step (7g) after execution, and transmitting the processed data back to the ground station according to the destination session user address;

[0028] (7i) When a new task arrives, the arrival time arr of the task is n Update the remaining time window and storage resources of the current satellite, complete the update of the current satellite status, and return to step (7b);

[0029] (8) After all tasks are offloaded, the low-orbit satellite m sends the global statistical data to the ground cloud management center to trigger the local statistical completion rate.

[0030] Compared with the prior art, the present invention has the following advantages:

[0031] 1. The present invention is aimed at a large-scale constellation network that integrates general computing and general operation. When scheduling tasks, the dynamic arrival of non-uniform tasks is taken into account. The task arrival obeys Poisson distribution and follows queuing calculation. In addition, the dynamic topology of the satellite network and the dynamic switching of visible satellites are taken into account. Therefore, it is helpful to deal with business emergencies in actual dynamic air networks, can reasonably schedule tasks, and has generalization.

[0032] 2. The present invention provides differentiated services according to different types of tasks in view of the differences in non-uniform business flow demands, which is not only conducive to the reasonable allocation of resources but also can improve the user service quality.

[0033] 3. The present invention aims to solve the problem that the optimization objective has variable coupling and a large decision solution space, and the movement of users and satellites causes dynamic changes in the environment, which is difficult to solve with traditional convex optimization methods. Taking into account the high-dimensional state space, the differentiated task air offloading DRL-DAO method based on deep reinforcement learning is adopted for task scheduling, which can maximize the task throughput and minimize the delay and computing energy consumption.

[0034] 4. Since the present invention divides the earth into different regions, based on the spatiotemporal correlation of channels and the correlation of tasks between adjacent regions, transfer learning is used to initialize the trained model, and then the network parameters are fine-tuned to obtain better task scheduling results. At the same time, since the present invention reasonably offloads tasks to different target satellites, a higher task completion rate is achieved, providing support for future air task scheduling and resource allocation solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is the overall flow chart of the implementation of the present invention;

[0036] Figure 2 is a large-scale constellation network diagram constructed in the present invention;

[0037] Figure 3 It is a structural diagram of the existing deep reinforcement learning DQN neural network used in the present invention;

[0038] Figure 4 This is a comparison chart of the number of tasks accepted by the present invention and the existing A3C-NDAO algorithm based on reinforcement learning;

[0039] Figure 5 It is a comparison chart of task completion rate simulation using the present invention and the existing A3C-NDAO algorithm based on reinforcement learning and the greedy algorithm. DETAILED DESCRIPTION

[0040] The present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0041] Reference Figure 1 , the implementation steps of this example are as follows:

[0042] Step 1: Build a large-scale constellation network.

[0043] like Figure 2 As shown in the figure, the large-scale constellation network constructed in this example includes low-orbit satellites LEO, user terminals UE, ground stations GS and ground cloud management centers TC. Among them: low-orbit satellites are responsible for accessing, processing and forwarding user data; user terminals include mobile phones, IoT devices or other mobile terminal nodes, which are used to generate mission data; ground stations are responsible for user information registration, maintaining the relationship between users and low-orbit satellites and inter-satellite switching decision control; the ground cloud management center is responsible for business management, periodically collecting, counting and analyzing business data, and adjusting the resource allocation and service strategy of large-scale constellation networks.

[0044] Step 2: Divide the management area and location area.

[0045] In order to enable the management center to centrally monitor the status and performance of each area and adjust the resource allocation strategy in a timely manner, it is necessary to divide the earth into multiple management areas and multiple location areas so that the system can more flexibly respond to real-time changing needs. In adjacent location intervals, the channel correlation and task correlation can be used to migrate the trained model to achieve more efficient resource allocation. The specific implementation includes the following:

[0046] 2.1) Division of management areas:

[0047] Divide the ground into N allMobility management areas: This example divides the ground into 12 mobility management areas based on its longitude and latitude. Each mobility management area is managed by a mobility management entity MG. The mobility management entity can be a medium-orbit satellite or a ground station. The basic principle of its selection is that when a ground station can be established in the mobility management area, the ground station is responsible for managing the area; when a ground station cannot be established in the mobility management area, the medium-orbit satellite is responsible for managing the area.

[0048] 2.2) Divide location area:

[0049] Each mobility management area is further divided into B la There are B location areas in the world: This example is based on the coverage radius of a single satellite of 574km, the satellite altitude of 550km, and the elevation angle of 40°. A single satellite network location area is divided into a coverage range of approximately 10°×10°, resulting in B=12*63=756 location areas in the world, where 63 is the number of satellite network location areas contained in each mobility management area, and 12 is the number of mobility management areas.

[0050] Step 3: Divide the operation time of the large-scale constellation.

[0051] With T as the time slot length, the operation time of the large-scale constellation network is divided into N time slots, where N ≥ 1.

[0052] In this embodiment, according to the set task arrival rate λ, it can be known that the task arrival interval is relatively dense, and the time slot length T is set to 1s, and the total number of time slots N is set to 7200.

[0053] Step 4: Define the task completion rate.

[0054] The existing task completion rate R'=P / Q' is defined based on the total task volume Q' generated by the user end and the number of tasks P successfully executed within the delay limit. In practical applications, the total task volume Q' generated by the user end and the number of tasks P successfully executed within the delay limit are often calculated by counting a time slot T v At the same time, since Q' is the total number of tasks generated by the user end in a time slot, when the network is congested or satellite resources are limited, the user may choose to perform tasks locally, so the task completion rate needs to be redefined.

[0055] In this embodiment, the global service statistics redefines the total task volume Q' generated by users as the total number of tasks Q that users choose to offload to the satellite, and redefines the task completion rate R as the satellite task execution completion rate R=P / Q.

[0056] Step 5: Use the improved deep reinforcement learning DRL-DAO algorithm to train the task scheduling and resource allocation model.

[0057] The DRL-DAO algorithm is improved on the basis of the existing deep reinforcement learning DQN algorithm, taking into account the differentiated requirements of task types. By performing differentiated processing on different types of tasks, user delay and energy consumption overhead are minimized. It is a differentiated air task offloading method.

[0058] The specific implementation of this step includes the following:

[0059] 5.1) Calculate the total delay between transmission and execution of task k generated by the user end:

[0060] 5.1.1) Let the size of the current task before processing be task1, the size of the task after processing be task2, and the computing power of the selected destination satellite be C m , calculate the energy consumption as Q m , server processing density η j , server processing energy consumption ε j .The energy consumption of transmitting from the ground station to the satellite is e gnd , the energy consumption of satellite transmission to the ground station is e sat , the propagation delay from the ground station to the satellite is t pro ;

[0061] 5.1.2) Calculate the transmission rate from user to satellite as r us and the transmission rate from satellite to user is r su :

[0062]

[0063] In the formula, B us represents the channel bandwidth of the user-satellite link, P u represents the user terminal transmit power, h us represents the channel fading coefficient, σ 2 represents the noise power, Λ represents the rain attenuation ratio; B su represents the channel bandwidth of the satellite-to-user link, P s represents the satellite transmission power, h su represents the channel fading coefficient, σ 2 represents the noise power, Λ represents the rain attenuation ratio;

[0064] 5.1.3) Calculate task processing delay:

[0065] 5.1.4) Calculate the task transmission delay: t tra =task1 / r us +task2 / r su ;

[0066] 5.1.5) The total delay for transmitting and executing task k generated by the user end is calculated based on the task processing delay and transmission delay:

[0067]

[0068] Among them, t wait is the waiting delay, t pro is the propagation delay;

[0069] 5.2) Calculate the total energy consumption of transmitting and executing task k generated by the user end:

[0070] 5.2.1) Calculate the energy consumption of task processing:

[0071] 5.2.2) Calculate the transmission energy consumption of the task:

[0072] 5.2.3) The total energy consumption of transmitting and executing task k generated by the user end is calculated based on the task processing energy consumption and transmission energy consumption:

[0073]

[0074] 5.3) Constructing differentiated air task scheduling and resource allocation model:

[0075] 5.3.1) Calculate the total latency and computing energy consumption of the system in time slot t C(t):

[0076]

[0077] Among them, M is the total number of low-orbit satellites, N t is the total number of tasks generated by the user terminal in the simulation time slot, β is the relative weight in J / s, which can be used to give different meanings to the computational cost and delay cost;

[0078] 5.3.2) Under the premise of ensuring the task completion rate, the weighted sum of the system user's delay and energy consumption is taken as the optimization target, and a joint optimization problem of dynamic resource allocation and task scheduling is constructed, which is expressed as follows:

[0079]

[0080]

[0081] in, is the actual maximum traffic that the MSC can handle in time slot t, L a is the average service demand of users in the current time slot, and the tasks are divided into enhanced mobile broadband eMBB services and ultra-reliable low-latency communication uRLLC services. For eMBB services, high bandwidth resources are required. B is the bandwidth resource allocated by MSC to this type of service in time slot t. a1 The actual bandwidth required for such services; for uRLLC services, ultra-low latency is required. is the delay of MSC in processing such services in time slot t, D a2 This is the delay that can be tolerated by this type of business.

[0082] 5.4) In order to minimize the system optimization objective, the resource allocation and task scheduling joint optimization problem is modeled as a Markov MDP decision process, that is, the optimization problem is transformed into<S,A,R> Triple, where S is the system state set, A is the visible satellite action set, and R is the reward function set;

[0083] A state S(n) of the system state set S in time slot n is expressed as:

[0084] S(n)=[[id1,T 1, Stor1,C1,Q1]...[id x ,T x, Stor x ,C x ,Q x ]...[id l ,T l, Stor l ,C l ,Q l ],[arr n ,type n ,P n ,L n ,req n ]]

[0085] Among them, id x is the index of the xth visible satellite, T x is the remaining execution time for the xth visible satellite, Stor x is the remaining storage resource of the xth visible satellite, C x is the computing power of the xth visible satellite, Q x is the computational energy consumption of the xth visible satellite, x∈{1,2...l}, l is the maximum number of visible satellites in the current time slot n, arr n is the arrival time of the current task, type n is the current task type, P n is the size before task execution, L n is the size after execution, req n The upper limit of task delay tolerance;

[0086] The satellite action set A, whose action index corresponds to the unloaded satellite;

[0087] The reward r(n) for executing task k in the reward function set R is expressed as:

[0088]

[0089] Among them, weight_time_delay is the delay weight, weight_energy_consumption is the energy consumption weight, and δ is the adjustment factor.

[0090] 5.5) Train the DQN neural network by learning the optimal strategy in the MDP to minimize the system optimization goal:

[0091] The DQN neural network includes a value network and a target value network, each of which includes four connection layers, where the number of neurons in the input layer is the size of the state dimension, and the number of neurons in the output layer is the size of the action space, such as Figure 3 shown.

[0092] The specific implementation of this step includes the following:

[0093] 5.5.1) The value network evaluates the action values ​​corresponding to all actions according to the current state s(n) of the input, and adopts a greedy strategy to select an output action a(n) and its corresponding action value Q(s(n),a(n); θ Q ) gets a reward of r and the next state s(n+1);

[0094] 5.5.2) Input state s(n+1) into the target value network and output the maximum action value Get the target action value y(n) of s(n):

[0095]

[0096] Among them, γ is the discount factor;

[0097] 5.5.3) According to the action value Q(s(n),a(n);θ Q ) and the target action value y(n) define the loss function of the DQN neural network as:

[0098]

[0099] Where N is the number of batches sampled in the buffer;

[0100] 5.5.4) Update network parameters:

[0101] The loss function is minimized by the following gradient descent formula:

[0102] ▽L(θ Q)=-▽E(y i (n)-Q(s i (n),a i (n); θ Q )) 2 ▽Q(s i (n),a i (n); θ Q )

[0103] Update the weights θ of the value network through backpropagation Q :

[0104] in, is the learning rate

[0105] Update the weights θ of the target value network using a soft update strategy Q' :

[0106] θ Q' ←τθ Q +(1-τ)θ Q' , where τ is the soft update parameter;

[0107] 5.5.5) Randomly extract small batches of sample data from the experience pool to train and update the value network and the target value network respectively, that is, repeat the above steps (5.5.1) to (5.5.4) to continuously optimize the weight parameters of the corresponding neural network until the cumulative reward value R tends to be stable to complete the network training process and obtain a trained task scheduling and resource allocation model.

[0108] Step 6: Use the trained task scheduling and resource allocation models to jointly optimize dynamic resource allocation.

[0109] 6.1) Deploy the trained model network to the base station platform, and obtain the satellite's orbit type, orbit altitude, inclination angle, and the longitude and latitude of the ground station C b Input the parameters into the STK analysis software, calculate the visibility between the satellite and the ground point, check the visibility time period, and obtain the .xml file of the visible time window between the ground station and M visible satellites. Read the file through the python software and traverse to generate the visible time window set.

[0110] 6.2) When the user generates a task, the ground cloud management center calculates the arrival time of the task n Check the current visible time window set to obtain visible satellites, input the visible satellite resource status and mission status into the trained model, and output the action index corresponding to the unloaded destination satellite;

[0111] 6.3) Query the remaining storage resources Stor of the unloading destination satellite m currAnd the remaining visible time window resources T curr , observe whether the remaining storage resources are sufficient for task caching and whether the remaining visible time window resources are sufficient to execute the current task and return:

[0112] If the above two conditions are met, the current task is accessed and (6.3) is executed;

[0113] Otherwise, discard the current task;

[0114] 6.4) The destination satellite allocates task transmission bandwidth resources according to the channel conditions, caches tasks and allocates required storage resources to the tasks, and updates the satellite queue status and storage status:

[0115] 6.4.1) According to the task bit size B n And the target satellite computing power C m Calculate the time T required to complete the task comp :

[0116]

[0117] where η j Processing density for servers;

[0118] 6.4.2) Obtain the mission start execution time T according to the destination satellite queue start , calculate the task completion time T end :

[0119] T end =T start +T comp ;

[0120] 6.4.3) T start and T end With array [T start ,T end ]Join the satellite alignment queue to get updated satellite queue status and storage status.

[0121] 6.5) Divide tasks into two categories: low-latency tasks and high-bandwidth tasks. Set different upper limits on task completion latency T according to the different tolerances of task types to latency. req , based on the fact that eMBB tasks have a high tolerance for latency and uRLLC tasks require ultra-low latency, the upper limit of the latency for eMBB tasks is set to T req1 Greater than the uRLLC delay upper limit T req2 , that is, T req2 <T req1 , determine whether the task is executed successfully:

[0122] If the task is completed within the upper limit of the delay, the task is successfully executed and a reward is given;

[0123] Otherwise, the task execution fails and a penalty is imposed;

[0124] 6.6) Release the storage resources and computing resources occupied by the task in step (6.4) after execution, and transmit the processed data back to the ground station according to the destination session user address;

[0125] 6.7) When a new task arrives, according to the arrival time arr of the task n Update the remaining time window and storage resources of the current satellite, complete the update of the current satellite status, and return to step (6.2);

[0126] 6.8) After all tasks are unloaded, complete the task scheduling and resource allocation for the current area;

[0127] 6.9) Use transfer learning to migrate the trained model to adjacent areas, take advantage of the spatiotemporal correlation of channels and task correlation between adjacent location areas, fine-tune the trained model network parameters by reducing the learning rate, and use the fine-tuned model to guide users to quickly select the destination satellite to offload tasks, thereby obtaining better task scheduling and resource allocation results.

[0128] Step 7: After all regional tasks are completed, the ground cloud management center triggers local statistics on the completion rate.

[0129] 7.1) The total number of successful missions that low-orbit satellite m will perform is P m and the total number of tasks received Q m Sent to ground station MG j :

[0130]

[0131] in, represents the total number of receiving tasks from location area b counted by low-orbit satellite m, represents the total number of successfully executed missions from location area b counted by low-orbit satellite m;

[0132] 7.2) Ground Station MG j Receive global statistics of low-orbit satellites in all location areas within the mobility management area a and summarize them into the total number of tasks Q received within the mobility management area a a and the total number of successfully executed tasks P a :

[0133]

[0134] in, represents the set of all low-orbit satellites passing over the mobility management area a during the simulation time slot, a∈{1,2,...,Nall},N all is the total number of mobile management areas;

[0135] 7.3) Ground Station MG j The total number of tasks Q received from all low-orbit satellites in area a a and the total number of successfully executed tasks P a The data is forwarded to the ground cloud management center, which calculates the task completion rate R of all areas. all and trigger vector C all :

[0136]

[0137] Among them, the trigger vector C all To record the task completion rate of an area;

[0138] 7.4) The ground cloud management center adjusts the network resource allocation strategy for the location areas included in the mobile management area based on local statistical data, and realizes the optimized management of constellation resources and task scheduling.

[0139] The effect of the present invention can be further illustrated by the following simulation results.

[0140] 1. Simulation software environment: STK 11x64, Python 3.7.0

[0141] 2. Simulation Content

[0142] Simulation 1, the present invention and the existing non-differentiated task offloading algorithm A3C-NDAO based on reinforcement learning respectively allocate resources to the computing type tasks generated by the user, so as to reasonably schedule the tasks and obtain the number of tasks successfully received by each satellite. The results are as follows Figure 4 shown.

[0143] from Figure 4 It can be seen that the learning process of the method adopted in the present invention converges quickly, starts to converge when the number of iterations is 50 rounds, and the number of received tasks tends to be stable; the A3C-NDAO algorithm converges slowly, and starts to converge and stabilize after 100 rounds, indicating that the present invention has low complexity and low computational overhead; at the same time, the convergence value of the present invention is higher, and the number of received tasks is significantly higher than that of A3C-NDAO, indicating that the present invention can reasonably allocate limited communication computing resources and improve user service quality.

[0144] Simulation 2, the present invention and the existing reinforcement learning-based non-differentiated task offloading algorithm A3C-NDAO and the greedy algorithm respectively allocate resources to the computing type tasks generated by the user to reasonably schedule the tasks and obtain the respective task completion rates when the number of tasks generated by the user is different. The results are as follows Figure 5 shown.

[0145] from Figure 5 It can be seen that when the number of tasks generated by users is 5000, the task completion rate of the present invention is 85%, the task completion rate of the greedy algorithm is 82%, and the task completion rate of the A3C-NDAO algorithm is 64%; when the number of tasks generated by users is 10000, the task completion rate of the present invention is 80%, the task completion rate of the greedy algorithm is 76%, and the task completion rate of the A3C-NDAO algorithm is 58%. This shows that the present invention significantly improves the task completion rate compared with the prior art.

[0146] The above description is only a specific example of the present invention and does not constitute any limitation to the present invention. It is obvious that for professionals in this field, after understanding the content and principles of the present invention, it is possible to make various modifications and changes in form and details without departing from the principles and structures of the present invention. However, these modifications and changes based on the ideas of the present invention are still within the scope of protection of the claims of the present invention.

[0147] It should be noted that the step numbers in the specification and claims of the present invention are only for the purpose of clearly describing the implementation scheme of the present invention and facilitating understanding, and the order of the step numbers is not limited.

Claims

1. A large-scale constellation computing collaborative resource allocation method for non-uniform business needs, characterized in that: The steps include: (1) Build a large-scale constellation network including low-orbit satellites, ground stations, and ground cloud management centers; (2) The ground is divided into multiple mobility management areas, and a ground station is set up in each mobility management area to be responsible for the control of the area in which it is located; a low-orbit satellite coverage area is set up above the area to receive the service data generated by user terminals in the area; (3) Taking T as the time slot length, the operation time of the large-scale constellation network is divided into N time slots, N ≥ 1; (4) According to the definition of task offloading, bit is used as the minimum unit to measure the amount of tasks, and the task completion rate is expressed as R = P / Q, where Q is the total amount of tasks sent and P is the total amount of tasks successfully received and executed; (5) Divide the world into B location areas, and determine the central longitude and latitude information C of the area based on the global longitude and latitude division and the location area distribution. b = {lat b ,lon b }, b∈{1,2,......,B}, let the center longitude and latitude be the location of the ground station, and the longitude and latitude range of each location area is R b = {lat1 b ,lat2 b lon1 b ,lon2 b }; (6) Taking one of the location areas b as the reference, according to the orbital parameters such as the satellite's orbital period and inclination angle and the longitude and latitude C of the ground station b , derive the K visible time windows between its ground station and the visible M low-orbit satellites in is the start time of the jth visible time window between the ground station and the mth visible satellite, is the end time of the jth visible time window between the ground station and the mth visible satellite; (7) The user terminal randomly generates data tasks, which are uploaded to the low-orbit satellite through the ground station or directly. The differentiated air task offloading DRL-DAO algorithm based on deep reinforcement learning is used to jointly optimize the dynamic resource allocation strategy for task offloading and data processing: (7a) Assume that the task arrival time follows a Poisson distribution: k=0,1,2..., λ is the arrival rate; (7b) The ground cloud management center traverses all visible time windows of M low-orbit satellites according to the time slot i of the mission arrival: If the time slot i is within the kth visible time window of the mth satellite, Then the subsequent time windows are no longer traversed and (7c) is executed; Otherwise, continue to traverse the remaining visible time window of the satellite; (7c) marking the satellite m as a visible satellite of the current time slot i and adding it to the current visible satellite list, and adding the time window k to the visible time window list of the current time slot; (7d) The differentiated air task offloading (DRL-DAO) algorithm based on deep reinforcement learning is used to assist task scheduling and select the target satellite for offloading; (7e) Query the remaining storage resources Stor of the unloading destination satellite m curr And the remaining visible time window resources T curr , observe whether the remaining storage resources are sufficient for task caching and whether the remaining visible time window resources are sufficient to execute the current task and return: If the above two conditions are met, the current task is accessed and (7f) is executed; Otherwise, discard the current task; (7f) The destination satellite allocates bandwidth resources corresponding to the task according to the channel status, adds the task to the selected satellite queue, allocates the time window resources and storage resources required for execution to the task, and updates the satellite storage and queue status; (7g) The tasks added to the satellite queue are divided into two categories: low-latency tasks and large-bandwidth tasks. Different task completion delay limits T are set according to the different delay tolerances of the task types. req , determine whether the task is successfully executed based on the upper limit of the delay: If the task is completed within the upper limit of the delay, the task is successfully executed and a reward is given; Otherwise, the task execution fails and a penalty is imposed; (7h) releasing the storage resources and computing resources occupied by the task in step (7g) after execution, and transmitting the processed data back to the ground station according to the destination session user address; (7i) When a new task arrives, the arrival time arr of the task is n Update the remaining time window and storage resources of the current satellite, complete the update of the current satellite status, and return to step (7b); (8) After all tasks are offloaded, the low-orbit satellite m sends the global statistical data to the ground cloud management center to trigger the local statistical completion rate.

2. The method according to claim 1, characterized in that In step (2), the ground is divided into multiple mobility management areas, which are divided into 12 mobility management areas based on the latitude and longitude of the ground. Each mobility management area contains 63 satellite network location areas.

3. The method according to claim 1, characterized in that In step (3), the world is divided into B location areas, based on the coverage radius of a single satellite of 574 km, the satellite altitude of 550 km, and the elevation angle of 40°. A single satellite network location area is divided into a coverage range of approximately 10°×10°, resulting in B=12*63=756 location areas worldwide, where 63 is the number of satellite network location areas included in each mobility management area, and 12 is the number of mobility management areas.

4. The method according to claim 1, characterized in that: Step (6) derives the K visible time windows of the ground station and the M visible low-orbit satellites, which is achieved by using the ground station and satellite visibility analysis software STK, that is, the orbit type, orbit altitude, inclination angle of the satellite and the longitude and latitude C of the ground station. b The parameters are input into the analysis software, the visibility between the satellite and the ground point is calculated, the visibility time period is checked, and the .xml file of the visible time window between the ground station and M visible satellites is obtained. The file is read by Python software, and an array list of visible time windows is generated by traversing.

5. The method according to claim 1, characterized in that In step (7d), the differentiated task offloading DRL-DAO algorithm based on deep reinforcement learning is used to assist task scheduling. The implementation steps include the following: (7d1) Based on the optimization goal of minimizing the weighted sum of latency and energy consumption of task offloading, a differentiated air task scheduling and resource allocation model is constructed: in, is the total delay and computing energy consumption in time slot t, M is the total number of low-orbit satellites, N t is the total number of tasks generated by the user terminal in the simulation time slot, is the total energy consumption of executing task k, is the total delay of transmission and calculation task k, β is the relative weight in J / s; is the actual maximum traffic that the large-scale satellite constellation MSC can handle in time slot t, L a is the average demand traffic of users in the current time slot, B is the bandwidth resource allocated by the large-scale satellite constellation MSC to the enhanced mobile bandwidth service in time slot t. a1 The actual bandwidth required for enhanced mobile bandwidth services; is the delay of MSC in processing ultra-reliable low-latency communication services in time slot t, D a2 The latency that can be tolerated by ultra-reliable low-latency communication services; (7d2) To minimize the system optimization objective, the task scheduling and resource allocation models are trained and the model is jointly optimized as a Markov decision MDP process 〈S, A, R〉, where S is the system state set, A is the visible satellite action set, and R is the reward function set; (7d3) A state s in the state set i Input DQN neural network generates an action a and returns a reward signal r and the next state s i ', through gradient descent and back propagation, the neural network parameters are iteratively updated to learn the optimal strategy in the MDP to achieve the goal of maximizing the reward, until the cumulative reward value R tends to a stable state, and the training stops to obtain a trained strategy network; (7d4) The trained strategy network is deployed to the base station platform. When the user generates a task, the satellite resource status and task status are input into the trained model, and the action index of the corresponding unloaded satellite is output to guide the user to quickly select the target satellite to unload the task, so as to minimize the system user delay and energy consumption.

6. The method according to claim 5, characterized in that The system state set S and reward function set R in the Markov decision process <S, A, R> are expressed as follows: S(n)=[[id1,T 1, Stor1,C1,Q1]...[id x ,T x, Stor x ,C x ,Q x ]...[id l ,T l, Stor l ,C l ,Q l ],[arr n ,type n ,P n ,L n ,req n ]] Among them, id x is the index of the xth visible satellite, T x is the remaining execution time for the xth visible satellite, Stor x is the remaining storage resource of the xth visible satellite, C x is the computing power of the xth visible satellite, Q x is the computational energy consumption of the xth visible satellite, x∈{1,2...l}, l is the maximum number of visible satellites in the current time slot n, arr n is the arrival time of the current task, type n is the current task type, P n is the size before task execution, L n is the size after execution, req n is the upper limit of task delay tolerance, S(n) is a state in time slot n of the state set; R is the reward function set; Among them, weight_time_delay is the delay weight, weight_energy_consumption is the energy consumption weight, and δ is the adjustment factor.

7. The method according to claim 1, characterized in that In step (7f), the time window resources and storage resources required for execution are allocated to the task, and the satellite storage and queue status are updated. The implementation steps include the following: (7f1) When the user terminal task arrives, the cache task is based on the task bit size B n And the target satellite computing power C m Calculate the time T required to complete the task comp : where η j Processing density for servers; (7f2) Get the task start time T according to the destination satellite queue start , calculate the task completion time T end : T end =T start +T comp (7f3) T start and T end With array [T start ,T end ]Join the satellite alignment queue to get updated satellite queue status and storage status.

8. The method according to claim 1, characterized in that Step (7g) sets different upper limits of task completion delay T according to the different tolerances of task types to delay req , is to divide the tasks into two categories: enhanced mobile bandwidth class eMBB and ultra-reliable low-latency communication class uRLLC. According to the high tolerance of eMBB tasks to latency and the ultra-low latency requirements of uRLLC tasks, the upper limit of the latency of the eMBB class is set T req1 Greater than the uRLLC delay upper limit T req2 , that is, T req2 <T req1 .

9. The method according to claim 1, characterized in that: In step (7d3), gradient descent and back propagation are used to iteratively update the neural network parameters. The implementation steps include the following: The DQN neural network includes a value network and a target value network, each of which includes four connection layers, where the number of neurons in the input layer is the size of the state dimension, and the number of neurons in the output layer is the size of the action space; The loss function of this network is: Among them, Q(s i (n),a i (n); θ Q ) Input the current state s for the value network i (n) Corresponding output action a i (n) value, Q'(s i (n+1),a i (n+1);θ Q’ ) is the target value network input next state s i (n+1) corresponds to output action a i The target value of (n+1), where N is the number of batches sampled in the buffer; Update the network: First update the weight θ of the value network by minimizing the above loss function Q ; Then according to the weight θ of the value network Q Use soft update strategy to update the weight θ of the target value network Q' : i Q’ ←tth Q +(1-τ)θ Q’ , Among them, τ is the soft update parameter.

10. The method according to claim 1, characterized in that In step (8), the low-orbit satellite m sends the global statistical data to the ground cloud management center to trigger the local statistical completion rate. The implementation steps include the following: (8a) The total number of successful missions that low-orbit satellite m will perform is P m and the total number of tasks received Q m Sent to ground station MG j ; in, represents the total number of receiving tasks from location area 1 counted by low-orbit satellite m, It represents the total number of successfully executed tasks from location area 1 counted by low-orbit satellite m; (8b) Ground Station MG j Receive global statistics of all low-orbit satellites in the mobility management area a and summarize them into the total number of tasks Q received in the mobility management area a a and the total number of successfully executed tasks P a : in, represents the set of all low-orbit satellites passing over the mobility management area a during the simulation time slot, a∈{1,2,...,N all },N all The total number of mobile management areas. (8c) Ground Station MG j The total number of tasks Q received from all low-orbit satellites in area a a and the total number of successfully executed tasks P a The data is forwarded to the ground cloud management center, which calculates the task completion rate R of all areas. all and trigger vector C all : Among them, the trigger vector C all To record the task completion rate of an area.

Images