Aircraft-oriented low-altitude task unloading method, device and system

Abstract

The invention discloses an aircraft-oriented low-altitude task unloading method, device and system, and relates to the field of aircrafts. The method comprises the following steps: collecting current flight state information of an aircraft; establishing a task unloading constraint optimization problem according to the current flight state information of the aircraft; constructing a deep reinforcement learning network model required by the deep reinforcement learning and task unloading scene; and solving the task offload constraint optimization problem by using the deep reinforcement learning network model to obtain a low-altitude task offload resource allocation planning result. According to the invention, by designing an intelligent resource planning algorithm based on deep reinforcement learning and actively learning a mapping relation between each task and a multi-dimensional resource in different scheduling time slots, cooperative adaptation from task data to low-altitude computing power equipment can be realized more quickly and efficiently; and the unloading success rate of the electric vertical take-off and landing aircraft task in each scheduling time slot is improved.

Description

Technical Field

[0001] The present invention relates to the field of aircraft technology, and in particular to a method and device for unloading low-altitude tasks from an aircraft. Background Art

[0002] In order to further develop urban transportation resources and alleviate the contradiction between the rapidly increasing ground transportation tools and limited urban road resources, electric vertical take-off and landing aircraft have become a research hotspot for urban transportation. To ensure the normal control and operation of electric vertical take-off and landing aircraft, the low latency and ultra-high reliability of data transmission during the communication process of electric vertical take-off and landing aircraft are the focus of existing research. By rationally planning resources such as time slots and frequencies on the communication link, the transmission efficiency of signaling data can be improved.

[0003] However, in order to carry out emerging industry applications, electric vertical take-off and landing aircraft often generate computing-intensive tasks. Task data requires a large amount of computing power and storage resources, and requires real-time edge computing devices to offload tasks and perform calculations. Existing methods for link transmission resource planning cannot meet the needs of electric vertical take-off and landing aircraft with strict requirements on computing and storage performance. Therefore, task offloading planning for electric vertical take-off and landing aircraft requires joint consideration of the coordinated allocation of multi-dimensional resources such as link transmission, computing, and storage.

[0004] Existing research on task unloading of electric vertical take-off and landing aircraft focuses on the allocation of wireless network link transmission resources such as time slots and frequencies to reduce data transmission delays. However, in practical applications, the allocation of computing, storage and other resources in the low-altitude network will also affect the overall delay and reliability of the low-altitude task unloading process of the electric vertical take-off and landing aircraft. Therefore, in order to achieve the purpose of more comprehensively optimizing the efficiency of low-altitude task unloading of electric vertical take-off and landing aircraft in multiple resource dimensions such as transmission, computing and storage, the present invention constructs a low-altitude task unloading method and device for electric vertical take-off and landing aircraft. Summary of the invention

[0005] The present invention provides a low-altitude task unloading method for an aircraft, comprising:

[0006] Collect the current flight status information of the aircraft;

[0007] Establish a task offloading constraint optimization problem based on the current flight status information of the aircraft;

[0008] Build deep reinforcement learning network models required for deep reinforcement learning and task offloading scenarios;

[0009] The deep reinforcement learning network model is used to solve the task offloading constraint optimization problem and obtain the low-altitude task offloading resource allocation planning result.

[0010] As described above, a low-altitude task unloading method for aircraft is established, wherein establishing a task unloading constraint optimization problem is specifically to establish a target optimization function for task unloading of an electric vertical take-off and landing aircraft, that is, maximizing the total number of successfully unloaded tasks in all time slots, including the number of tasks successfully unloaded to the base station determined according to the current flight status information and the number of tasks successfully unloaded to the edge computing device.

[0011] As described above, a low-altitude task offloading method for an aircraft is described, wherein task offloading constraints are set for the task offloading constraint optimization problem, including communication connection constraints and resource allocation ratio constraints.

[0012] A low-altitude task offloading method for aircraft as described above, wherein, after establishing the task offloading constraint optimization problem, it also includes converting the task offloading constraint optimization problem into a Markov decision process, designing the state space, action space and reward of the Markov decision process; using the current flight state information of the aircraft as the state space of the Markov decision process; using the task offloading constraint conditions as the action space of the Markov decision process; and the reward is used to optimize the objective optimization function using a reward shaping method.

[0013] As described above, a low-altitude task offloading method for aircraft is described, wherein building a deep reinforcement learning network model includes building a policy estimation neural network and creating a state judgment neural network, the policy estimation neural network calculates the direction of the task offloading action based on the current flight state information of the aircraft, and the state judgment neural network is used to judge the value of the current flight state information of the aircraft.

[0014] A low-altitude task offloading method for an aircraft as described above, wherein a deep reinforcement learning network model is used to solve the task offloading constraint optimization problem, specifically including: updating the parameters of a strategy estimation neural network and a state judgment neural network through a cyclic iteration method;

[0015] Update the policy estimation neural network, specifically including: in each iteration time step, obtain the state space characteristics of the current time slot, generate the corresponding action decision through the policy estimation neural network, and interact with the environment to perform the action; transfer the state to the state of the next time slot, and obtain the reward of the current time step, and determine the action execution result according to the state of the next time slot and the task offloading requirements; at the end of the iteration, generate a trajectory sample set for each iteration, and the trajectory sample set includes the current state space characteristics, action decision, next state space characteristics, the reward corresponding to the current state, and the action execution result;

[0016] Updating the state judgment neural network specifically includes: collecting a trajectory sample set; and using each trajectory sequence in the trajectory sample set to update the parameters of the state judgment neural network.

[0017] The present invention also provides a low-altitude task unloading device for aircraft, comprising: the device executes any one of the above-mentioned low-altitude task unloading methods for aircraft.

[0018] The present invention also provides a low-altitude task unloading system for aircraft, including a base station, an electric vertical take-off and landing aircraft and a low-altitude aircraft; the low-altitude task unloading device is installed on the base station side, and the electric vertical take-off and landing aircraft and the low-altitude aircraft are equipped with edge computing equipment, so that the electric vertical take-off and landing aircraft and each low-altitude aircraft can be covered by the base station signal during the task unloading time period.

[0019] The present invention also provides a computer storage medium, characterized in that it comprises: at least one memory and at least one processor;

[0020] The memory is used to store one or more program instructions;

[0021] A processor is used to run one or more program instructions to execute any of the above-mentioned low-altitude task offloading methods for aircraft.

[0022] The beneficial effects achieved by the present invention are as follows: Different from traditional task offloading optimization algorithms based on dynamic programming, heuristic algorithms, etc., the present invention designs an intelligent resource planning algorithm based on deep reinforcement learning. By actively learning the mapping relationship between tasks and multi-dimensional resources in different scheduling time slots, it can more quickly and efficiently realize the coordinated adaptation of task data to low-altitude computing equipment, and improve the unloading success rate of electric vertical take-off and landing aircraft tasks in each scheduling time slot. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0024] Figure 1 It is a schematic diagram of an application scenario of an embodiment of the present invention;

[0025] Figure 2 This is a flow chart of a low-altitude mission unloading method for an aircraft provided in Example 1 of the present application;

[0026] Figure 3 This is a schematic diagram of a low-altitude mission unloading device for an aircraft provided in Example 2 of the present application. DETAILED DESCRIPTION

[0027] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0028] Embodiment 1

[0029] Figure 1 An application scenario of an embodiment of the present invention is described. In this scenario, an electric vertical take-off and landing aircraft flies at a constant speed along a pre-set air corridor, with a flight altitude of about 300-500 meters from the ground. A low-altitude aircraft, such as a swarm of drones, is provided about 150 meters from the ground. Each drone is equipped with an edge computing device that can provide computing power, and a base station is provided on the ground. The base station is equipped with a high-performance edge server that provides high-performance computing services. The high-performance edge server is used to execute the following task unloading method; the electric vertical take-off and landing aircraft and each drone can be covered by the signal of the ground base station during the task unloading time period. This embodiment does not specifically limit the type of low-altitude aircraft, the number of aircraft, the flight altitude, and the terrain and landforms included in the electric vertical take-off and landing aircraft task unloading scenario.

[0030] The electric vertical take-off and landing aircraft has different computing-intensive tasks that need to be offloaded to the drone side or the base station side in each transmission time slot. The CPU computing power on the drone side is smaller than that on the base station side, and the computing time overhead is large. However, the drone is closer to the electric vertical take-off and landing aircraft, and the link transmission delay is short. Therefore, the overall return delay needs to be weighed.

[0031] The maximum frequency bandwidth, computing resources and storage resources that can be provided by the base station side are {2MHz, 30GHz, 6kb} respectively; the maximum frequency bandwidth, computing resources and storage resources that can be provided by the drone side are {10MHz, 250GHz, 50kb} respectively; the mission data size of the electric vertical take-off and landing aircraft is 0.5~1kb, the CPU computing frequency requirement is 50~100MHz, the backhaul delay requirement is 20~60ms, and the data transmission power is 20dBm.

[0032] The path loss from the electric vertical take-off and landing aircraft to the base station is L 1 =-40-35lg(D i,s ), where D i,s is the distance between the EVTOL aircraft and the base station; the path loss from the EVTOL aircraft to the UAV is L 2 =-30-35lg(D i,u ), where D i,uis the distance between the electric vertical take-off and landing aircraft and the UAV. In the above measurement process, the background noise power is set to -110dBm.

[0033] In order to solve the problem of joint allocation of resources such as transmission, computing and storage during low-altitude mission unloading of electric vertical take-off and landing aircraft, the embodiment of the present application provides a low-altitude mission unloading method for electric vertical take-off and landing aircraft, which is applied to a high-performance edge server equipped on the base station side, such as Figure 2 As shown, the low-altitude task unloading method includes the following steps:

[0034] Step 210: Collect the current flight status information of the aircraft;

[0035] The aircraft's current flight status information includes aircraft location, network and mission data information.

[0036] Specifically, the base station uniformly collects the location information of the electric vertical take-off and landing aircraft and the low-altitude aircraft providing edge computing services within its coverage area in each transmission time slot, and identifies the location of each aircraft in different time slots by establishing a horizontal, vertical, and vertical three-dimensional coordinate system. In addition, the base station uniformly collects network data information and mission data information within its coverage area in each transmission time slot. The network data information includes frequency bandwidth, number of low-altitude edge computing devices, computing power size, storage space size, etc.; the mission data information includes the size of mission data volume, CPU cycle requirements, maximum return delay requirements, etc.

[0037] Step 220: Establish a task offloading constraint optimization problem according to the current flight status information of the aircraft;

[0038] The task offloading constrained optimization problem includes the constraints optimization problems in three dimensions: transmission, computation and storage.

[0039] Specifically, the optimization goal of the electric vertical take-off and landing aircraft task unloading is established, that is, maximizing the total number of successfully unloaded tasks in all time slots. The formula is described as:

[0040]

[0041] Among them, Δ s (t) represents the number of tasks successfully offloaded to the base station in the tth time slot, t∈T, T is the total number of time slots, Δ u (t) represents the number of tasks successfully offloaded to the edge computing device in the tth time slot.

[0042] Since a task can be successfully offloaded only when both the return latency and storage space requirements of the task are met, Δ s (t) is related to the backhaul delay and storage resource allocation at the base station side, and the formula is described as:

[0043]

[0044] Among them, Θ i,s (t)∈{0,1} indicates whether the current task is offloaded to the base station side. is the return delay requirement of the ith task, is the i-th mission data size of the electric vertical take-off and landing aircraft, i∈Υ(t), Υ(t) is the total number of tasks, Γ(·) is used to indicate whether the service quality requirement of the offloaded task is met, and They represent the storage space size of the base station and the proportion of storage space allocated to the current task by the base station, T i,b (t) is the actual return delay of the i-th task data in the t-th time slot. This delay is equal to the total time spent on sending, calculating, and receiving data between the electric vertical take-off and landing aircraft and the edge computing device. The formula is expressed as:

[0045]

[0046] In formula (3), represents the uplink transmission delay required when the i-th task on the electric vertical take-off and landing aircraft is offloaded to the edge computing device in the t-th time slot;

[0047] When mission data is sent to the ground base station, the link transmission delay for:

[0048]

[0049] Among them, B s represents the frequency bandwidth of base station s, e i,s (t) represents the efficiency of the communication between the EVTOL aircraft and the base station in the tth time slot, P is the power of the EVTOL aircraft to send mission data, G i,s (t) is the link gain when the EVTOL vehicle communicates with the base station in the tth time slot, σ 2 is the power spectral density of white noise.

[0050] When mission data is sent to the edge computing device of the low-altitude aircraft device, the link transmission delay for:

[0051]

[0052] Among them, B f Indicates the frequency bandwidth provided by low-altitude aircraft equipment, e i,u (t) represents the efficiency of communication between the i-th electric vertical take-off and landing aircraft and the u-th low-altitude aircraft equipment in the t-th time slot, G i,u(t) represents the link gain when the i-th electric vertical take-off and landing aircraft communicates with the u-th low-altitude aircraft device in the t-th time slot, G i,v (t) represents the interference caused by the vth aircraft except the uth low-altitude aircraft to the i-th electric vertical take-off and landing aircraft in the t-th time slot.

[0053] In formula (3), when the base station calculates the task, T i,c (t) The calculation time is:

[0054]

[0055] in, represents the CPU cycles required for the i-th task on the electric vertical take-off and landing aircraft in the t-th time slot, is the computation frequency of the base station, It is the proportion of computing resources allocated by the base station to the current task in the tth time slot.

[0056] In formula (3), when computing tasks on the edge computing device of a low-altitude aircraft, T i,c (t) The calculation time is:

[0057]

[0058] in, Indicates the computing resources of the edge computing device of the low-altitude aircraft. It represents the proportion of computing resources allocated to the current task by the u-th low-altitude aircraft edge computing device in the t-th time slot.

[0059] In formula (3), It represents the downlink transmission delay required from the edge computing device to the electric vertical take-off and landing aircraft in the tth time slot. Considering that the amount of data fed back to the user on the electric vertical take-off and landing aircraft is generally small, and the edge computing device has a large transmission power, The value is much smaller than and T i,c (t), can be ignored in the calculation process.

[0060] It should be noted that the link transmission delay is related to the size of the transmitted task data and the link data rate, and can be calculated using the Shannon formula, which will not be elaborated here.

[0061] Similar to the above reasoning process, Δ u (t) can be described as:

[0062]

[0063] Among them, Θ i,u(t)∈{0,1} indicates whether the current i-th task is offloaded to the u-th low-altitude aircraft edge computing device, u∈Ω(t), Ω(t) represents the number of low-altitude aircraft edge computing devices in the t-th time slot, and They respectively represent the storage space size of the edge computing device of the low-altitude aircraft and the proportion of storage space allocated to the current task by each device.

[0064] The constraints of the task offloading process include communication connection conditions and resource allocation ratio conditions:

[0065] ① The communication connection condition requires that each task on the electric vertical take-off and landing aircraft can and can only be offloaded to an edge computing device in each time slot, that is, the formula must be satisfied:

[0066]

[0067] ② The resource allocation ratio condition requires that the sum of the ratios of frequency bandwidth, computing, and storage resources allocated to different tasks on each edge computing device cannot exceed 1.

[0068] For the edge computing server on the base station side, the constraints are described as:

[0069]

[0070] For the edge computing equipment on the low-altitude aircraft side, the constraints are described as:

[0071]

[0072] In the embodiment of the present application, the constrained optimization problem can be converted into a Markov decision process for calculation;

[0073] Specifically, the Markov decision process transformation includes state space, action space and reward.

[0074] The state space includes the position information of the electric vertical take-off and landing aircraft and the low-altitude aircraft in each time slot, the mission data size of the electric vertical take-off and landing aircraft, the CPU cycle requirement and the maximum return delay requirement. The formula is described as:

[0075] s(t)={x 1 (t),…,x N (t),y 1 (t),…,y N (t),z 1 (t),…,z N (t),d i size (t),d i cycle (t),di delay (t)} (12)

[0076] Among them, x 1 (t),…,x N (t),y 1 (t),…,y N (t),z 1 (t),…,z N (t) represents the position coordinate information of the electric vertical take-off and landing aircraft and the low-altitude aircraft in each time slot, represents the i-th mission data size of the electric vertical take-off and landing aircraft, represents the CPU cycles required for the i-th task on the electric vertical take-off and landing aircraft, Represents the return delay requirement of the i-th task.

[0077] The action space contains the communication connection indicator variables and the allocation ratio of various resources. The formula is described as:

[0078]

[0079] In order to optimize the objective function, the reward is designed by utilizing reward shaping as follows:

[0080]

[0081] Among them, α, β∈[0,1] are weight coefficients, and They are the return delay and storage space allocation reward when the current task data is unloaded to the edge computing device on the base station side, which are specifically described as:

[0082]

[0083] Similarly, and They are the return delay and storage space allocation reward when the current task data is unloaded to the edge computing device of the low-altitude aircraft, and are specifically described as:

[0084]

[0085] Step 230: construct a deep reinforcement learning network model required for deep reinforcement learning and task offloading scenarios;

[0086] Specifically, building a deep reinforcement learning network model includes building a strategy estimation neural network and a state judgment neural network.

[0087] The policy estimation neural network is used to calculate the corresponding action a(t) based on the current state s(t). The probability of generating an action under different states is πθ (a t |s t ), where θ is the parameter matrix of the policy estimation neural network. The policy estimation neural network consists of an input layer, an intermediate layer, and an output layer. The deep reinforcement learning algorithm uses the Adam optimizer with a learning rate of 0.0001. The intermediate layer contains two layers of fully connected neural networks and one layer of ReLU. The experience pool size is set to 2048, the sampling batch size is set to 64, and the discount factor is 0.95. Among them, the dimension of the input layer is equal to the feature dimension of s(t), and the dimension of the output layer is equal to the feature dimension of a(t). Specifically, the output layer is divided into four action groups, namely the offloading decision action group, the frequency resource decision action group, the computing resource decision action group, and the storage resource decision action group. The offloading decision action group is used to decide to which edge computing device the task of the electric vertical take-off and landing aircraft is to be offloaded. This action group has an embedded real domain extension layer to convert Θ i,s (t) and Θ i,u (t) is extended to the real number domain, and is denoted as R[Θ i,s (t)] and R[Θ i,u (t)], according to the communication connection constraints, the action group needs to decide the direction of the current task unloading according to the size of the real value. If R[Θ i,s (t)]>R[Θ i,u (t)], the task is offloaded to the base station side, otherwise the task is offloaded to the low-altitude aircraft side. Each resource decision action group is used to decide the proportion of the corresponding resources allocated to the current task. Each action group is embedded with a Softmax layer to meet the resource allocation ratio constraint. The policy estimation neural network contains M≤3 intermediate layers between the input layer and the output layer. The first M-1 intermediate layers are fully connected layers, and the Mth intermediate layer is a ReLU layer for nonlinear features.

[0088] The state judgment neural network is used to judge the value of the current state. It includes an input layer, an intermediate layer, and an output layer. The input layer dimension is equal to the feature dimension of s(t), the output layer dimension is equal to 1, and the output value is V μ (s t ), which is used to evaluate the value of the input state s(t). Where μ is the parameter matrix of the state evaluation neural network. The structure of the middle layer of the state evaluation neural network is the same as that of the middle layer of the policy estimation neural network.

[0089] Step 240: using a deep reinforcement learning network model to solve the task offloading constraint optimization problem, and obtaining a low-altitude task offloading resource allocation planning result;

[0090] The deep reinforcement learning network model is used to solve the task offloading constraint optimization problem, which includes the following sub-steps:

[0091] Step 241: design an intelligent solver algorithm structure based on deep reinforcement learning;

[0092] Specifically, the base station acts as an intelligent agent of the deep reinforcement learning algorithm. The algorithm updates the parameters of the policy estimation neural network and the state judgment neural network through a cyclic iteration. In each iteration time step, the agent observes the state space characteristics of the current time slot and generates the corresponding action decision a(t) through the policy estimation neural network. Then the agent interacts with the environment to perform actions, so that the state s(t) is transferred to the next state s(t+1), and the reward r(t) of the current step is obtained. The agent determines whether s(t+1) meets the task's return delay and storage space requirements. If it does not meet the task requirements, it returns done=True and ends the current round of iteration. Otherwise, it returns done=False and continues to iterate. The agent stores the trajectory {s(t), a(t), s(t+1), r(t), done} generated by each iteration in the experience pool and waits for sampling to update the neural network.

[0093] In the process of updating the neural network parameters, the agent collects trajectory samples from the experience pool in batches and in order to form a sample set containing multiple trajectory sequences, and then traverses each trajectory sequence from the set in turn to gradually update the parameters θ and μ. Specifically, the parameter θ uses the gradient ascent method to maximize the target J(θ):

[0094]

[0095] Where T is the step length of the trajectory sequence, t 0 is the initial step of the trajectory sequence. The clip(·) function is used to limit the return value between 1-ε and 1+ε to avoid excessive jitter during training. 0 is the old parameter before updating, and θ is the parameter after updating. is the advantage value, defined as:

[0096]

[0097] Among them, λ,ω∈[0,1] are discount factors, Indicates the step length starting from 0. V is the value function, μ 0 is the old parameter of the previous training step, μ represents the parameter of the state judgment neural network, μ is updated by minimizing the loss by the gradient descent method, and the loss function L(μ) is described as:

[0098]

[0099] Step 242, running the solver to solve the task offloading constraint optimization problem, and obtaining the low-altitude task offloading resource allocation planning result;

[0100] The low-altitude task offloading resource allocation planning results include but are not limited to the planning results of communication connections, channel bandwidth resources, computing resources and storage resources.

[0101] Embodiment 2

[0102] like Figure 3 As shown, Embodiment 2 of the present invention provides a low-altitude task unloading device for an aircraft, which is specifically arranged in a high-performance edge server on the base station side, and the device includes:

[0103] The flight status information collection module 310 is used to collect the current flight status information of the aircraft;

[0104] The task offloading constraint optimization problem establishing module 320 is used to establish the task offloading constraint optimization problem according to the current flight status information of the aircraft; establishing the task offloading constraint optimization problem is specifically to establish the target optimization function of the electric vertical take-off and landing aircraft task offloading, that is, to maximize the total number of successfully offloaded tasks in all time slots, including the number of tasks successfully offloaded to the base station and the number of tasks successfully offloaded to the edge computing device determined according to the current flight status information; and to set the task offloading constraint conditions for the task offloading constraint optimization problem, including the communication connection constraint conditions and the resource allocation ratio constraint conditions.

[0105] Preferably, after the task offloading constraint optimization problem establishment module is established, it also includes a Markov decision process conversion module, which is used to convert the task offloading constraint optimization problem into a Markov decision process, design the state space, action space and reward of the Markov decision process; use the current flight state information of the aircraft as the state space of the Markov decision process; use the task offloading constraint conditions as the action space of the Markov decision process; the reward is used to optimize the objective optimization function using the reward shaping method.

[0106] The deep reinforcement learning network model construction module 330 is used to construct the deep reinforcement learning network model required for deep reinforcement learning and task offloading scenarios; constructing the deep reinforcement learning network model includes constructing a policy estimation neural network and creating a state judgment neural network. The policy estimation neural network calculates the direction of the task offloading action based on the current flight state information of the aircraft, and the state judgment neural network is used to judge the value of the current flight state information of the aircraft.

[0107] The task offloading constraint optimization problem solving module 340 is used to solve the task offloading constraint optimization problem by using a deep reinforcement learning network model to obtain a low-altitude task offloading resource allocation planning result.

[0108] Specifically, a deep reinforcement learning network model is used to solve the task offloading constraint optimization problem, which includes: updating the parameters of the strategy estimation neural network and the state judgment neural network through a cyclic iteration method;

[0109] Update the policy estimation neural network, specifically including: in each iteration time step, obtain the state space characteristics of the current time slot, generate the corresponding action decision through the policy estimation neural network, and interact with the environment to perform the action; transfer the state to the state of the next time slot, and obtain the reward of the current time step, and determine the action execution result according to the state of the next time slot and the task offloading requirements; at the end of the iteration, generate a trajectory sample set for each iteration, and the trajectory sample set includes the current state space characteristics, action decision, next state space characteristics, the reward corresponding to the current state, and the action execution result;

[0110] Updating the state judgment neural network specifically includes: collecting a trajectory sample set; and using each trajectory sequence in the trajectory sample set to update the parameters of the state judgment neural network.

[0111] Corresponding to the above embodiment, an embodiment of the present invention provides a computer storage medium, including: at least one memory and at least one processor;

[0112] The memory is used to store one or more program instructions;

[0113] The processor is used for running one or more program instructions to execute a low-altitude task offloading method for an aircraft.

[0114] Corresponding to the above-mentioned embodiment, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer storage medium contains one or more program instructions, and the one or more program instructions are used by a processor to execute a low-altitude task unloading method for an aircraft.

[0115] The embodiments disclosed in the present invention provide a computer-readable storage medium, in which computer program instructions are stored. When the computer program instructions are executed on a computer, the computer executes the above-mentioned low-altitude task unloading method for an aircraft.

[0116] In the embodiment of the present invention, the processor may be an integrated circuit chip having the ability to process signals. The processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0117] The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present invention can be directly embodied as a hardware decoding processor for execution, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The processor reads the information in the storage medium and completes the steps of the above method in combination with its hardware.

[0118] The storage medium may be a memory, which may be, for example, a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memory.

[0119] Among them, the non-volatile memory can be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory.

[0120] The volatile memory may be a random access memory (RAM) which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDRSDRAM), enhanced synchronous DRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus DRAM (DRRAM).

[0121] The storage media described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memory.

[0122] Those skilled in the art will appreciate that in one or more of the above examples, the functions described in the present invention can be implemented using a combination of hardware and software. When software is used, the corresponding functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. Storage media can be any available media that can be accessed by general or special-purpose computers.

[0123] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solution of the present invention should be included in the scope of protection of the present invention.

Claims

1. A low-altitude mission unloading method for an aircraft, characterized in that: include: Collect the current flight status information of the aircraft; Establish a task offloading constraint optimization problem based on the current flight status information of the aircraft; Build deep reinforcement learning network models required for deep reinforcement learning and task offloading scenarios; The deep reinforcement learning network model is used to solve the task offloading constraint optimization problem and obtain the low-altitude task offloading resource allocation planning result.

2. A low-altitude mission unloading method for an aircraft as claimed in claim 1, characterized in that: Establishing the task offloading constraint optimization problem is specifically to establish the objective optimization function of electric vertical take-off and landing aircraft task offloading, that is, to maximize the total number of successfully unloaded tasks in all time slots, including the number of tasks successfully unloaded to the base station determined according to the current flight status information and the number of tasks successfully unloaded to the edge computing device.

3. A low-altitude mission unloading method for an aircraft as claimed in claim 2, characterized in that: Task offloading constraints are set for the task offloading constraint optimization problem, including communication connection constraints and resource allocation ratio constraints.

4. A low-altitude mission unloading method for an aircraft as claimed in claim 3, characterized in that: After establishing the task offloading constraint optimization problem, it also includes converting the task offloading constraint optimization problem into a Markov decision process, designing the state space, action space and reward of the Markov decision process; using the current flight state information of the aircraft as the state space of the Markov decision process; using the task offloading constraint conditions as the action space of the Markov decision process; and the reward is used to optimize the objective optimization function using a reward shaping method.

5. A low-altitude mission unloading method for an aircraft as claimed in claim 4, characterized in that: Building a deep reinforcement learning network model includes building a policy estimation neural network and creating a state judgment neural network. The policy estimation neural network calculates the direction of the task offloading action based on the current flight state information of the aircraft, and the state judgment neural network is used to judge the value of the current flight state information of the aircraft.

6. A low-altitude mission unloading method for an aircraft as claimed in claim 5, characterized in that: The deep reinforcement learning network model is used to solve the task offloading constraint optimization problem, including: updating the parameters of the strategy estimation neural network and the state judgment neural network through cyclic iteration; Update the policy estimation neural network, specifically including: in each iteration time step, obtain the state space characteristics of the current time slot, generate the corresponding action decision through the policy estimation neural network, and interact with the environment to perform the action; transfer the state to the state of the next time slot, and obtain the reward of the current time step, and determine the action execution result according to the state of the next time slot and the task offloading requirements; at the end of the iteration, generate a trajectory sample set for each iteration, and the trajectory sample set includes the current state space characteristics, action decision, next state space characteristics, the reward corresponding to the current state, and the action execution result; Updating the state judgment neural network specifically includes: collecting a trajectory sample set; and using each trajectory sequence in the trajectory sample set to update the parameters of the state judgment neural network.

7. A low-altitude mission unloading device for aircraft, characterized in that: include: The device executes a low-altitude mission unloading method for an aircraft as described in any one of claims 1-6.

8. A low-altitude mission unloading system for aircraft, characterized in that: It includes a base station, an electric vertical take-off and landing aircraft and a low-altitude aircraft; a low-altitude task unloading device as described in claim 7 is installed on the base station side, and edge computing equipment is provided on the electric vertical take-off and landing aircraft and the low-altitude aircraft side, so that the electric vertical take-off and landing aircraft and each low-altitude aircraft can be covered by the base station signal during the task unloading time period.

9. A computer storage medium, characterized in that include: at least one memory and at least one processor; The memory is used to store one or more program instructions; A processor, used to run one or more program instructions to execute a low-altitude task offloading method for an aircraft as described in any one of claims 1-6.

Images