Aircraft real-time resource allocation method and system

Abstract

The invention discloses an aircraft real-time resource allocation method and system, relates to the technical field of aircraft control, and is used for solving the technical problem of conflict between resource limitation and demand caused by aircraft resource allocation, and the aircraft real-time resource allocation method comprises the following steps: constructing a task priority queue and a resource state matrix of an aircraft, defining a state space; defining an action space, and defining a reward function according to the dynamic resource allocation target; building a real-time control software environment of the aircraft; preparing training data of a double-depth Q network, and constructing a double-depth Q network structure; constructing an experience playback buffer area, carrying out dual-depth Q network training, and updating parameters of a main network; obtaining a value function between the state and the action of the aircraft, and enabling the aircraft to make an optimal resource allocation decision through the value function; dynamic planning is used to refine and allocate resources in real time, maximization of task income under the condition of meeting resource constraints is achieved, and real-time resource allocation of the aircraft is completed.

Description

Technical Field

[0001] The present invention relates to the field of aircraft control technology, and more specifically, to a method and system for real-time resource allocation of an aircraft. Background Art

[0002] Aircraft real-time control systems are an important part of the modern aerospace field. Their functions cover key links such as flight attitude adjustment, path planning, and navigation control. With the increasing complexity of aircraft functions, the real-time and task diversity requirements faced by control systems are gradually increasing, which puts higher demands on the dynamic resource allocation and scheduling capabilities of aircraft real-time control software. The rationality of resource allocation directly affects the overall performance of the control system, especially in highly dynamic and complex mission environments. How to efficiently allocate limited computing resources to meet real-time challenges has become the focus and difficulty of current research.

[0003] The core requirements of aircraft real-time control software are concentrated on real-time performance and high efficiency. Real-time performance requires the system to respond quickly to changes in the external environment and adjustments to internal commands, which is crucial to ensuring flight safety. High efficiency is reflected in the need for the system to maximize task execution speed and resource utilization under limited processing power and storage resources to ensure that various control tasks can be completed at the best time to avoid delays or waste of resources. Dynamic resource allocation of aircraft real-time control software is a key technology to achieve real-time and high efficiency of aircraft real-time control software. The core of aircraft resource allocation lies in how to reasonably schedule limited computing resources, storage space and network bandwidth to support parallel processing of multiple tasks and rapid flow of data. However, the real-time resource allocation of existing aircraft real-time control software faces many challenges. The resources of aircraft, such as processor speed, memory size and energy consumption, have fixed upper limits, and the control software needs to complete complex data processing and task scheduling under these limited resources, which leads to conflicts between resource limitations and requirements, which may cause delays in key tasks, system crashes, and even safety accidents. Summary of the invention

[0004] The purpose of the present invention is to provide a method and system for real-time resource allocation of aircraft, which is used to solve the technical problem that resource limitation and demand conflict are caused by existing aircraft resource allocation. In view of this, the present invention is implemented through the following scheme.

[0005] In a first aspect, the present invention provides a method for real-time resource allocation of an aircraft, comprising: Construct the mission priority queue and resource state matrix of the aircraft and define the state space; define the action space by allocating the mission requirements of the aircraft to the overall system resources, and define the reward function based on the dynamic resource allocation goal; Build the real-time control software environment for the aircraft; prepare for dual depthQ The training data of the network and construct a double depth Q Network structure; Construct the experience replay buffer and perform the double depth Q Network training and updating the parameters of the main network; Obtaining a value function between the state and action of the aircraft, and enabling the aircraft to make an optimal decision on resource allocation through the value function; Use dynamic programming to allocate resources in real time and refine them, maximize mission benefits while satisfying resource constraints, and complete real-time resource allocation for aircraft.

[0006] Compared with the prior art, in the aircraft real-time resource allocation method of the present invention, the state space is defined by constructing the aircraft's task priority queue and resource state matrix, and the action space is defined by allocating the aircraft's task requirements to the system's overall resources, and the reward function is defined according to the dynamic resource allocation target; further, in constructing the dual-depth Q After the network structure and experience replay buffer, the double depth Q The network is trained and the parameters of the main network are updated. Then, by obtaining the value function between the state and action of the aircraft, the aircraft is enabled to make the optimal decision on resource allocation. Furthermore, dynamic programming is used to refine the allocation of resources in real time, maximize the mission benefits under the resource constraints, and complete the real-time resource allocation of the aircraft. Through the above technical solution, the present invention combines the adaptive learning ability of deep reinforcement learning for complex nonlinear problems and the rapid optimization advantage of dynamic programming in real-time decision-making, and realizes efficient real-time resource allocation through a two-way collaborative mechanism. Compared with the existing technology, the task completion rate, resource utilization rate, and average response time are significantly improved, and the aircraft's processing ability for the current resource limitations and requirements, and multi-task concurrency is enhanced; the adaptive strategy of the above technical solution of the present invention can dynamically adjust the algorithm parameters to adapt to environmental changes by real-time monitoring of the operating environment, thereby improving the adaptability of the aircraft in a complex environment. At the same time, the real-time resource allocation strategy based on reinforcement learning can be quickly optimized and updated by training iterations in a new environment when facing the evolving aircraft control algorithm, thereby supporting the efficient operation of the new algorithm and improving the performance of the aircraft; further, for emergencies encountered during the actual operation of the aircraft, the present invention can initiate an emergency response plan to ensure that the aircraft system can continue to operate or land safely when encountering an emergency task or a single point failure, and can respond quickly when encountering an emergency task, thereby significantly improving the safety performance and response capability of the aircraft. The present invention solves the technical problem of resource limitation and demand conflict caused by existing aircraft resource allocation through the above technical solution, and effectively avoids delays in key aircraft missions, system crashes, and even safety accidents.

[0007] Furthermore, in the aircraft real-time resource allocation method of the present invention, the step of constructing the aircraft's task priority queue and resource state matrix, and defining the state space, includes: The priority queue is defined and is represented by the following formula: ; in, P i Indicates the priority of the task. D i Indicates the deadline of the task. ε A buffer factor that represents task delays, used to avoid resource conflicts due to over-prioritization. W i represents the task weight; According to the current aircraft task queue Q and resource status matrix R Constructing the state space S ; Among them, the task queue , T n Represents a task queue Q The n aircraft missions; Resource Status Matrix , R c Represents system computing resources, R m Indicates system memory resources. R b Indicates system bandwidth resources; Mission the aircraft T i Assign to resources R j In the action space, A , action space ,in, Indicates Aircraft Mission T i Assign to resources R j Methods, resources , Indicates the allocated system computing resources, Indicates the allocated system memory resources. Indicates the allocated system bandwidth resources.

[0008] Furthermore, in the aircraft real-time resource allocation method of the present invention, the preparation of double depth Q The training data of the network includes: Prepare task data: Generate a task queue containing 1,000 tasks, and randomly assign requirements to each task; the requirements include task computing requirements, task memory requirements, and task bandwidth requirements; among them, task computing requirements ,unit MHz , task memory requirements , the unit is MB , task bandwidth requirements , the unit is Mbps , task deadline , the unit is s ; Preparing system resource data: total computing resources ; Total memory resources ; Total bandwidth resources ; and / or, the construction of double depth Q Network structure, including: Define the input layer: the task queue dimension is 10, the system resource status dimension is 3, and the total input layer dimension is 13; Define the hidden layer: There are two hidden layers, each with 128 neurons, and the activation function is ; Define the output layer: the dimension of the action set is 10.

[0009] Furthermore, in the aircraft real-time resource allocation method of the present invention, the experience playback buffer is constructed to perform the dual-depth Q Network training, including: Enter the state of each agent into a double depth Q The network, in the current action set space according to The greedy algorithm is used to select actions and the actions of all agents are applied to the environment. The action selection strategy can be expressed as: ;in, Indicates the current state. Indicates the optional actions in the current state. Indicates that the current state makes The biggest move, represents the exploration rate, , Indicates that the function is found When maximized, Represents the set of all possible actions in the current state; Introducing a target network, wherein the target network has the same structure as the main network and the two are independent of each other; Initialize the parameters of the main network and the target network; At each time step, according to the current state Use the main network to select an action ; Perform the selected action , observe the next state and instant rewards ; Determine the agent's training experience quadruple as , and the four-tuple Storing as training data in the experience playback buffer; And / or, updating the parameters of the primary network includes: Randomly sample a batch of experience quads from the experience replay buffer ; For any empirical quadruple sampled, use double depth Q The network estimates the current state value function With the next state The target value function ; Calculate the network parameters when minimizing the mean square error loss function and update the parameters of the main network.

[0010] Furthermore, in the aircraft real-time resource allocation method of the present invention, a batch of experience quadruplets are randomly sampled from the experience playback buffer. ; For any empirical quadruple sampled, use double depth Q The network estimates the current state value function With the next state The target value function ; Calculate the network parameters when minimizing the mean square error loss function and update the parameters of the main network, including: S100, use the target network to calculate the next state in the sampled experience quadruple The maximum value function Corresponding actions ; S200, use the main network to estimate the current state of the sampled experience quadruple The value function ; S300, use next state Execute an action The target value function Update current status The value function The value is: ; in, Indicates the current state. Indicates the optional actions in the current state. Indicates the current state QValue estimation, Indicates updated Q Value estimation, is the weight factor, and , Indicates the current state Execute an action After that, the reward value returned by the environment is represents the target value function, Indicates the next state of the current state. Indicates status Next Q The biggest move, is the discount factor, and ; S400, obtaining a mean square error loss function, whose formula is: ; in, represents the loss function, represents the expectation of all possible data distributions, Indicates the current state. Indicates the optional actions in the current state. Indicates in status Execute an action After that, the reward value returned by the environment is is the discount factor, and , Indicates the next state of the current state. Indicates status Next Q The biggest move, represents the parameters of the target network, Indicates the current state of the main network Q Value estimation, Represents the main network parameters, Indicates status The target network Q Value estimation; S500, update the parameters of the main network according to the stochastic gradient descent method, the formula is: ; in, represents the new parameter value of the main network after a gradient descent update, represents the parameter value of the main network at the beginning of the current iteration step, represents the learning rate, represents the loss function; is the gradient operator, which represents the parameter Find the gradient; S600, copying the parameters of the main network to the target network; S700, GPU parallel training for dual depth Q Network training is accelerated; And / or, enabling the aircraft to make an optimal decision on resource allocation through the value function includes: Using the double depth of training Q Network model, input system at a certain moment Status ; Double Depth Q The network model is based on the input state Output the best action , and obtain the optimal aircraft real-time resource allocation decision.

[0011] Furthermore, in the aircraft real-time resource allocation method of the present invention, after the aircraft makes the optimal resource allocation decision through the value function, the method further includes: S100 measures the contribution of a neuron to the result according to the gradient of the neuron to the loss function; the gradient of the neuron to the loss function is expressed by the following formula: ;in, , represents the network parameters associated with a neuron, represents the loss function, represents the gradient operator, which represents the parameter Find the gradient; S200, set a suitable pruning threshold according to the gradient of all neurons to the loss function, and delete neurons and links with low contribution based on this; S300 uses linear quantization technology to convert dual depth Q The weights represented by 32-bit floating point numbers in the network model are converted into 8-bit or 16-bit integers. The formula is: or ;in, Q Represents quantized fixed-point data, (•) indicates the rounding function, which is used to round the calculation result. Represents raw floating point data, represents the scaling factor, Indicates the offset; Set the maximum value of floating-point parameters and fixed-point parameters to , , , , then: ,in, represents the scaling factor, Indicates the maximum value of a floating-point parameter. Indicates the minimum value of a floating-point parameter. Indicates the maximum value of a fixed-point parameter, Indicates the minimum value of the fixed-point parameter; ,in, Indicates the offset, Indicates the maximum value of a fixed-point parameter, Indicates the maximum value of a floating-point parameter. Indicates the scaling factor.

[0012] Furthermore, in the aircraft real-time resource allocation method of the present invention, the use of dynamic programming to refine the allocation of resources in real time to achieve the maximum mission benefit under the resource constraint condition and complete the aircraft real-time resource allocation includes: For emergencies, estimate the amount of resources required for the emergencies and determine whether the reserved resources are sufficient; form a task queue with emergencies and non-critical tasks that release resources, and divide them into multiple task sub-queues according to priority; Perform initialization definition of dynamic programming algorithm; including definition of state variables in dynamic programming, definition of decision variables in dynamic programming, definition of dynamic programming state transfer matrix, and definition of task benefits in dynamic programming algorithm; Distributed computing is adopted, and the dynamic programming algorithm is used in each task sub-queue at the same time. The optimal decision of each stage is recorded recursively in order of priority from high to low, and then forward optimization is performed to finally obtain the optimal decision sequence of the original problem, thereby realizing the optimal strategy for allocating the remaining resources of the aircraft system for random burst tasks of the aircraft.

[0013] Furthermore, in the aircraft real-time resource allocation method of the present invention, the amount of resources required for the emergency is estimated, and whether the reserved resources are sufficient is determined; the emergency and the non-critical tasks for releasing resources are formed into a task queue, and divided into a plurality of task sub-queues according to priority, including: If the amount is sufficient, the reserved resources are used to deal with emergencies; if the amount is insufficient, the resources currently executing non-critical tasks are released and reallocated to critical tasks; The priority of each task is calculated according to the following formula: ; among them, among them, P i Indicates the priority of the task. D i Indicates the deadline of the task. ε A buffer factor that represents task delays, used to avoid resource conflicts due to over-prioritization. W i represents the task weight; Divide the task queue into priorities n Level task subqueue; And / or, the initialization definition of the dynamic programming algorithm includes defining the state variables in the dynamic programming, defining the decision variables in the dynamic programming, defining the dynamic programming state transfer matrix, and defining the task benefits in the dynamic programming algorithm, including: Get the total number of tasks in the task queue N ; Define the state variables in dynamic programming as ; Indicates allocation for i Tasks to The amount of resources for each task; Define the decision variables in dynamic programming as ; Indicates that the i The amount of resources for a task, ; And define the dynamic programming state transfer matrix, the formula is: ; Define the amount of resources allocated in the dynamic programming algorithm Give Tasks The profit function is , the formula is: ;in, Indicates the minimum amount of resources required to complete the task; Indicates that the The amount of resources; Indicates the task The weight of Represents the coefficient.

[0014] Furthermore, in the aircraft real-time resource allocation method of the present invention, the distributed computing is adopted, and the dynamic programming algorithm is used in each task subqueue at the same time, and the optimal decision of each stage is recorded recursively from high to low priority, and then the optimal decision sequence of the original problem is finally obtained, so as to realize the optimal strategy for the allocation of the remaining resources of the aircraft system for the random burst task of the aircraft, including: Distributed computing is used, with independent computing nodes processing dynamic programming algorithms and n The priority task sub-queues are processed and the allocation results are summarized at the same time; Use the dynamic programming recursive relationship to calculate step by step, and finally find the optimal solution to the problem; the formula is as follows: ; in, Indicates allocation for i Tasks to The amount of resources for a task, Indicates that the i The amount of resources for a task, Indicates the total number of tasks in the task queue. Indicates i Tasks in state The maximum profit under Indicates the amount of allocated resources Give Tasks The profit function, Indicates i+1 The maximum benefit of a task, Indicates The maximum benefit of a task, Represents the task in the dynamic programming algorithm The income is , Indicates that the The amount of resources for a task, Indicates allocation for The amount of resources for each task; A strategy combining heuristic search and dynamic programming is adopted to give priority to high-yield paths; the dynamic programming path is defined as the following formula: ,in, Indicates the number of tasks from the highest priority to the current priority. i The total actual benefit of the task; represents the total estimated revenue of the remaining tasks, Indicates the number of tasks from the highest priority to the current priority. i The tasks with the highest priority are i The comprehensive path benefit from the task to the last task of priority, j Indicates the number of remaining tasks.

[0015] In a second aspect, the present invention provides an aircraft real-time resource allocation system, comprising: The priority queue determination module is used to: construct the mission priority queue and resource state matrix of the aircraft and define the state space; define the action space by allocating the mission requirements of the aircraft to the overall system resources, and define the reward function according to the dynamic resource allocation target; Network structure building module, used to: build the real-time control software environment of the aircraft; prepare dual-depth Q The training data of the network and construct a double depth Q Network structure; The parameter update module of the main network is used to: construct an experience playback buffer and perform the dual depth Q Network training and updating the parameters of the main network; The optimal decision module is used to obtain the value function between the state and action of the aircraft, and enable the aircraft to make the optimal decision on resource allocation through the value function; The dynamic programming module is used to: use dynamic programming to refine resource allocation in real time, maximize mission benefits while satisfying resource constraints, and complete real-time resource allocation for aircraft.

[0016] Compared with the prior art, the beneficial effects of the aircraft real-time resource allocation system of the present invention are the same as the beneficial effects of the aircraft real-time resource allocation method described in the above technical solution, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 Double depth constructed for the present invention Q Schematic diagram of network structure; Figure 2 The double depth of the present invention Q Schematic diagram of the network training process; Figure 3 It is a schematic diagram of the overall process of the real-time resource allocation method for aircraft of the present invention; Figure 4 It is a bar chart diagram showing the performance comparison between the present invention and the prior art. DETAILED DESCRIPTION

[0018] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0019] It should be noted that when an element is referred to as being "fixed to" or "disposed on" another element, it can be directly on the other element or indirectly on the other element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element.

[0020] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined. The meaning of "several" is one or more, unless otherwise clearly and specifically defined.

[0021] The core requirements of aircraft real-time control software are concentrated on real-time performance and high efficiency. Real-time performance requires the system to respond quickly to changes in the external environment and adjustments to internal commands, which is crucial to ensuring flight safety. High efficiency is reflected in the need for the system to maximize task execution speed and resource utilization under limited processing power and storage resources to ensure that various control tasks can be completed at the best time to avoid delays or waste of resources. Dynamic resource allocation of aircraft real-time control software is a key technology to achieve real-time and high efficiency of aircraft real-time control software. The core of aircraft resource allocation lies in how to reasonably schedule limited computing resources, storage space and network bandwidth to support parallel processing of multiple tasks and rapid flow of data. However, the real-time resource allocation of existing aircraft real-time control software faces many challenges. The resources of aircraft, such as processor speed, memory size and energy consumption, have fixed upper limits, and the control software needs to complete complex data processing and task scheduling under these limited resources, which leads to conflicts between resource limitations and requirements, which may cause delays in key tasks, system crashes, and even safety accidents.

[0022] In order to solve the above technical problems, the present invention provides a method for real-time resource allocation of an aircraft, comprising: Construct the mission priority queue and resource state matrix of the aircraft and define the state space; define the action space by allocating the mission requirements of the aircraft to the overall system resources, and define the reward function based on the dynamic resource allocation goal; Build the real-time control software environment for the aircraft; prepare for dual depth Q The training data of the network and construct a double depth Q Network structure; Construct the experience replay buffer and perform the double depth Q Network training and updating the parameters of the main network; Obtaining a value function between the state and action of the aircraft, and enabling the aircraft to make an optimal decision on resource allocation through the value function; Use dynamic programming to allocate resources in real time and refine them, maximize mission benefits while satisfying resource constraints, and complete real-time resource allocation for aircraft.

[0023] In the case of adopting the above technical solution, in the real-time resource allocation method for aircraft of the present invention, the state space is defined by constructing the task priority queue and resource state matrix of the aircraft, and the action space is defined by allocating the task requirements of the aircraft to the overall resources of the system, and the reward function is defined according to the dynamic resource allocation target; further, in constructing the dual-depth Q After the network structure and experience replay buffer, the double depth QThe network is trained and the parameters of the main network are updated. Then, by obtaining the value function between the state and action of the aircraft, the aircraft is enabled to make the optimal decision on resource allocation. Furthermore, dynamic programming is used to refine the allocation of resources in real time, maximize the mission benefits under the conditions of satisfying resource constraints, and complete the real-time resource allocation of the aircraft. Through the above technical solution, the present invention combines the adaptive learning ability of deep reinforcement learning for complex nonlinear problems and the rapid optimization advantage of dynamic programming in real-time decision-making, and realizes efficient real-time resource allocation through a two-way collaborative mechanism. Compared with the existing technology, the task completion rate, resource utilization rate, and average response time are significantly improved, and the aircraft's processing ability for the current resource limitations and requirements, and multi-task concurrency is enhanced; the adaptive strategy of the above technical solution of the present invention can dynamically adjust the algorithm parameters to adapt to environmental changes by real-time monitoring of the operating environment, thereby improving the adaptability of the aircraft in a complex environment. At the same time, the real-time resource allocation strategy based on reinforcement learning can be quickly optimized and updated by training iterations in a new environment when facing the evolving aircraft control algorithm, thereby supporting the efficient operation of the new algorithm and improving the performance of the aircraft; further, for emergencies encountered during the actual operation of the aircraft, the present invention can initiate an emergency response plan to ensure that the aircraft system can continue to operate or land safely when encountering an emergency task or a single point failure, and can respond quickly when encountering an emergency task, thereby significantly improving the safety performance and response capability of the aircraft. The present invention solves the technical problem of resource limitation and demand conflict caused by existing aircraft resource allocation through the above-mentioned technical solution, and can effectively avoid delays in key aircraft missions, system crashes, and even safety accidents.

[0024] In order to better understand the present invention, the content of the present invention is further explained below in conjunction with specific embodiments, but the content of the present invention is not limited to the following embodiments.

[0025] Example 1

[0026] This embodiment provides a method for real-time resource allocation of an aircraft, including: Step 1: Construct the mission priority queue and resource state matrix of the aircraft and define the state space; define the action space by allocating the mission requirements of the aircraft to the overall system resources, and define the reward function according to the dynamic resource allocation goal; Step 2: Build the real-time control software environment for the aircraft; prepare for dual depth Q The training data of the network and construct a double depth Q Network structure; Step 3: Build the experience playback buffer and perform the double depth Q Network training and updating the parameters of the main network; Step 4, obtaining a value function between the state and action of the aircraft, and enabling the aircraft to make an optimal decision on resource allocation through the value function; Step 5: Use dynamic programming to refine resource allocation in real time to maximize mission benefits while satisfying resource constraints and complete real-time resource allocation for the aircraft.

[0027] Example 2

[0028] This embodiment provides a method for real-time resource allocation of an aircraft, including: S100, constructing a task priority queue and a resource state matrix of the aircraft, and defining a state space; defining an action space by allocating the task requirements of the aircraft to the overall resources of the system, and defining a reward function according to a dynamic resource allocation target; Furthermore, the construction of the mission priority queue and resource state matrix of the aircraft and the definition of the state space include: S101, defining the priority queue, wherein the priority queue is represented by the following formula: ; in, P i Indicates the priority of the task. D i Indicates the deadline of the task. ε A buffer factor that represents task delays, used to avoid resource conflicts due to over-prioritization. W i Indicates the task weight; S102, according to the current aircraft task queue Q and resource status matrix R Constructing the state space S ; Among them, the task queue , T n Represents a task queue Q The n aircraft missions; Resource Status Matrix , R c Represents system computing resources, R m Indicates system memory resources. R b Indicates system bandwidth resources; S103, the aircraft mission T i Assign to resources R j In the action space, A , action space ,in, Indicates Aircraft Mission T i Assign to resources R j Methods, resources , Indicates the allocated system computing resources, Indicates the allocated system memory resources. Indicates the allocated system bandwidth resources; Furthermore, for the above-mentioned reward function, this embodiment defines the reward function by introducing a comprehensive consideration of task completion time and task priority on the basis of the traditional maximization of resource utilization and task completion rate. r , the reward function r It is expressed by the following formula: ;in, r represents the reward function, represents the weight coefficient, Indicates the task weight, which can be adjusted according to actual needs. n Indicates the total number of tasks, represents the weight coefficient, Indicates the total system resources. Indicates the system resources required to complete the task. Indicates the number of completed tasks; is a Boolean variable representing the task status. When Success, when When Failure; Weight coefficient and Can be adjusted according to actual needs; S200, build the real-time control software environment of the aircraft; prepare for dual depth Q The training data of the network and construct a double depth Q Network structure; Further, the preparation of double depth Q The training data of the network includes: S201, prepare task data: generate a task queue containing 1000 tasks, and randomly assign requirements to each task; the requirements include task computing requirements, task memory requirements, and task bandwidth requirements; among which, task computing requirements ,unit MHz , task memory requirements , the unit is MB , task bandwidth requirements , the unit is Mbps , task deadline , the unit is s ; S202, prepare system resource data: total computing resources ; Total memory resources ; Total bandwidth resources ; For further information, see Figure 1 , the construction of double depth Q Network structure, including: S211, define the input layer: the task queue dimension is 10, the system resource status dimension is 3, and the total input layer dimension is 13; S212, define hidden layer: There are two hidden layers, each with 128 neurons, and the activation function is , activation function It refers to the ramp function in mathematics, which can be expressed as ,in, Indicates taking the maximum value, is the threshold value, Represents the input value; S213, define the output layer: the dimension of the action set is 10; S300, constructing an experience playback buffer and performing the dual-depth Q Network training and updating the parameters of the main network; For further information, see Figure 2 , the experience playback buffer is constructed, and the double depth Q Network training, including: S311, input the state of each agent into double depth Q The network, in the current action set space according to The greedy algorithm is used to select actions and the actions of all agents are applied to the environment. The action selection strategy can be expressed as: ;in, Indicates the current state. Indicates the optional actions in the current state. Indicates that the current state makes The biggest move, represents the exploration rate, , Indicates that the function is found When maximized , Represents the set of all possible actions in the current state; S312, introducing a target network, wherein the target network has the same structure as the main network and the two are independent of each other; S313, initializing parameters of the main network and the target network; S314, at each time step, according to the current state Use the main network to select an action ; S315, execute the selected action , observe the next state and instant rewards ; S316, determine the training experience quadruple of the agent: , and the four-tuple Storing as training data in the experience playback buffer; Further, the updating of the parameters of the primary network includes: S320, randomly sampling a batch of experience quadruplets from the experience playback buffer ; For any empirical quadruple sampled, use double depth Q The network estimates the current state value function With the next state The target value function ; Calculate the network parameters when minimizing the mean square error loss function and update the parameters of the main network; Further, step S320 specifically includes: S321, use the target network to calculate the next state in the sampled experience quadruple The maximum value function Corresponding actions ; S322, using the main network to estimate the current state of the sampled experience quadruple The value function ; S323, use the next state Execute an action The target value function Update current status The value function The value is: ; in, Indicates the current state. Indicates the optional actions in the current state. Indicates the current state Q Value estimation, Indicates updated Q Value estimation, is the weight factor, and , Indicates the current state Execute an action After that, the reward value returned by the environment is represents the target value function, Indicates the next state of the current state. Indicates status Next Q The biggest move, is the discount factor, and ; S324, obtaining a mean square error loss function, the formula of which is: ; in, represents the loss function, represents the expectation of all possible data distributions, Indicates the current state. Indicates the optional actions in the current state. Indicates in status Execute an action After that, the reward value returned by the environment is is the discount factor, and , Indicates the next state of the current state. Indicates status Next Q The biggest move, represents the parameters of the target network, Indicates the current state of the main network Q Value estimation, Represents the main network parameters, Indicates status The target network Q Value estimation; S325, update the parameters of the main network according to the stochastic gradient descent method, the formula is: ; in, represents the new parameter value of the main network after a gradient descent update, represents the parameter value of the main network at the beginning of the current iteration step, represents the learning rate, represents the loss function; is the gradient operator, which represents the parameter Find the gradient; S326, copying the parameters of the main network to the target network; S327, dual depth training through GPU parallel training Q The network training is accelerated; the above steps are repeated until the stopping condition is reached; Furthermore, in the above S327, it includes: S3271, setting is the total time steps of each training round, is the current time step, if ,but , jump to step S300, otherwise , jump to step S3272; for step S3272: if , is the total number of training rounds, is the current round number, then , jump to step S200, otherwise jump to step S3273; for step S3273: through iterative training, double depth Q The network finally obtains the value function between state and action , this function can guide the aircraft to make Q Real-time optimal resource allocation decision based on network; Furthermore, this embodiment uses pruning technology to delete parts with less impact on the prediction results by analyzing redundant neurons and invalid connections in the network; first, the contribution of each neuron is calculated by an importance evaluation method, and then neurons and connections with low contribution are deleted based on a set pruning threshold; specifically, as follows: S400, obtain the value function between the state and action of the aircraft; use the double depth completed by training Q Network model, input system at a certain moment Status ; Double Depth Q The network model is based on the input state Output the best action , get the optimal aircraft real-time resource allocation decision; S500 measures the contribution of the neuron to the result according to the gradient of the neuron to the loss function; the gradient of the neuron to the loss function is expressed by the following formula: ;in, , represents the network parameters associated with a neuron, represents the loss function, represents the gradient operator, which represents the parameter Find the gradient; S600, setting a suitable pruning threshold according to the gradient of all neurons to the loss function, and based on this, deleting neurons and links with low contribution; S700 uses linear quantization technology to convert dual depth Q The weights represented by 32-bit floating point numbers in the network model are converted into 8-bit or 16-bit integers. The formula is: or ;in, Q Represents quantized fixed-point data, (•) indicates the rounding function, which is used to round the calculation result. Represents raw floating point data, represents the scaling factor, Indicates the offset; Set the maximum value of floating-point parameters and fixed-point parameters to , , , , then: ,in, represents the scaling factor, Indicates the maximum value of a floating-point parameter. Indicates the minimum value of a floating-point parameter. Indicates the maximum value of a fixed-point parameter, Indicates the minimum value of the fixed-point parameter; ,in, Indicates the offset, Indicates the maximum value of a fixed-point parameter, Indicates the maximum value of a floating-point parameter. represents the scaling factor; S800, see Figure 3 , using dynamic programming to refine resource allocation in real time, maximize mission benefits under resource constraints, and complete real-time resource allocation for aircraft; Furthermore, the step S800 specifically includes: S810, for emergencies, estimate the amount of resources required for the emergencies and determine whether the reserved resources are sufficient; form a task queue with the emergencies and non-critical tasks for releasing resources, and divide them into multiple task sub-queues according to priority; S820, initializing and defining the dynamic programming algorithm; including defining the state variables in the dynamic programming, defining the decision variables in the dynamic programming, defining the dynamic programming state transfer matrix, and defining the task benefits in the dynamic programming algorithm; S830 uses distributed computing and a dynamic programming algorithm in each task subqueue. It recursively orders the priorities from high to low, records the optimal decision at each stage, and then searches forward to finally obtain the optimal decision sequence for the original problem, thus realizing the optimal strategy for allocating the remaining resources of the aircraft system for random burst tasks of the aircraft. Furthermore, step S810 specifically includes: S811, if sufficient, use the reserved resources to deal with emergencies; if insufficient, release the resources currently executing non-critical tasks and reallocate the resources to critical tasks; form a task queue with emergencies and non-critical tasks that release resources, and divide them into multiple sub-queues according to priority, and handle the total amount of currently remaining resources accordingly; S812, calculate the priority of each task according to the following formula: ; among them, among them, P i Indicates the priority of the task. D i Indicates the deadline of the task. ε A buffer factor that represents task delays, used to avoid resource conflicts due to over-prioritization. W i represents the task weight; S813, set standards and divide the task queue into priority groups n Level task subqueues; wherein, the standard settings can be set according to actual needs. For example, in this embodiment, tasks can be graded according to the absolute value of the priority, the top 20% is the first level task subqueue, 20-50% is the second level task subqueue, and 50-100% is the third level task subqueue; Further, step S820 specifically includes: S821, get the total number of tasks in the task queue N ; S822, define the state variables in dynamic programming as ; Indicates allocation for i Tasks to The amount of resources for each task; S823, define the decision variables in dynamic programming as ; Indicates that the i The amount of resources for a task, ; And define the dynamic programming state transfer matrix, the formula is: ; S824, define the amount of resources allocated in the dynamic programming algorithm Give Tasks The profit function is , the formula is: ;in, Indicates the minimum amount of resources required to complete the task; Indicates that the The amount of resources; Indicates the task The weight of represents the coefficient; Furthermore, step S830 specifically includes: S831 uses distributed computing, with independent computing nodes processing dynamic programming algorithms and n The priority task sub-queues are processed and the allocation results are summarized at the same time; S832, use the dynamic programming recursive relationship to calculate step by step, and finally find the optimal solution to the problem; the formula is as follows: ; in, Indicates allocation for i Tasks to The amount of resources for a task, Indicates that the i The amount of resources for a task, Indicates the total number of tasks in the task queue. Indicates i Tasks in state The maximum profit under Indicates allocation Resources for tasks The profit function, Indicates i+1 The maximum benefit of a task, Indicates The maximum benefit of a task, Represents the task in the dynamic programming algorithm The income is , Indicates that the The amount of resources for a task, Indicates allocation for The amount of resources for each task; S833, adopts a strategy combining heuristic search and dynamic programming, giving priority to high-yield paths; the dynamic programming path is defined as the following formula: ,in, Indicates the number of tasks from the highest priority to the current priority. i The total actual benefit of the task; represents the total estimated revenue of the remaining tasks, Indicates the number of tasks from the highest priority to the current priority. i The tasks with the highest priority are i The comprehensive path benefit from the task to the last task of priority, j Indicates the number of remaining tasks.

[0029] The real-time resource allocation method for aircraft in this embodiment adopts a distributed sensing network to divide the resource status into different regional subnets. Each subnet independently monitors and uploads resource usage data. The master node optimizes the allocation strategy based on the global resource status. The technical solution of this embodiment has an adaptive strategy. The adaptive strategy can dynamically adjust the algorithm parameters to adapt to environmental changes by real-time monitoring of the operating environment. Please refer to Figure 4 , Figure 4This is a bar diagram showing the performance comparison between this embodiment and other prior arts. Figure 4 It can be seen that the task completion rate of the technical solution of this embodiment reaches 97.8%, which is significantly higher than other algorithms, indicating that the dynamic resource allocation strategy of this embodiment effectively guarantees the completion of the task; the resource utilization rate of the allocation method of this embodiment is 92.5%, indicating that this embodiment can maximize the resource utilization efficiency under limited resource conditions; the response time of the allocation method is only 18.4 milliseconds, which is much lower than other algorithms, reflecting that the technical solution of this embodiment has significant advantages in real-time performance.

[0030] Furthermore, combined with Figures 1 to 3 The above technical solution of the present invention is described. Figure 1 Double depth constructed for the present invention Q Network structure diagram, from Figure 1 It can be seen that the double depth Q The network structure includes input layer, hidden layer, activation function and output layer. The state vector dimension is 13, the action vector dimension is 10, each hidden layer contains 128 neurons, and the activation function is ReLU function.

[0031] See also Figure 2 , Figure 2 Double depth Q Schematic diagram of the network training process, from Figure 2 It can be seen that the training experience quadruple is sampled from the experience replay buffer, based on the data of the experience replay buffer, using the dual depth Q Network (main network) estimates the current state Value Function And estimate the next state through the target network The target value function , and use the target value function Update current status The value function , through the target value function With the updated value function Calculate the minimum mean square error loss function, use the stochastic gradient descent method to update the parameters of the main network, and regularly copy the parameters of the main network to the target network; further, in the process of building the experience replay buffer, input the state of each agent into the double-depth Q Network (main network), in the current action set space according to The greedy algorithm is used to select actions (i.e. greedy action selection) and feedback to the environment; further, after the training is completed, the dual depth QThe lightweight design of the model is carried out based on the network model. The lightweight design includes neuron pruning technology and quantization technology. The pruning technology is to calculate the gradient of the mean square error loss function for the network parameters related to the neurons, and obtain the contribution of each neuron by comparing the absolute value of the gradient. The redundant neurons with low contribution and invalid connections are deleted based on the set pruning threshold; the quantization technology is to use the linear quantization technology to convert the double-depth Q The weights represented by 32-bit floating point numbers in the network model are converted to 8-bit or 16-bit integers.

[0032] See also Figure 3 , Figure 3 The overall process diagram of the real-time resource allocation method for an aircraft is shown in FIG. The existing aircraft may include a resource perception module, an intelligent decision-making module, and a task priority scheduling module. After the aircraft is started, it is determined whether an emergency occurs. If the determination is no ( N ), can be directly used through the intelligent decision module, using dual depth Q The optimal resource allocation strategy for network training completes resource allocation. If it is judged as yes ( Y ), on the one hand, through the resource perception module, the amount of resources required for the emergency is calculated, and whether the reserved resources are sufficient. On the other hand, through the task priority scheduling module, the priority of the emergency is evaluated, and the priority of the non-critical tasks of the removed resources is evaluated. Then, according to the priority evaluation, the task queue is divided into task sub-queues, and then distributed computing is adopted. The task sub-queues are dynamically programmed by independent computing nodes, and the allocation results are summarized at the same time, so as to realize real-time resource allocation; in the process of judging whether the reserved resources are sufficient, if the judgment is no ( N ), release the resources of the current non-critical tasks and reallocate them to the critical tasks. If the judgment is yes ( Y ), use reserved resources to solve emergency tasks; after the judgment is completed, it is processed by parallel processing technology to realize real-time resource allocation, that is, after the judgment is completed, sufficient resources are available, and then distributed computing is adopted. The task sub-queues are dynamically programmed by independent computing nodes, and the allocation results are summarized at the same time, thereby realizing a real-time resource allocation algorithm based on reinforcement learning and dynamic programming (that is, the real-time resource allocation method of the present invention).

[0033] In a second aspect, this embodiment provides an aircraft real-time resource allocation system, including: The priority queue determination module is used to: construct the mission priority queue and resource state matrix of the aircraft and define the state space; define the action space by allocating the mission requirements of the aircraft to the overall system resources, and define the reward function according to the dynamic resource allocation target; Network structure building module, used to: build the real-time control software environment of the aircraft; prepare dual-depth QThe training data of the network and construct a double depth Q Network structure; The parameter update module of the main network is used to: construct an experience playback buffer and perform the dual depth Q Network training and updating the parameters of the main network; The optimal decision module is used to obtain the value function between the state and action of the aircraft, and enable the aircraft to make the optimal decision on resource allocation through the value function; The dynamic programming module is used to: use dynamic programming to refine resource allocation in real time, maximize mission benefits while satisfying resource constraints, and complete real-time resource allocation for aircraft.

[0034] In the description of the above embodiments, specific features, structures, materials or characteristics may be combined in a suitable manner in any one or more embodiments or examples.

[0035] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for real-time resource allocation of an aircraft, characterized in that: include: Construct the mission priority queue and resource status matrix of the aircraft and define the state space; The action space is defined by allocating the mission requirements of the aircraft to the overall system resources, and the reward function is defined according to the dynamic resource allocation objective; Build the real-time control software environment for the aircraft; prepare for dual depth Q The training data of the network and construct a double depth Q Network structure; Construct the experience replay buffer and perform the double depth Q Network training and updating the parameters of the main network; Obtaining a value function between the state and action of the aircraft, and enabling the aircraft to make an optimal decision on resource allocation through the value function; Use dynamic programming to allocate resources in real time and refine them, maximize mission benefits while satisfying resource constraints, and complete real-time resource allocation for aircraft.

2. The method for real-time resource allocation of aircraft according to claim 1, characterized in that: The construction of the mission priority queue and resource state matrix of the aircraft and the definition of the state space include: The priority queue is defined and is represented by the following formula: ； in, P i Indicates the priority of the task. D i Indicates the deadline of the task. ε A buffer factor that represents task delays, used to avoid resource conflicts due to over-prioritization. W i represents the task weight; According to the current aircraft task queue Q and resource status matrix R Constructing the state space S ; Among them, the task queue , T n Represents a task queue Q The n aircraft missions; Resource Status Matrix , R c Represents system computing resources, R m Indicates system memory resources. R b Indicates system bandwidth resources; Mission of the aircraft T i Assign to resources R j In the action space, A , action space ,in, Indicates Aircraft Mission T i Assign to resources R j Methods, resources , Indicates the allocated system computing resources, Indicates the allocated system memory resources. Indicates the allocated system bandwidth resources.

3. The method for real-time resource allocation of aircraft according to claim 2, characterized in that: The preparation is double depth Q The training data of the network includes: Prepare task data: Generate a task queue containing 1,000 tasks, and randomly assign requirements to each task; the requirements include task computing requirements, task memory requirements, and task bandwidth requirements; among them, task computing requirements ,unit MHz , task memory requirements , the unit is MB , task bandwidth requirements , the unit is Mbps , task deadline , the unit is s ; Preparing system resource data: total computing resources ; Total memory resources ; Total bandwidth resources ; The double depth of the construction Q Network structure, including: Define the input layer: the task queue dimension is 10, the system resource status dimension is 3, and the total input layer dimension is 13; Define the hidden layer: There are two hidden layers, each with 128 neurons, and the activation function is ; Define the output layer: the dimension of the action set is 10.

4. The method for real-time resource allocation of aircraft according to claim 3, characterized in that: The experience playback buffer is constructed to perform the double depth Q Network training, including: Enter the state of each agent into a double depth Q The network, in the current action set space according to The greedy algorithm is used to select actions and the actions of all agents are applied to the environment. The action selection strategy can be expressed as: ;in, Indicates the current state. Indicates the optional actions in the current state. Indicates that the current state makes The biggest move, represents the exploration rate, , Indicates that the function is found When maximized , Represents the set of all possible actions in the current state; Introducing a target network, wherein the target network has the same structure as the main network and the two are independent of each other; Initialize the parameters of the main network and the target network; At each time step, according to the current state Use the main network to select an action ; Perform the selected action , observe the next state and instant rewards ; Determine the agent's training experience quadruple as , and the four-tuple Storing as training data in the experience playback buffer; The updating of the parameters of the main network includes: Randomly sample a batch of experience quads from the experience replay buffer ; For any empirical quadruple sampled, use double depth Q The network estimates the current state value function With the next state The target value function ; Calculate the network parameters when minimizing the mean square error loss function and update the parameters of the main network.

5. The method for real-time resource allocation of aircraft according to claim 4, characterized in that: The randomly sampling of a batch of experience quadruplets from the experience playback buffer ; For any empirical quadruple sampled, using double depth Q The network estimates the current state value function With the next state The target value function ; Calculate the network parameters that minimize the mean square error loss function and update the parameters of the main network, including: S100, use the target network to calculate the next state in the sampled experience quadruple The maximum value function Corresponding actions ; S200, use the main network to estimate the current state of the sampled experience quadruple The value function ; S300, use next state Execute an action The target value function Update current status The value function The value is: ； in, Indicates the current state. Indicates the optional actions in the current state. Indicates the current state Q Value estimation, Indicates updated Q Value estimation, is the weight factor, and , Indicates the current state Execute an action After that, the reward value returned by the environment is represents the target value function, Indicates the next state of the current state. Indicates status Next Q The biggest move, is the discount factor, and ; S400, obtaining a mean square error loss function, whose formula is: ； in, represents the loss function, represents the expectation of all possible data distributions, Indicates the current state. Indicates the optional actions in the current state. Indicates in status Execute an action After that, the reward value returned by the environment is is the discount factor, and , Indicates the next state of the current state. Indicates status Next Q The biggest move, represents the parameters of the target network, Indicates the current state of the main network Q Value estimation, Represents the main network parameters, Indicates status The target network Q Value estimation; S500, update the parameters of the main network according to the stochastic gradient descent method, the formula is: ； in, represents the new parameter value of the main network after a gradient descent update, represents the parameter value of the main network at the beginning of the current iteration step, represents the learning rate, represents the loss function; is the gradient operator, which represents the parameter Find the gradient; S600, copying the parameters of the main network to the target network; S700, GPU parallel training for dual depth Q Network training is accelerated; The step of enabling the aircraft to make an optimal decision on resource allocation by using the value function includes: Using the double depth of training Q Network model, input system at a certain moment Status ; Double Depth Q The network model is based on the input state Output the best action , and obtain the optimal aircraft real-time resource allocation decision.

6. The method for real-time resource allocation of aircraft according to claim 1, characterized in that: After the aircraft makes the optimal decision on resource allocation through the value function, the method further includes: S100 measures the contribution of a neuron to the result according to the gradient of the neuron to the loss function; the gradient of the neuron to the loss function is expressed by the following formula: ;in, , represents the network parameters associated with a neuron, represents the loss function, represents the gradient operator, which represents the parameter Find the gradient; S200, set a suitable pruning threshold according to the gradient of all neurons to the loss function, and delete neurons and links with low contribution based on this; S300 uses linear quantization technology to convert dual depth Q The weights represented by 32-bit floating point numbers in the network model are converted into 8-bit or 16-bit integers. The formula is: or ;in, Q Represents quantized fixed-point data, (•) indicates the rounding function, which is used to round the calculation result. Represents raw floating point data, represents the scaling factor, Indicates the offset; Set the maximum value of floating-point parameters and fixed-point parameters to , , , , then: ,in, represents the scaling factor, Indicates the maximum value of a floating-point parameter. Indicates the minimum value of a floating-point parameter. Indicates the maximum value of a fixed-point parameter, Indicates the minimum value of the fixed-point parameter; ,in, Indicates the offset, Indicates the maximum value of a fixed-point parameter, Indicates the maximum value of a floating-point parameter. Indicates the scaling factor.

7. The method for real-time resource allocation of aircraft according to claim 1, characterized in that: The method of using dynamic programming to allocate resources in real time and refine them, maximizing mission benefits under resource constraints, and completing real-time resource allocation for aircraft includes: For emergencies, estimate the amount of resources required for the emergencies and determine whether the reserved resources are sufficient; form a task queue with emergencies and non-critical tasks that release resources, and divide them into multiple task sub-queues according to priority; Perform initialization definition of dynamic programming algorithm; including definition of state variables in dynamic programming, definition of decision variables in dynamic programming, definition of dynamic programming state transfer matrix, and definition of task benefits in dynamic programming algorithm; Distributed computing is adopted, and the dynamic programming algorithm is used in each task sub-queue at the same time. The optimal decision of each stage is recorded recursively in order of priority from high to low, and then forward optimization is performed to finally obtain the optimal decision sequence of the original problem, thereby realizing the optimal strategy for allocating the remaining resources of the aircraft system for random burst tasks of the aircraft.

8. The method for real-time resource allocation of aircraft according to claim 7, characterized in that: For emergencies, the amount of resources required for emergencies is estimated, and it is determined whether the reserved resources are sufficient; emergencies and non-critical tasks for releasing resources are combined into a task queue, and divided into multiple task sub-queues according to priority, including: If the amount is sufficient, the reserved resources are used to deal with emergencies; if the amount is insufficient, the resources currently executing non-critical tasks are released and reallocated to critical tasks; The priority of each task is calculated according to the following formula: ;in, P i Indicates the priority of the task. D i Indicates the deadline of the task. ε A buffer factor that represents task delays, used to avoid resource conflicts due to over-prioritization. W i represents the task weight; Divide the task queue into priorities n Level task subqueue; The initialization definition of the dynamic programming algorithm includes defining the state variables in the dynamic programming, defining the decision variables in the dynamic programming, defining the dynamic programming state transfer matrix, and defining the task benefits in the dynamic programming algorithm, including: Get the total number of tasks in the task queue N ; Define the state variables in dynamic programming as ; Indicates allocation for i Tasks to The amount of resources for each task; Define the decision variables in dynamic programming as ; Indicates that the i The amount of resources for a task, ; And define the dynamic programming state transfer matrix, the formula is: ; Define the amount of resources allocated in the dynamic programming algorithm Give Tasks The profit function is , the formula is: ;in, Indicates the minimum amount of resources required to complete the task; Indicates that the The amount of resources; Indicates the task The weight of Represents the coefficient.

9. The method for real-time resource allocation of aircraft according to claim 7, characterized in that: The distributed computing is used, and the dynamic programming algorithm is used in each task subqueue at the same time, and the optimal decision of each stage is recorded recursively from high to low priority, and then the optimal decision sequence of the original problem is finally obtained, so as to realize the optimal strategy for allocating the remaining resources of the aircraft system for the random burst task of the aircraft, including: Distributed computing is used, with independent computing nodes processing dynamic programming algorithms and n The priority task sub-queues are processed and the allocation results are summarized at the same time; Use the dynamic programming recursive relationship to calculate step by step, and finally find the optimal solution to the problem; the formula is as follows: ; in, Indicates allocation for i Tasks to The amount of resources for a task, Indicates that the i The amount of resources for a task, Indicates the total number of tasks in the task queue. Indicates i Tasks in state The maximum profit under Indicates the amount of allocated resources Give Tasks The profit function, Indicates i+1 The maximum benefit of a task, Indicates The maximum benefit of a task, Represents the task in the dynamic programming algorithm The income is , Indicates that the The amount of resources for a task, Indicates allocation for The amount of resources for each task; A strategy combining heuristic search and dynamic programming is adopted to give priority to high-yield paths; the dynamic programming path is defined as the following formula: ,in, Indicates the number of tasks from the highest priority to the current priority. i The total actual benefit of the task; represents the total estimated revenue of the remaining tasks, Indicates the number of tasks from the highest priority to the current priority. i The tasks with the highest priority are i The comprehensive path benefit from the task to the last task of priority, j Indicates the number of remaining tasks.

10. A real-time resource allocation system for aircraft, characterized in that: include: The priority queue determination module is used to: construct the mission priority queue and resource state matrix of the aircraft and define the state space; define the action space by allocating the mission requirements of the aircraft to the overall system resources, and define the reward function according to the dynamic resource allocation target; Network structure building module, used to: build the real-time control software environment of the aircraft; prepare dual-depth Q The training data of the network and construct a double depth Q Network structure; The parameter update module of the main network is used to: construct an experience playback buffer and perform the dual depth Q Network training and updating the parameters of the main network; The optimal decision module is used to obtain the value function between the state and action of the aircraft, and enable the aircraft to make the optimal decision on resource allocation through the value function; The dynamic programming module is used to: use dynamic programming to refine resource allocation in real time, maximize mission benefits while satisfying resource constraints, and complete real-time resource allocation for aircraft.

Images