A multi-agent target assignment method based on SDQN algorithm

Abstract

A multi-agent target allocation method based on the SDQN algorithm sets a training environment for the background of swarm aircraft strike operations; sets different reward function types and optimization targets according to different task indicators; initializes the training environment; the environment returns the initialized task decision result to SDQN, and SDQN inputs the state returned by the environment into the network; the network gives the evaluation Q value of the next decision result of the current decision result; selects the allocation result with the largest Q value and integrates it into the environment. The present invention also decomposes the reward signal r that can only be obtained after all step-by-step actions are executed; according to the situation of the problem in the actual environment, the reward signal r is decomposed after each step-by-step action is executed, and at the same time, it is also ensured that the sum of the rewards obtained by each step-by-step action after the decomposition is equal to the original reward r. The advantage of the present invention is that it can provide a rich reward signal for the algorithm while ensuring that the original reward remains unchanged, thereby reducing the difficulty of training.

Description

Technical Field

[0001] The present invention belongs to the field of aerospace, and relates to a target allocation method for cluster aircraft combat, specifically a multi-agent target allocation method based on the SDQN algorithm. Background Art

[0002] In modern warfare and security operations, the application of swarm aircraft is rapidly becoming a focus. Swarm aircraft operations utilize the collaborative operation of multiple drones to improve the flexibility and efficiency of performing complex tasks. These swarms can perform multiple tasks such as reconnaissance, surveillance, and strike, and are highly adaptive and robust. However, with the increasing complexity and real-time requirements of combat missions, how to effectively allocate targets has become an important research area.

[0003] In swarm aircraft combat, the target allocation problem involves how to quickly and effectively assign tasks to swarm aircraft units based on battlefield conditions. The complexity of this problem comes from the following factors: 1. Each aircraft has different performance characteristics, including speed, load capacity, etc. 2. The battlefield environment changes dynamically, and new threats and target information need to be processed in real time. 3. The collaborative operation between aircraft needs to consider communication limitations and mutual interference.

[0004] In recent years, deep reinforcement learning has made important progress. Mnih et al. combined deep neural networks with traditional Q learning algorithms and proposed the DeepQ-Network (DQN) algorithm. In the DQN algorithm, the algorithm will gradually pass the final reward forward, but the Q value (estimate of expected benefits) of the corresponding state can only be passed one step, so it takes a long time to pass across multiple steps. In addition, the Q value must be attenuated for each step, and the Q value is very small after multiple steps, which will bring great difficulties to the training of the algorithm. Therefore, the Sequential DeepQ-Network (SDQN) algorithm is used to improve the above problems. The algorithm model can be applied to a variety of combat scenarios, significantly improving the decision-making quality and reaction speed of cluster aircraft in continuous and dynamic tasks, enhancing the autonomy and adaptability of cluster aircraft in complex environments, and improving the overall task execution efficiency and success rate. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes a multi-agent target allocation method based on the SequentialDeepQ-Network (SDQN) algorithm in combination with adversarial scenarios, which is designed to solve the target allocation of drone clusters in dynamic and complex environments. By combining advanced reinforcement learning techniques, SDQN can effectively handle decision-making problems, allowing drones to make efficient action choices in continuous state space.

[0006] A multi-agent target allocation method based on SDQN algorithm, characterized by:

[0007] Step 1: Set up a training environment for the context of swarm aircraft strike operations;

[0008] Step 2: Set different reward function types and optimization objectives according to different task indicators;

[0009] Step 3: Initialize the training environment and randomly set an initial distribution result to output to the environment;

[0010] Step 4: The environment returns the initialization task decision result to SDQN, and SDQN inputs the state returned by the environment into the network;

[0011] Step 5: The network gives the evaluation Q value of the next decision result of the current decision result;

[0012] Step 6: Select the allocation result with the largest Q value and integrate it into the environment;

[0013] Step 7: The environment determines all the route trajectories and mission critical states according to the current allocation results, and obtains the reward value according to the route trajectories and mission critical states;

[0014] Step 8: Modify the current state to the next state according to the current goal, and return the next state and reward value to the SDQN algorithm;

[0015] Step 9: The SDQN algorithm uses the return value of the environment to update the network parameters, making the network's estimate of the Q value more accurate;

[0016] Step 10: Determine whether the task can be completed and whether the performance value meets the requirements. If so, end the training. If not, return to step 4.

[0017] The present invention also discloses a non-volatile storage medium, characterized in that the non-volatile storage medium includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the above method.

[0018] The present invention also discloses an electronic device, characterized in that it comprises a processor and a memory; the memory stores computer-readable instructions, and the processor is used to execute the computer-readable instructions, wherein the computer-readable instructions execute the above method when they are executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 Schematic diagram of the training environment;

[0020] Figure 2 This is a schematic diagram of the SDQN principle;

[0021] Figure 3 This is a schematic diagram of scenario 1;

[0022] Figure 4 This is a schematic diagram of scenario 2;

[0023] Figure 5 This is a training flow chart based on the SDQN algorithm.

[0024] Figure 6 This is the algorithm output diagram under scenario 1: "The number of aircraft is 3, attacking important targets";

[0025] Figure 7 This is a schematic diagram of the allocation results under scenario 1: "The number of aircraft is 3, attacking important targets";

[0026] Figure 8 This is the algorithm output diagram under scenario 2: "The number of aircraft is 6, and the attack is on the front secondary target";

[0027] Fig. 9 This is a schematic diagram of the allocation results under scenario 2: "The number of aircraft is 6, and the attack is on the front secondary target." DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] The present invention is a multi-agent target allocation method based on the SDQN algorithm. The SDQN method is described as follows:

[0030] The core idea of ​​the SDQN method is to realize that complex functions on high-dimensional space can be modeled by a neural network that outputs 1 dimension at a time. Specifically, the SDQN algorithm uses the next step prediction model on the discretized dimension to model the Q value (estimate of expected return) and strategy on the continuous space. By parameterizing the prediction model, the composition structure of the action space can be exploited during training, and the maximum value of the action space can be calculated.

[0031] Figure 2 The schematic diagram of SDQN is shown in Figure 1. Figure 2 The upper part is the original Markov decision process, where the agent is in state , perform the action , get rewarded , arrive at the state ,in , that is, the action is a vector in three-dimensional space. Figure 2 The lower part is the Markov decision process after adopting SDQN. Figure 2 The original action can be seen in It is divided into three actions , , that is, the split action is a vector in one-dimensional space. The original SDQN algorithm performs actions The reward r=0 when After execution, the same algorithm as the original one was obtained. Same rewards .

[0032] The following are the specific steps of the multi-agent target allocation method based on the SDQN algorithm:

[0033] Step 1: Set up a training environment for the context of swarm aircraft strike operations.

[0034] The training environment is set up against the backdrop of swarm aircraft strike operations. Figure 1 This is a diagram of the training environment. The yellow circle in the figure is the threat range of each enemy target. If the aircraft enters this range, there is a certain probability of being intercepted. The aircraft's penetration probability is divided into lateral penetration probability and longitudinal penetration probability. The lateral penetration probability refers to the probability of the aircraft passing through but not attacking the enemy target, and the longitudinal penetration probability refers to the probability of the aircraft passing through when attacking the enemy target. Figure 1 In the figure, D1 to D4 represent aircraft, T2-1 to T2-4 represent forward escort targets, and the lateral penetration probability and longitudinal penetration probability are , T2-5 to T2-8 are peripheral guard targets, and the lateral penetration probability and longitudinal penetration probability are .

[0035] If the target and angle of attack of the aircraft are known, the attack trajectory of the aircraft can be drawn. The attack trajectory of D1-D4 is as follows: Figure 1 shown.

[0036] For D1, since it attacks T2-1 directly without passing through the threat range of other enemy targets, the penetration probability is ;

[0037] For D2, it attacks T2-6 and passes through the threat range of T2-3, so the total penetration probability is ;

[0038] For D3, it attacks T2-4 and passes through the threat range of T2-2, so the total penetration probability is ;

[0039] For D4, it attacks T2-3 and passes through the threat range of T2-1. However, since T2-1 has been attacked, it is considered that it has no interception capability. The total penetration probability is .

[0040] This mechanism is set up to prevent all aircraft from attacking the enemy target with the highest value. For example, it is generally believed that the important target T1 has the highest value. In the case of maximizing the total reward, all aircraft may attack the important target. It is generally believed that attacking and destroying secondary enemy targets is conducive to better attacking important targets. Therefore, the above mechanism will encourage the allocation of a certain number of aircraft to attack the escort target. In addition, if a large number of aircraft are indeed needed to attack important targets, this can be achieved by increasing the reward value of important targets.

[0041] Step 2: Set different reward function types and optimization objectives according to different task indicators.

[0042] Based on the differences in task indicators, the following types of reward functions are defined to guide the target allocation of the vehicle:

[0043] Maximizing Damage Effectiveness: When the primary objective of the mission is to cause maximum damage to enemy targets, the reward function will focus on measuring the damage effectiveness of targets assigned to a specific aircraft. With this reward function, aircraft are preferentially assigned targets that can cause the most damage.

[0044] Minimize time to complete the task: If the key to the task is speed, that is, completing the task as quickly as possible, the reward function will reflect the time required to complete the entire task. The goal allocation will tend to minimize the completion time of all goals.

[0045] Minimize the cost of completing the task: When the cost of task execution becomes the main consideration, the corresponding reward function will evaluate the resources and costs required to complete the goal, with the aim of minimizing resource consumption and cost expenditure without sacrificing task effectiveness.

[0046] According to specific mission requirements, the above reward functions can be flexibly adjusted or combined to adapt to diverse combat environments and objectives.

[0047] The optimization goal is to maximize the overall efficiency under the guidance of the reward function when there are limited aircraft. Specifically, it is to maximize the sum of the corresponding rewards of all aircraft:

[0048] Maximizing overall effectiveness: By dynamically adjusting the target allocation strategy and evaluating the total reward corresponding to each possible allocation scheme, we can find the target allocation strategy that maximizes the overall reward.

[0049] State: Information reflecting the current target allocation, such as the targets assigned to each aircraft and their status. This information is used as input to the neural network to evaluate the current allocation strategy.

[0050] Action: Based on the current state, the network outputs the target number that the aircraft should select, thereby determining the optimal target assignment for each aircraft.

[0051] Through the above steps, it is ensured that the target allocation of each aircraft not only meets the needs of a single mission, but also promotes the achievement of the optimization goals of the entire group, thereby achieving efficient and economical mission execution in a complex combat environment.

[0052] Step 3: Initialize the training environment and randomly set an initial distribution result to output to the environment.

[0053] The purpose of this step is to create an initial state for the training process of the multi-agent target allocation algorithm, including how to initialize the training environment and how to randomly set an initial target allocation result.

[0054] At the beginning of training, in order to explore possible target allocation schemes, the initial target allocation results are generated by a random strategy, which means that each aircraft is randomly assigned a target without considering the actual combat efficiency.

[0055] The main reason for using random allocation is to ensure the breadth and diversity of algorithm training and avoid falling into local optimal solutions at the beginning of the training process. By exploring a large number of allocation combinations, the algorithm can more comprehensively learn and evaluate the effects of different allocation strategies.

[0056] After the training environment is initialized, the algorithm generates a series of random numbers and pairs the aircraft with the targets according to these random numbers. This process ensures that the target allocation scheme is different at the beginning of each training, providing the algorithm with rich learning materials.

[0057] Through the above steps, it is ensured that the initialization of the training environment and the setting of the initial target allocation results not only have clear processes and reasons, but also provide a solid foundation for the effective learning and optimization of the algorithm.

[0058] Step 4: The environment returns the initialized task decision result to SDQN, and SDQN inputs the state returned by the environment into the network.

[0059] The mission decision result refers to the various indicators obtained after the aircraft group performs the mission in the simulated environment according to the initial target allocation results in step three, such as mission completion time, mission cost, target damage degree, etc.

[0060] The simulation environment generates a series of measurement indicator values ​​based on the results of the above decision execution. These values ​​together constitute the task decision results and are used as input to the SDQN algorithm to evaluate the effectiveness of the current decision.

[0061] In the SDQN framework, the state refers to a set of parameters that can describe the current mission execution status, including the position of the aircraft, target status, remaining mission volume, consumed resources, etc.

[0062] The simulation environment integrates the above task decision results and related parameters to form a state vector and returns it to the SDQN algorithm. This state vector provides the algorithm with a snapshot of the current environment and task execution status.

[0063] The main reason for inputting the state returned by the environment into the SDQN network is to use this information to evaluate the effectiveness of the current target allocation strategy and guide the algorithm to learn and optimize. By analyzing the state information, SDQN can determine the best action strategy for the next step to improve the efficiency and effectiveness of task execution.

[0064] Step 5: The network gives the evaluation Q value of the next decision result under the current decision result.

[0065] The current decision result refers to the actual effect of the action strategy output by the SDQN network in step 4 after execution in the simulation environment, including task completion, resource consumption, time consumption, etc.

[0066] The Q value represents the sum of future rewards expected from taking an action in a given state. It is an important indicator for measuring the quality of a decision.

[0067] The SDQN network calculates the Q value of the next possible decision based on the current decision results (including the status and effect of task execution). This process involves the forward propagation of the network, estimating the Q value of each possible action through the learned parameters and weights.

[0068] The Q-value evaluation mechanism enables the algorithm to quantify the pros and cons of each decision-making plan, provides a basis for selecting the optimal action strategy, and is the key to achieving efficient target allocation.

[0069] Step 6: Select the allocation result with the largest Q value and integrate it into the environment.

[0070] Choosing the action strategy with the largest Q value is based on the core principle of reinforcement learning - maximizing future rewards. This method aims to select the decision that is most likely to achieve the best task effect.

[0071] The SDQN network compares the calculated Q values ​​of all possible actions, selects the action with the largest Q value as the optimal decision for the next step, and instructs the simulation environment to execute the task according to this decision.

[0072] By applying the selected optimal decision to a simulation environment, the actual effect of the decision can be intuitively observed, including task execution efficiency, resource utilization, etc., providing experimental data for further learning and adjustment of the algorithm.

[0073] Step 7: The environment determines all route trajectories and mission critical states based on the current allocation results, and the environment derives the reward value based on the route trajectories and mission critical states.

[0074] By analyzing the current target allocation results and the resulting route trajectories and key status of task execution, the actual performance of each decision plan can be accurately evaluated, including path optimization, task execution time and efficiency.

[0075] Based on the route trajectory and mission critical status, the environment calculates the corresponding reward value. This value reflects the fitness and efficiency of the current decision-making plan relative to the mission goal and is the key feedback of the algorithm optimization process.

[0076] The reward value provides a clear learning goal for the algorithm, that is, how to adjust the action strategy to maximize the reward. It is the driving force for the algorithm to improve itself, gradually adapt to the environment, and ultimately achieve the optimal target allocation strategy.

[0077] Step 8: The environment modifies the current state to the next state according to the current goal, and returns the next state and reward value to the SDQN algorithm.

[0078] The update of the state reflects the changes in the environment caused by the execution of the optimal decision, which is the basis for continuous adaptation and optimization of strategies in the reinforcement learning process.

[0079] Based on the reward value determined in step 7, the environment updates its internal state to reflect the actual impact of the current decision result. This new state includes task progress, remaining resources, aircraft position, etc., and then returns this state and the corresponding reward value to the SDQN algorithm.

[0080] The return of new states and reward values ​​is key to the algorithm’s learning process, enabling SDQN to evaluate the effectiveness of its decisions and adjust its strategy accordingly for more efficient task execution.

[0081] Step 9: The SDQN algorithm uses the return value of the environment to update the network parameters, making the network's estimate of the Q value more accurate.

[0082] Through the actual reward values ​​and state changes fed back by the environment, the SDQN algorithm is able to evaluate the accuracy and effectiveness of its previous actions, and then optimize its action strategy by adjusting the network parameters.

[0083] Using the returned state and reward values, the SDQN algorithm performs parameter updates, which typically involve the calculation of a loss function and optimization techniques such as gradient descent. These updates allow the network to more accurately predict Q values, which are the rewards that may be obtained in the future.

[0084] Parameter updating is the key to the algorithm's continuous learning and adaptation to environmental changes. It ensures that each decision is based on the current optimal knowledge and strategy and is the core of achieving optimal target allocation.

[0085] Step 10: Determine whether the task can be completed and whether the performance value meets the requirements. If so, end the training. If not, return to step 4.

[0086] Regular evaluation of task completion and performance values ​​is an important step in ensuring the achievement of training goals and effective use of resources. Training can only be completed when the algorithm can meet the predetermined task effect and efficiency requirements.

[0087] After each round of training, the algorithm evaluates the current task completion and efficiency value, including task completion time, resource consumption, etc. If these indicators reach the preset goals, the training ends; otherwise, the algorithm will return to step 4 with the newly learned knowledge to continue optimization and learning.

[0088] This step ensures that the training process is goal-driven, that is, constantly pursuing higher task execution efficiency and effectiveness. At the same time, it is also a cyclical process of algorithm self-improvement, approaching the optimal solution through continuous iteration.

[0089] The training process based on SDQN is as follows Figure 5As shown in the figure, in general, during training, the neural network accepts the state given by the environment, outputs the target that the current aircraft should select, and inputs the selected target into the environment as an action. The environment calculates the reward obtained for this allocation based on a series of factors such as the hit rate and target value mentioned above and returns it to the network, and also returns the information of the next state. The network updates its evaluation of the allocation of each target under different states based on the returned reward information, and then evaluates the next state and its various actions, and selects the action that can obtain the highest reward.

[0090] After training is completed, simulation verification can be performed based on the network

[0091] Experiments and training were conducted on two typical combat scenarios, such as Figure 3 and Figure 4 shown.

[0092] In order to compare the impact of the attack angle of the aircraft, the number of aircraft is set in the above two scenarios. In addition, the attack requirements are divided into three situations, namely attacking secondary targets, attacking important targets, and indiscriminate attacks. In the case of attacking secondary targets, the value of secondary targets is set higher and the value of important targets is set lower; in the case of attacking important targets, the opposite is true, the value of important targets is set higher and the value of secondary targets is set lower; in the case of indiscriminate attacks, the value of all enemy targets is set to a uniform value. Here, the target allocation algorithm is trained separately in two scenarios, with different numbers of aircraft and three attack requirements.

[0093] Scenario 1 Allocation Algorithm Verification:

[0094] Figure 6 The algorithm output under the condition of "the number of aircraft is 3 and attacking important targets" is:

[0095] T1_1: [1, 2]

[0096] T2_1: [0]

[0097] T2_2: []

[0098] T2_3: []

[0099] T2_4: []

[0100] T2_5: []

[0101] T2_6: []

[0102] T2_7: []

[0103] T2_8: []

[0104] The content in [] is the aircraft assigned to the enemy target that will attack it. For example, T1_1: [1,2] means that the second and third aircraft attack T1_1.

[0105] Figure 7 This is a schematic diagram of the allocation results under the condition of "the number of aircraft is 3 and attacking important targets". It can be seen from the schematic diagram that the first aircraft directly attacks the T2_1 target which is closer, and the second and third aircraft cooperate to attack the T1_1 important target.

[0106] Scenario 2 Allocation Algorithm Verification:

[0107] Figure 8 The algorithm output under the condition of "the number of aircraft is 6 and the attack is the front secondary target":

[0108] T1_1: []

[0109] T1_2: []

[0110] T2_1: [1,3,4]

[0111] T2_2: [2,5]

[0112] T2_3: [0]

[0113] T2_4: []

[0114] T2_5: []

[0115] T2_6: []

[0116] T2_7: []

[0117] T2_8: []

[0118] The content in [] is the aircraft assigned to the enemy target that will attack it.

[0119] Fig. 9 This is a schematic diagram of the allocation results under the condition of "the number of aircraft is 6, and the front secondary target is attacked". It can be seen from the schematic diagram that the second, fourth, and fifth aircraft cooperate to attack the front secondary target T2_1 which is closer, the third and sixth aircraft cooperate to attack the front secondary target T2_2, and the first aircraft directly attacks the front secondary target T2_3.

[0120] It can be seen from the simulation examples that the target allocation method proposed in the present invention can quickly and reasonably complete the target allocation in a complex environment.

[0121] The present invention constructs a target allocation model based on SDQN, and decomposes the reward signal r that can only be obtained after all the step-by-step actions are executed. According to the actual situation of the problem in the actual environment, the reward signal r is decomposed after each step-by-step action is executed, and at the same time, it is also ensured that the sum of the rewards obtained by each step-by-step action after decomposition is equal to the original reward r. The advantage of the present invention is that it can ensure that the original reward remains unchanged, provide a rich reward signal for the algorithm, and reduce the difficulty of training.

[0122] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.

Claims

1. A multi-agent target allocation method based on the SDQN algorithm, characterized by: Step 1: Set up a training environment for the background of swarm aircraft strike operations; if an aircraft enters the threat zone, there is a certain probability of being intercepted, and set the aircraft's penetration probability to the lateral penetration probability and the longitudinal penetration probability. The lateral penetration probability refers to the probability of an aircraft passing through but not attacking an enemy target, and the longitudinal penetration probability refers to the probability of an aircraft passing through when attacking the enemy target to prevent all aircraft from attacking the enemy target with the highest value; Step 2: Set different reward function types and optimization goals according to different mission indicators; Based on the differences in mission indicators, define the following types of reward functions to guide the target allocation of aircraft: Maximize damage effectiveness: When the main goal of the mission is to cause maximum damage to enemy targets, the reward function will focus on measuring the damage effectiveness of targets assigned to specific aircraft. Through this reward function, the aircraft is given priority to targets that can cause the greatest damage; Minimize the time to complete the mission: If the key to the mission is speed, that is, to complete the mission as quickly as possible, the reward function at this time will reflect the time required to complete the entire mission; Target allocation will tend to minimize the completion time of all targets; Minimize the cost of completing the mission: When the cost of mission execution becomes the main consideration, the corresponding reward function will evaluate the resources and cost required to complete the goal, with the goal of achieving the best result without sacrificing the mission performance. Under the premise of sacrificing the mission effect, reduce resource consumption and cost expenditure as much as possible; according to specific mission requirements, flexibly adjust or combine the above reward functions to adapt to diverse combat environments and goals; the optimization goal is to maximize the overall effectiveness under the guidance of the reward function when the number of aircraft is limited, and to maximize the sum of the corresponding rewards of all aircraft: overall effectiveness maximization: by dynamically adjusting the target allocation strategy, evaluate the sum of rewards corresponding to each possible allocation scheme, so as to find the target allocation strategy that maximizes the overall reward; state: reflects the current target allocation situation, including the target assigned to each aircraft and its state information, which is used as the input of the neural network to evaluate the current allocation strategy; action: based on the current state, the network outputs the target number that the aircraft should select, and then determines the optimal target allocation for each aircraft; Step 3: Initialize the training environment and randomly set an initial allocation result to output to the environment; create an initial state for the training process of the multi-agent target allocation algorithm, including how to initialize the training environment and how to randomly set an initial target allocation result; Step 4: The environment returns the initialized task decision result to SDQN, and SDQN inputs the state returned by the environment into the network; the various indicators obtained by the aircraft group after performing the task in the simulated environment according to the initial target allocation result in step 3 are used to generate a series of measurement indicator values. These values ​​together constitute the task decision result, which is used as the input of the SDQN algorithm to evaluate the effect of the current decision; Step 5: The network gives the evaluation Q value of the next decision result of the current decision result; the SDQN network calculates the Q value of the next possible decision based on the current decision result; this process involves the forward propagation of the network, estimating the Q value of each possible action through the learned parameters and weights; Step 6: Select the allocation result with the largest Q value and integrate it into the environment; the SDQN network compares the Q values ​​of all possible actions calculated, selects the action with the largest Q value as the optimal decision for the next step, and instructs the simulation environment to execute the task according to this decision; Step 7: The environment determines all the route trajectories and mission critical states according to the current allocation results, and obtains the reward value according to the route trajectories and mission critical states; Step 8: Modify the current state to the next state according to the current goal, and return the next state and reward value to the SDQN algorithm; Step 9: The SDQN algorithm uses the returned values ​​from the environment to update the network parameters, making the network's estimate of the Q value more accurate. Using the returned state and reward values, the SDQN algorithm performs parameter updates, involving the calculation of the loss function and gradient descent optimization techniques. These updates allow the network to more accurately predict the Q value, which is the reward that may be obtained in the future. Step 10: Determine whether the task can be completed and whether the performance value meets the requirements. If so, end the training; if not, return to step 4. After each round of training, the algorithm evaluates the current task completion and performance value, including task completion time and resource consumption. If these indicators reach the preset goals, the training ends. Otherwise, the algorithm returns to step 4 with the newly learned knowledge to continue optimization and learning.

2. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored program, wherein when the program is executed, the device where the non-volatile storage medium is located is controlled to execute the method according to claim 1.

3. An electronic device, characterized in that: It comprises a processor and a memory; the memory stores computer-readable instructions, and the processor is used to execute the computer-readable instructions, wherein the computer-readable instructions execute the method according to claim 1 when executed.

Images