Unmanned aerial vehicle group scheduling and simulation integrated system based on digital twin and deep reinforcement learning

Abstract

The invention relates to the technical field of unmanned aerial vehicle control, and discloses an unmanned aerial vehicle group scheduling and simulation integrated system based on digital twinning and deep reinforcement learning, and the system comprises a digital twinning mapping module, a deep reinforcement learning task scheduling module, a heterogeneous interface adaptive module, a fault prediction and self-healing module, and a scenarized strategy optimization module. Through an unmanned aerial vehicle group intelligent scheduling and simulation integrated technology fusing digital twinning and deep reinforcement learning, all links of unmanned aerial vehicle group management and control can be comprehensively covered, common task scheduling and path planning indexes are concerned, and factors such as heterogeneous unmanned aerial vehicle compatibility, fault prediction and self-healing are also included; and the collaborative process of scheduling and simulation is deeply optimized, so that a system operation result more accurately reflects the real operation condition of the unmanned aerial vehicle group, and a more scientific basis is provided for intelligent scheduling and reliable management of the unmanned aerial vehicle group.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) control technology, and more specifically discloses an integrated system for UAV swarm scheduling and simulation based on digital twins and deep reinforcement learning. Background Technology

[0002] With the rapid development of drone technology and the continuous expansion of application scenarios, drones have evolved from simple aerial photography tools into intelligent aerial operation platforms, and are widely used in many fields such as surveying and mapping, logistics and distribution, agricultural plant protection, and power line inspection.

[0003] The prior art patent document with authorization announcement number CN120085686A discloses a "method and system for scheduling unmanned aerial vehicle (UAV) swarms". Specifically, it provides a method and system for scheduling UAV swarms. First, a swarm waypoint task is generated and sent to the master UAV in the UAV swarm. The master UAV decomposes the swarm waypoint task into multiple sub-waypoint tasks and assigns and sends the sub-waypoint tasks to the corresponding UAVs. The UAVs receive the sub-waypoint tasks to execute the task content. The patent document with authorization announcement number CN117693009A discloses "a scheduling method and system for a communication drone swarm", which includes: obtaining the user's current actual location using integrated sensing signals, and determining the first distribution position of the drone swarm based on the user's current actual location using a clustering algorithm; predicting the user's predicted location and the predicted value of the corresponding communication service request at the next moment based on the user's historical location and the corresponding historical communication service request, and using this as prior information; determining the transmission power allocation scheme that maximizes network utility using a reinforcement learning model based on the user's current actual location, the first distribution position, and the prior information; and adjusting the drone swarm to a second distribution position based on the first distribution position, the user's predicted location at the next moment, and the transmission power allocation scheme.

[0004] While existing technologies can acquire user location through integrated sensing signals and combine historical information to predict user location and communication service needs, alternately optimizing the distribution and transmission power of UAV swarms to ensure communication quality and network resource utilization in emergency scenarios, and can also enable task-level communication between the master UAV and the central control system and handle the decomposition and allocation of swarm tasks, reducing the information processing pressure on the central control system, optimizing the computational workload of swarm task allocation, and selecting or temporarily forming UAV swarms to adapt to tasks of different scales according to task requirements, existing technologies suffer from a "simulation-reality gap" that separates the real machine from the simulation environment. This often leads to a partial or even complete performance degradation or failure of algorithms after deployment on real machines, and also presents challenges in multi-UAV collaborative scheduling. Traditional heuristic algorithms require calculations on a second to minute basis, which cannot meet real-time requirements. Greedy algorithms are also difficult to obtain the global optimal solution. Heterogeneous UAV swarms require separate driver development for each model, resulting in high development and maintenance costs and poor scalability. In addition, fault handling is mostly a passive response, lacking intelligent prediction and automatic recovery mechanisms. Furthermore, it only focuses on communication scenarios and lacks the virtual-real bidirectional fusion achieved by digital twin technology to eliminate simulation-real machine deviation. It also does not involve unified management and control of heterogeneous UAVs and adaptive optimization for multiple scenarios. Finally, scheduling relies on static rules and clustering algorithms, lacking the millisecond-level near-optimal decision-making capabilities brought by deep reinforcement learning, and lacking fault prediction and self-healing mechanisms. It has poor adaptability to dynamic and complex scenarios (such as sudden failures and real-time changes in multiple tasks). Summary of the Invention

[0005] The main technical problem solved by this invention is to provide an integrated system for scheduling and simulation of unmanned aerial vehicle (UAV) swarms based on digital twins and deep reinforcement learning, which can solve the problems mentioned in the background art.

[0006] To address the aforementioned technical problems, according to one aspect of the present invention, more specifically, an integrated system for UAV swarm scheduling and simulation based on digital twins and deep reinforcement learning, comprising: a digital twin mapping module integrating UAV swarm physical attribute data and real-time operational status data to construct a digital mapping body of the UAV swarm and synchronize virtual and real states, predicting subsequent operational trends; a deep reinforcement learning task scheduling module combining UAV swarm status data and task requirement data to generate UAV swarm task allocation schemes and path planning strategies; a heterogeneous interface adaptive module processing communication data and control requirement data of different UAV models to generate a unified UAV control interface; a fault prediction and self-healing module monitoring UAV operational status data, identifying fault symptoms and predicting fault risks, and triggering task migration and system fault tolerance strategies when a fault warning occurs or occurs; and a scenario-based strategy optimization module combining task requirements and environmental data of different application scenarios to generate UAV swarm operation optimization strategies adapted to the corresponding scenarios.

[0007] Furthermore, the digital twin mapping module includes: a dynamic modeling module, an adaptive state synchronization module, and a simulation prediction module; Dynamics modeling module: The dynamics model of the UAV is constructed using the six-degree-of-freedom rigid body dynamics equations, while also considering aerodynamic effects; Adaptive State Synchronization Module: Employs an extended Kalman filter algorithm to dynamically adjust weight coefficients and integrates real UAV test data with digital twin prediction data to achieve state synchronization; Simulation prediction module: Runs a digital twin to predict the operational status of a drone swarm.

[0008] Furthermore, the deep reinforcement learning task scheduling module includes: a MA-DDPG algorithm module, a GNN state encoding module, an action space design module, a reward function module, and a training optimization module; MA-DDPG algorithm module: A distributed decision-making framework is constructed using a multi-agent deep deterministic policy gradient algorithm, with each UAV corresponding to an independent Actor network and sharing a Critic network; GNN State Encoding Module: The module uses a graph neural network to model the UAV and mission as graph nodes and their relationships as edges to encode the high-dimensional state space. Motion space design module: Design a hybrid motion space that includes discrete task allocation decisions and continuous speed, height, and path control; Reward function module: Constructs a multi-objective reward function that includes efficiency reward, safety reward, energy consumption reward, delay penalty, and collision penalty; Training optimization module: The training process is optimized by using techniques such as priority experience replay, multi-step learning, target network soft update, and course learning.

[0009] Furthermore, the heterogeneous interface adaptive module includes: a Transformer module, a learning strategy module, and a unified control interface generation module; Transformer module: Uses the Transformer architecture to build a protocol feature extraction and semantic decoding model to process raw data of different UAV communication protocols; Learning strategy module: The protocol mapping model is optimized by combining self-supervised pre-training, few-shot fine-tuning, and online learning. Unified control interface generation module: Generates standardized UAV control interfaces based on protocol semantic mapping results.

[0010] Furthermore, the fault prediction and self-healing module includes: an LSTM fault prediction module, a fault risk assessment module, a task migration module, and a multi-constraint verification module. LSTM Fault Prediction Module: The model is constructed using a multi-scale long short-term memory network. It takes data from motor, battery, IMU, GPS, communication, and environmental sensors as input and outputs the fault probability and remaining lifespan. Fault Risk Assessment Module: Sets fault probability thresholds and provides fault warnings; Task migration module: Employs an improved Hungarian algorithm to construct a task migration cost matrix and generate a task migration plan for faulty UAVs; Multi-constraint verification module: Verifies the task migration scheme based on power constraints, load constraints, time constraints, and spatial constraints.

[0011] Furthermore, the scenario-based strategy optimization module includes: a surveying scenario optimization module, a delivery scenario optimization module, an agricultural sowing scenario optimization module, and a power grid inspection scenario optimization module; Mapping Scene Optimization Module: Adopts an improved ox-plowing path algorithm, combines terrain models to generate mapping paths, and controls mapping coverage and overlap rates; Delivery scenario optimization module: Utilizes DBSCAN order clustering, improves the VRP algorithm, and combines real-time traffic and weather data to optimize delivery routes; Agricultural spreading scenario optimization module: Adaptive control algorithm is used to adjust spreading position compensation, nozzle angle and flow rate to optimize spreading uniformity; Power grid inspection scenario optimization module: It adopts a spiral inspection path algorithm, combines electromagnetic interference assessment to generate inspection paths, and enables AI defect detection function.

[0012] Furthermore, the parameters of the dynamic modeling module are identified and calibrated using actual flight data, the state synchronization delay of the adaptive state synchronization module is less than 100ms, and the prediction results of the simulation prediction module are used as the decision input for the deep reinforcement learning task scheduling module.

[0013] Furthermore, the heterogeneous interface adaptive module supports protocol adaptation for mainstream drone models such as DJI, Pixhawk, and Parrot.

[0014] The beneficial effects of this invention's integrated UAV swarm scheduling and simulation system based on digital twins and deep reinforcement learning are as follows: By integrating digital twins and deep reinforcement learning into a unified UAV swarm intelligent scheduling and simulation technology, it can comprehensively cover all aspects of UAV swarm management. It not only focuses on common task scheduling and path planning indicators but also incorporates elements such as heterogeneous UAV compatibility, fault prediction, and self-healing. Furthermore, it deeply optimizes the collaborative process of scheduling and simulation, enabling the system's operational results to more accurately reflect the real operational status of the UAV swarm, providing a more scientific basis for intelligent scheduling and reliable management of UAV swarms. In addition, through a unified heterogeneous interface and digital twin mapping technology, it achieves… It achieves effective integration of drones and systems from different manufacturers and models, breaking down protocol barriers and enabling real-time synchronization and data exchange between drones and systems. This improves the efficiency and intelligence of drone swarm resource scheduling, and enhances system compatibility and reliability. Simultaneously, through deep reinforcement learning task scheduling and intelligent fault self-healing technology, it provides more efficient and intelligent decision-making and safeguards, clearly planning task allocation and path changes for drone swarms. Furthermore, the self-healing mechanism based on real-time twin data and advanced reinforcement learning models can quickly provide prediction and migration strategies before faults occur, significantly improving the scheduling level of drone swarm management and its ability to respond to emergencies. Attached Figure Description

[0015] The present invention will now be described in further detail with reference to the accompanying drawings and specific implementation methods.

[0016] Figure 1 This is a schematic diagram of the system module architecture. Detailed Implementation

[0017] The present invention will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the present application can be combined with each other.

[0018] According to one aspect of the invention, such as Figure 1 As shown, an integrated system for UAV swarm scheduling and simulation based on digital twins and deep reinforcement learning is provided, including: a digital twin mapping module, which integrates UAV swarm physical attribute data and real-time operational status data to construct a digital mapping of the UAV swarm and synchronize virtual and real states, predicting subsequent operational trends. This module includes: Dynamics modeling module: The dynamics model of the UAV is constructed using the six-degree-of-freedom rigid body dynamics equations, while also considering aerodynamic effects; The six-degree-of-freedom rigid body dynamics equations used are as follows: In the formula, For the quality of drones, For position vectors, Here is the rotational inertia matrix. Let F be the angular velocity vector. thrust F drag F gravity F wind M motors M aero The force and torque terms were identified and calibrated using actual flight data. Specifically, real aircraft flight data were collected under different loads (e.g., 500g-10kg), wind speeds (e.g., 0-8m / s), and flight attitudes (e.g., hovering, diving, turning). The least squares method was used to iteratively optimize the model parameters to ensure that the model can accurately reproduce the force and motion state of the UAV in the real environment, avoiding the deviation caused by the "idealized assumptions" of traditional simulation models. Furthermore, this dynamic model does not only consider basic mechanical effects, but also fully incorporates complex aerodynamic factors such as turbulence and ground effects. For example, it obtains the aerodynamic drag coefficient at different flight speeds through wind tunnel tests, and combines the lift change data of the real aircraft when flying at low altitudes to correct the influence of ground effects on the hovering stability of the UAV. This greatly improves the accuracy of the model's reproduction of the motion characteristics of the real aircraft, providing a high-precision foundation for subsequent virtual-real state synchronization and simulation prediction.

[0019] Adaptive State Synchronization Module: Employs an extended Kalman filter algorithm to dynamically adjust weight coefficients and integrates real UAV test data with digital twin prediction data to achieve state synchronization; The core fusion formula for state synchronization is: In the formula, As a weight for trustworthiness of real device data, For time gradient compensation coefficients, This data is from actual test data of the drone. The predicted data for the digital twin is based on the output of the dynamic model. The gradient of the state over time; Simultaneously, the Extended Kalman Filter (EKF) algorithm is used for real-time dynamic adjustment. and The specific implementation process is as follows: First, the prior state estimate of the digital twin is obtained through the state transition function, as shown below: Secondly, the prior error covariance matrix is ​​calculated using the following formula: In the formula, Let Jacobian be the state transition matrix. The process noise covariance is then calculated. Subsequently, based on actual measured data, the optimal gain is calculated using the Kalman gain formula, as shown below: In the formula, For the observation matrix, To observe the noise covariance, the posterior state is finally updated: After obtaining the posterior state estimate, determine the trust weights of the real machine data. The calculation formula is: When the data quality of the actual device is high (e.g., strong GPS signal), Approaching 1, it relies more on actual device data; when actual device data is unreliable (e.g., weak GPS signal), The reliance on digital twins for predictions is decreasing; In addition, by deploying the Extended Kalman Filter (EKF) algorithm on edge computing nodes nearby, the transmission latency between the cloud and the physical device is reduced. Combined with a lightweight data compression protocol (such as MQTT-SN), the state synchronization latency is strictly controlled within 100ms, so as to avoid the digital twin from becoming disconnected from the physical device due to synchronization lag, thereby affecting the effectiveness of scheduling decisions.

[0020] Simulation prediction module: Runs a digital twin to predict the operational status of a drone swarm; Specifically, by calling the ultra-real-time simulation function of the digital twin (e.g., the simulation rate is configured to be 10 times that of the real-time rate), based on the calibrated dynamic model, real-time synchronized real machine status data (e.g., current position, battery level, task progress), and environmental data (e.g., real-time wind speed, visibility, obstacle distribution), the operating status of the drone swarm in the next 5-10 seconds can be predicted. For example, the location trajectory, remaining battery level change, arrival time at the next delivery point, and whether the energy consumption will exceed expectations due to increased wind speed can be predicted for a drone performing a delivery task in the next 6 seconds. Meanwhile, the simulation prediction results are not stored or displayed independently, but serve as the core decision input for the deep reinforcement learning task scheduling module. For example, when it is predicted that a drone will enter an area with wind speeds greater than 5 m / s within the next 8 seconds, this prediction information will be synchronized in real time to the state space of the MA-DDPG algorithm (incorporating wind speed features from the environmental factor dimension), allowing the scheduling engine to adjust the drone's flight speed (e.g., from 8 m / s to 5 m / s) and path (e.g., avoiding high-wind-speed areas) in advance. This avoids a surge in energy consumption or task delays due to sudden environmental changes. If it is predicted that multiple drones will have airspace conflict risks within the next 10 seconds, the prediction results will trigger the pre-calculation of the collision penalty term in the reward function module, allowing the DRL model to output obstacle avoidance adjustment strategies in advance, achieving closed-loop optimization of "prediction-decision-execution". Finally, during the simulation prediction process, the deviation between the predicted state and the subsequent actual measured state will be compared in real time. If the deviation exceeds 5% (e.g., the error between the predicted position and the actual position is greater than 3 meters), the dynamic model will be automatically calibrated twice. Through feedback adjustment, the model accuracy will be further improved, ensuring that the reliability of the simulation prediction continuously meets the scheduling requirements.

[0021] The deep reinforcement learning task scheduling module combines UAV swarm status data and task requirement data to generate UAV swarm task allocation schemes and path planning strategies. This module includes: MA-DDPG algorithm module: A distributed decision-making framework is constructed using a multi-agent deep deterministic policy gradient algorithm, with each UAV corresponding to an independent Actor network and sharing a Critic network; Specifically, a multi-agent deep deterministic policy gradient algorithm is adopted, in which each drone's Actor network independently outputs its own action decisions (such as flight speed adjustment and target task selection), while the shared Critic network evaluates the overall value of all Actor actions based on the global state (such as the comprehensive information of all drones and tasks), thus realizing a collaborative mode of "distributed decision-making + centralized value evaluation". Furthermore, the core reason for choosing this algorithm is that DDPG can effectively handle the continuous action space of drones, such as speed and altitude, while the multi-agent framework can support the large-scale expansion of drone swarms. For example, when the number of drones increases to more than 20, it can still maintain the independence of individual drone decisions. At the same time, by sharing Critic, it can ensure the global coordination of group actions and avoid the impact of single drone decision-making bias on overall mission efficiency.

[0022] GNN State Encoding Module: The module uses a graph neural network to model the UAV and mission as graph nodes and their relationships as edges to encode the high-dimensional state space. The state space comprises four dimensions: UAV state, mission attributes, environmental factors, and communication state. The UAV state covers the three-dimensional position of each UAV. 3D velocity Remaining battery percentage B i And the current task status (idle / in progress / returning home), the task attributes include the location coordinates of each task. Priority (Levels 1-10), Deadline Task type (delivery / surveying / inspection, etc.), estimated workload, environmental factors including wind speed and direction ( Visibility, rainfall intensity, and obstacle occupancy grid; communication status includes the quality matrix of inter-UAV communication links. Communication strength with ground stations.

[0023] Motion space design module: Design a hybrid motion space that includes discrete task allocation decisions and continuous speed, height, and path control; Specifically, the action space: In the formula, For discrete actions, it is responsible for assigning the next task to be performed to each drone (e.g., selecting a target task point from the idle task set). A continuous maneuver includes a waypoint sequence and a turning radius. For continuous operation, the adjustment range is [0, v]. max ], For continuous operation, the adjustment range is [h] min ,h max ]; The hybrid action space design can simultaneously meet the discrete decision-making requirements of "task allocation" and the continuous adjustment requirements of "flight control", avoiding the shortcomings of traditional single discrete or continuous action spaces that cannot take into account both scheduling flexibility and control accuracy. At the same time, all continuous action outputs will undergo physical constraint verification before being executed. If the performance limits of the drone are exceeded (e.g., speed exceeds v), the output will be verified. max If the value is 0, it will be automatically truncated to the limit value to ensure the feasibility of the action.

[0024] Reward function module: Constructs a multi-objective reward function that includes efficiency reward, safety reward, energy consumption reward, delay penalty, and collision penalty; Specifically, the reward function expression is: In the formula, - These are dynamic weighting coefficients; Efficiency bonus: In the formula, As task weight, This is an indicator function (1 for completion, 0 otherwise). Normalized execution time; The safety reward is in the form of a piecewise function: Severe penalties will be imposed when the distance between drones is less than a threshold. The energy consumption reward is as follows: It considers not only the remaining power, but also the uniformity of power distribution (measured by standard deviation). Final delay penalty: The delay penalty for high-priority tasks increases exponentially.

[0025] Training optimization module: The training process is optimized by using techniques such as priority experience replay, multi-step learning, target network soft update, and course learning. Among them, Priority Experience Replay (PER) assigns different sampling weights to experience samples based on TD error, and the sampling probability formula is: This allows the model to learn more from "unexpected" experiences (such as samples with large TD errors caused by sudden obstacles), thereby improving learning efficiency; Multi-step learning uses n-step benefit calculation, with the formula as follows: Reduce the variance of single-step gains to make training more stable; Furthermore, the target network soft update is achieved through the formula: To avoid model oscillations caused by excessive fluctuations in target values ​​during training; Finally, the course adopts a scenario progression strategy from simple to complex: the initial training scenario consists of 2-3 drones and 10 task points. After every 1000 rounds of training, the number of drones (e.g., up to 20) and the number of task points (e.g., up to 100) are gradually increased. At the same time, more complex environmental disturbances (e.g., random wind speed changes) are introduced to ensure that the model converges stably from easy to difficult and to avoid training crashes caused by the initial complex scenario.

[0026] The heterogeneous interface adaptive module processes communication data and control requirements data from different UAV models, generating a unified UAV control interface. This module includes: Transformer module: Uses the Transformer architecture to build a protocol feature extraction and semantic decoding model to process raw data of different UAV communication protocols; The core process of protocol processing is divided into three steps: preprocessing, feature extraction, and semantic decoding. First, the original protocol byte stream is processed by preprocess_protocol. Specifically, byte-level tokenization is performed to convert the original bytes into an input token that the model can recognize. Position encoding is generated to capture the sequence order relationship of protocol bytes. At the same time, the protocol type (such as MAVLink, DJI private protocol) is detected by the features of the first 16 bytes of the protocol header, and the embedding vector of the corresponding protocol type is loaded. Finally, the preprocessed data that integrates the token, position encoding and type embedding is output. The feature extraction stage is implemented by a 6-layer, 8-head TransformerEncoder, which captures the dependencies between protocol bytes through a self-attention mechanism and extracts high-dimensional features containing protocol syntax and semantic information. In addition, the semantic decoding stage calculates the attention weight of features and extracts key fields (such as GPS coordinates, motor speed, and attitude angle) from the protocol through a preset pattern matcher (such as position pattern matcher, speed pattern matcher, and command pattern matcher). Only when the confidence of a field extraction exceeds a threshold (such as 0.7) will it be included in the semantic field set to ensure the accuracy of subsequent mapping and avoid control command deviations caused by low-confidence fields.

[0027] Learning strategy module: The protocol mapping model is optimized by combining self-supervised pre-training, few-shot fine-tuning, and online learning. The self-supervised pre-training stage employs a masked language model (MLM) task, randomly masking a certain number of bytes (e.g., 15%) in the protocol message. This allows the model to learn the ability to recover the masked bytes from the context, thereby uncovering the protocol's inherent structural patterns. The loss function is: Through this task, the model can autonomously learn the syntax rules of different protocols (such as the message header format of MAVLink and the verification position of the DJI protocol) without manual annotation. For the few-sample fine-tuning stage, only 10-100 protocol samples need to be collected as the support set S for new drone models, and the loss function is: This enables rapid optimization of model parameters, allowing the model to adapt to new protocols and avoiding the problem of traditional static adaptation layers requiring thousands of lines of driver code for each model. Furthermore, during the online learning phase, protocol interaction data will be continuously collected during system operation, and model parameters will be updated through real-time gradient descent, as shown in the formula: This ensures that the model can adapt to minor protocol changes caused by drone firmware upgrades, maintaining long-term compatibility and stability.

[0028] Unified control interface generation module: Generates standardized UAV control interfaces based on protocol semantic mapping results; Specifically, the unified semantics output by the Transformer module (e.g., "take off to a height of 10 meters" and "adjust speed to 5 m / s") are first matched with the preset API template library (general drone control command template) to generate a standardized API. Simultaneously, after the API is generated, it will be actually called and verified through validate_api. That is, the generated API command is sent to the drone to be adapted, and the drone's response is checked to see if it is consistent with expectations (for example, after sending the take-off command, verify whether the drone leaves the ground and reaches the target altitude within 3 seconds). If the verification is successful, the API will be included in the unified control interface. If the verification fails (for example, the feedback command execution timed out), finetune_api will be started to fine-tune the protocol mapping parameters until the API call success rate is qualified. In addition, the generated unified control interface is compatible with mainstream drone models such as DJI, Pixhawk, and Parrot. When a new drone is connected, no professional protocol analysis personnel are required. Only a small number of protocol samples need to be collected to complete the adaptation. The connection time is shortened from "weeks" of the traditional static adaptation layer to "hours", which greatly reduces the management and control costs of heterogeneous drone swarms.

[0029] The fault prediction and self-healing module monitors the UAV's operational status data, identifies fault symptoms, predicts fault risks, and triggers mission migration and system fault tolerance strategies when a fault is warned or occurs. This module includes: LSTM Fault Prediction Module: The model is constructed using a multi-scale long short-term memory network. It takes data from motor, battery, IMU, GPS, communication, and environmental sensors as input and outputs the fault probability and remaining lifespan. Specifically, the input feature x t It covers six major categories of data: motor data (speed, current, temperature, vibration spectrum of each motor), battery data (voltage, current, temperature, internal resistance, discharge curve), IMU data (acceleration, angular velocity, magnetic field strength and its noise level), GPS data (position accuracy factor PDOP, number of satellites, signal strength), communication data (packet loss rate, delay, signal strength RSSI), and environmental data (temperature, humidity, air pressure, wind speed). The model employs a multi-scale LSTM architecture, including short-term LSTM (e.g., processing data from the most recent 1 minute with a sampling rate of 1Hz), medium-term LSTM (e.g., processing data from the most recent 10 minutes with a sampling rate of 0.1Hz), and long-term LSTM (e.g., processing data from the most recent 1 hour with a sampling rate of 0.01Hz). The hidden states of the three types of LSTMs at the last moment are concatenated by two linear + ReLU fusion layers. The model also outputs dual-task results: the failure probability is calculated using the softmax activation function, as shown below: Significantly reduces the risk of drone swarm mission interruption and crashes.

[0030] Fault Risk Assessment Module: Sets fault probability thresholds and provides fault warnings; Specifically, a fault probability threshold θ = 0.7 is set, and when the LSTM outputs a certain type of fault probability P... fault When the value is greater than θ, the system triggers an early warning. In addition, the early warning information will be synchronized to the digital twin mapping module to update the digital twin status of the faulty drone and provide real-time fault data support for the scheduling module.

[0031] Task migration module: Employs an improved Hungarian algorithm to construct a task migration cost matrix and generate a task migration plan for faulty UAVs; The optimization objective for task migration is to minimize the total migration cost and the incremental task latency, as shown in the formula: In the formula, For drones Towards The cost of migration tasks, For the task The delay increment; The cost matrix is ​​constructed by considering four factors: distance cost (straight-line distance between drone j and the faulty task point), time cost (distance / maximum speed of drone j), energy cost (estimated power consumption of the task), and priority penalty (additional cost for high-priority tasks not migrating in time). Furthermore, an improved Hungarian algorithm is used to solve the cost matrix in O(n log n) time. 3 Find the optimal allocation scheme within the time complexity; Finally, the migration plan will be optimized in sequence, prioritizing the migration of high-priority tasks with short remaining time to reduce the overall task interruption time.

[0032] Multi-constraint verification module: Verifies the task migration scheme based on power constraints, load constraints, time constraints, and spatial constraints; The power constraint requires that the remaining power of the takeover drone is sufficient to complete the mission, return to base, and allow for a safety margin. The formula is: In the formula, This is a safety margin (usually 10% of the total battery capacity). The load constraint applies to delivery tasks, requiring that the remaining load capacity of the takeover drone is not less than the task requirements. The formula is: In the formula, Healthy drones to take over missions Maximum load capacity For drones The weight it is currently bearing. For the first to be migrated The load required for each task; Meanwhile, time constraints ensure that the task can still be completed before the deadline after migration, as shown in the formula: In the formula, To allow time, For the current time, The time required for task migration, The time required for task execution. The deadline for the task; Finally, airspace constraints are verified using the airspace map of the digital twin to ensure that the migration path does not overlap with the no-fly zone. The formula is: If any constraint is not met, a takeover target is selected again from the list of healthy drones until all constraints are met, ensuring the feasibility of the migration plan.

[0033] The scenario-based strategy optimization module combines task requirements and environmental data from different application scenarios to generate optimized drone swarm operation strategies adapted to those scenarios. This module includes: Mapping Scene Optimization Module: Adopts an improved ox-plowing path algorithm, combines terrain models to generate mapping paths, and controls mapping coverage and overlap rates; The core optimization objective for the surveying scenario is a multi-objective balance of "coverage, overlap rate, and time consumption," and the objective function formula is: In the formula, For mapping coverage, Imaging resolution quality is inversely proportional to flight altitude. The overlap rate, Total time elapsed.

[0034] Delivery scenario optimization module: Utilizes DBSCAN order clustering, improves the VRP algorithm, and combines real-time traffic and weather data to optimize delivery routes; The optimization objectives for the delivery scenario focus on timeliness and cost, and the objective function formula is as follows: In the formula, For package weight, For service level coefficient, This refers to the actual delivery time. To ensure delivery time, This represents the total cost.

[0035] Agricultural spreading scenario optimization module: Adaptive control algorithm is used to adjust spreading position compensation, nozzle angle and flow rate to optimize spreading uniformity; The core optimization objective for agricultural seeding is to balance "uniformity, material utilization, and wind-induced displacement," and the objective function formula is: In the formula, To ensure even coverage, For the variance of seeding density, To improve material utilization, This is the wind-induced offset.

[0036] Power grid inspection scenario optimization module: adopts a spiral inspection path algorithm, combines electromagnetic interference assessment to generate inspection paths, and enables AI defect detection function; The optimization objective for power grid inspection is "defect detection rate - safe distance", and the objective function formula is: In the formula, For line segments The probability of defects, To detect indicator functions, For image quality, Punishment for security risks.

[0037] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention are also within the protection scope of the present invention.

Claims

1. An unmanned aerial vehicle group scheduling and simulation integrated system based on digital twinning and deep reinforcement learning, characterized in that, Comprise: The digital twin mapping module integrates unmanned aerial vehicle group physical attribute data, real-time running state data, constructs unmanned aerial vehicle group digital mapping body and synchronizes virtual and real state, predicts subsequent running trend; the deep reinforcement learning task scheduling module combines unmanned aerial vehicle group state data and task demand data, generates unmanned aerial vehicle group task allocation scheme and path planning strategy; the heterogeneous interface adaptive module processes communication data and control demand data of different models of unmanned aerial vehicles, generates unified unmanned aerial vehicle control interface; The fault prediction and self-healing module monitors unmanned aerial vehicle running state data, identifies fault signs and predicts fault risk, triggers task migration and system fault tolerance strategy when fault warning or occurrence; The scene-based strategy optimization module combines task demand and environment data of different application scenes, generates unmanned aerial vehicle group running optimization strategy adapted to corresponding scene.

2. The UAV swarm scheduling and simulation integrated system based on digital twin and deep reinforcement learning according to claim 1, characterized in that: The digital twin mapping module comprises: a dynamics modeling module, an adaptive state synchronization module and a simulation prediction module; The dynamics modeling module: adopts six-degree-of-freedom rigid body dynamics equation to construct unmanned aerial vehicle dynamics model, simultaneously considers aerodynamic effect; The adaptive state synchronization module: adopts extended Kalman filter algorithm to dynamically adjust weight coefficient, fuses unmanned aerial vehicle real machine measured data and digital twin prediction data to realize state synchronization; The simulation prediction module: runs digital twin, predicts unmanned aerial vehicle group running state.

3. The UAV swarm scheduling and simulation integrated system based on digital twin and deep reinforcement learning of claim 1, wherein: The deep reinforcement learning task scheduling module comprises: a MA-DDPG algorithm module, a GNN state coding module, an action space design module, a reward function module and a training optimization module; The MA-DDPG algorithm module: adopts multi-agent deep deterministic policy gradient algorithm to construct distributed decision-making framework, each unmanned aerial vehicle corresponds to independent Actor network and shared Critic network; The GNN state coding module: adopts graph neural network to model unmanned aerial vehicle and task as graph node and relationship as edge, encodes high-dimensional state space; The action space design module: designs hybrid action space containing discrete task allocation decision and continuous speed, height and path control; The reward function module: constructs multi-objective reward function containing efficiency reward, safety reward, energy consumption reward, delay penalty and collision penalty; The training optimization module: adopts priority experience replay, multi-step learning, target network soft update and curriculum learning technology to optimize model training process.

4. The UAV swarm scheduling and simulation integrated system based on digital twin and deep reinforcement learning of claim 1, wherein: The heterogeneous interface adaptive module comprises: a Transformer module, a learning strategy module and a unified control interface generation module; The Transformer module: adopts Transformer architecture to construct protocol feature extraction and semantic decoding model, processes different unmanned aerial vehicle communication protocol raw data; The learning strategy module: adopts self-supervised pre-training, few-shot fine-tuning and online learning combined strategy to optimize protocol mapping model; The unified control interface generation module: generates standardized unmanned aerial vehicle control interface based on protocol semantic mapping result.

5. The UAV swarm scheduling and simulation integrated system based on digital twin and deep reinforcement learning of claim 1, wherein: The fault prediction and self-healing module comprises: an LSTM fault prediction module, a fault risk assessment module, a task migration module and a multi-constraint checking module; The LSTM fault prediction module adopts a multi-scale long short-term memory network to construct a model, inputs motor, battery, IMU, GPS, communication, and environmental sensor data, and outputs fault probability and residual life; The fault risk assessment module sets a fault probability threshold to perform fault early warning; The task migration module adopts an improved Hungarian algorithm to construct a task migration cost matrix and generate a fault UAV task migration scheme; The multi-constraint checking module checks the task migration scheme for power consumption constraints, load constraints, time limit constraints, and airspace constraints.

6. The UAV swarm scheduling and simulation integrated system based on digital twin and deep reinforcement learning of claim 1, wherein: The scenario-based strategy optimization module includes a surveying and mapping scenario optimization module, a distribution scenario optimization module, an agricultural spreading scenario optimization module, and a power grid inspection scenario optimization module; The surveying and mapping scenario optimization module adopts an improved plough-type path algorithm to generate a surveying and mapping path in combination with a terrain model to control surveying and mapping coverage and overlap rates; The distribution scenario optimization module adopts DBSCAN order clustering and an improved VRP algorithm to optimize a distribution path in combination with real-time traffic and weather data; The agricultural spreading scenario optimization module adopts an adaptive control algorithm to adjust spreading position compensation, nozzle angle, and flow rate to optimize spreading uniformity; The power grid inspection scenario optimization module adopts a spiral inspection path algorithm to generate an inspection path in combination with electromagnetic interference evaluation and simultaneously enables an AI defect detection function.

7. The UAV swarm scheduling and simulation integrated system based on digital twin and deep reinforcement learning of claim 2, wherein: The parameters of the dynamics modeling module are identified and calibrated through actual flight data, the state synchronization delay of the adaptive state synchronization module is less than 100 ms, and the prediction results of the simulation prediction module are used as decision input for the deep reinforcement learning task scheduling module.

8. The UAV swarm scheduling and simulation integrated system based on digital twin and deep reinforcement learning of claim 4, wherein: The heterogeneous interface adaptive module supports protocol adaptation for mainstream DJI, Pixhawk, and Parrot UAV models.

Images