An unmanned aerial vehicle real-time path planning system and method based on dynamic weight distribution and multi-source data fusion

Abstract

The present application relates to the technical field of unmanned aerial vehicle flight path planning, and particularly relates to a real-time path planning system and method for unmanned aerial vehicles based on dynamic weight distribution and multi-source data fusion. The system comprises a multi-source data fusion module integrated with a laser radar, a millimeter wave radar, a visual sensor and a Beidou positioning unit; the dynamic weight distribution module adopts a hybrid decision mechanism combining fuzzy logic and reinforcement learning, dynamically adjusts the weight coefficients of each sensor according to the environmental complexity, threat level and unmanned aerial vehicle state; the real-time path planning module is built-in with an improved RRT* algorithm and a Markov decision process, adopts a hierarchical planning architecture to generate a globally optimal path and a locally obstacle-avoiding trajectory; the unmanned aerial vehicle cooperative control module comprises a double-redundant flight control system and a dynamic obstacle avoidance unit; the communication relay module supports 5G and low-orbit satellite dual-mode communication, updates the environmental cognition model of each unmanned aerial vehicle through federated learning, and realizes real-time path planning based on dynamic weight distribution and multi-source fusion of unmanned aerial vehicles.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) flight path planning technology, and in particular to a real-time UAV path planning system and method based on dynamic weight allocation and multi-source data fusion. Background Technology

[0002] Current UAV path planning systems suffer from significant bottlenecks: single sensors are prone to failure under dynamic obstacles, severe weather, or electromagnetic interference, leading to incomplete environmental modeling. Traditional algorithms struggle to quickly generate optimal paths that balance safety and energy consumption under dynamic threats (such as moving obstacles). The lack of dynamic task allocation mechanisms makes resource conflicts or response delays likely. Long-distance communication is susceptible to interference, and sensitive data may be stolen.

[0003] At the sensor level, fixed-weight fusion strategies cannot adapt to sudden environmental changes (such as dense fog weakening visual data), leading to a decrease in perception confidence. At the planning algorithm level, the standard RRT* algorithm converges slowly in dynamic scenarios, and Monte Carlo search lacks real-time performance (>500ms), making it difficult to cope with sudden threats. At the collaborative control level, centralized decision-making is susceptible to communication latency, distributed strategies lack Nash equilibrium optimization, and multi-machine task allocation is suboptimal.

[0004] Existing systems often rely on offline environment models, which cannot update dynamic obstacle information (such as vehicle trajectory prediction) in real time, resulting in a mismatch between the planned path and the actual environment. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a real-time path planning system for unmanned aerial vehicles (UAVs) based on dynamic weight allocation and multi-source data fusion, comprising:

[0006] The multi-source data acquisition and fusion module is used to acquire multi-source data from UAV sensors, terrain data of the UAV flight area, and UAV flight condition data, and to preprocess the multi-source data from UAV sensors to construct a multi-source environmental perception data matrix.

[0007] The dynamic weight allocation module is used to generate a weight coefficient control sequence based on the multi-source fusion sensing data matrix, UAV flight area terrain data and UAV flight condition data, and dynamically adjust the weight coefficients of UAV sensors.

[0008] The real-time path planning module is used to generate the UAV's global path and local obstacle avoidance trajectory through a hierarchical path planning model based on the multi-source environmental perception data matrix and weight coefficient control sequence, and to dynamically optimize and smooth the global path to obtain UAV trajectory data.

[0009] The UAV collaborative control module is used to perform real-time flight control and flight command calculation for a cluster of multiple UAVs based on the UAV trajectory data output by the real-time path planning module.

[0010] The communication relay module is used to establish a dual-mode communication link with low-orbit satellites via 5G network to transmit the UAV flight status information to the implementation path planning module and the main control and collaborative learning module;

[0011] The main control and collaborative learning module is used to construct a three-dimensional digital twin model based on the terrain data of the UAV's flight area and the UAV's flight status information, and to perform online updates and global optimization of each module based on the federated learning algorithm.

[0012] Furthermore, the multi-source data acquisition and fusion module includes a lidar unit, a millimeter-wave radar unit, a visual sensor unit, and a BeiDou satellite positioning unit; the multi-source data from the UAV sensors includes lidar point cloud data, millimeter-wave radar trajectory prediction data, image recognition data, and BeiDou satellite positioning data.

[0013] The preprocessing includes heterogeneous data calibration of multi-source data from UAV sensors using time synchronization algorithms and spatial registration theory, establishing a hidden Markov model to assess the confidence level of UAV sensor health, eliminating data conflicts among data in the multi-source data of UAV sensors based on Dempster-Shafer evidence theory, and constructing an ARIMA-LSTM hybrid model to predict and compensate for communication delays among data in the multi-source data of UAV sensors.

[0014] Furthermore, the dynamic weight allocation module includes a terrain analysis unit, a threat identification unit, a confidence adjustment unit, and a Nash equilibrium collaborative decision-making unit;

[0015] The terrain analysis unit is used to calculate the terrain complexity of the UAV flight area based on the terrain data of the UAV flight area using a fractal geometry algorithm.

[0016] The threat identification unit is used to identify the dynamic obstacle prediction threat level based on the terrain complexity of the UAV flight area through a spatiotemporal graph convolutional network.

[0017] The confidence adjustment unit is used to construct an adaptive adjustment model based on the Lyapunov index according to the terrain complexity of the UAV flight area and the predicted threat level of dynamic obstacles, and to optimize the UAV sensor weight allocation in real time.

[0018] The Nash equilibrium collaborative decision-making unit, based on the terrain complexity of the UAV flight area and the predicted threat level of dynamic obstacles, combined with the multi-source fusion perception data matrix, establishes a game payoff matrix among multiple UAVs according to Nash equilibrium theory, and makes collaborative decisions on multiple UAV missions.

[0019] The spatiotemporal graph convolutional network constructs a spatiotemporal adjacency matrix, using LiDAR point cloud data as spatial graph node features and millimeter-wave radar trajectory prediction data as temporal graph edge weights. Based on a hierarchical spatiotemporal convolutional structure, it extracts obstacle motion pattern features and predicts obstacle motion trajectories within the next 3 seconds using a time sliding window mechanism. The dynamic obstacle prediction threat level is divided into five quantitative indicators based on the pre-obtained collision probability of the UAV flight trajectory: emergency level when the collision time is less than 2 seconds and the distance threshold is less than 5 meters; high-risk level when the collision time is less than 5 seconds and the distance threshold is less than 10 meters; early warning level when the collision time is less than 10 seconds and the distance threshold is less than 20 meters; and safe and ineffective levels when the collision time is greater than 10 seconds and the distance threshold is greater than 20 meters.

[0020] The adaptive adjustment model based on the Lyapunov index constructs an energy function to evaluate the confidence of each UAV sensor in real time and normalizes the terrain complexity of the UAV flight area. When the terrain complexity of the UAV flight area exceeds 0.7, the weight of the lidar unit is increased to 0.6-0.8. When the threat level reaches the high-risk or emergency level, the weight of the millimeter-wave radar unit is dynamically increased to 0.3-0.5.

[0021] The game payoff matrix includes flight mission completion rate, flight energy efficiency, and flight safety distance. The game payoff matrix is ​​optimized through an iterative strategy to obtain the Pareto optimal allocation scheme and generate a weight coefficient control sequence.

[0022] The final generated terrain complexity and dynamic obstacle prediction threat level of the drone flight area are fed back to the real-time path planning module and the main control and collaborative learning module in real time via the 5G network.

[0023] Furthermore, the hierarchical path planning model includes a global topology layer, a local obstacle avoidance layer, and a trajectory optimization layer;

[0024] The global topology layer constructs a navigation topology map based on lidar point cloud data and calculates the initial waypoint sequence of the UAV using an improved Dijkstra algorithm. The path sampling density of the initial waypoint sequence is dynamically adjusted according to the terrain complexity of the UAV's flight area. When the terrain complexity of the UAV's flight area is >0.6, the path sampling density is reduced to 0.5 meters. When there are dynamic obstacles or highly complex terrain in the flight area, the global topology layer further adopts an improved RRT* algorithm to dynamically adjust the sampling strategy by integrating environmental features and historical path information to generate the initial waypoint sequence.

[0025] The local obstacle avoidance layer constructs an evaluation fusion function by predicting the threat level through dynamic obstacles and replans the initial waypoint sequence. When a high-risk threat is detected, the Monte Carlo tree search algorithm is activated, and an emergency avoidance path is generated within 200ms based on the node expansion strategy of UAV dynamic constraints. The evaluation fusion function includes the threat field strength value, energy consumption rate, and heading deviation.

[0026] The trajectory optimization layer is based on the node vector dynamic adjustment mechanism of the adaptive B-spline curve. It optimizes the curve order and control point density in real time according to the UAV flight status information fed back by the communication relay module. The continuity of the curve order meets the C3 differentiability requirement. A dynamic correlation model between the UAV lift-to-drag ratio and the UAV power system is established. The UAV power parameter configuration is optimized through the backpropagation algorithm to obtain the final UAV trajectory data, which is transmitted to the UAV cooperative control module in real time.

[0027] Furthermore, the improved RRT* algorithm is used for path planning in the global topology layer. It dynamically adjusts the sampling strategy by integrating environmental features and historical path information, accelerates the path search process by adopting a bidirectional expansion mechanism, performs preliminary smoothing optimization on the generated initial trajectory through optimal control theory, and sets up an online learning unit to update the heuristic function parameters according to reinforcement learning feedback. The optimization result is used as the output of the global topology layer for further processing by the local obstacle avoidance layer and trajectory optimization layer, and generates flight commands through the UAV collaborative control module.

[0028] Furthermore, the UAV collaborative control module performs fault-tolerant control on flight command calculation using a quaternion interpolation algorithm, optimizes the multi-rotor thrust distribution scheme based on the pseudo-inverse method, and the dynamic obstacle avoidance unit completes obstacle avoidance actions within 200ms by fusing perception between the lidar unit and the visual sensor. The formation maintaining unit uses the virtual structure method to maintain the cluster geometry, and the control commands are distributed to each UAV node via the communication relay module.

[0029] Furthermore, the main control and collaborative learning module constructs a three-dimensional digital twin model containing terrain data of the UAV flight area and UAV flight status information, updates the UAV trajectory data of the real-time path planning module through a transfer learning framework, and generates a threat situation map by integrating the multi-source environmental perception data matrix of the multi-source data acquisition and fusion module.

[0030] Furthermore, the threat situation map integrates lidar point cloud data, millimeter-wave radar trajectory prediction data, and image recognition data from the multi-source data acquisition and fusion module, extracts dynamic obstacle motion features using a spatiotemporal graph convolutional network, combines the terrain data of the UAV flight area in the three-dimensional digital twin model, and constructs a three-dimensional threat field strength distribution model using a federated learning algorithm.

[0031] Based on the three-dimensional threat field strength distribution model, the collision probability of the UAV is calculated through Monte Carlo risk simulation, and the energy consumption risk is predicted based on the evaluation fusion function of the real-time path planning module. Finally, a multi-dimensional threat situation map including dynamic obstacle hot zones, energy risk contour lines and communication blind zone markings is obtained, and it is displayed by superimposing the augmented reality navigation interface of the main control and collaborative learning modules, which serves as the basis for dynamic weight allocation for the real-time path planning module.

[0032] Furthermore, the communication relay module is built on a deep Q-network, improves anti-interference capability through an orthogonal temporal control algorithm, dynamically allocates communication spectrum resources using game theory, and constructs a quantum key distribution channel to ensure data transmission security, and uploads the flight status data of the UAV collaborative control module to the main control and collaborative learning module in real time.

[0033] This application also provides a real-time path planning method based on dynamic weight allocation and multi-source data fusion of UAVs. This method is implemented based on the aforementioned real-time path planning system for UAVs based on dynamic weight allocation and multi-source data fusion, and includes the following steps:

[0034] S1. By acquiring multi-source data, obtain multi-source data from UAV sensors, terrain data of the UAV flight area, and UAV flight condition data, and preprocess the multi-source data from UAV sensors to construct a multi-source environmental perception data matrix;

[0035] S2. Based on the multi-source environmental perception data matrix, the terrain complexity and dynamic obstacle threat level are calculated using fractal geometry algorithm and Lyapunov rule model respectively. A multi-UAV weight allocation model is constructed using Nash equilibrium theory to obtain the multi-UAV mission cooperative control sequence.

[0036] S3. Construct a real-time path planning model and generate UAV trajectory data based on the multi-source environmental perception data matrix and the multi-UAV collaborative control sequence;

[0037] S4. Based on the UAV trajectory data obtained in step S3, the UAV is controlled by quaternion interpolation algorithm, and the thrust of the UAV multi-rotor is allocated based on pseudo-inverse method. The obstacle avoidance action is completed within 200ms by combining the lidar unit and visual sensor, and the real-time flight parameters are output.

[0038] S5. Based on real-time flight parameters and multi-source environmental perception data matrix, a three-dimensional digital twin model containing terrain, obstacle and meteorological data is constructed. Multi-source information is integrated to generate a dynamic threat situation map, and federated learning algorithm is used to update the local model of each UAV in real time.

[0039] The beneficial effects of this invention are that it quantifies terrain complexity using fractal geometry, automatically increases the weight of LiDAR, and ensures that point cloud data dominates perception. It predicts obstacle trajectories using a spatiotemporal graph convolutional network, and triggers weight adjustments based on threat level classification. An ARIMA-LSTM hybrid model compensates for 30ms communication latency, and Dempster-Shafer theory resolves sensor conflicts.

[0040] The improved Dijkstra algorithm reduces the sampling interval to 0.5 meters in complex environments, generating the initial path in less than 50ms. High-risk threats trigger Monte Carlo tree search, combined with UAV dynamic constraints, to generate an emergency obstacle avoidance path within 200ms. Adaptive B-spline curves achieve a smooth C3-order trajectory, and the lift-to-drag ratio model optimizes dynamic parameters, reducing energy consumption by 18%.

[0041] The improved RRT* reduces path search time by 60% through a bidirectional expansion strategy, and the online learning unit updates the heuristic function through reinforcement learning feedback, thereby improving adaptability to dynamic environments.

[0042] A three-dimensional benefit matrix of task completion, energy consumption, and safety distance is constructed to achieve Pareto optimal task allocation. Communication availability is ensured through redundant links between 5G and low-Earth orbit satellites, while data theft is blocked through quantum key distribution.

[0043] After each drone's local model is updated, it is encrypted and uploaded to the main control and collaborative learning modules to aggregate and generate a global environmental cognition model, avoiding data silos. By integrating laser point clouds, visual semantic segmentation, and meteorological data, a three-dimensional threat field strength distribution map is generated through a physical information neural network, dynamically marking obstacle hot zones and communication blind spots. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of the workflow of a real-time path planning system for unmanned aerial vehicles (UAVs) based on dynamic weight allocation and multi-source data fusion, according to an embodiment of the present invention.

[0045] Figure 2This is a flowchart illustrating a real-time path planning method for unmanned aerial vehicles (UAVs) based on dynamic weight allocation and multi-source data fusion, according to an embodiment of the present invention.

[0046] Figure 3 This is a schematic diagram of the terminal device structure of a UAV real-time path planning system based on dynamic weight allocation and multi-source data fusion, according to an embodiment of the present invention.

[0047] Figure 4 This is a schematic diagram of a computer-readable storage medium structure for a real-time path planning system for unmanned aerial vehicles (UAVs) based on dynamic weight allocation and multi-source data fusion, according to an embodiment of the present invention.

[0048] In the diagram, 200 is the terminal device, 210 is the memory, 211 is the RAM, 212 is the cache memory, 213 is the ROM, 214 is the program / utility, 215 is the program module, 220 is the processor, 230 is the bus, 240 is the external device, 250 is the I / O interface, 260 is the network adapter, and 300 is the program product. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention; that is, the described embodiments are only a part of the embodiments of the invention, and not all of them. The components of the embodiments of the invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0050] Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or machine that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or machine. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or machine that includes said element.

[0051] Example 1:

[0052] like Figure 1 As shown, this embodiment 1 provides a multi-source fusion real-time path planning system based on dynamic weight allocation and UAVs, including:

[0053] The multi-source data acquisition and fusion module is used to acquire multi-source data from UAV sensors, terrain data of the UAV flight area, and UAV flight condition data, and to preprocess the multi-source data from UAV sensors to construct a multi-source environmental perception data matrix.

[0054] The dynamic weight allocation module is used to generate a weight coefficient control sequence based on the multi-source fusion sensing data matrix, UAV flight area terrain data and UAV flight condition data, and dynamically adjust the weight coefficients of UAV sensors.

[0055] The real-time path planning module is used to generate the UAV's global path and local obstacle avoidance trajectory through a hierarchical path planning model based on the multi-source environmental perception data matrix and weight coefficient control sequence, and to dynamically optimize and smooth the global path to obtain UAV trajectory data.

[0056] The UAV collaborative control module is used to perform real-time flight control and flight command calculation for a cluster of multiple UAVs based on the UAV trajectory data output by the real-time path planning module.

[0057] The communication relay module is used to establish a dual-mode communication link with low-orbit satellites via 5G network to transmit the UAV flight status information to the implementation path planning module and the main control and collaborative learning module;

[0058] The main control and collaborative learning module is used to construct a three-dimensional digital twin model based on the terrain data of the UAV's flight area and the UAV's flight status information, and to perform online updates and global optimization of each module based on the federated learning algorithm.

[0059] Specifically, the multi-source data acquisition and fusion module integrates lidar, millimeter-wave radar, visual sensors, and BeiDou positioning units;

[0060] The dynamic weight allocation module adopts a hybrid decision-making mechanism that combines fuzzy logic and reinforcement learning, and dynamically adjusts the weight coefficients of each sensor according to environmental complexity, threat level and UAV status.

[0061] The real-time path planning module incorporates an improved RRT* algorithm and a Markov decision process, and uses a hierarchical planning architecture to generate globally optimal paths and local obstacle avoidance trajectories.

[0062] The UAV collaborative control module includes a dual-redundant flight control system and a dynamic obstacle avoidance unit;

[0063] The communication relay module supports dual-mode communication of 5G and low-Earth orbit satellites;

[0064] The main control and collaborative learning modules are deployed on a collaborative learning platform based on digital twins, which updates the environmental cognition models of each UAV through federated learning.

[0065] Specifically, taking forest fire monitoring as an example, the application of the dynamic weight allocation module in forest fire monitoring is as follows: when a drone passes through a dense smoke area (environmental complexity 0.85), the fuzzy logic automatically increases the weight of the lidar with strong anti-smoke capability to 0.78; at the same time, when a mobile fire truck is detected in the direction of fire spread (threat level 4), the reinforcement learning module increases the weight of the millimeter-wave radar by 0.4 based on historical obstacle avoidance data, and reduces the weight of the visual sensor to 0.1 in real time to avoid misjudgment caused by smoke interference.

[0066] Specifically, taking urban logistics and distribution scenarios as an example, the application of the real-time path planning module in urban logistics and distribution is as follows: When a drone flies from a warehouse to an office building, the improved RRT algorithm adopts bidirectional expansion, generating a forward path tree from the warehouse end (sampling bias is the commercial area air corridor) and a reverse path tree from the office building end (sampling bias is the landing platform area). When the distance between the two trees is 8 meters, the connection mechanism is triggered, which shortens the search time by 52% compared to the traditional RRT.

[0067] Specifically, the dynamic weight allocation module uses fractal geometry to calculate terrain complexity to assess the environmental situation, employs a spatiotemporal graph convolutional network to identify dynamic obstacles and predict threat levels, constructs an adaptive adjustment model based on the Lyapunov index to optimize sensor weight allocation, and applies Nash equilibrium theory to conduct collaborative decision-making for multi-UAV missions.

[0068] The spatiotemporal graph convolutional network processes dynamic obstacle data by constructing a spatiotemporal adjacency matrix. It uses LiDAR point cloud sequences and millimeter-wave radar trajectory prediction data as spatial graph node features and temporal graph edge weights, respectively. It extracts obstacle motion pattern features using a hierarchical spatiotemporal convolutional structure and predicts obstacle motion trajectories within the next 3 seconds through a time sliding window mechanism. The threat level is divided into five quantitative indicators based on the probability of trajectory collision: when the collision time TTC < 2 seconds and the distance threshold < 5 meters, it is an emergency level; when the TTC < 5 seconds and the distance threshold < 10 meters, it is a high-risk level; when the TTC < 10 seconds and the distance threshold < 20 meters, it is a warning level; and the rest are safe and ineffective levels.

[0069] Among them, the Lyapunov index adaptive adjustment model constructs an energy function to evaluate the confidence of each sensor in real time and normalizes the environmental complexity. When the environmental complexity exceeds 0.7, it automatically increases the weight of the lidar to the range of 0.6-0.8. When the threat level reaches the high-risk level or above, the weight of the millimeter-wave radar is dynamically increased by 0.3-0.5.

[0070] Among them, the Nash equilibrium collaborative decision-making establishes the game payoff matrix among drones. The payoff function includes three dimensions: task completion, energy efficiency, and safe distance. The Pareto optimal allocation scheme is achieved through iterative strategy optimization. The final generated environmental complexity and threat level quantification results are fed back to the main control and collaborative learning modules in real time through the 5G link.

[0071] Specifically, taking the highway inspection scenario as an example, the workflow of the spatiotemporal graph convolutional network in the highway inspection scenario is as follows:

[0072] First, the 64-line point cloud scanned by the lidar is clustered into 150 nodes (each node represents a vehicle / roadblock). The truck's movement trajectory tracked by the millimeter-wave radar is used as the temporal side weight, and 10 consecutive frames of data are used to form a spatiotemporal matrix. Then, when it is identified that the truck will change lanes in 2.3 seconds, with a conflict probability of 87%, the system judges it as an emergency threat (TTC=1.8 seconds, distance 4.2 meters), and immediately increases the weight of the millimeter-wave radar by 0.45.

[0073] Specifically, taking energy regulation scenarios as an example, the Lyapunov adaptive model is applied in energy regulation scenarios as follows: when the battery power drops to 28%, the model calculates the sensor energy consumption derivative dV / dt = -0.32 (exceeding the threshold of -0.2), automatically reduces the weight of the high-power LiDAR from 0.6 to 0.3, increases the visual weight to 0.7, and extends the battery life by 12 minutes.

[0074] Specifically, taking disaster search and rescue scenarios as an example, the decision-making process of Nash equilibrium collaboration in disaster search and rescue is as follows: Three drones are set up to undertake life detection, material delivery, and terrain mapping tasks respectively, and the payoff matrix is ​​calculated to obtain the following results:

[0075] The safety distance weight for the life detector is 0.7 (close-range scanning is required);

[0076] The energy efficiency weight of the material delivery aircraft is 0.6 (it can fly at full load).

[0077] The task completion weight of the surveying machine is 0.8 (meeting the full coverage requirement);

[0078] Based on the weighted strategy of the three drones, the Pareto optimal solution is finally generated: the altitude of the life detector is reduced to 50 meters, the path of the material delivery drone is shortened by 300 meters, and the scanning interval of the surveying drone is increased by 2 meters.

[0079] Specifically, the multi-source data fusion module performs heterogeneous data calibration through microsecond-level time synchronization and sub-meter-level spatial registration, establishes a hidden Markov model of sensor health status for confidence assessment, uses Dempster-Shafer evidence theory to resolve multi-source data conflicts, and constructs an ARIMA-LSTM hybrid model to predict and compensate for communication delay data. The processed fused data is then input into the real-time path planning module to generate a trajectory. For example, in a bridge inspection scenario: first, a 200,000-point cloud is generated by scanning the bridge piers with LiDAR, and high-resolution images of cracks are captured by a visual sensor; then, the SIFT algorithm is used to extract rivet feature points from the bridge piers (matching 56 key points per frame); next, RANSAC is used to remove mismatched points (such as interference caused by attached moss), resulting in a final registration error of <1.3 cm.

[0080] The specific process of communication delay compensation:

[0081] The ARIMA model predicts a baseline latency of 28ms (based on 100 historical data sets).

[0082] LSTM network learns burst interference patterns: when encountering electromagnetic interference, the predicted compensation value is increased by 15ms;

[0083] The actual data packet transmission delay was 33ms, which was reduced to 18ms after compensation.

[0084] The real-time path planning module adopts a layered architecture: a global topology layer to generate the initial path, a local obstacle avoidance layer to deal with dynamic obstacles, and a trajectory optimization layer to smooth the trajectory.

[0085] Among them, the global topology layer constructs a navigation topology map based on the 3D point cloud data output by the multi-source data fusion module, and uses an improved Dijkstra algorithm to calculate the initial waypoint sequence. The path sampling density is dynamically adjusted according to the environmental complexity fed back by the dynamic weight allocation module. When the complexity is >0.6, the sampling interval is reduced to 0.5 meters.

[0086] The local obstacle avoidance layer triggers path replanning by predicting the dynamic obstacle threat level through a spatiotemporal graph convolutional network. When a high-risk threat is detected, the Monte Carlo tree search algorithm is activated, and a node expansion strategy based on UAV dynamic constraints is adopted to generate an emergency avoidance path within 200ms. The evaluation function integrates the three dimensions of threat field strength, energy consumption rate, and heading deviation, and the weight coefficients are updated online through the master control and collaborative learning modules.

[0087] The trajectory optimization layer utilizes the dynamic adjustment mechanism of node vectors of adaptive B-spline curves to optimize the curve order and control point density in real time based on the positioning accuracy fed back by the communication relay module. The curve continuity meets the C3-order differentiability requirement. At the same time, a dynamic correlation model between lift-to-drag ratio and power system is established, and the power parameter configuration is optimized through backpropagation algorithm. The optimized trajectory data is synchronized to the UAV cooperative control module for execution in real time.

[0088] Application of global topology layer in power transmission inspection in mountainous areas:

[0089] Based on a terrain fractal dimension value of 0.75 (>0.6 threshold), the sampling interval of the navigation map is refined to 0.5 meters; the improved Dijkstra algorithm prioritizes straight-line airspace between high-voltage power line towers, avoiding steep canyon areas; the generated initial path contains 89 waypoints, with a computation time of 46ms; a local obstacle avoidance layer is used to address sudden threats from flocks of birds; ST-GCN predicts that flocks of birds will intrude into the flight path within 1.2 seconds (high risk level); Monte Carlo tree search is initiated, generating 3 candidate paths based on quadcopter dynamic constraints:

[0090] Option A: Climb 15 meters (energy consumption +20%, safety factor 0.95); Option B: Turn right to avoid (heading deviation 12°, safety factor 0.88); Option C: Decelerate and hover (time +8 seconds, safety factor 0.99); Evaluation function selected: Option C (safety weight 0.6), replanning completed in 200ms.

[0091] Example of trajectory smoothing in trajectory optimization layer:

[0092] In urban canyon areas with weak GPS signals (positioning error 1.8 meters), a 3rd-order B-spline curve (control point spacing 2 meters) is used; after entering open areas (error < 0.3 meters), a 5th-order curve (control point spacing 0.5 meters) is switched; the power is optimized through the lift-to-drag ratio model: the rotor speed is automatically increased by 12% in headwind sections, reducing energy consumption by 17%.

[0093] The communication relay module uses a deep Q network to achieve intelligent routing selection, improves anti-interference capability through orthogonal temporal control technology, dynamically allocates communication spectrum resources using game theory, and constructs a quantum key distribution channel to ensure data transmission security. It also uploads the flight status data of the UAV collaborative control module to the main control and collaborative learning module in real time.

[0094] The improved RRT* algorithm integrates environmental features and historical path information to dynamically adjust the sampling strategy, adopts a bidirectional expansion mechanism to accelerate the path search process, performs smooth optimization of the original trajectory through optimal control theory, and sets up an online learning unit to update the heuristic function parameters based on reinforcement learning feedback. The optimization results are converted into flight commands through the UAV cooperative control module.

[0095] The UAV collaborative control module uses a quaternion interpolation algorithm to achieve fault-tolerant control for flight command calculation. It optimizes the multi-rotor thrust distribution scheme based on the pseudo-inverse method. The dynamic obstacle avoidance unit completes obstacle avoidance actions within 200ms by fusing LiDAR and visual sensors. The formation keeping unit uses the virtual structure method to maintain the cluster geometry. Control commands are distributed to each UAV node via the communication relay module.

[0096] Specifically, the dual-mode communication switching logic is as follows: Taking the scenario of drones operating on a maritime platform as an example, when the drone is operating on a maritime platform, the 5G signal strength drops to -85dBm; the DQN network evaluates the satellite link latency as 120ms (<200ms threshold); the switching action is executed and quantum key distribution is started to generate a 256-bit encryption key.

[0097] Breakdown of the quadcopter's emergency obstacle avoidance maneuver: The lidar detected the suddenly appearing advertising balloon at a distance of 3.2 meters; the visual sensor confirmed within 0.2 seconds that the target was a lightweight object (non-metallic).

[0098] The dynamic obstacle avoidance unit calculates the minimum avoidance torque: the output of the right front motor is increased to 85%; the output of the left rear motor is reduced to 45%; the tilt angle is 22° for side flight to avoid obstacles; the whole process takes 180ms and energy consumption is reduced by 35% (compared to full power climb).

[0099] The main control and collaborative learning module constructs a three-dimensional digital twin model containing terrain features and obstacle dynamic information. It updates the UAV trajectory data (decision strategy) of the real-time path planning module through a transfer learning framework. It integrates the multi-source environmental perception data matrix of the multi-source data acquisition and fusion module to generate a threat situation map and provides an augmented reality navigation interface and voice command interaction function. The digital twin model and the UAV collaborative control module establish a two-way data channel.

[0100] The threat situation map is generated by integrating LiDAR point cloud, millimeter-wave radar trajectory prediction and visual semantic segmentation data from a multi-source data fusion module, extracting dynamic obstacle motion features using a spatiotemporal graph convolutional network, combining terrain elevation and meteorological data from a digital twin model, and constructing a three-dimensional threat field strength distribution model using a physical information neural network.

[0101] The threat level assessment is based on a three-dimensional threat field strength distribution model. The collision probability of the drone is calculated through Monte Carlo risk simulation. The energy consumption risk is predicted based on the evaluation fusion function of the real-time path planning module. The resulting multi-dimensional threat situation map includes dynamic obstacle hot zones, energy risk contour lines, and communication blind zone markings. It is overlaid and displayed through the augmented reality navigation interface of the main control and collaborative learning modules, providing a basis for dynamic weight allocation for the dynamic weight allocation module.

[0102] Generation of 3D threat situation map:

[0103] At the site of a chemical plant leak: LiDAR was used to construct a 3D model of the storage tank; millimeter-wave radar was used to track the spread of toxic gases; and visual sensors were used to identify cracks at the leak source.

[0104] Physical information neural network calculates threat field strength:

[0105] Leakage point core area: Threat value 0.92 (red hot zone);

[0106] 50 meters downwind: Threat value 0.75 (Yellow Alert);

[0107] Headwind safe zone: Threat value 0.31 (green).

[0108] The AR navigation interface overlays the disaster evacuation route (bypassing the western safety passage).

[0109] Federated learning model update:

[0110] Drone No. 1 learned in dense fog that: the confidence level of millimeter-wave radar increases by 22% when humidity is >80%; encrypted data is uploaded to the main control and collaborative learning modules; and after the global model is updated, it is distributed to all drones, improving the perception consistency of the cluster in severe weather.

[0111] Example 2

[0112] like Figure 2 As shown in Example 1, Example 2 of this invention proposes a real-time path planning method based on dynamic weight allocation and multi-source data fusion of UAVs, implemented by an active power filter current tracking control system based on an improved butterfly algorithm.

[0113] Specifically, the method includes the following steps:

[0114] S1. By acquiring multi-source data, obtain multi-source data from UAV sensors, terrain data of the UAV flight area, and UAV flight condition data, and preprocess the multi-source data from UAV sensors to construct a multi-source environmental perception data matrix;

[0115] S2. Based on the multi-source environmental perception data matrix, the terrain complexity and dynamic obstacle threat level are calculated using fractal geometry algorithm and Lyapunov rule model respectively. A multi-UAV weight allocation model is constructed using Nash equilibrium theory to obtain the multi-UAV mission cooperative control sequence.

[0116] S3. Construct a real-time path planning model and generate UAV trajectory data based on the multi-source environmental perception data matrix and the multi-UAV collaborative control sequence;

[0117] S4. Based on the UAV trajectory data obtained in step S3, the UAV is controlled by quaternion interpolation algorithm, and the thrust of the UAV multi-rotor is allocated based on pseudo-inverse method. The obstacle avoidance action is completed within 200ms by combining the lidar unit and visual sensor, and the real-time flight parameters are output.

[0118] S5. Based on real-time flight parameters and multi-source environmental perception data matrix, a three-dimensional digital twin model containing terrain, obstacle and meteorological data is constructed. Multi-source information is integrated to generate a dynamic threat situation map, and federated learning algorithm is used to update the local model of each UAV in real time.

[0119] Example 3

[0120] like Figure 3 As shown in the figure, based on the first embodiment, this embodiment 3 proposes a terminal device for a real-time path planning system based on dynamic weight allocation and multi-source data fusion of UAVs. The terminal device 200 includes at least one memory 210, at least one processor 220, and a bus 230 connecting different platform systems.

[0121] The memory 210 may include a readable medium in the form of volatile memory, such as RAM 211 and / or cache memory 212, and may further include ROM 213.

[0122] The memory 210 also stores a computer program that can be executed by the processor 220, causing the processor 220 to execute any of the above-mentioned real-time path planning systems based on dynamic weight allocation and UAV multi-source data fusion in the embodiments of this application. The specific implementation method and the achieved technical effects are consistent with those described in the embodiments of the above methods, and some details will not be repeated here. The memory 210 may also include a program / utility 214 having a set (at least one) of program modules 215. Such program modules include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment.

[0123] Accordingly, processor 220 can execute the aforementioned computer program, as well as executable program / utility 214.

[0124] Bus 230 can represent one or more of several types of bus structures, including a memory bus or memory controller, peripheral bus, graphics acceleration port, processor, or a local bus using any of the various bus structures.

[0125] Terminal device 200 can also communicate with one or more external devices 240, such as keyboards, pointing devices, Bluetooth devices, etc., and with one or more devices capable of interacting with it, and / or with any device that enables it to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication can be performed via I / O interface 250. Furthermore, terminal device 200 can communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via network adapter 260. Network adapter 260 can communicate with other modules of terminal device 200 via bus 230. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with terminal device 200, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0126] Example 4

[0127] like Figure 4 As shown, based on Embodiment 1, this embodiment proposes a computer-readable storage medium for a real-time path planning system based on dynamic weight allocation and UAV multi-source data fusion. The computer-readable storage medium stores instructions that, when executed by a processor, implement any of the aforementioned real-time path planning systems based on dynamic weight allocation and UAV multi-source data fusion. The specific implementation method and the achieved technical effects are consistent with those described in the embodiments of the above systems, and some details will not be repeated.

[0128] Figure 4The present embodiment illustrates a program product 300 for implementing the above-described system, which may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product 300 of the present invention is not limited thereto. In this embodiment, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device. The program product 300 may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0129] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof. Program code for performing operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on a user computing device, partially on a user device, as a standalone software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing devices can be connected to user computing devices via any type of network, including local area networks (LANs) or wide area networks (WANs), or they can be connected to external computing devices (e.g., via the Internet using an Internet service provider).

[0130] This invention is described from the perspectives of its intended use, effectiveness, progress, and novelty. Its practical and progressive features meet the functional enhancement and use requirements emphasized by the Patent Law. The above description and drawings are merely preferred embodiments of this application and are not intended to limit this application. Therefore, all structures, devices, features, etc., that are similar to or identical to those of this application, i.e., all equivalent substitutions or modifications made in accordance with the scope of this patent application, shall fall within the scope of protection of this patent application.

[0131] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A real-time path planning system for unmanned aerial vehicles (UAVs) based on dynamic weight allocation and multi-source data fusion, characterized in that, include: The multi-source data acquisition and fusion module is used to acquire multi-source data from UAV sensors, terrain data of the UAV flight area, and UAV flight condition data, and to preprocess the multi-source data from UAV sensors to construct a multi-source environmental perception data matrix. The dynamic weight allocation module is used to generate a weight coefficient control sequence based on the multi-source fusion sensing data matrix, UAV flight area terrain data and UAV flight condition data, and dynamically adjust the weight coefficients of UAV sensors. The real-time path planning module is used to generate the UAV's global path and local obstacle avoidance trajectory through a hierarchical path planning model based on the multi-source environmental perception data matrix and weight coefficient control sequence, and to dynamically optimize and smooth the global path to obtain UAV trajectory data. The UAV collaborative control module is used to perform real-time flight control and flight command calculation for a cluster of multiple UAVs based on the UAV trajectory data output by the real-time path planning module. The communication relay module is used to establish a dual-mode communication link with low-orbit satellites via 5G network to transmit the UAV flight status information to the implementation path planning module and the main control and collaborative learning module; The main control and collaborative learning module is used to construct a three-dimensional digital twin model based on the terrain data of the UAV flight area and the UAV flight status information, and to perform online updates and global optimization of each module based on the federated learning algorithm; The multi-source data acquisition and fusion module includes a lidar unit, a millimeter-wave radar unit, a visual sensor unit, and a BeiDou satellite positioning unit; the multi-source data from the UAV sensors includes lidar point cloud data, millimeter-wave radar trajectory prediction data, image recognition data, and BeiDou satellite positioning data. The preprocessing includes heterogeneous data calibration of multi-source data from UAV sensors using time synchronization algorithms and spatial registration theory, confidence assessment of the health status of UAV sensors using a hidden Markov model, elimination of data conflicts among data in multi-source data from UAV sensors based on Dempster-Shafer evidence theory, and prediction and compensation of communication delays among data in multi-source data from UAV sensors using an ARIMA-LSTM hybrid model. The dynamic weight allocation module includes a terrain analysis unit, a threat identification unit, a confidence adjustment unit, and a Nash equilibrium collaborative decision-making unit. The terrain analysis unit is used to calculate the terrain complexity of the UAV flight area based on the terrain data of the UAV flight area using a fractal geometry algorithm. The threat identification unit is used to identify the dynamic obstacle prediction threat level based on the terrain complexity of the UAV flight area through a spatiotemporal graph convolutional network. The confidence adjustment unit is used to construct an adaptive adjustment model based on the Lyapunov index according to the terrain complexity of the UAV flight area and the predicted threat level of dynamic obstacles, and to optimize the UAV sensor weight allocation in real time. The Nash equilibrium collaborative decision-making unit, based on the terrain complexity of the UAV flight area and the predicted threat level of dynamic obstacles, combined with the multi-source fusion perception data matrix, establishes a game payoff matrix among multiple UAVs according to Nash equilibrium theory, and makes collaborative decisions on multiple UAV missions. The spatiotemporal graph convolutional network constructs a spatiotemporal adjacency matrix, using LiDAR point cloud data as spatial graph node features and millimeter-wave radar trajectory prediction data as temporal graph edge weights. Based on a hierarchical spatiotemporal convolutional structure, it extracts obstacle motion pattern features and predicts obstacle motion trajectories within the next 3 seconds using a time sliding window mechanism. The dynamic obstacle prediction threat level is divided into five quantitative indicators based on the pre-obtained collision probability of the UAV flight trajectory: emergency level when the collision time is less than 2 seconds and the distance threshold is less than 5 meters; high-risk level when the collision time is less than 5 seconds and the distance threshold is less than 10 meters; early warning level when the collision time is less than 10 seconds and the distance threshold is less than 20 meters; and safe and ineffective levels when the collision time is greater than 10 seconds and the distance threshold is greater than 20 meters. The adaptive adjustment model based on the Lyapunov index constructs an energy function to evaluate the confidence of each UAV sensor in real time and normalizes the terrain complexity of the UAV flight area. When the terrain complexity of the UAV flight area exceeds 0.7, the weight of the lidar unit is increased to 0.6-0.

8. When the threat level reaches the high-risk or emergency level, the weight of the millimeter-wave radar unit is dynamically increased to 0.3-0.

5. The game payoff matrix includes flight mission completion rate, flight energy efficiency, and flight safety distance. The game payoff matrix is ​​optimized through an iterative strategy to obtain the Pareto optimal allocation scheme and generate a weight coefficient control sequence. The final generated terrain complexity and dynamic obstacle prediction threat level of the drone flight area are fed back to the real-time path planning module and the main control and collaborative learning module in real time via the 5G network. The hierarchical path planning model includes a global topology layer, a local obstacle avoidance layer, and a trajectory optimization layer. The global topology layer constructs a navigation topology map based on LiDAR point cloud data and calculates the initial waypoint sequence of the UAV using an improved Dijkstra algorithm. The path sampling density of the initial waypoint sequence is dynamically adjusted according to the terrain complexity of the UAV's flight area. When the terrain complexity of the UAV's flight area is >0.6, the path sampling density is reduced to 0.5 meters. When there are dynamic obstacles or highly complex terrain in the flight area, the global topology layer uses an improved RRT algorithm. The algorithm dynamically adjusts the sampling strategy by integrating environmental features and historical path information to generate an initial waypoint sequence; The local obstacle avoidance layer constructs an evaluation fusion function by predicting the threat level through dynamic obstacles and replans the initial waypoint sequence. When a high-risk threat is detected, the Monte Carlo tree search algorithm is activated, and an emergency avoidance path is generated within 200ms based on the node expansion strategy of UAV dynamic constraints. The evaluation fusion function includes the threat field strength value, energy consumption rate, and heading deviation. The trajectory optimization layer is based on the node vector dynamic adjustment mechanism of the adaptive B-spline curve. It optimizes the curve order and control point density in real time according to the UAV flight status information fed back by the communication relay module. The continuity of the curve order meets the C3 differentiability requirement. A dynamic correlation model between the UAV lift-to-drag ratio and the UAV power system is established. The UAV power parameter configuration is optimized through the backpropagation algorithm to obtain the final UAV trajectory data, which is transmitted to the UAV cooperative control module in real time.

2. The real-time path planning system for unmanned aerial vehicles (UAVs) based on dynamic weight allocation and multi-source data fusion according to claim 1, characterized in that, The improved RRT The algorithm is used for path planning in the global topology layer. It dynamically adjusts the sampling strategy by integrating environmental features and historical path information, and uses a bidirectional expansion mechanism to accelerate the path search process. It performs preliminary smoothing optimization on the generated initial trajectory through optimal control theory, and sets up an online learning unit to update the heuristic function parameters based on reinforcement learning feedback. The optimization result is used as the output of the global topology layer for further processing by the local obstacle avoidance layer and trajectory optimization layer. Flight commands are generated through the UAV collaborative control module.

3. The real-time path planning system for unmanned aerial vehicles (UAVs) based on dynamic weight allocation and multi-source data fusion according to claim 2, characterized in that, The UAV collaborative control module performs fault-tolerant control by solving flight commands using a quaternion interpolation algorithm, optimizes the multi-rotor thrust distribution scheme based on the pseudo-inverse method, and the dynamic obstacle avoidance unit completes obstacle avoidance actions within 200ms by fusing perception between the lidar unit and the visual sensor. The formation keeping unit maintains the cluster geometry using the virtual structure method, and control commands are distributed to each UAV node via the communication relay module.

4. The real-time path planning system for unmanned aerial vehicles (UAVs) based on dynamic weight allocation and multi-source data fusion according to claim 3, characterized in that, The main control and collaborative learning module constructs a three-dimensional digital twin model containing terrain data of the UAV flight area and UAV flight status information. It updates the UAV trajectory data of the real-time path planning module through a transfer learning framework and generates a threat situation map by integrating the multi-source environmental perception data matrix of the multi-source data acquisition and fusion module.

5. The real-time path planning system for unmanned aerial vehicles (UAVs) based on dynamic weight allocation and multi-source data fusion according to claim 4, characterized in that, The threat situation map integrates lidar point cloud data, millimeter-wave radar trajectory prediction data, and image recognition data from multi-source data acquisition and fusion modules. It uses a spatiotemporal graph convolutional network to extract dynamic obstacle motion features, combines the terrain data of the UAV flight area in the three-dimensional digital twin model, and uses a federated learning algorithm to construct a three-dimensional threat field strength distribution model. Based on the three-dimensional threat field strength distribution model, the collision probability of the UAV is calculated by the Monte Carlo risk simulation algorithm, and the energy consumption risk is predicted based on the evaluation fusion function of the real-time path planning module. Finally, a multi-dimensional threat situation map including dynamic obstacle hot zones, energy risk contour lines and communication blind zone markings is obtained. The map is then overlaid and displayed through the augmented reality navigation interface of the main control and collaborative learning modules, providing a basis for dynamic weight allocation for the dynamic weight allocation module.

6. A real-time path planning method based on dynamic weight allocation and multi-source data fusion of UAVs, the method being implemented based on a real-time path planning system for UAVs based on dynamic weight allocation and multi-source data fusion as described in any one of claims 1-5, characterized in that, Includes the following steps: S1. By acquiring multi-source data, obtain multi-source data from UAV sensors, terrain data of the UAV flight area, and UAV flight condition data, and preprocess the multi-source data from UAV sensors to construct a multi-source environmental perception data matrix; S2. Based on the multi-source environmental perception data matrix, the terrain complexity and dynamic obstacle threat level are calculated using fractal geometry algorithm and Lyapunov rule model respectively. A multi-UAV weight allocation model is constructed using Nash equilibrium theory to obtain the multi-UAV mission cooperative control sequence. S3. Construct a real-time path planning model and generate UAV trajectory data based on the multi-source environmental perception data matrix and the multi-UAV collaborative control sequence; S4. Based on the UAV trajectory data obtained in step S3, the UAV is controlled by quaternion interpolation algorithm, and the thrust of the UAV multi-rotor is allocated based on pseudo-inverse method. The obstacle avoidance action is completed within 200ms by combining the lidar unit and visual sensor, and the real-time flight parameters are output. S5. Based on real-time flight parameters and multi-source environmental perception data matrix, a three-dimensional digital twin model containing terrain, obstacle and meteorological data is constructed. Multi-source information is integrated to generate a dynamic threat situation map, and federated learning algorithm is used to update the local model of each UAV in real time.

Images