Intelligent monitoring and management platform of unmanned aerial vehicle flight control architecture

Abstract

The invention discloses an intelligent monitoring and management platform of an unmanned aerial vehicle flight control architecture. The monitoring and management platform comprises a multi-mode sensing module, a dynamic decision module, a distributed cooperation module and a safety protection module. The multi-modal sensing module fuses GPS, radar and visual data through a space-time alignment algorithm and an attention mechanism, and has a sensor health degree self-diagnosis function; the dynamic decision-making module adopts a layered reinforcement learning architecture, inputs a task target, meteorological data and airspace limitation, outputs an optimal track, embeds a fast random tree and model prediction control, and generates an obstacle avoidance track in real time; according to the method, data of each sensor are dynamically weighted through an attention mechanism, and the problem of single-point faults such as GPS failure is solved; through a dynamic decision module, reinforcement learning and rolling optimization are combined to realize quick response from global planning to local obstacle avoidance; a distributed collaboration module is adopted, and continuous execution of tasks under communication interruption is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle (UAV) management platforms, and particularly to an intelligent monitoring and management platform for a UAV flight control architecture. Background Art

[0002] Currently, most UAV flight control systems are based on traditional proportional-integral-derivative (PID) control architectures, relying on preset rules and fixed parameters to achieve basic flight control, such as attitude stabilization and waypoint navigation. However, as UAV application scenarios develop towards complexity and clustering, such as urban logistics, disaster relief, and multi-UAV collaborative inspection, the existing technologies face the following problems: Single-sensor data is vulnerable to environmental interference. For example, in an urban environment, due to high-rise building blockages, the GPS signal of a UAV may be intermittently lost or interfered with, resulting in inaccurate positioning. Existing flight control systems often rely on a single sensor data source and lack effective fusion of multi-source data, such as vision and radar, and cannot provide a comprehensive perception of the flight environment. Flight strategies rely on offline planning and cannot respond to dynamic obstacles in real time. For example, a UAV may encounter dynamic obstacles such as birds, and it is difficult for the UAV to avoid obstacles in real time, resulting in the interruption of communication between the UAV and the ground control station. Summary of the Invention

[0003] The purpose of the present invention is to provide an intelligent monitoring and management platform for a UAV flight control architecture to solve the problems raised in the above background art.

[0004] To achieve the above purpose, the present invention provides the following technical solution: An intelligent monitoring and management platform for a UAV flight control architecture, the monitoring and management platform includes a multi-modal perception module, a dynamic decision-making module, a distributed cooperation module, and a security protection module; The multi-modal perception module fuses GPS, radar, and visual data through a spatio-temporal alignment algorithm and an attention mechanism, and has a self-diagnosis function for sensor health; The dynamic decision-making module adopts a hierarchical reinforcement learning architecture, with the input being task objectives, meteorological data, and airspace restrictions, and the output being the optimal flight path. It embeds a rapidly-exploring random tree (RRT) and model predictive control (MPC) to generate an obstacle avoidance trajectory in real time. The constraint conditions include: Wherein, is the state, is the control quantity, is the smoothing weight; The distributed cooperation module adopts an improved distributed contract network. Each UAV stores its local capability matrix based on a trusted execution environment. The matching degree calculation includes remaining battery power, sensor status, and task history. The status is broadcast through the Gossip protocol, and the task recipient achieves load balancing through the consistent hashing algorithm to realize decentralized task negotiation.

[0005] Preferably, the spatio-temporal alignment algorithm uses a spatial coordinate system transformation to apply a rotation plus translation transformation to the millimeter-wave radar point cloud data and map it to the coordinate system of the visual point cloud; the time stamp offset is optimized through a sliding window to align sensor data with a frequency difference exceeding 10%.

[0006] Preferably, the sensor health self-diagnosis and evaluation method includes calculating the residuals between the sensor data and the fusion result, marking as a fault when the residuals of continuous N frames exceed the dynamic threshold; triggering a redundancy switching strategy and preferentially enabling the same type of standby sensor.

[0007] Preferably, the hierarchical reinforcement learning architecture includes using the deep deterministic policy gradient algorithm with an input dimension of task objective encoding, meteorological data vector, no-fly zone coordinates and altitude layer, and an output of a waypoint sequence of the global flight path; using the proximal policy optimization algorithm with an input of the local segment of the high-level flight path and real-time attitude data, and an output of the motor speed and rudder surface deflection angle.

[0008] Preferably, a wind resistance model is introduced into the model predictive control optimization objective function of the dynamic decision module: where is the wind resistance coefficient, is the air density, and A is the cross-sectional area of the unmanned aerial vehicle facing the wind.

[0009] Preferably, the communication interruption response method of the distributed cooperation module includes using long-distance low-power and medium-distance medium-rate dual-mode communication. When unmanned aerial vehicle A cannot directly connect to the base station, the data packet is temporarily stored locally; when unmanned aerial vehicle A encounters another unmanned aerial vehicle B entering the communication range, the data is transmitted to B; unmanned aerial vehicle B continues to find the next relay node until the data reaches the target, and uses neighboring unmanned aerial vehicles to relay the transmission of key data, marking the urgency of the data, and giving high-priority data priority transmission; low-priority data is delayed or discarded.

[0010] Preferably, the method for optimizing the time stamp offset by the sliding window includes the following steps: S11: Construct a sliding window with a length of L and cache the original data of each sensor within the window; S12: Perform non-linear least squares optimization on the time stamp offset Δt between the radar and visual data within the window: ; S13: When the frequency difference of the sensors exceeds 10%, use cubic spline interpolation to upsample the data of the low-frequency sensors.

[0011] Preferably, the working method of the monitoring and management platform includes the following steps: S1: The multi-modal perception module fuses GPS, vision, and radar data to detect building obstacles and no-fly zones. When it self-diagnoses that the GPS signal is blocked by a building, it switches to the vision- and radar-dominated mode; S2: The dynamic decision-making module generates a global flight path based on the logistics task objective and real-time wind speed. When it detects a sudden flock of birds, the rapidly-exploring random tree generates three obstacle avoidance paths, and the energy consumption model predictive control selects the S-shaped bypass trajectory with the lowest energy consumption; S3: The distributed cooperation module assigns the parcel delivery task to the idle drone with the help of the consistent hashing algorithm. When the communication of drone C is interrupted, the DTN protocol is initiated, and the neighboring drone D relays the parcel receipt data.

[0012] Preferably, the energy consumption model calculates the total power consumption of the bypass path: where is the motor power - airspeed function, which is obtained by fitting measured data.

[0013] Preferably, the security protection module uses the TLS encryption protocol to ensure data transmission.

[0014] Compared with the prior art, the beneficial effects of the present invention are: The present invention dynamically weights the data of each sensor through the attention mechanism to solve the single-point failure problem of GPS failure; through the dynamic decision-making module, combining reinforcement learning and rolling optimization, it realizes a fast response from global planning to local obstacle avoidance; and adopts a distributed cooperation module to ensure the continuous execution of tasks under communication interruption. Description of the Drawings

[0015] Figure 1 is the system block diagram of the intelligent monitoring and management platform for the UAV flight control architecture of the present invention.

[0016] Figure 2 is the working flowchart of the intelligent monitoring and management platform for the UAV flight control architecture of the present invention. Detailed Embodiments

[0017] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0018] Please refer to Figures 1 to 2, the present invention provides a technical solution: an intelligent monitoring and management platform for an unmanned aerial vehicle flight control architecture, which includes a multi-modal perception module, a dynamic decision-making module, a distributed collaboration module, and a security protection module; The multi-modal perception module fuses GPS, radar, and visual data through a spatio-temporal alignment algorithm and an attention mechanism, and has a self-diagnosis function for sensor health.

[0019] The spatio-temporal alignment algorithm uses a spatial coordinate system transformation to apply a rotation and a translation transformation to the millimeter-wave radar point cloud data, and maps it to the coordinate system of the visual point cloud; the timestamp offset is optimized through a sliding window to align sensor data with a frequency difference exceeding 10%.

[0020] Spatial alignment: Establish a radar point cloud coordinate system and a visual point cloud coordinate system, and solve the SE(3) transformation matrix: where R is the rotation matrix and t is the translation vector; The iterative closest point algorithm is used for registration optimization, and the objective function is to minimize the distance between the radar and the visual point cloud: The optimal T is solved by singular value decomposition and iterated until the error converges.

[0021] The method for optimizing the timestamp offset by a sliding window includes the following steps: S11: Construct a sliding window with a length of L and cache the original data of each sensor within the window; S12: Perform non-linear least squares optimization on the timestamp offset Δt between the radar and the visual data within the window: ; S13: When the frequency difference of the sensors exceeds 10%, cubic spline interpolation is used to upsample the data of the low-frequency sensors.

[0022] The attention mechanism fusion mechanism includes the following steps: S21: Extract GPS data, including extracting longitude, latitude, altitude, and confidence; radar point cloud, extract obstacle distance, speed, and reflection intensity; visual data, extract feature points; S22: The encoder sets the GPS data as a fully connected network encoding; the radar point cloud is set as a point cloud neural network encoding; the visual data is set as a convolutional neural network encoding; S23: Calculate the correlation scores of each sensor data through a multi-head attention module; S24: When the GPS signal is weak, such as when the confidence is less than 0.7, automatically reduce the GPS weight and increase the radar and visual weights; when the vision is blocked, such as when the number of feature points drops sharply, increase the radar and GPS weights; S25: The final fusion result is the attention weights of GPS, radar, and vision.

[0023] The method for self-diagnosis and evaluation of sensor health in the multi-modal perception module includes calculating the residuals between the data of each sensor and the fusion result, calculating the mean μ and standard deviation σ based on a sliding window of historical data, such as the most recent 50 frames; the threshold thr = μ + 3σ, and if it exceeds the threshold, it is marked as abnormal; for example, when the main GPS fails, the backup GPS module is enabled; if both GPSs fail, switch to vision-radar fusion positioning.

[0024] The dynamic decision-making module adopts a hierarchical reinforcement learning architecture. The inputs are the task objective, meteorological data, and airspace restrictions, and the output is the optimal flight path. It embeds a rapid random tree and model predictive control to generate an obstacle avoidance trajectory in real time. The constraint conditions include: Among them, is the state, is the control variable, is the smoothing weight; The path generation method of the rapid random tree includes the following steps: S31: After detecting an obstacle, use the current UAV position as the root node and randomly sample the target direction to expand the tree; S32: Optimize the cost of the path nodes: Among them = 0.6, = 0.4; S33: Output the 3 candidate paths with the lowest cost for model predictive control to select.

[0025] The hierarchical reinforcement learning architecture includes using the deep deterministic policy gradient algorithm. The input dimension is the task objective encoding, meteorological data vector, coordinates of no-fly zones, and altitude layers, and the output is the sequence of waypoints of the global flight path; using the proximal policy optimization algorithm, the input is the local segment of the high-level flight path and real-time attitude data, and the output is the motor speed and rudder surface deflection angle.

[0026] A wind resistance model is introduced into the model predictive control optimization objective function of the dynamic decision-making module: Among them, is the wind resistance coefficient, is the air density, and A is the frontal area of the UAV.

[0027] The calibration method of the wind resistance coefficient includes: S41: Collect the wind resistance data of the UAV at different angles of attack α in a wind tunnel experiment; S42: Obtained through polynomial fitting The relational expression with the angle of attack α: S43: Dynamically calculate the windward area A according to the real-time attitude angle θ in the UAV: Where Is the maximum projected area of the UAV.

[0028] The distributed cooperation module adopts an improved distributed contract network. Each UAV stores the local capability matrix based on the trusted execution environment. The matching degree calculation includes the remaining power, sensor status, and task history. The status is broadcast through the Gossip protocol. The task recipient realizes load balancing through the consistent hashing algorithm to achieve decentralized task negotiation.

[0029] The task allocation objective function of the improved distributed contract network is: Where, = × remaining power + × sensor matching degree + × historical success rate; = task distance ⋅ unit distance energy consumption coefficient; α = 0.7, β = 0.3 are weight coefficients, = 0.4, = 0.4, = 0.4.

[0030] The communication interruption response method of the distributed cooperation module includes adopting long-distance low-power and medium-distance medium-rate dual-mode communication. When UAV A cannot directly connect to the base station, the data packet is temporarily stored locally; when UAV A encounters another UAV B entering the communication range, the data is transmitted to B; UAV B continues to search for the next relay node until the data reaches the target, using neighboring UAVs to relay the transmission of key data, marking the urgency of the data, and giving priority to transmitting high-priority data; delaying or discarding low-priority data.

[0031] The security protection module uses the TLS encryption protocol to ensure data transmission.

[0032] The working method of the monitoring and management platform includes the following steps: S1: The multimodal perception module fuses GPS, vision, and radar data to detect building obstacles and no-fly zones. When self-diagnosing that the GPS signal is blocked by a building, it switches to the vision and radar dominant mode; S2: The dynamic decision-making module generates a global flight path based on the logistics task objective and the real-time wind speed. When a sudden bird group is detected, the rapidly-exploring random tree generates three obstacle avoidance paths, and the energy consumption model predictive control selects the S-shaped bypass trajectory with the lowest energy consumption. S3: The distributed cooperation module distributes the parcel delivery task to the most idle drone through the consistent hashing algorithm. When the communication of drone C is interrupted, the DTN protocol is activated, and the adjacent drone D relays the parcel receipt data.

[0033] The energy consumption model calculates the total power consumption of the bypass path: where is the motor power - airspeed function, which is obtained by fitting the measured data.

[0034] In summary, the present invention dynamically weights the data of each sensor through the attention mechanism to solve the single-point failure problem such as GPS failure; through the dynamic decision-making module, combining reinforcement learning and rolling optimization, it realizes the rapid response from global planning to local obstacle avoidance; and adopts the distributed cooperation module to ensure the continuous execution of tasks under communication interruption.

[0035] Although the embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent monitoring and management platform for UAV flight control architecture, characterized by: The monitoring and management platform includes a multimodal perception module, a dynamic decision-making module, a distributed collaboration module, and a security protection module; The multimodal perception module integrates GPS, radar, and visual data through a spatiotemporal alignment algorithm and an attention mechanism, and has a sensor health self-diagnosis function; The dynamic decision module adopts a hierarchical reinforcement learning architecture, with mission objectives, meteorological data, and airspace restrictions as inputs, outputs the optimal trajectory, embeds fast random trees and model predictive control, and generates obstacle avoidance trajectories in real time. The constraints include: in, For status, To control the amount, is the smoothing weight; The distributed collaboration module adopts an improved distributed contract network. Each drone stores a local capability matrix based on a trusted execution environment. The matching degree calculation includes the remaining power, sensor status, and mission history. The status is broadcast through the Gossip protocol. The task recipient achieves load balancing through a consistent hashing algorithm to realize decentralized task negotiation.

2. The intelligent monitoring and management platform of the UAV flight control architecture according to claim 1, characterized in that: The spatiotemporal alignment algorithm uses spatial coordinate system conversion to apply rotation and translation transformation to the millimeter wave radar point cloud data and map it to the coordinate system of the visual point cloud; the timestamp offset is optimized through a sliding window to align sensor data with a frequency difference of more than 10%.

3. The intelligent monitoring and management platform of the UAV flight control architecture according to claim 1, characterized in that: The sensor health self-diagnosis evaluation method includes calculating the residual of each sensor data and the fusion result, marking it as a fault when the residual of N consecutive frames exceeds a dynamic threshold; triggering a redundant switching strategy, and giving priority to enabling spare sensors of the same type.

4. The intelligent monitoring and management platform of the UAV flight control architecture according to claim 1, characterized in that: The hierarchical reinforcement learning architecture includes a deep deterministic policy gradient algorithm, whose input dimensions are mission target encoding, meteorological data vector, no-fly zone coordinates and altitude layer, and whose output is a waypoint sequence of the global track; a proximal policy optimization algorithm, whose input is the local segment of the high-level track and real-time attitude data, and whose output is the motor speed and the deflection angle of the rudder surface.

5. The intelligent monitoring and management platform of the UAV flight control architecture according to claim 1, characterized in that: The wind resistance model is introduced into the model predictive control optimization objective function of the dynamic decision module: in, is the drag coefficient, is the air density, and A is the windward area of ​​the UAV.

6. The intelligent monitoring and management platform of the UAV flight control architecture according to claim 1, characterized in that: The communication interruption response method of the distributed collaborative module includes adopting long-distance low-power and medium-distance medium-speed dual-mode communications. When UAV A cannot directly connect to the base station, the data packet is temporarily stored locally; when UAV A encounters other UAVs B entering the communication range, the data is transmitted to B; UAV B continues to look for the next relay node until the data reaches the target, and uses neighboring UAVs to relay key data, marking the urgency of the data, and giving priority to high-priority data; low-priority data is delayed or discarded.

7. The intelligent monitoring and management platform of the UAV flight control architecture according to claim 2 is characterized by: The method for optimizing the timestamp offset by sliding window comprises the following steps: S11: construct a sliding window of length L to cache the raw data of each sensor in the window; S12: Perform nonlinear least squares optimization on the timestamp offset Δt between the radar and visual data in the window: ； S13: When the sensor frequency difference exceeds 10%, cubic spline interpolation is used to upsample the low-frequency sensor data.

8. The intelligent monitoring and management platform of the UAV flight control architecture according to claim 1, characterized in that: The working method of the monitoring and management platform comprises the following steps: S1: The multimodal perception module integrates GPS, vision and radar data to detect building obstacles and no-fly zones. If the GPS signal is blocked by a building, the module will switch to vision and radar dominant mode. S2: The dynamic decision module generates a global trajectory based on the logistics mission objectives and real-time wind speed, detects a sudden flock of birds, generates three obstacle avoidance paths using a fast random tree, and uses the energy consumption model to predict and control the S-shaped flight trajectory with the lowest energy consumption; S3: The distributed collaboration module assigns the package delivery task to the least busy drone through the consistent hashing algorithm. When the communication of drone C is interrupted, the DTN protocol is started and the neighboring drone D relays the package receipt data.

9. The intelligent monitoring and management platform of the UAV flight control architecture according to claim 1, characterized in that: The energy consumption model calculates the total power consumption of the detour path: in, It is the motor power consumption-airspeed function, which is obtained by fitting the measured data.

10. The intelligent monitoring and management platform of the UAV flight control architecture according to claim 1, characterized in that: The security protection module uses TLS encryption protocol to ensure data transmission.

Images