Traffic condition monitoring system assisted by low-altitude inspection unmanned aerial vehicle

Abstract

The invention relates to the field of unmanned aerial vehicles, and discloses a traffic condition monitoring system assisted by a low-altitude inspection unmanned aerial vehicle, which adopts intelligent task scheduling based on non-negative tensor decomposition, path optimization of deep reinforcement learning, a task-path double-closed-loop cooperation mechanism and a multi-modal sensor adaptive sensing technology. The system can adjust the inspection task of the unmanned aerial vehicle in real time, improve the response speed of sudden traffic events, optimize the inspection coverage rate, and enhance the perception ability in a low-light environment. Compared with a traditional inspection mode, the task scheduling precision, the path planning flexibility and the system stability are improved, it is ensured that the unmanned aerial vehicle operates efficiently in the complex dynamic traffic environment, and accurate and reliable technical support is provided for intelligent traffic management.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and specifically to a traffic condition monitoring system assisted by low-altitude inspection unmanned aerial vehicles. Background Art

[0002] With the development of intelligent transportation systems, the application of low-altitude unmanned aerial vehicle inspections in traffic condition monitoring has gradually increased. Due to their flexibility and mobility, unmanned aerial vehicles can cover a large-scale road network in a short time, providing real-time data for traffic management departments. However, existing low-altitude inspection unmanned aerial vehicle technologies still have many problems in task scheduling, path optimization, sensing capabilities, and system stability, which affect their practical application effects in complex traffic environments.

[0003] The task scheduling mode is single and difficult to match real-time traffic demands

[0004] Traditional traffic inspections mostly adopt task scheduling methods with fixed routes or preset time sequences. Usually relying on historical data for planning, they cannot dynamically adjust task priorities according to real-time traffic conditions. This results in unmanned aerial vehicles possibly conducting inspections in non-critical areas, with a slow response to emergencies and a reduction in inspection efficiency.

[0005] In addition, existing scheduling strategies are mostly based on single-machine task allocation, lacking overall optimization for unmanned aerial vehicle clusters, which easily leads to some unmanned aerial vehicles being idle for a long time or task overlap, reducing the utilization rate of system resources.

[0006] Static path planning lacks adaptability and is difficult to cope with complex traffic environments

[0007] Currently, unmanned aerial vehicle path planning mainly relies on traditional graph search algorithms such as A* and Dijkstra. These methods perform well in static environments but have limitations in dynamic traffic scenarios. For example, in the face of sudden accidents or road congestion, static paths are difficult to quickly adjust, resulting in a decline in the execution efficiency of inspection tasks.

[0008] In addition, existing path planning methods mainly focus on the shortest path or minimum energy consumption, without fully considering the inspection coverage rate and the priority of sensing targets, which easily causes insufficient monitoring of key areas and affects the quality of data collection.

[0009] Task and path planning are separated, and the system coordination is insufficient

[0010] Existing low-altitude unmanned aerial vehicle inspection systems often treat task management and path planning as two independent modules. The information transmission between them is limited, which easily leads to decision-making fragmentation. For example, the task management module may prioritize the allocation of a certain high-priority area, but the path optimization module selects a path with low energy consumption but low coverage rate, affecting the inspection effect.

[0011] In addition, when the drone mission is adjusted or the path is replanned, the system lacks a global coordination mechanism, which may lead to drone path conflicts, duplicate missions, or blank coverage areas, reducing the overall stability of the system.

[0012] Single or fixed-mode sensors affect the integrity of inspection data

[0013] Most existing drone inspection systems use a single sensor (such as a visible light camera) for traffic monitoring and are difficult to provide stable sensing capabilities under different environmental conditions. For example, in nighttime or low-light environments, the recognition effect of visible light cameras drops significantly, affecting the accuracy of inspection data.

[0014] In addition, although some systems integrate multimodal sensors (such as infrared, LiDAR, etc.), they usually operate in a fixed mode and fail to dynamically adjust the sensor switching strategy according to environmental characteristics, resulting in energy waste or missing key data.

[0015] Therefore, the present invention proposes a traffic condition monitoring system assisted by low-altitude inspection drones to solve the deficiencies of the prior art. Summary of the Invention

[0016] In view of the deficiencies of the prior art, the present invention provides a traffic condition monitoring system assisted by low-altitude inspection drones. By adopting technologies such as an intelligent task scheduling method based on non-negative tensor decomposition, a path optimization strategy based on deep reinforcement learning algorithms, a task-path double-closed-loop coordination mechanism, and the dynamic collaborative operation of multimodal sensors, the present invention can respond to traffic emergencies in real time, minimize the idle time and energy consumption of inspection tasks, improve the coverage rate and data acquisition quality of traffic inspections, and at the same time enhance the robustness and adaptability of the system to ensure that drones can operate continuously and stably in complex environments, improving the efficiency and reliability of intelligent traffic management.

[0017] To achieve the above objectives, the present invention is realized through the following technical solutions: A traffic condition monitoring system assisted by low-altitude inspection drones, comprising:

[0018] A drone swarm, each drone is equipped with multimodal sensors, including a visible light camera, an infrared sensor, and LiDAR, for collecting traffic data and performing data interaction with the edge computing unit through a communication module;

[0019] An edge computing unit, deployed at the drone end, receives the data collected by the multimodal sensors, performs feature extraction, encrypts and uploads it to the data fusion server, and at the same time receives the optimized model parameters returned by the data fusion server;

[0020] A data fusion server communicates with the edge computing unit on the drone side via a wireless network, receives the encrypted feature vectors uploaded by the drone, updates the global model based on the federated learning method, and sends the optimized model parameters back to the drone;

[0021] A task management module interacts with the data fusion server and the drone cluster, analyzes the traffic status based on dynamic spatio-temporal tensor, optimizes the inspection tasks of the drones, and adjusts the task allocation strategy;

[0022] A path optimization module communicates with the task management module, and dynamically adjusts the flight path and sensor switching strategy of the drone based on the flight status, battery level, task requirements and traffic monitoring data of the drone to minimize energy consumption and maximize the monitoring coverage;

[0023] Among them, the task management module extracts the key traffic event features based on the constructed three-dimensional dynamic spatio-temporal tensor model and the non-negative tensor decomposition method, and provides the analysis results to the path optimization module to optimize the drone inspection tasks.

[0024] Preferably, the task management module constructs a three-dimensional tensor based on the dynamic spatio-temporal tensor modeling method where T is the time window, representing the monitoring data in different time periods; S is the spatial grid, dividing the monitoring area into m×n cells; M is the set of types of multimodal sensors; the tensor element represents the data collected by the sensor m i at the time t and the spatial position s j .

[0025] Preferably, the task management module decomposes the three-dimensional tensor with the following optimization objective:

[0026]

[0027] where is the time factor matrix, with a piecewise sparse constraint imposed, and λ1 is the regularization coefficient for time sparsity; is the spatial factor matrix, with a total variation regularization imposed, and λ2 is the regularization coefficient for spatial smoothness; is the sensor factor matrix; R is the rank of the tensor decomposition.

[0028] Preferably, the edge computing unit uses a lightweight neural network f θ to extract the encrypted feature vector zi, with the optimization objective:

[0029]

[0030] where x iSensor data collected by the drone; y i is the traffic event label; θ is the neural network parameter; is the spatio-temporal feature obtained by tensor decomposition; β is the weight coefficient of the regularization term.

[0031] Preferably, the data fusion server performs global model update based on the federated averaging method, and its update formula is as follows:

[0032]

[0033] where n i is the data volume of the i-th drone; n is the total data volume of all drones, satisfying is the local model parameter of the i-th drone after the k-th iteration.

[0034] Preferably, the path optimization module optimizes the drone inspection path through a non-cooperative game model, and the optimization objective of each drone is:

[0035]

[0036] where Q i is the coverage quality, defined as:

[0037]

[0038] v j is the eigenvalue at position s in the spatial factor matrix V j ; w j is the eigenvalue of the corresponding mode in the sensor factor matrix W; E i is the energy consumption, defined as:

[0039]

[0040] ||p i || is the path length; c k is the energy consumption of sensor m k ; α, β are the energy consumption weight coefficients; and the optimal policy is solved through deep reinforcement learning.

[0041] Preferably, the path optimization module adjusts the switching strategy of the sensor based on the modal factor matrix W. When a certain mode w j > γ, the corresponding sensor is activated, where γ is the preset modal weight threshold.

[0042] Preferably, when the task management module detects a drone sensor failure, it encodes this state as a missing value of the tensor and triggers a tensor completion algorithm to reallocate tasks.

[0043] Preferably, the task management module performs global verification based on a closed-loop optimization mechanism and adjusts the sparsity parameter λ1 of tensor decomposition. The adjustment rules are as follows:

[0044]

[0045] where ||U (k) ||0 represents the sparsity of the time factor matrix U (k) ; η is an adjustment coefficient that controls the parameter update rate.

[0046] Preferably, a method for implementing the system includes the following steps:

[0047] Data acquisition: The UAV swarm collects multi-modal sensor data in the traffic monitoring area and constructs a dynamic spatio-temporal tensor;

[0048] Tensor decomposition: Perform non-negative tensor decomposition on the constructed spatio-temporal tensor, extract spatio-temporal features, and optimize task allocation;

[0049] Federated learning: Extract features on the UAV side and update the global model through federated learning;

[0050] Path optimization: Based on the flight state, battery level, and task requirements of the UAV, optimize the flight path and sensor switching strategy;

[0051] Closed-loop optimization: Adjust the tensor decomposition parameters according to the monitoring feedback of the UAV and dynamically optimize the monitoring tasks.

[0052] The present invention provides a traffic condition monitoring system assisted by low-altitude inspection UAVs. It has the following

[0053] beneficial effects:

[0054] 1. The present invention integrates deep reinforcement learning and non-cooperative game models, enabling the UAV to autonomously adjust the inspection path and dynamically adapt to changes in road traffic conditions. It achieves the technical effects of maximizing the inspection coverage and reducing repeated inspections under limited energy consumption. Compared with the traditional inspection method based on static path planning, it overcomes the shortcomings of slow response to sudden traffic accidents and lack of path flexibility, and is suitable for complex road network environments.

[0055] 2. The present invention adopts a dual closed-loop design of task management and path optimization, continuously adjusts parameters through federated learning, and ensures that the UAV can operate efficiently in different traffic scenarios. It achieves the technical effects of improving system stability and reducing the task execution failure rate. Compared with the traditional method of independent task scheduling and path planning, it solves the problems of UAV task conflicts and unreasonable path selection, making the entire traffic inspection system more adaptable in a dynamic environment.

[0056] 3. The intelligent control strategy of the present invention based on the multi-modal sensors of the unmanned aerial vehicle enables the inspection task to adjust the sensor modality according to the traffic environment, realizing the optimization of data fusion. It achieves the technical effects of accurately identifying traffic accidents and dynamically monitoring road congestion. Compared with the single-modal inspection scheme, it overcomes the problems of the decline in the perception ability of visible light cameras in low-light environments and the lack of detailed structures in infrared imaging, enhancing the adaptability of the all-weather low-altitude inspection system.

[0057] 4. The present invention designs a real-time detection and task reallocation mechanism for abnormal states, enabling the unmanned aerial vehicle to quickly adjust the inspection plan when problems such as sensor failures and communication interruptions occur. It achieves the technical effects of reducing task execution interruptions and enhancing system continuity. Compared with the traditional abnormal handling method relying on manual intervention, it solves the problems of lagging abnormal response and decreasing task coverage rate, making the low-altitude inspection system more intelligent and reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 The system architecture diagram of the traffic condition monitoring system assisted by the low-altitude inspection unmanned aerial vehicle;

[0059] Figure 2 The flowchart of the traffic condition monitoring method assisted by the low-altitude inspection unmanned aerial vehicle. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0060] Next, the technical solutions of the present invention will be clearly and completely described in conjunction with the drawings of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0061] Please refer to the attached Figure 1 , the embodiment of the present invention provides a traffic condition monitoring system assisted by a low-altitude inspection unmanned aerial vehicle. The following will elaborate on each module of the system of the present invention in detail.

[0062] Unmanned Aerial Vehicle Cluster

[0063] The unmanned aerial vehicle cluster of the present invention consists of multiple low-altitude inspection unmanned aerial vehicles with autonomous navigation capabilities, and realizes the efficient monitoring of traffic conditions through dynamic task allocation and real-time communication. The unmanned aerial vehicle cluster works in coordination with the edge computing unit, data fusion server, task management module, and path optimization module to form a closed-loop optimized monitoring system. The following will elaborate on the specific implementation manner of the unmanned aerial vehicle cluster in detail in conjunction with the embodiments.

[0064] Hardware Configuration and Data Acquisition of the Unmanned Aerial Vehicle Cluster

[0065] In this embodiment, each drone in the drone swarm is equipped with a multi-modal sensor and an edge computing unit for real-time collection and processing of traffic data.

[0066] In some embodiments, the hardware configuration of the drone is as follows:

[0067] Visible light camera: Adopts SONY IMX586 sensor, supports 4K resolution acquisition, frame rate 30fps, used to capture vehicle density, license plate information (after desensitization processing) and road abnormal events.

[0068] Infrared thermal imager: Selects FLIR Vue Pro module, temperature detection accuracy ±2°C, spatial resolution 640×512, used to identify thermal anomalies such as overheating of vehicles and icing on the road surface.

[0069] LiDAR: Configures Velodyne VLP-16 lidar, scanning frequency 20Hz, detection distance 100m, generates 3D point cloud to detect road deformation caused by obstacles or traffic accidents.

[0070] Edge computing unit: Equipped with NVIDIA Jetson Xavier NX module, built-in 384-core CUDA GPU, used to run lightweight neural network and tensor decomposition preprocessing in real time.

[0071] Specifically, the drone interacts with the ground control center through a 5G NR and LoRa dual-mode communication module. 5G NR is used for high-bandwidth data transmission (such as encrypted feature vectors), and LoRa is used to maintain the heartbeat signal in the low-power state. The communication delay ≤50ms and the packet loss rate <0.1%.

[0072] Dynamic spatio-temporal tensor modeling and feature extraction

[0073] The multi-modal data collected by the drone swarm is fused and analyzed through spatio-temporal tensor modeling.

[0074] In some embodiments, the spatio-temporal tensor construction rules are as follows:

[0075] Time window division: Taking 5 minutes as the time slice t∈T, continuously splicing data to form the time dimension.

[0076] Spatial grid division: Divide the monitoring area into grids s of 10m×10m i ∈S, and dynamically adjust to 5m×5m during peak hours.

[0077] Sensor modality: Define the modality set M = {visible light, infrared, LiDAR}, and the tensor element represents the corresponding data (such as filling the number of vehicles in the visible light modality and filling the point cloud density in the LiDAR modality).

[0078] In a possible implementation, the drone side performs non - negative tensor factorization (NTF) pre - processing, and the optimization objective is:

[0079]

[0080] where, is the time factor matrix, and λ1 = 0.1 is the piece - wise sparse constraint coefficient (sparsity enhancement during peak hours); is the space factor matrix, and λ2 = 0.05 is the total variation regularization coefficient, which enforces spatial continuity; is the sensor factor matrix, R = 10 is the rank of the tensor factorization; ° represents the outer product operation, ||·|| F is the Frobenius norm

[0081] Specifically, the decomposition result is used to generate the drone inspection priority:

[0082] If the standard deviation of the time factor u r in a certain time slice exceeds the threshold σ th = 2.0, it is determined as a sudden congestion event; if the value of the space factor v r in a specific grid v r > 3σ v (σ v is the standard deviation of the space factor), and the infrared modality w 红外 > 0.7, it is determined as an accident area.

[0083] Federated learning and privacy protection mechanism

[0084] The drone swarm realizes data collaborative analysis through federated learning while protecting privacy.

[0085] In some embodiments, the lightweight neural network (MobileNet - Edqe) on the drone side performs feature extraction, and the loss function is defined as:

[0086]

[0087] where, f θ is the neural network model, the input x i is the tensor spliced from multi - modal data; y i is the traffic event label (such as congestion, accident); β = 0.5 is the regularization weight coefficient, is the dynamic constraint term provided by the spatio - temporal tensor decomposition.

[0088] As an option, the encrypted feature vector z i = f θ (x i) Uploaded to the data fusion server through Paillier homomorphic encryption. The server executes federated averaging (FedAvg) to update the global model:

[0089]

[0090] where n i is the data volume of the i-th UAV; is the total data volume; the aggregation frequency is once every 5 minutes, synchronized with the spatio-temporal tensor window.

[0091] Path Optimization and Sensor Dynamic Control

[0092] The UAV swarm realizes cooperative path planning through a game model and reinforcement learning.

[0093] In a possible implementation, the path optimization module defines the action space of the UAV as a i = p i , τ i ), where p i is the flight path, selected from the set of candidate paths generated by the A* algorithm; τ i is the sensor switching strategy (binary vector), driven by the modal factor matrix W.

[0094] Specifically, the optimization objective function is:

[0095]

[0096] where is the spatial factor, w j is the sensor modal weight; α = 0.1, β = 0.01 are the energy consumption weights, c k is the sensor energy consumption; the optimal policy is solved by the Deep Deterministic Policy Gradient (DDPG) algorithm, the Actor network outputs the action probability, and the Critic network evaluates the action value.

[0097] In some embodiments, the sensor switching strategy is dynamically adjusted according to the modal factor threshold γ = 0.7:

[0098] When w 红外 > γ, turn off the visible light camera, and increase the infrared sensor sampling rate to 10Hz;

[0099] When w LiDAR > γ, expand the LiDAR scanning angle to 60°, and increase the point cloud density by 50%.

[0100] Anomaly Handling and Closed-Loop Optimization

[0101] The UAV swarm has the ability of fault self - checking and task re - allocation.

[0102] As an option, if the variance of the data of a certain UAV sensor is lower than the threshold for 5 consecutive frames (such as the variance of LiDAR point cloud < 0.1), it is determined to be faulty, and its status is encoded as a missing value in the spatio - temporal tensor. The task management module triggers the tensor completion algorithm to re - allocate the monitoring tasks

[0103] to other UAVs, and the response time < 2 seconds.

[0104] In a possible implementation, the closed - loop optimization module dynamically adjusts the tensor decomposition parameters:

[0105]

[0106] where, ||U (k) ||0 is the sparsity of the time - factor matrix; η = 0.01 is the adjustment rate coefficient, and the initial value of λ1 = 0.1.

[0107] Edge computing unit

[0108] As the core processing node of the UAV swarm, the edge computing unit is responsible for real - time processing of multi - modal sensor data and interacting with the data fusion server to achieve local feature extraction and privacy protection. Its technical solution is deeply coupled with UAV hardware and the federated learning module to form a distributed intelligent analysis network. The following describes the specific implementation of the edge computing unit in combination with embodiments.

[0109] Hardware Deployment and Data Processing of Edge Computing Unit

[0110] In this embodiment, the edge computing unit is deployed on the UAV side and adopts a heterogeneous computing architecture to achieve high - performance data processing.

[0111] In some embodiments, the hardware configuration is as follows:

[0112] Main control chip: Equipped with the NVIDIA Jetson Xavier NX module, with 384 CUDA cores, a computing power of 21 TOPS, and supporting INT8 quantization inference.

[0113] Memory configuration: LPDDR4 8GB, with a bandwidth of 51.2GB / s, meeting the real - time memory requirements of tensor decomposition and neural network models.

[0114] Storage module: Equipped with an NVMe SSD 256GB for caching raw sensor data and intermediate features.

[0115] Specifically, the edge computing unit is directly connected to the drone sensors through a PCIe4.0 interface, and the data throughput is ≥ 4GB / s. After the visible light, infrared, and LiDAR data are hardware-synchronized, they are input into the preprocessing pipeline.

[0116] Lightweight Neural Network and Feature Extraction

[0117] The edge computing unit runs a lightweight neural network to achieve traffic feature encryption and compression.

[0118] In a possible implementation, the neural network structure is as follows:

[0119] Input layer: Multimodal data is concatenated into a 256×256×3 tensor (RGB, infrared temperature map, LiDAR height map).

[0120] Convolutional layer: 5 layers of depthwise separable convolutions with a kernel size of 3×3, a stride of 2, and a ReLU activation function.

[0121] Output layer: A fully connected layer outputs a 128-dimensional encrypted feature vector z i , including vehicle density, average vehicle speed, and anomaly flag bits.

[0122] In some embodiments, the local training loss function is defined as:

[0123]

[0124] where f θ is the neural network model with 1.2M parameters; x i is the input sensor data; y i is the traffic event label (0 - normal, 1 - congestion, 2 - accident); θ is the model parameter, and the initial value is distributed by the data fusion server; is the dynamic constraint term provided by the spatio-temporal tensor decomposition; β = 0.5 is the regularization weight coefficient, which is determined by cross-validation.

[0125] Federated Learning and Privacy Protection Mechanism

[0126] The edge computing unit performs local training of federated learning to ensure data privacy.

[0127] As an option, the feature vector z i is processed using the Paillier homomorphic encryption algorithm, and the encryption process is as follows:

[0128] Key generation: The drone generates a public-private key pair (pk i , sk i ), the private key sk i is stored locally, and the public key pk i is uploaded to the server.

[0129] Encryption operation: For the elements z of the feature vector i,j is encrypted as where g is the generator, r is a random number, and N is the key length (2048 bits).

[0130] Specifically, the encrypted features are uploaded to the data fusion server via the 5G NR link, and the bandwidth occupancy is reduced to 30% of the unencrypted data.

[0131] Dynamic parameter constraint and model update

[0132] The edge computing unit receives the global model parameters sent by the data fusion server and injects the spatio-temporal tensor decomposition results to optimize local training.

[0133] In some embodiments, the model parameter update rule is as follows:

[0134] Initial synchronization: When the drone starts, it downloads the pre-trained model from the server

[0135] Incremental update: Receive the updated one every 5 minutes and perform weighted fusion with the local parameter θ local Weighted fusion:

[0136]

[0137] Constraint injection: The spatio-temporal factor matrix is directly written into the GPU video memory through the DMA channel to reduce the CPU intervention delay.

[0138] Exception handling and resource scheduling

[0139] The edge computing unit has the ability of real-time resource monitoring and task priority adjustment.

[0140] In a possible implementation, the exception handling mechanism includes:

[0141] Computing overload detection: If the GPU utilization rate continuously exceeds 90% for 10 seconds, automatically switch to the low-precision mode (FP16). Data disconnection compensation: When the sensor signal is lost, enable the historical tensor decomposition results U (t-1) , V (t-1) and interpolate to generate temporary features.

[0142] As an option, the resource scheduling strategy dynamically allocates computing power based on task priority:

[0143] The traffic event recognition task occupies 70% of the computing power:

[0144] The tensor decomposition preprocessing occupies 20% of the computing power;

[0145] Communication and encryption occupy 10% of the computing power.

[0146] Data Fusion Server

[0147] As the global coordination node of the system, the Data Fusion Server is responsible for aggregating the encrypted features of the UAV cluster and updating the federated learning model. At the same time, it interacts with the task management module to optimize the monitoring strategy. Its technical solution is deeply integrated with the edge computing unit and UAV hardware to form a closed-loop optimization network. The following describes the specific implementation of the Data Fusion Server in conjunction with embodiments.

[0148] In this embodiment, the Data Fusion Server receives the encrypted feature vectors uploaded by the UAV cluster and executes the federated averaging algorithm to generate a global model.

[0149] In some embodiments, the model update process is as follows:

[0150] Data reception: Receive the encrypted feature {Enc(z i )} through the 5GNR base station, and the single-node throughput > 1 Gbps.

[0151] Aggregation calculation: Use the additive homomorphic property compatible with Paillier homomorphic encryption to sum the encrypted features:

[0152]

[0153] where n i is the data volume of the i-th UAV; is the total data volume; N is the Paillier key length (2048 bits).

[0154] Parameter decryption: Use the server private key to decrypt z global , and generate the global model parameter θ global .

[0155] Specifically, the aggregation frequency is synchronized with the spatio-temporal tensor window and is executed once every 5 minutes, and the latency is controlled within +2 seconds.

[0156] Model distribution and spatio-temporal feature injection

[0157] The Data Fusion Server distributes the global model parameters to the UAV cluster and injects the spatio-temporal tensor decomposition results to enhance the local training constraints. In one possible implementation, the distribution mechanism includes:

[0158] Differential coding: Only transmit the parameter difference Δθ = θ global -θ local , reducing the bandwidth occupancy by 40%.

[0159] Priority scheduling: Prioritize the distribution to high-task-priority UAVs (such as monitoring congested areas), with a delay ≤ 100 ms.

[0160] In some embodiments, spatio-temporal features are injected into the edge computing unit through an independent channel and transmitted separately from the model parameters to avoid crosstalk.

[0161] Data storage and anomaly tracing

[0162] The data fusion server stores historical model parameters and spatio-temporal tensor snapshots to support backtracking analysis of abnormal events.

[0163] As an option, the storage architecture is as follows:

[0164] Time series database: Use InfluxDB to store the model parameter sequence The timestamp accuracy is 1 ms.

[0165] Tensor snapshot: Save the full spatio-temporal tensor every 15 minutes The compression ratio is 80% (based on the Zstandard algorithm).

[0166] Specifically, the anomaly tracing process includes:

[0167] Retrieve the abnormal time point t error The corresponding model parameters

[0168] Load the spatio-temporal tensor Rerun non-negative tensor decomposition;

[0169] Compare the decomposition result with the original data to locate sensor faults or communication packet loss events.

[0170] Resource scheduling and load balancing

[0171] The data fusion server dynamically allocates computing resources to handle high-concurrency requests.

[0172] In a possible implementation, the resource allocation strategy is as follows:

[0173] Computing node grouping: Divide the server cluster into a real-time group (processing model aggregation) and an offline group (performing tracing analysis).

[0174] Elastic scaling: The number of nodes K in the real-time group is dynamically adjusted according to the request volume Q:

[0175]

[0176] where Q is the number of requests per second, with an upper limit of 500.

[0177] In some embodiments, the load balancing algorithm uses the weighted least connections (WLC), and the weights are calculated from the node GPU utilization and memory margin:

[0178] Wj = 0.6·(1 - GPU j ) + 0.4·(1 - Mem j );

[0179] where GPU j , Mem j are the resource utilization rates (normalized to [0, 1]) of the j-th node.

[0180] Security Protection and Fault Tolerance Mechanism

[0181] The data fusion server integrates multi-layer security protection to ensure the reliability of the system.

[0182] As an option, the protection measures include:

[0183] Traffic cleaning: Perform feature matching filtering on DDoS attack traffic, with a misjudgment rate < 0.1%.

[0184] Model signature: The global model parameters are distributed after being signed by the SM3 hashing algorithm to prevent tampering.

[0185] Dual-machine hot standby: The state synchronization delay between the primary and standby nodes ≤ 10 ms, and the switching time < 1 second.

[0186] Specifically, the fault tolerance mechanism covers the following scenarios:

[0187] Node rock failure: The standby node takes over the task, and the data loss is ≤ 5 items

[0188] Data corruption: The original data is restored through RS erasure code (10 + 4), and the time consumption < 200 ms.

[0189] Task Management Module

[0190] The task management module, as the decision-making center of the system, is responsible for parsing spatio-temporal tensor features and dynamically allocating drone inspection tasks, and at the same time coordinating the interaction between the path optimization module and the data fusion server. Its technical solution deeply integrates the results of tensor decomposition and the feedback of federated learning to achieve closed-loop task scheduling. The following describes the specific implementation method of the task management module in combination with embodiments.

[0191] Spatio-temporal Feature Parsing and Task Priority Calculation

[0192] In this embodiment, the task management module generates a task priority queue based on the non-negative tensor decomposition result.

[0193] In some embodiments, the priority calculation rules are as follows:

[0194] Sudden congestion detection: If the standard deviation σ(u r ) of a certain time slice of the time factor matrix U > 2.0, it is marked as a high-priority task.

[0195] Accident area identification: When the spatial factor v r in the grid s j value exceeds 3σ v , and the infrared mode weight w 红外 > 0.7, trigger the accident response task.

[0196] Specifically, the task priority P j The calculation formula is:

[0197]

[0198] where α = 0.6, β = 0.3, γ = 0.1 are weight coefficients, determined by fitting historical data;

[0199] σ th = 2.0 is the congestion determination threshold; The priority queue is sorted in descending order of P j , and the real-time update period ≤ 1 second.

[0200] Dynamic task allocation and conflict resolution

[0201] The task management module allocates tasks to the UAV cluster and resolves multi-UAV task conflicts.

[0202] In a possible implementation, the allocation strategy includes:

[0203] Proximity principle: Prioritize selecting the UAV closest to the target grid s j (Euclidean distance m);

[0204] Energy consumption balance: If multiple UAVs are close in distance, select the node with a battery remaining of 260% to execute the task.

[0205] As an option, conflict resolution adopts a distributed bidding mechanism:

[0206] The UAV submits a bid price

[0207] The task management module selects the UAV with the highest B i to win the bid, and the response delay ≤ 500ms.

[0208] Exception handling and task reassignment

[0209] The task management module monitors the UAV status in real time and triggers the exception handling process. In some embodiments, the exception determination conditions include:

[0210] Sensor failure: The variance of LiDAR point cloud is continuously < 0.1 for 5 frames, or the infrared temperature data exceeds the range of [-20°C, 150°C];

[0211] Communication interruption: If the heartbeat signal of the UAV is lost for more than 30 seconds, it is determined to be offline.

[0212] Specifically, the task reallocation process is as follows:

[0213] Mark the unfinished tasks of the abnormal UAV as "to be allocated";

[0214] Based on the spatio-temporal tensor decomposition results V and W, recalculate the task priority P′ j ;

[0215] Allocate to other UAVs according to the updated queue to ensure that the loss of task coverage rate ≤ 5%.

[0216] Closed-loop optimization and dynamic parameter adjustment

[0217] The task management module dynamically optimizes the tensor decomposition parameters according to the federated learning feedback. As an option, the sparsity constraint parameter λ1 is adjusted according to the following rules:

[0218]

[0219] Among them, ||U (k) ||0 is the number of non-zero elements of the time factor matrix; R = 10 is the rank of the tensor decomposition; T is the time window length; η = 0.01 is the adjustment rate, and the initial value

[0220] In a possible implementation, the adjustment result is synchronized to the data fusion server in real time to ensure the consistency of the system-wide parameters.

[0221] Multi-modal task collaboration

[0222] The task management module coordinates the task execution strategies of different sensor modalities.

[0223] In some embodiments, the modality collaboration rules include:

[0224] LiDAR and visible light linkage: When LiDAR detects road deformation (point cloud density change rate > 10%), trigger local high-definition shooting (30x zoom) of the visible light camera;

[0225] Infrared and communication collaboration: If the temperature in a certain area is abnormally high for 3 minutes, force the neighboring UAVs to switch to the 5G high-frequency band to increase the data transmission rate to 100 Mbps.

[0226] Path optimization module

[0227] As the core of the navigation decision-making for the UAV swarm, the path optimization module is responsible for generating efficient flight paths and sensor control strategies, while responding to the dynamic scheduling requirements of the task management module. Its technical solution deeply integrates spatio-temporal tensor features and reinforcement learning algorithms to achieve the dual goals of minimizing energy consumption and maximizing monitoring coverage. The following describes the specific implementation of the path optimization module in combination with embodiments.

[0228] Non-cooperative game model construction and state definition

[0229] In this embodiment, the path optimization module models each UAV as an independent agent and constructs a multi-agent game environment.

[0230] In some embodiments, the state space S includes the following elements:

[0231] UAV state: current position (x i , y i ), battery remaining capacity E 剩余 , sensor switch state τ i

[0232] Environmental characteristics: spatio-temporal tensor decomposition result (time-space coupling characteristics), modal factor matrix W;

[0233] Task priority: priority score P issued by the task management module j .

[0234] Specifically, the action space A is defined as:

[0235] Path selection: select a path from the pre-generated set of candidate paths {p1, p2,..., p K} (generated based on the A* algorithm, minimum obstacle avoidance distance 5m);

[0236] Sensor control: binary vector τ i ∈{0, 1} M , where M = 3 (visible light, infrared, LiDAR).

[0237] Deep reinforcement learning algorithm implementation

[0238] The path optimization module uses the Deep Deterministic Policy Gradient (DDPG) algorithm to solve the optimal policy.

[0239] In a possible implementation, the DDPG network structure and training parameters are as follows:

[0240] Actor network: 3-layer fully connected (256-128-64), with the state S as the input and the action probability distribution π(a|s) as the output;

[0241] Critic Network: 3 - layer fully connected (256 - 128 - 1), with the input being the state - action pair (S, A) and the output being the value evaluation Q(s, a);

[0242] Experience Replay Buffer: Capacity 10,000, prioritized sampling ratio 20% (sorted based on TD error);

[0243] Training Parameters: Learning rate α = 10 -4 , discount factor γ = 0.99, exploration noise is an OU process (mean 0, variance 0.1).

[0244] In some embodiments, the reward function is designed as:

[0245]

[0246] where, is the spatial factor, w j is the sensor modality weight; ||p i || is the path length (km), c k is the sensor energy consumption (W); Conflict penalty: If the distance from other drones < 10m, 0.5 points will be deducted each time; η = 0.2, μ = 0.5 are weight coefficients.

[0247] Dynamic Sensor Control Strategy

[0248] The path optimization module adjusts the sensor switch strategy in real - time according to the modality factor matrix W.

[0249] As an option, the control logic is as follows:

[0250] Visible - light camera: Start when w 可见光 > 0.6 and the light intensity is 21000 lux, and the resolution switches to 4K@30fps;

[0251] Infrared sensor: Start when w 红外 > 0.7 or the environmental temperature gradient > 5℃ / m, and the sampling rate is increased to 10Hz;

[0252] LiDAR: Start when w LiDAR > 0.8 or the detected obstacle distance < 50m, and the scanning angle is extended to 60°.

[0253] Specifically, the sensor state τ i is controlled in real - time through GPIO pins, and the response delay ≤ 10ms.

[0254] Abnormal Path Re - planning and Conflict Resolution

[0255] The path optimization module triggers replanning when a sudden obstacle or task change is detected.

[0256] In some embodiments, the replanning process includes:

[0257] Local path correction: Generate an obstacle - avoiding path based on the RRT* algorithm, with a maximum yaw angle of 30°, and the calculation time consumption < 500ms;

[0258] Global task synchronization: Send the updated path p′ i to the task management module, triggering re - sorting of the priority queue;

[0259] Collision detection: If the overlapping area between the new path p′ i and the paths of other UAVs is > 20%, start the distributed negotiation mechanism (based on the contract net protocol).

[0260] Energy consumption optimization and parameter adaptation

[0261] The path optimization module dynamically adjusts the parameters of the game model to adapt to different scenarios.

[0262] In a possible implementation, the energy consumption weight coefficients α and β are updated according to the following rules:

[0263]

[0264] where D max is the maximum allowable flight distance of a single aircraft (default 10km); E 剩余 is the current battery remaining capacity, and E 总 is the full - charge value.

[0265] Please refer to the appendix Figure 2 , the present invention also provides a method for implementing the said system. The following describes the specific implementation manners of each step in combination with the working process of the method of the present invention.

[0266] A traffic condition monitoring method assisted by low - altitude inspection UAVs includes the following steps:

[0267] Data collection: The UAV cluster collects multi - modal sensor data of the traffic monitoring area and constructs a dynamic spatio - temporal tensor;

[0268] Tensor decomposition: Perform non - negative tensor decomposition on the constructed spatio - temporal tensor, extract spatio - temporal features, and optimize task allocation;

[0269] Federated learning: Extract features at the UAV end and update the global model through federated learning;

[0270] Path optimization: Optimize the flight path and sensor switch strategy based on the flight state, battery remaining capacity, and task requirements of the UAV;

[0271] Closed-loop optimization: Adjust the tensor decomposition parameters according to the monitoring feedback of the drone, and dynamically optimize the monitoring task.

[0272] The method of this embodiment can be used to implement the above system embodiment, and its principle and technical effects are similar, so they will not be elaborated here.

[0273] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principle and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A traffic condition monitoring system assisted by a low-altitude inspection drone, characterized in that: include: Drone swarm, where each drone is equipped with multimodal sensors, including visible light cameras, infrared sensors, and LiDAR, to collect traffic data and interact with edge computing units through communication modules; The edge computing unit is deployed on the drone side, receives the data collected by the multimodal sensors, performs feature extraction, and uploads it to the data fusion server after encryption, while receiving the optimization model parameters returned by the data fusion server; The data fusion server communicates with the edge computing unit on the drone side through a wireless network, receives the encrypted feature vectors uploaded by the drone, updates the global model based on the federated learning method, and sends the optimized model parameters back to the drone; The task management module interacts with the data fusion server and the drone cluster, analyzes traffic conditions based on dynamic spatiotemporal tensors, optimizes drone inspection tasks, and adjusts task allocation strategies; The path optimization module communicates with the task management module and dynamically adjusts the flight path and sensor switching strategy of the drone based on the drone's flight status, battery remaining, mission requirements, and traffic monitoring data to minimize energy consumption and maximize monitoring coverage; Among them, the task management module is based on the constructed three-dimensional dynamic space-time tensor model, combined with the non-negative tensor decomposition method to extract key traffic event characteristics, and provide the analysis results to the path optimization module to optimize the drone inspection tasks.

2. A low-altitude inspection drone-assisted traffic condition monitoring system according to claim 1, characterized in that: The task management module constructs a three-dimensional tensor based on the dynamic space-time tensor modeling method. Among them, T is the time window, which represents the monitoring data of different time periods; S is the spatial grid, which divides the monitoring area into m×n units; M is the type set of multimodal sensors; the tensor element Indicates that at time t, spatial position s i At, by sensor m j Collected data.

3. The low-altitude inspection drone-assisted traffic condition monitoring system according to claim 1 is characterized in that: The task management module is based on the non-negative tensor decomposition method to perform three-dimensional tensor Decomposed, the optimization objectives are as follows: in, is the time factor matrix, which imposes piecewise sparse constraints, and λ1 is the regularization coefficient of time sparsity; is the spatial factor matrix, total variation regularization is applied, and λ2 is the regularization coefficient of spatial smoothness; is the sensor factor matrix; R is the rank of tensor decomposition.

4. The low-altitude inspection drone-assisted traffic condition monitoring system according to claim 1 is characterized in that: The edge computing unit uses a lightweight neural network f θ Extract the encrypted feature vector zi, and the optimization goal is: Among them, x i Sensor data collected for the drone; i is the traffic event label; θ is the neural network parameter; U T v i is the spatiotemporal feature obtained by tensor decomposition; β is the weight coefficient of the regularization term.

5. The low-altitude inspection drone-assisted traffic condition monitoring system according to claim 1 is characterized in that: The data fusion server performs global model update based on the federated average method, and the update formula is as follows: Among them, n i is the data volume of the i-th drone; n is the total data volume of all drones, satisfying is the local model parameter of the i-th UAV after the k-th iteration.

6. The low-altitude inspection drone-assisted traffic condition monitoring system according to claim 1, characterized in that: The path optimization module optimizes the inspection path of the drone through a non-cooperative game model. The optimization goal of each drone is: Among them, Q i is the coverage quality, defined as: v j is the position s in the spatial factor matrix V j The characteristic value of w j is the eigenvalue of the corresponding mode in the sensor factor matrix W; E i is the energy consumption, defined as: ∥p i ∥ is the path length; c k is the sensor m k energy consumption; α, β are energy consumption weight coefficients; and the optimal strategy is solved through deep reinforcement learning.

7. The low-altitude inspection drone-assisted traffic condition monitoring system according to claim 1 is characterized in that: The path optimization module adjusts the switch strategy of the sensor based on the modal factor matrix W. j >γ, the corresponding sensor is started, where γ is the preset modal weight threshold.

8. The low-altitude inspection drone-assisted traffic condition monitoring system according to claim 7, characterized in that: When the mission management module detects a drone sensor failure, it encodes the state as a tensor ’s missing values ​​and trigger the tensor completion algorithm to reallocate tasks.

9. The low-altitude inspection drone-assisted traffic condition monitoring system according to claim 1, characterized in that: The task management module performs global verification based on a closed-loop optimization mechanism and adjusts the sparsity parameter λ1 of the tensor decomposition. The adjustment rules are as follows: Among them, ∥U( k) ∥0 represents the time factor matrix U( k) The sparsity of ; η is the adjustment coefficient, which controls the parameter update rate.

10. A method for implementing the system according to any one of claims 1 to 9, characterized in that: The following steps are involved: Data collection: The drone cluster collects multimodal sensor data in the traffic monitoring area and constructs a dynamic spatiotemporal tensor; Tensor decomposition: Perform non-negative tensor decomposition on the constructed spatiotemporal tensor to extract spatiotemporal features and optimize task allocation; Federated learning: Feature extraction is performed on the drone side, and the global model is updated through federated learning; Path optimization: Optimize the flight path and sensor switching strategy based on the UAV’s flight status, battery remaining and mission requirements; Closed-loop optimization: Adjust the tensor decomposition parameters according to the monitoring feedback from the drone to dynamically optimize the monitoring task.

Images