Traffic monitoring method and traffic monitoring system based on unmanned aerial vehicle image recognition
By using a drone swarm network for collaborative data acquisition and multimodal sensor fusion, combined with lightweight image recognition algorithms and a four-level data transmission network, the system solves the problems of insufficient coverage and data timeliness in traditional traffic monitoring systems under complex road conditions and emergency scenarios, and achieves accurate identification of traffic targets and real-time emergency response.
Patent Information
- Application Number
- CN202610295454.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-11
- Publication Date
- 2026-05-08
AI Technical Summary
Traditional traffic monitoring systems suffer from limited coverage, poor adaptability, low data timeliness, and poor multi-device coordination in complex road conditions, special weather, and emergency scenarios, resulting in low efficiency in real-time traffic monitoring and emergency response.
By adopting a drone swarm collaborative networking and dynamic complementation mechanism, combined with multimodal sensor fusion acquisition and lightweight image recognition algorithms, a four-level data transmission network is built to achieve real-time data processing and multi-terminal data synchronization, providing accurate traffic parameter calculation and emergency response.
It enables clear capture of traffic targets in complex environments, eliminates monitoring blind spots, improves the real-time performance of data processing and analysis, and enhances emergency response efficiency and decision support capabilities.
Smart Images

Figure CN121999616A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of traffic monitoring technology, specifically to a traffic monitoring method and system based on UAV image recognition. Background Technology
[0002] With the acceleration of urbanization, the mileage of urban roads and the number of motor vehicles continue to grow, and problems such as traffic congestion, traffic accidents, and illegal driving are becoming increasingly prominent, making the demand for real-time, comprehensive, and accurate traffic monitoring more and more urgent. Traditional traffic monitoring mainly relies on equipment such as fixed cameras, inductive loops, and microwave radar. Although it can achieve some monitoring functions in core road sections, it suffers from limited coverage, poor adaptability, and low data timeliness in complex road conditions (such as suburban roads, temporary construction sections, and cross-river bridges), special weather conditions (heavy rain, heavy fog, nighttime), and emergency scenarios (traffic accident scenes, traffic control areas).
[0003] Drones possess advantages such as flexibility, high-altitude perspective, and rapid deployment. Combined with image recognition technology, they can overcome the spatial limitations of traditional monitoring. At the same time, the maturity of current artificial intelligence algorithms (such as deep learning object detection and semantic segmentation) provides efficient analysis capabilities for traffic images collected by drones. This enables accurate identification of targets such as vehicles, pedestrians, and traffic signs, as well as real-time calculation of traffic parameters (flow rate, speed, and density), providing data support for traffic management, emergency response, and road condition prediction. Therefore, developing a traffic monitoring system based on drone image recognition has significant practical application value.
[0004] Based on the above, traditional traffic monitoring has the following problems:
[0005] 1. Insufficient coverage of fixed monitoring points, with many blind spots.
[0006] Traditional fixed cameras and inductive loop detectors can only cover core urban roads or intersections, leaving monitoring blind spots in suburban roads, urban-rural fringe areas, and mountain highways. For example, in 2024, in a suburban county road in a certain province, due to the lack of fixed monitoring equipment, a collision between a truck and a non-motorized vehicle occurred. Because of the lack of real-time image data, the determination of liability for the accident took more than 72 hours, and it was impossible to trace traffic flow changes on the road before the accident, leading to delays in developing subsequent traffic management plans.
[0007] 2. Image acquisition is greatly affected by environmental interference, resulting in low data quality.
[0008] Current drone monitoring relies heavily on single visible light cameras. In complex environments such as heavy rain, dense fog, and low-light conditions at night, images are prone to blurring, noise, and low contrast, leading to a significant decrease in subsequent recognition accuracy. For example, during a heavy rainstorm in a certain city in 2024 (visibility less than 50 meters), vehicle outlines were blurred in drone-collected images of urban expressways. Traditional recognition algorithms misidentified 20% of small cars as non-motorized vehicles, and traffic flow statistics had an error exceeding 30%, failing to provide traffic management departments with accurate congestion control information.
[0009] 3. Data processing and analysis are lagging and lack real-time performance.
[0010] Existing systems mostly adopt a "drone data collection - cloud transmission - centralized processing" model, which results in long data transmission and analysis processes. This is especially problematic during peak traffic hours or in scenarios involving multiple drones monitoring collaboratively, easily leading to data backlogs. For example, during the morning rush hour (7:30-8:30) in a certain city, five drones simultaneously monitored three main roads in the city center. The collected image data needed to be uploaded to the city-level traffic cloud platform via a 4G network. From data collection to the generation of a traffic congestion warning, it took up to 28 minutes, preventing traffic management departments from timely deploying police forces for traffic control. As a result, the congested sections continued to expand to two surrounding side roads.
[0011] 4. Poor coordination among multiple devices leads to low emergency response efficiency.
[0012] In the existing system, there is a lack of a unified data interaction interface between devices / terminals such as drones, traffic management command centers, traffic police patrol cars, and emergency rescue vehicles. After a drone detects an abnormal event (such as a vehicle breakdown or road collapse), it cannot directly synchronize image data and location information to the associated terminal. For example, in 2024, a rear-end collision occurred on a city expressway. Although the drone detected the accident within 5 minutes, it had to first upload the data to the cloud, and then the cloud forwarded it to the traffic police patrol car terminal. The entire process took 12 minutes, causing a delay in the arrival of traffic police at the scene, and the congestion behind the accident site expanded from 500 meters to 1.2 kilometers. Summary of the Invention
[0013] To solve the above-mentioned technical problems, the present invention provides the following technical solution:
[0014] A traffic monitoring method based on UAV image recognition includes the following specific steps:
[0015] S1: First, combine real-time traffic data to plan the optimal path using the A* algorithm and adjust it dynamically; then, use Mesh self-organizing network technology to form an aerial monitoring network, supporting elastic expansion and automatic replacement to avoid blind spots; at the same time, monitor the drone status in real time, automatically return to home when the battery is low and dispatch a backup drone to ensure stable image acquisition.
[0016] S2: First, multi-source data is acquired using a visible light camera, an infrared thermal imaging sensor, and a millimeter-wave radar; then, images from different environments are preprocessed using dedicated algorithms; finally, invalid areas are removed and data is compressed to reduce the amount of data transmitted.
[0017] S3: First, a lightweight Transformer algorithm is used to identify information; then, image and radar data are fused to accurately calculate parameters; after that, anomalies are identified, timely warnings are issued, historical data is traced, and emergency response departments are coordinated.
[0018] S4: First, build a four-level transmission network; then, display the monitoring data in multiple dimensions on the large screen in the command center to facilitate quick location; finally, provide solutions based on the data.
[0019] As a preferred embodiment of the traffic monitoring method based on UAV image recognition described in this invention, the specific steps of S1 are as follows:
[0020] S11: Based on the monitoring area preset by the traffic management department and combined with real-time traffic data, the optimal flight path of the drone is planned through the A* algorithm; and it supports dynamic path adjustment. When a new abnormal event is detected, a detour or focused monitoring path is automatically generated.
[0021] S12: Employs Mesh self-organizing network technology to enable real-time data interaction between multiple drones, forming an aerial monitoring network; automatically dispatches nearby drones to fill the gap when a drone leaves due to insufficient power or signal interruption, avoiding monitoring blind spots.
[0022] S13: Real-time collection of the drone's battery level, remaining flight time, and sensor status, so as to automatically trigger a return-to-home command when the battery level is below the threshold and dispatch a backup drone to take over the task; at the same time, real-time calibration of the drone's flight attitude to ensure the stability of image acquisition.
[0023] As a preferred embodiment of the traffic monitoring method based on UAV image recognition described in this invention, the specific steps of S2 are as follows:
[0024] S21: Each drone is equipped with a three-modal device, namely a visible light camera, an infrared thermal imaging sensor, and a millimeter-wave radar, to collect multi-source data; and through sensor time synchronization technology, the three-modal data are aligned at the same time and in the same location, providing multi-dimensional data support for subsequent identification.
[0025] S22: Construct an environment-adaptive preprocessing model to automatically match the optimal preprocessing algorithm based on image features without manual intervention;
[0026] S23: Perform region cropping on the preprocessed image, retaining only the region containing traffic targets and removing invalid regions to reduce data transmission volume.
[0027] As a preferred embodiment of the traffic monitoring method based on UAV image recognition described in this invention, the specific steps of S3 are as follows:
[0028] S31: Based on the lightweight Transformer object detection algorithm, object recognition is performed on the preprocessed image; at the same time, few-shot transfer learning is introduced, which requires only a small number of labeled samples to update the model for special scenarios, thereby improving the system's adaptability to new scenarios;
[0029] S32: After fusing vehicle speed data acquired by millimeter-wave radar with image recognition results, calculate traffic parameters such as flow rate, speed, and density;
[0030] S33: When an abnormal event is detected, it automatically triggers warning, tracing, and linkage operations;
[0031] S34: First, based on the vehicle / pedestrian trajectory data output by the multi-target accurate recognition unit, combined with the traffic risk behavior feature library, potential dangerous behaviors are predicted through a temporal convolutional network; then, for special traffic scenarios, the accurate recognition capability is supplemented. For new energy vehicles, the system identifies the illegal occupation of roadside charging station parking spaces by fuel vehicles and the over-time parking of new energy vehicles, and pushes reminders in conjunction with the charging station management system; for the auxiliary verification of cloned vehicles, the license plate recognition result is compared with the suspected cloned vehicle database updated in real time by the traffic management department. If a match is found, it is automatically marked as a high-risk vehicle, and the vehicle's appearance details are retrieved simultaneously for manual secondary verification.
[0032] As a preferred embodiment of the traffic monitoring method based on UAV image recognition described in this invention, the specific steps of S4 are as follows:
[0033] S41: Construct a four-level data transmission network of drone-edge node-cloud-terminal to enable the use of a real-time data priority transmission mechanism, so that critical data occupies bandwidth first to ensure timeliness, while non-critical data is transmitted when bandwidth is idle to avoid network congestion;
[0034] S42: On the large screen of the traffic management command center, based on a map, it realizes the visualization display of real-time monitoring images, traffic parameter heat maps, and abnormal event markers; at the same time, it supports multi-dimensional filtering, which makes it convenient for traffic management personnel to quickly locate areas of interest and improve decision-making efficiency.
[0035] S43: Based on historical traffic data and real-time monitoring results, provide congestion mitigation suggestions, road condition predictions, and emergency plan generation decision support; and link decision suggestions with actual traffic effects, continuously optimize the model through feedback mechanisms, and improve the effectiveness of decision-making.
[0036] A traffic monitoring system based on UAV image recognition includes:
[0037] The UAV swarm dynamic scheduling module first combines real-time traffic data to plan the optimal path using the A* algorithm and dynamically adjust it; then it uses Mesh self-organizing network technology to form an aerial monitoring network, supporting elastic expansion and automatic replacement to avoid blind spots; at the same time, it monitors the status of UAVs in real time, automatically returns to home when the battery is low and dispatches backup UAVs to ensure stable image acquisition.
[0038] The multimodal image acquisition and preprocessing module is used to first acquire multi-source data using a visible light camera, an infrared thermal imaging sensor, and a millimeter-wave radar; then, it uses dedicated algorithms to preprocess images for different environments; finally, it removes invalid areas and compresses the data to reduce the amount of data transmitted.
[0039] The intelligent image recognition and traffic analysis module first uses a lightweight Transformer algorithm to identify information; then it integrates image and radar data to accurately calculate parameters; and finally, it issues timely warnings, traces historical data, and coordinates with emergency departments after identifying anomalies.
[0040] The data interaction and decision support module is used to first build a four-level transmission network; then, the monitoring data is displayed in multiple dimensions on the large screen in the command center for easy and rapid location; and finally, solutions are provided based on the data.
[0041] As a preferred embodiment of the traffic monitoring system based on UAV image recognition described in this invention, the UAV swarm dynamic scheduling module includes:
[0042] The task planning and path optimization unit is used to plan the optimal flight path of the UAV based on the monitoring area preset by the traffic management department and combined with real-time traffic data, using the A* algorithm; it also supports dynamic path adjustment and automatically generates detour or focused monitoring paths when new abnormal events are detected.
[0043] The multi-drone collaborative networking unit is used to achieve real-time data interaction between multiple drones using Mesh self-organizing network technology to form an aerial monitoring network; it can automatically dispatch surrounding drones to fill the gap when a drone leaves due to insufficient power or signal interruption, thus avoiding the generation of monitoring blind spots.
[0044] The energy and status management unit is used to collect the drone's battery level, remaining flight time, and sensor status in real time. When the battery level is below a threshold, it automatically triggers a return-to-home command and dispatches a backup drone to take over the task. At the same time, it performs real-time calibration of the drone's flight attitude to ensure the stability of image acquisition.
[0045] As a preferred embodiment of the traffic monitoring system based on UAV image recognition described in this invention, the multimodal image acquisition and preprocessing module includes:
[0046] The multi-sensor fusion acquisition unit enables each drone to be equipped with a three-modal device consisting of a visible light camera, an infrared thermal imaging sensor, and a millimeter-wave radar for multi-source data acquisition; and through sensor time synchronization technology, it achieves simultaneous and locational alignment of the three-modal data, providing multi-dimensional data support for subsequent identification;
[0047] The image enhancement and denoising unit is used to build an environment-adaptive preprocessing model to automatically match the optimal preprocessing algorithm based on image features without manual intervention.
[0048] The image cropping and compression unit is used to crop regions of the preprocessed image, retaining only the regions containing traffic targets and removing invalid regions to reduce data transmission volume.
[0049] As a preferred embodiment of the traffic monitoring system based on UAV image recognition described in this invention, the intelligent image recognition and traffic analysis module includes:
[0050] The multi-target accurate recognition unit is used to perform target recognition on the preprocessed image based on the lightweight Transformer target detection algorithm. At the same time, few-shot transfer learning is introduced, which requires only a small number of labeled samples to update the model for special scenarios, thereby improving the system's adaptability to new scenarios.
[0051] The real-time traffic parameter calculation unit is used to calculate traffic parameters such as flow rate, speed, and density after fusing vehicle speed data acquired by millimeter-wave radar with image recognition results.
[0052] The abnormal event early warning and tracing unit is used to automatically trigger early warning, tracing, and linkage operations when an abnormal event is detected.
[0053] The dynamic risk behavior prediction and specialized identification unit is used to first predict potential dangerous behaviors based on vehicle / pedestrian trajectory data output by the multi-target accurate identification unit, combined with a traffic risk behavior feature database, through a temporal convolutional network. Then, it supplements the accurate identification capability for special traffic scenarios. For new energy vehicles, it identifies the illegal occupation of roadside charging station parking spaces by fuel vehicles and the over-time parking of new energy vehicles, and pushes reminders in conjunction with the charging station management system. For the auxiliary verification of cloned vehicles, it compares the license plate recognition results with the suspected cloned vehicle database updated in real time by the traffic management department. If a match is found, it is automatically marked as a high-risk vehicle, and the vehicle's appearance details are retrieved for secondary manual verification.
[0054] As a preferred embodiment of the traffic monitoring system based on UAV image recognition described in this invention, the data interaction and decision support module includes:
[0055] The multi-terminal data transmission unit is used to build a four-level data transmission network of UAV-edge node-cloud-terminal. It can adopt a real-time data priority transmission mechanism to prioritize the use of bandwidth for critical data to ensure timeliness, while non-critical data is transmitted when bandwidth is idle to avoid network congestion.
[0056] The traffic situation visualization unit is used to visualize real-time monitoring images, traffic parameter heat maps, and abnormal event markers on the large screen of the traffic management command center, based on a map. At the same time, it supports multi-dimensional filtering, which makes it easy for traffic management personnel to quickly locate areas of interest and improve decision-making efficiency.
[0057] The traffic decision support unit is used to provide congestion mitigation suggestions, road condition predictions, and emergency plan generation decision support based on historical traffic data and real-time monitoring results; it also links the decision suggestions with actual traffic effects and continuously optimizes the model through a feedback mechanism to improve the effectiveness of decision-making.
[0059] Compared with existing technologies:
[0060] 1. By using a drone swarm collaborative networking and dynamic supplementation mechanism, combined with a regional priority resource allocation strategy, it has the advantage of achieving comprehensive monitoring of blind spots covered by traditional fixed equipment, such as suburban roads, urban-rural fringe roads, and mountain highways, thus eliminating blank areas in traffic monitoring.
[0061] 2. By using multimodal sensor fusion acquisition and combined with environment-adaptive image preprocessing algorithms (fog defogging, nighttime brightening, and shake-based blur removal), it has the advantages of clearly capturing traffic targets in complex environments such as heavy rain, dense fog, and nighttime, ensuring the validity and usability of image data.
[0062] 3. By completing some real-time analysis locally through edge computing nodes, and combining a four-level data transmission network of "drone-edge node-cloud-terminal" with a real-time data priority transmission mechanism, it has the advantages of enabling rapid processing, analysis and timely generation of traffic data and early warning information, avoiding delays caused by excessively long data transmission and processing processes.
[0063] 4. By connecting multiple terminals such as drones, traffic management command centers, traffic police patrol cars, and emergency rescue vehicles through a unified data interaction interface, and combined with an automatic linkage and push mechanism for abnormal events, it has the advantages of realizing real-time synchronization of image data and location information between various devices, improving the efficiency and response speed of multi-device collaboration in emergency scenarios. Attached Figure Description
[0064] Figure 1 This is a schematic diagram of the overall framework of the present invention;
[0065] Figure 2 This is a schematic diagram of the framework of the UAV cluster dynamic scheduling module of the present invention;
[0066] Figure 3 This is a schematic diagram of the framework of the multimodal image acquisition and preprocessing module of the present invention;
[0067] Figure 4 This is a schematic diagram of the intelligent image recognition and traffic analysis module framework of the present invention;
[0068] Figure 5 This is a schematic diagram of the data interaction and decision support module framework of the present invention. Detailed Implementation
[0069] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.
[0070] This invention provides a traffic monitoring method based on UAV image recognition. Please refer to [link / reference]. Figures 1-5 The specific steps are as follows:
[0071] S1: First, combine real-time traffic data to plan the optimal path using the A* algorithm and adjust it dynamically; then, use Mesh self-organizing network technology to form an aerial monitoring network, supporting elastic expansion and automatic replacement to avoid blind spots; at the same time, monitor the drone status in real time, automatically return to home when the battery is low and dispatch a backup drone to ensure stable image acquisition.
[0072] The specific steps of S1 are as follows:
[0073] S11: Based on the monitoring areas preset by the traffic management department (such as core road sections, blind spot road sections, and emergency areas), combined with real-time traffic data (such as congestion points and accident points), the optimal flight path of the drone is planned through the A* algorithm; and it supports dynamic path adjustment. When new abnormal events (such as temporary congestion) are detected, detour or focused monitoring paths are automatically generated.
[0074] S12: Employs Mesh self-organizing network technology to enable real-time data interaction (such as location information and image clips) between multiple drones, forming an aerial monitoring network; automatically dispatches surrounding drones to fill the gap when a drone leaves due to insufficient power or signal interruption, avoiding the generation of monitoring blind spots;
[0075] S13: Real-time collection of the drone's battery level, remaining flight time, and sensor status (such as camera clarity and radar signal strength) to automatically trigger a return-to-home command when the battery level falls below a threshold (such as 20%) and dispatch a backup drone to take over the task; at the same time, real-time calibration of the drone's flight attitude (such as altitude and speed) to ensure the stability of image acquisition.
[0076] S2: First, multi-source data is acquired using a visible light camera, an infrared thermal imaging sensor, and a millimeter-wave radar; then, images from different environments are preprocessed using dedicated algorithms; finally, invalid areas are removed and data is compressed to reduce the amount of data transmitted.
[0077] The specific steps of S2 are as follows:
[0078] S21: Each drone is equipped with a three-modal device, namely a visible light camera, an infrared thermal imaging sensor, and a millimeter-wave radar, to collect multi-source data; and through sensor time synchronization technology, the three-modal data are aligned at the same time and in the same location, providing multi-dimensional data support for subsequent identification.
[0079] The visible light camera: captures high-definition traffic images under good conditions such as sunny days and daytime, for the recognition of vehicle colors, license plates, and traffic signs;
[0080] The infrared thermal imaging sensor captures the outlines of vehicles and pedestrians by measuring the temperature differences of objects in low-visibility environments such as nighttime, heavy fog, and heavy rain, avoiding ambient light interference.
[0081] The millimeter-wave radar can penetrate obstacles such as rain, fog, and snow to help obtain parameters such as vehicle speed and distance, thus compensating for errors in dynamic parameter calculations caused by image recognition.
[0082] S22: Construct an environment-adaptive preprocessing model to automatically match the optimal preprocessing algorithm based on image features (such as brightness, contrast, and noise intensity) without manual intervention;
[0083] For foggy images: remove fog effects and improve image contrast based on dark channel prior algorithm;
[0084] For nighttime images: The Retinex algorithm is used to enhance brightness while suppressing noise;
[0085] For motion-blurred images (such as those caused by drone shaking): use a convolutional neural network (CNN) to deblur the image and restore the clear outline of the vehicle;
[0086] S23: Perform region cropping on the preprocessed image, retaining only the region containing traffic targets (vehicles, pedestrians, roads), and removing invalid regions such as sky and trees to reduce data transmission volume; adopt the H.265 video compression standard to compress the data volume to 30%-40% of the original volume while ensuring image quality (compressed resolution ≥1080P), thereby improving transmission speed.
[0087] S3: First, a lightweight Transformer algorithm is used to identify information; then, image and radar data are fused to accurately calculate parameters; after that, anomalies are identified, timely warnings are issued, historical data is traced, and emergency response departments are coordinated.
[0088] The specific steps of S3 are as follows:
[0089] S31: Based on a lightweight Transformer object detection algorithm (such as YOLOv8-Tiny), target recognition is performed on the preprocessed image; at the same time, few-shot transfer learning is introduced, and for special scenarios (such as new electric vehicles, temporary traffic signs), only a small number of labeled samples (50-100 images) are needed to update the model, improving the system's adaptability to new scenarios;
[0090] Vehicle recognition: Distinguishes between cars, trucks, buses, and non-motorized vehicles with an accuracy rate of ≥98%; supports license plate recognition (character recognition accuracy rate ≥95%), and can be linked to vehicle registration information (such as whether it is a commercial vehicle, whether it has a traffic violation record);
[0091] Pedestrian and non-motorized vehicle recognition: Distinguishes between pedestrians, electric vehicles, and bicycles with an accuracy rate of ≥97%, avoiding confusion with small vehicles;
[0092] Traffic incident recognition: Identifies traffic accidents (vehicle collisions, rollovers), illegal driving (driving against traffic, running red lights, occupying emergency lanes), and road anomalies (road collapses, obstacles), with a recognition response time of ≤1 second;
[0093] S32: After fusing vehicle speed data acquired by millimeter-wave radar with image recognition results, calculate traffic parameters such as flow rate, speed, and density;
[0094] Traffic flow: Statistical analysis of the number of vehicles passing through a road segment within a unit of time (e.g., 5 minutes), categorized by vehicle type;
[0095] Speed: The real-time speed of the vehicle is calculated by combining the changes in the vehicle's position in adjacent frames with the drone's flight altitude, with an error of ≤5km / h;
[0096] Density: Based on the road segment length and the current number of vehicles, calculate the vehicle distribution density and determine the congestion level (smooth traffic, light congestion, heavy congestion).
[0097] S33: When an abnormal event is detected, it automatically triggers warning, tracing, and linkage operations;
[0098] Warning: Send warning information containing event type, location, and real-time images to the traffic management command center;
[0099] Tracing: Retrieve historical image data from 3-5 minutes before the incident to provide a basis for determining liability for the accident and securing evidence of violations;
[0100] Linkage: Based on the severity of the event (such as a major accident), automatically push information to emergency rescue departments (such as 120, fire department) and mark the optimal rescue route;
[0101] S34: First, based on the vehicle / pedestrian trajectory data output by the multi-target accurate recognition unit (such as position changes in 5 consecutive frames of images), combined with the traffic risk behavior feature library (including risk features such as rapid vehicle acceleration, continuous lane changes, and pedestrians lingering on the shoulder), a temporal convolutional network (TCN) is used to predict potential dangerous behaviors; for example, when a vehicle is detected continuously driving in the emergency lane of a highway with fluctuating speed, it is predicted that there may be a risk of "temporary stopping to drop off passengers"; when a pedestrian repeatedly crosses traffic outside the zebra crossing at an intersection, the risk of "pedestrian running a red light" is predicted, and a warning is triggered 1-2 seconds in advance; then... To enhance the accuracy of identification capabilities for specific traffic scenarios, for new energy vehicles, it identifies situations where fuel vehicles illegally occupy roadside charging station parking spaces or new energy vehicles are parked for extended periods (such as not leaving the vehicle more than 30 minutes after charging is completed), and sends reminders to the charging station management system. For assisted verification of cloned license plates, it compares the license plate recognition results with the traffic management department's real-time updated database of suspected cloned license plates (including records of the same license plate appearing simultaneously in different areas and abnormal license plate styles). If a match is found, the vehicle is automatically marked as a high-risk vehicle, and the vehicle's exterior details (such as body stickers and wheel styles) are retrieved for secondary manual verification.
[0102] S4: First, establish a four-level transmission network; then, display monitoring data in multiple dimensions on the command center's large screen for easy and rapid location; finally, provide solutions based on the data.
[0103] The specific steps of S4 are as follows:
[0104] S41: Construct a four-level data transmission network of drone-edge node-cloud-terminal to enable the use of a real-time data priority transmission mechanism, so that critical data such as abnormal event warnings and real-time traffic parameters occupy bandwidth first to ensure timeliness, while non-critical data (such as historical images) are transmitted when bandwidth is idle to avoid network congestion;
[0105] Drones → Edge Nodes: Pre-processed image data and recognition results are transmitted to roadside edge computing nodes (such as edge servers mounted on streetlight poles) via 5G private networks to achieve some real-time analysis (such as abnormal event early warning).
[0106] Edge node → Cloud: Upload historical data (such as 24-hour traffic flow) and complete image data to the traffic cloud platform for long-term storage and big data analysis;
[0107] Cloud / Edge Node → Terminal: Traffic data and early warning information are synchronized to the traffic management command center's large screen, traffic police patrol car Pad, and citizen travel APP through API interfaces to achieve data sharing.
[0108] S42: On the large screen of the traffic management command center, based on a map, it realizes the visualization display of real-time monitoring images, traffic parameter heat maps, and abnormal event markers; at the same time, it supports multi-dimensional filtering (such as by time, road segment, and event type), which makes it convenient for traffic management personnel to quickly locate the area of interest and improve decision-making efficiency.
[0109] Real-time monitoring screen: Displays the flight position of each drone and real-time images, with the option to switch between them by clicking;
[0110] Traffic parameter heat map: The congestion situation of road sections is shown by color intensity (red - severe congestion, yellow - mild congestion, green - smooth flow);
[0111] Abnormal event markers: Different icons (such as red triangle - accident, orange circle - wrong turn) mark the location of abnormal events. Click to view details and real-time images;
[0112] S43: Based on historical traffic data and real-time monitoring results, provide congestion mitigation suggestions, road condition predictions, and emergency plan generation decision support; and link decision suggestions with actual traffic effects, continuously optimize the model through a feedback mechanism (if the congestion relief rate after the implementation of a certain mitigation plan is lower than expected, the suggested parameters will be automatically adjusted) to improve the effectiveness of decision-making;
[0113] Congestion mitigation recommendations: For congested road sections, we recommend police deployment locations (e.g., 200 meters upstream of the congestion point) and temporary traffic control measures (e.g., prohibiting left turns).
[0114] Traffic prediction: Based on traffic flow changes over the past hour, the LSTM neural network predicts traffic conditions for the next 30 minutes (e.g., "a certain road segment will enter a period of mild congestion in 10 minutes").
[0115] Emergency plan generation: In the event of a major accident, the system automatically generates the scope of road closures and detour routes, and pushes them to the citizen's travel APP to guide vehicle diversion.
[0116] A traffic monitoring system based on UAV image recognition includes:
[0117] The UAV swarm dynamic scheduling module first combines real-time traffic data to plan the optimal path using the A* algorithm and dynamically adjust it; then it uses Mesh self-organizing network technology to form an aerial monitoring network, supporting elastic expansion and automatic replacement to avoid blind spots; at the same time, it monitors the status of UAVs in real time, automatically returns to home when the battery is low and dispatches backup UAVs to ensure stable image acquisition.
[0118] The multimodal image acquisition and preprocessing module is used to first acquire multi-source data using a visible light camera, an infrared thermal imaging sensor, and a millimeter-wave radar; then, it uses dedicated algorithms to preprocess images for different environments; finally, it removes invalid areas and compresses the data to reduce the amount of data transmitted.
[0119] The intelligent image recognition and traffic analysis module first uses a lightweight Transformer algorithm to identify information; then it integrates image and radar data to accurately calculate parameters; and finally, it issues timely warnings, traces historical data, and coordinates with emergency departments after identifying anomalies.
[0120] The data interaction and decision support module is used to first build a four-level transmission network; then, the monitoring data is displayed in multiple dimensions on the large screen in the command center for easy and rapid location; and finally, solutions are provided based on the data.
[0121] The dynamic scheduling module for the drone swarm includes:
[0122] The task planning and path optimization unit is used to plan the optimal flight path of the UAV based on the monitoring areas preset by the traffic management department (such as core road sections, blind spot road sections, and emergency areas) and real-time traffic data (such as congestion points and accident points). It also supports dynamic path adjustment and automatically generates detour or focused monitoring paths when new abnormal events (such as temporary congestion) are detected.
[0123] The multi-drone collaborative networking unit is used to achieve real-time data interaction (such as location information and image clips) between multiple drones using Mesh self-organizing network technology, forming an aerial monitoring network; it can automatically dispatch surrounding drones to fill the gap when a drone leaves due to insufficient power or signal interruption, thus avoiding the generation of monitoring blind spots.
[0124] The energy and status management unit is used to collect the drone's battery level, remaining flight time, and sensor status (such as camera clarity and radar signal strength) in real time. When the battery level is below a threshold (such as 20%), it automatically triggers a return-to-home command and dispatches a backup drone to take over the task. At the same time, it performs real-time calibration of the drone's flight attitude (such as altitude and speed) to ensure the stability of image acquisition.
[0125] The multimodal image acquisition and preprocessing module includes:
[0126] The multi-sensor fusion acquisition unit enables each drone to be equipped with a three-modal device consisting of a visible light camera, an infrared thermal imaging sensor, and a millimeter-wave radar for multi-source data acquisition; and through sensor time synchronization technology, it achieves simultaneous and locational alignment of the three-modal data, providing multi-dimensional data support for subsequent identification;
[0127] The visible light camera: captures high-definition traffic images under good conditions such as sunny days and daytime, for the recognition of vehicle colors, license plates, and traffic signs;
[0128] The infrared thermal imaging sensor captures the outlines of vehicles and pedestrians by measuring the temperature differences of objects in low-visibility environments such as nighttime, heavy fog, and heavy rain, avoiding ambient light interference.
[0129] The millimeter-wave radar can penetrate obstacles such as rain, fog, and snow to help obtain parameters such as vehicle speed and distance, thus compensating for errors in dynamic parameter calculations caused by image recognition.
[0130] The image enhancement and denoising unit is used to build an environment-adaptive preprocessing model to automatically match the optimal preprocessing algorithm based on image features (such as brightness, contrast, and noise intensity) without manual intervention.
[0131] For foggy images: remove fog effects and improve image contrast based on dark channel prior algorithm;
[0132] For nighttime images: The Retinex algorithm is used to enhance brightness while suppressing noise;
[0133] For motion-blurred images (such as those caused by drone shaking): use a convolutional neural network (CNN) to deblur the image and restore the clear outline of the vehicle;
[0134] The image cropping and compression unit is used to crop regions of the preprocessed image, retaining only the regions containing traffic targets (vehicles, pedestrians, roads) and removing invalid regions such as the sky and trees to reduce the amount of data transmission. It adopts the H.265 video compression standard to compress the data volume to 30%-40% of the original volume while ensuring image quality (compressed resolution ≥1080P) and improving transmission speed.
[0135] The intelligent image recognition and traffic analysis module includes:
[0136] The multi-target accurate recognition unit is used to perform target recognition on preprocessed images based on a lightweight Transformer target detection algorithm (such as YOLOv8-Tiny). At the same time, it introduces few-shot transfer learning, which allows the model to be updated with only a small number of labeled samples (50-100 images) for special scenarios (such as new electric vehicles and temporary traffic signs), thereby improving the system's adaptability to new scenarios.
[0137] Vehicle recognition: Distinguishes between cars, trucks, buses, and non-motorized vehicles with an accuracy rate of ≥98%; supports license plate recognition (character recognition accuracy rate ≥95%), and can be linked to vehicle registration information (such as whether it is a commercial vehicle, whether it has a traffic violation record);
[0138] Pedestrian and non-motorized vehicle recognition: Distinguishes between pedestrians, electric vehicles, and bicycles with an accuracy rate of ≥97%, avoiding confusion with small vehicles;
[0139] Traffic incident recognition: Identifies traffic accidents (vehicle collisions, rollovers), illegal driving (driving against traffic, running red lights, occupying emergency lanes), and road anomalies (road collapses, obstacles), with a recognition response time of ≤1 second;
[0140] The real-time traffic parameter calculation unit is used to calculate traffic parameters such as flow rate, speed, and density after fusing vehicle speed data acquired by millimeter-wave radar with image recognition results.
[0141] Traffic flow: Statistical analysis of the number of vehicles passing through a road segment within a unit of time (e.g., 5 minutes), categorized by vehicle type;
[0142] Speed: The real-time speed of the vehicle is calculated by combining the changes in the vehicle's position in adjacent frames with the drone's flight altitude, with an error of ≤5km / h;
[0143] Density: Based on the road segment length and the current number of vehicles, calculate the vehicle distribution density and determine the congestion level (smooth traffic, light congestion, heavy congestion).
[0144] The abnormal event early warning and tracing unit is used to automatically trigger early warning, tracing, and linkage operations when an abnormal event is detected.
[0145] Warning: Send warning information containing event type, location, and real-time images to the traffic management command center;
[0146] Tracing: Retrieve historical image data from 3-5 minutes before the incident to provide a basis for determining liability for the accident and securing evidence of violations;
[0147] Linkage: Based on the severity of the event (such as a major accident), automatically push information to emergency rescue departments (such as 120, fire department) and mark the optimal rescue route;
[0148] The dynamic risk behavior prediction and specialized identification unit is used to predict potential dangerous behaviors based on vehicle / pedestrian trajectory data (such as position changes in 5 consecutive frames of images) output by the multi-target accurate identification unit, combined with a traffic risk behavior feature library (including risk features such as rapid vehicle acceleration, continuous lane changes, and pedestrians lingering on the shoulder). For example, when a vehicle is detected continuously driving in the emergency lane of a highway with fluctuating speeds, it is predicted that there may be a risk of "temporary stopping to drop off passengers"; when a pedestrian repeatedly crosses traffic outside the zebra crossing at an intersection, the risk of "pedestrian running a red light" is predicted, and predictions are made 1-2 seconds in advance. The system triggers an alert instantly. Then, it adds precise recognition capabilities for specific traffic scenarios. For new energy vehicles, it identifies instances of fuel vehicles illegally occupying roadside charging station spaces or new energy vehicles exceeding the permitted parking time (e.g., not leaving the station more than 30 minutes after charging is complete), and sends reminders to the charging station management system. For vehicles using counterfeit license plates, it compares the license plate recognition results with the traffic management department's real-time updated database of suspected counterfeit vehicles (including records of the same license plate appearing simultaneously in different areas or abnormal license plate styles). If a match is found, the vehicle is automatically marked as a high-risk vehicle, and vehicle exterior details (such as body stickers and wheel styles) are retrieved for secondary manual verification.
[0149] The data interaction and decision support module includes:
[0150] The multi-terminal data transmission unit is used to build a four-level data transmission network of UAV-edge node-cloud-terminal. It can adopt a real-time data priority transmission mechanism to prioritize the use of bandwidth for critical data such as abnormal event warnings and real-time traffic parameters to ensure timeliness. Non-critical data (such as historical images) are transmitted when the bandwidth is idle to avoid network congestion.
[0151] Drones → Edge Nodes: Pre-processed image data and recognition results are transmitted to roadside edge computing nodes (such as edge servers mounted on streetlight poles) via 5G private networks to achieve some real-time analysis (such as abnormal event early warning).
[0152] Edge node → Cloud: Upload historical data (such as 24-hour traffic flow) and complete image data to the traffic cloud platform for long-term storage and big data analysis;
[0153] Cloud / Edge Node → Terminal: Traffic data and early warning information are synchronized to the traffic management command center's large screen, traffic police patrol car Pad, and citizen travel APP through API interfaces to achieve data sharing.
[0154] The traffic situation visualization unit is used to visualize real-time monitoring images, traffic parameter heat maps, and abnormal event markers on the large screen of the traffic management command center, based on a map. At the same time, it supports multi-dimensional filtering (such as by time, road segment, and event type), which makes it easier for traffic management personnel to quickly locate areas of interest and improve decision-making efficiency.
[0155] Real-time monitoring screen: Displays the flight position of each drone and real-time images, with the option to switch between them by clicking;
[0156] Traffic parameter heat map: The congestion situation of road sections is shown by color intensity (red - severe congestion, yellow - mild congestion, green - smooth flow);
[0157] Abnormal event markers: Different icons (such as red triangle - accident, orange circle - wrong turn) mark the location of abnormal events. Click to view details and real-time images;
[0158] The traffic decision support unit is used to provide congestion mitigation suggestions, road condition predictions, and emergency plan generation decision support based on historical traffic data and real-time monitoring results; and to link the decision suggestions with actual traffic effects, continuously optimize the model through a feedback mechanism (if the congestion relief rate after the implementation of a certain mitigation plan is lower than expected, the suggested parameters will be automatically adjusted) to improve the effectiveness of decision-making.
[0159] Congestion mitigation recommendations: For congested road sections, we recommend police deployment locations (e.g., 200 meters upstream of the congestion point) and temporary traffic control measures (e.g., prohibiting left turns).
[0160] Traffic prediction: Based on traffic flow changes over the past hour, the LSTM neural network predicts traffic conditions for the next 30 minutes (e.g., "a certain road segment will enter a period of mild congestion in 10 minutes").
[0161] Emergency plan generation: In the event of a major accident, the system automatically generates the scope of road closures and detour routes, and pushes them to the citizen's travel APP to guide vehicle diversion.
[0162] Although the present invention has been described above with reference to embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of the invention. In particular, as long as there is no structural conflict, the features in the disclosed embodiments can be combined with each other in any manner. The lack of an exhaustive description of these combinations in this specification is merely for the sake of brevity and resource conservation. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.
Claims
1. A traffic monitoring method based on UAV image recognition, characterized in that, The specific steps are as follows: S1: First, combine real-time traffic data to plan the optimal path using the A* algorithm and adjust it dynamically; then, use Mesh self-organizing network technology to form an aerial monitoring network, supporting elastic expansion and automatic replacement to avoid blind spots; at the same time, monitor the drone status in real time, automatically return to home when the battery is low and dispatch a backup drone to ensure stable image acquisition. S2: First, multi-source data is acquired using a visible light camera, an infrared thermal imaging sensor, and a millimeter-wave radar; then, images from different environments are preprocessed using dedicated algorithms; finally, invalid areas are removed and data is compressed to reduce the amount of data transmitted. S3: First, a lightweight Transformer algorithm is used to identify information; then, image and radar data are fused to accurately calculate parameters; after that, anomalies are identified, timely warnings are issued, historical data is traced, and emergency response departments are coordinated. S4: First, build a four-level transmission network; then, display the monitoring data in multiple dimensions on the large screen in the command center to facilitate quick location; finally, provide solutions based on the data.
2. The traffic monitoring method based on UAV image recognition according to claim 1, characterized in that, The specific steps of S1 are as follows: S11: Based on the monitoring area preset by the traffic management department and combined with real-time traffic data, the optimal flight path of the drone is planned through the A* algorithm; and it supports dynamic path adjustment. When a new abnormal event is detected, a detour or focused monitoring path is automatically generated. S12: Employs Mesh self-organizing network technology to enable real-time data interaction between multiple drones, forming an aerial monitoring network; automatically dispatches nearby drones to fill the gap when a drone leaves due to insufficient power or signal interruption, avoiding monitoring blind spots. S13: Real-time collection of the drone's battery level, remaining flight time, and sensor status, so as to automatically trigger a return-to-home command when the battery level is below the threshold and dispatch a backup drone to take over the task; at the same time, real-time calibration of the drone's flight attitude to ensure the stability of image acquisition.
3. The traffic monitoring method based on UAV image recognition according to claim 1, characterized in that, The specific steps of S2 are as follows: S21: Each drone is equipped with a three-modal device, namely a visible light camera, an infrared thermal imaging sensor, and a millimeter-wave radar, to collect multi-source data; and through sensor time synchronization technology, the three-modal data are aligned at the same time and in the same location, providing multi-dimensional data support for subsequent identification. S22: Construct an environment-adaptive preprocessing model to automatically match the optimal preprocessing algorithm based on image features without manual intervention; S23: Perform region cropping on the preprocessed image, retaining only the region containing traffic targets and removing invalid regions to reduce data transmission volume.
4. The traffic monitoring method based on UAV image recognition according to claim 1, characterized in that, The specific steps of S3 are as follows: S31: Based on the lightweight Transformer object detection algorithm, object recognition is performed on the preprocessed image; at the same time, few-shot transfer learning is introduced, which requires only a small number of labeled samples to update the model for special scenarios, thereby improving the system's adaptability to new scenarios; S32: After fusing vehicle speed data acquired by millimeter-wave radar with image recognition results, calculate traffic parameters such as flow rate, speed, and density; S33: When an abnormal event is detected, it automatically triggers warning, tracing, and linkage operations; S34: First, based on the vehicle / pedestrian trajectory data output by the multi-target accurate recognition unit, combined with the traffic risk behavior feature library, potential dangerous behaviors are predicted through a temporal convolutional network; then, for special traffic scenarios, the accurate recognition capability is supplemented. For new energy vehicles, the system identifies the illegal occupation of roadside charging station parking spaces by fuel vehicles and the over-time parking of new energy vehicles, and pushes reminders in conjunction with the charging station management system; for the auxiliary verification of cloned vehicles, the license plate recognition result is compared with the suspected cloned vehicle database updated in real time by the traffic management department. If a match is found, it is automatically marked as a high-risk vehicle, and the vehicle's appearance details are retrieved simultaneously for manual secondary verification.
5. The traffic monitoring method based on UAV image recognition according to claim 1, characterized in that, The specific steps of S4 are as follows: S41: Construct a four-level data transmission network of drone-edge node-cloud-terminal to enable the use of a real-time data priority transmission mechanism, so that critical data occupies bandwidth first to ensure timeliness, while non-critical data is transmitted when bandwidth is idle to avoid network congestion; S42: On the large screen of the traffic management command center, based on a map, it realizes the visualization display of real-time monitoring images, traffic parameter heat maps, and abnormal event markers; at the same time, it supports multi-dimensional filtering, which makes it convenient for traffic management personnel to quickly locate areas of interest and improve decision-making efficiency. S43: Based on historical traffic data and real-time monitoring results, provide congestion mitigation suggestions, road condition predictions, and emergency plan generation decision support; and link decision suggestions with actual traffic effects, continuously optimize the model through feedback mechanisms, and improve the effectiveness of decision-making.
6. A traffic monitoring system based on UAV image recognition, characterized in that, include: The UAV swarm dynamic scheduling module first combines real-time traffic data to plan the optimal path using the A* algorithm and dynamically adjust it; then it uses Mesh self-organizing network technology to form an aerial monitoring network, supporting elastic expansion and automatic replacement to avoid blind spots; at the same time, it monitors the status of UAVs in real time, automatically returns to home when the battery is low and dispatches backup UAVs to ensure stable image acquisition. The multimodal image acquisition and preprocessing module is used to first acquire multi-source data using a visible light camera, an infrared thermal imaging sensor, and a millimeter-wave radar; then, it uses dedicated algorithms to preprocess images for different environments; finally, it removes invalid areas and compresses the data to reduce the amount of data transmitted. The intelligent image recognition and traffic analysis module first uses a lightweight Transformer algorithm to identify information; then it integrates image and radar data to accurately calculate parameters; and finally, it issues timely warnings, traces historical data, and coordinates with emergency departments after identifying anomalies. The data interaction and decision support module is used to first build a four-level transmission network; then, the monitoring data is displayed in multiple dimensions on the large screen in the command center for easy and rapid location; and finally, solutions are provided based on the data.
7. A traffic monitoring system based on UAV image recognition according to claim 6, characterized in that, The dynamic scheduling module for the drone swarm includes: The task planning and path optimization unit is used to plan the optimal flight path of the UAV based on the monitoring area preset by the traffic management department and combined with real-time traffic data, using the A* algorithm; it also supports dynamic path adjustment and automatically generates detour or focused monitoring paths when new abnormal events are detected. The multi-drone collaborative networking unit is used to achieve real-time data interaction between multiple drones using Mesh self-organizing network technology to form an aerial monitoring network; it can automatically dispatch surrounding drones to fill the gap when a drone leaves due to insufficient power or signal interruption, thus avoiding the generation of monitoring blind spots. The energy and status management unit is used to collect the drone's battery level, remaining flight time, and sensor status in real time. When the battery level is below a threshold, it automatically triggers a return-to-home command and dispatches a backup drone to take over the task. At the same time, it performs real-time calibration of the drone's flight attitude to ensure the stability of image acquisition.
8. A traffic monitoring system based on UAV image recognition according to claim 6, characterized in that, The multimodal image acquisition and preprocessing module includes: The multi-sensor fusion acquisition unit enables each drone to be equipped with a three-modal device consisting of a visible light camera, an infrared thermal imaging sensor, and a millimeter-wave radar for multi-source data acquisition; and through sensor time synchronization technology, it achieves simultaneous and locational alignment of the three-modal data, providing multi-dimensional data support for subsequent identification; The image enhancement and denoising unit is used to build an environment-adaptive preprocessing model to automatically match the optimal preprocessing algorithm based on image features without manual intervention. The image cropping and compression unit is used to crop regions of the preprocessed image, retaining only the regions containing traffic targets and removing invalid regions to reduce data transmission volume.
9. A traffic monitoring system based on UAV image recognition according to claim 6, characterized in that, The intelligent image recognition and traffic analysis module includes: The multi-target accurate recognition unit is used to perform target recognition on the preprocessed image based on the lightweight Transformer target detection algorithm. At the same time, few-shot transfer learning is introduced, which requires only a small number of labeled samples to update the model for special scenarios, thereby improving the system's adaptability to new scenarios. The real-time traffic parameter calculation unit is used to calculate traffic parameters such as flow rate, speed, and density after fusing vehicle speed data acquired by millimeter-wave radar with image recognition results. The abnormal event early warning and tracing unit is used to automatically trigger early warning, tracing, and linkage operations when an abnormal event is detected. The dynamic risk behavior prediction and specialized identification unit is used to first predict potential dangerous behaviors based on vehicle / pedestrian trajectory data output by the multi-target accurate identification unit, combined with a traffic risk behavior feature database, through a temporal convolutional network. Then, it supplements the accurate identification capability for special traffic scenarios. For new energy vehicles, it identifies the illegal occupation of roadside charging station parking spaces by fuel vehicles and the over-time parking of new energy vehicles, and pushes reminders in conjunction with the charging station management system. For the auxiliary verification of cloned vehicles, it compares the license plate recognition results with the suspected cloned vehicle database updated in real time by the traffic management department. If a match is found, it is automatically marked as a high-risk vehicle, and the vehicle's appearance details are retrieved for secondary manual verification.
10. A traffic monitoring system based on UAV image recognition according to claim 6, characterized in that, The data interaction and decision support module includes: The multi-terminal data transmission unit is used to build a four-level data transmission network of UAV-edge node-cloud-terminal. It can adopt a real-time data priority transmission mechanism to prioritize the use of bandwidth for critical data to ensure timeliness, while non-critical data is transmitted when bandwidth is idle to avoid network congestion. The traffic situation visualization unit is used to visualize real-time monitoring images, traffic parameter heat maps, and abnormal event markers on the large screen of the traffic management command center, based on a map. At the same time, it supports multi-dimensional filtering, which makes it easy for traffic management personnel to quickly locate areas of interest and improve decision-making efficiency. The traffic decision support unit is used to provide congestion mitigation suggestions, road condition predictions, and emergency plan generation decision support based on historical traffic data and real-time monitoring results; it also links the decision suggestions with actual traffic effects and continuously optimizes the model through a feedback mechanism to improve the effectiveness of decision-making.
