Road traffic all-region monitoring method and system based on air-ground integration

Abstract

The application discloses a kind of based on air-ground integration's road traffic global monitoring method and system, it is related to traffic monitoring and intelligent equipment technical field, the method uses unmanned law enforcement vehicle, according to the route of tight coupling positioning fusion algorithm planning travels on road, real-time collection road traffic information;Unmanned aerial vehicle is used, through space-time cube segmentation algorithm, according to the route of setting is patrolled in the air and is monitored, collects the road information under the aerial view, and unmanned law enforcement vehicle constitutes air-ground integration's stereoscopic perception network;Receive the real-time data collected by unmanned law enforcement vehicle and unmanned aerial vehicle, introduce artificial intelligence algorithm to quickly identify traffic police situation, and according to the identification result sends instruction to unmanned law enforcement vehicle and unmanned aerial vehicle, monitors the police situation area key.Solve the problem that there is monitoring blind area in road traffic static monitoring method, improve traffic data acquisition and analysis ability, improve police situation discovery and disposal ability.

Description

Technical Field

[0001] This application relates to the field of traffic monitoring and intelligent equipment technology, and in particular to a method and system for comprehensive road traffic monitoring based on air-ground integration. Background Technology

[0002] Road traffic monitoring refers to the real-time or periodic observation, measurement and evaluation of road traffic conditions using various technical means, including sensors, cameras and radar. It covers the collection and analysis of information on traffic flow, speed, traffic incidents, road infrastructure status and meteorological environment, providing data support and decision-making basis for traffic management, road maintenance, travel services and traffic planning.

[0003] Existing road traffic monitoring methods generally employ static traffic monitoring equipment installed at fixed locations, such as red-light violation cameras, speed radar, and surveillance cameras. These static monitoring devices are fixed in specific locations on the road, monitoring only a limited area around the installation point, thus severely restricting the monitoring range. For road sections without static monitoring equipment, remote areas, and sections with complex road conditions, blind spots are created, failing to achieve seamless coverage of the entire road. This results in traffic violations and emergencies not being quickly detected and handled, seriously affecting the efficiency and accuracy of road traffic management and posing numerous hidden dangers to road traffic safety. Therefore, there is an urgent need for a technical solution that can overcome the limitations of fixed monitoring equipment and achieve comprehensive, blind-spot-free, and efficient monitoring of road traffic, thereby improving traffic data collection and analysis capabilities and enhancing the ability to detect and respond to emergencies. Summary of the Invention

[0004] To address the technical problems of the prior art, this application provides a road traffic full-area monitoring method and system based on air-ground integration, which overcomes the shortcomings of existing fixed static traffic monitoring equipment, such as limited monitoring range, blind spots, and untimely detection of incidents. This achieves full-area coverage monitoring of road traffic, improves the comprehensiveness and accuracy of traffic data collection and analysis, and enhances the ability to detect and handle incidents.

[0005] This application provides a method for comprehensive road traffic monitoring based on air-ground integration, including:

[0006] (1) An unmanned law enforcement vehicle is used to drive on the road according to the route planned by the tightly coupled positioning fusion algorithm, and collect information on the vehicle driving status, road environment and traffic violations in real time;

[0007] (2) Using unmanned aerial vehicles, the spatiotemporal cube segmentation algorithm is used to conduct aerial patrol and monitoring according to the set route, and collect traffic flow, vehicle distribution and road anomaly information on the road from the aerial perspective, forming an integrated air-ground three-dimensional perception network with unmanned law enforcement vehicles.

[0008] (3) Receive real-time data collected by unmanned law enforcement vehicles and unmanned aerial vehicles, introduce artificial intelligence recognition algorithms to analyze the data, quickly identify traffic incidents, and send instructions to unmanned law enforcement vehicles and unmanned aerial vehicles based on the recognition results to focus on monitoring and managing the incident area.

[0009] Furthermore, the tightly coupled positioning fusion algorithm integrates positioning data collected by three positioning technologies: Global Navigation Satellite System (GNSS), Ultra-Wideband Positioning (UWB), and Visual Inertial Odometry (VIO), to provide accurate path planning data and location information for the autonomous driving process of unmanned law enforcement vehicles.

[0010] A Global Navigation Satellite System (GNSS) consists of three parts: a constellation of space satellites, ground control stations, and user receiving equipment. Ground control stations are responsible for monitoring, calculating, and transmitting navigation information to the satellites; user receiving equipment is responsible for receiving satellite signals and calculating its own position. A GNSS receiver receives radio signals from multiple navigation satellites via an antenna, including timestamp information and satellite position information. The receiver analyzes and processes the signals, extracting pseudorange information and calculating the distance between the receiver and the satellites by measuring the signal propagation time. Then, using least squares methods, Kalman filtering, and other techniques, combined with the known position information of the satellites, the receiver's three-dimensional position (longitude, latitude, and altitude) is calculated.

[0011] Ultra-wideband (UWB) positioning achieves location by transmitting narrow, non-sinusoidal pulse signals on the order of nanoseconds or shorter. Utilizing the high time resolution of UWB pulse signals, it accurately measures the propagation time of the signal between the transmitter and receiver, then calculates the distance between them based on the speed of light. The ranging method employs time difference of arrival (TDOA) calculation, determining the tag's location by measuring the time difference between the signal's arrival at different base stations, without requiring strict time synchronization between the tag and the base station.

[0012] The visual inertial odometry system includes an onboard camera and an IMU sensor. The camera acquires image sequences during the vehicle's movement and calculates the vehicle's motion changes between consecutive image frames through image matching and feature point detection algorithms, i.e., visual odometry. The IMU sensor measures the vehicle's acceleration and angular velocity in real time. By integrating this information, the vehicle's pose changes are obtained. By fusing the visual odometry and IMU data, an accurate and stable vehicle positioning result is obtained.

[0013] This application also provides a road traffic all-area monitoring system based on air-ground integration, including:

[0014] Unmanned vehicle monitoring module: It adopts an unmanned law enforcement vehicle, which travels on the road according to the route planned by the tightly coupled positioning fusion algorithm, and collects information on vehicle driving status, road environment and traffic violations in real time;

[0015] Unmanned aerial vehicle (UAV) monitoring module: Using unmanned aerial vehicles and spatiotemporal cube segmentation algorithm, it conducts aerial patrol and monitoring according to a set route, collecting traffic flow, vehicle distribution and road anomaly information from an aerial perspective, forming an integrated air-ground three-dimensional perception network with unmanned law enforcement vehicles;

[0016] Air-Ground Integrated Data Processing Module: This module receives real-time data collected by unmanned law enforcement vehicles and unmanned aerial vehicles, incorporates artificial intelligence recognition algorithms to analyze the data, quickly identifies traffic incidents, and sends instructions to unmanned law enforcement vehicles and unmanned aerial vehicles based on the identification results for key monitoring and management of the incident area.

[0017] The present invention discloses the following technical effects:

[0018] This invention proposes a method and system for comprehensive road traffic monitoring based on integrated air-ground technology. By combining seamless mobile monitoring by unmanned vehicles (UAVs) with aerial patrol monitoring by drones, it breaks through the range limitations of fixed static monitoring equipment, enabling comprehensive monitoring of road traffic and eliminating blind spots. Furthermore, the integrated air-ground data processing module in the system performs real-time comparative analysis of data collected by UAVs and drones, making the collected data more comprehensive and accurate, and the analysis results more valuable, thus improving the efficiency of traffic data utilization. Due to the expanded monitoring range and real-time data processing and analysis, various traffic incidents can be detected more quickly and accurately, providing strong support for traffic management departments to handle incidents promptly, helping to improve road traffic management and road traffic efficiency, and ensuring road traffic safety. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings of the embodiments of the present invention will be briefly described below. Flowcharts are used in this application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed precisely in sequence. Instead, various steps can be processed in reverse order or simultaneously as needed. Furthermore, other operations can be added to these processes, or one or more steps can be removed from these processes.

[0020] Figure 1 This is a flowchart illustrating the air-ground integrated road traffic monitoring method provided in an embodiment of this application.

[0021] Figure 2A schematic diagram of the structure of the air-ground integrated road traffic monitoring system provided in this application embodiment. Detailed Implementation

[0022] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application.

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description of this application will be provided in conjunction with the accompanying drawings. The described embodiments should not be considered as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] In the following description, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those explicitly listed, but may include other steps or modules not explicitly listed or inherent to such processes, methods, products, or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only.

[0025] Example 1: This application provides a method for comprehensive road traffic monitoring based on integrated air-ground systems, such as... Figure 1 As shown, the method includes:

[0026] Step S10: An unmanned law enforcement vehicle is used to travel on the road according to the route planned by the tightly coupled positioning fusion algorithm, and collect information on vehicle driving status, road environment and traffic violations in real time.

[0027] In this embodiment, the unmanned law enforcement vehicle is equipped with a high-definition camera, millimeter-wave radar, lidar, GPS positioning module, and data transmission module.

[0028] The GPS positioning module receives positioning data collected by millimeter-wave radar and lidar, while the data transmission module is used to transmit image data collected by a high-definition camera and positioning data obtained by the GPS positioning module.

[0029] During operation, high-definition cameras are used to capture images of vehicle license plates and traffic violations; millimeter-wave radar and lidar are used to detect vehicle distance, acceleration, and speed; and the GPS positioning module integrates positioning data collected by three positioning sensors—Global Navigation Satellite System (GNSS), Ultra-Wideband (UWB), and Visual Inertial Odometry (VIO)—to plan the driving path of the unmanned law enforcement vehicle and record precise monitoring locations.

[0030] Based on the tightly coupled localization fusion algorithm, the planning of the driving route for the unmanned law enforcement vehicle includes the following steps:

[0031] The system collects positioning data from different positioning sources during the operation of the unmanned law enforcement vehicle: GNSS provides global positioning information, UMB provides high-precision relative positioning information within a short distance, and VIO estimates pose information based on the unmanned law enforcement vehicle's own motion.

[0032] The collected positioning data is preprocessed to remove obvious errors and interference:

[0033] The positioning data from GNSS, UWB and VIO sensors are timestamped to unify their timestamps into a common time coordinate system. The second pulse signal in the GNSS signal is used as the synchronization reference, and the timestamps of the UWB and VIO data are aligned with it.

[0034] Transform different positioning data into the same coordinate system. GNSS provides location information in a global geographic coordinate system, while UWB and VIO use local coordinate systems; transform all positioning data into a unified local horizontal coordinate system.

[0035] The collected positioning data is sorted according to time series, and a consecutive odd number of data points are selected. The data point in the middle position is used as the output value to replace the original data point, which effectively removes spike noise and impulse interference in the data. When the positioning data is missing, the missing data is filled by interpolation estimation based on the adjacent data points before and after the missing data.

[0036] By employing an adaptive robust Kalman filter algorithm and fusing preprocessed GNSS, UWB, and VIO positioning data, accurate location information of the unmanned law enforcement vehicle can be obtained.

[0037] A state vector containing the position, velocity, and attitude angles of the unmanned law enforcement vehicle is determined. The state vector is initialized using GNSS, UWB, and VIO sensor positioning data at the initial moment, and the covariance matrix is ​​initialized based on the uncertainty of the initial estimate. Initial state vector Represented as:

[0038]

[0039] in, This indicates the transpose operation. , and These represent the quaternions for position vector, velocity vector, and attitude angle, respectively.

[0040] Based on the system's dynamic characteristics, a state equation is established to describe the evolution of the state vector over time. The state equation is expressed by the formula:

[0041]

[0042] in, and These represent the state vectors before and after the update, respectively. Represents the state transition matrix. It is a control input matrix. Represents the process noise vector;

[0043] For positioning data from GNSS, UWB, and VIO sensors, measurement equations are established to express the relationship between the data from each sensor and the state vector. The measurement equations are expressed by the following formulas:

[0044]

[0045]

[0046]

[0047] in, , and These represent the GNSS, UWB, and VIO sensors, respectively. The measurement vector at time t. , and These represent the measurement matrices of GNSS, UWB, and VIO sensors, respectively, and map the state vectors to the measurement space. , and These represent the GNSS, UWB, and VIO sensors, respectively. The measurement noise vector at time step;

[0048] Based on the state equation, the current state vector is predicted using the state estimate from the previous time step, and the error covariance matrix is ​​also predicted, completing the prediction step of the adaptive robust Kalman filter. The state vector update process is expressed as follows:

[0049]

[0050] in, This represents the state vector at the current moment. The state estimate is given at the previous moment; the prediction process for the error covariance matrix is ​​expressed as follows:

[0051]

[0052] in, This represents the prediction error covariance matrix at the current time. The estimation error covariance matrix for the previous time step. The process noise covariance matrix at the previous time step;

[0053] Substitute the measurement data from each sensor into the corresponding measurement equation to calculate the innovation vector:

[0054]

[0055] in, This represents the innovation vector of the GNSS sensor; similarly, the innovation vectors of the UWB and VIO sensors can be obtained. and Based on the innovation vector and the prior threshold, the robust covariance matrix corresponding to the measurement noise is constructed using the Huber function;

[0056] By using Kalman gain to update the state estimate and error covariance matrix, more accurate vehicle positioning results and vehicle state information are obtained. The state vector update results from GNSS, UWB, and VIO are fused together. Time Kalman Gain The calculation process is expressed as follows:

[0057]

[0058] in, This represents a measurement matrix that integrates GNSS, UWB, and VIO measurements. This represents the robust covariance matrix after fusion. Represent the inverse matrix; based on the fusion parameters, update the state estimates and covariance matrix:

[0059]

[0060]

[0061] in, This represents the updated final state vector. The fused measurement vector is obtained by weighted summation of the measurement data from each sensor. The identity matrix is ​​used to obtain the final positioning result of the unmanned law enforcement vehicle from the final state vector.

[0062] During the operation of the unmanned law enforcement vehicle, the updated position and status of the unmanned law enforcement vehicle are received in real time from the tightly coupled positioning fusion algorithm. At the same time, combined with the real-time collected road traffic data and the distribution of traffic violations, the route is dynamically adjusted using an A*-based path planning algorithm.

[0063] Step S20: Using an unmanned aerial vehicle, an aerial patrol and monitoring system is conducted along a set route using a spatiotemporal cube segmentation algorithm. This system collects traffic flow, vehicle distribution, and road anomaly information from an aerial perspective, forming an integrated air-ground three-dimensional perception network with the unmanned law enforcement vehicle.

[0064] In this embodiment, the unmanned aerial vehicle is equipped with a high-definition zoom camera, an infrared thermal imager, a positioning module, a data processing module, and an autonomous navigation module.

[0065] The data processing module receives image data from the high-definition zoom camera and infrared thermal imager, positioning data of the unmanned aerial vehicle (UAV) collected by the positioning module, and location information of road vehicles, and transmits the data; the autonomous navigation module processes the positioning data in the positioning module and dynamically adjusts the flight path of the UAV.

[0066] The unmanned aerial vehicle is a multi-rotor unmanned aerial vehicle. Its patrol area is divided into multiple blocks according to the urban road layout, and each block is set with a fixed patrol route. The high-definition zoom camera can clearly capture vehicles and road conditions at a distance, and the infrared thermal imager can still effectively monitor road conditions at night or in bad weather conditions.

[0067] Before planning the flight path of the unmanned aerial vehicle, information on the target patrol area is collected, including geographic information data, meteorological data, airspace control data, and traffic flow data; the collected data is preprocessed to remove noise and missing values ​​and to complete the standardization process.

[0068] Based on the collected target area data, a spatiotemporal cube model is constructed, which discretizes the target area in three-dimensional space (longitude, latitude, and altitude) and time dimension to form multiple spatiotemporal units:

[0069] The granularity of spatial division is adjusted according to the patrol area of ​​the unmanned aerial vehicle. In areas with heavy traffic and high monitoring requirements, the side length of the spatial unit is set to 10-50m. During the division process, the safe flight altitude range of the unmanned aerial vehicle and the effective monitoring distance of the monitoring equipment need to be considered to ensure that the divided spatial units can cover the entire monitoring area.

[0070] When dividing the time dimension, the time axis is divided into multiple time intervals. The length of the time interval is determined according to the dynamic changes in road traffic. For urban road traffic monitoring, the time interval is set to 5-15 minutes to capture the dynamic changes in traffic flow in a timely manner. For the stable traffic environment of highways, the time interval is set to 30-60 minutes. During peak traffic periods, the time interval is shortened to improve the monitoring frequency and data update speed.

[0071] Combining the segmentation results of the spatiotemporal cube model with the performance parameters of the unmanned aerial vehicle, a path planning algorithm is adopted to determine the objective function. With the constraints of avoiding no-fly zones and reducing the energy consumption of the unmanned aerial vehicle, an optimal initial patrol route from the take-off point to the target area is searched in the spatiotemporal cube model.

[0072] The generated initial patrol route is mapped to a spatiotemporal cube model to determine the sequence of spatiotemporal cube units that the unmanned aerial vehicle (UAV) needs to pass through in different time periods, thus establishing a spatiotemporal trajectory model for the UAV. During the patrol and monitoring mission, the UAV receives real-time data on road traffic conditions and its own position and attitude, and feeds the received real-time information back to the model to dynamically adjust the UAV's patrol route.

[0073] This embodiment employs a "global-local" two-layer decision-making model, based on a spatiotemporal cube model to achieve dynamic hierarchical management of the airspace:

[0074] Based on the segmentation results of the spatiotemporal cube model and the flight altitude range of the aircraft, the airspace is divided into multiple levels, each level corresponding to different altitude ranges and usage classifications. The relationships and conversion rules between different airspace levels are defined, and it is clarified under what circumstances the aircraft can make inter-level conversions, as well as the conditions and approval processes that need to be met during the conversion.

[0075] By utilizing various ground monitoring devices deployed within the monitoring area, feedback information from other unmanned aerial vehicles, and data provided by meteorological departments, real-time airspace status information is collected, including the number of aircraft in the airspace, flight altitude, flight speed, flight direction, and weather conditions. The real-time airspace status information is analyzed and processed using a spatiotemporal cube segmentation algorithm, and airspace dynamic adjustment decisions are automatically generated based on preset airspace usage strategies and safety rules.

[0076] By combining unmanned law enforcement vehicles and unmanned aerial vehicles (UAVs), an integrated air-ground three-dimensional perception network is constructed, combining "wide-area aerial perception with fine-grained ground detection." UAVs are responsible for macro-level road network monitoring, while unmanned law enforcement vehicles perform fine-grained detection of key areas. An improved Grey Wolf optimization algorithm is used to achieve intelligent task allocation, automatically switching inspection modes based on real-time road conditions. The intelligent task allocation process includes the following steps:

[0077] Based on the number of deployed unmanned law enforcement vehicles and unmanned aerial vehicles, as well as their initial positions and battery status, initialize the optimized gray wolf swarm:

[0078] Each gray wolf represents a task allocation scheme, and its location vector contains the proportion and order in which unmanned equipment (law enforcement vehicles and aircraft) are assigned to different traffic monitoring task areas;

[0079] Define a fitness function to measure the merits of each task allocation scheme. The fitness function takes into account factors such as task completion time, task completion quality, equipment energy consumption, and task allocation balance.

[0080] Based on the location update formula of the gray wolf optimization algorithm, an adaptive mechanism is introduced to dynamically adjust the algorithm parameters (population size, maximum number of iterations, and step size parameter for location update). Traffic flow and congestion level are added as weight factors to the location update parameters to simulate the group behavior of gray wolves.

[0081] Iteratively update each gray wolf position (task allocation scheme), calculate the fitness value of the task scheme corresponding to each gray wolf, and select the current alpha wolf (the task scheme with the smallest fitness value) to guide the search direction of other gray wolves.

[0082] Set a maximum iteration threshold. When the algorithm reaches the maximum number of iterations, stop the iteration process and take the task allocation scheme represented by the alpha wolf at this point as the final scheme.

[0083] Step S30: Receive real-time data collected by unmanned law enforcement vehicles and unmanned aerial vehicles, introduce artificial intelligence recognition algorithms to analyze the data, quickly identify traffic incidents, and send instructions to unmanned law enforcement vehicles and unmanned aerial vehicles based on the recognition results to focus on monitoring and managing the incident area.

[0084] In this embodiment, cloud computing and big data technologies are used to establish an integrated air-ground data processing module. This module includes a data receiving module, a data storage module, a data comparison and analysis module, an alarm identification module, and an instruction sending module.

[0085] The data receiving module is used to receive data transmitted by the unmanned vehicle monitoring module and the drone monitoring module. It adopts 5G communication technology to ensure the real-time performance and stability of data transmission.

[0086] The data storage module is used to store the received data, and to build a distributed file system, object storage and time-series database to store log data (operation status and operation records of unmanned law enforcement vehicles and unmanned aerial vehicles), video data (dynamic and static information of vehicles, pedestrians and traffic signs in traffic scenes collected by unmanned law enforcement vehicles, recording traffic violations) and sensor data (vehicle speed, acceleration, distance and vehicle status data collected by unmanned law enforcement vehicles, and road traffic flow changes and vehicle distribution data collected by unmanned aerial vehicles).

[0087] The data comparison and analysis module correlates ground-based near-field data collected by unmanned law enforcement vehicles with aerial global data collected by unmanned aerial vehicles to compensate for the shortcomings of a single monitoring method and improve the richness and accuracy of data collection.

[0088] When an unmanned aerial vehicle detects abnormal traffic flow in a certain area, it instructs an unmanned law enforcement vehicle to go to that area. The unmanned vehicle then conducts close-range monitoring to confirm whether a traffic accident or road congestion has occurred, and compares and verifies the data from both.

[0089] The incident identification module analyzes data and uses artificial intelligence algorithms to identify traffic incidents.

[0090] Embedded pre-trained target detection models can analyze vehicle images and road environment images in real time, promptly detect traffic accident scenes, and detect obstacles on the road.

[0091] For temporary roadside parking, vehicle license plate information is collected based on license plate recognition algorithms, the parking time of the vehicle is recorded, and parking thresholds are set to determine whether the vehicle has engaged in illegal parking behavior.

[0092] Embedding a pre-trained time series prediction model based on graph neural networks, the system predicts changes in traffic flow over a period of time and identifies congested road sections based on collected historical traffic density data.

[0093] An embedded lane segmentation model is used to identify vehicles' illegal driving behavior based on high-altitude road images collected by unmanned aerial vehicles.

[0094] Based on the alarm identification results, the command sending module sends commands to the unmanned law enforcement vehicle and the unmanned aerial vehicle, instructing the unmanned law enforcement vehicle to go to the alarm location for close verification, and instructing the unmanned aerial vehicle to adjust the monitoring angle and position for key monitoring.

[0095] Example 2: The air-ground integrated road traffic monitoring system provided in this embodiment of the invention can execute the air-ground integrated road traffic monitoring method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method. The detailed structure of the system is as follows: Figure 2 As shown, it includes the following modules:

[0096] Unmanned vehicle monitoring module: It adopts an unmanned law enforcement vehicle, which travels on the road according to the route planned by the tightly coupled positioning fusion algorithm, and collects information on vehicle driving status, road environment and traffic violations in real time;

[0097] Unmanned aerial vehicle (UAV) monitoring module: Using unmanned aerial vehicles and spatiotemporal cube segmentation algorithm, it conducts aerial patrol and monitoring according to a set route, collecting traffic flow, vehicle distribution and road anomaly information from an aerial perspective, forming an integrated air-ground three-dimensional perception network with unmanned law enforcement vehicles;

[0098] Air-Ground Integrated Data Processing Module: This module receives real-time data collected by unmanned law enforcement vehicles and unmanned aerial vehicles, incorporates artificial intelligence recognition algorithms to analyze the data, quickly identifies traffic incidents, and sends instructions to unmanned law enforcement vehicles and unmanned aerial vehicles based on the identification results for key monitoring and management of the incident area.

[0099] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application. In some cases, the actions or steps described in this application can be performed in a different order than in the embodiments and still achieve the desired results.

Claims

1. A method for comprehensive road traffic monitoring based on integrated air-ground systems, characterized in that: The method includes: (1) An unmanned law enforcement vehicle is used to drive on the road according to the route planned by the tightly coupled positioning fusion algorithm, and collect information on the vehicle driving status, road environment and traffic violations in real time; (2) Using unmanned aerial vehicles, the spatiotemporal cube segmentation algorithm is used to conduct aerial patrol and monitoring according to the set route, and collect traffic flow, vehicle distribution and road anomaly information on the road from the aerial perspective, forming an integrated air-ground three-dimensional perception network with unmanned law enforcement vehicles. The unmanned aerial vehicle is equipped with a high-definition zoom camera, an infrared thermal imager, a positioning module, a data processing module, and an autonomous navigation module. The data processing module receives image data from the high-definition zoom camera and infrared thermal imager, positioning data of the unmanned aerial vehicle (UAV) collected by the positioning module, and location information of road vehicles, and transmits the data; the autonomous navigation module processes the positioning data in the positioning module and dynamically adjusts the flight path of the UAV. The unmanned aerial vehicle is a multi-rotor unmanned aerial vehicle. The patrol area is divided into multiple blocks according to the urban road layout, and each block is assigned a fixed patrol route. The high-definition zoom camera clearly captures the vehicles and road conditions from an aerial perspective, and the infrared thermal imager monitors the road conditions at night or in bad weather conditions. Before planning the flight path of the unmanned aerial vehicle, information on the target patrol area is collected, including geographic information data, meteorological data, airspace control data, and traffic flow data; the collected data is preprocessed to remove noise and missing values ​​and to complete the standardization process. The planned flight path for the unmanned aerial vehicle includes the following steps: Based on the collected target area data, a spatiotemporal cube model is constructed, and the target area is discretized in three-dimensional space and time dimension to form multiple spatiotemporal units; Combining the partitioning results of the spatiotemporal cube model with the performance parameters of the unmanned aerial vehicle, a path planning algorithm is adopted to determine the objective function. With the constraints of avoiding no-fly zones and reducing the energy consumption of the unmanned aerial vehicle, an optimal initial patrol route from the take-off point to the target area is searched in the spatiotemporal cube model. The generated initial patrol route is mapped to the spatiotemporal cube model to establish the spatiotemporal trajectory model of the unmanned aerial vehicle (UAV). During the patrol and monitoring mission, the UAV receives real-time road traffic data and its own position and attitude information, and feeds the received real-time information back to the model to dynamically adjust the UAV's patrol route. The three-dimensional perception network is an integrated air-ground three-dimensional perception network that combines unmanned law enforcement vehicles and unmanned aerial vehicles to form a "wide-area air perception + fine ground detection" system. Unmanned aerial vehicles are responsible for macro-level road network monitoring, while unmanned law enforcement vehicles perform fine detection of key areas. Based on a three-dimensional perception network, an improved gray wolf optimization algorithm is used to achieve intelligent task allocation and automatically switch inspection modes according to real-time traffic conditions. The detailed steps include: Based on the number of deployed unmanned law enforcement vehicles and unmanned aerial vehicles, as well as their initial positions and battery status, initialize the optimized gray wolf swarm: Each gray wolf represents a task allocation scheme, and its position vector contains the proportion and order in which the unmanned vehicle is assigned to different traffic monitoring task areas; a fitness function is defined, which takes into account factors such as task completion time, task completion quality, equipment energy consumption and task allocation balance. Based on the location update formula of the gray wolf optimization algorithm, an adaptive mechanism is introduced to dynamically adjust the algorithm parameters. Traffic flow and congestion level are added as weight factors to the location update parameters to simulate the group behavior of gray wolves. For each gray wolf's position, iterative updates are performed, the fitness value of the task plan corresponding to each gray wolf is calculated, and the task allocation plan with the smallest fitness value of the current alpha wolf is selected to guide the search direction of other gray wolves. Set a maximum iteration threshold. When the algorithm reaches the maximum number of iterations, stop the iteration process and take the task allocation scheme represented by the alpha wolf at this time as the final scheme. (3) Receive real-time data collected by unmanned law enforcement vehicles and unmanned aerial vehicles, introduce artificial intelligence recognition algorithms to analyze the data, quickly identify traffic incidents, and send instructions to unmanned law enforcement vehicles and unmanned aerial vehicles based on the recognition results to focus on monitoring and managing the incident area.

2. The road traffic monitoring method based on air-ground integration as described in claim 1, characterized in that, In step (1), the unmanned law enforcement vehicle is equipped with a high-definition camera, millimeter-wave radar, lidar, GPS positioning module, and data transmission module: The GPS positioning module receives positioning data collected by millimeter-wave radar and lidar, while the data transmission module is used to transmit image data collected by a high-definition camera and positioning data obtained by the GPS positioning module. During operation, high-definition cameras capture images of vehicle license plates and traffic violations; millimeter-wave radar and lidar detect vehicle distance, acceleration, and speed; and the GPS positioning module integrates positioning data collected by three positioning sensors—GNSS, UWB, and VIO—to plan the autonomous enforcement vehicle's route and record precise monitoring locations.

3. The air-ground integrated road traffic monitoring method as described in claim 2, characterized in that, The planned driving path for the unmanned law enforcement vehicle includes the following steps: First, location information is obtained through a tightly coupled positioning fusion algorithm: The system collects positioning data from different positioning sources during the operation of the unmanned law enforcement vehicle: GNSS provides global positioning information, UMB provides high-precision relative positioning information within a short distance, and VIO estimates pose information based on the unmanned law enforcement vehicle's own motion. The collected positioning data is preprocessed to remove obvious errors and interference. By using an adaptive robust Kalman filter algorithm, preprocessed GNSS, UWB and VIO positioning data are fused to obtain accurate vehicle location information; Secondly, during the driving process, the unmanned law enforcement vehicle receives updated vehicle position and status information from the tightly coupled positioning fusion algorithm in real time. At the same time, it combines real-time collected road traffic data and the distribution of traffic violations, and uses an A*-based path planning algorithm to dynamically adjust the route.

4. The road traffic full-area monitoring method based on air-ground integration as described in claim 3, characterized in that, The detailed steps of the adaptive robust Kalman filter algorithm include: The state vector containing the position, velocity, and attitude angle of the unmanned law enforcement vehicle is determined. The state vector is initialized using the GNSS, UWB, and VIO sensor positioning data at the initial moment, and the covariance matrix is ​​initialized based on the uncertainty of the initial estimate. State equations are established based on the system dynamics characteristics, and measurement equations are established for the positioning data of GNSS, UWB and VIO sensors respectively. The state vector at the current moment is predicted using the state estimate from the previous moment, and the error covariance matrix is ​​also predicted, thus completing the prediction step of the adaptive robust Kalman filter. The measurement data of each sensor is substituted into the corresponding measurement equation to calculate the innovation vector. Based on the innovation vector and the prior threshold, the robust covariance matrix corresponding to the measurement noise is constructed using the Huber function. By using Kalman gain to update the state estimate and error covariance matrix, the localization result and state information of the unmanned law enforcement vehicle are obtained. The state vector update results of GNSS, UWB and VIO are fused to obtain the final localization result of the unmanned law enforcement vehicle.

5. The road traffic full-area monitoring method based on air-ground integration as described in claim 1, characterized in that, In step (3), the methods for identifying traffic incidents include: By applying a pre-trained target detection model, vehicle images and road environment images on the road are analyzed in real time to promptly detect traffic accident scenes and obstacles on the road. For temporary roadside parking, vehicle license plate information is collected based on license plate recognition algorithms, the parking time of the vehicle is recorded, and parking thresholds are set to determine whether the vehicle has engaged in illegal parking behavior. By applying a pre-trained time series prediction model based on graph neural networks, traffic flow changes over a period of time are predicted based on collected historical traffic density data, and traffic congestion sections are identified. By applying a lane segmentation model and using high-altitude road images collected by an unmanned aerial vehicle, illegal driving behavior of vehicles is identified.

6. A road traffic monitoring system based on integrated air-ground systems, characterized in that: The system includes: Unmanned vehicle monitoring module: It adopts an unmanned law enforcement vehicle, which travels on the road according to the route planned by the tightly coupled positioning fusion algorithm, and collects information on vehicle driving status, road environment and traffic violations in real time; Unmanned aerial vehicle (UAV) monitoring module: Using unmanned aerial vehicles and spatiotemporal cube segmentation algorithm, it conducts aerial patrol and monitoring according to a set route, collecting traffic flow, vehicle distribution and road anomaly information from an aerial perspective, forming an integrated air-ground three-dimensional perception network with unmanned law enforcement vehicles; The unmanned aerial vehicle is equipped with a high-definition zoom camera, an infrared thermal imager, a positioning module, a data processing module, and an autonomous navigation module. The data processing module receives image data from the high-definition zoom camera and infrared thermal imager, positioning data of the unmanned aerial vehicle (UAV) collected by the positioning module, and location information of road vehicles, and transmits the data; the autonomous navigation module processes the positioning data in the positioning module and dynamically adjusts the flight path of the UAV. The unmanned aerial vehicle is a multi-rotor unmanned aerial vehicle. The patrol area is divided into multiple blocks according to the urban road layout, and each block is assigned a fixed patrol route. The high-definition zoom camera clearly captures the vehicles and road conditions from an aerial perspective, and the infrared thermal imager monitors the road conditions at night or in bad weather conditions. Before planning the flight path of the unmanned aerial vehicle, information on the target patrol area is collected, including geographic information data, meteorological data, airspace control data, and traffic flow data; the collected data is preprocessed to remove noise and missing values ​​and to complete the standardization process. The planned flight path for the unmanned aerial vehicle includes the following steps: Based on the collected target area data, a spatiotemporal cube model is constructed, and the target area is discretized in three-dimensional space and time dimension to form multiple spatiotemporal units; Combining the partitioning results of the spatiotemporal cube model with the performance parameters of the unmanned aerial vehicle, a path planning algorithm is adopted to determine the objective function. With the constraints of avoiding no-fly zones and reducing the energy consumption of the unmanned aerial vehicle, an optimal initial patrol route from the take-off point to the target area is searched in the spatiotemporal cube model. The generated initial patrol route is mapped to the spatiotemporal cube model to establish the spatiotemporal trajectory model of the unmanned aerial vehicle (UAV). During the patrol and monitoring mission, the UAV receives real-time road traffic data and its own position and attitude information, and feeds the received real-time information back to the model to dynamically adjust the UAV's patrol route. The three-dimensional perception network is an integrated air-ground three-dimensional perception network that combines unmanned law enforcement vehicles and unmanned aerial vehicles to form a "wide-area air perception + fine ground detection" system. Unmanned aerial vehicles are responsible for macro-level road network monitoring, while unmanned law enforcement vehicles perform fine detection of key areas. Based on a three-dimensional perception network, an improved gray wolf optimization algorithm is used to achieve intelligent task allocation and automatically switch inspection modes according to real-time traffic conditions. The detailed steps include: Based on the number of deployed unmanned law enforcement vehicles and unmanned aerial vehicles, as well as their initial positions and battery status, initialize the optimized gray wolf swarm: Each gray wolf represents a task allocation scheme, and its position vector contains the proportion and order in which the unmanned vehicle is assigned to different traffic monitoring task areas; a fitness function is defined, which takes into account factors such as task completion time, task completion quality, equipment energy consumption and task allocation balance. Based on the location update formula of the gray wolf optimization algorithm, an adaptive mechanism is introduced to dynamically adjust the algorithm parameters. Traffic flow and congestion level are added as weight factors to the location update parameters to simulate the group behavior of gray wolves. For each gray wolf's position, iterative updates are performed, the fitness value of the task plan corresponding to each gray wolf is calculated, and the task allocation plan with the smallest fitness value of the current alpha wolf is selected to guide the search direction of other gray wolves. Set a maximum iteration threshold. When the algorithm reaches the maximum number of iterations, stop the iteration process and take the task allocation scheme represented by the alpha wolf at this time as the final scheme. Air-Ground Integrated Data Processing Module: This module receives real-time data collected by unmanned law enforcement vehicles and unmanned aerial vehicles, incorporates artificial intelligence recognition algorithms to analyze the data, quickly identifies traffic incidents, and sends instructions to unmanned law enforcement vehicles and unmanned aerial vehicles based on the identification results for key monitoring and management of the incident area.

Images