A monitoring and cooperative management system based on intelligent identification of traffic violations

Abstract

The application belongs to the technical field of intelligent traffic, vehicle-road cooperation and traffic monitoring, and discloses a monitoring and cooperative management system based on intelligent identification of traffic violations, which comprises a multi-modal heterogeneous perception layer, an edge and cloud cooperative transmission layer, a dynamic game decision layer, a cross-domain cooperative disposal layer connected in sequence in communication, and an intelligent traceability and optimization layer in bidirectional communication with the dynamic game decision layer. By constructing the four-modal heterogeneous perception layer of space base, road base, vehicle-mounted and human base, multi-dimensional data cross verification is realized, solving the problem of single or binary perception blind area and high misjudgment rate in the prior art. By designing a dynamic game decision algorithm linked in real time with multi-modal fusion data, introducing a dynamic weight adjustment, scene correction and illegal hazard level coefficient calibration mechanism, the illegal level can be accurately distinguished, and different traffic scenes can be adapted, solving the problem that the existing algorithm cannot adapt to complex scenes and has low accuracy.

Description

Technical Field

[0001] This invention belongs to the fields of intelligent transportation, vehicle-road cooperation, and traffic monitoring technology, specifically a monitoring and collaborative management system based on intelligent identification of traffic violations. Background Technology

[0002] With the rapid growth of road traffic flow, traffic accidents caused by traffic violations occur frequently. Existing traffic violation monitoring systems use multimodal perception combining airborne and roadbed elements to intelligently identify and monitor traffic violations, which is more efficient than traditional manual detection.

[0003] However, existing multimodal sensing is mostly limited to a binary combination of airborne and roadbed sensing. Each sensing module works independently and the data is not verified by each other, which easily leads to blind spots and misjudgments. Moreover, its dynamic game decision-making algorithm is mostly applied to risk assessment alone and does not form a real-time linkage with multimodal sensing data. It is not convenient to dynamically adjust the algorithm weights according to the sensing data, and cross-domain collaboration can only achieve three-level linkage of vehicle, road and cloud or road, cloud and people.

[0004] A search revealed that application CN202510019584.5 provides a road traffic information monitoring system and method that uses video surveillance, sensors, drones and satellite equipment to acquire traffic data in real time. This application is a three-level collaborative linkage mechanism of vehicle, road and cloud, but it lacks the participation of human-based supplementary perception, which is not conducive to forming a closed loop of perception, decision-making, handling, tracing and optimization, and is not conducive to achieving rapid response. Application CN202110319298.2 provides a traffic management system based on facial recognition. It uses a monitoring module to comprehensively monitor road conditions, transmits signals to a reading module, and then through a 3D modeling module, a comparison module, and a storage module to confirm personnel information before transmitting it to the traffic control console. After confirmation by the authorization module, the control module operates traffic lights to direct traffic. This application is a three-level collaborative linkage mechanism of road, cloud, and human, but it lacks the participation of vehicle-based systems, which is not conducive to forming a closed loop of perception, decision-making, handling, tracing, and optimization, and is not conducive to achieving rapid response.

[0005] Therefore, a monitoring and collaborative management system based on intelligent identification of traffic violations is needed. Summary of the Invention

[0006] The purpose of this invention is to provide a monitoring and collaborative management system based on intelligent identification of traffic violations, so as to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a monitoring and collaborative management system based on intelligent identification of traffic violations, comprising a multimodal heterogeneous perception layer, an edge and cloud collaborative transmission layer, a dynamic game decision layer, a cross-domain collaborative processing layer, and an intelligent tracing and optimization layer that communicates bidirectionally with the dynamic game decision layer; The multimodal heterogeneous sensing layer includes an airborne dynamic sensing module, a roadbed holographic sensing module, a vehicle-mounted active sensing module, and a human-based supplementary sensing module. The edge and cloud collaborative transmission layer includes a dynamic priority transmission protocol module and a heterogeneous data adaptation module. The dynamic game decision-making layer runs a dynamic game decision-making algorithm for illegality and compliance, which is based on multimodal fusion data. The cross-domain collaborative processing layer establishes a four-level collaborative mechanism involving vehicles, roads, cloud, and people. The intelligent tracing and optimization layer includes an illegal source analysis module and an algorithm self-iteration module. The intelligent tracing and optimization layer and the dynamic game decision layer communicate bidirectionally.

[0008] Furthermore, the airborne dynamic perception module is a tethered drone swarm equipped with micro-Doppler radar and behavior recognition LiDAR. The tethered drone swarm is equipped with micro-Doppler radar, and the deployment density of the tethered drone swarm is 1 drone per 5 square kilometers, with a hovering height of 50-80m and a sampling frequency of 10Hz, supporting swarm collaborative networking.

[0009] Furthermore, the roadbed holographic perception module includes a piezoelectric and visual fusion road surface sensor and a violation prediction unit. One set of the piezoelectric and visual fusion road surface sensor is deployed every 200m, and data is collected synchronously with the piezoelectric sensor, with a collection frequency of 10Hz.

[0010] Furthermore, the vehicle-mounted active perception module connects to the vehicle's OBD interface and ADAS system, and adopts the ISO15765-4 interface protocol. The data transmission format of the vehicle-mounted active perception module is the CAN bus standard format. The human-based supplementary perception module is a passenger violation monitoring terminal deployed in buses and taxis, equipped with Android 11 system, a built-in 2-megapixel camera, 8GB storage capacity, supports 4G / 5G wireless transmission, and adopts the TCP / IP protocol. The passenger violation monitoring terminal has a built-in digital signature module and adopts the RSA256 encryption algorithm.

[0011] Furthermore, the dynamic priority transmission protocol module establishes a mechanism for binding violation level with transmission priority. The heterogeneous data adaptation module completes the cleaning and spatiotemporal registration of radar point cloud, video stream, piezoelectric signal, and OBD data at the edge. The specific threshold and implementation steps of the data cleaning are to use the 3σ criterion to remove outliers. The calculation basis of σ is the standard deviation of the corresponding sensing data in the past 3 months.

[0012] Furthermore, the dynamic game decision-making algorithm for illegality and compliance running in the dynamic game decision-making layer is specifically implemented through the following formula: ; ; The The number of perception dimensions participating in the game is 4, corresponding to airborne, roadborne, vehicle-mounted, and human-based. for The dynamic weights of the i-th class of sensing data at time i, and satisfying and The The behavioral feature value collected by the i-th type of sensing device has a value range of [0,1]. This is a scene correction coefficient, with a value range of [0.2, 1.5]. The coefficient for the level of illegality and harm has a value range of [1, 10]. for Real-time violation risk value, ranging from [0,10]. for Real-time dynamic violation threshold.

[0013] Furthermore, the four-level collaborative mechanism process of vehicles, roads, cloud, and people in the cross-domain collaborative processing layer is as follows: A1: Vehicle-to-vehicle collaboration: Push voice and visual warnings to vehicles after violation prediction or identification. Voice warnings provide real-time voice prompts indicating the type of violation, while visual warnings are pop-up prompts on the in-vehicle display screen. A2: Roadside coordination: Link traffic lights and electronic screens at intersections to trigger temporary light control for high-risk vehicles. Specifically, vehicles that run red lights will trigger a 10-15 second extension of the red light at the intersection, and vehicles that exceed the speed limit by more than 50% will trigger the green light at the intersection ahead to turn red. At the same time, the electronic screen will display the vehicle's license plate number, violation type, and warning information in real time. A3: Cloud Collaboration: Second-level solidification of multi-device integrated illegal evidence chain, which includes multi-dimensional perception data, illegal scene video, time and location information, and is simultaneously pushed to the traffic police command platform and the illegal processing system; A4: Personnel Collaboration: Issue handling instructions containing the location, type, and risk level of the illegal vehicle to nearby patrolling traffic police and traffic assistants to achieve rapid on-site intervention.

[0014] Furthermore, the handling rules corresponding to the handling instructions include flexible handling of minor violations and rigid closed-loop handling of high-risk violations.

[0015] Furthermore, the flexible handling of minor violations refers to first-time illegal parking that does not affect traffic, while the rigid closed loop for high-risk violations refers to drunk driving and speeding by more than 50%.

[0016] A method for intelligent identification, monitoring, and collaborative management of traffic violations includes the following steps: S1: The multimodal heterogeneous sensing layer synchronously collects multi-dimensional data from airborne, roadbed, vehicle-mounted, and human-based systems. The edge end completes data cleaning and spatiotemporal registration. First, the collected data is denoised and deduplicated. Then, spatiotemporal registration is achieved through timestamp synchronization and GPS positioning calibration, and standardized feature data is output. S2: The edge and cloud collaborative transmission layer transmits standardized data to the dynamic game decision-making layer according to dynamic priority; S3: The dynamic game decision-making layer runs a dynamic game decision-making algorithm for violations and compliance, calculates real-time violation risk values, and determines illegal behaviors, warning levels, or exemption situations. The criteria for exemption situations are: first minor violation, no safety hazards caused, and proactive rectification. After system verification, the penalty can be waived. S4: The cross-domain collaborative response layer initiates four-level collaboration among vehicles, roads, cloud, and people to execute early warning, evidence consolidation, and on-site intervention operations; S5: Intelligent source tracing and optimization layer analyzes the causes of violations, automatically optimizes algorithm parameters and the deployment location of sensing devices. The optimization logic for the deployment location of sensing devices is to adjust the hovering position of airborne drones and the embedding density of roadside sensors according to the location and range of high-incidence areas of violations, so as to ensure that there are no blind spots in the perception coverage of high-incidence areas.

[0017] The beneficial effects of this invention are as follows: 1. This invention constructs a four-modal heterogeneous sensing layer consisting of airborne, roadbed, vehicle-mounted, and human-based sensors to achieve multi-dimensional data cross-validation, thus solving the problems of blind spots and high misjudgment rates in existing single or binary sensing technologies.

[0018] 2. This invention designs a dynamic game-theoretic decision-making algorithm that is linked with multimodal fusion data in real time. It introduces dynamic weight adjustment, scenario correction and violation hazard level coefficient calibration mechanism, which can accurately distinguish the violation level and adapt to different traffic scenarios. This solves the problem that existing algorithms cannot adapt to complex scenarios and have low judgment accuracy.

[0019] 3. This invention constructs a four-level collaborative mechanism involving vehicles, roads, cloud, and people, and adds a personnel collaborative handling link, realizing a closed-loop management of perception, decision-making, handling, tracing, and optimization. The four-level collaborative handling response time is faster, improving the efficiency of handling violations. At the same time, the human-based supplementary perception module solves the problem of human-based perception being easily affected by subjective factors through digital signatures, spatiotemporal consistency verification, etc., ensuring data validity, supplementing the post-event evidence chain, and reducing the situation of missed judgments of violations.

[0020] 4. This invention achieves adaptive system upgrades through intelligent traceability and optimization layer algorithm self-iteration and sensing device deployment optimization. Algorithm parameters and device deployment can be optimized without manual intervention, reducing operation and maintenance costs while improving the system's long-term stability and applicability. Attached Figure Description Figure 1 This is a diagram of the overall system architecture of the present invention; Figure 2 This is the logic diagram for cross-validation of multimodal sensing data in this invention; Figure 3 This is a four-level collaborative working logic diagram of the cross-domain collaborative processing layer of the present invention; Figure 4 This is a flowchart of the algorithm self-iteration of the intelligent tracing and optimization layer of this invention; Figure 5 This is a flowchart illustrating the overall implementation of the method of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Example 1: like Figures 1 to 5 As shown, this embodiment of the invention provides a monitoring and collaborative management system based on intelligent identification of traffic violations, including a multimodal heterogeneous perception layer, an edge and cloud collaborative transmission layer, a dynamic game decision layer, a cross-domain collaborative processing layer, and an intelligent tracing and optimization layer that communicates bidirectionally with the dynamic game decision layer. The multimodal heterogeneous sensing layer includes an airborne dynamic sensing module, a roadbed holographic sensing module, a vehicle-mounted active sensing module, and a human-based supplementary sensing module; The edge and cloud collaborative transmission layer includes a dynamic priority transmission protocol module and a heterogeneous data adaptation module; The dynamic game decision-making layer operates a dynamic game decision-making algorithm for illegality and compliance, which is based on multimodal fusion data; A cross-domain collaborative handling layer is established to construct a four-level collaborative mechanism involving vehicles, roads, cloud, and people; The intelligent tracing and optimization layer includes an illegal source analysis module and an algorithm self-iteration module. The intelligent tracing and optimization layer and the dynamic game decision-making layer communicate bidirectionally.

[0023] Multi-dimensional heterogeneous data on vehicle behavior, driver operation, and environmental status are collected through the space-based dynamic sensing module, road-based holographic sensing module, vehicle-mounted active sensing module, and human-based supplementary sensing module of the multimodal heterogeneous sensing layer. The dynamic priority transmission protocol module and heterogeneous data adaptation module of the edge and cloud collaborative transmission layer can realize hierarchical transmission of multi-source data and edge pre-fusion. Edge pre-fusion is to initially associate and integrate radar point clouds, video streams, piezoelectric signals and OBD data collected from airborne, road-based, vehicle-mounted and human-based sources, so as to reduce the data processing pressure on the cloud and improve data transmission efficiency. By dynamically allocating the weights of each perception dimension and combining the scenario with the level of harm caused by the violation, the illegal risk value is calculated in real time, thereby dynamically determining the illegal behavior and the warning level. Cross-domain refers to cross-regional traffic control areas and cross-departmental traffic control areas, such as traffic police and traffic management departments. The four-level collaborative interaction logic is as follows: vehicle-end receives early warning, road-end links control equipment, cloud-based evidence is solidified and instructions are issued, and personnel intervene on-site, realizing a closed-loop handling of violation early warning, evidence solidification, instruction issuance and on-site intervention. The illegal source analysis module and algorithm self-iteration module of the intelligent source tracing and optimization layer can be used to output illegal cause reports and automatically optimize algorithm parameters. The two-way communication between the intelligent source tracing and optimization layer and the dynamic game decision layer allows the dynamic game decision layer to transmit real-time illegal data, misjudgment cases and algorithm running parameters to the intelligent source tracing and optimization layer. The intelligent source tracing and optimization layer feeds back the optimized algorithm parameters and sensing device deployment suggestions to the dynamic game decision layer, realizing the dynamic optimization of the algorithm and sensing strategy.

[0024] Among them, the airborne dynamic perception module is a tethered drone swarm equipped with micro-Doppler radar and behavior recognition LiDAR. The tethered drone swarm is equipped with micro-Doppler radar, and the deployment density of the tethered drone swarm is 1 drone per 5 square kilometers, with a hovering height of 50-80m and a sampling frequency of 10Hz, supporting swarm collaborative networking.

[0025] Drone swarms can hover and cover a large area, and the swarms can work together to achieve full coverage. Simultaneously, they can capture the three-dimensional motion trajectory of vehicles. At the same time, the road-based holographic perception module can capture the driver's operation behavior, such as predicting lane changes without using turn signals and changes in head posture due to fatigue. LiDAR captures changes in the driver's head pitch and yaw angles and combines them with preset thresholds to predict fatigue driving status, while micro-Doppler radar captures the vehicle's motion trajectory. The data from both are complementary in real time, enabling synchronous recognition of trajectory and operation behavior.

[0026] The roadbed holographic perception module includes a piezoelectric and visual fusion road surface sensor and a violation prediction unit. One set of piezoelectric and visual fusion road surface sensors is deployed every 200m, and data is collected synchronously with the piezoelectric sensors. The collection frequency is 10Hz.

[0027] By embedding a piezoelectric and vision-fusion road surface sensor into the road surface, vehicle wheel pressure, speed, axle count data and 8K infrared video stream can be collected simultaneously. The fusion method is to spatiotemporally correlate the vehicle physical parameters collected by the piezoelectric sensor with the vehicle image features in the infrared video stream, and verify the image recognition results through physical parameters, thus solving the visual recognition blind spots at night and in rainy and foggy weather. The illegal behavior prediction unit uses millimeter-wave radar to capture early signs of abnormal vehicle operation, such as sudden acceleration before running a red light (the threshold for judgment is an increase in vehicle speed of ≥10km / h within 0.3 seconds) and sudden drop in vehicle speed before illegal parking (the threshold for judgment is a decrease in vehicle speed of ≥8km / h within 0.5 seconds). The time interval from identifying the early signs of abnormal operation to the occurrence of the illegal behavior is 0.5-2 seconds. The warning trigger condition is that the above-mentioned early signs of abnormal operation are captured and the duration is ≥0.2 seconds.

[0028] The vehicle-mounted active perception module connects to the vehicle's OBD interface and ADAS system, and adopts the ISO15765-4 interface protocol. The data transmission format of the vehicle-mounted active perception module is the CAN bus standard format. The human-based supplementary perception module is a passenger violation monitoring terminal deployed in buses and taxis. It is equipped with the Android 11 system, has a built-in 2-megapixel camera, 8GB of storage capacity, supports 4G / 5G wireless transmission, and adopts the TCP / IP protocol. The passenger violation monitoring terminal has a built-in digital signature module and adopts the RSA256 encryption algorithm.

[0029] The vehicle-mounted active sensing module collects real-time data on vehicle braking status, steering operation, and lighting control. This data is then cross-validated with airborne and road-based sensing data to eliminate equipment misjudgments. The cross-validation logic is that at least two of the three types of sensing data (airborne, road-based, and vehicle-mounted) detect the same illegal behavior, and the data deviation is ≤5%. This confirms the illegal behavior and eliminates misjudgments from a single device. The human-based supplementary sensing module supports passengers uploading illegal evidence from their dashcams. After system verification, the evidence is included in the illegal database. Verification criteria include that the evidence time is consistent with the on-site monitoring time, the evidence image is clear and the vehicle license plate and illegal scene are identifiable. Human-based data is only used for post-event supplementary evidence and review.

[0030] Among them, the dynamic priority transmission protocol module establishes a mechanism for binding violation level with transmission priority, and the heterogeneous data adaptation module completes the cleaning and spatiotemporal registration of radar point cloud, video stream, piezoelectric signal and OBD data at the edge. The specific threshold and implementation steps of data cleaning are to use the 3σ criterion to remove outliers. The calculation basis of σ is the standard deviation of the corresponding sensing data in the past 3 months.

[0031] High-risk illegal data is transmitted in milliseconds within 50ms, while general illegal data is transmitted in compressed form. The spatiotemporal registration method is based on timestamp synchronization and geographic location calibration, and standardized feature data is output.

[0032] The dynamic game decision-making algorithm for illegality and compliance, which operates at the dynamic game decision-making level, is specifically implemented through the following formula: ; ; The number of perception dimensions participating in the game is 4, corresponding to airborne, roadborne, vehicle-mounted, and human-based. for The dynamic weights of the i-th class of sensing data at time i, and satisfying and , The value represents the behavioral feature value collected by the i-th type of sensing device, and its range is [0,1]. This is a scene correction factor, with a value range of [0.2, 1.5]. This is the coefficient for the severity level of illegality, with a value range of [1, 10]. for Real-time violation risk value, ranging from [0, 10]. for Real-time dynamic violation thresholds; This is the convergence accuracy threshold. This represents the maximum number of iterations.

[0033] The behavioral feature values ​​collected by the i-th type of sensing device are obtained by standardizing the raw data collected by the sensing device, and the calculation method is as follows: ; Rainy day scene Emergency lane traffic scenarios Scenes around the school The values ​​are determined based on the degree of harm caused by illegal acts in different scenarios and the accuracy of the sensing devices. Level of violation severity coefficient, drunk driving scenarios Sealing scene The value is determined based on the probability and harmful consequences of illegal acts causing traffic accidents; Real-time violation risk value, with different values ​​corresponding to different violation types. The interval is used to distinguish specific types of violations; The threshold for violations is set dynamically at all times. The threshold for running a red light during normal hours is 3, the threshold for running a red light during morning and evening rush hours is 2.4, and the threshold for other types of violations is set according to their degree of harm. This is the convergence accuracy threshold, used to determine whether the algorithm iteration has converged. The number of algorithm iterations is taken as... When the number of iterations reaches the maximum value, the algorithm stops iterating and outputs the result; The behavioral feature value collected by the i-th type of sensing device is calculated as follows: The calibration cycle is one month; The scene correction coefficient, the classification standard and specific values ​​are as follows: emergency lane traffic scene 0.2, rural roads 0.6, urban main roads 0.8, urban expressways 1.0, school surroundings, such as school commuting hours, is 1.5, and the calibration cycle is 3 months; The severity level coefficient of illegal acts is defined by the following standards and specific values: crossing the line, not driving in the designated lane (1) is low risk, illegal parking (2) is medium-low risk, running a red light (6) is medium-high risk, speeding more than 50% (8) is high risk, and driving under the influence of alcohol (10) is extremely high risk. The calibration cycle is 6 months and is adjusted based on the traffic accident statistics in the jurisdiction over the past year. The real-time violation risk value at time t, with different ranges corresponding to different violation types, such as 0-2 for minor violations, 2-6 for general violations, and 6-10 for high-risk violations; The dynamic violation thresholds at time t are as follows: 3 for running a red light during normal hours, 2.4 for running a red light during morning and evening rush hours, 1.8 for minor violations, 3.0 for general violations, and 6.0 for high-risk violations. The calibration cycle is 1 month. The determination of illegality requires cross-validation of at least two types of data from airborne, roadbed, and vehicle-mounted sensing data, with dynamic weighting. Adjustments are made in real time according to the scenario. In rainy weather, the weight of visual perception data is reduced and the weight of radar perception data is increased. After the adjustment, the sum of all weights is still 1, ensuring the accuracy of data recognition in severe weather. The specific implementation details of the dynamic game decision-making algorithm include defining the weights. The dynamic adjustment logic, combined with real-time feedback from multimodal sensing data, enables calibration, specifically... Initially, the weights are allocated as follows: airborne (30%), roadbed (30%), vehicle-mounted (25%), and human-based (15%). These weights are dynamically adjusted in real-time based on the data accuracy of each sensing module. If the accuracy of a module falls below 85%, its weight is reduced by 5%-10%, and the reduced weight is simultaneously redistributed to modules with an accuracy ≥90%, ensuring... , The parameters are based on specific criteria and calibration methods, for rainy weather scenarios with rainfall ≥ 5 mm / h. Decreased accuracy of visual perception in emergency lane traffic scenarios The degree of harm caused by the violation is relatively low, and the scene is located around schools during school hours. The severity of the violations has increased. The calibration method involves collecting on-site violation data quarterly, comparing the algorithm's judgment results with the actual violations, and making corrections accordingly. The deviation of the value is ≤3%, and it is determined based on the probability of traffic accidents caused by illegal behavior, including drunk driving and driving under the influence of alcohol. This is extremely dangerous, exceeding the speed limit by more than 50%. Running a red light is considered high-risk. This is considered a medium-to-high risk area, involving driving over the lane lines or not following lane markings. For low-risk cases, the calibration method involves combining traffic accident statistics within the jurisdiction over the past year. If the probability of a certain type of violation causing an accident changes by ≥10%, then adjustments are made accordingly. Values; The passenger violation monitoring terminal is an embedded vehicle-mounted terminal, with a selectable size of 10cm×8cm×3cm. When uploading evidence, a unique signature is automatically generated. During cloud verification, the signature is compared with the terminal device number. Simultaneously, the spatiotemporal consistency of video frame timestamps, GPS location information, and on-site perception data is used to clarify the intelligent tracing and optimization layer. The algorithm's self-iteration module is triggered under two conditions: first, a cumulative number of misjudged cases ≥ 50; second, the accuracy rate for judging a certain type of violation is below 95% for three consecutive months. The mathematical model for parameter optimization uses the gradient descent algorithm, with the objective function being min |judgment result − actual violation situation|. The iteration step size is set to 0.01, and the iteration termination condition is consistent with the dynamic game algorithm. The system automatically iterates once per quarter, while also supporting manual iteration. Data cleaning uses the 3σ criterion to remove outliers, such as radar point cloud data deviation > 3σ, blurred frames or resolution < 720P in video streams, piezoelectric signal noise > 50dB, and OBD data missing rate > 10%. Deduplication uses a dual deduplication method based on timestamps and data features. For example, only one duplicate data point collected by the same device at the same timestamp is retained. Spatiotemporal registration uses timestamp synchronization with the cloud clock as the reference, ensuring that the time synchronization deviation of each sensing device is ≤ 50ms. GPS positioning calibration uses the WGS-84 coordinate system and corrects deviations through base station-assisted positioning. The final output is standardized feature data in JSON format, which includes data type, collection time, positioning information, and feature values. Human-based data is divided into two categories: real-time auxiliary verification and post-event supplementary verification. In real-time auxiliary verification, if the violation clues uploaded by the human-based terminal, such as drunk driving videos taken by passengers, are consistent with the spatiotemporal range of airborne and road-based sensing data, and the data features match (e.g., the vehicle license plate in the video matches the license plate collected by the road-based sensor), then the human-based data is incorporated into the data source for dynamic game theory decision-making to improve the accuracy of the judgment. In post-event supplementary verification, if the system determines there is a suspected violation but airborne and road-based data are insufficient, human-based data for the corresponding time period can be called for review to ensure no violation is missed. The collaborative logic between each layer is clearly defined. After the multimodal heterogeneous sensing layer transmits standardized data to the edge and cloud collaborative transmission layer, the dynamic priority transmission protocol module allocates transmission bandwidth according to the violation level of the data, classifying high-risk, general, and minor violations. High-risk illegal data is allocated bandwidth ≥10Mbps with a transmission delay ≤50ms; general illegal data is allocated bandwidth 5-10Mbps with a transmission delay ≤100ms; and minor illegal data is allocated bandwidth ≤5Mbps with a transmission delay ≤200ms. After receiving the data, the dynamic game decision layer calculates the real-time illegal risk value every 100ms. If it is determined to be a high-risk illegal activity, a trigger signal is immediately sent to the cross-domain collaborative handling layer, and real-time data is pushed to the intelligent tracing and optimization layer. After the cross-domain collaborative handling layer completes the handling, it feeds back the handling result to the intelligent tracing and optimization layer for algorithm iteration and optimization of sensing device deployment. The optimization results, algorithm parameters, and device deployment suggestions of the intelligent tracing and optimization layer are fed back to the dynamic game decision layer and the multimodal heterogeneous sensing layer in real time, forming a closed-loop collaboration.

[0034] The four-level collaborative mechanism process of vehicles, roads, cloud, and people in the cross-domain collaborative processing layer is as follows: A1: Vehicle-to-vehicle collaboration: Push voice and visual warnings to vehicles after violation prediction or identification. Voice warnings provide real-time voice prompts indicating the type of violation, while visual warnings are pop-up prompts on the in-vehicle display screen. A2: Roadside coordination: Link traffic lights and electronic screens at intersections to trigger temporary light control for high-risk vehicles. Specifically, vehicles that run red lights will trigger a 10-15 second extension of the red light at the intersection, and vehicles that exceed the speed limit by more than 50% will trigger the green light at the intersection ahead to turn red. At the same time, the electronic screen will display the vehicle's license plate number, violation type, and warning information in real time. A3: Cloud Collaboration: Second-level solidification of multi-device integrated illegal evidence chain, which includes multi-dimensional perception data, illegal scene video, time and location information, and is simultaneously pushed to the traffic police command platform and the illegal processing system; A4: Personnel Collaboration: Issue handling instructions containing the location, type, and risk level of the illegal vehicle to nearby patrolling traffic police and traffic assistants to achieve rapid on-site intervention.

[0035] The communication protocol, data interaction format, and response time threshold for the four-level collaboration are as follows: the vehicle-side warning uses a Bluetooth 5.0+4G dual-mode push protocol, the data interaction format is JSON, and the cloud pushes the evidence chain to the traffic police command platform using the HTTP / 2 protocol, with the data format being an encrypted compressed package (AES256 encryption).

[0036] The handling rules corresponding to the handling instructions include flexible handling of minor violations and rigid closed-loop handling of high-risk violations.

[0037] Among them, minor violations are handled flexibly if it is a first-time illegal parking violation that does not affect traffic, while high-risk violations are handled rigidly if it is drunk driving or speeding by more than 50%.

[0038] The criteria for determining minor violations that do not affect traffic are that the illegally parked vehicle does not occupy the motor vehicle lane or pedestrian crossing and does not cause congestion for vehicles behind it. The criteria for congestion are that there are ≥3 vehicles queuing behind the vehicle and the duration is ≥30 seconds. The criteria for proactive rectification are that the vehicle leaves the illegally parked area and corrects minor violations such as crossing the line within 1 minute after receiving the warning, and the rectification is confirmed by the system in real time. Minor violations can be handled flexibly with automatic reminders for rectification. The criteria for determining non-traffic disruption are that illegally parked vehicles do not occupy motor vehicle lanes or pedestrian crossings and do not cause congestion for vehicles behind them. High-risk violations are handled in a rigid closed-loop system that links roadside access control and toll stations to achieve precise vehicle interception. Meanwhile, the violation source analysis module uses big data analysis to identify the characteristics of high-incidence road sections, time periods, and vehicle types, and outputs violation cause reports. The cause is unreasonable traffic light timing. The big data analysis method involves collecting violation data from the past three months, and the analysis indicators include the distribution of violation time periods, traffic flow on road sections, and the matching degree between traffic light timing duration and traffic flow. When the violation rate of a certain road section at the same time period is higher than the average by 30%, and the matching degree between traffic light timing duration and traffic flow is lower than 60%, it is determined that the traffic light timing is unreasonable. The algorithm self-iteration module automatically optimizes the weight coefficients and scenario correction coefficients of the dynamic game algorithm every quarter based on new violation data and misjudgment cases, without manual intervention. The optimization logic is to reverse the adjustment of the weight coefficients and correction coefficients of the corresponding scenarios for misjudgment cases, so that the subsequent misjudgment rate of similar cases is reduced by ≥80%.

[0039] Example 2: A method for intelligent identification, monitoring, and collaborative management of traffic violations includes the following steps: S1: The multimodal heterogeneous sensing layer synchronously collects multi-dimensional data from airborne, roadbed, vehicle-mounted, and human-based systems. The edge end completes data cleaning and spatiotemporal registration. First, the collected data is denoised and deduplicated. Then, spatiotemporal registration is achieved through timestamp synchronization and GPS positioning calibration, and standardized feature data is output. S2: The edge and cloud collaborative transmission layer transmits standardized data to the dynamic game decision-making layer according to dynamic priority; S3: The dynamic game decision-making layer runs a dynamic game decision-making algorithm for violations and compliance, calculates real-time violation risk values, and determines illegal behaviors, warning levels, or exemption situations. The criteria for exemption situations are: first minor violation, no safety hazards caused, and proactive rectification. After system verification, the penalty can be waived. S4: The cross-domain collaborative response layer initiates four-level collaboration among vehicles, roads, cloud, and people to execute early warning, evidence consolidation, and on-site intervention operations; S5: Intelligent source tracing and optimization layer analyzes the causes of violations, automatically optimizes algorithm parameters and the deployment location of sensing devices. The optimization logic for the deployment location of sensing devices is to adjust the hovering position of airborne drones and the embedding density of roadside sensors according to the location and range of high-incidence areas of violations, so as to ensure that there are no blind spots in the perception coverage of high-incidence areas.

[0040] The specific methods for data cleaning and spatiotemporal registration in step S1 are as follows: Denoising uses wavelet transform denoising algorithm, decomposes into 3 layers, threshold 0.05, deduplication uses dual deduplication based on timestamp and data features, timestamp synchronization uses NTP protocol, GPS positioning calibration uses base station assisted positioning correction to ensure positioning error ≤5m, and the standardized feature data format is JSON, which includes data type, collection time, positioning information, and feature value. In step S3, the criterion for proactive rectification in the exemption case is to correct the illegal behavior within 1 minute after receiving the warning. The system verifies the rectification effect in real time through multimodal perception data. After confirming that there is no continuous violation, it is determined that the proactive rectification is qualified.

[0041] The specific parameters and process of the algorithm's self-iterative module in the intelligent tracing and optimization layer are the same as those of the gradient descent algorithm, with a learning rate of 0.01, a momentum factor of 0.9, and an objective function of [missing information]. The specific process for parameter optimization involves collecting illegal data and misjudgment cases from the past month, inputting the objective function, calculating the parameter gradient, adjusting the algorithm parameters according to the learning rate and momentum factor, and weighting. Scene correction coefficient Level coefficient of illegality and harm Verify the accuracy of the optimized parameters. If the accuracy is ≥95%, the iteration terminates; otherwise, repeat steps 2-3. The iteration termination condition is the same as that of the dynamic game algorithm. It automatically iterates once per quarter, and also supports manual iteration.

[0042] The collaborative working logic of the four modules of the multimodal heterogeneous sensing layer is as follows: the acquisition frequency of the four modules is synchronized: 10Hz for airborne and roadbed sensors, 5Hz for vehicle-mounted sensors, and human-based sensors are uploaded on demand. The specific judgment rule for data cross-verification is that at least two of the three types of sensing data (airborne, roadbed, and vehicle-mounted) detect the same illegal behavior, and the deviation of the data feature value is ≤5%. The deviation is calculated as |data1-data2| / max(data1,data2)×100%, which confirms the illegal behavior. Human-based data is divided into two categories: real-time auxiliary verification and post-event supplementary verification. In real-time auxiliary verification, if the illegal clues uploaded by the human-based terminal are consistent with the spatiotemporal range of the airborne and roadbed sensing data, and the data features match, such as the license plate of the vehicle in the video being consistent with the license plate collected by the roadbed sensor, then the human-based data is included in the data source of dynamic game decision-making. In post-event supplementary verification, if the system determines that there is a suspected illegal behavior but the airborne and roadbed data are insufficient, the human-based data of the corresponding time period can be called for verification.

[0043] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0044] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A monitoring and collaborative management system based on intelligent identification of traffic violations, characterized in that... It includes a multimodal heterogeneous perception layer, an edge and cloud collaborative transmission layer, a dynamic game decision layer, a cross-domain collaborative processing layer, and an intelligent tracing and optimization layer that communicates bidirectionally with the dynamic game decision layer. The multimodal heterogeneous sensing layer includes an airborne dynamic sensing module, a roadbed holographic sensing module, a vehicle-mounted active sensing module, and a human-based supplementary sensing module. The edge and cloud collaborative transmission layer includes a dynamic priority transmission protocol module and a heterogeneous data adaptation module. The dynamic game decision-making layer runs a dynamic game decision-making algorithm for illegality and compliance, which is based on multimodal fusion data. The cross-domain collaborative processing layer establishes a four-level collaborative mechanism involving vehicles, roads, cloud, and people. The intelligent tracing and optimization layer includes an illegal source analysis module and an algorithm self-iteration module. The intelligent tracing and optimization layer and the dynamic game decision layer communicate bidirectionally.

2. The monitoring and collaborative management system based on intelligent identification of traffic violations according to claim 1, characterized in that: The airborne dynamic perception module is a tethered drone swarm equipped with micro-Doppler radar and behavior recognition LiDAR. The tethered drone swarm is equipped with micro-Doppler radar, and the deployment density of the tethered drone swarm is 1 drone per 5 square kilometers, with a hovering height of 50-80m and a sampling frequency of 10Hz, supporting swarm collaborative networking.

3. The monitoring and collaborative management system based on intelligent identification of traffic violations according to claim 1, characterized in that: The roadbed holographic perception module includes a piezoelectric and visual fusion road surface sensor and a violation prediction unit. One set of the piezoelectric and visual fusion road surface sensor is deployed every 200m, and data is collected synchronously with the piezoelectric sensor. The collection frequency is 10Hz.

4. The monitoring and collaborative management system based on intelligent identification of traffic violations according to claim 1, characterized in that: The vehicle-mounted active perception module connects to the vehicle's OBD interface and ADAS system, and adopts the ISO15765-4 interface protocol. The data transmission format of the vehicle-mounted active perception module is the CAN bus standard format. The human-based supplementary perception module is a passenger violation monitoring terminal deployed in buses and taxis, equipped with Android 11 system, a built-in 2-megapixel camera, 8GB storage capacity, supports 4G / 5G wireless transmission, and adopts the TCP / IP protocol. The passenger violation monitoring terminal has a built-in digital signature module and adopts the RSA256 encryption algorithm.

5. A monitoring and collaborative management system based on intelligent identification of traffic violations according to claim 1, characterized in that: The dynamic priority transmission protocol module establishes a mechanism for binding violation level with transmission priority. The heterogeneous data adaptation module completes the cleaning and spatiotemporal registration of radar point cloud, video stream, piezoelectric signal, and OBD data at the edge. The specific threshold and implementation steps of the data cleaning are to use the 3σ criterion to remove outliers. The calculation basis of σ is the standard deviation of the corresponding sensing data in the past 3 months.

6. The monitoring and collaborative management system based on intelligent identification of traffic violations according to claim 1, characterized in that: The dynamic game decision-making algorithm for illegality and compliance, which runs in the dynamic game decision-making layer, is specifically implemented through the following formula: ； ； The The number of perception dimensions participating in the game is 4, corresponding to airborne, roadborne, vehicle-mounted, and human-based. for The dynamic weights of the i-th class of sensing data at time i, and satisfying and The The behavioral feature value collected by the i-th type of sensing device has a value range of [0,1]. This is a scene correction coefficient, with a value range of [0.2, 1.5]. The coefficient for the level of illegality and harm has a value range of [1, 10]. for Real-time violation risk value, ranging from [0,10]. for Real-time dynamic violation threshold.

7. A monitoring and collaborative management system based on intelligent identification of traffic violations according to claim 1, characterized in that: The four-level collaborative mechanism process of vehicles, roads, cloud, and people in the cross-domain collaborative processing layer is as follows: A1: Vehicle-to-vehicle collaboration: Push voice and visual warnings to vehicles after violation prediction or identification. Voice warnings provide real-time voice prompts indicating the type of violation, while visual warnings are pop-up prompts on the in-vehicle display screen. A2: Roadside coordination: Link traffic lights and electronic screens at intersections to trigger temporary light control for high-risk vehicles. Specifically, vehicles that run red lights will trigger a 10-15 second extension of the red light at the intersection, and vehicles that exceed the speed limit by more than 50% will trigger the green light at the intersection ahead to turn red. At the same time, the electronic screen will display the vehicle's license plate number, violation type, and warning information in real time. A3: Cloud Collaboration: Second-level solidification of multi-device integrated illegal evidence chain, which includes multi-dimensional perception data, illegal scene video, time and location information, and is simultaneously pushed to the traffic police command platform and the illegal processing system; A4: Personnel Collaboration: Issue handling instructions containing the location, type, and risk level of the illegal vehicle to nearby patrolling traffic police and traffic assistants to achieve rapid on-site intervention.

8. A monitoring and collaborative management system based on intelligent identification of traffic violations according to claim 7, characterized in that: The handling rules corresponding to the handling instructions include flexible handling of minor violations and rigid closed-loop handling of high-risk violations.

9. A monitoring and collaborative management system based on intelligent identification of traffic violations according to claim 8, characterized in that: The flexible handling of minor violations refers to first-time illegal parking that does not affect traffic, while the rigid closed loop for high-risk violations refers to drunk driving and speeding by more than 50%.

10. A method for intelligent identification, monitoring, and collaborative management of traffic violations, characterized in that, The system implementation based on any one of claims 1-9 includes the following steps: S1: The multimodal heterogeneous sensing layer synchronously collects multi-dimensional data from airborne, roadbed, vehicle-mounted, and human-based systems. The edge end completes data cleaning and spatiotemporal registration. First, the collected data is denoised and deduplicated. Then, spatiotemporal registration is achieved through timestamp synchronization and GPS positioning calibration, and standardized feature data is output. S2: The edge and cloud collaborative transmission layer transmits standardized data to the dynamic game decision-making layer according to dynamic priority; S3: The dynamic game decision-making layer runs a dynamic game decision-making algorithm for violations and compliance, calculates real-time violation risk values, and determines illegal behaviors, warning levels, or exemption situations. The criteria for exemption situations are: first minor violation, no safety hazards caused, and proactive rectification. After system verification, the penalty can be waived. S4: The cross-domain collaborative response layer initiates four-level collaboration among vehicles, roads, cloud, and people to execute early warning, evidence consolidation, and on-site intervention operations; S5: Intelligent source tracing and optimization layer analyzes the causes of violations, automatically optimizes algorithm parameters and the deployment location of sensing devices. The optimization logic for the deployment location of sensing devices is to adjust the hovering position of airborne drones and the embedding density of roadside sensors according to the location and range of high-incidence areas of violations, so as to ensure that there are no blind spots in the perception coverage of high-incidence areas.

Images