Highway traffic data labeling system based on deep learning

Abstract

This invention relates to the field of highway traffic data processing and artificial intelligence annotation technology, specifically disclosing a deep learning-based highway traffic data annotation system, including a data acquisition module, a data preprocessing module, a multimodal feature extraction module, an annotation model training module, a quality assessment and correction module, a data management module, and an application adaptation module. This invention employs multimodal feature fusion and a deep learning model, combining active learning and weakly supervised learning strategies to achieve fully automated annotation of highway traffic data, solving the problems of low efficiency, poor consistency, and insufficient accuracy of traditional annotation methods. Standardized acquisition specifications and multimodal fusion technology adapt to the annotation needs of multi-source heterogeneous highway traffic data, breaking the limitations of single-modal annotation.

Description

Technical Field

[0001] This invention relates to the field of highway traffic data processing and artificial intelligence annotation technology, and in particular to a highway traffic data annotation system based on deep learning. Background Technology

[0002] With the advancement of intelligent transportation and the construction of a strong transportation nation, artificial intelligence technology is increasingly widely used in the entire chain of highway "construction-management-maintenance-operation-service". High-quality labeled datasets are the core foundation for AI model training, inference and verification. Highway traffic data is characterized by multimodality, heterogeneity, and dynamism, encompassing various data types such as video, radar, sensors, meteorology, and text. Traditional annotation methods primarily rely on manual annotation, which presents the following significant problems: 1. Extremely low annotation efficiency: Highway traffic data is massive in scale, and annotating a single data point requires a significant amount of manpower, especially for complex data such as video and radar. The long manual annotation cycle makes it difficult to meet the needs of large-scale dataset construction. 2. Poor annotation consistency: Differences in the professional level and subjective judgment of different annotators lead to inconsistent annotation results for the same batch of data, affecting dataset quality and AI model training effects. 3. Difficulty in multimodal data fusion annotation: Traditional annotation methods struggle to handle the spatiotemporal alignment and semantic association issues of multi-source heterogeneous data, resulting in annotation results that cannot accurately reflect the comprehensive information of traffic scenarios. 4. High labor costs: Large-scale data annotation requires a significant investment of manpower, and the highly repetitive annotation process further increases data construction costs. 5. Limited adaptability: Traditional annotation methods are mostly designed for single traffic scenarios, such as independent road surface detection and traffic flow statistics, making it difficult to adapt to the diverse annotation needs throughout the entire lifecycle of highways.

[0003] In existing technologies, some studies have attempted to improve efficiency using semi-automated annotation methods, but these still suffer from problems such as insufficient annotation accuracy, poor adaptability to multimodal data, and imperfect quality assessment mechanisms. For example, existing semi-automated annotation models often rely on single-modal features and fail to fully integrate complementary information from multi-source data; quality assessment focuses only on annotation accuracy, ignoring key indicators such as data integrity and spatiotemporal alignment; and the lack of a dynamic correction mechanism makes it difficult to correct annotation errors in real time. Therefore, there is an urgent need to propose a fully automated, high-precision, and multi-scenario adaptable highway traffic data annotation method and system to overcome the technical bottlenecks of traditional annotation methods. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a deep learning-based highway traffic data annotation system.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: A deep learning-based highway traffic data annotation system includes: The data acquisition module is used to realize the real-time, batch, and incremental acquisition of multi-source highway traffic data. The data acquisition module includes a high-definition camera, millimeter-wave radar, meteorological sensor, vehicle terminal, and positioning device. The data preprocessing module is used for cleaning and standardizing multi-source data. The data preprocessing module includes data denoising, data completion, and spatiotemporal semantic alignment. The multimodal feature extraction module deploys ViT, PointPillars, LSTM, and BERT models based on the PyTorch framework. It constructs a multimodal feature association map through graph neural networks to achieve feature extraction and cross-modal fusion of multi-source data. The annotation model training module, based on active learning and weakly supervised learning strategies, constructs a weakly supervised annotation model training framework, supporting the initialization training, iterative updates, and parameter optimization of the weakly supervised annotation model; The quality assessment and correction module integrates a multi-dimensional quality assessment model and a domain knowledge base. This module evaluates annotation quality through rule validation, cross-validation, and consistency checks, and dynamically corrects annotation errors using generative adversarial networks. The data management module employs a distributed storage architecture to securely store large-scale labeled data, supporting petabyte-level data storage capacity; it also includes a built-in data indexing unit. The application adaptation module provides customized labeled data output.

[0006] Preferably, the data acquisition module has a built-in device status monitoring unit that monitors the operating status of devices including high-definition cameras, millimeter-wave radar, weather sensors, vehicle terminals, and positioning devices in real time through current sensors and network status monitoring tools. When a device malfunctions, an alarm is automatically triggered, and a backup device is switched to ensure the continuity of data acquisition.

[0007] Preferably, the data denoising uses Gaussian filtering to remove sensor noise; the data imputation uses a linear interpolation algorithm for datasets with missing values ​​≤5%, and a random forest-based interpolation algorithm for datasets with missing values ​​>5%; the spatiotemporal semantic alignment uses Zhang Zhengyou calibration method to calibrate the spatial coordinates of the visual sensor and radar, uses the NTP protocol to achieve time synchronization of multi-source data, and combines the positioning data collected by the positioning device with high-precision map matching to achieve spatial coordinate unification and time synchronization of the data.

[0008] Preferably, the annotation model training module has a built-in model compression unit, which performs lightweight processing on the weakly supervised annotation model through knowledge distillation and model pruning techniques, enabling the weakly supervised annotation model to be deployed on edge computing devices and support real-time annotation tasks. The knowledge distillation adopts a Teacher-Student architecture, where the Teacher model is a pre-trained complete model and the Student model is a lightweight model. The model pruning adopts L1 regularized pruning with a pruning rate of ≤50%.

[0009] Preferably, it also includes a data sharing module, which provides a standardized RESTful API interface and a web-based sharing platform to support online querying, downloading, and accessing of labeled datasets.

[0010] Preferably, the data sharing module integrates a privacy protection mechanism, including data anonymization, access control, and data transmission encryption.

[0011] The beneficial effects of this invention are: This invention employs multimodal feature fusion and a deep learning model, combining active learning and weakly supervised learning strategies to achieve fully automated annotation of highway traffic data, improving annotation efficiency by more than three times compared to traditional manual annotation. Through multidimensional quality assessment and dynamic correction mechanisms, the annotation accuracy is improved to over 95%, with an annotation consistency Kappa coefficient ≥ 0.85 and a spatiotemporal alignment error ≤ 0.5s, solving the problems of low efficiency, poor consistency, and insufficient accuracy of traditional annotation methods. The standardized collection specifications and multimodal fusion technology adapt to the annotation needs of multi-source heterogeneous highway traffic data, breaking the limitations of single-modal annotation. Detailed Implementation

[0012] A deep learning-based highway traffic data annotation system includes: The data acquisition module is used to realize real-time, batch, and incremental acquisition of multi-source highway traffic data. The data acquisition module includes a high-definition camera, millimeter-wave radar, weather sensor, vehicle-mounted terminal, and positioning device. In this embodiment, the high-definition camera is a Hikvision DS-2CD7A26G0-IZS, the millimeter-wave radar is a Continental ARS548, the weather sensor is a Davis Vantage Pro2, the vehicle-mounted terminal supports BeiDou / GPS dual-mode positioning, and the positioning device is a BeiDou / GPS positioning device. It is understood that the data acquisition module has a built-in device status monitoring unit that monitors the operating status of the high-definition camera, millimeter-wave radar, weather sensor, vehicle-mounted terminal, and positioning device in real time through current sensors and network status monitoring tools. When a device malfunctions, an alarm is automatically triggered, and a backup device is switched to ensure the continuity of data acquisition.

[0013] It also includes a data preprocessing module for cleaning and standardizing multi-source data. The data preprocessing module includes data denoising, data completion, and spatiotemporal semantic alignment. The data denoising uses Gaussian filtering to remove sensor noise. For datasets with missing values ​​≤5%, linear interpolation is used for data completion, and random forest-based interpolation is used for datasets with missing values ​​>5%. The spatiotemporal semantic alignment uses Zhang Zhengyou calibration method to calibrate the spatial coordinates of the visual sensor and radar, and uses the NTP protocol to achieve time synchronization of multi-source data. Combined with the positioning data collected by the positioning device and high-precision map matching, the spatial coordinates of the data are unified and the time is synchronized, with a spatiotemporal alignment error ≤0.5s.

[0014] It also includes a multimodal feature extraction module, which implements the deployment of ViT model, PointPillars model, LSTM model and BERT model based on the PyTorch framework, and constructs a multimodal feature association map through graph neural network to realize feature extraction and cross-modal fusion of multi-source data.

[0015] It also includes a labeling model training module, which constructs a weakly supervised labeling model training framework based on active learning and weakly supervised learning strategies. This framework supports the initialization training, iterative updates, and parameter optimization of the weakly supervised labeling model. The labeling model training module has a built-in model compression unit, which performs lightweight processing on the weakly supervised labeling model through knowledge distillation and model pruning techniques. This enables the weakly supervised labeling model to be deployed on edge computing devices and supports real-time labeling tasks. The knowledge distillation adopts a Teacher-Student architecture, where the Teacher model is a pre-trained complete model and the Student model is a lightweight model. The model pruning uses L1 regularized pruning with a pruning rate of ≤50%.

[0016] It also includes a quality assessment and correction module, integrating a multi-dimensional quality assessment model and a domain knowledge base. The domain knowledge base is stored in a MySQL database and includes structured and unstructured data such as traffic regulations, road defect types, traffic incident definitions, and road network topology information. An incremental update mechanism is used, crawling data from traffic industry standards and specifications, academic papers, and engineering cases to ensure the accuracy and timeliness of the knowledge base content. The quality assessment and correction module evaluates annotation quality through rule validation, cross-validation, and consistency checks, and uses generative adversarial networks to dynamically correct annotation errors. The accuracy rate of the corrected annotations is ≥95%.

[0017] It also includes a data management module, which uses a distributed storage architecture to achieve secure storage of large-scale labeled data and supports petabyte-level data storage capacity; it has a built-in data indexing unit, which uses Elasticsearch to build a full-text search index, supports fast retrieval by target category, time range, spatial range and other conditions, and provides data version management function to record the update content and time of each labeled dataset, and supports version backtracking and comparison.

[0018] It also includes a data sharing module, providing a standardized RESTful API interface and a web-based sharing platform to support online querying, downloading, and accessing of labeled datasets; it integrates privacy protection mechanisms, including data anonymization, access control, and data transmission encryption, to ensure the security and compliance of the data sharing process.

[0019] It also includes an application adaptation module that provides customized labeled data output. Specifically, in construction scenarios, it outputs labeled data on road infrastructure and construction process to adapt to the training of AI models for construction quality monitoring; in management scenarios, it outputs labeled data on traffic incident detection and traffic flow statistics to adapt to the training of AI models for traffic control; in maintenance scenarios, it outputs labeled data on road defects and pavement performance to adapt to the training of AI models for intelligent maintenance decision-making; in operation scenarios, it outputs labeled data on vehicle operating status and road network traffic efficiency to adapt to the training of AI models for operation optimization; and in service scenarios, it outputs labeled data on travel demand and service facility status to adapt to the training of AI models for travel service recommendations.

[0020] The annotation system in this embodiment is deployed on a smart transportation cloud platform, and the hardware environment includes: Edge computing node: NVIDIA Jetson Xavier NX, 8GB memory; Cloud server: Intel Xeon Gold 6248 processor, 128GB RAM, NVIDIA Tesla V100 graphics card; Distributed storage cluster: 10-node HDFS cluster, with a total storage capacity of 10PB; The software environment includes: Ubuntu 20.04LTS operating system, PyTorch 1.12.1 deep learning framework, MySQL 8.0 database, Hadoop 3.3.4 distributed storage system, and Elasticsearch 7.17.0 search engine.

[0021] The data collection area is a section of highway with a total length of 100km. 100 sets of high-definition cameras, 50 sets of millimeter-wave radar, 20 sets of meteorological sensors, and 10 sets of vehicle-mounted terminals are deployed to cover typical scenarios such as bridges, tunnels, toll stations, and high slopes. The data collection period is 3 months, from March 2025 to June 2025.

[0022] The implementation steps are as follows: Step S1: Data Acquisition and Preprocessing. The data preprocessing module performs denoising, completion, and spatiotemporal semantic alignment on the data acquired by the data acquisition module. Video data, radar data, sensor data, meteorological data, and GPS positioning data are acquired through deployed acquisition equipment. This includes 100,000 video segments (30 seconds each), 5 million radar data entries, 10 million sensor data entries, 216,000 sets of meteorological data, and 3 million GPS positioning data entries. The data preprocessing module performs denoising, completion, and spatiotemporal semantic alignment, removing approximately 50,000 noisy sensor data entries and filling in missing data. The data qualification rate after spatiotemporal alignment reaches 99.2%. Step S2: Multimodal feature extraction. The multimodal feature extraction module extracts and fuses features from the data collected and preprocessed in step S1. Visual features are extracted from frequency data using the ViT model, distance features are extracted from radar data using the PointPillars model, temporal features are extracted from sensor data using the LSTM model, and semantic features are extracted from text data using the BERT model. Cross-modal fusion is then achieved using the GNN model. The fused feature vector has a dimension of 1024, and the feature extraction rate is 1200 samples / minute. Step S3: Model Training and Iteration. 10% of the preprocessed data (approximately 1.8 million records) was manually labeled to construct an initial labeled sample set. Based on this initial labeled sample set, the weakly supervised labeling model was fine-tuned. The pre-trained model used was the ViT-B / 16 model trained on the nuScenes dataset. After fine-tuning, the initial labeling accuracy of the model was 88%. This model was then used to predict the remaining 90% of unlabeled data (approximately 8.2 million records). Samples with a confidence score ≥ 0.8 were selected as valid labels (approximately 7 million records), and samples with a confidence score < 0.8 were manually supplemented with additional labels (approximately 1.2 million records). The supplemented labeled samples were added to the initial labeled sample set, the model parameters were updated, and the unlabeled data was predicted again. After three iterations, all data was labeled, with a total of 3 million manually labeled records, accounting for 16.7% of the total data. Step S4, Quality Assessment and Correction: The annotation results are evaluated using a multi-dimensional quality assessment model, including assessment of the completeness missing rate, accuracy, consistency Kappa coefficient, and spatiotemporal alignment error. Non-compliant annotated data is validated using domain knowledge base rules and dynamically corrected using a GAN model. The corresponding data showed a completeness missing rate of 2.1%, an accuracy assessment of 96.8%, a consistency Kappa coefficient of 0.87, and a spatiotemporal alignment error of 0.3s. Approximately 320,000 non-compliant annotations were identified. After validation using domain knowledge base rules and dynamic correction using a GAN model, the annotation accuracy improved to 97.5%. Step S5: Generation and application of structured datasets. The corrected labeled data is structured into traffic flow datasets, road defect datasets, traffic incident detection datasets, and road condition detection and evaluation datasets. The traffic flow dataset contains 300 records, the road defect dataset contains 1.5 million records, the traffic incident detection dataset contains 800,000 records, and the road condition detection and evaluation dataset contains 2 million records. The data is converted to TFRecord format and indexed. A standardized interface is provided to the AI ​​model training platform of the provincial expressway administration through the data sharing module to support the training of AI models for traffic flow prediction, road defect identification, and intelligent maintenance decision-making.

[0023] In this embodiment, multimodal traffic data annotation for a highway section over three months was completed, generating a total of 9.3 million structured annotation data. The annotation efficiency was improved by 3.2 times compared to traditional manual annotation, the manual annotation cost was reduced by 65%, the annotation accuracy reached 97.5%, the spatiotemporal alignment error was 0.3s, and the consistency Kappa coefficient was 0.87. This meets the high-quality data requirements for training traffic AI models. The annotated dataset was applied to the training of traffic flow prediction models, road defect identification models, and intelligent maintenance decision-making models on a smart highway AI platform. The model prediction accuracy was improved by 18%, the road defect identification false negative rate was reduced by 25%, and the maintenance decision-making cost was reduced by 30%.

Claims

1. A deep learning-based highway traffic data annotation system, characterized in that, include: The data acquisition module is used to realize the real-time, batch, and incremental acquisition of multi-source highway traffic data. The data acquisition module includes a high-definition camera, millimeter-wave radar, meteorological sensor, vehicle terminal, and positioning device. The data preprocessing module is used for cleaning and standardizing multi-source data. The data preprocessing module includes data denoising, data completion, and spatiotemporal semantic alignment. The multimodal feature extraction module deploys ViT, PointPillars, LSTM, and BERT models based on the PyTorch framework. It constructs a multimodal feature association map through graph neural networks to achieve feature extraction and cross-modal fusion of multi-source data. The annotation model training module, based on active learning and weakly supervised learning strategies, constructs a weakly supervised annotation model training framework, supporting the initialization training, iterative updates, and parameter optimization of the weakly supervised annotation model; The quality assessment and correction module integrates a multi-dimensional quality assessment model and a domain knowledge base. This module evaluates annotation quality through rule validation, cross-validation, and consistency checks, and dynamically corrects annotation errors using generative adversarial networks. The data management module employs a distributed storage architecture to securely store large-scale labeled data, supporting petabyte-level data storage capacity; it also includes a built-in data indexing unit. The application adaptation module provides customized labeled data output.

2. The deep learning-based highway traffic data annotation system according to claim 1, characterized in that, The data acquisition module has a built-in device status monitoring unit that monitors the operating status of devices including high-definition cameras, millimeter-wave radar, weather sensors, vehicle terminals, and positioning devices in real time through current sensors and network status monitoring tools. When a device malfunctions, an alarm is automatically triggered and a backup device is switched to ensure the continuity of data acquisition.

3. The deep learning-based highway traffic data annotation system according to claim 1, characterized in that, The data denoising uses Gaussian filtering to remove sensor noise. For data imputation, linear interpolation is used to fill in missing values ​​for datasets with a missing value ratio of ≤5%, and random forest-based interpolation is used for datasets with a missing value ratio >5%. The spatiotemporal semantic alignment uses Zhang Zhengyou calibration method to calibrate the spatial coordinates of the visual sensor and radar, and uses the NTP protocol to achieve time synchronization of multi-source data. By combining the positioning data collected by the positioning device with high-precision map matching, the spatial coordinates of the data are unified and the time is synchronized.

4. The deep learning-based highway traffic data annotation system according to claim 1, characterized in that, The annotation model training module has a built-in model compression unit. It uses knowledge distillation and model pruning techniques to lightweight the weakly supervised annotation model, enabling it to be deployed on edge computing devices and supporting real-time annotation tasks. The knowledge distillation adopts a Teacher-Student architecture, where the Teacher model is a pre-trained complete model and the Student model is a lightweight model. The model pruning uses L1 regularization pruning with a pruning rate of ≤50%.

5. The deep learning-based highway traffic data annotation system according to claim 1, characterized in that, It also includes a data sharing module, which provides a standardized RESTful API interface and a web-based sharing platform, supporting online querying, downloading, and access to labeled datasets.

6. The deep learning-based highway traffic data annotation system according to claim 5, characterized in that, The data sharing module integrates privacy protection mechanisms, including data anonymization, access control, and data transmission encryption.