A big data processing system based on vehicle-road collaboration

Abstract

The present invention relates to a big data processing system based on vehicle-road collaboration, comprising: a data base layer for collecting vehicle-side data, road-side data, and network-side data, and performing mining, generalization, governance, annotation, and storage processing on the collected data; an algorithm support layer for providing an algorithm framework and model for the data base layer, feeding back the intermediate results and output results of the algorithm framework and model during data processing to the data base layer, and sending the output results to the data application layer; a data application layer for receiving the output results of the algorithm support layer, planning the vehicle's driving trajectory based on the output results, and performing intelligent vehicle-side data visualization on the output results. Compared with the existing technology, the present invention can receive vehicle-side and road-side data, and effectively store and process the data, including data enhancement and annotation.

Description

Technical Field

[0001] The present invention relates to the field of vehicle-road collaboration technology, and in particular to a big data processing system based on vehicle-road collaboration. Background Art

[0002] In vehicle-side data collection, different types of sensors need to be used and integrated. The system generally collects data through a combination of the Global Positioning System (GPS) and the Inertial Measurement Unit (IMU), LiDAR, cameras and other sensor devices to perceive the surrounding environment. Among them, the GPS and IMU combined sensing system is mainly used for the positioning of autonomous vehicles; the data collected by LiDAR is used for drawing high-precision maps, positioning and obstacle avoidance; the videos and images collected by the camera are widely used for target detection, recognition and tracking. In terms of the use of autonomous driving datasets, the most widely used public datasets are the KITTI dataset, the CityScapes dataset and the Oxford RobotCar dataset, which can be applied to tasks such as target detection, 3D reconstruction, and path planning, providing real traffic data for autonomous driving algorithms.

[0003] In the collection of road-side data, communication with the road side is mainly carried out through the vehicle-side device OBU (OnBoard Unit). The OBU also needs to exchange data with the vehicle-side ADAS domain controller. The road-side device RSU (RoadSide Unit) is mainly responsible for communicating with the vehicle-side OBU. It also includes perception devices such as cameras, millimeter-wave radars, lidars, and high-performance computing units required for data processing.

[0004] During the driving process of an autonomous vehicle, a large amount of data is generated on both the vehicle side and the road side. However, the existing technology lacks a big data processing system to effectively store and process this data, including automatic labeling and data enhancement of the big data. Summary of the Invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned existing technologies and provide a big data processing system based on vehicle-road collaboration. The present invention can receive vehicle-side and road-side data, and effectively store and process the data, including data enhancement and labeling.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] The present invention provides a big data processing system based on vehicle-road collaboration, comprising:

[0008] Data base layer: used to collect vehicle-side data, road-side data, and network-side data, and to mine, generalize, manage, label, and store the collected data;

[0009] Algorithm support layer: used to provide algorithm framework and model for the data base layer, feed back the intermediate results and output results of the algorithm framework and model in the process of processing data to the data base layer, and send the output results to the data application layer;

[0010] Data application layer: used to receive the output results of the algorithm support layer, plan the vehicle's driving trajectory based on the output results, and visualize the output results on the intelligent vehicle side.

[0011] Preferably, the data base layer includes:

[0012] Data collection module: used to set and collect vehicle-side data, road-side data, and network-side data according to collection strategies and collection parameters; update collection strategies and collection parameters based on the output results of the algorithm support layer;

[0013] Data augmentation module: This module uses the data augmentation method based on the combination of autoencoders and generative adversarial networks provided by the algorithm support layer to obtain a low-dimensional vector representation of the image data. The other half of the network structure is used for decoding to generate image data reconstructed based on the low-dimensional vector representation.

[0014] Data annotation module: used to set annotation rules and annotation tools, annotate images obtained by extracting frames of video data in one or more sessions according to the annotation rules, and update the annotation rules and annotation tools based on the information fed back by the algorithm support layer;

[0015] Data storage module: used to clean, integrate and serialize the data collected by the data acquisition module and store it in the data warehouse.

[0016] Preferably, the data collected by the data collection module is cleaned and integrated through the client / server mechanism of EMQTT.

[0017] Preferably, on the Web platform, the integrated data is converted into Key-Value serialized data in Json format.

[0018] Preferably, the serialized data is stored in a data warehouse according to a classification table in a MySQL database.

[0019] Preferably, the data base layer further includes:

[0020] Data mining module: used to process the data collected by the data acquisition module according to the algorithm framework and model provided by the algorithm support layer, output the processing results corresponding to the algorithm framework and model, and optimize the algorithm model, adjust parameters and select features based on the information sensed by the algorithm support layer;

[0021] Data generalization module and data governance module: used to perform attribute-oriented induction of data and manage data usage based on the output results of the data mining module.

[0022] Preferably, the algorithm support layer includes:

[0023] Model building tools: used to build the algorithm framework and model structure as well as the loss function corresponding to each framework and model;

[0024] Standardized model library: used to store built and trained algorithm frameworks and models;

[0025] Open source algorithm framework: used to store open source algorithm framework;

[0026] Model management module: used to receive data stored in the data storage module, train the constructed algorithm framework and model, and feed back the intermediate results and output results of the algorithm framework and model processing data to the data acquisition module, data mining module, data annotation module and data application layer.

[0027] Preferably, the vehicle-side data includes data collected by cameras, lidars, millimeter-wave radars, navigation locators and on-board buses installed on the vehicle; the road-side data includes data collected by cameras and lidars installed on the roadside, and also includes data interacting with the vehicle-side through V2X; the network-side data includes data interacting with the vehicle-side and road-side through V2X, and data interacting with the Internet.

[0028] Preferably, the data application layer receives and displays image data collected by cameras installed on the vehicle and cameras installed on the roadside, receives the output results of the algorithm support layer, and displays message data.

[0029] Preferably, the message data and the image data are displayed respectively based on the Nginx server and the Flask web page framework.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] This invention provides a big data processing system based on vehicle-road collaboration. By constructing a data foundation layer and an algorithm support layer, it can annotate vehicle-roadside data and images, and automatically audit the annotation quality to ensure data quality. Automatically annotated data is imported into the algorithm support layer to enable model training, evaluation, and prediction, seamlessly integrating data analysis and prediction applications, and reducing the difficulty of lifecycle management of machine learning models. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A schematic diagram of the structure of a big data processing system based on vehicle-road collaboration provided in this embodiment. DETAILED DESCRIPTION

[0033] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0034] refer to Figure 1 As shown, this embodiment provides a big data processing system based on vehicle-road collaboration, including:

[0035] Data base layer: used to collect vehicle-side data, road-side data and network-side data, and to mine, generalize, govern, enhance, label and store the collected data.

[0036] As an optional implementation method, the vehicle-side data includes data collected by cameras, lidar, millimeter-wave radar, navigation locator and on-board bus installed on the vehicle; the road-side data includes data collected by cameras and lidar installed on the roadside, and also includes data interacting with the vehicle-side through V2X; the network-side data includes data interacting with the vehicle-side and road-side through V2X, and data interacting with the Internet.

[0037] Algorithm support layer: used to provide algorithm framework and model for the data base layer, and feed back the intermediate results and output results of the algorithm framework and model in the process of processing data to the data base layer, and send the output results to the data application layer.

[0038] Data application layer: used to receive the output results of the algorithm support layer, plan the vehicle's driving trajectory based on the output results, and visualize the output results on the intelligent vehicle side.

[0039] As an optional implementation, the data base layer collects, processes, and applies traffic information in a collaborative vehicle-infrastructure environment, enabling the annotation, training, identification, and inference of autonomous driving traffic datasets. This enables integrated vehicle-infrastructure collaborative operations management, holographic intersections, traffic flow, and accident replay. The acquired real-world vehicle-infrastructure data can simulate rare real-world situations and reduce the need for large-scale data collection. Data generalization and enhancement of small samples enable high-quality model training with small samples, reducing reliance on large-scale data and improving model generalization and robustness.

[0040] The data base layer includes:

[0041] Data collection module: used to set and collect vehicle-side data, road-side data and network-side data according to collection strategies and collection parameters;

[0042] Data mining module: used to process the data collected by the data acquisition module according to the algorithm framework and model provided by the algorithm support layer, output the processing results corresponding to the algorithm framework and model, and optimize the algorithm model, adjust parameters and select features based on the information sensed by the algorithm support layer;

[0043] Data generalization module and data governance module: used to perform attribute-oriented induction of data and manage data usage based on the output results of the data mining module.

[0044] Data enhancement module: It is used to obtain a low-dimensional vector representation of the image data according to the data enhancement method based on the combination of autoencoder and generative adversarial network provided by the algorithm support layer, and use the other half of the network structure for decoding to generate image data reconstructed according to the low-dimensional vector representation.

[0045] The amount of raw data uploaded determines, to a certain extent, the size of the dataset available to users. Since data size significantly impacts model performance, it's necessary to provide an incremental generation function for image data to increase the size of training data. The data augmentation module enables the expansion of the capacity of both training and noise data samples, enabling the use of data augmentation techniques to improve the generalization and robustness of neural networks. Based on a method combining variational autoencoders and generative adversarial networks, conditional adversarial generation is employed for high-resolution images. This method is applied to traffic participant target recognition in vehicle-road collaboration, resulting in a better similarity metric than element-based VAEs.

[0046] Data annotation module: used to set annotation rules and annotation tools, annotate images obtained by extracting frames of video data in one or more sessions according to the annotation rules, and update the annotation rules and annotation tools based on the information feedback from the algorithm support layer.

[0047] As an optional implementation, the video data is frame-extracted to obtain the image. There are two strategies for frame extraction: one is to extract frames at a specified time interval, and the other is to include the video data and its corresponding GPS file when uploading the data.

[0048] Data storage module: used to clean, integrate and serialize the data collected by the data acquisition module and store it in the data warehouse.

[0049] As an optional implementation method, the data collected by the data acquisition module is cleaned and integrated through the client / server (C / S) mechanism of EMQTT. On the Web platform, the integrated data is converted into Key-Value serialized data in Json format. The serialized data is then stored in the data warehouse according to the classification table in the MySQL database for subsequent data analysis and processing.

[0050] As an optional implementation, the algorithm support layer supports statistical analysis and mining using multiple programming languages ​​and runs within a distributed computing framework. For example, it leverages R's rich statistical analysis libraries and extensive graphical visualization methods to analyze data stored in Hadoop. Users query SQL and receive result data. By building this support platform, a miniaturized algorithm model based on vehicle-road collaboration and cloud-edge collaboration can be established, improving the operational and deployment efficiency of autonomous driving models.

[0051] The algorithm support layer includes:

[0052] Model building tools: used to build the algorithm framework and model structure as well as the loss function corresponding to each framework and model;

[0053] Constructing a multi-scale discriminator: In the discriminator, a convolutional neural network is used to extract image features. By downsampling the original image to varying degrees, input images of different sizes are constructed, forming an input image pyramid. These are then fed into the discriminator network for discrimination. This results in output feature maps with receptive fields of varying sizes, balancing global image information with local details.

[0054] Construct loss function: extract the feature maps used by the discriminator to distinguish the generated image and the real image respectively, perform feature matching on the generated feature map and the real feature map of the same layer, and use the L1 norm to measure the feature matching loss.

[0055] Standardized model library: used to store built and trained algorithm frameworks and models;

[0056] Open source algorithm framework: used to store open source algorithm framework;

[0057] In this embodiment, the stored open-source algorithm framework includes: Generative Adversarial Network (GAN): This transforms the input noise data into something close to real data. The generative model G and the discriminative model D compete with each other, and both models are strengthened through competition.

[0058] Model management module: used to receive data stored in the data storage module, train the constructed algorithm framework and model, and feed back the intermediate results and output results of the algorithm framework and model processing data to the data acquisition module, data mining module, data annotation module and data application layer.

[0059] As an optional implementation, the data application layer's display of intelligent connected vehicle data is broadly divided into two parts: one for displaying real-time uploaded message data, and the other for displaying real-time uploaded image data, achieving a remote video surveillance effect. The data visualization subsystem, designed primarily based on the Nginx server and the Flask web design framework, visualizes both real-time message data and real-time image data. Therefore, the cloud-based data visualization subsystem utilizes two different architectures based on the Flask web framework to process the two types of intelligent vehicle data.

[0060] In summary, the present invention is different from traditional data processing systems. By constructing a data base layer and an algorithm support layer, it can annotate vehicle-side and road-side data and images, and automatically review the annotation quality to ensure data quality. The automatically annotated data is imported into the algorithm support layer, and the model training, evaluation and prediction are realized through a visual operation interface, seamlessly connecting data analysis and prediction applications, and reducing the difficulty of life cycle management of machine learning models. The platform also has open services, which can encapsulate the capabilities of general models and directly provide services in the form of external APIs and core models. The platform can realize industry data capability customization, accelerate industry implementation, and has commercial value.

[0061] The above describes in detail the preferred embodiments of the present invention. It should be understood that those skilled in the art can make numerous modifications and variations based on the concepts of the present invention without inventive effort. Therefore, any technical solutions that can be derived by those skilled in the art through logical analysis, reasoning, or limited experimentation based on the concepts of the present invention and the prior art should be within the scope of protection defined by the claims.

Claims

1. A big data processing system based on vehicle-road collaboration, characterized in that: include: Data base layer: used to collect vehicle-side data, road-side data, and network-side data, and to mine, generalize, manage, label, and store the collected data; Algorithm support layer: used to provide algorithm framework and model for the data base layer, feed back the intermediate results and output results of the algorithm framework and model in the process of processing data to the data base layer, and send the output results to the data application layer; Data application layer: used to receive the output results of the algorithm support layer, plan the vehicle's driving trajectory based on the output results, and visualize the output results on the intelligent vehicle side; The data base layer includes: Data collection module: used to set and collect vehicle-side data, road-side data, and network-side data according to collection strategies and collection parameters; update collection strategies and collection parameters based on the output results of the algorithm support layer; Data augmentation module: This module uses the data augmentation method based on the combination of autoencoders and generative adversarial networks provided by the algorithm support layer to obtain a low-dimensional vector representation of the image data. The other half of the network structure is used for decoding to generate image data reconstructed based on the low-dimensional vector representation. Data annotation module: used to set annotation rules and annotation tools, annotate images obtained by extracting frames of video data in one or more sessions according to the annotation rules, and update the annotation rules and annotation tools based on the information fed back by the algorithm support layer; Data storage module: used to clean, integrate and serialize the data collected by the data acquisition module and store it in the data warehouse; Clean and integrate the data collected by the data acquisition module through the client / server mechanism of EMQTT; The data base layer also includes: Data mining module: used to process the data collected by the data acquisition module according to the algorithm framework and model provided by the algorithm support layer, output the processing results corresponding to the algorithm framework and model, and optimize the algorithm model, adjust parameters and select features based on the information sensed by the algorithm support layer; Data generalization module and data governance module: used to perform attribute-oriented induction and data usage management based on the output results of the data mining module respectively; The algorithm support layer includes: Model building tools: used to build the algorithm framework and model structure as well as the loss function corresponding to each framework and model; Standardized model library: used to store built and trained algorithm frameworks and models; Open source algorithm framework: used to store open source algorithm framework; Model management module: used to receive data stored in the data storage module, train the constructed algorithm framework and model, and feed back the intermediate results and output results of the algorithm framework and model processing data to the data acquisition module, data mining module, data annotation module and data application layer.

2. The big data processing system based on vehicle-road collaboration according to claim 1 is characterized in that: On the Web platform, the integrated data is converted into Key-Value serialized data in Json format.

3. The big data processing system based on vehicle-road collaboration according to claim 1 is characterized in that: The serialized data is stored in the data warehouse according to the classification table in the MySQL database.

4. The big data processing system based on vehicle-road collaboration according to claim 1 is characterized in that: The vehicle-side data includes data collected by cameras, lidar, millimeter-wave radar, navigation locator and on-board bus installed on the vehicle; the road-side data includes data collected by cameras and lidar installed on the roadside, and also includes data interacting with the vehicle-side through V2X; the network-side data includes data interacting with the vehicle-side and road-side through V2X, and data interacting with the Internet.

5. The big data processing system based on vehicle-road collaboration according to claim 4 is characterized in that: The data application layer receives and displays image data collected by cameras installed on the vehicle and cameras installed on the roadside, and receives the output results of the algorithm support layer to display message data.

6. The big data processing system based on vehicle-road collaboration according to claim 5 is characterized in that: Message data and image data are displayed based on Nginx server and Flask web page framework respectively.

Images