Data processing method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a data processing method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: collecting driving data through various sensors, and adding a timestamp to the driving data collected by each sensor; aligning the driving data collected by each sensor according to the timestamps to obtain a plurality of groups of aligned data, each group of aligned data corresponding to one timestamp; for each group of alignment data, performing data fusion on the alignment data to obtain intelligent driving data, and storing the intelligent driving data to a cloud; when model training needs to be carried out, data needing to be labeled are obtained from all intelligent driving data stored in the cloud end, data to be labeled are obtained, according to the requirement of a labeling task, the data to be labeled are automatically labeled, labeled data are obtained, and the labeled data are used for model training. By adopting the method, the data can be collected, processed and used to form an efficient closed loop.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and particularly to a data processing method, apparatus, computer device, computer-readable storage medium, and computer program product. Background Art

[0002] In the current era of rapid technological development, the field of intelligent driving is advancing at an astonishing speed. With the increasing maturity and widespread application of intelligent driving technology, the demand for data in this field has shown an explosive growth trend. For an intelligent driving system to achieve accurate environmental perception, safe decision-making and planning, and stable vehicle control, a vast amount of data is an indispensable support. From real-time road condition information during vehicle driving, to dynamic data of surrounding vehicles and pedestrians, to various state parameters of the vehicle itself, all this data needs to be accurately received, quickly transmitted back, and then analyzed and processed in a timely manner. And for the reception, transmission back, and use of this series of data, there is an urgent need for a platform with sufficient bandwidth that can efficiently interact with a vast amount of data, thereby ensuring the stable and reliable operation of the intelligent driving system and providing a safer and more convenient travel experience for people.

[0003] With the increasing complexity of AI algorithms, the traditional software development model is difficult to support the iteration efficiency, and the current training, simulation, and other links of vehicle manufacturers are separated. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide a data processing method, apparatus, computer device, computer-readable storage medium, and computer program product that can form an efficient closed-loop for data collection, processing, and use.

[0005] In a first aspect, this application provides a data processing method, including:

[0006] Collect driving data through multiple sensors, and add timestamps to the driving data collected by each sensor;

[0007] Align the driving data collected by each sensor according to the timestamps to obtain multiple groups of aligned data, and each group of aligned data corresponds to a timestamp;

[0008] For each group of aligned data, perform data fusion on the aligned data to obtain intelligent driving data, and store the intelligent driving data in the cloud;

[0009] When model training is required, obtain the data to be labeled from all the intelligent driving data stored in the cloud to obtain the data to be labeled, perform automated labeling on the data to be labeled according to the requirements of the labeling task to obtain labeled data, and use the labeled data for model training.

[0010] In one embodiment, before collecting driving data through a variety of sensors, it further includes:

[0011] After the vehicle starts, detect and initialize a variety of sensors on the vehicle end; after the initialization is completed, set the sampling frequency and the format of the collected data for the variety of sensors, and calibrate the variety of sensors.

[0012] In one embodiment, the variety of sensors include vehicle-end sensors and environmental sensors; the vehicle-end sensors include high-definition cameras, lidar, inertial measurement units (IMUs), and global positioning systems (GPSs); the collecting driving data through a variety of sensors includes:

[0013] Collect vehicle-end data through the high-definition camera, the lidar, the inertial measurement unit (IMU), and the global positioning system (GPS); collect weather data and traffic data through the environmental sensors.

[0014] In one embodiment, before performing data fusion on the aligned data, it further includes:

[0015] Perform preprocessing on each group of aligned data respectively to obtain processed data of a variety of sensors; the preprocessing includes at least one of noise reduction, format conversion, and desensitization processing.

[0016] In one embodiment, the cloud includes: a raw data layer, an annotated data layer, and an algorithm analysis structure layer;

[0017] Storing the intelligent driving data in the cloud includes: storing the intelligent driving data in the raw data layer of the cloud; after obtaining the annotated data, it further includes: storing the annotated data in the annotated data layer of the cloud; using the annotated data for model training includes: obtaining the annotated data from the annotated data layer of the cloud, using the annotated data for model training, and storing the obtained analysis results in the algorithm analysis result layer of the cloud.

[0018] In one embodiment, the automatic annotation of the data to be annotated includes:

[0019] Automatically annotate the data to be annotated by using 2D annotation, 3D annotation, and joint annotation methods.

[0020] In a second aspect, the present application further provides a data processing device, including:

[0021] A collection module, configured to collect driving data through a variety of sensors and add timestamps to the data collected by each sensor;

[0022] An alignment module, configured to align the data collected by each type of sensor according to timestamps to obtain multiple sets of aligned data, where each set of aligned data corresponds to a timestamp;

[0023] A fusion module, configured to perform data fusion on the aligned data for each set of aligned data to obtain intelligent driving data, and store the intelligent driving data in the cloud;

[0024] A labeling module, configured to, when model training is required, obtain the data to be labeled from all the intelligent driving data stored in the cloud to obtain the data to be labeled, perform automated labeling on the data to be labeled according to the requirements of the labeling task to obtain labeled data, and use the labeled data for model training.

[0025] In a third aspect, the present application further provides a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0026] Collect driving data through multiple sensors, and add timestamps to the driving data collected by each type of sensor;

[0027] Align the driving data collected by each type of sensor according to timestamps to obtain multiple sets of aligned data, where each set of aligned data corresponds to a timestamp;

[0028] For each set of aligned data, perform data fusion on the aligned data to obtain intelligent driving data, and store the intelligent driving data in the cloud;

[0029] When model training is required, obtain the data to be labeled from all the intelligent driving data stored in the cloud to obtain the data to be labeled, perform automated labeling on the data to be labeled according to the requirements of the labeling task to obtain labeled data, and use the labeled data for model training.

[0030] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0031] Collect driving data through multiple sensors, and add timestamps to the driving data collected by each type of sensor;

[0032] Align the driving data collected by each type of sensor according to timestamps to obtain multiple sets of aligned data, where each set of aligned data corresponds to a timestamp;

[0033] For each set of aligned data, perform data fusion on the aligned data to obtain intelligent driving data, and store the intelligent driving data in the cloud;

[0034] When model training is required, obtain the data to be labeled from all the intelligent driving data stored in the cloud to get the data to be labeled. According to the requirements of the labeling task, automatically label the data to be labeled to obtain labeled data, and use the labeled data for model training.

[0035] In a fifth aspect, the present application also provides a computer program product, including a computer program, which when executed by a processor implements the following steps:

[0036] Collect driving data through multiple sensors and add timestamps to the driving data collected by each sensor;

[0037] Align the driving data collected by each sensor according to the timestamps to obtain multiple sets of aligned data, and each set of aligned data corresponds to a timestamp;

[0038] For each set of aligned data, perform data fusion on the aligned data to obtain intelligent driving data, and store the intelligent driving data in the cloud;

[0039] When model training is required, obtain the data to be labeled from all the intelligent driving data stored in the cloud to get the data to be labeled. According to the requirements of the labeling task, automatically label the data to be labeled to obtain labeled data, and use the labeled data for model training.

[0040] The above data processing method, device, computer device, computer-readable storage medium, and computer program product collect driving data through multiple sensors, add timestamps to the driving data collected by each sensor; align the driving data collected by each sensor according to the timestamps to obtain multiple sets of aligned data, and each set of aligned data corresponds to a timestamp; for each set of aligned data, perform data fusion on the aligned data to obtain intelligent driving data, and store the intelligent driving data in the cloud; when model training is required, obtain the data to be labeled from all the intelligent driving data stored in the cloud to get the data to be labeled. According to the requirements of the labeling task, automatically label the data to be labeled to obtain labeled data, and use the labeled data for model training. Through the above method, it is possible to form an efficient closed-loop for data collection, processing, and use. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for the description of the embodiments of the present application or related technologies. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0042] Figure 1 It is a schematic flowchart of a data processing method in an embodiment;

[0043] Figure 2 It is a detailed flowchart of a data processing method in an embodiment;

[0044] Figure 3 It is a structural block diagram of a data processing device in an embodiment;

[0045] Figure 4 It is an internal structure diagram of a computer device in an embodiment. Specific embodiments

[0046] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0047] In one embodiment, as Figure 1 shown, a data processing method is provided. In this embodiment, it is exemplified that the method is applied to a terminal. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0048] Step 102, collect driving data through multiple sensors, and add time stamps to the driving data collected by each sensor.

[0049] Among them, to ensure that the data of the sensors can be aligned, it is first necessary to add accurate time stamps to the data of the sensors.

[0050] Exemplarily, collect vehicle-end data and vehicle-external environment data through multiple sensors, and add time stamps to the driving data collected by each sensor.

[0051] Step 104, perform alignment processing on the driving data collected by each sensor according to the time stamps to obtain multiple groups of aligned data, and each group of aligned data corresponds to a time stamp.

[0052] Among them, based on the time stamps, use the hardware synchronization mechanism at the vehicle end and use algorithms to complete the data stream alignment of multiple sensors.

[0053] Exemplarily, classify the driving data collected by each sensor according to the time stamps, and the driving data with the same time stamp is divided into a group to obtain multiple groups of aligned data.

[0054] Step 106: For each set of aligned data, perform data fusion on the aligned data to obtain intelligent driving data, and store the intelligent driving data in the cloud.

[0055] Optionally, compress the intelligent driving data first and then store it in the cloud.

[0056] Optionally, use the Kalman filter algorithm to perform data fusion on the aligned data.

[0057] Exemplarily, for each set of aligned data, use the Kalman filter algorithm to perform data fusion on the aligned data to obtain intelligent driving data, compress the intelligent driving data, and then store the compressed intelligent driving data in the cloud.

[0058] Step 108: When model training is required, obtain the data to be labeled from all the intelligent driving data stored in the cloud to get the data to be labeled. According to the requirements of the labeling task, perform automated labeling on the data to be labeled to obtain labeled data, and use the labeled data for model training.

[0059] Optionally, in addition to model training, there are also virtual simulation tasks.

[0060] The above data processing method, device, computer device, computer-readable storage medium, and computer program product collect driving data through multiple sensors, add timestamps to the driving data collected by each sensor; perform alignment processing on the driving data collected by each sensor according to the timestamps to obtain multiple sets of aligned data, and each set of aligned data corresponds to a timestamp; for each set of aligned data, perform data fusion on the aligned data to obtain intelligent driving data, and store the intelligent driving data in the cloud; when model training is required, obtain the data to be labeled from all the intelligent driving data stored in the cloud to get the data to be labeled. According to the requirements of the labeling task, perform automated labeling on the data to be labeled to obtain labeled data, and use the labeled data for model training. Through the above method, it is possible to form an efficient closed-loop for data collection, processing, and use.

[0061] In an exemplary embodiment, before collecting driving data through multiple sensors, it further includes:

[0062] After the vehicle is started, detect and initialize multiple sensors on the vehicle end; after the initialization is completed, set the sampling frequency and the format of the collected data for the multiple sensors, and calibrate the multiple sensors.

[0063] Exemplarily, after the vehicle is started, detect and initialize multiple sensors on the vehicle end; after the initialization is completed, set the sampling frequency and the format of the collected data for the multiple sensors according to the type and operation scenario of the vehicle, and calibrate the multiple sensors.

[0064] In this embodiment, by calibrating the sensors, the errors in the sensor manufacturing process can be eliminated, ensuring the accuracy and subsequent usability of the data.

[0065] In an exemplary embodiment, the multiple sensors include vehicle-end sensors and environmental sensors; the vehicle-end sensors include high-definition cameras, lidar, inertial measurement units (IMUs), and global positioning systems (GPSs); the collection of driving data by the multiple sensors includes:

[0066] Collecting vehicle-end data through the high-definition camera, the lidar, the inertial measurement unit (IMU), and the global positioning system (GPS); collecting weather data and traffic data through the environmental sensors.

[0067] Among them, the sources of the current data include two parts: vehicle-end sensor data collection and external environment data collection. The vehicle-end sensors are mainly high-definition cameras, lidar, IMUs, GPSs, etc., and the external environment data are mainly maps, weather, and traffic information.

[0068] Exemplarily, vehicle-end data is collected through a high-definition camera, lidar, inertial measurement unit (IMU), and global positioning system (GPS); weather data and traffic data are collected through environmental sensors.

[0069] In this embodiment, by collecting vehicle-end data and vehicle external environment data through multiple sensors, richer data can be obtained.

[0070] In an exemplary embodiment, before performing data fusion on the aligned data, it further includes:

[0071] Performing preprocessing on each group of aligned data to obtain processed multiple sensor data; the preprocessing includes at least one of noise reduction, format conversion, and desensitization processing.

[0072] Among them, due to limited vehicle-end computing power, only a simple filtering algorithm is used to complete data noise reduction, and various types of data are converted into a unified format according to the data format conversion specification. According to the design scheme of the data closed-loop platform, data compliance and desensitization are also completed at the vehicle end.

[0073] Exemplarily, noise reduction, format conversion, and desensitization processing are performed on each group of aligned data to obtain processed multiple sensor data.

[0074] In this embodiment, by performing preprocessing on the aligned data, more effective and accurate multiple sensor data can be obtained.

[0075] In an exemplary embodiment, the cloud includes: a raw data layer, an annotated data layer, and an algorithm analysis structure layer; storing the intelligent driving data in the cloud includes: storing the intelligent driving data in the raw data layer of the cloud; after obtaining the annotated data, it further includes: storing the annotated data in the annotated data layer of the cloud; using the annotated data for model training includes: obtaining the annotated data from the annotated data layer of the cloud, using the annotated data for model training, and storing the obtained analysis result in the algorithm analysis result layer of the cloud.

[0076] Exemplarily, the cloud is divided into a raw data layer, an annotated data layer, and an algorithm analysis structure layer; storing the intelligent driving data in the raw data layer of the cloud; storing the annotated data in the annotated data layer of the cloud; obtaining the annotated data from the annotated data layer of the cloud, using the annotated data for model training, and storing the obtained analysis result in the algorithm analysis result layer of the cloud.

[0077] In this embodiment, by classifying and storing data in the raw data layer, the annotated data layer, and the algorithm analysis structure layer, the required data can be obtained more timely and conveniently.

[0078] In an exemplary embodiment, the automatic annotation of the data to be annotated includes:

[0079] Automatically annotating the data to be annotated by using 2D annotation, 3D annotation, and joint annotation methods.

[0080] Among them, methods such as 2D annotation, 3D annotation, and joint annotation are used to label the data. An automatic tool integrating the above various annotation methods can select a suitable tool according to the annotation task. After the annotation is completed, the data set can also be manually checked to ensure the annotation quality.

[0081] Exemplarily, the data to be annotated is automatically annotated by using 2D annotation, 3D annotation, and joint annotation methods.

[0082] In this embodiment, through automatic annotation, the pressure of manual data annotation can be alleviated.

[0083] In an exemplary embodiment, such as Figure 2As shown in the figure, a data processing method includes: after the vehicle is started, detecting and initializing various sensors on the vehicle end; after the initialization is completed, setting the sampling frequencies and data acquisition formats of various sensors according to the vehicle type and operation scenarios, and calibrating various sensors. Collecting vehicle-end data through a high-definition camera, a lidar, an inertial measurement unit (IMU), and a global positioning system (GPS); collecting weather data and traffic data through environmental sensors, and adding timestamps to the driving data collected by each sensor. Classifying the driving data collected by each sensor according to the timestamps, grouping the driving data with the same timestamp into one group to obtain multiple groups of aligned data. For each group of aligned data, performing noise reduction, format conversion, and desensitization processing on the aligned data, and using the Kalman filtering algorithm to perform data fusion on the processed aligned data to obtain intelligent driving data, compressing the intelligent driving data, and then storing the compressed intelligent driving data in the original data layer of the cloud. When model training is required, obtaining the data to be labeled from all the intelligent driving data stored in the cloud to obtain the data to be labeled, and automatically labeling the data to be labeled using 2D labeling, 3D labeling, and joint labeling according to the requirements of the labeling task to obtain labeled data, and storing the labeled data in the labeled data layer of the cloud; obtaining the labeled data from the labeled data layer of the cloud, using the labeled data for model training, and storing the analysis results obtained from the training in the algorithm analysis result layer of the cloud.

[0084] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limitation, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0085] In an exemplary embodiment, as Figure 3 shown, a data processing device is provided, including: a collection module 301, an alignment module 302, a fusion module 303, and a labeling module 304, where:

[0086] The collection module is used to collect driving data through various sensors and add timestamps to the data collected by each sensor;

[0087] An alignment module for aligning the data collected by each type of sensor according to timestamps to obtain multiple sets of aligned data, with each set of aligned data corresponding to a timestamp;

[0088] A fusion module for performing data fusion on the aligned data for each set of aligned data to obtain intelligent driving data and storing the intelligent driving data in the cloud;

[0089] A labeling module for, when model training is required, obtaining the data to be labeled from all the intelligent driving data stored in the cloud to get the data to be labeled, automatically labeling the data to be labeled according to the requirements of the labeling task to obtain labeled data, and using the labeled data for model training.

[0090] In an exemplary embodiment, the acquisition module is further configured to:

[0091] After the vehicle is started, detect and initialize multiple sensors on the vehicle end; after the initialization is completed, set the sampling frequency and the format of the collected data of the multiple sensors, and calibrate the multiple sensors.

[0092] In an exemplary embodiment, the multiple sensors include vehicle-end sensors and environmental sensors; the vehicle-end sensors include a high-definition camera, a lidar, an inertial measurement unit (IMU), and a global positioning system (GPS); the acquisition module is further configured to:

[0093] Collect vehicle-end data through the high-definition camera, the lidar, the inertial measurement unit (IMU), and the global positioning system (GPS); collect weather data and traffic data through the environmental sensors.

[0094] In an exemplary embodiment, the fusion module is further configured to:

[0095] Perform preprocessing on each set of aligned data respectively to obtain processed multiple-sensor data; the preprocessing includes at least one of noise reduction, format conversion, and desensitization processing.

[0096] In an exemplary embodiment, the cloud includes: a raw data layer, a labeled data layer, and an algorithm analysis structure layer;

[0097] The storage module is further configured to: store the intelligent driving data in the raw data layer of the cloud;

[0098] The labeling module is further configured to: store the labeled data in the labeled data layer of the cloud; obtain the labeled data from the labeled data layer of the cloud, use the labeled data for model training, and store the analysis result obtained from the training in the algorithm analysis result layer of the cloud.

[0099] In an exemplary embodiment, the annotation module is further configured to:

[0100] Automatically annotate the data to be annotated using 2D annotation, 3D annotation, and joint annotation methods.

[0101] Each module in the above data processing device can be implemented in whole or in part by software, hardware, and their combination. Each of the above modules can be embedded in the processor of the computer device in hardware form or be independent of it, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each of the above modules.

[0102] In an exemplary embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 4 shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store intelligent driving data. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, it implements a data processing method.

[0103] Those skilled in the art can understand that Figure 4 the structure shown in

[0104] is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0105] Collect driving data through multiple sensors and add timestamps to the driving data collected by each sensor;

[0106] Align the driving data collected by each sensor according to the timestamps to obtain multiple groups of aligned data, and each group of aligned data corresponds to a timestamp;

[0107] For each set of aligned data, perform data fusion on the aligned data to obtain intelligent driving data, and store the intelligent driving data in the cloud;

[0108] When model training is required, obtain the data to be labeled from all the intelligent driving data stored in the cloud to get the data to be labeled. According to the requirements of the labeling task, perform automated labeling on the data to be labeled to obtain labeled data, and use the labeled data for model training.

[0109] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0110] After the vehicle starts, detect and initialize various sensors on the vehicle end; after the initialization is completed, set the sampling frequency and the format of the collected data of the various sensors, and calibrate the various sensors.

[0111] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0112] The various sensors include vehicle-end sensors and environmental sensors; the vehicle-end sensors include a high-definition camera, a lidar, an inertial measurement unit (IMU), and a global positioning system (GPS); the driving data collected by the various sensors includes: collecting vehicle-end data through the high-definition camera, the lidar, the inertial measurement unit (IMU), and the global positioning system (GPS); collecting weather data and traffic data through the environmental sensors.

[0113] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0114] Perform preprocessing on each set of aligned data respectively to obtain processed data of various sensors; the preprocessing includes at least one of noise reduction, format conversion, and desensitization processing.

[0115] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0116] The cloud includes: a raw data layer, a labeled data layer, and an algorithm analysis structure layer; storing the intelligent driving data in the cloud includes: storing the intelligent driving data in the raw data layer of the cloud; after obtaining the labeled data, it further includes: storing the labeled data in the labeled data layer of the cloud; using the labeled data for model training includes: obtaining the labeled data from the labeled data layer of the cloud, using the labeled data for model training, and storing the analysis result obtained by training in the algorithm analysis result layer of the cloud.

[0117] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0118] Automatically annotate the data to be annotated by using 2D annotation, 3D annotation, and joint annotation methods.

[0119] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0120] Collect driving data through multiple sensors, and add timestamps to the driving data collected by each sensor;

[0121] Align the driving data collected by each sensor according to the timestamps to obtain multiple sets of aligned data, and each set of aligned data corresponds to a timestamp;

[0122] For each set of aligned data, perform data fusion on the aligned data to obtain intelligent driving data, and store the intelligent driving data in the cloud;

[0123] When model training is required, obtain the data to be annotated from all the intelligent driving data stored in the cloud to obtain the data to be annotated. According to the requirements of the annotation task, automatically annotate the data to be annotated to obtain annotated data, and use the annotated data for model training.

[0124] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0125] After the vehicle is started, detect and initialize multiple sensors on the vehicle end; after the initialization is completed, set the sampling frequency and the format of the collected data of the multiple sensors, and calibrate the multiple sensors.

[0126] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0127] The multiple sensors include vehicle-end sensors and environmental sensors; the vehicle-end sensors include high-definition cameras, lidars, inertial measurement units (IMUs), and global positioning systems (GPSs); the collection of driving data through multiple sensors includes: collecting vehicle-end data through the high-definition cameras, the lidars, the inertial measurement units (IMUs), and the global positioning systems (GPSs); collecting weather data and traffic data through the environmental sensors.

[0128] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0129] Perform preprocessing on each set of aligned data respectively to obtain processed multiple-sensor data; the preprocessing includes at least one of noise reduction, format conversion, and desensitization processing.

[0130] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0131] The cloud includes: a raw data layer, an annotated data layer, and an algorithm analysis structure layer; storing the intelligent driving data in the cloud includes: storing the intelligent driving data in the raw data layer of the cloud; after obtaining the annotated data, it further includes: storing the annotated data in the annotated data layer of the cloud; using the annotated data for model training includes: obtaining the annotated data from the annotated data layer of the cloud, using the annotated data for model training, and storing the obtained analysis result in the algorithm analysis result layer of the cloud.

[0132] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0133] Automatically annotate the data to be annotated using 2D annotation, 3D annotation, and joint annotation methods.

[0134] In one embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0135] Collect driving data through multiple sensors, and add timestamps to the driving data collected by each sensor;

[0136] Align the driving data collected by each sensor according to the timestamps to obtain multiple sets of aligned data, and each set of aligned data corresponds to a timestamp;

[0137] For each set of aligned data, perform data fusion on the aligned data to obtain intelligent driving data, and store the intelligent driving data in the cloud;

[0138] When model training is required, obtain the data to be annotated from all the intelligent driving data stored in the cloud to obtain the data to be annotated, automatically annotate the data to be annotated according to the requirements of the annotation task to obtain annotated data, and use the annotated data for model training.

[0139] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0140] After the vehicle is started, detect and initialize multiple sensors on the vehicle end; after the initialization is completed, set the sampling frequency and the format of the collected data of the multiple sensors, and calibrate the multiple sensors.

[0141] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0142] The multiple sensors include vehicle-end sensors and environmental sensors; the vehicle-end sensors include high-definition cameras, lidar, inertial measurement units (IMUs), and global positioning systems (GPSs); the driving data is collected through the multiple sensors, including: collecting vehicle-end data through the high-definition cameras, the lidar, the inertial measurement units (IMUs), and the global positioning systems (GPSs); collecting weather data and traffic data through the environmental sensors.

[0143] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented:

[0144] Preprocess each set of aligned data to obtain processed multiple-sensor data; the preprocessing includes at least one of noise reduction, format conversion, and desensitization processing.

[0145] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented:

[0146] The cloud includes: a raw data layer, an annotated data layer, and an algorithm analysis structure layer; storing the intelligent driving data in the cloud includes: storing the intelligent driving data in the raw data layer of the cloud; after obtaining the annotated data, it further includes: storing the annotated data in the annotated data layer of the cloud; using the annotated data for model training includes: obtaining the annotated data from the annotated data layer of the cloud, using the annotated data for model training, and storing the analysis result obtained from the training in the algorithm analysis result layer of the cloud.

[0147] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented:

[0148] Automatically annotate the data to be annotated using 2D annotation, 3D annotation, and joint annotation methods.

[0149] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in this application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, artificial intelligence (AI) processors, etc., without limitation.

[0150] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as within the scope recorded in this application.

[0151] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A data processing method, characterized in that: The method comprises: Collect driving data through multiple sensors and add timestamps to the driving data collected by each sensor; The driving data collected by each sensor is aligned according to the timestamp to obtain multiple groups of aligned data, each group of aligned data corresponds to a timestamp; For each set of aligned data, data fusion is performed on the aligned data to obtain intelligent driving data, and the intelligent driving data is stored in the cloud; When model training is required, the data that needs to be labeled is obtained from all intelligent driving data stored in the cloud to obtain the data to be labeled. According to the requirements of the labeling task, the data to be labeled is automatically labeled to obtain labeled data, and the labeled data is used for model training.

2. The method according to claim 1, characterized in that Before collecting driving data through multiple sensors, the method further includes: After the vehicle is started, detect and initialize various sensors on the vehicle side; After the initialization is completed, the sampling frequencies and data acquisition formats of the various sensors are set, and the various sensors are calibrated.

3. The method according to claim 1, characterized in that The multiple sensors include vehicle-side sensors and environmental sensors; the vehicle-side sensors include high-definition cameras, laser radars, inertial measurement units (IMUs), and global positioning systems (GPS); The driving data is collected through a variety of sensors, including: Collect vehicle-side data through the high-definition camera, the laser radar, the inertial measurement unit IMU, and the global positioning system GPS; The weather data and traffic data are collected by the environmental sensors.

4. The method according to claim 1, characterized in that Before fusing the aligned data, the method further includes: Each group of aligned data is preprocessed to obtain processed multiple sensor data; the preprocessing includes at least one of noise reduction, format conversion and desensitization processing.

5. The method according to claim 1, characterized in that The cloud includes: an original data layer, an annotated data layer and an algorithm analysis structure layer; The storing of the intelligent driving data to the cloud comprises: Storing the intelligent driving data in a raw data layer of the cloud; After obtaining the labeled data, the method further includes: Storing the annotated data in an annotated data layer in the cloud; The using the labeled data to perform model training includes: The annotated data is obtained from the annotated data layer in the cloud, the annotated data is used to perform model training, and the analysis results obtained from the training are stored in the algorithm analysis result layer in the cloud.

6. The method according to claim 1, characterized in that The step of automatically labeling the data to be labeled includes: The data to be labeled is automatically labeled using 2D labeling, 3D labeling and combined labeling.

7. A data processing device, characterized in that: The device comprises: A collection module, used to collect driving data through multiple sensors and add a timestamp to the data collected by each sensor; An alignment module is used to align the data collected by each sensor according to the timestamp to obtain multiple groups of aligned data, each group of aligned data corresponds to a timestamp; A fusion module, for performing data fusion on each set of aligned data to obtain intelligent driving data, and storing the intelligent driving data in the cloud; The labeling module is used to obtain the data that needs to be labeled from all the intelligent driving data stored in the cloud when model training is needed, obtain the data to be labeled, automatically label the data to be labeled according to the requirements of the labeling task, obtain the labeled data, and use the labeled data for model training.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images