Data processing method and device, equipment and medium

Abstract

The embodiment of the invention provides a data processing method and device, equipment and a medium. The method comprises the following steps: acquiring automatic driving data of an automatic driving vehicle; classifying and storing the automatic driving data to different storage positions; receiving a data reading request sent by an external platform; and adjusting the storage position of the automatic driving data corresponding to the data reading request. According to the embodiment of the invention, multi-source heterogeneous data is deeply integrated, and meanwhile, an intelligent caching strategy is combined, so that the real-time data processing requirement in an automatic driving scene is met, the vehicle can be ensured to make an accurate decision in time, and the driving safety and smoothness are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving, and particularly to a data processing method, apparatus, device, and medium. Background Art

[0002] With the development of autonomous driving technology, the amount of data generated by vehicles has increased exponentially, including sensor data (such as cameras, radars, lidars), vehicle status data (such as vehicle speed, steering angle), environmental data (such as road conditions, meteorological information), etc. These data have characteristics such as multi-source heterogeneity, high throughput, and high real-time requirements. The autonomous driving system needs to store a large amount of historical data (such as driving records, road condition information) and real-time data (such as sensor data, vehicle status data). Traditional centralized storage systems cannot meet the requirements of high concurrency and high throughput. The autonomous driving system has extremely high real-time requirements for data, especially vehicle real-time status data and environmental perception data. Traditional data access methods (such as directly reading from disks) cannot meet the requirements of low latency. Summary of the Invention

[0003] In view of the above problems, the present invention aims to provide a data processing method, apparatus, device, and medium to solve the problem of how to efficiently, accurately, and safely process autonomous driving data in the links of acquisition, storage, processing, model training, and verification, and improve the performance and reliability of the autonomous driving system.

[0004] According to the first aspect of the present invention, first, a data processing method is provided, and the method includes:

[0005] Obtain the autonomous driving data of an autonomous driving vehicle;

[0006] Classify and store the autonomous driving data in different storage locations;

[0007] Receive a data reading request sent by an external platform;

[0008] Adjust the storage location of the autonomous driving data corresponding to the data reading request.

[0009] Optionally, before obtaining the autonomous driving data of the autonomous driving vehicle, it further includes:

[0010] Collect the autonomous driving data through an offline collection method and / or a real-time collection method;

[0011] Blur the privacy data in the autonomous driving data and desensitize the sensitive information in the autonomous driving data.

[0012] Optionally, the different storage locations include a memory cache layer and a bottom storage layer;

[0013] Storing the autonomous driving data in different storage layers includes:

[0014] Classifying the autonomous driving data into hot data and cold data according to at least one of the data type, access frequency, and importance of the autonomous driving data;

[0015] Storing the hot data in the memory cache layer;

[0016] Storing the cold data in the underlying storage layer.

[0017] Optionally, the external platform includes an annotation platform;

[0018] Adjusting the storage location of the autonomous driving data corresponding to the read data request includes:

[0019] Determining the first target autonomous driving data requested by the annotation platform according to the data type currently being processed by the annotation platform, the annotation progress, and the access pattern of the historical annotation data;

[0020] Adjusting the storage location of the first target autonomous driving data to the memory cache layer;

[0021] In the case where the first target autonomous driving data requested by the annotation platform changes, determining the second target autonomous driving data currently requested by the annotation platform and adjusting the storage location of the second target autonomous driving data to the memory cache layer.

[0022] Optionally, the external platform includes a model training platform;

[0023] Adjusting the storage location of the autonomous driving data corresponding to the read data request includes:

[0024] Determining the third target autonomous driving data requested by the model training platform according to the requirements of the model training algorithm, the current convergence state of the model, and the training data required by the model;

[0025] Adjusting the storage location of the third target autonomous driving data to the memory cache layer;

[0026] In the case where the third target autonomous driving data requested by the model training platform changes, determining the fourth target autonomous driving data currently requested by the model training platform and adjusting the storage location of the fourth target autonomous driving data to the memory cache layer.

[0027] Optionally, the external platform includes a simulation platform;

[0028] Adjusting the storage location of the autonomous driving data corresponding to the read data request includes:

[0029] Determine fifth target autonomous driving data requested by the simulation platform according to the characteristics of the driving scenario and the real-time access requirements of the simulation platform for the autonomous driving data;

[0030] Adjust the storage location of the fifth target autonomous driving data to the memory cache layer;

[0031] In the case where the fifth target autonomous driving data requested by the simulation platform changes, determine sixth target autonomous driving data currently requested by the simulation platform, and adjust the storage location of the sixth target autonomous driving data to the memory cache layer.

[0032] Optionally, the method further includes:

[0033] Monitor the access frequency, last use time, and data type of hot data and cold data in real time;

[0034] In the case of meeting a first preset condition, adjust the storage location of the hot data; wherein, the first preset condition includes at least one of the access frequency of the hot data not being higher than a first preset threshold, the last use time of the hot data not being lower than a second preset threshold, and the hot data belonging to historical data;

[0035] And / or,

[0036] In the case of meeting a second preset condition, adjust the storage location of the cold data; wherein, the second preset condition includes at least one of the access frequency of the cold data being higher than a first preset threshold, the last use time of the cold data being lower than a second preset threshold, and the cold data belonging to real-time data.

[0037] According to a second aspect of the present invention, there is also provided a data processing device, the device includes:

[0038] A data acquisition module, configured to acquire autonomous driving data of an autonomous driving vehicle;

[0039] A storage module, configured to classify and store the autonomous driving data to different storage locations;

[0040] A receiving module, configured to receive a read data request sent by an external platform;

[0041] An adjustment module, configured to adjust the storage location of the autonomous driving data corresponding to the read data request.

[0042] According to a third aspect of the present invention, there is also provided an electronic device, including a processor, a memory, and a computer program stored on the memory and capable of running on the processor, where when the computer program is executed by the processor, the data processing method as described above is implemented.

[0043] According to the fourth aspect of the present invention, there is also provided a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the data processing method described above is implemented.

[0044] The data processing method provided by the embodiments of the present invention includes obtaining the autonomous driving data of an autonomous vehicle; classifying and storing the autonomous driving data in different storage locations; receiving a read data request sent by an external platform; and adjusting the storage location of the autonomous driving data corresponding to the read data request. The embodiments of the present invention deeply integrate multi-source heterogeneous data, construct an intelligent collaboration mechanism, fully explore the data value, provide accurate and comprehensive basis for autonomous driving decision-making, and achieve all-round data collaboration; at the same time, combined with an intelligent caching strategy, it meets the real-time data processing requirements in the autonomous driving scenario, ensures that the vehicle can make accurate decisions in a timely manner, and improves driving safety and fluency.

[0045] The above description is only an overview of the technical solution of the present invention. In order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of the present invention more obvious and understandable, the specific embodiments of the present invention are specifically given below. Description of the Drawings

[0046] By reading the following detailed description of the preferred embodiments, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present invention. Moreover, throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0047] Figure 1 is a step flowchart of a data processing method provided by an embodiment of the present invention;

[0048] Figure 2 is a data closed-loop processing flowchart provided by an embodiment of the present invention;

[0049] Figure 3 is a structural schematic diagram of a data processing device provided by an embodiment of the present invention. Detailed Embodiments

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will elaborate on various embodiments of the present invention in conjunction with the accompanying drawings. However, those of ordinary skill in the art can understand that in various embodiments of the present invention, many technical details are presented to help readers better understand the present invention. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in the present invention can still be implemented. The division of the following embodiments is for convenience of description and should not constitute any limitation to the specific implementation of the present invention. Various embodiments can be combined and cross-referenced with each other on the premise of no contradiction.

[0051] The following problems exist in related data processing technologies:

[0052] 1. Data integration and collaboration dilemmas: The multi-source heterogeneous data generated by autonomous driving is stored dispersedly, lacking an effective integration mechanism, forming data silos, which hinders the comprehensive utilization of data and the in-depth understanding of driving scenarios, and further affects the accuracy of driving decisions;

[0053] 2. Bottlenecks in data processing efficiency: Autonomous driving scenarios have extremely high requirements for the real-time processing of data. However, existing technologies are limited by the storage architecture and data access mechanism. In complex road conditions or emergencies, it is difficult to process a large amount of sensor data in a timely manner, resulting in decision-making delays and endangering driving safety;

[0054] 3. Complicated and costly data management: With the development of autonomous driving technology, the amount of data has increased explosively. Traditional data management methods rely on conventional tools and have many problems such as chaotic data classification, imperfect backup, and loopholes in permission management. This not only increases the management complexity but also leads to cost increases, affecting the data utilization efficiency and the overall benefit of the system;

[0055] 4. Limitations in model training and optimization: Conventional model training methods lack fine processing and efficient optimization mechanisms for the characteristics of autonomous driving data. The untimely and incomplete data acquisition affects the training quality, and there is a lack of an effective feedback mechanism to adjust the model, making it difficult for autonomous driving models to adapt to changes in road conditions and driving requirements.

[0056] This application makes use of the advantages of the intelligent caching strategy in the Alluxio memory-first hierarchical architecture to efficiently, precisely, and safely process autonomous driving data in the aspects of data collection, storage, processing, model training, and verification, thereby improving the performance and reliability of the autonomous driving system.

[0057] Alluxio is an open-source distributed memory acceleration layer designed to provide efficient data access and caching services for big data and machine learning workloads. By caching data in memory, it significantly accelerates data access speed while providing seamless integration with multiple underlying storage systems. The core design concept of Alluxio is "memory first", optimizing the processing performance of data-intensive applications through a hierarchical architecture and intelligent caching strategies.

[0058] As a key enabling technology of this application, Alluxio's hierarchical architecture includes an underlying storage layer (which can be seamlessly docked with multiple mainstream storage systems), a data caching layer (which implements intelligent caching using an adaptive intelligent caching algorithm), and an application layer interface (which provides a convenient access channel for upper-layer data-driven applications). Among them, the adaptive intelligent caching of the data caching layer is the core, which can dynamically adjust the caching strategy based on various data factors, thereby significantly improving the data access speed and bringing high-efficiency data processing capabilities to the entire autonomous driving data closed-loop processing system.

[0059] Refer to Figure 1 , which shows the step flowchart of a data processing method provided by an embodiment of the present invention, and specifically may include the following steps:

[0060] Step 101, obtain the autonomous driving data of the autonomous vehicle.

[0061] When collecting autonomous driving data, a multi-source collection strategy is adopted, including vehicle-end offline collection (dynamically setting the collection interval according to factors such as vehicle driving mileage and road conditions, and copying vehicle operation data, including multi-source heterogeneous data such as videos, audios, pictures, point clouds, signals, etc.) and real-time collection (the vehicle-end is equipped with a TBOX (Telematics BOX, remote information processor / vehicle networking terminal) device to collect signal data in real time in combination with an intelligent sensor network, which is relayed by kafka and adaptively adjusts the collection frequency according to real-time road conditions and vehicle states), and at the same time introducing third-party data source collection (obtaining external data such as road condition information and meteorological data by cooperating with traffic management departments, meteorological departments, etc.). Among them, Kafka is a high-throughput distributed publish-subscribe messaging system that can process all action stream data of consumers in a website.

[0062] During the collection process, anonymize the vehicle-end data related to the personal information of the vehicle owner or passengers in the vehicle (such as the pictures taken by the in-vehicle camera, voice records, etc.) to ensure user privacy.

[0063] Step 102, classify and store the autonomous driving data in different storage locations.

[0064] Build a unified intelligent storage architecture with Alluxio as the core hub, integrating multiple third-party storage systems (such as HDFS (Hadoop Distributed File System), S3 (Amazon Simple Storage Service), local file systems, etc.). Among them, Alluxio, as the core hub, is responsible for the unified management and intelligent allocation of data; the third-party storage system is used for persistent data storage.

[0065] The third-party storage system is integrated with Alluxio to provide a unified access interface (i.e., the application layer interface), and has two storage levels in the unified intelligent storage architecture: the memory cache layer and the underlying storage layer. Among them, the memory cache layer is used to store hot data (i.e., frequently accessed data, highly important data, real-time data), and the underlying storage layer is used to store cold data (i.e., infrequently accessed data, less important data, historical data). By caching hot data in the memory cache layer, Alluxio can significantly improve the data access speed and meet the real-time requirements of the autonomous driving system.

[0066] The unified intelligent storage architecture with Alluxio as the core hub can significantly improve the data access efficiency, optimize the storage resource utilization rate, and meet the requirements of the autonomous driving system for high performance, high reliability, and high scalability by intelligently allocating data to different storage locations (memory cache layer, underlying storage layer).

[0067] Step 103: Receive a data reading request sent by an external platform.

[0068] When an external platform (annotation platform, model training platform, or simulation platform) needs data, it can send a data reading request to Alluxio in the unified intelligent storage architecture through the application layer interface.

[0069] Step 104: Adjust the storage location of the autonomous driving data corresponding to the data reading request.

[0070] Specifically, Alluxio first searches for the requested data in the memory cache layer. If the data is found in the memory cache layer (cache hit), the data is directly returned to achieve the fastest access speed (usually at the microsecond level). If the data is not found in the memory cache layer (cache miss), Alluxio will extract the required data from the underlying storage layer (underlying storage system, such as HDFS) according to the data storage location information and transfer it to the memory cache layer for subsequent access. Finally, the data is returned to the requester (such as annotation platform, model training platform, simulation platform) through the application layer interface.

[0071] Among them, the intelligent data management module maintains the metadata information of autonomous driving data (such as data location, data type, creation time, etc.), and then Alluxio can extract data from the underlying storage system according to the metadata information of the data.

[0072] The intelligent data management module is one of the core components in the unified intelligent storage architecture, responsible for monitoring and managing the entire life cycle of data. Through intelligent data management strategies, this module can automatically perform operations such as cache updates and backups, and realize the automatic transfer of data between different storage layers based on factors such as data access frequency, last use time, and data type.

[0073] Through the collaborative work of the memory cache layer of Alluxio and the underlying storage system, the annotation platform, model training platform, and simulation platform can achieve efficient data access. This mechanism not only improves the data access speed but also optimizes resource utilization, and is applicable to application scenarios that require frequent access to large-scale data.

[0074] In the embodiment of the present invention, autonomous driving data of an autonomous driving vehicle is obtained; the autonomous driving data is classified and stored in different storage locations; a data reading request sent by an external platform is received; the storage location of the autonomous driving data corresponding to the data reading request is adjusted. The embodiment of the present invention deeply integrates multi-source heterogeneous data, constructs an intelligent collaboration mechanism, fully explores the data value, provides accurate and comprehensive basis for autonomous driving decisions, and realizes all-round data collaboration; at the same time, combined with intelligent cache strategies, it meets the real-time data processing requirements in the autonomous driving scenario, ensures that the vehicle can make accurate decisions in time, and improves driving safety and fluency.

[0075] In an alternative embodiment of the present invention, the following steps are further included before step 101:

[0076] S1011, collecting autonomous driving data through an offline collection method and / or a real-time collection method;

[0077] S1012, performing fuzzy processing on the privacy data in the autonomous driving data and desensitizing the sensitive information in the autonomous driving data.

[0078] When obtaining autonomous driving data, a multi-source collection strategy is adopted for collection, combining the offline collection method and the real-time collection method.

[0079] Offline collection: Collect vehicle operation data, including videos, audios, pictures, point clouds, signals, etc., covering multi-modal data required by the autonomous driving system. During collection, the data collection interval can be dynamically adjusted according to factors such as vehicle mileage and road conditions to ensure the representativeness and efficiency of the data. For example, increase the collection frequency in complex road conditions (such as congestion, construction areas), and reduce the collection frequency on smooth sections.

[0080] Real-time acquisition: Equip the vehicle with a TBOX device and combine it with an intelligent sensor network to collect signal data in real time. Such as vehicle speed, acceleration, steering angle, environmental perception data, etc. The collected real-time data is relayed through the Kafka message queue to ensure high-throughput and low-latency data transmission. During acquisition, the acquisition frequency can be dynamically adjusted according to real-time road conditions and vehicle status. For example, increase the acquisition frequency during emergency braking or complex road conditions to ensure data integrity and real-time performance.

[0081] Among them, as a message queue, Kafka can buffer a large amount of real-time data from the vehicle end, avoiding direct coupling between data producers and consumers. Even if the downstream system (such as a data analysis platform) is temporarily unable to process data, Kafka can ensure that the data is not lost, and Kafka supports a high throughput of millions of messages per second, which is suitable for the large amount of data generated by the vehicle-end sensor network. The collected real-time data can be temporarily stored in the Kafka message queue, and then Alluxio obtains the data by subscribing to Kafka.

[0082] In addition, when obtaining autonomous driving data, it is also necessary to collect in combination with third-party data sources. By cooperating with traffic management departments, meteorological departments, etc., external data such as road condition information (such as traffic flow, accident information) and meteorological data (such as temperature, humidity, visibility) are obtained. The third-party data is fused with the data collected from the vehicle end to provide more comprehensive environmental perception and decision support for the autonomous driving system.

[0083] During the acquisition process of autonomous driving data, anonymize the vehicle-end data (such as the images captured by the in-vehicle camera, voice records, etc.) that involve the personal information of the vehicle owner or in-vehicle passengers to ensure user privacy. For example, blur the in-vehicle face information and desensitize the sensitive information in the voice records.

[0084] Through the multi-source acquisition strategy, it covers vehicle-end offline data, real-time data, and third-party data, providing comprehensive data support for the autonomous driving system. The caching and distributed characteristics of Alluxio ensure efficient data transmission and storage, meeting the requirements of the autonomous driving system for real-time performance and low latency. Through anonymization processing and compliance design, user privacy is effectively protected.

[0085] In an optional embodiment of the present invention, step 102 further includes the following sub-steps:

[0086] S1021, classify the autonomous driving data into hot data and cold data according to at least one of the data type, access frequency, and importance of the autonomous driving data;

[0087] S1022, store the hot data in the memory cache layer;

[0088] S1023, store the cold data into the underlying storage layer.

[0089] Hot data refers to real-time data with high access frequency, crucial for real-time decision-making, or urgently needed in the current scenario. Its characteristics are: high access frequency: data that is frequently accessed; high importance: data that plays a key role in system operation and decision-making; strong real-time performance: data that needs to be accessed quickly.

[0090] Examples of hot data include vehicle real-time status data such as vehicle speed, steering angle, and brake signal. Current driving scenario data such as current road conditions, weather information, and sensor data (cameras, radars, lidars, etc.). Frequently accessed labeled data such as pictures, videos, and point cloud data being labeled. Key data in model training such as the dataset of the current training batch.

[0091] Cold data refers to historical data with low access frequency, little impact on real-time decision-making, or long-term preservation. Its characteristics are: low access frequency: data that is rarely accessed; low importance: data that has little impact on system operation and decision-making; long-term preservation: data that needs to be stored long-term but does not need to be accessed frequently.

[0092] Examples of cold data include historical driving records such as past road conditions and vehicle driving trajectories. Archived logs such as system logs and test logs. Low-frequency accessed labeled data such as historical data that has been labeled. Backup data such as backups of model training data.

[0093] Specifically, classify autonomous driving data with data types of real-time data (real-time sensor data, current driving scenario data, frequently accessed labeled data), and / or frequently accessed high-frequency data (such as real-time sensor data, dataset of the current training batch), and / or high-importance data crucial for real-time decision-making (such as vehicle speed, steering angle, brake signal) as hot data. Classify data with data types of historical data (such as historical driving records, archived logs, low-frequency accessed labeled data), and / or rarely accessed low-frequency data (such as historical driving records, archived logs), and / or low-importance data with little impact on real-time decision-making (such as historical logs, backup data) as cold data.

[0094] Store hot data in the memory cache layer to ensure the fastest access speed, and store cold data in the underlying storage layer to reduce storage costs.

[0095] In an alternative embodiment of the present invention, step 104 further includes the following sub-steps:

[0096] S1041. Determine the first target autonomous driving data requested by the annotation platform according to the data type currently being processed by the annotation platform, the annotation progress, and the access pattern of historical annotation data.

[0097] S1042. Adjust the storage location of the first target autonomous driving data to the memory cache layer.

[0098] S1043. In the case where the first target autonomous driving data requested by the annotation platform changes, determine the second target autonomous driving data currently requested by the annotation platform, and adjust the storage location of the second target autonomous driving data to the memory cache layer.

[0099] When the annotation platform needs data for annotation operations (sending a data read request), the intelligent data management module first anticipates some of the data that the annotation platform may need next based on factors such as the data type currently being processed by the annotation platform, the operation progress of the annotator, and the access pattern of historical annotation data. According to the anticipation result, it decides which data needs to be cached to the memory cache layer (caching policy). Then Alluxio transfers the anticipated data to the memory cache layer for high-speed access.

[0100] The intelligent data management module and Alluxio work together. The intelligent data management module is responsible for data lifecycle management, caching policy formulation, and data requirement anticipation. According to the anticipation result, it decides which data needs to be cached to the memory cache layer (i.e., formulates the caching policy). The intelligent data management module is the decision-making layer, responsible for analyzing data requirements and formulating caching policies. Alluxio is a distributed memory acceleration layer, responsible for actual data caching and distribution. It caches data to the memory cache layer according to the instructions of the intelligent data management module. Alluxio is the execution layer, responsible for implementing the caching policy of the intelligent data management module.

[0101] For example, if the annotation platform is annotating vehicle video data and currently focusing on annotating the driving posture of the vehicle, the intelligent data management module will analyze that in previous similar annotation tasks, it is usually necessary to further annotate the interaction between the vehicle and the surrounding environment. Thus, it determines that video segments related to the vehicle's surrounding environment (such as close contact scenarios between the vehicle and pedestrians, other vehicles, etc.) may need to be cached. Then Alluxio transfers the video segment data related to the vehicle's surrounding environment to the memory cache layer. Among them, if the video segment data related to the vehicle's surrounding environment is already in the memory cache layer, there is no need to transfer it, and the data is directly returned to achieve the fastest access speed.

[0102] The intelligent data management module continuously monitors the working status of the annotation platform (such as annotation progress, annotation speed) and changes in data requirements (such as the shift of annotation focus). As the annotation work progresses, if the annotation focus of the annotation platform shifts to the detailed features of vehicles (such as the identification of vehicle parts, etc.), the intelligent data management module will adjust the caching strategy in a timely manner according to the new requirements, and determine that video frames or picture data related to the detailed features of vehicles need to be preferentially cached to the memory cache layer close to the annotation platform, so that the annotation platform can quickly obtain the required data and improve the annotation efficiency. Then Alluxio is responsible for implementing the caching strategy of the intelligent data management module, and caching video frames or picture data related to the detailed features of vehicles to the memory cache layer.

[0103] Through the caching update and adjustment mechanism of the intelligent data management module, the annotation platform can dynamically adjust the cached content according to actual needs to ensure that the cached data is always consistent with the annotation focus; by preferentially caching data related to the current annotation focus, the waiting time for the annotation platform to load data is reduced, and the annotation efficiency is improved.

[0104] In an optional embodiment of the present invention, step 104 further includes the following sub-steps:

[0105] S104-1, determine the third target autonomous driving data requested by the model training platform according to the requirements of the model training algorithm, the current convergence state of the model, and the training data required by the model;

[0106] S104-2, adjust the storage location of the third target autonomous driving data to the memory cache layer;

[0107] S104-3, in the case where the third target autonomous driving data requested by the model training platform changes, determine the fourth target autonomous driving data currently requested by the model training platform, and adjust the storage location of the fourth target autonomous driving data to the memory cache layer.

[0108] During the process that the model training platform needs data for model training, the intelligent data management module allocates and caches data based on the requirements of the training algorithm, the current convergence state of the model, and the characteristics of the training data (such as data type, access frequency, importance, etc.).

[0109] Different training algorithms have different requirements for data. For example, deep learning algorithms require a large amount of basic data samples at the initial stage of training to start the training process.

[0110] The requirements for data in different stages of model training (such as the initial stage, the middle stage, and the later stage) are different. Initial stage: A large amount of basic data samples are required to start the training; Middle stage: Diversified data samples are required to improve the generalization ability of the model; Later stage: High-quality data samples are required to fine-tune the model.

[0111] For example, when using a deep learning algorithm for model training, at the initial stage of training, a large number of basic data samples (such as labeled data of various vehicle driving scenarios) are required to start the training process. The intelligent data management module will determine that these basic data samples need to be preferentially cached in the memory cache layer according to their importance and access frequency to speed up data access and thus accelerate the initial stage of model training. Then Alluxio caches the data in the memory cache layer according to the instructions of the intelligent data management module.

[0112] As the model training progresses, when the model is approaching the convergence state, the training algorithm may have a change in the demand for certain specific data (such as the driving data of vehicles under specific road conditions), and these data are needed for the final fine-tuning. The intelligent data management module will detect this change in demand in real time, and through its adaptive intelligent caching algorithm, dynamically adjust the caching strategy to cache these specific data in the memory cache layer in a timely manner to ensure that the model can successfully complete the final convergence process and improve the overall efficiency of model training.

[0113] By dynamically adjusting the caching strategy, it is ensured that the training algorithm can quickly obtain the required data, improving the efficiency of model training; dynamically adjusting the cached content according to the change in demand can maximize the efficient use of the cache space and reduce the access frequency to the underlying storage system.

[0114] In an alternative embodiment of the present invention, step 104 further includes the following sub-steps:

[0115] S4-1, determining the fifth target autonomous driving data requested by the simulation platform according to the characteristics of the driving scenario and the real-time access demand of the simulation platform for the autonomous driving data;

[0116] S4-2, adjusting the storage location of the fifth target autonomous driving data to the memory cache layer;

[0117] S4-3, in the case where the fifth target autonomous driving data requested by the simulation platform changes, determining the sixth target autonomous driving data currently requested by the simulation platform and adjusting the storage location of the sixth target autonomous driving data to the memory cache layer.

[0118] When the simulation platform needs to build a simulation scenario to simulate various driving scenarios, Alluxio is responsible for providing the required data for the simulation platform and optimizing the caching of the data according to the characteristics of the simulation scenario (such as different weather conditions, road conditions, etc.) and the real-time access demand of the simulation platform for the data.

[0119] For example, when the simulation platform needs to simulate a driving scenario in a heavy rainstorm, the intelligent data management module will anticipate that the simulation platform requires a large amount of data related to the impact of rain on visibility, the driving characteristics of vehicles on slippery roads, etc. Alluxio caches this relevant data in the memory cache layer in advance according to the decision of the intelligent data management module, ensuring that the simulation platform can quickly obtain the required data during operation and guaranteeing the accuracy and efficiency of the simulation test.

[0120] During the simulation test, if the simulation scenario of the simulation platform changes (such as switching from a sunny driving scenario to a night driving scenario), the intelligent data management module will immediately sense this change and, according to the new simulation requirements, re-evaluate the data content and priority to be cached. For example, for a night driving scenario, the intelligent data management module will quickly determine that data related to the vehicle's lighting effect and the visible range at night needs to be cached in the memory cache layer to meet the real-time data requirements of the simulation platform and ensure that the simulation test can accurately simulate the driving conditions in different scenarios. Then Alluxio implements the caching strategy of the intelligent data management module and transfers the data related to the vehicle's lighting effect and the visible range at night to the memory cache layer.

[0121] Through the real-time response mechanism of the intelligent data management module to changes in simulation requirements, the simulation platform can quickly obtain the required data when the scenario switches, ensuring that the simulation test can accurately simulate the driving conditions in different scenarios.

[0122] In an optional embodiment of the present invention, the following steps may further be included:

[0123] S1, Monitor the access frequency, last use time, and data type of hot data and cold data in real time;

[0124] S2, Adjust the storage location of the hot data when the first preset condition is met; wherein, the first preset condition includes at least one of the access frequency of the hot data not being higher than the first preset threshold, the last use time of the hot data not being lower than the second preset threshold, and the hot data belonging to historical data;

[0125] And / or,

[0126] S3, Adjust the storage location of the cold data when the second preset condition is met; wherein, the second preset condition includes at least one of the access frequency of the cold data being higher than the first preset threshold, the last use time of the cold data being lower than the second preset threshold, and the cold data belonging to real-time data.

[0127] Among them, the first preset threshold and the second preset threshold can be set according to actual needs, and the present application does not make any restrictions.

[0128] The intelligent data management module monitors and manages the entire lifecycle of autonomous driving data. Based on factors such as the access frequency of the data, the last usage time, and the data type, it automatically determines and executes cache updates (i.e., storing the data in different storage layers), enabling the automatic transfer of data between different storage layers to meet the data requirements at different stages.

[0129] Specifically, the intelligent data management module monitors hot data and cold data in real time. The monitoring metrics include: access frequency: the number of times or frequency at which the data is accessed; last usage time: the time when the data was last accessed; data type: the type and format of the data (such as real-time data, historical data, pictures, videos, etc.). The data access information can be collected in real time through API interfaces (Application Programming Interface), log records, or database queries, and this application does not impose restrictions.

[0130] When the access frequency of hot data decreases and it no longer belongs to high-frequency access data, and / or the hot data has not been accessed for a long time and may no longer be needed, and / or the hot data has changed from real-time data to historical data, the hot data is transferred from the memory cache layer to the underlying storage layer. For example, when a driving scenario ends, the relevant real-time data may change from hot data to cold data and be demoted from the memory cache layer to the underlying storage layer.

[0131] When the access frequency of cold data increases and it has changed to hot data, and / or the cold data has been frequently accessed recently and needs to be promoted to a higher storage location, and / or the cold data has changed from historical data to real-time data, the cold data is promoted from the underlying storage layer to the memory cache layer. For example, when analyzing historical driving records, the relevant data may change from cold data to hot data and be promoted from the underlying storage layer to the memory cache layer.

[0132] In addition, it should be noted that for vehicle real-time status data that is frequently accessed and crucial for real-time decision-making (such as vehicle speed, steering angle, brake signal, etc.), regardless of the interaction with which platform (annotation platform, model training platform, simulation platform, etc.), the intelligent data management module preferentially stores it in the memory cache layer to achieve the fastest access speed, ensuring that each platform can quickly obtain these key data when needed and guaranteeing the efficient operation of the entire autonomous driving data closed-loop processing system.

[0133] For historical data that is not frequently accessed but needs to be stored for a long time (such as the vehicle's past driving records, past road conditions, etc.), the intelligent data management module will store it in the underlying storage system (such as HDFS), and through its unified management function, ensure that these data can be extracted and used orderly when needed. At the same time, according to the access requirements of different platforms for these historical data in specific situations, the intelligent data management module will timely cache some relevant historical data to the memory cache layer to improve data access efficiency. Among them, it can be specified through configuration information which data belongs to the vehicle real-time status data that is frequently accessed and crucial for real-time decision-making (such as vehicle speed, steering angle, brake signal, etc.), and set high priority for these data to ensure that they can always be stored in the memory cache layer.

[0134] By real-time monitoring the access frequency, last usage time, and data type of hot data and cold data, the intelligent data management module can dynamically adjust their storage locations, maximize the utilization rate of storage resources, ensure that frequently accessed data can be quickly obtained, and at the same time reduce the storage cost of low-frequency accessed data, providing strong support for the autonomous driving data closed-loop processing system.

[0135] Refer to Figure 2 , which shows a data closed-loop processing flow chart provided by an embodiment of the present invention, specifically as follows:

[0136] Data collection:

[0137] Vehicle-end offline collection: Collect data offline from the vehicle end. The collection interval can be dynamically set according to factors such as vehicle driving mileage and road conditions, and copy vehicle operation data, including multi-source heterogeneous data such as videos, audios, pictures, point clouds, signals, etc.;

[0138] Vehicle-end real-time collection: The TBOX device equipped at the vehicle end combines with the intelligent sensor network to collect signal data in real time;

[0139] Third-party data source collection: Obtain external data such as road conditions information and meteorological data by cooperating with traffic management departments, meteorological departments, etc.;

[0140] During the collection process, anonymize the vehicle-end data involving the personal information of the vehicle owner or passengers in the vehicle (such as the pictures taken by the in-vehicle camera, voice records, etc.) to ensure user privacy.

[0141] Write the anonymized data into Alluxio through the application layer interface layer. According to the characteristics of the data (such as data type, access frequency, importance, etc.) and the requirements of the current driving scenario, intelligently allocate different types of data to different storage layers. For example, for vehicle real-time status data (such as vehicle speed, steering angle, brake signal, etc.) that is frequently accessed and crucial for real-time decision-making, it is preferentially stored in the memory cache layer of Alluxio; for historical data that is not frequently accessed but needs to be stored for a long time, it is stored in a third-party storage system (underlying storage layer) (such as HDFS), and is uniformly managed by Alluxio.

[0142] Monitor and manage the entire life cycle of the data. According to factors such as the access frequency of the data, the last usage time, and the data type, automatically judge and execute cache updates, and realize the automatic transfer of data between different storage layers to meet the data requirements at different stages. For example, when the access frequency of low-frequency data stored in a third-party storage system (underlying storage layer) becomes higher, this part of the data can be automatically transferred to the memory cache layer.

[0143] When the downstream platform (external platform, annotation platform, model training platform, or simulation platform) needs data, first search in the memory cache layer. If found, directly obtain it to achieve the fastest access speed. If the required data is not found in the memory cache layer, Alluxio will extract and cache it from the third-party storage system (such as HDFS) according to the storage location information of the data into the memory cache layer for subsequent fast access.

[0144] Real-time monitor the data requirements of the downstream platform, detect the data requirements of the downstream platform, and dynamically adjust the cache policy. Alluxio will adjust the storage location of the data in real time according to the cache policy to ensure that the data can be quickly obtained by the downstream platform and improve efficiency. For example, when the annotation focus of the annotation platform shifts from the driving posture of the vehicle to the detailed features of the vehicle, the cache policy is adjusted in a timely manner. Previously, video clips related to the surrounding environment of the vehicle related to the driving posture of the vehicle (such as close contact scenes between the vehicle and pedestrians, other vehicles, etc.) were cached in the memory cache layer. After adjustment, video frames or picture data related to the detailed features of the vehicle are preferentially cached in the memory cache layer close to the annotation platform.

[0145] Refer to Figure 3 , which shows a schematic structural diagram of a data processing device provided by an embodiment of the present invention. The device includes:

[0146] A data acquisition module 201, configured to acquire autonomous driving data of an autonomous vehicle;

[0147] A storage module 202, configured to classify and store the autonomous driving data at different storage locations;

[0148] A receiving module 203, configured to receive a read data request sent by an external platform;

[0149] An adjustment module 204, configured to adjust the storage location of the autonomous driving data corresponding to the read data request.

[0150] In an alternative embodiment of the present invention, the device further includes:

[0151] A data acquisition module, configured to acquire autonomous driving data through an offline acquisition method and / or a real-time acquisition method;

[0152] A data preprocessing module, configured to perform blurring processing on the privacy data in the autonomous driving data and desensitize the sensitive information in the autonomous driving data.

[0153] In an alternative embodiment of the present invention, the different storage locations include a memory cache layer and a bottom storage layer; the storage module 202 includes:

[0154] A classification module, configured to classify the autonomous driving data into hot data and cold data according to at least one of the data type, access frequency, and importance of the autonomous driving data;

[0155] A hot data storage module, configured to store the hot data in the memory cache layer;

[0156] A cold data storage module, configured to store the cold data in the bottom storage layer.

[0157] In an alternative embodiment of the present invention, the external platform includes an annotation platform; the adjustment module 204 includes:

[0158] A first target autonomous driving data determination module, configured to determine the first target autonomous driving data requested by the annotation platform according to the data type, annotation progress, and access mode of the historical annotation data currently being processed by the annotation platform;

[0159] A first adjustment module, configured to adjust the storage location of the first target autonomous driving data to the memory cache layer

[0160] A second adjustment module, configured to determine the second target autonomous driving data currently requested by the annotation platform in the case where the first target autonomous driving data requested by the annotation platform changes, and adjust the storage location of the second target autonomous driving data to the memory cache layer.

[0161] In an alternative embodiment of the present invention, the external platform further includes a model training platform; the adjustment module 204 includes:

[0162] A third target autonomous driving data determination module, configured to determine third target autonomous driving data requested by the model training platform according to the requirements of the model training algorithm, the current convergence state of the model, and the training data required by the model;

[0163] A third adjustment module, configured to adjust the storage location of the third target autonomous driving data to the memory cache layer;

[0164] A fourth adjustment module, configured to, when the third target autonomous driving data requested by the model training platform changes, determine fourth target autonomous driving data currently requested by the model training platform, and adjust the storage location of the fourth target autonomous driving data to the memory cache layer.

[0165] In an optional embodiment of the present invention, the external platform further includes a simulation platform; the adjustment module 204 includes:

[0166] A fifth target autonomous driving data determination module, configured to determine fifth target autonomous driving data requested by the simulation platform according to the characteristics of the driving scenario and the real-time access requirements of the simulation platform for the autonomous driving data;

[0167] A fifth adjustment module, configured to adjust the storage location of the fifth target autonomous driving data to the memory cache layer;

[0168] A sixth adjustment module, configured to, when the fifth target autonomous driving data requested by the simulation platform changes, determine sixth target autonomous driving data currently requested by the simulation platform, and adjust the storage location of the sixth target autonomous driving data to the memory cache layer.

[0169] In an optional embodiment of the present invention, the device further includes:

[0170] A monitoring module, configured to monitor in real time the access frequency, the last use time, and the data type of hot data and cold data;

[0171] A hot data adjustment module, configured to adjust the storage location of the hot data when a first preset condition is met; wherein the first preset condition includes at least one of that the access frequency of the hot data is not higher than a first preset threshold, the last use time of the hot data is not lower than a second preset threshold, and the hot data belongs to historical data;

[0172] And / or,

[0173] A cold data adjustment module is used to adjust the storage location of the cold data when a second preset condition is met; wherein, the second preset condition includes at least one of the access frequency of the cold data being higher than a first preset threshold, the last usage time of the cold data being lower than a second preset threshold, and the cold data belonging to real-time data.

[0174] In an embodiment of the present invention, by acquiring the autonomous driving data of an autonomous vehicle; classifying and storing the autonomous driving data in different storage locations; receiving a read data request sent by an external platform; and adjusting the storage location of the autonomous driving data corresponding to the read data request. The embodiments of the present invention deeply integrate multi-source heterogeneous data, construct an intelligent collaboration mechanism, fully explore the data value, provide accurate and comprehensive basis for autonomous driving decision-making, and achieve all-round data collaboration; at the same time, combined with an intelligent caching strategy, it meets the real-time data processing requirements in the autonomous driving scenario, ensures that the vehicle can make accurate decisions in a timely manner, and improves driving safety and fluency.

[0175] An embodiment of the present invention further provides an electronic device, including a processor, a memory, and a computer program stored on the memory and capable of running on the processor. When the computer program is executed by the processor, it implements the data processing method as described above.

[0176] The memory may include a random access memory (Random Access Memory, abbreviated as RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0177] The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, abbreviated as CPU), a network processor (Network Processor, abbreviated as NP), etc.; it may also be a digital signal processor (DigitalSignal Processing, abbreviated as DSP), an application specific integrated circuit (Application Specific IntegratedCircuit, abbreviated as ASIC), a field programmable gate array (Field-Programmable Gate Array, abbreviated as FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0178] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the data processing method as described above.

[0179] For the apparatus embodiments, since they are basically similar to the method embodiments, they are described relatively simply. For the related parts, please refer to the corresponding descriptions in the method embodiments.

[0180] In the above embodiments, they can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, they can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from a website, computer, server, or data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or a wireless manner (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)).

[0181] It should be noted that in this document, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including", or any other variation thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or device that includes a series of elements includes not only those elements but also other elements not expressly listed, or elements that are inherent to such process, method, article, or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article, or device that includes the element.

[0182] Each embodiment in this specification is described in a related manner. For the same or similar parts between the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiments, since they are basically similar to the method embodiments, they are described relatively simply. For the related parts, please refer to the corresponding descriptions in the method embodiments.

[0183] The above are only the preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention are all included in the protection scope of the present invention.

Claims

1. A data processing method, characterized in that: The method comprises: Obtaining autonomous driving data from autonomous vehicles; Classify and store the autonomous driving data in different storage locations; Receive data read requests sent by external platforms; Adjust the storage location of the autonomous driving data corresponding to the data read request.

2. The method according to claim 1, characterized in that Before obtaining the autonomous driving data of the autonomous driving vehicle, the method further includes: Collecting autonomous driving data through offline collection method and / or real-time collection method; The privacy data in the autonomous driving data is blurred, and the sensitive information in the autonomous driving data is desensitized.

3. The method according to claim 1, characterized in that The different storage locations include a memory cache layer and an underlying storage layer; The classifying and storing the autonomous driving data into different storage layers includes: Classify the autonomous driving data into hot data and cold data according to at least one of a data type, an access frequency, and an importance of the autonomous driving data; storing the hot data in the memory cache layer; The cold data is stored in the underlying storage layer.

4. The method according to claim 1, characterized in that: The external platform includes a marking platform; The adjusting the storage location of the autonomous driving data corresponding to the data reading request includes: Determining first target autonomous driving data requested by the annotation platform according to a data type currently being processed by the annotation platform, an annotation progress, and an access mode of historical annotation data; Adjusting the storage location of the first target autonomous driving data to a memory cache layer; When the first target autonomous driving data requested by the annotation platform changes, the second target autonomous driving data currently requested by the annotation platform is determined, and the storage location of the second target autonomous driving data is adjusted to the memory cache layer.

5. The method according to claim 1, characterized in that The external platform includes a model training platform; The adjusting the storage location of the autonomous driving data corresponding to the data reading request includes: Determining the third target autonomous driving data requested by the model training platform according to the requirements of the model training algorithm, the current convergence state of the model, and the training data required by the model; Adjusting the storage location of the third target autonomous driving data to the memory cache layer; When the third target autonomous driving data requested by the model training platform changes, the fourth target autonomous driving data currently requested by the model training platform is determined, and the storage location of the fourth target autonomous driving data is adjusted to the memory cache layer.

6. The method according to claim 1, characterized in that The external platform includes a simulation platform; The adjusting the storage location of the autonomous driving data corresponding to the data reading request includes: determining fifth target autonomous driving data requested by the simulation platform according to the characteristics of the driving scenario and the real-time access requirements of the simulation platform to the autonomous driving data; Adjusting the storage location of the fifth target autonomous driving data to the memory cache layer; When the fifth target autonomous driving data requested by the simulation platform changes, the sixth target autonomous driving data currently requested by the simulation platform is determined, and the storage location of the sixth target autonomous driving data is adjusted to the memory cache layer.

7. The method according to claim 1 or 3, characterized in that: The method further comprises: Monitor the access frequency, last use time, and data type of hot and cold data in real time; If a first preset condition is met, adjusting the storage location of the hot data; wherein the first preset condition includes at least one of the following: the access frequency of the hot data is not higher than a first preset threshold, the last use time of the hot data is not lower than a second preset threshold, and the hot data belongs to historical data; and / or, When a second preset condition is met, the storage location of the cold data is adjusted; wherein the second preset condition includes at least one of the following: the access frequency of the cold data is higher than a first preset threshold, the last use time of the cold data is lower than a second preset threshold, and the cold data is real-time data.

8. A data processing device, characterized in that: The device comprises: A data acquisition module, used to acquire autonomous driving data of the autonomous driving vehicle; A storage module, used for storing the autonomous driving data in different storage locations according to categories; A receiving module, used for receiving a data read request sent by an external platform; An adjustment module is used to adjust the storage location of the autonomous driving data corresponding to the read data request.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the data processing method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the data processing method according to any one of claims 1 to 7 is implemented.

Images