Method, system and vehicle for processing data

Abstract

This application provides a data processing method, system, and vehicle. The method includes: acquiring driving data from different data acquisition platforms, wherein the driving data represents data collected by multiple types of sensors in a driving scenario; encapsulating the driving data to obtain structured intermediate data, wherein the structured intermediate data represents initial data units, and the initial data units are identified by a unified semantic identifier; standardizing the structured intermediate data to obtain first target data, wherein the first target data conforms to a standardized data structure, and the standardization degree of the first target data is higher than that of the structured intermediate data; performing time synchronization processing on the first target data to obtain second target data, wherein the second target data represents time-synchronized multimodal data frames; and performing data service processing on the second target data. This application solves the technical problem of ineffective data processing.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a data processing method, system, and vehicle. Background Technology

[0002] Currently, due to the lack of a unified data processing architecture, the heterogeneity of various data sources cannot be effectively masked, requiring upper-layer algorithm developers to adapt different data access interfaces for different acquisition systems. This not only increases development complexity but also leads to inconsistent training sample quality and unreproducible simulation results due to inconsistent data standards.

[0003] Furthermore, when the data source format is updated or the hardware platform changes, downstream applications that rely on this data must be modified synchronously, resulting in poor scalability of the architecture and high long-term maintenance costs, making it difficult to support the large-scale and standardized requirements of data-driven development for intelligent driving. Therefore, there are still technical problems that prevent the effective processing of data.

[0004] There is currently no good solution to the above problems. Summary of the Invention

[0005] This application provides a data processing method, system, and vehicle to at least solve the technical problem of the inability to effectively process data.

[0006] According to one aspect of the embodiments of this application, a data processing method is provided. The method may include: acquiring driving data from different data acquisition platforms, wherein the driving data represents data collected by multiple types of sensors in a driving scenario; encapsulating the driving data to obtain structured intermediate data, wherein the structured intermediate data represents initial data units, which are identified by a unified semantic identifier; standardizing the structured intermediate data to obtain first target data, wherein the first target data conforms to a standardized data structure, and the degree of standardization of the first target data is higher than that of the structured intermediate data; performing time synchronization processing on the first target data to obtain second target data, wherein the second target data represents time-synchronized multimodal data frames; and performing data service processing on the second target data.

[0007] Furthermore, the structured intermediate data is standardized to obtain the first target data, including: parsing the structured intermediate data to obtain initial semantic data, wherein the initial semantic data includes at least one of the following: the content, data type, time data, and metadata of the structured intermediate data; and transforming the initial semantic data to obtain the first target data.

[0008] Further, the structured intermediate data is standardized to obtain the first target data, including: calling the first data processing module to standardize the structured intermediate data to obtain the first target data; and performing time synchronization processing on the first target data to obtain the second target data, including: in response to the second data processing module receiving the first target data, aligning the first target data using the timestamp or global synchronization clock in the second data processing module, wherein the first data processing module and the second data processing module are connected through a data management bus; and reassembling the aligned first target data to obtain the second target data.

[0009] Furthermore, data service processing is performed on the second target data, including: performing at least one of the following processing on the second target data: query processing, export processing, and push processing.

[0010] Furthermore, the second target data is processed using data services, including: using a data service module to process the second target data; the data service module includes a target interface module, a data export plugin module, and a push module, and the method further includes one of the following: using the target interface module to query the second target data; using the data export plugin module to export the second target data; using the push module to push the second target data to the data access module or the data processing module.

[0011] Furthermore, the driving data is encapsulated to obtain structured intermediate data, including: parsing the driving data; and encapsulating the parsed driving data to obtain structured intermediate data.

[0012] Furthermore, the driving data is encapsulated to obtain structured intermediate data, including: responding to the driving data format being a standardized binary container format, using a first plugin to obtain first structural information of the driving data, and parsing the first structural information to obtain a first initial data unit, wherein the first initial data unit includes the semantic content of the driving data; responding to the driving data format being a custom format, using a second plugin to obtain second structural information of the driving data, and parsing the second structural information to obtain a second initial data unit, wherein the second initial data unit includes the semantic content of the driving data; the first initial data unit and the second initial data unit are encapsulated respectively to obtain structured intermediate data.

[0013] Furthermore, the method also includes: grouping the second target data; and storing the grouped second target data.

[0014] According to another aspect of the embodiments of this application, a data processing system is also provided, comprising: a data access module for acquiring driving data from different data acquisition platforms, wherein the driving data represents data collected by multiple types of sensors in a driving scenario; encapsulating the driving data to obtain structured intermediate data, wherein the structured intermediate data represents initial data units, and the initial data units are identified by a unified semantic identifier; a data processing module for standardizing the structured intermediate data to obtain first target data, wherein the first target data conforms to a standardized data structure, and the standardization degree of the first target data is higher than that of the structured intermediate data; performing time synchronization processing on the first target data to obtain second target data, wherein the second target data represents time-synchronized multimodal data frames; and a data service module for performing data service processing on the second target data.

[0015] According to another aspect of the embodiments of this application, a data processing apparatus is also provided, comprising: an acquisition unit for acquiring driving data from different data acquisition platforms, wherein the driving data represents data collected by multiple types of sensors in a driving scenario; an encapsulation unit for encapsulating the driving data to obtain structured intermediate data, wherein the structured intermediate data represents initial data units, and the initial data units are identified by a unified semantic identifier; a first processing unit for standardizing the structured intermediate data to obtain first target data, wherein the first target data conforms to a standardized data structure, and the standardization degree of the first target data is higher than that of the structured intermediate data; a second processing unit for time synchronization processing of the first target data to obtain second target data, wherein the second target data represents a time-synchronized multimodal data frame; and a third processing unit for data service processing of the second target data.

[0016] According to another aspect of the embodiments of this application, a processor is also provided. The processor is used to run a program, wherein the program is executed by the processor to perform the methods described in the embodiments of this application.

[0017] According to another aspect of the embodiments of this application, a vehicle is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of this application when it runs.

[0018] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of this application.

[0019] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods of various embodiments of this application.

[0020] According to another aspect of the embodiments of this application, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods in various embodiments of this application.

[0021] According to another aspect of the embodiments of this application, a computer program is also provided, which, when executed by a processor, implements the methods of the various embodiments of this application.

[0022] In this embodiment, if it is necessary to process data in the vehicle, driving data from different data acquisition platforms can be obtained. This driving data represents data collected by multiple sensors in a driving scenario. The driving data can be encapsulated to obtain structured intermediate data, which represents initial data units identified by a unified semantic identifier. The structured intermediate data can be standardized to obtain first target data, which conforms to a standardized data structure and has a higher degree of standardization than the structured intermediate data. The first target data can be time-synchronized to obtain second target data, which represents time-synchronized multimodal data frames. Finally, the second target data can be processed for data services. In other words, this application embodiment acquires driving data from different data acquisition platforms in driving scenarios. Then, the driving data can be encapsulated to obtain structured intermediate data identified by a unified semantic identifier. After standardizing the structured intermediate data, the first target data is obtained. The first target data is then time-synchronized to ensure that the data conforms to the standardized data structure and is time-synchronized. This enables data service processing of the second target data, overcoming the limitations of the inability to effectively shield the heterogeneity of various data sources and the inconsistent data standards leading to inconsistent quality. This solves the technical problem of ineffective data processing and achieves the technical effect of effective data processing. Attached Figure Description

[0023] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0024] Figure 1 This is a flowchart of a data processing method according to an embodiment of this application;

[0025] Figure 2 This is a schematic diagram of an architecture system for general data acquisition and processing of intelligent driving according to an embodiment of this application;

[0026] Figure 3 This is a schematic diagram of a data processing system according to an embodiment of this application;

[0027] Figure 4 This is a schematic diagram of a data processing apparatus according to an embodiment of this application. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0030] According to an embodiment of this application, a data processing method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0031] This embodiment provides a data processing method. Figure 1 This is a flowchart of a data processing method according to an embodiment of this application, such as... Figure 1 As shown, the process may include the following steps.

[0032] Step S102: Obtain driving data from different data collection platforms.

[0033] In the technical solution provided in step S102 of this application, driving data can be used to represent data collected by multiple types of sensors during a vehicle's driving scenario. This driving data can include raw data synchronously collected by multiple types of sensors during vehicle operation, including but not limited to: image frames output by cameras (e.g., image data), three-dimensional point clouds generated by lidar (e.g., point cloud data), a list of targets detected by millimeter-wave radar, acceleration and angular velocity provided by an inertial measurement unit (IMU), pose information output by a Global Navigation Satellite System (GNSS) and a high-precision positioning module, vehicle controller area network (CAN) signals, or control signals such as vehicle speed, steering angle, gear position, and braking status transmitted on an Ethernet bus, as well as metadata such as timestamps, device identifiers (IDs), and sampling frequencies corresponding to each sensor.

[0034] In this embodiment, driving data from different data acquisition platforms is acquired. For example, during the development, testing or verification of the vehicle's intelligent driving system, the original data recorded by the vehicle in real or simulated driving scenarios can be actively read and completely extracted from various heterogeneous data acquisition devices or software systems.

[0035] Optionally, the aforementioned data acquisition platform may include an onboard recording system based on Robot Operating System (ROS), an Advanced Data Tracking Framework (ADTF) analysis platform, a CAN bus recorder in Measurement Data Format version 4 (MDF4) format, a manufacturer-defined binary acquisition device, an onboard black box, etc. These data acquisition platforms can be deployed independently on the test vehicle, each employing different file encapsulation formats, communication protocols, and data encoding methods. This is merely an illustrative example and does not impose specific limitations on the content of the data acquisition platform.

[0036] Optionally, the aforementioned driving data is not an intermediate result that has undergone preprocessing or cleaning, but rather raw, unprocessed, high-fidelity data, providing a data foundation for subsequent driving data processing.

[0037] In this embodiment, after acquiring driving data from different data acquisition platforms, the data format of the driving data can be automatically identified through a pre-built multi-type data access plugin. For example, the data format of the driving data can be determined by the file extension, header signature, or metadata identifier, and the corresponding parsing engine can be called to complete the file reading and raw stream extraction of the driving data, ensuring that no sensor channel or time series data is missed.

[0038] Step S104: Encapsulate the driving data to obtain structured intermediate data.

[0039] In the technical solution provided in step S104 of this application, the structured intermediate data can be used to represent the initial data unit, which is identified by a unified semantic identifier.

[0040] In this embodiment, after acquiring driving data from different data acquisition platforms, the driving data can be encapsulated to obtain structured intermediate data, such as an intermediate general data object. This intermediate general data object retains the basic information and data body of the driving data (e.g., the original data), but removes the specific encapsulation format of the driving data.

[0041] Optionally, encapsulating driving data to obtain structured intermediate data involves removing the original data acquisition platform's unique file structure, protocol header, encoding method, or proprietary container format after acquiring the driving data. Instead, it extracts the individual data units from the driving data. These individual data units can be initial data units, such as an image frame, a point cloud set, or a CAN signal message. These individual data units are then reorganized and packaged according to unified semantic information to generate a lightweight, platform-independent intermediate data representation, i.e., structured intermediate data.

[0042] Optionally, the aforementioned structured intermediate data is not a simple copy of the original data, but rather encapsulates the core semantic information of each data unit in the form of structured objects. Each object can contain a set of standardized and unified semantic identifiers. For example, data types include "image", "pointcloud", and "can_signal". Sensor identifiers include "cam_front_01" for the front-facing camera and "lidar_top" for the top-mounted LiDAR, etc. These are just examples and no specific limitations are imposed here.

[0043] In this embodiment, by encapsulating driving data to obtain structured intermediate data, the tightly coupled encapsulation shell between the driving data and the data acquisition platform is removed, achieving decoupling of data semantics and data carrier. Through unified semantic identification, the complexity of data parsing is significantly reduced, and a programmable, verifiable, and traceable unified input foundation is provided for subsequent standardized processing. As an intermediate abstraction layer, the structured intermediate data allows for the development of a new parsing plugin to map driving data to a unified structure when a new driving data source is added, without modifying other processing modules. This significantly improves the scalability and maintainability of the data processing system.

[0044] Step S106: Standardize the structured intermediate data to obtain the first target data.

[0045] In the technical solution provided by step S106 of this application, the first target data conforms to a standardized data structure, and the standardization degree of the first target data is higher than that of the structured intermediate data.

[0046] In this embodiment, after encapsulating the driving data to obtain structured intermediate data, the structured intermediate data can be standardized to obtain the first target data. The first target data can be fully standardized data.

[0047] Optionally, the structured intermediate data is standardized by further aligning and standardizing the semantic connotation, physical expression and system specifications of each initial data unit according to the preset internal unified data standard, based on the structured intermediate data defined by the unified semantic identifier, so as to generate the first target data that meets the requirements.

[0048] For example, image data from different sensors is standardized to a red, green, blue (RGB) or blue, green, red (BGR) channel order, with pixels classified as 8-bit unsigned integers and a unified resolution benchmark. Point cloud data is standardized to a right-handed coordinate system with "front, left, top, and sky" as the origin, and the field order of points is standardized to x, y, z, intensity, and timestamp, with units of meters and nanoseconds. Vehicle speed in CAN signals is converted from "km / h" to "m / s", angles from "degrees" to "radians", and pose data is standardized to represent orientation as quaternions. Timestamps are standardized to Coordinated Universal Time (UTC) nanosecond-level absolute time benchmark to eliminate errors caused by system clock drift or time zone differences. The naming of each field, data structure, unit definition, and coordinate system convention follows an internally published data dictionary to ensure that the semantic and numerical meanings of the same type of data remain consistent across different sources, projects, and times.

[0049] In this embodiment, the structured intermediate data possesses a unified semantic identifier, but the data body of the structured intermediate data still retains the diversity of the original data, with potential ambiguities such as inconsistent coordinate systems, inconsistent units, and mismatched precision. This will directly lead to algorithm training failure, unreproducible simulation results, and misjudgments during multi-team collaboration. The first target data, through standardization, ensures consistency in semantics, structure, dimensions, and spatiotemporal reference among the structured intermediate data. By standardizing the structured intermediate data, the first target data is obtained, providing a highly reliable, reusable, and comparable data foundation for subsequent data processing.

[0050] Step S108: Perform time synchronization processing on the first target data to obtain the second target data.

[0051] In the technical solution provided by step S108 of this application, the second target data can be used to represent a time-synchronized multimodal data frame.

[0052] In this embodiment, after standardizing the structured intermediate data to obtain the first target data, the first target data can be time-synchronized to obtain the second target data, such as a multimodal data frame that is strictly synchronized in time.

[0053] Optionally, due to differences in sampling frequencies, hardware clock drift, data transmission delays, and system scheduling among different sensors, even after unifying the coordinate system and units of the structured intermediate data, the first target data may still have millisecond or even microsecond-level offsets on the time axis. For example, a camera outputs an image frame every 40 milliseconds, a LiDAR scans a circle every 100 milliseconds, and a CAN signal is continuously updated at a frequency of 10 milliseconds. If these first target data are directly spliced ​​together in the order of acquisition, it will lead to a mismatch between the "vehicle position in the image" and the "actual position in the point cloud" in the perception result, thereby causing serious problems such as target tracking drift and inaccurate trajectory prediction. Therefore, the first target data can be time-synchronized. Based on the fact that the semantic structure and physical units of the data from each sensor have been unified, the first target data is cross-modal time-aligned and reassembled at the frame level according to the high-precision timestamps carried by each sensor, thereby generating a set of multimodal data frames that are strictly synchronized in the time dimension, i.e., the second target data.

[0054] Optionally, time synchronization processing constructs a global time base with nanosecond-level precision, mapping data from various sensors onto the same time axis. Algorithms such as interpolation, timestamp matching, or sliding window alignment can be used to aggregate valid data units within the same time window (e.g., with a ±5 millisecond tolerance) into a complete data frame. Each second target data frame contains complete sensing elements synchronized at that moment, including the image, point cloud, radar target, positioning pose, and CAN signal.

[0055] Optionally, during the time synchronization processing of the first target data to obtain the second target data, intelligent compensation can be performed on missing or abnormal first target data. For example, when the lidar is not triggered, the nearest valid pose interpolation is used to complete the data frame, ensuring the continuity and availability of the data frame. Specifically, during multi-sensor data synchronization, when the pose data of a certain sensor is lost or abnormal, the nearest valid pose value in time is used. A linear or higher-order interpolation algorithm is then used to dynamically estimate the pose information at the missing moment to complete the data frame, ensuring the integrity and continuity of the data stream after time alignment.

[0056] In this embodiment of the application, the second target data is obtained by performing time synchronization processing on the first target data, which solves the problem of data asynchrony in the intelligent driving system of the vehicle and enables the multi-sensor fusion algorithm to make decisions based on environmental information at the same moment.

[0057] Step S110: Perform data service processing on the second target data.

[0058] In the technical solution provided by step S110 of this application, after time synchronization processing of the first target data to obtain the second target data, data service processing can be performed on the second target data.

[0059] In this embodiment, the data service processing of the second target data is carried out by using the time-synchronized multimodal data frames as a unified data asset to build a standardized access and distribution capability for upper-layer applications. This enables various downstream systems such as algorithm development, simulation testing, data annotation, and system verification to obtain the required data on demand through a consistent, efficient, and scalable interface, ignoring the original source, processing process, or storage location.

[0060] Optionally, by building a complete service system, such as a data service layer including a unified data access application programming interface (API), a data query engine, export plugins, and real-time streaming services, the second target data can be transformed into data services that can be invoked, retrieved, and shared.

[0061] Optionally, by providing a multi-dimensional query interface based on timestamp, sensor type, scene label, or vehicle ID, a complete data frame containing "forward-looking camera image, LiDAR point cloud, and vehicle longitudinal velocity at 14:23:15.123456789 nanoseconds on May 10, 2024" can be requested with a single line of code. Associated synchronous data frames can be automatically retrieved and exported according to semantic scenes (e.g., "pedestrians crossing at an intersection in rainy weather"). The simulation platform can subscribe to continuous time-series sensor data through a real-time streaming interface to achieve high-fidelity closed-loop simulation. When collaborating with third parties, data can be exported to a universal standard format with a single click, enabling seamless data exchange. The aforementioned data services support access control, metadata management, data version tracking, and access log recording to ensure the security and auditability of data usage.

[0062] In this embodiment, by processing the second target data through data services, data silos are broken down, transforming driving data into intelligent assets that can be flexibly invoked and efficiently reused. This eliminates the need to write cumbersome parsing code to adapt to different data sources and avoids the maintenance burden caused by data format changes. The system can operate by connecting to a stable, clear, and well-documented service interface.

[0063] It should be noted that in the data processing method of this embodiment, the vehicle can also interact with roadside equipment and terminal equipment. Optionally, the vehicle can send an information subscription request to the roadside equipment. This message subscription request can include specific types of information that the vehicle needs to receive, such as road conditions, traffic signal status, and obstacle warnings ahead. In response to the information subscription request, the roadside equipment can send roadside perception information to the vehicle. For example, the roadside equipment will filter out roadside perception information that meets the vehicle's needs based on its own perception capabilities and stored information, and send it to the vehicle at a certain frequency. In addition to communicating with the roadside equipment, the vehicle can also receive driving scenario switching instructions transmitted by the terminal equipment through the network. For example, the driving scenario switching instructions can be used to switch the vehicle to energy-saving mode, sport mode, autonomous driving mode, etc., so that the vehicle can adapt to new driving scenarios.

[0064] Through steps S102 to S110 above, if it is necessary to process the data in the vehicle, driving data from different data acquisition platforms can be obtained. This driving data represents data collected by multiple sensors in a driving scenario. The driving data can be encapsulated to obtain structured intermediate data, which represents initial data units identified by a unified semantic identifier. The structured intermediate data can be standardized to obtain first target data, which conforms to a standardized data structure and has a higher degree of standardization than the structured intermediate data. The first target data can be time-synchronized to obtain second target data, which represents time-synchronized multimodal data frames. Data service processing can then be performed on the second target data. In other words, this application embodiment acquires driving data from different data acquisition platforms in driving scenarios. Then, the driving data can be encapsulated to obtain structured intermediate data identified by a unified semantic identifier. After standardizing the structured intermediate data, the first target data is obtained. The first target data is then time-synchronized to ensure that the data conforms to the standardized data structure and is time-synchronized. This enables data service processing of the second target data, overcoming the limitations of the inability to effectively shield the heterogeneity of various data sources and the inconsistent data standards leading to inconsistent quality. This solves the technical problem of ineffective data processing and achieves the technical effect of effective data processing.

[0065] The above-mentioned method of this application will be further described below.

[0066] As an optional implementation, step S106, standardizing the structured intermediate data to obtain the first target data, includes: parsing the structured intermediate data to obtain initial semantic data, wherein the initial semantic data includes at least one of the following: the content, data type, time data, and metadata of the structured intermediate data; and transforming the initial semantic data to obtain the first target data.

[0067] In this embodiment, during the process of standardizing the structured intermediate data to obtain the first target data, the structured intermediate data can be parsed and transformed to obtain the first target data.

[0068] Optionally, the structured intermediate data is parsed to extract the original elements constituting the semantic essence of the data, i.e., the initial semantic data. This initial semantic data is not a replica of the original acquisition format, but rather the core information units stripped, encapsulated, and restored from the structured intermediate data. The initial semantic data may include the specific content of the data (e.g., image pixel arrays, point cloud coordinate sets, CAN message values), explicit data types (e.g., image data "camera_image", point cloud data "lidar_pointcloud", vehicle speed "vehicle_speed"), high-precision timestamps (nanosecond-level absolute time), and metadata that helps describe the data attributes (e.g., sensor model, resolution, coordinate system origin, sampling frequency, measurement range, unit definition). These elements together constitute the semantic skeleton of the data and serve as the basis for subsequent standardization conversion.

[0069] Optionally, after obtaining the initial semantic data, semantic enhancement and physical alignment transformation operations can be performed on each type of data according to a preset internal unified data standard. For example, all image data can be uniformly decoded into an RGB channel order, 8-bit precision structure, and a standard coordinate system label can be attached, such as "FRONT_LEFT_UP" for the front left-top coordinate system. The coordinates in the point cloud data can be converted from any original coordinate system to a unified "vehicle front left-top" right-handed coordinate system, with the point fields fixedly sorted by x, y, z, reflection intensity, and timestamp, and the units forced to be meters and nanoseconds. The vehicle speed in the CAN signal can be converted from "km / h" to "m / s", and the angle can be converted from "degree" to "radian", with a unified naming convention of vehicle speed (meters per second) "vehicle_speed_m_s" and heading angle (radians) "heading_rad". The timestamp is unified to UTC reference to eliminate device clock differences.

[0070] Optionally, during the conversion process, the rationality of the data can be automatically verified. For example, out-of-range values, missing interpolation frames, and missing metadata can be removed to ensure that the output first target data is not only in a uniform format, but also consistent with the numerical meaning, physical meaning and system context.

[0071] In this embodiment, the above steps achieve a transformation from "semantically identifiable" to "semantically computable." While structured intermediate data possesses a unified identifier, its content still retains the diversity and uncertainty of the original data collection, making it unsuitable for direct algorithm training or system testing. However, by standardizing the structured intermediate data, the resulting first target data achieves a high degree of consistency across the semantic, numerical, unit, and spatiotemporal layers, eliminating cognitive ambiguity and computational errors caused by differences in data sources.

[0072] As an optional implementation, step S106, standardizing the structured intermediate data to obtain first target data, includes: calling the first data processing module to standardize the structured intermediate data to obtain first target data; step S108, performing time synchronization processing on the first target data to obtain second target data, includes: in response to the second data processing module receiving the first target data, aligning the first target data using the timestamp or global synchronization clock in the second data processing module, wherein the first data processing module and the second data processing module are connected through a data management bus; and reassembling the aligned first target data to obtain second target data.

[0073] In this embodiment, the structured intermediate data is standardized to obtain the first target data, which can be accomplished by calling the first data processing module. The first data processing module can be a data parsing and standardization module.

[0074] Optionally, the first data processing module incorporates conversion rules and algorithm logic that are completely consistent with the internal unified data standard, and can receive structured intermediate data. By semantically recognizing the type, content, and metadata of each data unit, it automatically performs standardized operations such as format conversion, unit normalization, coordinate system correction, and time base unification, ultimately outputting the first target data that meets the requirements.

[0075] Optionally, the first data processing module adopts a plug-in design, with different data types (e.g., images, point clouds, CAN signals) corresponding to independent sub-processing engines. These engines can be independently upgraded, reused, and verified, ensuring that the standardization process is both flexible and reliable. For example, the first data processing module parses and transforms intermediate common data objects according to a predefined internal unified data standard. The internal unified data standard specifies the data structure, coordinate system, units, field naming, etc., for example, unifying camera images into OpenCV matrix (OpenCV Mat) objects in BGR or RGB format, with a unified timestamp and coordinate system label.

[0076] Optionally, after obtaining the first target data, time synchronization processing can be performed on the first target data. The timestamps in the second data processing module or a global synchronization clock can be used to align the first target data. For example, when the first target data is pushed to the second data processing module via the data management bus, the second data processing module can read the nanosecond-level timestamp inherent in each data entry and combine it with the system-level global synchronization clock, such as a Precision Time Protocol (PTP) or Global Positioning System (GPS) time reference, to perform precise time alignment of the various sensor data.

[0077] Optionally, the alignment process is not a simple sorting, but rather uses dynamic window matching, interpolation compensation, and tolerance control algorithms to precisely aggregate images, point clouds, positioning, and CAN signals distributed in different sampling periods into multimodal data frames that are highly consistent in time.

[0078] Optionally, after alignment is completed, the aligned first target data can be recombined to obtain the second target data. The sensor data within the same time window are encapsulated into a structurally complete and semantically consistent data frame unit, namely the second target data.

[0079] Optionally, the first data processing module and the second data processing module are completely decoupled and can transmit data asynchronously, with high throughput and low latency through the data management bus, ensuring that the standardization and synchronous processing stages do not block each other and evolve independently.

[0080] In this embodiment, a first data processing module, using a data management bus as the communication channel, enables the replacement or enhancement of corresponding modules when a new sensor is added or the time synchronization algorithm is upgraded, without requiring a complete reconstruction of the entire processing chain. Simultaneously, the data management bus ensures the reliability and orderliness of data flow, maintaining stable processing throughput even in complex scenarios with high concurrency and multi-source asynchronous input. This modular, bus-based design allows for independent observation, logging, and performance evaluation of data output, providing a foundation for intelligent driving data processing in vehicles.

[0081] As an optional implementation, step S110 involves performing data service processing on the second target data, including performing at least one of the following processing on the second target data: query processing, export processing, and push processing.

[0082] In this embodiment, the data service processing of the second target data can be carried out using time-synchronized multimodal data frames as a unified data asset. Through three core service mechanisms—query, export, and push—flexible, efficient, and on-demand data access capabilities can be provided to various upper-layer applications.

[0083] Optionally, query processing can be designed for offline analysis and data exploration scenarios. Based on a semantic query interface with multiple dimensions such as timestamps, sensor types, scene tags, vehicle IDs, and environmental conditions, users can precisely request complete data frames within a specific time window through a unified API. For example, "Retrieve synchronized data from the forward-facing camera, LiDAR, and high-precision positioning within ±50 milliseconds before and after 14:23:15.123456789 nanoseconds on June 15, 2024." Millisecond-level response times can be achieved through metadata indexing and a high-efficiency storage engine, supporting fuzzy matching, conditional filtering, and result pagination, greatly improving data retrieval efficiency and enabling rapid identification of abnormal samples, construction of test cases, or analysis of specific scenarios.

[0084] Optionally, the export process can be tailored to data exchange and standards integration scenarios. Various export plugins can convert secondary target data into universal formats as needed, such as ROS2 Bag files, OpenX standards (OpenLABEL, OpenSCENARIO), and Autoware Universe compatible formats, automatically preserving time synchronization relationships, sensor metadata, and semantic annotation information. This allows for seamless data output to third-party evaluation platforms, partner manufacturers, or open-source communities without manual intervention or customized scripts, achieving standardization once and exporting multiple times, breaking down data silos and accelerating industry collaboration. OpenLABEL is an open format used to annotate and describe autonomous driving perception data (e.g., object categories, bounding boxes, and semantic labels in images and point clouds). OpenSCENARIO is an Extensible Markup Language (XML) format standard used to describe autonomous driving simulation scenarios (e.g., vehicle lane changes, pedestrian crossings, weather conditions, and traffic signal changes).

[0085] Optionally, push processing can serve real-time systems and online algorithm verification scenarios. Secondary target data can be continuously pushed to online algorithm modules, simulation platforms, or visualization tools in real-time streams, supporting low latency, high throughput, breakpoint resumption, and subscription mechanisms. For example, a vehicle's autonomous driving decision-making system can receive synchronous perception data streams in real-time during the testing phase for closed-loop verification. Training platforms can collect and push data simultaneously, enabling data-driven online learning. The push service can also support access control and bandwidth management to ensure that critical tasks receive data first.

[0086] In this embodiment, the above steps transform passively stored data packets into actively serviced data assets. Through the coordinated use of query, export, and push mechanisms, users no longer need to download original files, write parsing code, or manually align timelines. Instead, they can obtain data through standard interfaces, much like calling a cloud service. This not only significantly reduces the access costs for upper-layer applications and improves development efficiency, but also standardizes data usage behavior through a unified service interface, enhancing data traceability, security, and compliance.

[0087] As an optional implementation, step S110, performing data service processing on the second target data, includes: using a data service module to perform data service processing on the second target data; the data service module includes a target interface module, a data export plugin module, and a push module, and the method further includes one of the following: using the target interface module to query the second target data; using the data export plugin module to export the second target data; using the push module to push the second target data to the data access module or the data processing module.

[0088] In this embodiment, data service processing of the second target data can be implemented through a unified data service module. The data service module serves as the top-level interaction entry point of the data processing architecture, not directly participating in data parsing or synchronization, but rather acting as an intelligent service hub connecting the underlying standardized data with upper-level applications. Internally, the data service module can integrate three core components: a target interface module, a data export plugin module, and a push module. These correspond to three typical use cases: querying, exporting, and real-time pushing, respectively, forming a complete, flexible, and service-oriented data service system.

[0089] Optionally, when the algorithm team, simulation platform, or testing system needs to use the data, they no longer need to directly access the original files or write parsing scripts. Instead, they can initiate semantic query requests through the target interface module, such as "retrieving frame data synchronized between the forward-looking camera and LiDAR within a certain time period" or "retrieving multimodal data frames containing pedestrian crossing scenes." The target interface module can quickly locate and return complete data frames that meet the requirements based on the metadata index of the second target data. It supports multi-dimensional conditional queries such as time range, sensor type, scene label, and vehicle status. It is highly efficient and has a unified interface, enabling developers to obtain high-quality data as if calling a database, greatly reducing the threshold for data acquisition.

[0090] Optionally, for scenarios requiring integration with external systems or industry standards, the data export plugin module offers multiple output formats. It can convert secondary target data into common formats such as ROS2 Bag, OpenLABEL, and OpenSCENARIO as needed, automatically preserving time synchronization relationships, sensor calibration parameters, and semantic tags. This ensures that the exported data can still be correctly parsed and used on external platforms. This reduces the problems of manual conversion, data loss, or time misalignment caused by format incompatibility, enabling seamless integration of data into external evaluation systems, collaborative projects, or open-source ecosystems, achieving standardization once and export multiple times.

[0091] Optionally, in scenarios with high real-time requirements, such as online algorithm verification, closed-loop simulation, or in-vehicle system debugging, the push module can continuously and with low latency push the second target data to the subscriber in a real-time streaming manner. For example, synchronized perception frames can be sent to the autonomous driving decision-making module or visualization platform in real time. This push module can support on-demand subscription, bandwidth control, breakpoint resumption, and security authentication to ensure that critical tasks receive data first, while avoiding network congestion.

[0092] For example, the target interface module can include a unified data access interface, providing a standard API that allows upper-layer applications to query and retrieve standardized data by timestamp, sensor type, topic, etc. The data export plugin module can provide various output plugins to easily export internal standardized data to common formats, such as ROS2 Bag, OpenX standards (OpenLABEL, OpenSCENARIO), and AutowareUniverse compatible formats, to meet data exchange needs. The push module can support real-time data streaming services to push standardized data as a real-time stream to online algorithm modules or visualization tools.

[0093] In this embodiment, the previously statically and passively stored second target data is transformed into a data service asset that can be actively consumed and delivered on demand through the above processing. By coordinating the three service mechanisms and declaring requirements according to standard interfaces, data query processing, export processing, and push processing can be completed automatically.

[0094] As an optional implementation, step S104 encapsulates the driving data to obtain structured intermediate data, including: parsing the driving data; and encapsulating the parsed driving data to obtain structured intermediate data.

[0095] In this embodiment, during the process of encapsulating driving data to obtain structured intermediate data, deep parsing of driving data from different acquisition platforms can be performed based on the private format characteristics of the driving data. For example, the message header of the ROS Bag file, the frame structure of the Advanced Driver Assistance Systems Tool Data File (ADTF DAT), and the protocol header and byte layout of custom binary files are considered. The corresponding parsing engine is dynamically loaded to extract the core content from the original data line by line, such as key fields like original sensor sampling values, timestamps, device identifiers, sampling frequency, and data length. During the parsing process, lossless restoration of the driving data can be maintained, without semantic modification or content deletion; only the original encapsulation form is stripped away. For example, the ROS namespace, the ADTF container structure, or vendor-specific header and footer information are extracted, thus obtaining a set of exposed but complete data elements.

[0096] Optionally, after parsing, these exposed data elements can be uniformly encapsulated to generate structured intermediate data. This structured intermediate data adopts a platform-independent, semantically clear, internally common data object format. Each piece of structured intermediate data can contain three core pieces of information. First, the content body of the driving data, such as an image pixel array, a point cloud coordinate list, or CAN message values. Second, the metadata of the driving data source, such as sensor type, acquisition device ID, original format identifier, sampling frequency, and original timestamp. Third, standardized structural identifiers, such as unified type labels (e.g., "IMAGE_RAW" for image data, "POINTCLOUD_3D" for point cloud data), data dimension definitions, and data encoding methods.

[0097] Optionally, the parsed driving data can be encapsulated. This encapsulation does not involve semantic conversion or unit normalization, but rather unifies the originally different data formats into a universal intermediate form that is recognizable, routable, and traceable within the system.

[0098] In this embodiment, structured intermediate data serves as an intermediate carrier, preserving the information integrity of driving data and providing a traceable and verifiable input foundation for subsequent standardization and synchronization processing. When data anomalies or algorithm deviations occur, the process can be traced back to the intermediate data layer to clearly identify whether the problem lies in the original data acquisition, parsing errors, or subsequent processing defects, greatly improving problem localization efficiency. This design of encapsulating before transforming avoids the risk of misjudgment caused by performing semantic processing before the data is fully parsed, ensuring the rigor and reliability of the entire processing chain.

[0099] As an optional implementation, step S104 encapsulates the driving data to obtain structured intermediate data, including: in response to the driving data being in a standardized binary container format, using a first plugin to obtain first structural information of the driving data, and parsing the first structural information to obtain a first initial data unit, wherein the first initial data unit includes the semantic content of the driving data; in response to the driving data being in a custom format, using a second plugin to obtain second structural information of the driving data, and parsing the second structural information to obtain a second initial data unit, wherein the second initial data unit includes the semantic content of the driving data; and encapsulating the first initial data unit and the second initial data unit respectively to obtain structured intermediate data.

[0100] In this embodiment, a plug-in mechanism can be used to achieve seamless and compatible processing of driving data from heterogeneous data sources. The first plug-in can be a ROS Bag access plug-in. The second plug-in can be a custom binary access plug-in.

[0101] Optionally, when the input driving data is detected to be in a standardized binary container format (e.g., ROS Bag format), the corresponding first plugin can be automatically loaded. This first plugin has complete parsing capabilities for this standardized binary container format, and can identify the internal organization methods such as file headers, message indexes, and data flow structures to extract the first structural information of the data. The first structural information may include meta-structural information such as message type, channel identifier, timestamp sequence, and data payload location. Subsequently, the first plugin can parse the first structural information line by line according to the predefined format specifications to reconstruct the first initial data unit carried by the driving data. The first initial data unit not only contains the original sensor sampling values ​​(e.g., image pixels, point cloud coordinates, CAN messages), but also retains the semantic context, such as sensor name, data source device ID, and original sampling frequency, ensuring that the semantic integrity of the data is not lost due to format stripping.

[0102] Optionally, when the driving data is identified as being in a custom format, i.e., originating from a manufacturer-defined proprietary binary format, a second plugin is dynamically loaded. This second plugin, based on a pre-configured protocol document or reverse engineering model, can identify unique file header signatures, data block length encoding, field offset layouts, and time synchronization mechanisms, thereby extracting second structural information. During parsing, the second plugin does not assume the semantic meaning of the data; instead, it uses rule mapping and field matching to restore the original byte stream into a second initial data unit structurally isomorphic to the first initial data unit. This also fully preserves the semantic content of the original data; that is, regardless of whether the source is a black-box device or a closed system, the final output is a fragment of original data with a unified semantic expression.

[0103] Optionally, after obtaining the first and second initial data units, a unified encapsulation process can be performed. The encapsulation process does not change the data content itself, but rather packages these initial data units into a platform-independent, well-structured intermediate object to obtain structured intermediate data.

[0104] For example, when a data packet from platform A is identified as being in ROS Bag format, the ROS Bag access plugin is automatically loaded and invoked. This plugin reads the bag file, parses out sensor image messages (sensor_msgs / Image), sensor point cloud messages (sensor_msgs / PointCloud2), etc., and encapsulates them into intermediate general data objects. When a data packet from vendor B is identified as being in a proprietary format, the corresponding custom binary access plugin is loaded. This custom binary access plugin, based on the known file structure and protocol, parses out images, CAN signals, etc., and similarly encapsulates them into intermediate general data objects.

[0105] In this embodiment, a plug-in design enables plug-and-play capability for unknown or future data sources without modifying the core architecture. New devices can be supported simply by adding corresponding plug-ins. Furthermore, since data is restored to a unified semantic unit before entering core processing, errors caused by incomplete format parsing or semantic misinterpretation are reduced.

[0106] As an optional implementation, the method further includes: grouping the second target data; and storing the grouped second target data.

[0107] In this embodiment, the second target data can be grouped and the grouped data can be persistently stored.

[0108] Optionally, after obtaining time-synchronized and semantically unified second target data, the second target data can be intelligently classified and organized according to the semantic context and engineering application scenario to form a data set with clear semantic boundaries and searchable attributes.

[0109] Optionally, grouping criteria may include, but are not limited to: driving scenario type (e.g., urban congestion, highway following, left turn at intersection), environmental conditions (e.g., day / night, sunny / rainy weather), vehicle status (e.g., emergency braking, lane keeping assist activated), sensor configuration, data collection time window (e.g., one data segment every 5 minutes), or test task number. Each group is assigned metadata identifiers, including key attributes such as time range, geographical location, scenario label, data collection vehicle ID, sensor version, and data quality score, forming a structured index of the data segments.

[0110] Optionally, after grouping, the second target data and associated metadata can be stored in a high-performance object storage system. Hierarchical directories or distributed hash mechanisms can be used for management to ensure efficient writing and fast retrieval of large amounts of data.

[0111] Optionally, the stored procedure supports data compression, index pre-generation, and metadata snapshots, while preserving the integrity of the original data frames for subsequent playback, annotation, training, or auditing. The grouped data no longer exists as a stack of original files, but becomes a semantic data asset that can be quickly located and accessed by algorithm engineers, testing teams, or data platforms using keywords, tags, time intervals, and other methods.

[0112] In this embodiment, by grouping the second target data and persistently storing the grouped data, the availability and management efficiency of the data are improved. Through grouping and structured storage, the original data is transformed into a data asset library that can be semantically retrieved and reused in various scenarios, shortening the time required to obtain data for specific scenarios.

[0113] The data processing method of this application embodiment, if it is necessary to process data in a vehicle, can acquire driving data from different data acquisition platforms, wherein the driving data represents data collected by multiple types of sensors in a driving scenario; the driving data can be encapsulated to obtain structured intermediate data, wherein the structured intermediate data represents initial data units, and the initial data units are identified by a unified semantic identifier; the structured intermediate data can be standardized to obtain first target data, wherein the first target data conforms to a standardized data structure, and the standardization degree of the first target data is higher than that of the structured intermediate data; the first target data can be time-synchronized to obtain second target data, wherein the second target data represents time-synchronized multimodal data frames; and the second target data can be processed for data services. In other words, this application embodiment acquires driving data from different data acquisition platforms in driving scenarios. Then, the driving data can be encapsulated to obtain structured intermediate data identified by a unified semantic identifier. After standardizing the structured intermediate data, the first target data is obtained. The first target data is then time-synchronized to ensure that the data conforms to the standardized data structure and is time-synchronized. This enables data service processing of the second target data, overcoming the limitations of the inability to effectively shield the heterogeneity of various data sources and the inconsistent data standards leading to inconsistent quality. This solves the technical problem of ineffective data processing and achieves the technical effect of effective data processing.

[0114] The above technical solutions of the embodiments of this application will be further illustrated below with reference to preferred embodiments.

[0115] Currently, data-driven approaches have become the core of algorithm iteration and system optimization. During the development, testing, and verification of intelligent vehicles, a large amount of data needs to be recorded through various data acquisition systems. This data includes, but is not limited to, camera images, LiDAR point clouds, millimeter-wave radar targets, inertial navigation unit information, vehicle bus signals, and time synchronization information.

[0116] The following are the main technical problems encountered in the relevant technologies.

[0117] Inconsistent data formats exist. Data acquisition platforms from different vendors, and data acquisition hardware and software at different stages, result in significant differences in the output data packet formats, encoding methods, and storage structures. For example, some use ROS Bag, some use ADTFDAT, and some use custom binary formats, which makes it impossible for data to be directly interoperable.

[0118] Inconsistent data standards exist. Even with similar data formats, data from different sources can differ significantly in terms of coordinate system definitions, data units, signal naming, timestamp precision, and synchronization strategies. For example, coordinate systems may have different conventions, such as front left at the top and front right at the bottom.

[0119] The processing workflow is fragmented. A dedicated data parsing, preprocessing, and visualization toolchain needs to be developed for each specific data source. This results in a large amount of repetitive development work, high maintenance costs, and difficulty in standardizing the data processing workflow.

[0120] Data exchange is difficult. Data from different project teams and at different stages of development is hard to seamlessly integrate. Exchanging data with partners and third-party organizations requires a lot of manpower for data conversion and integration, resulting in low efficiency.

[0121] In summary, existing solutions typically involve writing one-off transformation scripts for specific data sources or relying on a closed commercial software platform. These methods lack scalability and universality, and cannot fundamentally solve the problem of multi-source data fusion and general processing.

[0122] To address the aforementioned issues, this application provides a general data acquisition and processing architecture for intelligent driving. Through a pluggable data access layer, support for new data sources can be extended; only new access plugins need to be developed, without modifying the core architecture. By enforcing standardization in the core processing layer, data adheres to the same set of specifications, eliminating data ambiguity and providing a high-quality, consistent data foundation for algorithm training and testing. Upper-layer application developers do not need to concern themselves with the data source; they only need to obtain data through a unified data service layer interface, greatly reducing the workload of adapting to different data sources and allowing teams to focus on core algorithm development. When the format of a data source changes, only the corresponding access plugin needs to be updated, without modifying all downstream applications that depend on that data, significantly reducing maintenance costs.

[0123] Figure 2 This is a schematic diagram of an architecture system for general data acquisition and processing in intelligent driving according to an embodiment of this application, as shown below. Figure 2 As shown, the architecture system includes: a data access layer 201, a core processing layer 202, and a data service layer 203.

[0124] The data access layer 201 defines a series of data source access plugins, each specifically designed to parse raw data packets in a particular format. These plugins include at least the ROS Bag plugin, ADTF plugin, custom binary plugin, and MDF4 plugin. Each plugin can read the file structure of a raw data packet (e.g., a raw data source), parse the raw binary stream within it, and encapsulate the parsed raw data units into an intermediate generic data object. This object retains the basic information and data body of the raw data but removes its specific encapsulation format.

[0125] Optionally, the original data source may include ROS packet files (ROS Bag), multimodal sensor data stream files (CyberRecord), packaged sensor data files (Pack), and sensor time-aligned files (STF). The data access layer 201 may then include ROS read-write interfaces (ROS Read-Write, RosRW), real-time communication middleware data read-write interfaces (CyberRW), packaged data read-write interfaces (PackRW), and sensor time-aligned file read-write interfaces (STF_RW).

[0126] The core processing layer 202 is the heart of the architecture, responsible for converting intermediate generic data objects from the data access layer into fully standardized data. The core components of the core processing layer 202 may include a data parsing and standardization module, a data reassembly and synchronization module, and a data management bus.

[0127] Optionally, the data parsing and standardization module (corresponding to data parsing & standardization) can parse and transform intermediate common data objects according to a predefined internal unified data standard. This internal unified data standard specifies the data structure, coordinate system, units, field naming, etc., for all data types. For example, it standardizes all camera images into OpenCV Mat objects in BGR or RGB format, with a unified timestamp and coordinate system label.

[0128] Optionally, the data reassembly and synchronization module can receive standardized data streams, align and reassemble the data streams based on the data's built-in timestamps or a global synchronization clock, and output multimodal data frames that are strictly synchronized in time. This includes tasks such as frame alignment, distortion correction, point cloud fusion, unified verification, and unified internal and external parameters.

[0129] Optionally, the data management bus, as the communication backbone within the layer, is responsible for efficiently and reliably transmitting standardized data streams between various modules.

[0130] Optionally, data storage and bus management can include metadata management and object storage.

[0131] The data service layer 203 provides a unified and convenient data access interface for upper-layer applications. Data service layer 203 may include a unified data access interface, data export plugins, and real-time data streaming services, which can be used for data mining and model training.

[0132] Optionally, a unified data access interface can provide a standard API that allows upper-layer applications to query and retrieve standardized data by timestamp, sensor type, topic, etc.

[0133] Optionally, the data export plugin can provide a variety of output plugins to easily export internal standardized data to common formats, such as ROS2 Bag, OpenX series standards (OpenLABEL, OpenSCENARIO), Autoware Universe compatible formats, etc., to meet data exchange needs.

[0134] Optionally, the real-time data streaming service can support pushing standardized data as a real-time stream to online algorithm modules or visualization tools.

[0135] In this embodiment, the data access layer 201, core processing layer 202, and data service layer 203 enable rapid and seamless compatibility with raw data packets recorded on different platforms and using different software. Heterogeneous raw data can be converted into internally unified standardized data, achieving full lifecycle compatibility. Flexible data output interfaces can be provided to meet the data exchange needs with industry partners and third parties. This reduces the development and maintenance costs of the data processing chain and improves data utilization efficiency.

[0136] The following example illustrates how to process a ROS Bag data packet from platform A and a custom binary data packet from vendor B for algorithm testing.

[0137] First, the data access layer recognizes that data packets from platform A are in ROS Bag format and automatically loads and invokes the ROS Bag access plugin. This plugin reads the bag file, parses out messages such as sensor_msgs / Image (image) and sensor_msgs / PointCloud2 (point cloud), and encapsulates them into intermediate general data objects. It can also recognize that data packets from vendor B are in a proprietary format and loads the corresponding custom binary access plugin. This plugin, based on the known file structure and protocol, parses out images, CAN signals, etc., and similarly encapsulates them into intermediate general data objects. Afterwards, the data parsing and standardization module in the core processing layer can receive these intermediate objects.

[0138] Optionally, for images, regardless of whether the source is ROS or a binary packet, they can be decoded and uniformly converted into an internal standard image format, while the timestamps are unified to UTC time with nanosecond precision.

[0139] Optionally, point clouds can be uniformly converted into internally defined PointCloud objects, where the order of point fields, data types, and coordinate systems are all forcibly standardized.

[0140] Optionally, the positioning and pose information can be standardized and output, such as position (including latitude), velocity, angular velocity, linear velocity, heading, etc.

[0141] Optionally, the data frame can receive the standardized data stream described above, and combine images, point clouds, and positioning information from the same moment (or within the allowed time tolerance) into a single data frame based on the timestamps of each data point. Then, the data can be managed as metadata at a granular level of data fragments + data frames (clip + frame), and the corresponding files can be saved in object storage.

[0142] Optionally, in the data service layer, users can query and manage data on the data portal, or mine relevant data according to the scenario through the data mining service. The algorithm team can also obtain relevant data based on metadata and download the data to the server through the data download service.

[0143] Optionally, the above process enables plug-and-play processing of heterogeneous data sources, providing a seamless data experience for upper-layer applications.

[0144] In this embodiment, if it is necessary to process data in the vehicle, driving data from different data acquisition platforms can be obtained. This driving data represents data collected by multiple sensors in a driving scenario. The driving data can be encapsulated to obtain structured intermediate data, which represents initial data units identified by a unified semantic identifier. The structured intermediate data can be standardized to obtain first target data, which conforms to a standardized data structure and has a higher degree of standardization than the structured intermediate data. The first target data can be time-synchronized to obtain second target data, which represents time-synchronized multimodal data frames. Finally, the second target data can be processed for data services. In other words, this application embodiment acquires driving data from different data acquisition platforms in driving scenarios. Then, the driving data can be encapsulated to obtain structured intermediate data identified by a unified semantic identifier. After standardizing the structured intermediate data, the first target data is obtained. The first target data is then time-synchronized to ensure that the data conforms to the standardized data structure and is time-synchronized. This enables data service processing of the second target data, overcoming the limitations of the inability to effectively shield the heterogeneity of various data sources and the inconsistent data standards leading to inconsistent quality. This solves the technical problem of ineffective data processing and achieves the technical effect of effective data processing.

[0145] According to an embodiment of this application, a data processing system embodiment is provided. Figure 3 This is a schematic diagram of a data processing system according to an embodiment of this application, such as... Figure 3 As shown, the data processing system 300 may include: a data access module 302, a data processing module 304, and a data service module 306.

[0146] The data access module 302 is used to acquire driving data from different data acquisition platforms. The driving data represents the data collected by multiple sensors in the driving scenario. The driving data is encapsulated to obtain structured intermediate data, which represents the initial data unit. The initial data unit is identified by a unified semantic identifier.

[0147] The data processing module 304 is used to standardize the structured intermediate data to obtain first target data, wherein the first target data conforms to a standardized data structure and the standardization degree of the first target data is higher than that of the structured intermediate data; and to perform time synchronization processing on the first target data to obtain second target data, wherein the second target data is used to represent time-synchronized multimodal data frames.

[0148] The data service module 306 is used to perform data service processing on the second target data.

[0149] In this embodiment, in the data input layer, driving data from different data acquisition platforms can be obtained through the data access module 302. The driving data is used to represent the data collected by multiple sensors in the driving scenario. The driving data is encapsulated to obtain structured intermediate data, which is used to represent the initial data unit. The initial data unit is identified by a unified semantic identifier.

[0150] In this embodiment, in the core processing layer, the structured intermediate data can be standardized by the data processing module 304 to obtain the first target data, wherein the first target data conforms to the standardized data structure and the standardization degree of the first target data is higher than that of the structured intermediate data; the first target data is time-synchronized to obtain the second target data, wherein the second target data is used to represent the time-synchronized multimodal data frame.

[0151] In this embodiment, in the data service layer, the second target data can be processed by the data service module 306.

[0152] In this embodiment, if it is necessary to process data in the vehicle, driving data from different data acquisition platforms can be obtained. This driving data represents data collected by multiple sensors in a driving scenario. The driving data can be encapsulated to obtain structured intermediate data, which represents initial data units identified by a unified semantic identifier. The structured intermediate data can be standardized to obtain first target data, which conforms to a standardized data structure and has a higher degree of standardization than the structured intermediate data. The first target data can be time-synchronized to obtain second target data, which represents time-synchronized multimodal data frames. Finally, the second target data can be processed for data services. In other words, this application embodiment acquires driving data from different data acquisition platforms in driving scenarios. Then, the driving data can be encapsulated to obtain structured intermediate data identified by a unified semantic identifier. After standardizing the structured intermediate data, the first target data is obtained. The first target data is then time-synchronized to ensure that the data conforms to the standardized data structure and is time-synchronized. This enables data service processing of the second target data, overcoming the limitations of the inability to effectively shield the heterogeneity of various data sources and the inconsistent data standards leading to inconsistent quality. This solves the technical problem of ineffective data processing and achieves the technical effect of effective data processing.

[0153] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0154] According to an embodiment of this application, a data processing apparatus is provided. It should be noted that the data processing apparatus can be used to execute the above-described data processing method.

[0155] Figure 4 This is a schematic diagram of a data processing apparatus according to an embodiment of this application, such as... Figure 4 As shown, the data processing device 400 may include: an acquisition unit 402, an encapsulation unit 404, a first processing unit 406, a second processing unit 408, and a third processing unit 410.

[0156] The acquisition unit 402 is used to acquire driving data from different data acquisition platforms, wherein the driving data represents the data collected by multiple types of sensors in the driving scenario.

[0157] The encapsulation unit 404 is used to encapsulate driving data to obtain structured intermediate data, wherein the structured intermediate data is used to represent the initial data unit, and the initial data unit is identified by a unified semantic identifier.

[0158] The first processing unit 406 is used to standardize the structured intermediate data to obtain the first target data, wherein the first target data conforms to the standardized data structure and the standardization degree of the first target data is higher than that of the structured intermediate data.

[0159] The second processing unit 408 is used to perform time synchronization processing on the first target data to obtain the second target data, wherein the second target data is used to represent the time-synchronized multimodal data frame.

[0160] The third processing unit 410 is used to perform data service processing on the second target data.

[0161] Optionally, the first processing unit 406 includes: a first parsing subunit, used to parse the structured intermediate data to obtain initial semantic data, wherein the initial semantic data includes at least one of the following: the content, data type, time data, and metadata of the structured intermediate data; and a transformation subunit, used to transform the initial semantic data to obtain first target data.

[0162] Optionally, the first processing unit 406 includes: a first processing subunit, used to call the first data processing module to perform standardization processing on the structured intermediate data to obtain the first target data; the first processing unit 406 includes: an alignment subunit, used to perform an alignment operation on the first target data using a timestamp or global synchronization clock in the second data processing module in response to the second data processing module receiving the first target data, wherein the first data processing module and the second data processing module are connected through a data management bus; and a recombination subunit, used to recombine the aligned first target data to obtain the second target data.

[0163] Optionally, the third processing unit 410 includes: a second processing subunit, used to perform at least one of the following processes on the second target data: query processing, export processing, and push processing.

[0164] Optionally, the third processing unit 410 includes: a third processing subunit, used to perform data service processing on the second target data using the data service module; the data service module includes a target interface module, a data export plugin module, and a push module, and the method further includes one of the following: using the target interface module to query the second target data; using the data export plugin module to export the second target data; using the push module to push the second target data to the data access module or the data processing module.

[0165] Optionally, the encapsulation unit 404 includes: a second parsing subunit for parsing driving data; and a first encapsulation subunit for encapsulating the parsed driving data to obtain structured intermediate data.

[0166] Optionally, the encapsulation unit 404 includes: a third parsing subunit, used to, in response to the driving data being in a standardized binary container format, use a first plugin to obtain first structural information of the driving data and parse the first structural information to obtain a first initial data unit, wherein the first initial data unit includes the semantic content of the driving data; a fourth parsing subunit, used to, in response to the driving data being in a custom format, use a second plugin to obtain second structural information of the driving data and parse the second structural information to obtain a second initial data unit, wherein the second initial data unit includes the semantic content of the driving data; and a second encapsulation subunit, used to encapsulate the first initial data unit and the second initial data unit respectively to obtain structured intermediate data.

[0167] Optionally, the data processing apparatus 400 further includes: a grouping unit for grouping the second target data; and a storage unit for storing the grouped second target data.

[0168] In the data processing apparatus of this embodiment, the acquisition unit 402 acquires driving data from different data acquisition platforms, wherein the driving data represents data collected by multiple types of sensors in a driving scenario; the encapsulation unit 404 encapsulates the driving data to obtain structured intermediate data, wherein the structured intermediate data represents initial data units, which are identified by a unified semantic identifier; the first processing unit 406 performs standardization processing on the structured intermediate data to obtain first target data, wherein the first target data conforms to a standardized data structure and the standardization degree of the first target data is higher than that of the structured intermediate data; the second processing unit 408 performs time synchronization processing on the first target data to obtain second target data, wherein the second target data represents time-synchronized multimodal data frames; and the third processing unit 410 performs data service processing on the second target data, thereby solving the technical problem of ineffective data processing and achieving the technical effect of effective data processing.

[0169] Embodiments of this application also provide a vehicle, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods described in various embodiments of this application when it runs.

[0170] Embodiments of this application also provide a computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of this application.

[0171] Embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the methods of various embodiments of this application.

[0172] Embodiments of this application also provide a computer program product, including a non-volatile computer-readable storage medium for storing a computer program that, when executed by a processor, implements the methods in various embodiments of this application.

[0173] Embodiments of this application also provide a computer program that, when executed by a processor, implements the methods described in the various embodiments of this application.

[0174] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0175] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0176] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0177] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0178] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0179] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A data processing method, characterized in that, include: Acquire driving data from different data acquisition platforms, wherein the driving data is used to represent data collected by multiple types of sensors in a driving scenario; The driving data is encapsulated to obtain structured intermediate data, wherein the structured intermediate data is used to represent initial data units, and the initial data units are identified by a unified semantic identifier; The structured intermediate data is standardized to obtain first target data, wherein the first target data conforms to a standardized data structure and the degree of standardization of the first target data is higher than that of the structured intermediate data. The first target data is subjected to time synchronization processing to obtain the second target data, wherein the second target data is used to represent a time-synchronized multimodal data frame; The second target data is processed using data services.

2. The method according to claim 1, characterized in that, The structured intermediate data is standardized to obtain the first target data, including: The structured intermediate data is parsed to obtain initial semantic data, wherein the initial semantic data includes at least one of the following: the content, data type, time data, and metadata of the structured intermediate data; The initial semantic data is transformed to obtain the first target data.

3. The method according to claim 1, characterized in that, The structured intermediate data is standardized to obtain the first target data, including: The first data processing module is invoked to standardize the structured intermediate data to obtain the first target data; The first target data is time-synchronized to obtain the second target data, including: In response to the second data processing module receiving the first target data, the first target data is aligned using the timestamp or global synchronization clock in the second data processing module, wherein the first data processing module and the second data processing module are connected via a data management bus. The aligned first target data is reassembled to obtain the second target data.

4. The method according to claim 1, characterized in that, Data service processing is performed on the second target data, including: The second target data is processed by at least one of the following methods: query processing, export processing, and push processing.

5. The method according to claim 1, characterized in that, Data service processing is performed on the second target data, including: The data service module is used to perform data service processing on the second target data; The data service module includes a target interface module, a data export plugin module, and a push module. The method also includes one of the following: Using the target interface module, query the second target data; The second target data is exported using the data export plugin module. The push module is used to push the second target data to the data access module or the data processing module.

6. The method according to claim 1, characterized in that, The driving data is encapsulated to obtain structured intermediate data, including: The driving data is analyzed; The parsed driving data is encapsulated to obtain the structured intermediate data.

7. The method according to claim 1, characterized in that, The driving data is encapsulated to obtain structured intermediate data, including: In response to the fact that the driving data is in a standardized binary container format, a first plugin is used to obtain the first structural information of the driving data and to parse the first structural information to obtain a first initial data unit, wherein the first initial data unit includes the semantic content of the driving data. In response to the fact that the data format of the driving data is a custom format, the second plugin is used to obtain the second structural information of the driving data and parse the second structural information to obtain a second initial data unit, wherein the second initial data unit includes the semantic content of the driving data; The first initial data unit and the second initial data unit are encapsulated to obtain the structured intermediate data.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: The second target data is grouped and processed. Store the second target data after grouping and processing.

9. A data processing system, characterized in that, include: The data access module is used to acquire driving data from different data acquisition platforms, wherein the driving data represents data collected by multiple types of sensors in a driving scenario; the driving data is encapsulated to obtain structured intermediate data, wherein the structured intermediate data represents an initial data unit, and the initial data unit is identified by a unified semantic identifier; The data processing module is used to standardize the structured intermediate data to obtain first target data, wherein the first target data conforms to a standardized data structure and the degree of standardization of the first target data is higher than that of the structured intermediate data; and to perform time synchronization processing on the first target data to obtain second target data, wherein the second target data is used to represent a time-synchronized multimodal data frame. The data service module is used to perform data service processing on the second target data.

10. A vehicle, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the method according to any one of claims 1 to 8.

Images