An automatic driving data collection system and method based on a modular double channel

Abstract

The application belongs to the technical field of automatic driving, and provides an automatic driving data collection system and method based on a modular double channel, which comprises a data collection industrial personal computer, a fixed transmission channel for collecting and transmitting sensor and controller data preset by a vehicle factory, and an extensible collection channel comprising a domain controller and at least one extensible sensor, wherein the extensible sensor is connected with the domain controller, the extensible collection channel is used for forwarding data of the extensible sensor to the data collection industrial personal computer through the domain controller, and the fixed transmission channel and the extensible collection channel are connected with the data collection industrial personal computer through different hardware interfaces and communication protocols to form a double-channel parallel collection architecture with independent data streams. The application can synchronously collect vehicle controller information, main stream sensors fixedly installed, and non-standard sensor data temporarily added, and can realize high-precision time alignment across channels.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to an autonomous driving data acquisition system and method based on modular dual-channel. Background Technology

[0002] The training and reliable verification of autonomous driving algorithms rely on acquiring massive amounts of high-quality real-world road scene data. This data not only needs to be large in scale but also possess multimodal and high synchronization accuracy characteristics to accurately recreate complex driving environments. However, current mainstream data acquisition systems mostly adopt fixed, closed integrated architectures, meaning they involve deeply customized hardware connections and proprietary protocol adaptations for specific brands or models of sensors, such as specific models of LiDAR, cameras, and millimeter-wave radar. However, this fixed approach suffers from the following problems in long-term application:

[0003] 1. Insufficient system flexibility and scalability: After system integration, the sensor types, quantities, and data flow paths are fixed. When algorithm development tasks change, requiring the acquisition of new sensors, such as higher-resolution image sensors or 4D imaging radar, or adjustments to the sensor combination scheme, the existing system often requires complex hardware modifications, driver rewriting, and even overall architecture adjustments. It cannot quickly respond to the dynamic and differentiated data type requirements of algorithm training, resulting in extended development cycles and high costs.

[0004] 2. Difficulty in guaranteeing the time synchronization accuracy of multi-source heterogeneous data: Traditional systems, when integrating multiple sensors, typically rely on software timing or a general-purpose hardware clock, which has limited synchronization accuracy. In high-speed autonomous driving scenarios, even microsecond-level time deviations can cause misalignment between image, LiDAR point cloud, and inertial measurement unit data during fusion, severely affecting the accuracy of subsequent perception and localization algorithms.

[0005] 3. Insufficient Data Diversity: Existing systems are primarily designed around the sensors pre-installed by the vehicle manufacturer, lacking a standardized, open data access interface. This makes it difficult for R&D personnel to easily and quickly integrate third-party sensors, custom-designed add-ons, or new testing equipment that are still in the experimental stage into the system. The limited availability of data sources restricts the diversity and richness of the dataset, failing to meet the needs of cutting-edge algorithm verification for data from specific scenarios and extreme cases.

[0006] Currently, industry professionals have developed various autonomous driving data acquisition technologies, but none have been able to effectively solve the aforementioned problems. For example, patent CN120320890B discloses a modular data acquisition device and system for special vehicles that integrates multiple sensors. This device includes a switch, a master clock module, multiple slave clock modules, and a mission computer, all connected to the switch. The master clock module tracks the frequency and time of GNSS satellites. The multiple slave clock modules are synchronized with the master clock module. Each slave clock module includes a trigger controller and an inertial measurement unit. The multiple slave clock modules constitute a distributed clock for coordinated and precise triggering among multiple sensors, implementing hardware triggering or capturing hardware timestamps on each sensor. The mission computer is used to acquire data from multiple sensors and perform consistency calculations on the hardware timestamps. Compared with existing technologies, this system has advantages such as achieving nanosecond-level precise clock synchronization for multiple sensors, using different trigger sources for different sensors, and improving the stability of the acquisition device. However, this system still cannot solve the problem of severely insufficient system flexibility and scalability; it does not support the replacement of new sensors without configuration. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention provides a modular dual-channel autonomous driving data acquisition system and method that can simultaneously acquire vehicle controller information, data from fixed mainstream sensors, and data from temporarily added non-standard sensors, and can achieve high-precision time alignment across channels.

[0008] To achieve the above and related objectives, the present invention employs the following technical means:

[0009] The first aspect of this invention provides an autonomous driving data acquisition system based on a modular dual-channel architecture, comprising:

[0010] Data acquisition industrial control computer;

[0011] A fixed transmission channel is used to collect and transmit sensor and controller data preset by the vehicle at the factory.

[0012] A scalable acquisition channel includes a domain controller and at least one scalable sensor, wherein the scalable sensor is connected to the domain controller.

[0013] The scalable acquisition channel is used to forward data from scalable sensors to the data acquisition industrial control computer via the domain controller;

[0014] The fixed transmission channel and the expandable acquisition channel are connected to the data acquisition industrial control computer through different hardware interfaces and communication protocols, forming a dual-channel parallel acquisition architecture with independent data streams.

[0015] Furthermore, the data from the scalable sensors can be packaged by the domain controller and transmitted to the data acquisition industrial control computer via a pre-defined standardized streaming media protocol.

[0016] Furthermore, standardized streaming media protocols include TCP and RTP protocols.

[0017] Furthermore, the fixed transmission channel includes:

[0018] Ethernet switch for transmitting lidar point cloud and / or millimeter-wave radar data in UDP protocol;

[0019] Image acquisition card, used to receive GMSL2 video signals to acquire camera images;

[0020] A CAN acquisition card is used to receive CAN, CANFD, or vehicle Ethernet data from the vehicle controller.

[0021] Furthermore, the Ethernet switch supports the Precision Time Protocol and is configured as a transparent clock to provide a high-precision time synchronization channel for lidar point cloud and / or millimeter-wave radar data transmitted through it.

[0022] Furthermore, the scalable acquisition channel is configured to support the replacement of scalable sensors during the operation of the data acquisition industrial control computer, and the communication parameters between the domain controller and the data acquisition industrial control computer remain unchanged during the replacement process.

[0023] Furthermore, the system also includes a time synchronization source for providing a reference time signal to sensors in the fixed transmission channel and domain controllers in the scalable acquisition channel.

[0024] Furthermore, the system also includes at least one USB camera, which is directly connected to the data acquisition industrial control computer for acquiring image data.

[0025] A second aspect of this invention provides a method for acquiring autonomous driving data based on a modular dual-channel architecture, comprising the following steps:

[0026] Establish a fixed transmission channel and a scalable acquisition channel. The fixed transmission channel and the scalable acquisition channel are connected to the data acquisition industrial control computer through different hardware interfaces and communication protocols.

[0027] Data from the vehicle's factory-preset sensors and controllers is collected and transmitted to the data acquisition industrial control computer via a fixed transmission channel.

[0028] Data from at least one scalable sensor is acquired synchronously via a scalable acquisition channel and forwarded to the data acquisition industrial control computer via a domain controller.

[0029] Furthermore, in the data acquisition industrial control computer, data from fixed transmission channels and scalable acquisition channels are processed for time synchronization based on a unified time reference.

[0030] The beneficial technical effects of this invention are as follows:

[0031] This invention integrates all interface types of current mainstream autonomous driving sensors into a fixed transmission channel, which can meet most daily data acquisition needs, and the data path is stable and reliable. Considering the rapid pace of sensor hardware updates and the different data requirements of different autonomous driving algorithms, this invention establishes an expandable acquisition channel. This channel forwards some new sensor data that is not yet adapted or on the market to the data acquisition industrial control computer through a domain controller. This can break through the limitation of traditional data acquisition systems that only support fixed sensors, and meet the multi-data fusion needs of vehicle sensors and external sensors in autonomous driving research and development.

[0032] This invention provides a targeted configuration for scalable acquisition channels. When a scalable sensor is replaced, the new sensor only needs to send data to the domain controller according to a preset standardized streaming media protocol format, and then the domain controller will automatically forward it to the data acquisition industrial control computer. Unlike traditional data acquisition systems, there is no need for re-adaptation and redesign.

[0033] The present invention aligns the data streams of the fixed transmission channel and the expandable acquisition channel with timestamps based on a unified time reference at the data acquisition industrial control computer. This ensures that the data of all channels have an accurate and consistent time reference, guarantees high-precision time alignment, and improves data availability.

[0034] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0035] The accompanying drawings, incorporated in and forming part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without inventive effort. In the drawings:

[0036] Figure 1 This is a schematic diagram of the autonomous driving data acquisition system structure of this application;

[0037] Figure 2 This is a flowchart of the autonomous driving data acquisition method of this application. Detailed Implementation

[0038] Unless otherwise defined, all technical and / or scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should be understood that certain features of the invention (described in the context of separate embodiments for clarity) may also be provided in a single embodiment. Conversely, multiple features of the invention (described in the context of a single embodiment for brevity) may also be provided separately or in any suitable combination or, where appropriate, in any other described embodiment of the invention. Certain features described in the context of various embodiments will not be considered essential features of those embodiments unless the embodiment is inoperable without those elements. The invention is further illustrated below by specific examples; however, it should be noted that the specific process conditions and results described in the embodiments of the invention are merely illustrative and should not be construed as limiting the scope of protection of the invention. All equivalent changes or modifications made in accordance with the spirit and essence of the invention should be covered within the scope of protection of the invention.

[0039] like Figure 1 As shown, this application provides a modular dual-channel autonomous driving data acquisition system, including: a data acquisition industrial control computer; a fixed transmission channel for acquiring and transmitting sensor and controller data preset by the vehicle at the factory; and a scalable acquisition channel, including a domain controller and at least one scalable sensor, wherein the scalable sensor is connected to the domain controller, and the scalable acquisition channel is used to forward the data from the scalable sensor to the data acquisition industrial control computer through the domain controller; the fixed transmission channel and the scalable acquisition channel are connected to the data acquisition industrial control computer through different hardware interfaces and communication protocols, forming a dual-channel parallel acquisition architecture with independent data streams.

[0040] Furthermore, the data acquisition system of this application is used to acquire information such as images, point clouds, and poses during the vehicle's driving process.

[0041] Furthermore, the sensor and controller data preset by the manufacturer for the vehicle in this application are various sensor data and vehicle controller information selected at the initial stage of vehicle design. The types of sensors are relatively limited and can be replaced within a small range. Adapted sensors can be replaced, while data that is not adapted or does not meet system requirements cannot be collected.

[0042] Furthermore, such as Figure 1 As shown, the fixed transmission channel of this application includes the following hardware units:

[0043] Ethernet switch for transmitting lidar point cloud and / or millimeter-wave radar data in UDP protocol;

[0044] The image acquisition card is used to receive GMSL2 video signals to acquire camera images. The GMSL2 signal supports a 1-to-2 bypass function, which can output the video stream to other display or recording devices while acquiring the image.

[0045] Zhou Ligong CAN acquisition card is used to receive CAN, CANFD or vehicle Ethernet data from the vehicle controller, as well as combined inertial navigation data;

[0046] The integrated navigation system connects to the data acquisition industrial control computer via Ethernet, CAN, or serial port.

[0047] Furthermore, the Ethernet switch supports Precision Time Protocol (PTP) and is configured as a transparent clock to provide a high-precision time synchronization channel for LiDAR point cloud and / or millimeter-wave radar data transmitted through it. To address the time alignment issue of multi-sensor data, especially LiDAR and millimeter-wave radar data, this application configures the Ethernet switch as a PTP transparent clock to correct the dwell time of PTP messages within the switch, thereby maintaining high clock synchronization accuracy throughout the entire data transmission link. Even further, this application configures the integrated navigation system with a highly stable PTP master clock to provide a reference time for the entire network; then, the Ethernet switch is deployed on the critical data channel between the master clock and the sensors, and all PTP synchronization messages and data service messages from all sensors flow through this switch, thus synchronizing with the master clock.

[0048] Furthermore, the data from the scalable sensor is packaged by the domain controller and transmitted to the data acquisition industrial control computer via a preset standardized streaming media protocol. The standardized streaming media protocol includes TCP and RTP protocols. Even further, the scalable sensor in this application is an external sensor, such as a novel LiDAR or camera, which is not directly connected to the industrial control computer but is first connected to a domain controller. The domain controller is responsible for encapsulating and streaming the sensor's raw or pre-processed data according to a preset standardized streaming media protocol (such as TCP / RTP). For example, point cloud data is packaged by the domain controller and transmitted to the data acquisition industrial control computer via TCP; image data is packaged by the domain controller and transmitted to the data acquisition industrial control computer via RTP.

[0049] Furthermore, the scalable acquisition channel is configured to support the replacement of scalable sensors during the operation of the data acquisition industrial control computer, and the communication parameters between the domain controller and the data acquisition industrial control computer remain unchanged during the replacement process. The communication parameters between the domain controller and the data acquisition industrial control computer in this application are the aforementioned standardized streaming media protocol. Regardless of the type of scalable sensor connected, all data will be converted to this standard protocol format by the domain controller before uploading. The domain controller in this application can pre-integrate various drivers or protocol parsing plugins to adapt to the private protocols of different scalable sensors. When a new sensor is connected, the domain controller calls the corresponding driver or protocol parsing plugin to convert the raw data of the new sensor into the standardized streaming media protocol format in real time. Consequently, the data receiving program on the data acquisition industrial control computer in this application does not require restarting or configuration modification; the network configuration, listening port, and data parsing code of the industrial control computer do not need to be changed.

[0050] Furthermore, the system also includes at least one USB camera, which is directly connected to the data acquisition industrial control computer for acquiring image data. The USB camera in this application can be used to quickly acquire images of viewpoints not covered in fixed transmission channels or expandable acquisition channels.

[0051] Furthermore, the system also includes a time synchronization source for providing reference time signals to sensors in the fixed transmission channel and domain controllers in the scalable acquisition channel. The time synchronization source in this application includes a master clock generator and a time signal distribution module, which generates PPS hardware pulse signals, serial time messages, and PTP protocol messages. In this application, the master clock generator, under the action of the integrated navigation system, generates a highly stable and accurate local absolute time, and distributes the PPS hardware pulse signals and serial time messages via hardware lines, or PTP protocol messages via a network, to the fixed transmission channel and the scalable acquisition channel. The fixed transmission channel directly sends signals to sensors supporting external time input, such as LiDAR, image acquisition cards, and data acquisition industrial control computers; the scalable acquisition channel sends signals to the domain controller, which, upon receiving a unified time reference, generates a unified timestamp for all scalable sensors connected to it. The data acquisition industrial control computer itself also receives signals from the same time synchronization source. Upon receiving data packets from each channel, it can sort, align, and fuse all data under a unified time base based on the hardware timestamps embedded in the data packets, fundamentally solving the time deviation problem of multi-source data.

[0052] like Figure 2 As shown, this application also provides a method for acquiring autonomous driving data based on modular dual-channel, including the following steps:

[0053] Establish a fixed transmission channel and a scalable acquisition channel. The fixed transmission channel and the scalable acquisition channel are connected to the data acquisition industrial control computer through different hardware interfaces and communication protocols.

[0054] Data from the vehicle's factory-preset sensors and controllers is collected and transmitted to the data acquisition industrial control computer via a fixed transmission channel.

[0055] Data from at least one scalable sensor is acquired synchronously via a scalable acquisition channel and forwarded to the data acquisition industrial control computer via a domain controller.

[0056] Furthermore, in the data acquisition industrial control computer, data from fixed transmission channels and scalable acquisition channels are processed for time synchronization based on a unified time reference.

[0057] Furthermore, the method of this application first starts the data acquisition industrial control computer, loads the drivers and configuration parameters of the fixed transmission channel and the expandable acquisition channel, and then establishes the fixed transmission channel and the expandable acquisition channel respectively.

[0058] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. An autonomous driving data acquisition system based on modular dual-channel, characterized in that, include: Data acquisition industrial control computer; A fixed transmission channel is used to collect and transmit sensor and controller data preset by the vehicle at the factory. A scalable acquisition channel includes a domain controller and at least one scalable sensor, wherein the scalable sensor is connected to the domain controller. The scalable acquisition channel is used to forward the data from the scalable sensor to the data acquisition industrial control computer through the domain controller; The fixed transmission channel and the expandable acquisition channel are connected to the data acquisition industrial control computer through different hardware interfaces and communication protocols, forming a dual-channel parallel acquisition architecture with independent data streams.

2. The data acquisition system according to claim 1, characterized in that, The data from the scalable sensor is packaged by the domain controller and transmitted to the data acquisition industrial control computer via a preset standardized streaming media protocol.

3. The data acquisition system according to claim 2, characterized in that, The standardized streaming media protocols include TCP and RTP.

4. The data acquisition system according to claim 1, characterized in that, The fixed transmission channel includes: Ethernet switch for transmitting lidar point cloud and / or millimeter-wave radar data in UDP protocol; Image acquisition card, used to receive GMSL2 video signals to acquire camera images; A CAN acquisition card is used to receive CAN, CANFD, or vehicle Ethernet data from the vehicle controller.

5. The data acquisition system according to claim 4, characterized in that, The Ethernet switch supports a precise time protocol and is configured as a transparent clock to provide a high-precision time synchronization channel for the lidar point cloud and / or millimeter-wave radar data transmitted through it.

6. The data acquisition system according to claim 3, characterized in that, The scalable acquisition channel is configured to support the replacement of the scalable sensor during the operation of the data acquisition industrial control computer, and the communication parameters between the domain controller and the data acquisition industrial control computer remain unchanged during the replacement process.

7. The data acquisition system according to claim 1, characterized in that, The system also includes a time synchronization source for providing a reference time signal to the sensors in the fixed transmission channel and the domain controller in the scalable acquisition channel.

8. The data acquisition system according to claim 1, characterized in that, The system also includes at least one USB camera, which is directly connected to the data acquisition industrial control computer for acquiring image data.

9. A method for acquiring autonomous driving data based on modular dual-channel, characterized in that, Includes the following steps: A fixed transmission channel and an expandable acquisition channel are established. The fixed transmission channel and the expandable acquisition channel are connected to the data acquisition industrial control computer through different hardware interfaces and communication protocols. The data from the vehicle's factory-preset sensors and controllers is collected and transmitted to the data acquisition industrial control computer through the fixed transmission channel. Data from at least one scalable sensor is simultaneously acquired through the scalable acquisition channel and forwarded to the data acquisition industrial control computer via the domain controller.

10. The data acquisition method according to claim 9, characterized in that, In the data acquisition industrial control computer, data from the fixed transmission channel and the expandable acquisition channel are processed for time synchronization based on a unified time reference.

Images