A modular data acquisition device and system for special vehicles integrating multiple sensors

Abstract

The present invention relates to the field of data acquisition, and in particular to a modular data acquisition device and system for special vehicles that integrates multiple sensors. The device includes a switch, a master clock module, multiple slave clock modules, and a mission computer, each of which is connected to the switch. The master clock module is used to track the frequency and time of GNSS satellites; the multiple slave clock modules are synchronized with the master clock module, each of which includes a trigger controller and an inertial measurement unit. The multiple slave clock modules constitute a distributed clock for coordinated and precise triggering between multiple sensors, implementing hardware triggering or capturing hardware timestamps on each sensor; the mission computer is used to collect data from multiple sensors and perform consistency calculations on the hardware timestamps. Compared with the existing technology, the present invention has the advantages of achieving precise nanosecond-level clock synchronization of multiple sensors, using different trigger sources for different sensors, and improving the stability of the acquisition device.

Description

Technical Field

[0001] The present invention relates to the technical field of special vehicle data acquisition technology, and in particular to a special vehicle modular data acquisition device and system integrating multiple sensors. Background Art

[0002] In autonomous driving and intelligent robotics systems, multi-sensor data fusion technology is the core foundation for environmental perception and decision-making. To overcome the physical limitations of single sensors, modern intelligent systems generally use heterogeneous sensors such as lidar, millimeter-wave radar, cameras, and inertial measurement units (IMUs) to build multimodal perception systems. This multi-source heterogeneity places stringent demands on data acquisition devices, requiring precise synchronization of data acquisition timing across different sensors to eliminate spatiotemporal misalignment caused by hardware differences and ensure strict alignment of multimodal data in both time and space. However, existing systems often use a single trigger source or software trigger mode, making it difficult to adapt to the timing characteristics of different sensors. For example, sensors such as lidar cannot use a trigger source, while cameras typically require customized trigger moments to compensate for exposure delay. These requirements require multiple, customizable, and precise trigger sources, which cannot be achieved with a universal trigger source. In terms of timestamp generation, traditional solutions rely on the operating system kernel time or driver acquisition time stamping, but the real-time limitations of general systems such as Windows / Linux make it difficult for timestamp accuracy to exceed the millisecond level. Timestamps based on the operating system clock will produce jitter greater than 10ms, which is much greater than the frequency of high-speed sensors. For example, the sampling rate of high-speed IMU is usually less than 1ms. For multi-sensor fusion systems that require microsecond synchronization accuracy, this will lead to an increase in the error rate of cross-modal data matching. In terms of condition monitoring, the existing architecture cannot complete performance monitoring tasks and cannot detect short-term sensor anomalies. When accidental failures cause frame loss, related problems often require complex offline analysis to identify. It is difficult for the system to detect faults in a timely manner and reorganize data streams, which poses a challenge to the reliability of systems such as navigation and positioning. Therefore, the current sensor data acquisition device has the following problems:

[0003] 1) Existing solutions typically require all required sensors to be connected to the same hardware. This hardware also places strict restrictions on the type and number of sensors, making it suitable only for a specific sensor fusion system and unsuitable for more complex and specialized multi-sensor fusion systems that require high sensor diversity and quantity. Therefore, the design and implementation of a modular and scalable hardware-software architecture, as described in this invention, is essential.

[0004] 2) Existing solutions reuse common trigger signals as trigger sources to simplify design. However, these trigger sources cannot be directly controlled, do not support accurate configuration of sensor triggering processes, cannot support the trigger signals required by different sensors, are limited in the types of sensors they support, cannot adjust the trigger timing based on the sensor's fixed delay, and cannot compensate for sensor delays. Therefore, it is necessary to implement a flexible and configurable multi-channel synchronous trigger signal source.

[0005] 3) Existing sensor data acquisition solutions do not prioritize sensor clock stamps, often relying on the data acquisition driver to set timestamps based on the data acquisition time. However, general operating systems lack real-time performance, limiting the accuracy of acquired timestamps and making them inaccurate for precise sensor data fusion. Therefore, it is essential to output the precise time of data acquisition for each sensor based on the precise timestamps in the data acquisition device.

[0006] After searching, Chinese invention patent application publication number CN114614934A discloses a time synchronization trigger device for use in a true value test system for autonomous vehicles. The device comprises: a satellite inertial navigation module, an industrial computer, a time synchronization hardware module, and at least one camera. The satellite inertial navigation module is configured to receive GPS radio frequency signals via a GPS antenna and output uninterrupted time data frames with GPS timestamps. The industrial computer is configured to use time synchronization control software to obtain the GPS timestamps and synchronously retrieve the CPU hardware timestamps to perform time signal fitting to obtain the fitted system time. The device then sends a timestamp-bearing PTP message based on the fitted system time to synchronize with sensors supporting the PTP protocol. The time synchronization hardware module receives the fitted system time via the PTP protocol to complete time synchronization and sends a time hard synchronization trigger signal to synchronize with the at least one camera. This existing patent application suffers from inaccurate system time, which in turn affects the time of each sensor, and an inability to accurately align the system time with the GPS time.

[0007] How to realize a data acquisition device with modularization, precise time synchronization, and flexible configuration of trigger signal sources for special vehicles has become a technical problem that needs to be solved. Summary of the Invention

[0008] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a special vehicle modular data acquisition device and system integrating multiple sensors.

[0009] The purpose of the present invention can be achieved by the following technical solutions:

[0010] According to one aspect of the present invention, a modular data acquisition device for special vehicles integrating multiple sensors is provided. The device includes a switch, a master clock module, multiple slave clock modules, and a task computer respectively connected to the switch.

[0011] The master clock module is used to track the frequency and time of GNSS satellites;

[0012] Multiple slave clock modules are synchronized with the master clock module. Each slave clock module includes a trigger controller and an inertial measurement unit. Multiple slave clock modules constitute a distributed clock for coordinated and precise triggering between multiple sensors, and implement hardware triggering or capture hardware timestamps on each sensor.

[0013] The task computer is used to collect data from multiple sensors and perform consistency calculation on hardware timestamps.

[0014] Preferably, the slave clock module is connected to the triggerable sensor via an I / O interface of the trigger controller, and the triggerable sensor performs clock synchronization among multiple sensors via an active synchronization protocol.

[0015] More preferably, the process of triggering the sensor to synchronize clocks among multiple sensors through the active synchronization protocol includes the following steps:

[0016] Clock synchronization: Ensure that all slave clocks are synchronized within the clock deviation limit before triggering;

[0017] Hardware initialization: Configure sensor-specific delay compensation parameters during setup ;

[0018] Time buffer: establishing a safe initialization offset To reduce the impact of operating system jitter;

[0019] Triggered execution: Maintains deterministic sampling through hardware-triggered acquisition cycles; stops sampling if the monitored clock deviation exceeds the tolerance threshold;

[0020] Graceful termination: Ensure data collection is complete before the system shuts down.

[0021] Preferably, the process of performing consistency calculation on the hardware timestamp includes:

[0022] ROS nodes generate timestamps in the following ways:

[0023]

[0024] in, is the timestamp of the final calculation of the ROS node, is the start time of data collection, is the signal trigger period, is the data frame sequence number;

[0025] The actual triggering moment and sensor acquisition time are as follows:

[0026]

[0027]

[0028] in, The trigger moment for the plan; is the actual sampling time of the sensor; is the total sensor delay, for An estimate of , used for compensation; It is the deviation caused by clock asynchrony; The number of frame index error deviations;

[0029] The resulting timestamp error is:

[0030]

[0031] when Below the operating system jitter threshold, real-time detection becomes infeasible;

[0032] After acquisition, compare the number of frame index errors between the trigger controller and the sensor ,like If it is 0, the collected data set is valid, otherwise the collected data set is invalid.

[0033] Preferably, for non-triggerable sensors, different strategies are adopted to meet synchronization requirements depending on whether they support time synchronization, specifically:

[0034] Sensors that support time synchronization are connected to the master clock module to maintain clock synchronization;

[0035] Sensors that do not support time synchronization can be connected to a slave clock module to capture hardware timestamps.

[0036] More preferably, the timestamp of the sensor supporting time synchronization is calculated as:

[0037]

[0038] in, represents the measured sensor processing delay, Indicates the timestamp measured by the sensor based on its own clock. Indicates the deviation caused by clock asynchrony.

[0039] More preferably, the timestamp of sensors that do not support time synchronization is calculated as:

[0040]

[0041] in, Represents the timing uncertainty generated during signal acquisition, Indicates the timestamp measured by the sensor based on its own clock. Indicates the deviation caused by clock asynchrony.

[0042] Preferably, for a hardware-triggered sensor, the total delay consists of a fixed delay and a variable delay, and an estimated total delay of the sensor is determined by end-to-end calibration to compensate for the fixed delay.

[0043] For camera-type triggerable sensors, the sampling delay of the sensor itself includes the exposure time Half:

[0044]

[0045] in, Indicates the sensor latency for the camera type to trigger the sensor.

[0046] Preferably, the slave clock module includes an active temperature-compensated crystal oscillator and an output selector;

[0047] The output selector is used to expand a single I / O output into a configurable number of multiple I / O outputs to achieve trigger signals and data capture.

[0048] According to another aspect of the present invention, a modular data acquisition system for special vehicles integrating multiple sensors is provided. The system includes an external information processing system, a chassis and sensors. The modules or devices of the system are connected via a CANopen bus and Ethernet. The device includes a management computer for managing the data acquisition system.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] 1) The present invention uses a master clock module and a slave clock module to perform hardware triggering or capture hardware timestamps on multiple sensors, achieving nanosecond-level clock precision synchronization among multiple sensors, thereby tightly integrating multi-sensor data and solving the synchronization problem among multiple sensors.

[0051] 2) This invention flexibly adopts different synchronization methods for different sensors: triggerable sensors synchronize with a slave clock via an active synchronization protocol; non-triggerable sensors that support time synchronization synchronize with the master clock; and non-triggerable sensors that do not support time synchronization connect to a slave clock module to capture hardware timestamps to meet synchronization requirements, thus achieving multi-sensor triggering flexibility. During sensor synchronization, this invention does not rely on software time or internet connectivity to obtain timestamps, ensuring stable and reliable synchronization.

[0052] 3) The present invention designs a comprehensive triggering process to accurately control the triggering process of sensors supported by hardware triggering, so that each sensor collects data at the preset time and ensures that the acquired timestamp matches the sensor data accurately. At the same time, the timestamp is calculated for consistency to verify the validity of the collected data, thereby improving the stability and robustness of the collection device.

[0053] 4) The present invention adds an output selector to the slave clock module to achieve access to a configurable number of multiple sensors, thus achieving modular scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a structural schematic diagram of the collection device in the present invention;

[0055] Figure 2 Schematic diagram of the hardware architecture of the acquisition system in the present invention;

[0056] Figure 3 It is a timing diagram of the synchronization algorithm in the present invention;

[0057] Figure 4 Schematic diagram of the SLAM experimental platform;

[0058] Figure 5 for Figure 4 Schematic diagram of multiple sensors in the system;

[0059] Figure 6 Schematic diagram of SLAM experimental results in the present invention;

[0060] Figure 7 for Figure 6 A partial enlarged schematic diagram of part A in the middle;

[0061] Figure 8 This is a result diagram of the GNSS clock to the master clock in the synchronization and triggering performance experiment of the present invention;

[0062] Figure 9 This is a result diagram of the GNSS clock to slave clock in the synchronization and triggering performance experiment of the present invention;

[0063] Figure 10This is a result diagram of the GNSS clock to hardware trigger signal in the synchronization and trigger performance experiment of the present invention;

[0064] In the attached figure, 1: chassis, 2: synchronization system, 3: sensor platform. DETAILED DESCRIPTION

[0065] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0066] With technological advancements and changing needs, many traditional sensor fusion systems fail to fully consider the potential need for new features or modules at the outset of their design. This often leads to significant challenges when upgrading or expanding existing systems. This invention, through its modular hardware-software architecture, makes the system more adaptable to future expansion needs, whether adding new sensors or introducing more advanced processing algorithms. In practical applications, different types of sensors (such as cameras, radars, and lidars) typically operate at different sampling rates and time bases. Ensuring that data from these heterogeneous sensors is collected and processed simultaneously is a challenge. The configurable multi-sensor precision synchronization trigger mechanism proposed in this invention effectively addresses this issue. It not only ensures precise time synchronization between sensors but also supports flexible configuration to meet the unique requirements of specific application scenarios. For certain applications such as high-precision positioning, navigation, and environmental perception, even small time errors can have significant impacts. For example, in autonomous vehicles, not knowing the exact time of each sensor reading can lead to erroneous decisions. However, software-based timestamp methods are susceptible to buffering in operating systems, drivers, and other systems, making it difficult to accurately align data frames and timestamps. This invention achieves high-precision time synchronization at the microsecond or even nanosecond level between two different types of sensors by optimizing the sensor timestamp acquisition method, greatly improving the overall performance and reliability of the system.

[0067] To address the aforementioned technical issues, the present invention designs a scalable, modular, hardware-software architecture tailored to the needs of a multi-sensor data fusion system. This architecture not only supports flexible hardware expansion to accommodate the changing number and type of sensors required in different application scenarios, but also enhances system flexibility and adaptability through software configuration, providing an efficient and stable platform for multi-sensor fusion.

[0068] Regarding synchronous triggering, to address the challenge of configuring trigger sources for a large number of diverse sensors, this paper proposes a solution based on hardware precision triggers, capable of generating multiple, precisely controllable trigger signal sources. More importantly, considering the unique requirements of different applications, we have also implemented a modeled adjustment mechanism for each sensor's trigger source parameters. This allows users to freely set trigger conditions based on their specific needs, ensuring precise synchronization of all sensors at the microsecond or even nanosecond level, significantly improving the quality and efficiency of multi-sensor data fusion.

[0069] Furthermore, to further meet the time alignment requirements of multi-sensor data fusion systems, the present invention has developed a high-precision timestamp acquisition method using hardware triggering technology. This method not only improves the timestamp accuracy of sensor data to the nanosecond level but also allows for cross-calibration between two or more different types of sensors, ensuring time consistency across devices, which is crucial for subsequent data processing and analysis.

[0070] At the same time, in order to ensure the stable operation of the entire system, the present invention also includes a data flow monitoring solution that combines software and hardware, which can detect and report any potential data loss in real time, effectively solving the time jitter problem caused by non-real-time operating systems, and enhancing the reliability and maintenance convenience of the system.

[0071] The present embodiment relates to a modular data acquisition device for special vehicles that integrates multiple sensors. In response to the needs of multi-sensor fusion algorithms, the acquisition device supports modular data acquisition devices for multiple sensor modalities including cameras, lidars, millimeter-wave radars, IMUs, GNSS, CAN signals, etc. More specifically, when collecting data from multiple sensors, the types of sensors required are flexible and varied, and multiple sensors need to use different external trigger sources to coordinate sensor operations. Therefore, it is not easy to coordinate the operation of multiple types of sensors in real time, and it is necessary to use a modular approach to build a multi-sensor data acquisition system to facilitate subsequent sensor adjustment and maintenance. At the same time, it is also necessary to accurately obtain the operating status and operating time of each sensor for real-time monitoring of the sensor's operating status. Therefore, the present invention designs an expandable modular data acquisition device to meet the needs of data acquisition for multiple sensor fusion tasks.

[0072] A modular data acquisition device for special vehicles integrating multiple sensors includes an expandable modular hardware and software joint architecture, configurable multi-sensor precise synchronous triggering, and accurate acquisition of sensor timestamps.

[0073] 1) A scalable modular hardware and software joint architecture designed to achieve seamless integration of multi-sensor synchronization and fusion technologies across multiple domains, such as Figure 1 and Figure 2 . The architecture supports efficient exploration by independent robots, collaborative work among multiple agents, and intelligent roadside infrastructure. With a modular design, the boards within the system can be flexibly reused, expanded, or combined to meet different research needs. In addition, the architecture is versatile and scalable to meet various future challenges, especially in integrating multiple sensors. By pre-building such an open and easily adjustable technology framework, the time and resources required to develop new systems can be significantly reduced while ensuring performance.

[0074] like Figure 1 The device includes a master clock module, a slave clock module, a switch, a task computer, a management computer, and a power management module.

[0075] The master clock module tracks the frequency and time of GNSS satellites, providing a globally consistent, high-precision time reference for all modules. To reduce system complexity, the frequency reference is provided by the GPS Disciplined Oscillator (GPSDO), while the time reference is provided by the PTP master card.

[0076] All clocks to the master clock module are provided by a phase-locked loop (PLL) on a controlled daughter board to achieve the most accurate frequency reference. PPS alignment, timing, and PTP master functions are performed by an ARM Cortex-M4 microcontroller and its MAC, which supports PTP hardware timestamping.

[0077] The master clock module features 10 output signals routed from the IPEX connector, with each channel length-matched for latency balance. High-speed buffers are used to drive capacitive loads on the lines and provide impedance matching at the front and back ends. Four SMA outputs connect to the outside world via coaxial cables, allowing for flexible routing of the output signals. Two serial ports output standard NMEA protocols and connect to the master clock module internally.

[0078] The slave clock module inherits the core design and layout of the master clock module. Since it can synchronize with the master clock module via Ethernet or SMA interfaces, this design omits the GPSDO (GPSDO) and RTC (real-time clock) modules. Instead, it uses an active temperature-compensated crystal oscillator with an accuracy of 1.5ppm and adds external interface circuits for trigger signals and data capture.

[0079] The trigger controller is integrated into the slave clock module. The external interface features four opto-isolated inputs and seven outputs, the latter supporting various level or open-drain (OD) trigger modes. All outputs have integrated load drivers, making them suitable for camera triggering or controlling small loads. Precise calibration and compensation are implemented to account for time delays introduced by opto-isolation and load drivers.

[0080] The inertial measurement unit (IMU) is a tactical-grade ADIS16505-2 model with a PPS trigger interface and an internal phase lock mechanism to ensure synchronization. Data exchange between it and the microcontroller is accomplished via the serial peripheral interface (SPI). Notably, the IMU is connected to the slave clock module via a carrier board, allowing for flexible replacement of IMUs of different models and specifications for performance evaluation and comparative experiments.

[0081] The switch provides Ethernet packet switching services and allows a management computer to monitor and control the status of data exchange. A 100BASE-T1 interface is designed for power supply and connection to the 4D millimeter-wave radar, while a Gigabit fiber interface is reserved to support long-distance telemetry requirements. In addition, a Gigabit electrical port is configured for efficient interconnection with other cards. The card adopts a symmetrical architecture design and can be flexibly adjusted by replacing the front panel to meet the need for providing additional network interfaces externally or internally. Factors such as port speed and number, PTP protocol support, PHY type, and management interface were comprehensively considered when selecting the switching chip.

[0082] The mission computer, used to collect and store data from multiple sensors, consists of the NVIDIA Jetson Orin module and its expansion components. Key factors in selecting the Orin series were its outstanding 12-core CPU performance and support for the CUDA ecosystem, which significantly improves the execution efficiency of high-bandwidth data compression and acquisition tasks. The platform is equipped with a 32GB unified memory architecture, which facilitates the deployment of large language models in a compact form and effectively reduces inter-memory data transfer latency. Furthermore, given the widespread application of the Orin series chips in the field of intelligent driving, this solution can provide a development and testing environment that is closer to actual application scenarios. System storage is provided by a large-capacity solid-state drive (SSD).

[0083] The management computer, used to manage the data acquisition system, consists of a Raspberry Pi 5 (a high-performance single-board computer), a power supply, audio and video indicators, and expansion modules. A key reason for choosing the Raspberry Pi 5 was that its network interface card supports hardware timestamping, while previous generations only had software timestamping. Furthermore, the Raspberry Pi's robust ecosystem makes it easy to purchase and use expansion modules, which facilitates a variety of research and exploration needs.

[0084] If the system freezes, the management computer will not receive the heartbeat signal from the device, triggering a watchdog timeout. The management computer will then control the power management module via the CANopen bus to automatically reset the device.

[0085] In a modular hardware-software joint architecture, design precise master clocks, slave clocks, and switches that support clock synchronization; based on the hardware clock sources of each system, design trigger controllers and corresponding programs; 3) Design task computers and management computers for data processing and system management.

[0086] 2) This invention designs a configurable multi-sensor precision synchronization trigger to achieve precise multi-sensor triggering. It manages the clocks of a distributed system consisting of multiple sensors and a trigger controller, and implements precise hardware triggering or hardware timestamp acquisition on each sensor. Specifically, it provides a master clock source synchronized with global time; precisely synchronizes the clocks of each slave clock with the master clock; and implements hardware-based synchronization triggering based on the slave clocks.

[0087] Multi-sensor synchronization requires coordinated clock domain management between the slave clock modules and heterogeneous sensors distributed in multiple locations. Figure 3 As shown, the acquisition device includes three levels of clock domains:

[0088] 21) The GNSS clock domain, as a long-term global reference, has an inherent random jitter of 20 nanoseconds;

[0089] 22) A master clock domain synthesized by phase-locking the GNSS signal with a local high-stability oscillator to ensure short-term stability;

[0090] 23) A slave clock synchronization domain aligned with the master clock via the IEEE 1588 protocol for sensor control and timestamp generation.

[0091] To avoid trigger timing uncertainty due to software scheduling, each sensor must be triggered based on the hardware clock output. However, the hardware clock's pulse signal output is difficult to configure, requiring a special design to expand the output signal.

[0092] a) Master clock synchronization

[0093] To synchronize the system with global time, a stable clock source is required. However, while a high-quality local clock source can be very accurate over short periods of time, it lacks stability over long periods. While GNSS (Global Navigation Satellite System) time can be stable over longer periods of time, it exhibits jitter of approximately 20 nanoseconds. Therefore, neither is ideal as a single clock source.

[0094] To provide the clock source required for multi-sensor fusion, we use a GPS Disciplined Oscillator (GPSDO) to combine the advantages of both, achieving long-term and short-term stability and keeping synchronized with global time. The local clock source is synchronized with the GNSS clock through a PI (proportional-integral) controller, following the following formula:

[0095]

[0096] in, represents the frequency deviation between the local clock source and the GNSS clock, and It represents the phase deviation between the two. Indicates the base frequency of the local clock source. Represents the gain of the proportional control part, represents the gain of the integral control part, Indicates the final adjusted local clock source frequency.

[0097] b) Synchronize from the clock

[0098] The PTP protocol operates based on a master-slave architecture and includes the following synchronization phases:

[0099] Initialization: The synchronization protocol enters the Initialization state under two operating conditions: a device power-up event or the detection of a loss of synchronization. During this phase, slaves terminate existing time relationships and initiate the master discovery process. This reset mechanism ensures recovery from temporary network failures or master unavailability by clearing previous synchronization parameters and preparing to reconfigure the hierarchy.

[0100] Grandmaster selection: Potential grandmasters periodically broadcast Announce messages containing time quality metrics and clock characteristics. Slave devices evaluate these announcements using the Best Master Clock Algorithm (BMCA), which hierarchically selects a grandmaster based on clock accuracy, stratum level, and variance metrics. Alternatively, in constrained networks, the system may employ a static grandmaster configuration, bypassing the BMCA through pre-defined master-slave assignments. This selection process establishes the basic time hierarchy required for subsequent synchronization operations.

[0101] Synchronization: After the master clock is selected, the time alignment between the slave clock and the master clock is achieved through the continuous exchange of timestamp messages. When the measured clock offset exceeds the predefined maximum threshold When the slave clock is synchronized to the master clock's reference timestamp, it will immediately synchronize its local time to eliminate the large difference. Based on the deviation from the master clock, a proportional-integral-derivative (PID) control algorithm gradually adjusts the phase and frequency of the slave clock to converge asymptotically to the time scale of the master clock. This two-parameter correction method maintains frequency consistency while minimizing transient synchronization errors.

[0102] State transition: When the offset between the slave clock and the master clock is continuously lower than Synchronization is considered complete when the slave clock is synchronized. Conversely, synchronization is declared lost if either of the following two conditions occurs: 1) the measured offset exceeds an application-specific stability threshold for multiple synchronization cycles; or 2) a prolonged communication failure prevents the exchange of timestamps. Once synchronization loss is detected, the slave clock reinitializes the master clock discovery process, resetting to its initial state to reestablish the hierarchical timing relationship. This state machine design ensures the system can robustly recover from brief network disturbances or master clock failures.

[0103] Message exchange mechanism. The following four message types are used to achieve slave clock synchronization:

[0104] Sync: The master clock module is Time broadcast, from the clock module Time record;

[0105] Follow_Up: Contains the ;

[0106] Delay_Req: From the clock module The master clock module sends Always receive;

[0107] Delay_Resp: Return to the slave clock module.

[0108] Round trip delay and clock skew are calculated as follows:

[0109]

[0110]

[0111] The implementation of the PID controller is as follows:

[0112]

[0113] in, Indicates the basic frequency of the slave clock module. Indicates the The offset measurement value of the Indicates the offset threshold, and n is the number of measurements.

[0114] The following is the pseudo code of the clock synchronization algorithm:

[0115] Input: master clock set M, threshold δ max , δ 目标 =100ns

[0116] Output: Clock status after synchronization

[0117] 1: Current state ← Initializing

[0118] 2: Select the main clock←empty

[0119] 3: δ clock error ← 0

[0120] 4: loop

[0121] 5:if current state = initializing then

[0122] 6: Broadcast announcement message

[0123] 7: Select the master clock ← BMCA(M)

[0124] 8:if selected master clock ≠ empty then

[0125] 9: Current state ← Uncalibrated

[0126] 10:end if

[0127] 11:else if current state = uncalibrated or current state = slave mode then

[0128] 12: Exchange timestamps and calculate δ 时钟差

[0129] 13:if |δclock difference|>δmaximum then

[0130] 14: Set the clock to T immediately 主钟

[0131] 15:else

[0132] 16: Frequency adjustment through PID controller

[0133] 17:end if

[0134] 18:if |δclock difference|≤δtargetthen

[0135] 19: Current state←Slave mode

[0136] 20:else if timeout occurs or |δ 钟差 |>δ 故障 then

[0137] 21: Current status ← Initializing

[0138] 22:else

[0139] 23: Current status ← Uncalibrated

[0140] 24:end if

[0141] 25:end if

[0142] 26: end loop

[0143] c) Hardware precise triggering

[0144] Typically, for the sake of flexibility, most sensor triggers use software to control I / O signals, thereby controlling sensor operation. For example, a common approach is to set a timer, trigger an interrupt when the timer overflows, and control the GPIO peripheral within the interrupt routine. However, this introduces uncertainty. First, the timer interrupt may not respond immediately. For example, the program may be executing an operating system function that disables interrupts. This means that the timer interrupt must wait until the function completes and the interrupt is re-enabled. This introduces uncertain delays. Second, even if the interrupt routine is guaranteed to respond promptly, the system hardware may not enable the GPIO peripheral at the exact moment. Instead, it must wait until communication resources such as the system's internal bus are idle before sending control signals to the corresponding peripheral. This also introduces uncertainty into the precise hardware triggering.

[0145] Therefore, to achieve nanosecond-level hardware-precision triggering, the present invention uses clock events generated by the hardware clock based on system time to directly control the peripherals corresponding to I / O pins, achieving deterministic triggering. However, the number of I / O pins that can be controlled by the hardware clock is limited. To achieve flexibly configurable trigger outputs, a high-speed output selector must be added to the microcontroller of the slave clock to form a complete trigger controller. This expands a single I / O output into multiple configurable outputs, thereby meeting the complex frequency and phase combinations required when combining multiple sensors.

[0146] 3) Accurately Acquire Sensor Timestamps: After using hardware I / O to precisely control sensor operation, a crucial challenge remains: how to match sensor data with timestamps. Due to operating system timing jitter and the buffering of data acquisition drivers within the operating system, it's possible that the Robot Operating System (ROS) node reads data after a gap exceeding the sensor sampling time. Mismatched sensor timestamps can cause time deviations across several sampling intervals, significantly impacting system performance. Therefore, this invention incorporates a combined hardware and software sensor timestamp acquisition mechanism to ensure accurate sensor timestamp matching.

[0147] 31) Triggerable sensor

[0148] 311) The triggering process is implemented by the slave clock module.

[0149] Triggerable sensors maintain time consistency across sensors using an active synchronization protocol. The synchronization process consists of five consecutive phases that, unlike the subsequent monitoring and calibration mechanisms, are executed sequentially:

[0150] Clock synchronization: Ensure that all slave clocks reach Bounded synchronization is different from continuous clock monitoring in later stages.

[0151] Hardware initialization: Configure sensor-specific delay compensation parameters during setup , which is different from the runtime latency adjustment discussed in Latency Decomposition.

[0152] Time buffer: establishing a safe initialization offset To mitigate the effects of operating system jitter, this occurs before real-time fault detection mechanisms.

[0153] Triggered execution: Maintain deterministic sampling through hardware-triggered acquisition cycles, as opposed to post-hoc data validation procedures.

[0154] Graceful termination: Ensures data collection is complete before system shutdown, separate from ongoing clock drift monitoring.

[0155] Clock drift monitoring: real-time monitoring of clock deviation , and stops sampling when the tolerance threshold is exceeded. Continuous clock deviation indicates system degradation and the need for recalibration.

[0156] 312) Hardware timestamp consistency calculations are performed by the mission computer.

[0157] ROS nodes generate timestamps in the following ways:

[0158]

[0159] in, is the timestamp of the final calculation of the ROS node, is the start time of data collection, is the signal trigger period, The data frame sequence number.

[0160] The actual triggering moment and sensor acquisition time are as follows:

[0161]

[0162]

[0163] in, The trigger moment for the plan; is the actual sampling time of the sensor; is the total sensor delay, for An estimate of , used for compensation; It is the deviation caused by clock asynchrony; The number of frame index error deviations.

[0164] The resulting timestamp error is:

[0165]

[0166] Fault detection mechanism: Hardware / software instability may cause frame index mismatch ( ).when Below the operating system jitter threshold, real-time detection becomes unfeasible. After acquisition, compare the number of deviations in frame index errors between the trigger controller and the sensor. , it is possible to identify whether data integrity is violated, and if so, the collected data set is considered invalid.

[0167] Latency breakdown for hardware-triggered sensors: Total latency Delayed by fixed portion and variable partial delay composition:

[0168]

[0169]

[0170] in, Indicates signal output delay, 、 and They represent the transmission delay of the electrical signal on the cable, the sensor input stage delay and the sampling delay of the sensor itself.

[0171] End-to-end calibration determines the total sensor latency estimate To compensate for the fixed delay. For triggerable sensors like cameras, Including exposure time Half:

[0172]

[0173] in, Indicates the sensor latency for the camera type to trigger the sensor.

[0174] 32) Sensor cannot be triggered

[0175] For sensors that lack hardware triggering capabilities, time synchronization relies on accurate timestamp acquisition and post-processing alignment algorithms. Two complementary strategies can meet synchronization requirements:

[0176] Sensors that support time synchronization are synchronized with the master clock module.

[0177] PTP-supported LiDAR and millimeter-wave radar synchronize their clocks with the master clock module to keep the timestamps synchronized. Source:

[0178]

[0179] in, represents a specific sensor processing delay that can be measured through controlled experiments, Indicates the timestamp measured by the sensor based on its own clock. This method requires a network interface that supports PTP (Precision Time Protocol) and a high-precision oscillator, which limits its deployment in resource-constrained edge devices.

[0180] Sensors that do not support time synchronization are connected to slave clock modules.

[0181] Maintaining precise clock synchronization requires a complex hardware infrastructure that is often not met by cost-optimized edge devices constrained by power budgets and physical size. For such devices (cameras and I / O devices in Figure 2), time coordination is achieved through dedicated synchronization interface cards that timestamp the hardware interrupts from actively sampled signals or the start of communication frames. The resulting timestamp, based on the master reference time, is as follows:

[0182]

[0183] in, Represents the timing uncertainty generated during the signal acquisition process.

[0184] Therefore, accurate acquisition of sensor timestamps includes the following:

[0185] 3-1) Design a timestamp acquisition protocol for hardware-triggered sensors;

[0186] 3-2) Provide a synchronization clock source for sensors that support synchronization;

[0187] 3-3) Use hardware capture to obtain hardware timestamps for sensors that do not support triggering and synchronization;

[0188] 3-4) Obtain the protocol based on the timestamp to diagnose and analyze possible errors.

[0189] In order to verify the performance of the above system, we designed Figure 4The experimental platform includes an off-road robot chassis 1, the synchronization system 2 (Syncube system) of the present invention, and a sensor plane 3. It supports sensors including GNSS receivers (U-blox F9P), 4D millimeter-wave radar (Continental 548), barometers (Goertek SPL06) x8, 1.2 million pixel cameras x4, 5 million pixel cameras x2, 400,000 pixel high-speed camera x1, lidar (supporting rotating radars such as Hesai PandaR64 and non-repeating scanning radars such as Livox Avia), IMUs (ADIS16505, Lunqu H30), etc. Figure 5 .

[0190] The GVINS algorithm was used to conduct simultaneous localization and mapping (SLAM) experiments on the above platform. The positioning results of the experimental groundtruth, the unsynchronized GVINS algorithm (Unsync GVINS), and the synchronized GVINS algorithm (Syncube GVINS) are as follows: Figure 6 and Figure 7 When using the GVINS algorithm without synchronization, the root mean square error of the positioning result is 2.056m, while when using the GVINS algorithm with synchronization (this application), the root mean square error of the positioning result is 1.30m, which improves the positioning accuracy by 58%.

[0191] To verify the synchronization performance of the device, the GPS PPS, PTP master PPS, PTP slave PPS, and an IO trigger output were connected to channels A through D of a RIGOL 1104 2GSa / s 4-channel oscilloscope. GPSPPS was set as the trigger source. The IO trigger output was set to 250Hz, and the clock offset was set to 0. A program was written to remotely control the oscilloscope to complete a 1000-second sampling cycle and automatically perform data statistics and analysis tasks. The experimental results are shown in Figure 2. Figures 8 to 10 As shown, a waveform screenshot and a statistical histogram are displayed.

[0192] Due to the unique MAC timestamp hard trigger mechanism and hardware-software joint compensation mechanism, not only is there no software on the critical trigger path, that is, no software interrupts or program involvement, but the host can also perform large-scale calibration and compensation with nanosecond accuracy for any delay caused by peripheral hardware.

[0193] The results show that the device solves the synchronization problem between sensors on the one hand, and maintains the timestamp accuracy at the nanosecond level on the other.

[0194] This embodiment also relates to a data acquisition system, which includes a data acquisition device, as shown in the system diagram. Figure 2 ,In addition to this device, it supports access to external information processing systems (such as ,a host computer), chassis, sensors and other external devices.,The modules are connected through CANopen bus and Ethernet.

[0195] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A modular data acquisition device for special vehicles integrating multiple sensors, the device including a switch, characterized in that: The device includes a master clock module, multiple slave clock modules, and a task computer, each of which is connected to a switch. The master clock module is used to track the frequency and time of GNSS satellites; Multiple slave clock modules are synchronized with the master clock module. Each slave clock module includes a trigger controller and an inertial measurement unit. Multiple slave clock modules constitute a distributed clock for coordinated and precise triggering between multiple sensors, and implement hardware triggering or capture hardware timestamps on each sensor. The task computer is used to collect data from multiple sensors and perform consistency calculations on hardware timestamps; The slave clock module is connected to the triggerable sensor via the I / O interface of the trigger controller, and the triggerable sensor performs clock synchronization among multiple sensors via an active synchronization protocol; The process of calculating the consistency of the hardware timestamp includes: ROS nodes generate timestamps in the following ways: ， in, is the timestamp of the final calculation of the ROS node, is the start time of data collection, is the signal trigger period, is the data frame sequence number; The actual triggering moment and sensor acquisition time are as follows: ， ， in, The trigger moment for the plan; is the actual sampling time of the sensor; is the total sensor delay, for An estimate of , used for compensation; It is the deviation caused by clock asynchrony; The number of frame index error deviations; The resulting timestamp error is: ， when Below the operating system jitter threshold, real-time detection becomes infeasible; After acquisition, compare the number of frame index errors between the trigger controller and the sensor ,like If it is 0, the collected data set is valid, otherwise the collected data set is invalid; For non-triggerable sensors, different strategies are adopted to meet synchronization requirements based on whether they support time synchronization. Specifically: Sensors that support time synchronization are connected to the master clock module to maintain clock synchronization; Sensors that do not support time synchronization can be connected to a slave clock module to capture hardware timestamps.

2. The modular data acquisition device for special vehicles integrating multiple sensors according to claim 1, characterized in that: The process of clock synchronization between multiple sensors using the active synchronization protocol includes the following steps: Clock synchronization: Ensure that all slave clocks are synchronized within the clock deviation limit before triggering; Hardware initialization: Configure sensor-specific delay compensation parameters during setup ; Time buffer: establishing a safe initialization offset To reduce the impact of operating system jitter; Triggered execution: Maintains deterministic sampling through hardware-triggered acquisition cycles; stops sampling if the monitored clock deviation exceeds the tolerance threshold; Graceful termination: Ensure data collection is complete before the system shuts down.

3. The modular data acquisition device for special vehicles integrating multiple sensors according to claim 1, characterized in that: The timestamp for sensors that support time synchronization is calculated as: ， in, represents the measured sensor processing delay, Indicates the timestamp measured by the sensor based on its own clock. Indicates the deviation caused by clock asynchrony.

4. The modular data acquisition device for special vehicles integrating multiple sensors according to claim 1, characterized in that: The timestamp calculation for sensors that do not support time synchronization is: ， in, Represents the timing uncertainty generated during signal acquisition, Indicates the timestamp measured by the sensor based on its own clock. Indicates the deviation caused by clock asynchrony.

5. The modular data acquisition device for special vehicles integrating multiple sensors according to claim 1, characterized in that: For hardware-triggered sensors, the total delay consists of a fixed delay and a variable delay. The total sensor delay estimate is determined through end-to-end calibration to compensate for the fixed delay. For camera-type triggerable sensors, the sampling delay of the sensor itself includes the exposure time Half: ， in, Indicates the sensor latency for the camera type to trigger the sensor.

6. The modular data acquisition device for special vehicles integrating multiple sensors according to claim 1, characterized in that: The slave clock module includes an active temperature-compensated crystal oscillator and an output selector; The output selector is used to expand a single I / O output into a configurable number of multiple I / O outputs to achieve trigger signals and data capture.

7. A data acquisition system comprising the special vehicle modular data acquisition device according to any one of claims 1 to 6, characterized in that: The system includes an external information processing system, a chassis and sensors. The modules or devices of the system are connected via a CANopen bus and Ethernet. The device includes a management computer for managing the data acquisition system.

Images