Comprehensive perception fusion system

Abstract

The utility model discloses a comprehensive perception fusion system, relates to the technical field of sensor data fusion, and solves the problems of insufficient clock synchronization precision and lack of fault tolerance and redundancy mechanisms in the prior art. Comprising a multi-sensor module, a distributed clock synchronization module, a data processing and fusion module, a fault-tolerant and redundancy management module and a power management module, the multi-sensor module realizes data acquisition and timestamp addition through a sensor interface unit, a local clock unit and a timestamp generation unit; the distributed clock synchronization module adopts a high-precision clock source and redundancy design, and the synchronization precision is improved through a clock distribution circuit and a clock synchronization detection circuit; the fault-tolerant and redundancy management module realizes rapid switching and recovery of the system through a fault detection unit, a switching control unit and a state recovery unit; according to the invention, the clock synchronization precision and the system fault-tolerant capability are greatly improved.

Description

Technical Field

[0001] This utility model relates to the field of sensor data fusion technology, and more specifically to a comprehensive sensing fusion system. Background Technology

[0002] With the rapid development of technology, sensor fusion systems have become a key technology in fields such as autonomous driving and intelligent transportation. In complex and ever-changing traffic environments, this system can integrate information from multiple sensors, improve perception accuracy and robustness, and provide strong support for safe autonomous driving.

[0003] Existing sensor fusion systems integrate information from various sensors, such as cameras, radar, and lidar, through multi-sensor fusion technology to improve the accuracy of perception and information processing. The data collected by these sensors, after preprocessing, feature extraction, and fusion algorithms, provides comprehensive and accurate environmental perception for decision support systems. Existing systems typically employ a centralized or distributed architecture, relying on clock synchronization units to ensure the temporal consistency of multi-sensor data. However, current clock synchronization technologies are mainly based on master-slave synchronization or network synchronization protocols (such as PTP). Master-slave synchronization depends on the accuracy of the master clock, while network synchronization is limited by communication latency and jitter, making it difficult to achieve microsecond or even nanosecond-level synchronization accuracy. Furthermore, the system's hardware design lacks fault tolerance and redundancy mechanisms. If the master clock or synchronization unit fails, the clock synchronization of the entire system will fail, leading to a significant performance degradation in complex environments or under hardware failure conditions. These issues limit the system's application in highly dynamic and high-reliability scenarios. Utility Model Content

[0004] The purpose of this invention is to provide a comprehensive sensing fusion system to solve the problems mentioned in the background art.

[0005] To achieve the above-mentioned technical effects, the present invention adopts the following technical solution:

[0006] A comprehensive sensing fusion system includes: a multi-sensor module, a distributed clock synchronization module, a data processing and fusion module, a fault-tolerant and redundancy management module, and a power management module;

[0007] The multi-sensor module includes a sensor interface unit, a local clock unit, and a timestamp generation unit;

[0008] The distributed clock synchronization module includes a first clock source, a clock distribution circuit, a second clock source, and a clock synchronization detection circuit;

[0009] The data processing and fusion module includes a data receiving unit, a timestamp alignment unit, and a data calculation unit;

[0010] The fault tolerance and redundancy management module includes a fault detection unit, a switching control unit, and a state recovery unit;

[0011] The power management module includes a multi-channel isolated power supply circuit, a dynamic load adjustment circuit, and a backup power supply circuit.

[0012] The output of the multi-sensor module is connected to the input of the distributed clock synchronization module and the data processing and fusion module; the output of the distributed clock synchronization module is connected to the input of the multi-sensor module and the fault tolerance and redundancy management module; the output of the data processing and fusion module is connected to the input of the fault tolerance and redundancy management module; the output of the fault tolerance and redundancy management module is connected to the input of the distributed clock synchronization module, the multi-sensor module, and the power management module; the power management module supplies power to all modules.

[0013] Preferably, the sensor interface unit is connected to the local clock unit via a clock signal line; the timestamp generation unit receives the local clock signal from the local clock unit, adds a timestamp to the sensor data of the sensor interface unit, and then transmits it to the data processing and fusion module.

[0014] Preferably, the first clock source is connected to the local clock unit of the multi-sensor module through the clock distribution unit; the second clock source is connected in parallel with the clock distribution circuit and switched by a switching switch; the clock synchronization detection circuit is connected to the timestamp generation unit of the multi-sensor module.

[0015] Preferably, the first clock source and the second clock source are temperature-compensated crystal oscillators or temperature-controlled crystal oscillators.

[0016] Preferably, the clock distribution circuit includes a clock source, a buffer, a clock distribution chip, a resistor network, and a capacitor; the output pin of the clock source is connected to the input pin of the buffer; the output pin of the buffer is connected to the input pin of the clock distribution chip; the output pin of the clock distribution chip is connected to the local clock circuit input pin of each sensor module; the resistor network is connected in series on the clock signal line and connected to the output pin of the clock distribution chip and the input pin of the multi-sensor module; the capacitor is connected in parallel between the power supply pin and the ground pin.

[0017] Preferably, the data receiving unit is connected to the timestamp generation unit of the multi-sensor module; the timestamp alignment unit is connected to the data receiving unit, and transmits the aligned data to the data calculation unit.

[0018] Preferably, the fault detection unit is connected to the clock synchronization detection circuit of the distributed clock synchronization module and the sensor interface unit of the multi-sensor module; the switching control unit is connected to the second clock source of the distributed clock synchronization module and the backup power circuit of the power management module; and the state recovery unit is connected to the switching control unit.

[0019] Preferably, the multi-channel isolated power supply circuit is connected to the power interface of each module; the dynamic load adjustment circuit is connected to the multi-channel isolated power supply circuit; and the backup power supply circuit is connected in parallel with the multi-channel isolated power supply circuit, with the main and backup switching achieved through a switching switch.

[0020] Preferably, the multi-channel isolated power supply circuit includes a DC-DC isolation module, an LDO regulator, a filter capacitor, a diode, and an inductor; the input pin of the DC-DC isolation module is connected to the main power supply, and the output pin is connected to the input pin of the LDO regulator; the output pin of the LDO regulator is connected to the power input pin of each sensor module; the filter capacitor is connected in parallel between the input and output pins and the ground pin of the LDO regulator; the diode is connected in series between the input pin of the DC-DC isolation module and the main power supply; and the inductor is connected in series between the output pin of the DC-DC isolation module and the input pin of the LDO regulator.

[0021] Preferably, the dynamic load adjustment circuit includes a digital potentiometer, a Buck-Boost converter, a current sensor, an operational amplifier, and a MOSFET; the output pin of the digital potentiometer is connected to the feedback pin of the Buck-Boost converter; the output pin of the Buck-Boost converter is connected to the power input pin of each module; the input pin of the current sensor is connected in series between the output pin of the Buck-Boost converter and the load, and its output pin is connected to the input pin of the operational amplifier; the output pin of the operational amplifier is connected to the control pin of the digital potentiometer; the gate pin of the MOSFET is connected to the control pin of the Buck-Boost converter, the drain pin is connected to the output pin of the Buck-Boost converter, and the source pin is connected to the load.

[0022] In summary, due to the adoption of the above technical solution, the beneficial effects of this utility model are:

[0023] 1. This solution introduces independent first and second clock sources, combined with a precise clock distribution circuit and synchronization detection circuit, achieving high-precision clock synchronization at the microsecond and even nanosecond levels. Compared to traditional methods based on master-slave synchronization or PTP protocols, this solution no longer relies on the stability of a single master clock, effectively overcoming the impact of communication delays and jitter, ensuring the time consistency of multi-sensor data acquisition, and greatly improving the accuracy and real-time performance of information processing.

[0024] 2. The fault detection unit, switching control unit, and state recovery unit in the fault tolerance and redundancy management module work together to provide the system with robust self-protection capabilities. Once any sensor or clock synchronization component malfunctions, the system can quickly switch to a backup path or use redundant components to continue operating, ensuring normal operation even in complex environments or under hardware failure conditions. This design significantly enhances the system's stability and reliability, making it particularly suitable for fields with extremely high safety requirements, such as autonomous driving and aerospace.

[0025] 3. The design of multi-channel isolated power supply circuits, dynamic load adjustment circuits, and backup power supply circuits not only ensures a stable power supply for each module but also allows for flexible energy allocation under different operating conditions, reducing energy waste and extending system runtime. This is especially important for mobile devices such as drones and electric vehicles, as it directly translates into longer working hours and higher operational efficiency. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this utility model or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this utility model. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein:

[0027] Figure 1 This is a schematic diagram of the system working principle architecture of this utility model;

[0028] Figure 2 This is a schematic diagram of the multi-sensor module structure of this utility model;

[0029] Figure 3 This is a schematic diagram of the distributed clock synchronization module structure of this utility model;

[0030] Figure 4 This is a schematic diagram of the data processing and fusion module structure of this utility model;

[0031] Figure 5 This is a schematic diagram of the fault tolerance and redundancy management module structure of this utility model;

[0032] Figure 6 This is a schematic diagram of the power management module structure of this utility model;

[0033] The diagram is labeled as follows: 1. Multi-sensor module; 2. Distributed clock synchronization module; 3. Data processing and fusion module; 4. Fault tolerance and redundancy management module; 5. Power management module; 101. Sensor interface unit; 102. Local clock unit; 103. Timestamp generation unit; 201. First clock source; 202. Clock distribution circuit; 203. Second clock source; 204. Clock synchronization detection circuit; 301. Data receiving unit; 302. Timestamp alignment unit; 303. Data calculation unit; 401. Fault detection unit; 402. Switching control unit; 403. State recovery unit; 501. Multi-channel isolated power supply circuit; 502. Dynamic load adjustment circuit; 503. Backup power supply circuit. Detailed Implementation

[0034] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this utility model.

[0035] like Figures 1-6 As shown, a comprehensive sensing fusion system includes:

[0036] The system comprises a multi-sensor module 1, a distributed clock synchronization module 2, a data processing and fusion module 3, a fault-tolerant and redundancy management module 4, and a power management module 5. The output of the multi-sensor module 1 is connected to the inputs of the distributed clock synchronization module 2 and the data processing and fusion module 3. The output of the distributed clock synchronization module 2 is connected to the inputs of the multi-sensor module 1 and the fault-tolerant and redundancy management module 4. The output of the data processing and fusion module 3 is connected to the input of the fault-tolerant and redundancy management module 4. The output of the fault-tolerant and redundancy management module 4 is connected to the inputs of the distributed clock synchronization module 2, the multi-sensor module 1, and the power management module 5. The power management module 5 supplies power to all modules.

[0037] The multi-sensor module 1 includes a sensor interface unit 101, a local clock unit 102, and a timestamp generation unit 103. The sensor interface unit 101 and the local clock unit 102 are connected via a clock signal line. The timestamp generation unit 103 receives the local clock signal from the local clock unit 102, adds a timestamp to the sensor data of the sensor interface unit 101, and then transmits it to the data processing and fusion module 3.

[0038] In implementation, the multi-sensor module 1 serves as the sensing front-end of the entire sensor integrated sensing and fusion system. Its core function is to integrate and coordinate the operation of different types of sensors. The sensor interface unit 101 is responsible for physical connections and electrical signal interaction with various sensors such as cameras, LiDAR, millimeter-wave radar, and ultrasonic sensors. This unit includes multiple types of interfaces, such as the CSI Camera Serial Interface for cameras and the SPI Serial Peripheral Interface for radar, to adapt to the data transmission protocols of different sensors. The local clock unit 102 provides a unified local clock signal to the multi-sensor module 1, connected to the sensor interface unit 101 via a clock signal line. This clock signal is generated based on a high-precision crystal oscillator, typically set to a frequency of tens of megahertz. For example, the crystal oscillator used is an HC-49S with a frequency of 25MHz, ensuring a certain degree of consistency and synchronization in the data acquisition from each sensor. The timestamp generation unit 103 receives the local clock signal from the local clock unit 102 and uses its precise time information to add timestamps to the sensor data acquired from the sensor interface unit 101. This process involves precise time stamping. The counter inside the timestamp generation unit 103 counts according to the rising or falling edge of the local clock pulse, and the count value is matched one-to-one with the data collected by the sensor. This associates the data with precise time information, enabling the subsequent data processing and fusion module 3 to accurately process the data collected by different sensors at different times based on the timestamp.

[0039] In practical operation, the sensor interface unit 101 continuously polls each connected sensor to determine if new data has been generated. When the camera captures an image frame, it transmits the image data to the sensor interface unit 101 via the CSI interface at a certain data rate, such as 30 Mbps. The sensor interface unit 101 immediately notifies the timestamp generation unit 103. The timestamp generation unit 103 adds a timestamp accurate to the microsecond level to this image frame based on the clock signal of the local clock unit 102. For radar sensors, when they transmit detected distance and speed data via the SPI interface at a certain rate, such as 1 Mbps, the timestamp generation unit 103 is also triggered to add a timestamp. Furthermore, data from different sensors are processed sequentially by the timestamp generation unit 103 under the synchronization of the local clock, according to their data update frequency and transmission rate. During the sensor data acquisition process, the local clock unit 102 continuously and stably outputs a clock signal to ensure the temporal continuity of the entire multi-sensor module 1 and avoid data time deviations caused by clock jitter or frequency deviations.

[0040] Among them, the sensor interface unit 101 uses MAX3232 for RS-232 interface to realize communication with some traditional sensors; for high-speed image sensors, multiple CSI interfaces are implemented using FPGA such as Xilinx Spartan-7 series. Its internal logic units can be flexibly configured to support parallel data acquisition of multiple image sensors, and the data transmission bandwidth can reach 1Gbps to meet the high-speed data transmission requirements of high-resolution cameras.

[0041] The local clock unit 102 uses a high-stability crystal oscillator, such as the HC-49S type mentioned above, with a frequency of 25MHz. It is also paired with a clock buffer chip, such as SN74LVC1G125, to buffer and shape the clock signal, ensuring the quality and driving capability of the clock signal and providing a reliable clock source for the entire multi-sensor module 1.

[0042] The timestamp generation unit 103 uses a high-precision timestamp counter chip, such as the TSC7200. This chip can achieve nanosecond-level timestamp marking. It contains high-precision timers and counters that can be precisely synchronized with the clock signal of the local clock unit 102, accurately adding timestamp information to the sensor data.

[0043] In implementation, the multi-sensor module 1 is typically installed close to the monitored object or in a location where environmental sensing is required to minimize signal attenuation and delay during transmission. For vehicle-mounted systems, cameras can be installed at the front, rear, left, and right of the vehicle for comprehensive environmental monitoring. The camera is connected to the CSI interface module in the sensor interface unit 101 via shielded cables, with the cable length kept as short as possible, generally not exceeding 3 meters, to reduce electromagnetic interference. The sensor interface unit 101, timestamp generation unit 103, and local clock unit 102 can be mounted on a specially designed printed circuit board (PCB). This PCB is placed inside the vehicle's control box and must be properly electromagnetically shielded, for example, using a metal shielding cover, to prevent external electromagnetic interference from affecting clock signals and data transmission. In industrial automation scenarios, sensors may be distributed across different locations on the production line, while the PCB of the multi-sensor module 1 can be centrally mounted in a control cabinet. Each sensor is connected to the sensor interface unit 101 via dedicated cables. The cables should be selected based on the electromagnetic interference intensity of the industrial environment, using either shielded or unshielded cables. For environments with high electromagnetic interference, double-shielded cables are used.

[0044] In implementation, the local clock unit 102 provides a unified clock signal to all sensors, providing a solid time synchronization foundation for subsequent data time alignment and fusion. This avoids data time deviation caused by inconsistent sensor clocks and improves the accuracy of data fusion. Furthermore, the precise marking function of the timestamp generation unit 103 enables subsequent data processing modules to accurately sort and correlate data based on timestamps. Even in complex dynamic environments, it allows for clear tracing of sensor data at different times, providing more accurate information for the decision-making system. Secondly, the sensor interface unit 101 employs various interface chips and FPGAs to implement multiple types of interfaces, ensuring compatibility with different types of sensors. This significantly expands the application scope of the multi-sensor module 1, allowing the system to flexibly select and add sensors according to different application scenarios, thus improving the system's versatility and scalability.

[0045] The distributed clock synchronization module 2 includes a first clock source 201, a clock distribution circuit 202, a second clock source 203, and a clock synchronization detection circuit 204. The first clock source 201 is connected to the local clock unit 102 of the multi-sensor module 1 through the clock distribution unit. The second clock source 203 is connected in parallel with the clock distribution circuit 202 and is switched by a switch. The clock synchronization detection circuit 204 is connected to the timestamp generation unit 103 of the multi-sensor module 1. The first clock source 201 and the second clock source 203 use temperature-compensated crystal oscillators or temperature-controlled crystal oscillators.

[0046] In implementation, the first clock source 201 uses a temperature-controlled crystal oscillator (OCXO) as the main clock source, such as the TXC OCXO-935. This crystal oscillator internally consists of a quartz crystal, a heating device, a temperature sensor, and a control circuit. The quartz crystal is the core oscillation element, and its oscillation frequency is stable. The heating device maintains the internal environment of the crystal oscillator at a constant temperature to reduce the impact of temperature changes on the frequency. The temperature sensor monitors the temperature in real time, and the control circuit adjusts the heating power according to temperature changes, ensuring the crystal oscillator operates at a stable frequency. Its output frequency can achieve extremely high stability, typically tens of megahertz, such as 10MHz.

[0047] The clock distribution circuit 202 consists of multiple clock buffer chips and clock distribution chips, such as the IDT8T49N241 clock distribution chip. The output pin of the first clock source 201 is connected to the input pin of the clock distribution chip. The clock distribution chip distributes the input clock signal to multiple output pins through internal logic and buffer circuits. Internally, it employs a multi-level buffer and fan-out structure to ensure the driving capability and signal integrity of the clock signal. Through the output pins, the clock signal is transmitted to the local clock unit 102 of the multi-sensor module 1, providing it with a stable clock reference.

[0048] The second clock source 203 uses a temperature-compensated crystal oscillator (TCXO) as a backup clock source, such as the EPSON TG-5035CA model. Internally, it contains a quartz crystal and a temperature compensation circuit. The temperature compensation circuit compensates for the output frequency according to the crystal's temperature characteristics, keeping the frequency relatively stable within a certain temperature range, with an output frequency of approximately 10MHz. It is connected in parallel with the clock distribution circuit 202, and can be connected to the clock distribution circuit 202 via a switch using an ADG708 analog switch chip. The control pin of this analog switch chip is connected to the system controller, allowing the clock source to be switched according to instructions.

[0049] The clock synchronization detection circuit 204 mainly consists of a comparator, a counter, and logic gates, such as using an LM311 comparator and a 74HC393 counter. It is connected to the timestamp generation unit 103 of the multi-sensor module 1. By comparing the clock signal from the timestamp generation unit 103 with the clock signal from the first clock source 201, it generates a feedback signal based on the comparison result. The comparator compares the two clock signals, the counter counts the clock deviation, and the logic gates determine whether the deviation exceeds the allowable range based on the counter result. If it does, an alarm signal is issued to notify the system to perform clock source switching or other adjustment operations.

[0050] In the initial stage of system operation, the first clock source 201, a temperature-compensated crystal oscillator, begins operation, and its stable clock signal is transmitted to the local clock unit 102 of the multi-sensor module 1 via the clock distribution circuit 202. The clock distribution circuit 202 multiplexes the input clock signal to ensure that each local clock unit 102 receives a high-quality clock signal. Simultaneously, the clock synchronization detection circuit 204 continuously monitors the clock signal in the timestamp generation unit 103 of the multi-sensor module 1 and the clock signal from the first clock source 201. Under normal circumstances, the system operates stably due to the high stability of the first clock source 201. When the system detects a rapid change in ambient temperature, power fluctuations, or other abnormal conditions, the performance of the first clock source 201 may be affected. In this case, the clock synchronization detection circuit 204 will detect a clock deviation exceeding the allowable range. At this time, the system will switch the clock source to the second clock source 203, a temperature-compensated crystal oscillator, by controlling the control pin of the ADG708 analog switch chip to ensure uninterrupted clock signal transmission.

[0051] The specific circuit operation principle is as follows: The clock signal generated by the first clock source 201 is processed by the clock distribution circuit 202 and flows to each local clock unit 102 according to a predetermined path to provide clock signals for the multi-sensor module 1. The clock synchronization detection circuit 204 continuously compares and analyzes the clock signal fed back from the timestamp generation unit 103 and the clock signal of the first clock source 201. Once the clock deviation is abnormal, a signal is triggered to the controller. The controller changes its internal switching state by controlling the logic level of the ADG708 analog switch chip, thereby realizing the switching from the first clock source 201 to the second clock source 203 and ensuring the stability of the system clock.

[0052] In implementation, the use of the primary clock source 201, a temperature-controlled crystal oscillator, ensures extremely high clock stability under normal conditions, providing a high-precision time reference for sensor data acquisition and processing. This meets the requirements of applications with high time accuracy, such as high-precision positioning and target recognition in autonomous driving. The presence of the secondary clock source 203, a temperature-compensated crystal oscillator, and a switching switch provides clock redundancy for the system. When the primary clock source fails, the system can quickly switch to the backup clock source, preventing clock synchronization failure of the entire system due to clock source failure and ensuring the continuous and stable operation of the system.

[0053] The clock distribution circuit 202 includes a clock source, a buffer, a clock distribution chip, a resistor network, and a capacitor. The output pin of the clock source is connected to the input pin of the buffer. The output pin of the buffer is connected to the input pin of the clock distribution chip. The output pin of the clock distribution chip is connected to the local clock circuit input pin of each sensor module. The resistor network is connected in series on the clock signal line and is connected to the output pin of the clock distribution chip and the input pin of the multi-sensor module 1. The capacitor is connected in parallel between the power supply pin and the ground pin.

[0054] In implementation, a 74HC244 buffer is used. Its main function is to buffer and enhance the clock source signal, preventing subsequent circuits from affecting the clock source's load. The clock source's output pin is connected to the 74HC244 buffer's input pin; for example, the clock source's output pin OUT is connected to the 74HC244's input pins A1-A8. The buffer's internal structure contains multiple input and output channels, using internal logic circuitry to buffer the input signal, thereby improving its driving capability and anti-interference ability.

[0055] The clock distribution chip uses the IDT8T49N241, which can distribute one input clock signal into multiple output clock signals. The output pins Y1-Y8 of the buffer are connected to the input pins of the clock distribution chip, such as IN. Internally, there are dedicated logic circuits and signal distribution circuits that distribute the input clock signal to multiple output pins, such as OUT1-OUT8, through multiplexing and other techniques.

[0056] The resistor network consists of multiple precision resistors, such as 100Ω metal film resistors connected in series on the clock signal line, and connected to the output pins of the clock distribution chip and the input pins of the multi-sensor module 1. The function of the resistor network is to properly match and current-limit the clock signal, prevent signal reflection and overshoot, and ensure the integrity of the clock signal during transmission. It is connected in series between each output pin of the clock distribution chip and the local clock circuit input pin of the multi-sensor module 1; for example, one end of the resistor is connected to the OUT1 pin of the clock distribution chip, and the other end is connected to the local clock circuit input pin of the corresponding sensor module.

[0057] In practical operation, the clock source TXC OCXO-750 first begins oscillation, generating a stable clock signal. This signal is output through its output pin and enters the input pin of the 74HC244 buffer. The buffer buffers the input signal, enhancing its driving capability and stability, and outputs the processed signal from its output pin. Then, the signal is transmitted to the input pin of the IDT8T49N241 clock distribution chip. The clock distribution chip receives the buffered clock signal and, through its internal logic circuitry, splits it into multiple signals, outputting them from its output pin. During the clock signal's transmission from the clock distribution chip to each sensor module, a series resistor network appropriately limits and matches the signal, ensuring stable transmission of the clock signal on the transmission line and preventing signal reflection and distortion. Simultaneously, a capacitor connected in parallel between the power supply pin and the ground pin decouples and filters the power supply, ensuring its stability and thus guaranteeing the stable operation of the entire clock distribution circuit 202. The electrical flow logic is as follows: The clock signal generated by the clock source is sequentially buffered and amplified, distributed by the clock distribution chip, and then transmitted and matched by a resistor network before finally being transmitted to the local clock circuit of each sensor module. Throughout this process, capacitors continuously provide a stable power environment for the circuit, preventing power supply noise from interfering with the normal transmission and distribution of the clock signal.

[0058] The data processing and fusion module 3 includes a data receiving unit 301, a timestamp alignment unit 302, and a data calculation unit 303; the data receiving unit 301 is connected to the timestamp generation unit 103 of the multi-sensor module 1; the timestamp alignment unit 302 is connected to the data receiving unit 301, and transmits the aligned data to the data calculation unit 303.

[0059] In implementation, the data receiving unit 301 mainly consists of a high-speed data interface chip and a data buffer. An FPGA, such as the Xilinx Spartan-7 series, can be used as the high-speed data interface chip, which has abundant I / O pins and powerful parallel processing capabilities. Multiple I / O pins are used to receive timestamp data from the multi-sensor module 1, and these pins are connected to the output of the timestamp generation unit 103 of the multi-sensor module 1. Simultaneously, a high-speed data buffer, such as the IDT72V36512, is used. Its input pins are connected to the output pins of the FPGA, and it internally contains storage units and control logic to temporarily store the received data, preventing data loss or overflow. The FPGA's internal logic circuit decodes and performs preliminary processing on the data from different sensors according to the set interface protocol, temporarily storing the received data in the data buffer.

[0060] The timestamp alignment unit 302 consists of a high-speed comparator and synchronous logic circuitry, such as using an AD8661 as the high-speed comparator. This unit receives data from the data receiving unit 301 and compares the timestamps of data from different sensors using the comparator to determine the data's chronological order. The synchronous logic circuitry uses a programmable logic device, such as the Altera MAX II series from the CPLD Complex Programmable Logic Device, with its input pins connected to the comparator's output. Based on the comparison results, it sorts and adjusts the timestamps of the data, ensuring accurate time alignment of data from different sensors. Through internal logic gates, flip-flops, and other components, the data is reorganized according to chronological order, ensuring a one-to-one correspondence between data collected by different sensors within the same time window.

[0061] The data processing unit 303 mainly consists of a high-performance digital signal processor (DSP) and a dedicated arithmetic logic unit (ALU), such as the TI TMS320C6678 DSP chip. Its input pins are connected to the output of the timestamp alignment unit 302 to receive aligned data. The DSP chip has a powerful computing core and storage unit, capable of performing complex calculations on the data, including but not limited to weighted averaging and Kalman filtering fusion algorithms for data from different sensors. The ALU, as part of the DSP, performs basic arithmetic and logical operations, assisting the DSP in completing the mathematical operations required for data fusion. ## II. Operating Mode When the system is working, the FPGA of the data receiving unit 301 first receives timestamp data from the multi-sensor module 1. When data from different sensors arrives at the data receiving unit 301, the FPGA inputs the data through its I / O pins according to a predetermined data protocol and stores it in the data buffer IDT72V36512. Then, the timestamp alignment unit 302 starts working; its internal high-speed comparator AD8661 compares the timestamps of different sensor data, and the comparison result is input to the synchronization logic circuit of the CPLD. The synchronization logic circuit rearranges the data according to the timestamp order based on the comparison result, ensuring time alignment. The timestamp-aligned data is then transmitted to the TMS320C6678 DSP chip in the data computing unit 303. The DSP chip, based on a preset fusion algorithm, calls its internal computing core and ALU to perform fusion processing on the aligned data. For example, data from both the camera and LiDAR at the same time are weighted according to their respective characteristics and reliability before fusion to obtain more accurate target information, such as the object's position, speed, and shape. The electrical flow logic is as follows: Timestamp data from the multi-sensor module 1 enters the data receiving unit 301 through the FPGA's I / O pins, is processed by the FPGA, and stored in the data buffer. The comparator and synchronization logic circuit in the timestamp alignment unit 302 compare and reorder the stored data's timestamps, transmitting the time-aligned data to the DSP chip in the data computing unit 303. The DSP chip and its internal ALU complete the final data fusion calculation, ultimately outputting the fused data.

[0062] The fault tolerance and redundancy management module 4 includes a fault detection unit 401, a switching control unit 402, and a state recovery unit 403; the fault detection unit 401 is connected to the clock synchronization detection circuit 204 of the distributed clock synchronization module 2 and the sensor interface unit 101 of the multi-sensor module 1; the switching control unit 402 is connected to the second clock source 203 of the distributed clock synchronization module 2 and the backup power circuit 503 of the power management module 5; the state recovery unit 403 is connected to the switching control unit 402;

[0063] In implementation, the fault detection unit 401 mainly consists of multiple sensors and a microcontroller, with an STM32F4 series microcontroller selected as the core chip. It connects to the clock synchronization detection circuit 204 of the distributed clock synchronization module 2 via multiple analog input pins to receive clock synchronization deviation information; simultaneously, it connects to the sensor interface unit 101 of the multi-sensor module 1 via multiple digital input pins to acquire sensor status information. The microcontroller runs a dedicated fault detection program, using its built-in analog-to-digital converter (ADC) to convert the analog signals from the clock synchronization detection circuit 204 into digital signals, while directly reading the digital status signals from the sensor interface unit 101 for real-time monitoring and analysis. For example, it checks whether the clock synchronization deviation exceeds a preset threshold and whether the sensor has experienced data transmission interruptions or data anomalies.

[0064] The switching control unit 402 consists of logic gate circuits and analog switch chips. Taking the ADG708 analog switch chip as an example, its control pins are connected to the output pins of the fault detection unit 401, such as the GPIO pins of the STM32F4 microcontroller. The input pins of the analog switch chip are connected to the second clock source 203 of the distributed clock synchronization module 2, such as a temperature-compensated crystal oscillator, and the backup power supply circuit 503 of the power management module 5, respectively. The logic gate circuits perform logical judgment and processing on the signals from the fault detection unit 401. When a fault signal is received, the control signal of the ADG708 analog switch chip is changed to realize the switching operation of the clock source or the power supply. For example, when a fault of the main clock source is detected, the logic level output by the logic gate circuit will cause the ADG708 analog switch chip to connect the second clock source 203 to the system; when a power supply fault occurs, it switches to the backup power supply circuit 503.

[0065] The state recovery unit 403 mainly consists of a memory chip and a reset circuit, and can use an EEPROM such as the AT24C02 as the memory chip. Its data pins are connected to the switching control unit 402 to receive system status information. The reset circuit can use a dedicated reset chip, such as the MAX810. After the fault is cleared, the state recovery unit 403, based on the system status information stored in the EEPROM, sends a reset signal to the system through the reset circuit, restoring the system to its normal operating state. The memory chip stores critical information about the system before the fault, and the reset circuit receives the information from the memory chip and generates a reset pulse as needed to reset the system to a stable state.

[0066] During normal system operation, the STM32F4 microcontroller in the fault detection unit 401 continuously monitors the status of the clock synchronization detection circuit 204 and the sensor interface unit 101. For the clock synchronization detection circuit 204, the microcontroller continuously reads its output clock deviation signal via an ADC. Once the deviation exceeds a set threshold, such as 100 ps, ​​it sends a fault signal to the switching control unit 402. For the sensor interface unit 101, it checks for data interruptions or abnormalities. If a sensor fails to send data for an extended period or its data error rate exceeds 10%, it also sends a corresponding fault signal. Upon receiving the fault signal, the switching control unit 402 uses logic gates to make logical judgments based on the nature and severity of the signal. For example, if it's a clock source fault signal, it changes the control level of the ADG708 analog switch chip, switching the system to the second clock source 203. For power supply failures, it connects the backup power supply circuit 503 to the system. After the fault is cleared, the state recovery unit 403 reads the previously stored system state information from the EEPROM, such as the previously used clock source and power supply information, and sends a reset signal to the system via the MAX810 reset chip, restoring the system to normal operating status. The electrical flow logic is as follows: the microcontroller of the fault detection unit 401 monitors the system state and transmits the fault signal to the switching control unit 402. The switching control unit 402 then switches the clock source or power supply according to logic gate circuits and analog switch chips. After the fault is cleared, the state recovery unit 403 uses the storage chip and reset circuit to restore the system to normal operation, ensuring continuous and stable system operation.

[0067] In implementation, real-time monitoring of clock synchronization and sensor interfaces enables timely detection of system faults, preventing system failure due to localized faults and improving system reliability in complex environments. For example, in scenarios with high system reliability requirements, such as industrial automation, it ensures continuous and stable production processes. Furthermore, the switching control unit 402 uses logic gates and analog switching chips to quickly switch between clock and power sources, reducing system downtime due to faults and improving system availability and fault tolerance. Secondly, the state recovery unit 403 stores system state information and restores the system to normal operation after fault resolution, reducing maintenance costs and system recovery difficulty, ensuring the system quickly returns to optimal operating conditions after a fault. Simultaneously, the use of configurable microcontrollers and logic gates allows for flexible adjustment of fault detection thresholds and switching strategies according to different application scenarios and system requirements, exhibiting strong adaptability and scalability to meet the needs of various complex systems.

[0068] The power management module 5 includes a multi-channel isolated power supply circuit 501, a dynamic load adjustment circuit 502, and a backup power supply circuit 503. The multi-channel isolated power supply circuit 501 is connected to the power interface of each module. The dynamic load adjustment circuit 502 is connected to the multi-channel isolated power supply circuit 501. The backup power supply circuit 503 is connected in parallel with the multi-channel isolated power supply circuit 501, and a switching switch is used to achieve main / backup switching. The multi-channel isolated power supply circuit 501 includes a DC-DC isolation module, an LDO regulator, a filter capacitor, a diode, and an inductor. The input pin of the DC-DC isolation module is connected to the main power supply, and the output pin is connected to the input pin of the LDO regulator. The output pin of the LDO regulator is connected to the power input pin of each sensor module. The filter capacitor is connected in parallel between the input and output pins and the ground pin of the LDO regulator. The diode is connected in series between the input pin of the DC-DC isolation module and the main power supply. An inductor is connected in series between the output pin of the DC-DC isolation module and the input pin of the LDO regulator; the dynamic load adjustment circuit 502 includes a digital potentiometer, a Buck-Boost converter, a current sensor, an operational amplifier, and a MOSFET; the output pin of the digital potentiometer is connected to the feedback pin of the Buck-Boost converter; the output pin of the Buck-Boost converter is connected to the power input pin of each module; the input pin of the current sensor is connected in series between the output pin of the Buck-Boost converter and the load, and its output pin is connected to the input pin of the operational amplifier; the output pin of the operational amplifier is connected to the control pin of the digital potentiometer; the gate pin of the MOSFET is connected to the control pin of the Buck-Boost converter, the drain pin is connected to the output pin of the Buck-Boost converter, and the source pin is connected to the load.

[0069] In practical implementation, the DC-DC isolation module uses TDK-Lambda's i7A series DC-DC isolation module, such as the i7A12S05. Its input pin Vin is connected to the main power supply, and its output pin Vout is connected to the input pin of the LDO regulator. Internally, this module contains a transformer, switching transistors, and control circuitry. It converts the input voltage to the required isolation voltage through high-frequency switching and achieves electrical isolation between the input and output through the transformer, effectively preventing power supply interference between different modules. For example, it converts the 12V main power supply voltage to a 5V isolation voltage.

[0070] The LDO regulator selected is the LM317. Its input pin receives the output voltage from the DC-DC isolation module, and its output pin is connected to the power input pin of each sensor module. It utilizes its internal regulating transistor and feedback circuit to stabilize the input voltage at the required low value, such as further regulating 5V to 3.3V, providing a stable operating voltage for the sensors.

[0071] Multiple ceramic capacitors, such as 0.1μF and 10μF capacitors, are used for filtering. These are connected in parallel between the input and output pins and ground pin of the LDO regulator to filter out high-frequency and low-frequency noise in the power supply, ensuring power purity. A 1N4007 diode is connected in series between the input pin of the DC-DC isolation module and the main power supply to prevent reverse connection and protect the power circuit. A 10μH power inductor is connected in series between the output pin of the DC-DC isolation module and the input pin of the LDO regulator. Together with the capacitors and other components within the DC-DC isolation module, they form a filtering circuit to further improve the ripple characteristics of the output voltage.

[0072] The digital potentiometer used is the AD5242, whose output pin is connected to the feedback pin of the Buck-Boost converter. Internally, it contains multiple resistor arrays and switches; digital signals control the switch states, adjusting the output resistance to achieve precise regulation of the feedback voltage. The Buck-Boost converter uses the TPS63070, whose output pin is connected to the power input pin of each module, enabling voltage boost / buck conversion under different input voltage and load conditions. The current sensor is the ACS712, whose input pin is connected in series between the Buck-Boost converter's output pin and the load. Its output pin is connected to the operational amplifier's input pin, allowing real-time monitoring of the load current. The operational amplifier is the LM358, whose output pin is connected to the control pin of the digital potentiometer, amplifying and processing the current sensor's output signal to provide precise control signals for the digital potentiometer. The MOSFET is the IRF540, whose gate pin is connected to the Buck-Boost converter's control pin. The MOSFET's on / off state is controlled by the Buck-Boost converter's control signals, achieving dynamic load regulation.

[0073] In implementation, the multi-channel isolation power supply circuit 501 draws power from the main power supply. The main power supply, after passing through a 1N4007 diode to prevent reverse connection, enters the i7A12S05 DC-DC isolation module. This module converts the voltage to the required isolation voltage and filters it through an inductor and a filter capacitor. Subsequently, an LM317 LDO regulator stabilizes the voltage to the operating voltage required by the sensors and supplies power to each sensor module through its output pin. For the dynamic load adjustment circuit 502, when the load changes, the ACS712 current sensor monitors the load current in real time and transmits the signal to the LM358 operational amplifier. The operational amplifier amplifies and processes the current signal and outputs a signal to the control pin of the AD5242 digital potentiometer. The digital potentiometer adjusts its output resistance based on this signal, thereby regulating the feedback voltage of the TPS63070 Buck-Boost converter. Simultaneously, the control signal of the TPS63070 controls the conduction level of the IRF540 MOSFET, dynamically adjusting the load to keep the output voltage stable when the load changes.

[0074] The circuit logic is as follows: the main power supply provides stable power to each module through diodes, a DC-DC isolation module, inductors, filter capacitors, and an LDO regulator. Simultaneously, load current changes are monitored by a current sensor, processed by an operational amplifier, and then the feedback voltage of the Buck-Boost converter is adjusted via a digital potentiometer. Combined with the control of the MOSFET, dynamic load adjustment is achieved to ensure stable power output.

[0075] In implementation, the multi-channel isolated power supply circuit 501 provides stable and isolated power to each module through a DC-DC isolation module and an LDO regulator, preventing power interference between modules, ensuring stable operation of sensors and other modules, and improving system reliability and stability, especially in complex electromagnetic environments. The dynamic load adjustment circuit 502 adjusts the power output in real time according to load changes, ensuring the system maintains a stable voltage under different load conditions, avoiding voltage fluctuations caused by load changes, improving system performance and efficiency, and extending the lifespan of system equipment. Furthermore, through the coordinated operation of the Buck-Boost converter and the dynamic load adjustment circuit 502, the power supply can be adjusted according to actual needs, avoiding unnecessary energy waste, reducing overall system power consumption, and extending battery life for battery-powered systems. Diodes prevent reverse connection of the power supply, avoiding circuit damage caused by incorrect power connection, providing a certain degree of protection for the system, and enhancing system safety.

[0076] The system will be illustrated below through specific embodiments:

[0077] In autonomous vehicles, the sensor integrated perception fusion system achieves environmental perception and decision-making through the following steps: First, the camera, radar, and lidar in the multi-sensor module 1 collect environmental data respectively. The timestamp generation unit 103 adds a high-precision timestamp to the data from each sensor, achieving nanosecond-level accuracy. The distributed clock synchronization module 2 provides a unified clock reference for each sensor through the first clock source 201TCXO and the clock distribution circuit 202. The clock synchronization detection circuit 204 monitors the clock synchronization status of each sensor in real time to ensure timestamp consistency. The timestamp alignment unit 302 in the data processing and fusion module 3 aligns the multi-sensor data according to the timestamps.

[0078] The data processing unit 303 runs Kalman filtering and deep learning algorithms, fusing multi-sensor data to generate environmental perception results such as obstacle position, speed, and category. These perception results are transmitted to the vehicle control system for path planning and decision-making.

[0079] The fault detection unit 401 monitors the system status in real time. If the master clock fails, the switching control unit 402 automatically switches to the backup clock source 203. If a sensor module fails, the system switches to the backup sensor module to ensure uninterrupted data acquisition. The multi-channel isolated power supply circuit 501 provides independent power to each module, reducing voltage fluctuations. The dynamic load adjustment circuit 502 dynamically adjusts the power output according to the system load, ensuring stable operation of the system under high load.

[0080] In this implementation, cameras, radar, and lidar are deployed in the front, rear, left, and right directions of the vehicle to ensure all-around coverage. The local clock unit 102 and timestamp generation unit 103 are integrated inside the sensors and connected to the data processing and fusion module 3 via a high-speed data cable.

[0081] The first clock source 201 and the second clock source 203 are located within the vehicle's central control unit, close to the clock distribution circuit 202. The clock distribution circuit 202 is connected to the local clock units 102 of each sensor via shielded cables to reduce signal interference.

[0082] The data receiving unit 301, timestamp alignment unit 302, and data calculation unit 303 are integrated within the vehicle's central control unit and communicate with each sensor module via a high-speed bus. The fault detection unit 401 and switching control unit 402 are located within the central control unit, close to the distributed clock synchronization module 2. The state recovery unit 403 is connected to the power management circuits of each module to achieve rapid recovery.

[0083] The multi-channel isolated power supply circuit 501 and the dynamic load adjustment circuit 502 are located within the vehicle power distribution unit and are connected to each module via independent power lines. The backup power supply circuit 503 is connected in parallel with the main power supply, and the main / backup switching is achieved through a transfer switch.

[0084] While specific embodiments of this utility model have been described above, those skilled in the art should understand that these specific embodiments are merely illustrative. Those skilled in the art can omit, substitute, and modify the details of the above methods and systems in various ways without departing from the principles and essence of this utility model. For example, combining the above method steps to perform substantially the same function and achieve substantially the same result according to substantially the same method falls within the scope of this utility model. Therefore, the scope of this utility model is defined only by the appended claims.

Claims

1. A comprehensive sensing fusion system, characterized in that, include: Multi-sensor module (1), distributed clock synchronization module (2), data processing and fusion module (3), fault tolerance and redundancy management module (4), and power management module (5); The multi-sensor module (1) includes a sensor interface unit (101), a local clock unit (102), and a timestamp generation unit (103); The distributed clock synchronization module (2) includes a first clock source (201), a clock distribution circuit (202), a second clock source (203), and a clock synchronization detection circuit (204); The data processing and fusion module (3) includes a data receiving unit (301), a timestamp alignment unit (302), and a data calculation unit (303); The fault tolerance and redundancy management module (4) includes a fault detection unit (401), a switching control unit (402), and a state recovery unit (403); The power management module (5) includes a multi-channel isolated power supply circuit (501), a dynamic load adjustment circuit (502), and a backup power supply circuit (503); The output of the multi-sensor module (1) is connected to the input of the distributed clock synchronization module (2) and the data processing and fusion module (3); the output of the distributed clock synchronization module (2) is connected to the input of the multi-sensor module (1) and the fault tolerance and redundancy management module (4); the output of the data processing and fusion module (3) is connected to the input of the fault tolerance and redundancy management module (4); the output of the fault tolerance and redundancy management module (4) is connected to the input of the distributed clock synchronization module (2), the multi-sensor module (1) and the power management module (5); the power management module (5) supplies power to all modules.

2. The integrated sensing fusion system according to claim 1, characterized in that: The sensor interface unit (101) is connected to the local clock unit (102) via a clock signal line; the timestamp generation unit (103) receives the local clock signal from the local clock unit (102), adds a timestamp to the sensor data of the sensor interface unit (101), and then transmits it to the data processing and fusion module (3).

3. The integrated sensing fusion system according to claim 1, characterized in that: The first clock source (201) is connected to the local clock unit (102) of the multi-sensor module (1) through the clock distribution unit; the second clock source (203) is connected in parallel with the clock distribution circuit (202) and is switched by a switching switch; the clock synchronization detection circuit (204) is connected to the timestamp generation unit (103) of the multi-sensor module (1).

4. The integrated sensing fusion system according to claim 3, characterized in that: The first clock source (201) and the second clock source (203) are temperature-compensated crystal oscillators or temperature-controlled crystal oscillators.

5. The integrated sensing fusion system according to claim 1, characterized in that: The clock distribution circuit (202) includes a clock source, a buffer, a clock distribution chip, a resistor network, and a capacitor; the output pin of the clock source is connected to the input pin of the buffer; the output pin of the buffer is connected to the input pin of the clock distribution chip; the output pin of the clock distribution chip is connected to the local clock circuit input pin of each sensor module; the resistor network is connected in series on the clock signal line and connected to the output pin of the clock distribution chip and the input pin of the multi-sensor module (1); the capacitor is connected in parallel between the power supply pin and the ground pin.

6. The integrated sensing fusion system according to claim 1, characterized in that: The data receiving unit (301) is connected to the timestamp generation unit (103) of the multi-sensor module (1); the timestamp alignment unit (302) is connected to the data receiving unit (301), and transmits the aligned data to the data calculation unit (303).

7. The integrated sensing fusion system according to claim 1, characterized in that: The fault detection unit (401) is connected to the clock synchronization detection circuit (204) of the distributed clock synchronization module (2) and the sensor interface unit (101) of the multi-sensor module (1); the switching control unit (402) is connected to the second clock source (203) of the distributed clock synchronization module (2) and the backup power circuit (503) of the power management module (5); the state recovery unit (403) is connected to the switching control unit (402).

8. The integrated sensing fusion system according to claim 1, characterized in that: The multi-channel isolated power supply circuit (501) is connected to the power interface of each module; the dynamic load adjustment circuit (502) is connected to the multi-channel isolated power supply circuit (501); the backup power supply circuit (503) is connected in parallel with the multi-channel isolated power supply circuit (501), and the main backup switching is realized through a switching switch.

9. The integrated sensing fusion system according to claim 1, characterized in that: The multi-channel isolated power supply circuit (501) includes a DC-DC isolation module, an LDO regulator, a filter capacitor, a diode, and an inductor; the input pin of the DC-DC isolation module is connected to the main power supply, and the output pin is connected to the input pin of the LDO regulator; the output pin of the LDO regulator is connected to the power input pin of each sensor module; the filter capacitor is connected in parallel between the input and output pins and the ground pin of the LDO regulator; the diode is connected in series between the input pin of the DC-DC isolation module and the main power supply; and the inductor is connected in series between the output pin of the DC-DC isolation module and the input pin of the LDO regulator.

10. A comprehensive sensing fusion system according to claim 1, characterized in that: The dynamic load adjustment circuit (502) includes a digital potentiometer, a Buck-Boost converter, a current sensor, an operational amplifier, and a MOSFET. The output pin of the digital potentiometer is connected to the feedback pin of the Buck-Boost converter. The output pin of the Buck-Boost converter is connected to the power input pin of each module. The input pin of the current sensor is connected in series between the output pin of the Buck-Boost converter and the load, and its output pin is connected to the input pin of the operational amplifier. The output pin of the operational amplifier is connected to the control pin of the digital potentiometer. The gate pin of the MOSFET is connected to the control pin of the Buck-Boost converter, the drain pin is connected to the output pin of the Buck-Boost converter, and the source pin is connected to the load.

Images