A sensor integration device for autonomous driving

Abstract

The utility model provides a kind of sensor integrated device for automatic driving.The sensor integrated device includes at least two sensors integrated sensor integrated unit;Sensor data serial unit is connected with each described sensor respectively, and according to the sensing original data of at least one sensor generates serial data stream;Transmission interface is connected with the central domain control processor of described sensor data serial unit and described;Central domain control processor receives the serial data stream that the sensor data serial unit issues by the transmission interface, and according to the serial data stream, data fusion is carried out to the at least two sensors.The sensor integrated device provided by the utility model solves the problems of large space occupation, complex wiring, data synchronization difficulty caused by scattered installation of sensors, and improves the environmental perception ability and reliability of automatic driving system.

Description

Technical Field

[0001] This utility model relates to the field of autonomous driving technology, and in particular to a sensor integration device for autonomous driving. Background Technology

[0002] With the rapid development of autonomous driving technology, vehicles are placing higher demands on the accuracy and reliability of their environmental perception. Autonomous driving systems typically require the fusion of information from multiple sensors to achieve comprehensive perception of the surrounding environment. Currently, common sensor combinations include camera modules, millimeter-wave radar, lidar, ultrasonic radar, 4D imaging radar, positioning-type wheel speedometers / IMU inertial sensors, RTK, collision sensors, and rain / light sensors. Among these sensors, for example, camera modules can provide rich visual information, such as object color and shape; millimeter-wave radar has strong penetration capabilities and can operate normally in adverse weather conditions such as rain and fog, providing information such as object distance and speed; lidar can achieve accurate 3D spatial perception and is suitable for nighttime and low-light environments, but rain, fog, and dust can still affect lidar performance.

[0003] However, in existing technologies, sensors are typically installed as independent sensor units on vehicles, and this decentralized installation approach has several drawbacks. First, decentralized installation occupies a significant amount of vehicle space, which is detrimental to vehicle layout design, especially for smaller vehicles or autonomous vehicles with high space requirements. Second, multiple independent sensors require separate wiring, leading to complex wiring and increasing vehicle production costs and maintenance difficulties. Furthermore, due to the different installation positions and angles of each sensor, the data they collect exhibit temporal and spatial discrepancies, posing significant challenges to subsequent data fusion and reducing the accuracy and real-time performance of environmental perception.

[0004] Therefore, how to provide a sensor integration device with a compact structure, high data synchronization accuracy, and strong anti-interference capability has become an urgent problem to be solved by those skilled in the art. Utility Model Content

[0005] This invention provides a sensor integration device for autonomous driving, which solves the problems of large space occupation, complex wiring, and difficulty in data synchronization caused by the dispersed installation of sensors in the prior art, thereby improving the environmental perception capability and reliability of the autonomous driving system.

[0006] This invention provides a sensor integration device for autonomous driving. The sensor integration device includes: a sensor integration unit integrating at least two sensors; a sensor data serialization unit connected to each of the sensors and generating a serial data stream based on the raw sensor data from the at least one sensor; a transmission interface connected to both the sensor data serialization unit and a central domain controller; and a central domain controller receiving the serial data stream from the sensor data serialization unit through the transmission interface and performing data fusion on the at least two sensors based on the serial data stream.

[0007] In some embodiments, the at least two sensors include: at least two different types of sensors; the at least two different types of sensors are mounted side by side on the same cascade plate and / or base plate.

[0008] In some embodiments, the at least two sensors include: at least one millimeter-wave radar and at least one camera module.

[0009] In some embodiments, an opening is provided between the transmitting antenna and the receiving antenna of the millimeter-wave radar on the PCBA board where the millimeter-wave radar is located; the camera module is assembled in the opening.

[0010] In some embodiments, the at least two sensors include at least two different types of sensors; the pointing deviation of the field-of-view center axis of the at least two different types of sensors is less than or equal to a preset angle.

[0011] In some embodiments, the at least two sensors are connected to the central domain controller via the transmission interface; the at least two sensors and the central domain controller transmit synchronization trigger signals for the at least two sensors via the transmission interface.

[0012] In some embodiments, the transmission interface includes a coaxial cable employing high-speed serial technology.

[0013] In some embodiments, the at least two sensors include: a plurality of millimeter-wave radars, wherein the antennas of the plurality of millimeter-wave radars are cascaded to form an antenna array.

[0014] In some embodiments, the sensor data serialization unit includes: a time-division multiplexing encoder connected to each of the sensors, used to divide the raw sensor data of at least one sensor into time slices; an address encoder connected to the time-division multiplexing encoder, used to encode the divided raw sensor data into data frames with address identifiers; a code-division encoder connected to the address encoder, used to encode the data frames; and a serializer connected to the code-division encoder and the transmission interface, used to convert the encoded data frames into a serial data stream and send it to the central domain controller processor through the transmission interface.

[0015] In some embodiments, the central domain controller processor includes: a deserializer connected to the transmission interface for deserializing the serial data stream to obtain encoded data frames; a code division decoder connected to the deserializer for decoding the encoded data frames to obtain data frames; an address decoder connected to the code division decoder for parsing the address identifier of the data frames to separate the raw sensing data from the at least two sensors; and a fusion processor connected to the address decoder for fusion processing of the raw sensing data from the at least two sensors.

[0016] This utility model achieves the following beneficial effects through a sensor integration device:

[0017] Compact structure: By physically integrating the camera module and millimeter-wave radar through side-by-side or through-hole methods, the space occupied by the sensor system is greatly reduced, which is beneficial to the vehicle layout design.

[0018] Simplified wiring: The sensor data serialization unit generates a serial data stream from multiple sensors and transmits it to the central domain controller via a coaxial cable bundle, reducing the number of wires and lowering production costs and maintenance difficulty.

[0019] High data synchronization accuracy: The central domain controller processor achieves spatiotemporal coordinate alignment of data from different sensors by comparing trigger signals and timestamps, thereby improving the accuracy of data fusion.

[0020] Strong anti-interference capability: The perforation method involves opening a hole between the transmitting and receiving antennas of the millimeter-wave radar. The reasonable design of the hole position and size reduces interference to the performance of the millimeter-wave radar.

[0021] High transmission efficiency: The SerDes technology, with a speed of at least 10Gbps, ensures high-speed data transmission and meets the real-time requirements of autonomous driving systems. Attached Figure Description

[0022] To make the content of this utility model easier to understand, the present utility model will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein:

[0023] Figure 1 This is a schematic diagram of a sensor device with multiple sensors transmitting data independently, as is common in the prior art.

[0024] Figure 2 This is a schematic block diagram of the sensor integration device 200 according to some embodiments of the present invention;

[0025] Figure 3 This is a schematic diagram showing the physical integration of a camera module and a millimeter-wave radar in a side-by-side manner;

[0026] Figure 4 This is a schematic diagram showing the structure of the camera module and millimeter-wave radar physically integrated through a perforation method;

[0027] Figure 5 This is a schematic diagram showing the structure in which a camera module and a millimeter-wave radar are physically integrated simultaneously through a side-by-side and perforated method;

[0028] Figure 6 This is a schematic diagram showing the block diagram of the sensor data serial unit;

[0029] Figure 7 This is a schematic diagram showing the block diagram of the central domain controller processor. Detailed Implementation

[0030] Figure 1 This is a schematic diagram of a sensor device in the prior art where multiple sensors transmit data independently. For example... Figure 1 As shown, the device includes a sensor group 101, a sensor data processor 102, a transmission interface 103, a processor 104, and any other components that can be used to perform sensor data processing. In some embodiments, the sensor group 101 may include a first sensor, a second sensor, and a third sensor. The sensor group 101 may include more or fewer sensors; for simplicity, this invention uses three sensors as an example. The first sensor may be one or more camera modules, the second sensor may be at least one millimeter-wave radar, and the third sensor may be at least one lidar.

[0031] In some embodiments, the number of processors in the sensor data processor 102 can be determined according to the number of different sensors in the sensor group 101. According to an example of the present invention, the sensor data processor 102 includes a first sensor data processor, a second sensor data processor, and a third sensor data processor, wherein the first sensor data processor processes the sensing data of the first sensor, the second sensor data processor processes the sensing data of the second sensor, and the third sensor data processor processes the sensing data of the third sensor. In some embodiments, the first sensor processor may be an image processor, an image signal processor (ISP), etc.; the second sensor processor may be a millimeter-wave radar processor; and the third sensor processor may be a lidar signal processor.

[0032] Similarly, the transmission interface 103 includes a first transmission interface, a second transmission interface, and a third transmission interface, which correspond to the first sensor and the first sensor data processor, the second sensor and the second sensor data processor, and the third sensor and the third sensor data processor, respectively.

[0033] exist Figure 1 In the aforementioned device, the sensors are installed separately. This separate sensor architecture, coupled with the increasing number of sensors in autonomous driving, results in dozens of transmission harnesses, leading to a significant increase in cost and a decrease in reliability, while also posing a major challenge to lightweight harnesses. Furthermore, with the continuous development of autonomous driving sensor computing methods, camera perception methods, millimeter-wave and laser data processing algorithms are constantly being updated. The pre-processing of sensor data from each sensor makes OTA upgrades of the sensor data processor extremely labor-intensive and difficult. Finally, due to differences in data acquisition frequency and accuracy among various sensors, different sensors exhibit different errors and drifts, and each sensor operates independently without temporal and spatial correlation. Therefore, the processor cannot effectively correct for these differences when fusing sensor data, ensuring the timeliness and accuracy of data fusion.

[0034] To address the aforementioned problems, this invention provides a sensor integration device for autonomous driving. Figure 2 This is a schematic block diagram of a sensor integration device 200 according to some embodiments of the present invention.

[0035] like Figure 2As shown, the sensor integration device includes a sensor integration unit 201, a sensor data serialization unit 202, a transmission interface 203, a central domain controller 204, and any other components that can be used for sensor data processing. In some embodiments, the sensor integration unit 201 may integrate at least two sensors. In some embodiments, the sensor integration unit 201 may include a first sensor, a second sensor, and a third sensor. The sensor integration unit 201 may include more or fewer sensors; for simplicity, this invention uses three sensors as an example. The first sensor may include at least one camera module, the second sensor may include at least one millimeter-wave radar, and the third sensor may include at least one lidar.

[0036] In some embodiments, the sensor integration unit 201 can integrate at least two different types of sensors. The at least two different types of sensors can be mounted side-by-side on the same cascade board and / or base plate. For example, when the at least two different types of sensors include at least one camera module or at least one millimeter-wave radar, the at least one camera module and the at least one millimeter-wave radar can be mounted side-by-side on the same cascade board and / or base plate.

[0037] In some embodiments, when the at least two different types of sensors include at least one camera module or at least one millimeter-wave radar, the at least one camera module or at least one millimeter-wave radar can be mounted via a through-hole. For example, an opening is provided between the transmitting antenna and the receiving antenna of the millimeter-wave radar on the PCBA board where the millimeter-wave radar is located; the camera module is mounted in the opening. This opening can be located between the transmitting antenna and the receiving antenna.

[0038] In some embodiments, the sensor integration unit 201 can integrate multiple millimeter-wave radars. In some embodiments, the antennas of the multiple millimeter-wave radars in the sensor integration unit 201, such as receiving antennas or transmitting antennas, can be cascaded into an antenna array. The multiple millimeter-wave radars can constitute an integrated phased array radar.

[0039] In some embodiments, when at least two different types of sensors are integrated in the sensor integration unit 201, the pointing deviation of the central axis of the field of view of the at least two different types of sensors is less than or equal to a preset angle. In some embodiments, the preset angle may be 3 degrees or a smaller angle.

[0040] In some embodiments, the sensor data serialization unit 202 is connected to the sensor integration unit 201. In some embodiments, the sensor data serialization unit 202 may be connected to at least one sensor in the sensor integration unit 201, and generate a serial data stream based on the raw sensing data of the at least one sensor. For example, the sensor data serialization unit 202 may generate a serial data stream based on the raw sensing data of at least one camera module, the raw sensing data of at least one millimeter-wave radar, and / or the raw sensing data of at least one lidar. In some embodiments, the sensor data serialization unit 202 may include a time-division multiplexing encoder, wherein the time-division multiplexing encoder is connected to each sensor and is used to divide the raw sensing data of each sensor in the sensor group into time slices; the sensor data serialization unit also includes an address encoder, connected to the time-division multiplexing encoder, for encoding the segmented raw sensing data into data frames with address identifiers to ensure the consistency of sampling by the back-end chip. The sensor serialization unit 202 may also include a code division encoder, wherein the code division encoder is connected to the address encoder and is used to encode the data frames into data suitable for high-speed transmission. The sensor serial unit 202 may further include a serializer, which is connected to the code division encoder and the transmission interface 203 respectively. The serializer is used to convert the encoded data frame into a serial data stream and send it to the central domain controller processor through the transmission interface 203.

[0041] In some embodiments, the transmission interface 203 may be a coaxial cable. In some embodiments, the transmission interface 203 may be a coaxial cable employing high-speed serial (SerDes) technology. That is, the transmission interface 203 may employ at least 10Gbps-level high-speed serial (SerDes) technology to transmit the serial data stream to the central domain controller processor via the coaxial cable.

[0042] In some embodiments, high-speed serial technology is widely used in the field of autonomous driving. For example, in autonomous driving and advanced driver assistance systems (ADAS), SerDes technology is used to transmit video data collected by onboard cameras at high speed to the central processing unit for real-time image processing and analysis. SerDes technology is also used to rapidly transmit data from radar sensors to the computing unit, improving the vehicle's perception of its surroundings, which is particularly important for obstacle detection and avoidance functions in autonomous driving. SerDes technology transmits high-resolution video signals from the main control chip to the instrument panel, rearview mirror display, etc., ensuring the driver receives clear information. SerDes technology is also used to fuse data from different sensors (such as cameras and radar) to improve the accuracy and reliability of environmental perception, supporting autonomous driving. Furthermore, SerDes technology can be combined with traditional protocols such as CAN and LIN to achieve even higher data transmission rates.

[0043] In some specific embodiments, when targeting high-resolution camera modules, SerDes technology can select one or more of the following: Gigabit Multimedia Serial Link (GMSL), Flat Panel Display Link (FPD-Link), High-Speed ​​Moving Terminal (HSMT), A-PHY (Automotive-Physical Layer), or ASA (ASA Motion Link), supporting at least 6Gbps, to ensure real-time data transmission without loss. In other specific embodiments, when targeting radar data, FPD-Link or EthernetAVB with speeds around 1Gbps can be selected to meet the requirements for radar data transmission. In some embodiments, the radar data can share a transmission link with other sensors; for example, it can use the transmission link of the camera module. To achieve more efficient and reliable transmission, SerDes technology preferentially selects a transmission link with speeds of at least 10Gbps.

[0044] In some embodiments, the central domain controller 204 is configured to communicate with the sensor data serial unit via the transmission interface 203 and receive the serial data stream from the sensor data serial unit. In some embodiments, the central domain controller 204 is further configured to deserialize and decode the serial data stream to obtain the raw sensor data of the sensor group; wherein the raw sensor data includes raw data from the first sensor, the second sensor, and the third sensor, etc.

[0045] In some embodiments, the central domain controller 204 includes a deserializer, a code division decoder, an address decoder, and a fusion processor. The deserializer is connected to the transmission interface 203 and is used to deserialize the serial data stream to obtain encoded data frames. The code division decoder is connected to the deserializer and is used to decode the encoded data frames to obtain data frames. The address decoder is connected to the code division decoder and is used to parse the address identifier of the data frames to separate the raw sensing data from the at least two sensors. The fusion processor, connected to the address decoder, is used to perform fusion processing on the raw sensing data from the at least two sensors.

[0046] In some embodiments, the central domain controller 204 can also be used to compare the trigger signals of the different sensors with the data timestamps of the original sensor data, thereby achieving spatiotemporal coordinate alignment between the original data of different sensors.

[0047] In some embodiments, the at least two sensors are connected to the central domain controller via the transmission interface; the at least two sensors and the central domain controller transmit synchronization trigger signals for the at least two sensors via the transmission interface. The central domain controller can send synchronization trigger signals to at least two sensors in the sensor integration unit via the transmission interface. For example, the central domain controller can send a trigger signal to the first sensor and to other sensors, such as the second and / or the third sensor, via the transmission interface. In some embodiments, each sensor in the sensor integration unit, such as the first, second, and / or third sensor, can send a synchronization trigger signal to the central domain controller via the transmission interface for synchronization within the central domain controller.

[0048] In some embodiments, the fusion processor performs fusion processing on the raw sensor data from different sensors after aligning the spatiotemporal coordinates between the raw data from different sensors. In some embodiments, fusion processing of sensor data refers to combining data information from multiple sensors to achieve more accurate environmental perception and decision-making. In some embodiments, sensor data fusion algorithms include Kalman filtering, extended Kalman filtering, unscented Kalman filtering, etc.

[0049] The aforementioned sensor integration device reduces the number of sensor processing chips, significantly lowering BOM costs. For example, in a traditional five-camera-five-millimeter-wave radar-five-LiDAR (5V5R5L) architecture, five camera image processing chips, five radar processing chips, and five LiDAR processing chips are required. However, in the sensor integration device, all of these are integrated into the central domain controller processor (CNC). The number of wiring harnesses is reduced from 15 to 5, significantly lowering costs and meeting the requirements for lightweight wiring harnesses. Furthermore, it enables the independent MCU for the front-end radar head and reduces a large number of core board components, thereby reducing power consumption and battery consumption.

[0050] Furthermore, the aforementioned sensor integration device also achieves significant performance improvements and simplifies and facilitates system maintenance. For example, it enables pixel-level real-time spatiotemporal alignment of sensor data, significantly improving fusion accuracy and greatly enhancing confidence and robustness. As another example, the device reduces the number of software types from four to one, allowing OTA upgrades to be performed only in the central domain controller, greatly reducing software development and maintenance workload, simplifying system architecture, improving system reliability, and facilitating future maintenance and upgrades.

[0051] The table below compares the application of the traditional 5V5R5L system architecture with the aforementioned sensor integration device:

[0052] Camera image processing chip 5 0 Radar processing chip 5 0 LiDAR processing chip 5 0 Sensor harness 15 articles 5 items Maintenance software types 4 types 1 type

[0053] In some embodiments, the first sensor, the second sensor, and / or the third sensor in the sensor integration unit 201 of the sensor integration device can be physically integrated in a side-by-side or through-hole manner. Taking a camera module and a millimeter-wave radar as examples, embodiments of physical integration of a camera module and a millimeter-wave radar in a side-by-side manner and / or physical integration of a camera module and a millimeter-wave radar in a through-hole manner are provided below, respectively.

[0054] Example 1

[0055] Figure 3 This is a schematic diagram illustrating the physical integration of a camera module and a millimeter-wave radar in a side-by-side manner. In this embodiment, the sensor integration unit includes a camera module and a millimeter-wave radar. Figure 3As shown, the camera module and millimeter-wave radar are mounted side-by-side on a cascaded board. The cascaded board is made of high-strength PCB material, possessing excellent mechanical and electrical properties. The housings of the camera module and the millimeter-wave radar are secured to the cascaded board with screws, ensuring a firm installation. To precisely control the deviation of the field-of-view center axis of the camera module and the millimeter-wave radar, positioning slots and adjustment mechanisms are provided on the cascaded board. The positioning slots are used to initially determine the installation positions of the camera module and the millimeter-wave radar. The adjustment mechanism includes fine-tuning screws and shims; by adjusting the fine-tuning screws, the angles of the camera module and the millimeter-wave radar can be precisely adjusted so that the deviation of their field-of-view center axes is less than or equal to 3 degrees.

[0056] Example 2

[0057] Figure 4 This is a schematic diagram illustrating the physical integration of a camera module and a millimeter-wave radar via a perforated design. In this embodiment, the sensor integration unit also includes a camera module and a millimeter-wave radar, but they are physically integrated using a perforated method. Figure 4 As shown, a hole is first made between the transmitting and receiving antennas on the PCBA board where the millimeter-wave radar is located. The position and size of the hole were precisely calculated and simulated to ensure that it would not significantly affect the antenna performance of the millimeter-wave radar. The hole is circular with a diameter of 5mm and is located precisely in the middle of the transmitting and receiving antennas. Next, a camera module is mounted at the hole, with its lens passing through the hole and facing forward of the vehicle. The circuit board of the camera module is connected to the PCBA board by soldering to ensure the stability of the electrical connection. To further reduce interference, a metal shielding layer is placed around the camera module, which is connected to the ground plane of the PCBA board to form a good electromagnetic shielding effect.

[0058] In some embodiments, when a camera module and a millimeter-wave radar are physically integrated via a perforation method, the data acquisition synchronization can be higher due to their closer physical location, thereby making the sensor data serialization unit more efficient in processing raw sensor data.

[0059] Example 3

[0060] Figure 5 This is a schematic diagram illustrating the physical integration of a camera module and a millimeter-wave radar using both side-by-side and perforated methods. In this embodiment, the sensor integration unit includes two camera modules and two millimeter-wave radars, physically integrated using a combination of side-by-side and perforated methods. Figure 5As shown, two millimeter-wave radars are mounted side-by-side on the base plate. Each millimeter-wave radar's PCBA board has a hole for mounting a camera module. This integration method fully utilizes the advantages of both side-by-side and through-hole methods, ensuring compact installation of multiple sensors while improving the synchronization and accuracy of data acquisition.

[0061] In some embodiments, the installation methods described in Embodiments 1, 2, and 3 above can ensure that the pointing deviation of the central axis of the field of view of the at least two different types of sensors is less than or equal to a preset angle. In some embodiments, the preset angle can be 3 degrees or less.

[0062] In some embodiments, the sensing data serialization unit in the sensor integration device is primarily used to generate a serial data stream based on the raw sensing data of the sensor group. Figure 6 This is a schematic diagram illustrating a block diagram of a sensor data serialization unit according to some embodiments of the present invention. As shown, the sensor data serialization unit includes a time-division multiplexing encoder 601 and an address encoder 602. The time-division multiplexing encoder 601 is connected to at least two sensors in the sensor group and employs a high-speed digital signal processing chip, capable of segmenting the raw sensor data of the sensor group (e.g., a camera module, millimeter-wave radar, or lidar) according to time slices. Specifically, the time-division multiplexing encoder 601 divides a time period into multiple time slices, each time slice being allocated to the camera module or millimeter-wave radar for sequential data acquisition and transmission. The address encoder 602 is connected to the time-division multiplexing encoder. The address encoder can encode the segmented raw sensor data into data frames with address identifiers. Each data frame contains data content and corresponding sensor address information, enabling the central domain controller processor to identify the source of the data. In some embodiments, the sensor data serialization unit further includes a code division encoder 603, which is connected to the address encoder 602. The code division encoder 603 can be used to convert the raw sensor data, i.e., data frames, processed by the time division multiplexing encoder and the address encoder into a serial data stream suitable for high-speed transmission. In some embodiments, the sensor data serialization unit further includes a serializer 604, which is connected to the code division encoder 604 and the transmission interface, and is used to convert the encoded data frames into a serial data stream and send it to the central domain controller processor through the transmission interface.

[0063] In some embodiments, the central domain controller processor in the sensor integration device is primarily used to receive and process the serial data stream of the sensor data serial unit. Figure 7This is a schematic diagram illustrating the block diagram of the central domain controller processor. In some embodiments, the central domain controller processor is connected to the sensor data serialization unit via a transmission interface, wherein the transmission interface can be a coaxial cable using at least 10Gbps SerDes technology. As shown, the central domain controller processor includes a deserializer 701, a code division decoder 702, an address decoder 703, and a fusion processor 704. The deserializer is connected to the transmission interface, receives the serial data stream, and deserializes the serial data stream, converting the serial data into parallel data to obtain encoded data frames. The code division decoder 702 is connected to the deserializer 701 and is used to decode the encoded data frames to obtain data frames. The address decoder 703 is connected to the code division decoder 702 and is used to parse the address representation in the data frames to obtain raw sensor data from different sensors (e.g., at least one camera module and / or at least one millimeter-wave radar). The fusion processor 704 is connected to the address decoder and can align the raw sensor data from different sensors, such as aligning spatiotemporal coordinates. The fusion processor can achieve spatiotemporal coordinate alignment between different sensor raw data by comparing the data timestamps of the sensor raw data generated under the synchronous trigger signal (e.g., at least one camera module and / or at least one millimeter-wave radar).

[0064] In some embodiments, the synchronization trigger signals of the different sensors can be sent to the central domain processor by the different sensors through a transmission interface. For example, the camera module can send a trigger signal to the central domain processor, and the millimeter-wave radar can also send a synchronization trigger signal to the central domain processor, thereby enabling the central domain processor to align the raw data of the different sensors based on the trigger signal and the synchronization trigger signal. In other embodiments, the trigger signals of the different sensors can be sent by the central domain processor to different sensors through a transmission interface; for example, the central domain processor can send a trigger signal to the camera module and a synchronization trigger signal to the millimeter-wave radar. Then, the central domain processor can align the raw sensing data of the different sensors (e.g., at least one camera module and / or at least one millimeter-wave radar) based on the trigger signal and the synchronization trigger signal.

[0065] Because this embodiment involves a large number of sensors, the spatiotemporal coordinate alignment algorithm is more complex, requiring consideration of the relative positions and angular relationships between multiple sensors. Finally, the central domain controller performs fusion processing on the aligned raw sensor data, employing algorithms such as Kalman filtering to integrate information from various sensors and generate more accurate and reliable environmental perception results.

[0066] After aligning the raw sensor data from the different sensors, the fusion processor can perform fusion processing on the aligned raw sensor data, combining the information from each sensor to generate a more accurate and reliable environmental perception result. The sensor data fusion algorithm may include one or more of the following: Kalman filtering algorithm, extended Kalman filtering algorithm, and unscented Kalman filtering algorithm.

[0067] This invention provides a sensor integration device for autonomous driving. It achieves physical integration of a camera module and a millimeter-wave radar through side-by-side or through-hole methods. Combined with a sensor data serialization unit and a central domain control processor, it solves the problems of large space occupation, complex wiring, and difficult data synchronization caused by the dispersed installation of sensors in existing technologies. Experimental results show that this device has advantages such as compact structure, high data synchronization accuracy, strong anti-interference ability, and high transmission efficiency. It can significantly improve the environmental perception capability and reliability of autonomous driving systems and has broad application prospects.

Claims

1. A sensor integration device for autonomous driving, characterized in that, include: A sensor integration unit that integrates at least two sensors; A sensing data serialization unit is connected to each of the sensors respectively, and generates a serial data stream based on the raw sensing data of at least one sensor; The transmission interface is connected to the sensor data serial unit and the central domain controller processor, respectively. The central domain controller receives the serial data stream from the sensor data serial unit through the transmission interface, and performs data fusion on the at least two sensors based on the serial data stream.

2. The sensor integration device according to claim 1, characterized in that, The at least two sensors include: at least two different types of sensors; The at least two different types of sensors are mounted side-by-side on the same cascade plate and / or base plate.

3. The sensor integration device according to claim 1, characterized in that, The at least two sensors include: at least one millimeter-wave radar and at least one camera module.

4. The sensor integration device according to claim 3, characterized in that, An opening is provided between the transmitting antenna and the receiving antenna of the millimeter-wave radar on the PCBA board where the millimeter-wave radar is located. The camera module is assembled in the opening.

5. The sensor integration device according to claim 1, characterized in that, The at least two sensors include: at least two different types of sensors; The pointing deviation of the central axis of the field of view of the at least two different types of sensors is less than or equal to a preset angle.

6. The sensor integration device according to claim 1, characterized in that, The at least two sensors are connected to the central domain controller processor via the transmission interface; The at least two sensors and the central domain controller processor transmit synchronous trigger signals for the at least two sensors through the transmission interface.

7. The sensor integration device according to claim 1, characterized in that, The transmission interface includes a coaxial cable employing high-speed serial technology.

8. The sensor integration device according to claim 1, characterized in that, The at least two sensors include: multiple millimeter-wave radars, the antennas of the multiple millimeter-wave radars being cascaded to form an antenna array.

9. The sensor integration device according to claim 1, characterized in that, The sensing data serialization unit includes: A time-division multiplexing encoder is connected to each of the sensors and is used to divide the raw sensing data of at least one sensor into time slices. An address encoder, connected to the time-division multiplexing encoder, is used to encode the segmented raw sensing data into data frames with address identifiers. A code division encoder, connected to the address encoder, is used to encode the data frame; The serializer is connected to both the code division encoder and the transmission interface, and is used to convert the encoded data frame into a serial data stream, which is then sent to the central domain controller processor through the transmission interface.

10. The sensor integration device according to claim 1, characterized in that, The central domain controller processor includes: A deserializer, connected to the transmission interface, is used to deserialize the serial data stream and obtain the encoded data frame; A code divider decoder, connected to the deserializer, is used to decode the encoded data frame to obtain the data frame; The address decoder, connected to the code division decoder, is used to parse the address identifier of the data frame and separate the original sensing data of the at least two sensors. A fusion processor, connected to the address decoder, is used to perform fusion processing on the raw sensing data from the at least two sensors.

Images