Interface system on a chip for automotive applications

Abstract

An interface system (12) on a chip comprises a plurality of input interfaces (42) designed for at least two different communication protocols and designed to receive sensor data (22) from a plurality of sensors (16a, 16b, 16c); a preprocessing unit (46) for preprocessing the sensor data (22); and at least one output interface (44) designed for a specific communication protocol for outputting the sensor data (22).

Description

[0001] The invention relates to an interface system on a chip and a vehicle system with such an interface system.

[0002] Semi-autonomous and fully autonomous vehicles have significantly increased the number of sensors installed in vehicles. Camera systems, lidar systems, and radar systems with high bandwidth requirements present new challenges regarding data distribution, preprocessing, and security. Currently, various systems-on-a-chip (SoCs) exist that can handle the distribution (PCIe and Ethernet switches), processing (performance SoCs), time synchronization, and other tasks of this data.

[0003] One objective of the invention is to abstract the distribution, preprocessing and synchronization of a vehicle's sensor data from the final processing of the sensor data for controlling the vehicle.

[0004] This problem is solved by the subject matter of the independent claims. Further embodiments of the invention are described in the dependent claims and in the following description.

[0005] One aspect of the invention relates to an interface system on a chip. A system-on-a-chip is an integrated circuit (IC), particularly one implemented on a single chip. The interface system comprises a plurality of interfaces with different protocols for reading sensor data from various sensors and outputting it using a specific interface or protocol. The interface system preprocesses and / or synchronizes the data. It is designed to perform safety-compliant preprocessing and, optionally, synchronization of sensor data from a plurality of sensors.The interface system on a chip enables the preprocessing of data, the aggregation and synchronization of data, and the distribution of this data via a standard interface, such as PCIe (Peripheral Component Interconnect Express) or UCIe (Universal Chiplet Interconnect Express).

[0006] According to one embodiment, the interface system comprises a plurality of input interfaces configured for at least two different communication protocols and designed to receive sensor data from a plurality of sensors. The sensors may include a camera, radar, and / or lidar, and / or may be configured to monitor the vehicle's environment. An input interface may be provided for each sensor. The communication protocols may provide serial data and / or data packets. The communication protocols may include MIPI (Mobile Industry Processor Interface), CSI (Camera Serial Interface), GMSL (Gigabit Multimedia Serial Link), FPD-Link (Flat Panel Display Link), and / or Ethernet.

[0007] An input interface of the interface system can be connected to the respective sensor via a physical interface that implements protocol components of the physical layer. It is also possible that a deserializer, which generates data packets from a serial data stream, is connected between the sensor and the input interface.

[0008] According to one embodiment, the interface system includes a preprocessing unit for preprocessing the sensor data, for example, image signal processing (ISP), digital signal processing for radar and lidar, and / or AI accelerators for general operations, such as object detection. The data from a large number of sensors can be preprocessed in real time with high bandwidth by efficient hardware, ideally reducing energy consumption.

[0009] According to one embodiment, the interface system comprises at least one output interface configured for a specific communication protocol for outputting the data. The specific communication protocol can be PCIe or UCIe. The interface system can have multiple output interfaces with the specific communication protocol, each of which can be connected to an application system on a chip, another interface system, and / or other systems, such as a diagnostic interface for other vehicle computing systems (such as ECUs), memory, cloud systems, data logging systems, etc.

[0010] An application system can be a system on a chip designed to execute application software for the vehicle system. The interface system enables the implementation of specific application software independent of embedded software. The preprocessing provided by the interface system can improve the vehicle system's software architecture and offload the application systems, allowing them to perform better on more intensive tasks, such as AI processing.

[0011] According to one embodiment, the interface system further includes a synchronization unit for synchronizing the sensor data, whereby, for example, sensor data recorded at the same time are given the same timestamp. All input data can be synchronized by the synchronization unit, which may have its own internal clock. During synchronization, the sensor data can be timestamped. The interface system can be designed to operate in streaming mode, resulting in a more deterministic system. Timestamping and sensor synchronization are much easier to implement when all data is combined in just one component, i.e., the interface system.

[0012] In streaming mode, data is processed sequentially without the processor having to wait for the end of an instruction cycle. This enables efficient processing of data streams, such as those found in multimedia applications (e.g., video or audio streams). Streaming architectures often use parallel pipelines to process multiple data streams simultaneously. Each pipeline performs specialized operations, such as arithmetic, filtering, or transformation. Overall, streaming mode enables efficient processing of data streams by being optimized for continuous data flows.

[0013] According to one embodiment, the interface system further includes an automotive interface for data communication with the vehicle's control units. The automotive interface can operate with communication protocols typical for vehicles, such as CAN, CAN-FD, I2C, SPI, FlexRay, etc.

[0014] According to one embodiment, at least one of the input interfaces is implemented for the CSI protocol. The CSI protocol is primarily used for communication with a camera acting as a sensor.

[0015] According to one embodiment, at least one of the input interfaces is implemented for the Ethernet protocol. Communication with a radar or lidar sensor, for example, can be established via the Ethernet protocol.

[0016] According to one embodiment, the interface system comprises a plurality of output interfaces for the specific communication protocol. All output interfaces are implemented for the same communication protocol, such as PCIe or UCIe. The interface system translates the data in the form of the communication protocols of the input interfaces into data that is provided via the output interfaces using the specific communication protocol.

[0017] According to one embodiment, the preprocessing unit includes an image signal processor for processing image data. The image signal processor can convert image data in an image format specific to the camera as a sensor into an image format that can be processed by the application systems.

[0018] According to one embodiment, the preprocessing unit includes a digital signal processor, for example, for processing radar data and / or lidar data. In general, sensor data provided by a sensor in a specific format can be transferred or converted into a format that can be processed by the application systems.

[0019] According to one embodiment, the interface system further comprises a safety unit, for example with at least two or three parallel processors, whose results are compared by the safety unit. The safety unit can execute safety-relevant functions that still deliver safe results even if one processor fails or malfunctions.

[0020] According to one embodiment, the interface system further comprises a processor unit with a plurality of processors. The processor unit can execute further functions of the interface system, for example, classic functions in embedded software such as the control of cameras and sensors, the monitoring of temperature and voltage, and the control of fans.

[0021] According to one embodiment, the interface system further comprises a data communication network for distributing data between units of the interface system. The data communication network can be a so-called NoC (network on chip) or a bus system.

[0022] According to one embodiment, the interface system further includes an internal memory, which is used, for example, by the preprocessing unit and the processors.

[0023] According to one embodiment, the interface system further includes a memory interface for accessing external storage, which can be used, for example, to expand the internal storage.

[0024] Another aspect of the invention relates to a vehicle system with an interface system, as described herein. The vehicle system can be the main computer or ECU of a vehicle, enabling autonomous or semi-autonomous control of the vehicle.

[0025] According to one embodiment, the vehicle system comprises a plurality of sensors for monitoring the vehicle's environment; at least one application system on a chip designed to control the vehicle; and an interface system on a chip whose input interfaces are connected to the plurality of sensors and whose at least one output interface is connected to the at least one application system.

[0026] According to one embodiment, the vehicle system further comprises at least two interface systems on a single chip, which are interconnected via a respective output interface, wherein the interface systems are connected to different sensor sets. By using multiple interface systems on a single chip, all of which can be identically structured, the vehicle system can be implemented redundantly and in a functionally safety-compliant manner.

[0027] In the following, exemplary embodiments of the invention are described in detail with reference to the accompanying figures. Fig. Figure 1 schematically shows a vehicle system according to an embodiment of the invention. Fig. Figure 2 schematically shows a vehicle system according to a further embodiment of the invention. Fig. Figure 3 schematically shows an interface system according to a further embodiment of the invention.

[0028] The reference symbols used in the figures and their meanings are summarized in the list of reference symbols. Generally, identical or similar parts are designated with the same reference symbols.

[0029] Fig. Figure 1 schematically shows a vehicle system 10 consisting of an interface system 12 on a chip and an application system 14 on a chip. The vehicle system 10 provides part of the control system for an autonomous or semi-autonomous vehicle. Embedded software 13 is executed in the interface system 12; that is, software adapted to specific hardware of the vehicle system 10, such as a camera 16. Application software 15 is executed in the application system 14; that is, software implemented independently of the hardware of the vehicle system 10.

[0030] The two systems 12, 14 are connected via a data communication link 18 based on PCIe or UCIe. The data communication link 18 represents an abstraction layer 20 between the application software layer 15 and the embedded software layer 13.

[0031] The interface system 12 receives the data 22, such as sensor data from the hardware of the vehicle system 10, and converts it into standardized data, which is then transferred to the application system 14. Conversely, the interface system 12 converts control commands from the application system 14 into hardware-specific control commands 24 for hardware components, such as the camera 16. Thus, the interface system 12 handles the implementation of the hardware-specific requirements of the sensors and / or actuators of the vehicle system 10. The application software 15 of the application system 14 can be implemented independently of specific hardware components. Furthermore, the application system 14 can be replaced without requiring any modifications to the embedded software 13 of the interface system 12.

[0032] The Fig. Figure 2 shows another vehicle system 10, which has a plurality of interface systems 12 and application systems 14, each arranged on a chip. The vehicle system 10 can, for example, be a high-performance computer for an ECU (electronic control unit) of an autonomous or semi-autonomous vehicle.

[0033] An interface unit 26 of the vehicle system 10 has two interface systems 12 (each on a single chip) which are connected to two sensor sets 28 via various interfaces. Each sensor set 28 can include a camera 16a, a radar 16b, and / or a lidar 16c. It is understood that multiple cameras 16a, radars 16b, and / or lidars 16c may also be present.

[0034] The interface systems 12 are interconnected via a data communication link 30 based on PCIe or UCIe. The interface systems 12 are connected to the application systems 14 via further data communication links 18 based on PCIe or UCIe. A first interface system 12 can be connected to a first application system 14, and a second interface system 12 can be connected to one or more further application systems 14.

[0035] The application systems 14 can perform various functions. The two sensor sets 28 and the two interface systems 12 divide the interface unit 26 into two redundant areas, so that even if one of the sensor sets 28 and / or the interface systems 12 fails, the interface unit 26 can continue to perform its function. The data communication link 30 can be used to share sensor data from the sensor sets 28 and / or control data from the application systems 14 between the interface systems 12 to create additional redundancy.

[0036] Through the abstraction layer 20, the application software 15 becomes independent of the implementation of the interface unit 26 and / or the hardware components, such as the sensor sets 28, and generally independent of sensors, fans, coolers, AD and DA converters, controllers, etc. The application software 15 can be used purely for object recognition, pixel segmentation, L4 functions, etc.

[0037] The Fig. Figure 2 further shows that the interface unit 26, in addition to the interface systems 12, each of which is located on a chip, has further components which can also each be located on an associated chip.

[0038] To support the application system 12, a microcontroller (MCU, micro controller unit) 32 may be present. The MCU performs safety monitoring and control through fault detection. For example, it constantly monitors the state of the control unit and detects faults or deviations from the expected functionality. If a fault occurs, the MCU implements safety mechanisms to respond to the fault, e.g., through redundancy, monitoring of sensors and actuators, and safe state transitions.

[0039] Each sensor 16a, 16b, 16c can have a physical interface 34a, 34b, which receives the sensor data of the respective sensor at a physical layer and forwards it to the interface system 12. To process sensor data from a camera 16a, a deserializer 34a can be provided, which generates data packets for the interface system 12 from the serial data of the camera 16a. As a further example, each sensor 16b, 16c that communicates via Ethernet can have a physical Ethernet interface 34b.

[0040] An external RAM 36 can expand the internal RAM of the interface system 12.

[0041] Another data communication link 38 via PCIe or UCIe can be used to provide a diagnostic interface (MDI) and / or an interface to another high-performance computer or ECU 40 of the vehicle system 10.

[0042] The Fig.Figure 3 shows an embodiment of an interface system 12 in more detail. The interface system 12 comprises a plurality of input interfaces 42, which are configured for at least two different communication protocols and which are configured to receive sensor data 22 from a plurality of sensors 16a, 16b, 16c. The interface system 12 further comprises at least one output interface 44, which is configured for a specific communication protocol, for outputting the sensor data that has been preprocessed and optionally synchronized by the interface system 12.

[0043] One of the input interfaces 42a can be configured for the CSI protocol. The CSI protocol is primarily used for communication with a camera 16a as a sensor. One of the input interfaces 42b can be configured for the Ethernet protocol. The Ethernet protocol is used for communication with, for example, a radar 16b or lidar 16c as a sensor. Each input interface 42 can be connected to the respective sensor 16a, 16b, or 16c via a physical interface 34a or 34b, which implements protocol components of the physical layer. A deserializer 34a, which generates data packets from serial data from the sensor, can also be connected between the sensor 16a and the input interface.

[0044] Some of the sensor data 22, in particular image data, can be preprocessed by a preprocessing unit 46. The preprocessing unit 46 can comprise one or more image signal processors 46a for processing image data and / or one or more digital signal processors 46b for processing more general data.

[0045] The interface system 12 comprises a data communication network 48, such as a bus system, for distributing data between the input interfaces 42, the output interfaces 44, the preprocessing unit 46, and other components of the interface system 12. The optionally preprocessed sensor data 22 from the sensors 16a, 16b, 16c are forwarded via the data communication network 48 to the output interfaces 44, which then make the sensor data 22 available to other components of the vehicle system 10 using a specific communication protocol, such as PCIe or UCIe.

[0046] The sensor data 22 from camera 16a flows through a digital signal processor 46b and then via the data communication network 48 to the output interfaces 44. The sensor data 22 from radar 16b and lidar 16c flows directly into the data communication network 48, which forwards it, for example, to a digital signal processor 46b and / or a processor unit 50. The processor unit 50 can comprise a plurality of processors that preprocess the sensor data 22 using software.

[0047] Intermediate results and software can be stored in internal memory 52 or external memory 36. The internal memory 52 is directly connected to the data communication network 48. A storage interface 54 connected to the data communication network 48 is used to access the external memory 36. The internal memory 52 is used primarily to avoid slower external access to the external memory 36.

[0048] The interface system 12 further includes a synchronization unit 56 for synchronizing the sensor data 22, whereby sensor data 22 recorded at the same time are given the same timestamp. This can also occur if the frequency of the sensor data or of the sensors 16a, 16b, 16c is different.

[0049] The interface system 12 can further include an automotive interface 58 for data communication with control units of the vehicle, such as a CAN bus.

[0050] The interface system 12 can also include a safety unit 60, which enables functions to be executed in a secure manner. The safety unit 60 can have at least two parallel processors, the results of which are compared by the safety unit 60.

[0051] The pre-processed and synchronized sensor data 22 are forwarded by the data communication network 48 to the output interfaces 44.

[0052] The output interfaces 44 can provide this sensor data 22 to a plurality of components of the vehicle system 10 at the application level. Performance systems 14, such as CPUs and GPUs, can perform computationally intensive tasks, such as AI classification, autonomous driving functions, etc. Furthermore, data logging and recording devices, such as external SSDs, HDDs, and / or a cloud connection, can be connected to the output interfaces 44.

[0053] Additional high-performance computers and / or ECUs 40 can be connected to the output interfaces 44.

[0054] It should also be noted that "comprehensive" does not exclude any other elements or steps, and "a" or "an" does not exclude a plurality. Furthermore, it should be noted that features or steps described with reference to one of the above embodiments may also be used in combination with other features or steps from other embodiments described above. Reference numerals in the claims are not to be considered as limitations. Reference sign 10 Vehicle systems 12 interface systems on one chip 13 embedded software 14 Application system on a chip 15 Application software 16 Vehicle sensor, camera 16a Camera 16b Radar 16c Lidar 18 Data communication connection 20 Abstraction layer 22 data points, sensor data 24 data points, control commands 26 Interface unit 28 Sensor set 30 Data communication connection 32 microcontrollers 34a physical interface, deserializer 34b physical interface, physical Ethernet interface 36 GB external RAM 38 Data communication connection 40 high-performance computers, ECUs 42, 42a Input interface 42, 42b Input interface 44 Output interface 46 Pre-processing unit 46a DSP 46b ISP 48 Data communication network 50 processor units 52 GB internal storage 54 memory interface 56 Synchronization unit 58 Automotive interface 60 security units

Claims

[1] Interface system (12) on a chip, comprising: a plurality of input interfaces (42) configured for at least two different communication protocols and configured to receive sensor data (22) from a plurality of sensors (16a, 16b, 16c); a preprocessing unit (46) for preprocessing the sensor data (22); at least one output interface (44) designed for a specific communication protocol for outputting the sensor data (22). [2] Interface system (12) according to claim 1, further comprising: a synchronization unit (56) for synchronizing the sensor data (22), wherein sensor data (22) that were recorded at the same time are given the same timestamp. [3] Interface system (12) according to claim 1 or 2, further comprising: an automotive interface (58) for data communication with control units of the vehicle. [4] Interface system (12) according to any one of the preceding claims, where one of the input interfaces (42a) is implemented for the CSI protocol; and / or one of the input interfaces (42b) is implemented for the Ethernet protocol. [5] Interface system (12) according to one of the preceding claims, wherein the interface system (12) comprises a plurality of output interfaces (44) for the specific communication protocol. [6] Interface system (12) according to any of the preceding claims, wherein the specific communication protocol for the output interface (44) is PCI express or UCI express. [7] Interface system (12) according to any one of the preceding claims, wherein the preprocessing unit (46) comprises an image signal processor (46a) for processing image data; and / or wherein the preprocessing unit (46) comprises a digital signal processor (46b) for processing radar data and / or lidar data. [8] Interface system (12) according to any one of the preceding claims, further comprising at least one of the following: a security unit (60) with at least two parallel processors, the results of which are compared with each other by the security unit (60); a processor unit (50) with a plurality of processors; a data communication network (48) for distributing data between units of the interface system (12); an internal memory (52); a storage interface (54) for accessing an external storage device (36). [9] Vehicle system (10), comprising: a plurality of sensors (16) for monitoring the vehicle's surroundings; at least one application system (14) on a chip designed to control the vehicle; an interface system (12) on a chip according to one of the preceding claims, the input interfaces (42) of which are connected to the plurality of sensors (16a, 16b, 16c) and the at least one output interface (44) of which is connected to the at least one application system (14). [10] Vehicle system (10), further comprising: at least two interface systems (12) on a chip, which are connected to each other via a respective output interface (44); wherein the interface systems (12) are connected to different sensor sets (28).

Images