Apparatus for synchronizing sensor data, sensor system and method for testing a sensor system

Abstract

In summary, the present invention relates to the synchronization of sensor data, such as its potential use for testing subsequent processing of sensor data. Three operating modes are provided for this purpose. In one operating mode, real sensor data from an environmental sensor is received and forwarded for subsequent processing according to the timing of the sensor data delivery. In another operating mode, previously stored sensor data is forwarded according to the timing of the sensor data delivery. Furthermore, a third operating mode is provided, in which previously stored sensor data is forwarded in a timing sequence corresponding to the timing of additional data simultaneously provided by the environmental sensor.

Description

Technical Field

[0001] This invention relates to a device for synchronizing sensor data and a sensor system having such a device. The invention also relates to a method for testing a sensor system. Background Technology

[0002] Radar systems play a central role in modern applications, particularly in driver assistance systems, autonomous mobility, and industrial automation. Such systems enable the detection of precise information about the environment, including the position, velocity, and direction of motion of objects. The reliability and accuracy of the data from such systems depend decisively on the efficiency and reliability of signal assessment.

[0003] Therefore, a crucial aspect of developing such radar systems, or other systems for environmental monitoring, is the verification and validation of the evaluation logic. This requires efficient testing methods to check the correct operation of the signal evaluation under realistic conditions. In addition to testing under real-world conditions, further testing based on simulated or previously stored scenarios is desirable. In particular, the last mentioned test ensures that comparable tests can be performed during development, even after modifications to the evaluation logic.

[0004] Reference DE 10 2018 222 195 A1, for example, describes a method for locating or classifying objects using radar sensors. To this end, an evaluation system is proposed that utilizes an artificial neural network to evaluate radar signals. Summary of the Invention

[0005] This invention provides an apparatus for synchronizing sensor data, a sensor system, and a method for testing the sensor system. Further advantageous embodiments are described below.

[0006] Configure accordingly: A device for synchronizing sensor data, comprising a first input interface, a second input interface, and a processing unit, is disclosed. The first input interface is designed to receive sensor data from an environmental sensor. The sensor data can be received, in particular, in the form of data packets. Specifically, the data received through the first input interface can be provided in real time by the environmental sensor. The second input interface is designed to receive previously stored sensor data. This previously stored sensor data can be received through the second input interface in the form of data packets. The processing unit is designed to process the sensor data from the environmental sensor in at least three operating modes. Specifically, the processing unit is designed to read and process data packets from the environmental sensor at the first input interface in the first operating mode. Here, the data packets are handled and processed in the first operating mode at a timing corresponding to the timing of the data packets provided by the environmental sensor at the first input interface. Furthermore, the processing unit is designed to read and process data packets from the second input interface in the second operating mode. Here, the data packets are handled and processed in the second operating mode at a timing corresponding to the timing of the data packets provided at the second input interface. Furthermore, the processing device is designed to read and process data packets from the second input interface in a third operating mode, wherein the data packets are taken over and processed in the third operating mode at a timing corresponding to the timing of additional data packets provided by the environmental sensor at the first input interface.

[0007] Additional settings: A sensor system having at least one environmental sensor and a device for synchronizing sensor data according to the invention. The environmental sensor is designed to provide sensor data in the form of data packets at a first input interface of the device for synchronizing sensor data.

[0008] Final settings: A method for testing a sensor system, particularly a sensor system according to the present invention. The method includes at least three operating modes. In a first operating mode, data packets from an environmental sensor are received and forwarded. Here, the data packets are received and forwarded in a timing sequence corresponding to the timing of the data packets provided by the environmental sensor in the first operating mode. In a second operating mode, previously stored data packets are received and forwarded. Here, the data packets are received and forwarded in a timing sequence corresponding to the timing of the previously stored data packets in the second operating mode. In a third operating mode, data packets from previously stored data packets are received and forwarded. Here, in the third operating mode, the data packets are received and forwarded in a timing sequence corresponding to the timing of additional data packets provided by the environmental sensor.

[0009] Advantages of the invention This invention is based on the understanding that, in order to test environmental sensors (e.g., radar sensors), it can be advantageous to use previously stored data in addition to testing with real sensor data. It is precisely the stored data that provides the possibility of simulating specifically defined scenarios under controlled conditions and examining the particular procedural manner of the evaluation logic. This is especially helpful for tests that require repeatability or aim to cover scenarios that are difficult to reproduce in real-world environments. Therefore, it can also be used, for example, to verify rare traffic incidents or complex multi-target detection.

[0010] However, a key difference between real sensor data and stored data may lie in the timing of the data delivery. While real environmental sensors typically provide data according to a hardware-defined timeline, this timing can deviate in the case of stored data. However, for certain tests, it may be desirable to process stored data with a timing sequence corresponding to that of real sensors. This allows for a more realistic check of the evaluation logic, taking into account real, operating system timing conditions.

[0011] Based on this understanding, one concept of the present invention is to provide stored data in addition to real data for testing evaluation logic, especially evaluation logic for processing sensor data for environmental sensors, and to synchronize the stored data to the timing of the real sensors if necessary.

[0012] In particular, for this purpose, data can be provided to the evaluation logic in the form of data packets. Here, the timing of forwarding the stored data to the downstream evaluation logic and processing it can be synchronized to data packets as if they were provided by real sensors.

[0013] According to one embodiment, the environmental sensor includes a radar sensor, an ultrasonic sensor, and / or a lidar sensor. Here, the environmental sensor can be not only a pure sensor unit that detects only physical signals, but also a complete sensor system. In addition to sensor detection via microwave, ultrasonic, or laser signals, such a system may also include signal preprocessing and (if necessary) analog-to-digital conversion (A / D conversion). Through such integrated processing steps, the detected raw signal can be converted into a form that can be directly used for subsequent evaluation logic.

[0014] According to one implementation, sensor data from an environmental sensor and / or previously stored sensor data is provided in the Advanced Microcontroller Bus Architecture (AMBA) protocol and received in this form by a synchronization device. The AMBA protocol is a widely used standard for efficiently designing communication between different components of a System-on-Chip (SoC). In addition to standardized data communication, the AMBA protocol offers scalability and modularity, making it well-suited for use in a variety of applications.

[0015] According to one embodiment, the device for processing sensor data includes a processing unit. This processing unit may be designed to receive and process data provided by a synchronization device. The result of the processing can then be output by the processing unit. Here, the output of the result may also be performed in the AMBA protocol, where appropriate.

[0016] According to one embodiment, the device for synchronizing sensor data is implemented as a system-on-a-chip (SoC). In particular, the device can also be implemented as a monolithic microwave integrated circuit (MMIC) in combination with other components of the environmental sensor (e.g., a radar sensor), where appropriate. This allows for a particularly compact construction for processing and evaluating sensor data from the environmental sensor. Here, with the solution according to the invention, reliable verification of the evaluation can be achieved even in such a highly integrated circuit.

[0017] According to one embodiment, the sensor system includes a storage device. This storage device can be designed to store and provide sensor data for testing processing devices. In particular, the sensor data can be stored and / or provided in the form of data packets.

[0018] According to one embodiment, the stored sensor data may include analog sensor data. Additionally or alternatively, the stored sensor data may also include previously detected sensor data from environmental sensors. Therefore, this stored sensor data provides a data foundation for repeatable inspection and verification of the sensor system.

[0019] The above configurations and extensions can be combined arbitrarily with each other, as long as they are meaningful. Other configurations, extensions, and implementations of the invention also include combinations of features of the invention not explicitly mentioned in the preceding or subsequent descriptions of the embodiments. In particular, those skilled in the art will also add individual aspects as improvements or additions to the corresponding basic forms of the invention. Attached Figure Description

[0020] Further features and advantages of the invention will now be explained with reference to the accompanying drawings. As shown here: Figure 1A schematic diagram of a sensor system in a device for synchronizing sensor data according to one embodiment; Figure 2 A first operating mode of a device for synchronizing sensor data according to one embodiment may be based on a timing diagram; Figure 3 A second operating mode of a device for synchronizing sensor data according to one embodiment may be based on a timing diagram; Figure 4 A third operating mode of a device for synchronizing sensor data, according to one embodiment, can be based on a timing diagram; and Figure 5 The flowchart illustrates a method for testing a sensor system according to one embodiment. Detailed Implementation

[0021] Figure 1 A schematic diagram of a sensor system having a device 1 for processing sensor data according to one embodiment is shown. The device 1 for processing sensor data includes a first input interface 11 and a second input interface 12. Furthermore, the device 1 for synchronizing sensor data includes a synchronization device 13 and a processing device 14.

[0022] Sensor data from the environmental sensor 2 can be provided at the first input interface 11. The environmental sensor 2 can be, for example, a radar sensor, an ultrasonic sensor, or a lidar sensor. In particular, the environmental sensor 2 can be not only a pure sensor unit that detects only physical signals, but also a complete sensor system that includes signal preprocessing and, where appropriate, A / D conversion in addition to sensor detection of the environment. Through this integrated processing, the detected raw signal can be converted into a form that can be directly used for subsequent evaluation. For example, the signal can be transmitted from the environmental sensor 2 to the first input interface 11 based on the Advanced Microcontroller Bus Architecture (AMBA) protocol. Optionally, the environmental sensor, or at least a portion thereof, and the device 1 for processing the sensor data can be implemented as a system-on-a-chip (SoC) and, where appropriate, implemented on a common chip.

[0023] At the second input interface 12, sensor data from an additional data source 3 can be provided, for example. This additional data source 3 could provide previously stored sensor data. The previously stored sensor data could be, for example, real sensor data previously recorded using the environmental sensor 2 or a comparable sensor system. Alternatively or additionally, analog sensor data can also be provided via the additional data source 3. The sensor data from the additional data source 3 can also be provided, for example, in the same form as the sensor data from the environmental sensor 2. In particular, the sensor data can also be provided within the AMBA protocol.

[0024] Data from environmental sensor 2 and data from another data source 3 can be provided, for example, in chunks or in data packets. In principle, any type of data communication, especially in the form of packet-based data transmission, is possible. The timing of the data packets will be explained in more detail below.

[0025] The synchronization device 13 of the device 1 for synchronizing sensor data can receive sensor data from a first input interface or a second input interface and forward it to a downstream processing device 14 according to the principles explained in more detail below. For the processing of the sensor data performed by the downstream processing device 14, any type of data processing based on sensor data provided by the environmental sensor 2 or another data source 3 is possible. This processing may include, for example, object detection, or determining the position, distance, speed, or similar information of an object. Furthermore, object classification, clustering of individual targets, or similar processes are also possible. In addition, the processing device 14 may, where appropriate, derive further action instructions for subsequent systems based on the sensor data. For example, driver assistance systems such as brake assist, lane keeping assist, or similar systems can be controlled based on the sensor data. Depending on the application, any other processing steps may also be performed by the processing device 14.

[0026] The results from the processing device 14 can then be output.

[0027] The following describes three possible operating modes of device 1 for synchronizing sensor data.

[0028] In the first operating mode, the device 1 for synchronizing sensor data performs data processing based solely on sensor data from the environmental sensor 2. For this purpose, sensor data from the environmental sensor 2 is provided at the first input interface 11.

[0029] Figure 2A timing diagram for forwarding sensor data in a first operating mode is shown. Clock signal 100 is shown in the row clk above. Based on clock signal 100, data can be transmitted in digital form, for example, via a bus system or similar system. Clock signal 100 should be understood as illustrative only. The number of clock pulses per data packet does not constitute a limitation of the invention.

[0030] The data signal 210 from the sensor data of the environmental sensor 2 is shown in the following row (Data 1). As can be seen here, the environmental sensor 2 provides sensor data in a series of data packets ADn.

[0031] The other data source 3 does not provide relevant data (Data 2) in this first operating mode. Accordingly, data is forwarded from the first input interface 11 to the synchronization device 13. Here, the timing of the forwarded data 300 corresponds to the timing of the data packet 210 from the environmental sensor 2.

[0032] Figure 3 A schematic diagram of the timing of a second operating mode of the device 1 for synchronizing sensor data is shown. In this second operating mode, data pre-stored by an additional data source 3 is provided at the second input interface 12. The corresponding data signal (Data 2) is... Figure 3 The signal is shown as 220. Here, data 220 from another data source 3 is also provided in the form of data packet BDn. In this second operating mode, the environmental sensor 2 either does not provide sensor data, or the sensor data 210 provided by the environmental sensor 2 is ignored. Therefore, the data packet 220 provided by the other data source 3 is provided to the processing device 14 as data packet 300 (Data 3) according to the timing from the other data source 3.

[0033] In addition, a third operating mode is provided in device 1 for synchronizing sensor data, such as... Figure 4 As shown in the diagram, data 220 in the form of data packets BDn is provided by another data source 3. Simultaneously, data packets 210 are also provided by the environmental sensor 2. In this third operating mode, data packets from the other data source 3 provided at the second input interface 12 are forwarded to the processing device 14 according to the timing corresponding to the timing of data packets 210 from the environmental sensor 2 at the first input interface 11. In other words, if a data packet from the environmental sensor 2 is received at the first input interface 11, data packets from the other data source 3 are forwarded to the processing device 14. In this way, data packets from the other data source 3 can be forwarded to the processing device 14 according to the packet rate from the environmental sensor 2.

[0034] Figure 5 A flowchart illustrating a method for testing a sensor system according to one embodiment is shown. This method may, in principle, include any steps, as previously described in conjunction with device 1 for synchronizing sensor data in a sensor system. Similarly, device 1 for synchronizing sensor data, or a sensor system having such device 1, may also include any components suitable for implementing the method described below.

[0035] In the first operating mode S1, data packets from the environmental sensor 2 are received and forwarded, wherein the data packets are taken over and forwarded in the first operating mode in a timing sequence corresponding to the timing sequence of the data packets provided by the environmental sensor 2.

[0036] In the second operating mode S2, previously stored data packets, for example, from data source 3, are received and forwarded, wherein the data packets are received and forwarded in the second operating mode in a timing sequence corresponding to the timing sequence of the previously stored data packets provided by another data source 3.

[0037] Finally, in the third operating mode S3, previously stored data packets are received and forwarded, wherein the data packets are received and forwarded in this third operating mode in a timing sequence corresponding to the timing sequence of other data packets simultaneously provided by the environmental sensors.

[0038] The method may also include a step in which data from environmental sensors is pre-stored. The data from the environmental sensors may come from environmental sensors that also provide data in the first operating mode. Alternatively, the data may be provided by other environmental sensors of the same or similar nature. Furthermore, data generated based on simulations for such environmental sensors may also be recorded and stored.

[0039] In summary, the present invention relates to the synchronization of sensor data, such as its potential use for testing subsequent processing of sensor data. Three operating modes are provided for this purpose. In one operating mode, real sensor data from an environmental sensor is received and forwarded for subsequent processing according to the timing of the sensor data delivery. In another operating mode, previously stored sensor data is forwarded according to the timing of the sensor data delivery. Furthermore, a third operating mode is provided, in which previously stored sensor data is forwarded in a timing sequence corresponding to the timing of additional data simultaneously provided by the environmental sensor.

Claims

1. A device (1) for synchronizing sensor data, comprising: The first input interface (11) is designed to receive sensor data in the form of data packets from the environmental sensor (2), wherein, The data packets received through the first input interface (11) are provided in real time by the environmental sensor (2); The second input interface (12) is designed to receive previously stored sensor data in the form of data packets; as well as Synchronization device (13), the synchronization device being designed for: In the first operating mode, data packets from the environmental sensor (2) are read at the first input interface (11) and forwarded to the downstream processing device (14), wherein the data packets are forwarded in the first operating mode in a timing sequence corresponding to the timing of the data packets provided by the environmental sensor (2) at the first input interface (11). In the second operating mode, data packets from the second input interface (12) are read in and forwarded to the downstream processing device (14), wherein the data packets are forwarded in the second operating mode at a timing corresponding to the timing at which the data packets are provided at the second input interface (12), and In the third operating mode, data packets from the second input interface (12) are read in and forwarded to the downstream processing device (14), wherein the data packets are forwarded in the third operating mode in a timing corresponding to the timing of additional data packets provided by the environmental sensor (2) at the first input interface (11).

2. The device (1) according to claim 1, wherein, The environmental sensor (2) includes a radar sensor, an ultrasonic sensor and / or a lidar sensor.

3. The device (1) according to claim 1 or 2, wherein, Sensor data from the environmental sensor (2) and / or the previously stored sensor data are received in the Advanced Microcontroller Bus Architecture (AMBA) protocol.

4. The device (1) according to any one of claims 1 to 3, having a processing device (14) designed to process data packets provided by the synchronization device (13) and output the result of the processing.

5. The device (1) according to any one of claims 1 to 4, wherein, The device (1) used for synchronizing sensor data is implemented as a system-on-a-chip (SoC), and more particularly as a monolithic microwave integrated circuit (MMIC).

6. A sensor system having: At least one environmental sensor (2), and The device (1) for synchronizing sensor data according to any one of claims 1 to 5. in, The environmental sensor (2) is designed to provide sensor data in the form of data packets at the first input interface.

7. The sensor system according to claim 6, having a storage device (3) designed to store sensor data in the form of data packets and to provide the stored sensor data at the second input interface (12).

8. The sensor system according to claim 6 or 7, wherein, The stored sensor data includes simulated sensor data and / or previously detected sensor data from environmental sensors.

9. A method for testing a sensor system (1), wherein, The method In the first operating mode (S1), data packets from the environmental sensor (2) are received and forwarded for further processing, wherein the data packets are taken over and processed in a timing sequence corresponding to the timing of the data packets provided by the environmental sensor (2) in the first operating mode; In the second operating mode (S2), previously stored data packets are received and forwarded for further processing, wherein the data packets are received and processed in a timing sequence corresponding to the timing of providing the previously stored data packets; and In the third operating mode (S3), previously stored data packets are received and forwarded for further processing, wherein the data packets are received and processed in a timing sequence corresponding to the timing of additional data packets provided by the environmental sensor (2).

10. The method according to claim 9, comprising the steps of recording and storing data from at least one environmental sensor (2).

Images