METHOD AND DEVICE FOR PROCESSING SENSOR DATA

Description

[0001] The invention relates to a method for processing sensor data. The invention further relates to a device for processing sensor data. The invention further relates to a control device. The invention further relates to a computer program. The invention further relates to a machine-readable storage medium. State of the art

[0002] The published patent application WO 2018 / 126215 A1 discloses a method for creating a digital road map.

[0003] The document "MASSOW K ET AL: "Deriving HD maps for highly automated driving from vehicular probe data", 2016 IEEE 19TH INTERNATIONAL CONFERENCE ON INTELLIGENT TRANSPORTATION SYSTEMS (ITSC), IEEE, November 1, 2016 (2016-11-01), pages 1745-1752, XP033028575, DOI: 10.1109 / ITSC.2016.7795794" discloses a concept for creating road maps using vehicle data.

[0004] Technologies that enable automated driving typically have very high expectations and requirements. One of these requirements is a detailed, accurate, up-to-date, and complete digital map of the environment, sufficiently accurate to ensure robust, accurate, and safe localization, planning, and navigation of AD (automatic driving) or ADAS (advanced driver assistance system) vehicles in such an environment.

[0005] Traditionally, data from such "environmental maps" is collected by dedicated mapping vehicles equipped with high-precision and expensive environmental sensors. These vehicles are capable of capturing virtually all environmental features. However, to provide the aforementioned requirements for a complete, accurate, and up-to-date environmental map of the entire world, such an approach is very costly and practically impractical.

[0006] It is known that today vehicles are equipped with numerous low-cost sensors (e.g. GPS, radar, camera, etc.) for use in driver assistance systems.

[0007] In practice, however, the bandwidth of each vehicle to the map creation device, which is usually located in the cloud, is usually very limited (e.g., to approximately 10 KB / km), which means that only a small amount of the environment data can be transmitted, but definitely not the raw sensor data.

[0008] Data from environmental sensors is already being used today to perform specific driving tasks in a vehicle. For example, environmental sensor data is used to control autonomous emergency braking (AEB) or adaptive cruise control (ACC). Typically, the environmental sensor data used to implement functions like AEB or ACC is obtained from radar sensors and / or cameras. What all of these functions have in common is that their effective range is very limited.

[0009] DE 10 2016 205 436 A1 discloses a method and a system for creating a digital map. It teaches that only parts of the vehicle's surroundings are taken into account to create the digital map, whereby during communication between the vehicle and the cloud, the vehicle sensors capture only a portion of the entire vehicle's surroundings, not the entire vehicle's surroundings.

[0010] S. Shobhit and N. Michael, "Approximate continuous belief distributions for precise autonomous inspection," in 2016 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), Lausanne, 2016, discloses a hierarchical environment model in which each hierarchy level represents the world at a certain level of abstraction. I. Ulrich and I. Nourbakhsh, "Appearance-based place recognition for topological localization," in Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings Cat.No.00CH37065), San Francisco, 2000, discloses a method for topological accuracy, or low-accuracy localization. Disclosure of the invention

[0011] An object of the invention is to provide an improved method for processing sensor data.

[0012] The object is achieved according to a first aspect with a method as defined in claim 1.

[0013] Advantageously, this allows environmental data to be transmitted even with a reduced bandwidth of transmission channels between the transmitting device and the control device, or between the control device and the processing device. This advantageously reduces the raw environmental data by a factor of approximately 100. Advantageously, this allows a mapping fleet used exclusively for the purpose of creating a digital map to be significantly reduced, because the environmental data can be advantageously recorded and transmitted using standard production vehicles.

[0014] According to a second aspect, the object is achieved with a device as defined in claim 7.

[0015] Advantageous further developments of the method are the subject of dependent claims.

[0016] An advantageous refinement of the method involves selecting the surrounding data using the data limitation criterion. This implements an initial data limitation strategy.

[0017] A further advantageous refinement of the method provides for the transmission of randomly selected environmental data. This implements a further data limitation strategy.

[0018] A further advantageous development of the method provides for prioritizing the sensor device's acquisition activity in defined directions when determining the environmental data. This implements a further data limitation strategy.

[0019] Further advantageous refinements of the method are characterized by the fact that, when providing the environmental data, a spatially and / or temporally defined dense acquisition of measurement points is carried out. This method produces so-called point cloud measurements, which implement a further data limitation strategy. This allows sections of the environment to be captured with very high resolution or with a rather coarse or low resolution, resulting in a large amount of environmental data.

[0020] A further advantageous development of the method provides for the use of the environmental data to create a digital map. This provides a beneficial application for the proposed method.

[0021] The invention is described in detail below with further features and advantages using several figures. The technical features and advantages of the described device result analogously from the technical features and advantages of the described method.

[0022] In the figures shows: Fig. 1 shows a schematic diagram of a proposed device for processing sensor data; and Fig. 2 shows a schematic diagram of a flowchart of a proposed method for processing sensor data. Description of embodiments

[0023] A key concept of the invention is, in particular, to transmit sensor data within a vehicle with limited bandwidth. As an example, the proposed application is to provide data for creating an optimized digital map.

[0024] Advantageously, the environmental data provided by the invention can be used to create highly accurate and robust digital maps that are as complete and highly available as possible. The general principle is very simple and enables many implementations and use cases, which are described below only as examples.

[0025] Each use case requires one or more data transmission strategies specifically suited to it. Given the specific problem, the information available for decision-making is the degree of freedom regarding the available and transmitted environmental data, as well as the weightings assigned to each optimization objective regarding environmental data (accuracy, completeness, etc.). Different strategies can be derived by solving optimization problems resulting from each individual use case.

[0026] Depending on the connectivity status of each vehicle, several scenarios can be considered based on Fig. 1 be explained in more detail. Fig. 1 shows an embodiment of a proposed device 100 for processing sensor data. It shows a sensor device 10 (e.g., radar sensor, lidar sensor, camera, etc.) of a vehicle (not shown), which is functionally connected via a first line 11 (e.g., CAN bus) to a central control device 30 (e.g., electronic control unit, ECU), which in turn is connected via a second line 31 to a processing device 200, which is arranged, for example, in the cloud. The second line 31 is not to be understood in the physical sense here, but rather as a data connection channel (e.g., a radio connection). Each of the two lines 11, 31 has a bandwidth limitation that can only transmit a limited portion of x percent of sensor data to the control device 30 and only y percent of the sensor data to the processing device 200, where x and y are generally much smaller than 100 percent.

[0027] In this context, sensor data refers to either raw sensor data or (as is usually the case) processed measurement data (e.g., through filtering, feature extraction, modeling, etc.). Since the control device 30 has additional access to other information sources, e.g., via a third line 21 to a positioning device 20, stored maps, etc., and since, as already mentioned, an optimal transmission strategy is positively influenced by the scope of available information, two fundamentally different use cases can be distinguished for this situation: Use case 1 (according to the invention): The sensor device 10 is already delivered with correspondingly preconfigured software and cannot be reconfigured. Use case 2 (not covered by the invention): The sensor device 10 can be controlled or configured by the control device 30, which can influence which sensor data the sensor device 10 transmits to the control device 30.

[0028] In the first use case, which is the norm for conventional vehicles, the optimal strategy will consist of a fixed data transmission strategy for sensor data, whereas in the second use case, a data transmission strategy for sensor data is adapted to a specific existing situation. With both use cases, an optimized digital map can be realized by exploiting a system-specific bandwidth limitation.

[0029] In this way, with the help of a data limitation criterion, for example, a fixed or randomized transmission of the sensor data can be carried out via the lines 11, 31.

[0030] It can also be provided that the control device 30 can communicate with the sensor device 10 via a feedback line 33, wherein the control device 30 can inform the sensor device 10 via the feedback line 33 which sensor data it would like to receive. For example, in this way, sensor data in the form of environmental data SD, which was recorded from a defined side of the vehicle, can be transmitted within a defined time period.

[0031] Furthermore, in accordance with a request / response principle, the processing device 200 can decide whether complete environmental data SD should be transmitted to the processing device 200. In this case, environmental data SD of a defined area (e.g., a geo-rectangle area) can be requested, which can be specified, for example, in the header of the thus spatially assigned environmental data SD'. For this purpose, a feedback line 32 can be provided from the processing device 200 to the control device 30.

[0032] It can be seen that, for the purpose of implementing the data limitation criterion, the sensor device 10 has a first limitation device 12, by means of which it is decided which environmental data SD are presumably the most relevant, which are then transmitted to the control device 30. In this way, a prioritization of important environmental data SD already takes place on the sensor device 10.

[0033] Optionally or additionally, a second limiting device 34 can also be formed on the control device 30, which implements a data limiting criterion on the control device 30.

[0034] For example, using the positioning device 20 (application case 2), the vehicle can determine whether the road it is traveling on is the one it previously traveled. Assume that during the first trip, it transmitted a portion of the environment data SD that is specific to a part of the environment. With this information provided, optimal behavior (e.g., for optimal coverage of the digital map) may consist of transmitting a different portion of the environment data SD, which was not transmitted during the first trip, during the second trip. In application case 1, such adaptation toward better coverage cannot be realized.

[0035] Improved strategies regarding limited data transmission can result from the vehicle's ability to communicate with other vehicles in the vicinity. These strategies can, for example, adapt data transmission behavior to minimize the disadvantage of mutual coverage, improve accuracy in certain sections of the map, improve map coverage, etc., all of which are performed under the assumption of limited bandwidth.

[0036] What applies to the control or reconfiguration options between the control device 20 and the sensor device 10 can also be applied to the connection between the processing device 200 and the control device 30. It makes a difference, for example, whether the processing device 200 has the ability to request environmental data SD in the form of a specific type of information from the sensor device 20 via the feedback line 32 from the vehicle or to focus on certain sections of the environment, etc., based on the current status of the digital map stored on the processing device 200.

[0037] Another aspect concerns the dimensionality of the space from which the environmental data SD originates. For example, a fleet of vehicles can be equipped with radar sensors, with each radar sensor performing so-called point cloud measurements of the environment while the vehicle is moving. Each point of the point cloud measurement contains information that spans a space in various dimensions (e.g., x-direction, y-direction, z-direction, RCS, etc.), and contains variance / covariance values ​​for some of these dimensions. Additionally, at least one of the following parameters: recording time, vehicle type, sensor mounting position, sensor software version, etc., can also establish other dimensions of the data space.

[0038] The strategy that optimizes the accuracy, coverage, and / or novelty of the digital map should take this spatial dimensionality into account. This can be advantageously reflected in reduced costs for the transmitted environmental data SD.

[0039] For example, a cost function related to spatial accuracy may be given greater weight than a cost function related to RCS accuracy, etc., resulting in different data transmission strategies. For example, the optimal data transmission strategy for each part of the vehicle fleet may focus on certain spatial directions.

[0040] Other weighting functions can be defined for each spatial dimension with a desired degree of resolution per dimension and modified accordingly. For example, a high weighting can be placed on the spatial accuracy of the environment data SD until the digital map reaches a defined level of accuracy (e.g., on the order of approximately 10 cm). After that, the weighting of spatial accuracy can be lowered and the strategy for limiting sensor data can be directed toward other dimensions.

[0041] Another example is the recording or acquisition time of the environmental data SD. Different resolutions of sensor data recorded during the day or at night could be the focus of interest, although the exact time is of secondary importance. This can also significantly reduce the amount of transmitted environmental data SD.

[0042] As a result, the proposed method can advantageously define a data transmission strategy that allows the development of a model known in the state of the art, starting from high to low levels. This way, depending on the number of vehicles traveling in a defined region, the accuracy and timeliness of the map data, and thus also the localization, can be improved.

[0043] Although the method was explained above in connection with the creation of a digital map, the method is not limited thereto. Advantageously, the invention can be used not only to create an optimized digital map, but also to achieve an optimal goal. This can be, for example, optimized localization, for example in the form of accuracy, availability, robustness, security, etc., or a general optimized functionality (e.g., localization, planning, perception, or other functions that the vehicle performs using the map to be created).

[0044] Fig. 2 shows a basic flow of a proposed method for processing sensor data, which concerns use cases 1 and 2.

[0045] In a step 300, environmental data SD of a vehicle is provided by means of a sensor device 10.

[0046] In a step 310, the environment data SD are assigned to corresponding location data OD.

[0047] In a step 320, the locally assigned environmental data SD' is transmitted to a processing device 200, wherein the transmission of the environmental data SD between the sensor device 10 and a control device 30 and / or the transmission of the locally assigned environmental data SD' between the control device 30 and the processing device 200 is carried out according to a defined data limitation criterion.

[0048] The method according to the invention can advantageously be implemented as software, which runs, for example, on the electronic control device 30 in the form of a control unit in the vehicle or cloud-based on a server device. This supports easy adaptability of the method.

Claims

1. Method for processing sensor data, with the steps of: - providing surroundings data (SD) relating to a vehicle; - assigning the surroundings data (SD) to corresponding locating data (OD); and - transmitting the locally assigned surroundings data (SD') to a processing apparatus (200); characterized in that - the surroundings data (SD) are transferred between a sensor device (10) and a control device (30) according to a defined data limitation criterion which is defined by the sensor device (10) that has correspondingly preconfigured software and is not reconfigurable.

2. Method according to Claim 1, wherein the surroundings data (SD) are selected with the data limitation criterion.

3. Method according to Claim 2, wherein randomly selected surroundings data (SD") are transmitted.

4. Method according to one of the preceding claims, wherein the detection activity of the sensor device (10) in defined directions is prioritized when determining the surroundings data (SD).

5. Method according to one of the preceding claims, wherein locally and / or temporally defined dense detection of measuring points is carried out when providing the surroundings data (SD).

6. Method according to one of the preceding claims, wherein the surroundings data (SD) are used to create a digital map.

7. Apparatus (100) for transferring sensor data, comprising: - a sensor device (10) for providing surroundings data (SD) relating to a vehicle; - a locating device (20); - a control device (30) for assigning the surroundings data (SD) to corresponding locating data (OD); - a first transmission device (11) for transmitting the surroundings data (SD) from the sensor device to the control device (30) and a second transmission device (31) for transmitting the locally assigned surroundings data (SD') to a processing apparatus (200) arranged outside the apparatus (100); characterized in that - the surroundings data (SD) are transferred between the sensor device (10) and the control device (30) according to a defined data limitation criterion which is defined by the sensor device (10) that has correspondingly preconfigured software and is not reconfigurable.