Method for processing measurement data of an environmental sensor of a vehicle and compensating for delayed vehicle data, computing device and computer program

Abstract

The present invention relates to a method for processing measurement data from environmental sensors of a vehicle, comprising the steps of: receiving measurement data from environmental sensors, wherein the measurement data describes the environment of the vehicle; determining vehicle data describing the position and / or movement of the vehicle; processing the measurement data in consideration of the vehicle data; receiving driving planning data describing the future movement of the vehicle during at least partially automated operation of the vehicle; predicting vehicle data at at least one point in time during the future movement of the vehicle based on the driving planning data; and processing the measurement data received for said at least one point in time in consideration of the predicted vehicle data.

Description

Technical Field

[0001] This invention relates to a method for processing measurement data from environmental sensors in a vehicle. Furthermore, this invention relates to a computing device for a vehicle. The invention also relates to a computer program and a computer-readable (storage) medium. Background Technology

[0002] Modern vehicles include various driver assistance systems that support the driver or user while driving. Multiple environmental sensors are associated with these systems, enabling the detection of the vehicle's environment. These sensors typically provide measurement data describing the environment or objects within it. Furthermore, vehicle data describing the vehicle's position and / or movement is used by the driver assistance systems. Vehicle data may describe, for example, speed, yaw rate, etc. The environmental sensors and the sensors used to provide vehicle data may be associated with various control devices within the vehicle.

[0003] The increasingly complex automotive electrical network architecture introduces latency within the vehicle, hindering the timely delivery of sensor signals. Furthermore, additional latency within control equipment increases time skew. Another impact is on signals using time filtering. If vehicle data arrives delayed and / or is time-filtered, this can cause instantaneous or currently detected measurements from environmental sensors to become irrelevant to the vehicle data or automotive electrical network data. In such cases, only vehicle data containing very small changes over a sufficiently long period can be used. Consequently, environmental detection can be delayed or rendered inaccurate.

[0004] The calibration process for environmental sensors, which involves calibrating the vehicle while it is in motion, also requires vehicle data. For accurate calibration of environmental sensors, vehicle data must be received without delay, or at least with only very small changes over a sufficiently long period. If the vehicle data is delayed, the calibration process will slow down because the premise of minute data changes is usually not present. If non-static, i.e., dynamic and delayed vehicle data is used, the quality of environmental sensor calibration will be affected, and sometimes it may not be able to complete. Summary of the Invention

[0005] The purpose of this invention is to provide a solution for how to more reliably process measurement data from environmental sensors of the type of vehicle mentioned at the beginning.

[0006] According to the invention, the objective is achieved by a method, computing device, computer program, and computer-readable (storage) medium having the features according to the independent claims. Advantageous improvements of the invention are described in the dependent claims.

[0007] The method according to the invention is used to process measurement data from environmental sensors of a vehicle. The method includes receiving measurement data from the environmental sensors, wherein the measurement data describes the vehicle's environment. The method also includes determining vehicle data describing the vehicle's position and / or movement. The method further includes processing the measurement data while taking the vehicle data into account. Furthermore, the method includes receiving driving planning data describing the vehicle's future movement during at least partially automated operation. Additionally, the method includes predicting vehicle data at at least one point in time during the vehicle's future movement based on the driving planning data. The method also includes processing the measurement data received for said at least one point in time while taking the predicted vehicle data into account.

[0008] The method should further process the measurement data or sensor data from the environmental sensors. Here, the term "processing" can be understood in particular as: evaluating and / or adapting the measurement data while taking into account vehicle data. It can also be understood as: performing further calculations and / or determining a model based on the measurement data and vehicle data. This method can be performed using the vehicle's computing device. The computing device can be, for example, a control device associated with the environmental sensors. The environmental sensors can be radar sensors, lidar sensors, cameras, ultrasonic sensors, etc. The measurement data can, for example, describe sensor signals transmitted by the environmental sensors and reflected in the environment. The measurement data can also be image data describing the environment. The measurement data can be transmitted from the environmental sensors to the computing device. In principle, it can also be proposed that the computing device receive measurement data from multiple environmental sensors. In particular, objects and / or other traffic participants in the vehicle environment can be identified based on the measurement data. Here, the term "environment" should be understood as the area outside the vehicle, which can be detected by the environmental sensors.

[0009] Furthermore, vehicle data describing the vehicle's current position and / or movement is received from the computing device. The vehicle data can describe the vehicle's own movement. Here, the vehicle's movement can describe its speed and yaw rate. Additionally, the vehicle's movement can also describe its pitch and / or roll. Furthermore, the installation positions of the environmental sensors at or within the vehicle are known. The environment can then be detected based on the known installation positions of the environmental sensors, measurement data, and vehicle data. This specifically means determining the distance between the vehicle and objects in the environment. Furthermore, the relative speed and / or angle between the vehicle and objects can be determined. Moreover, an environmental model of the vehicle's environment can be derived based on the measurement data and vehicle data.

[0010] Furthermore, driving planning data can be received using a computing device. The driving planning data can be determined using a driving planning device. The driving planning device can plan or pre-plan the future movement of the vehicle. For example, the driving planning device can plan future driving maneuvers or future driving strategies for at least partially automated driving operation of the vehicle. The driving planning device can pre-plan or control the longitudinal and / or lateral guidance of the vehicle. Based on the driving planning data, movement data can be predicted for at least one point in time for the vehicle's future at least partially automated driving. During at least partially automated driving, the driver assistance system or the driving planning device can take over the longitudinal and lateral guidance of the vehicle. Therefore, during future driving, the vehicle should operate at least according to Level 2 of the SAE J3016 standard. Therefore, for example, speed and yaw rate can be estimated for at least one point in time. Preferably, vehicle data can also be estimated for multiple points in time based on the driving planning data.

[0011] This invention is based on the understanding that modern vehicles are not solely driven by drivers, but rather have the option of being controlled by engineering systems or driving planning devices. Based on driving planning data, vehicle data can be predicted with high confidence. Now, measurement data describing the environment at at least one point in time can be processed based on the predicted vehicle data, which is also used to predict the vehicle data for that point in time. Using the predicted vehicle data, for example, the vehicle's environment can be detected more reliably than based on the known latency-affected and / or time-filtered vehicular network signals themselves. Overall, measurement data can be processed more reliably.

[0012] Furthermore, onboard electrical network data is preferably received from at least one motion sensor of the vehicle, wherein the onboard electrical network data also describes the vehicle's position and / or movement. Additionally, measurement data is preferably processed based on the onboard electrical network data. Similarly, as vehicle data predicted based on driving planning data, the onboard electrical network data describes the vehicle's position and / or movement. As described initially, the onboard electrical network data may be transmitted to environmental sensors or their control devices with a time delay due to the onboard electrical network architecture and / or filtering. Here, the onboard electrical network data can be used to check the reasonableness of the predicted vehicle data. Here, the onboard electrical network data and / or the predicted vehicle data are weighted. Furthermore, based on the predicted vehicle data and the onboard electrical network data, the time delay or latency of the onboard electrical network data can be determined and considered in the future if necessary.

[0013] In another embodiment, driving planning data is received from the vehicle's driving planning device. In other words, the driving planning data can be determined by the driving planning device within the vehicle and transmitted to a computing device. The driving planning device can be a driver assistance system of the vehicle or part of a driver assistance system. The driving planning device can be used, for example, for adaptive speed regulation with lateral guidance or for remote control of the vehicle. The driving planning device can also be part of highway autonomous driving, parking assistance, etc. Therefore, the driving planning data from the driving planning device within the vehicle can be used to predict vehicle data.

[0014] According to an alternative implementation, driving planning data is received from a driving planning device external to the vehicle. This external driving planning device can wirelessly exchange data with the vehicle. Here, measurement data describing the environment can be transmitted from the vehicle to the external driving planning device. Thus, by means of the external driving planning device, driving planning data can be determined based on the measurement data and transmitted to the vehicle.

[0015] In another embodiment, detection data is received from an external detection device, wherein the detection data describes the vehicle's position and / or movement, and the detection data is compared with predicted vehicle data. The external monitoring device can, in particular, continuously determine the vehicle's position, speed, and / or rotational movement about the vehicle's axis. The external detection device may have at least one sensor device, by which an area of ​​the vehicle or its body can be continuously detected. Reference points for detecting the vehicle may also be proposed using the sensor device.

[0016] The sensor devices of the external detection equipment can be, in particular, optical sensors, such as cameras, lidar sensors, etc. The location of the sensor devices is known. Data provided by the sensor devices, describing the detected areas of the vehicle body, can be further processed by the external detection equipment. Specifically, detection data describing the movement of the vehicle at at least one point in time can be determined by means of the external detection equipment or its computer. The detection data can then be transmitted from the external detection equipment to the vehicle or its computing device. The vehicle's computing device can then compare this data with predicted vehicle data for inspection.

[0017] Specifically, it is proposed here that a control signal for controlling and / or regulating vehicle movement be output based on a comparison. The vehicle's position and / or movement can be determined with high precision based on detection data from external detection equipment. If a comparison between predicted vehicle data and detection data shows that the predicted vehicle data deviates from the detection data, a control signal for controlling vehicle movement can be provided. It is also proposed that the control signal be adapted based on the deviation between the predicted vehicle data and the detection data. The control signal is particularly useful for controlling the vehicle during at least partially automated operation. By taking the detection data into account, control or regulation of vehicle movement can be achieved during at least partially automated operation. Therefore, for example, the effects of tires, road surface, and / or environmental factors can be compensated for.

[0018] To improve predictions based on vehicle data, additional data can be used. For example, data from onboard and / or external sensors, such as motion sensors, inertial sensors, satellite-supported position determination systems, and / or other environmental sensors, can be used. Depending on the system, predictions can be performed intrinsically or extrinsically. Additionally, in addition to internal and external sensors, predictions can also use recursive loops as input variables.

[0019] Preferably, the measurement data is processed to calibrate the environmental sensors. Here, the measurement data can be processed or corrected taking into account the vehicle data used for calibration. Specifically, this method can be used when the vehicle is maneuvering along a calibration route. Here, the vehicle can be maneuvered at least partially automatically along the calibration route. The vehicle's movement along the calibration route can be planned by a driving plan device. Therefore, the driving plan data describes the vehicle's movement along the calibration route. Here, at least one reference object can be detected by means of an environmental sensor. Thus, the measurement data describes the reference object. In this case, the measurement data is uncalibrated or comes from an uncalibrated environmental sensor.

[0020] Furthermore, the position of the reference object can be known. Additionally, the installation position of the environmental sensor at or within the vehicle can be known. Accurate knowledge of vehicle data is crucial for calibrating the environmental sensor. Vehicle data can be predicted based on driving planning data. Measurement variables, such as the distance to the reference object, the relative speed between the vehicle and the reference object, and / or the angle between the vehicle and the reference object, can be determined based on the measurement data. These measurement variables can be compared with variables derived from the vehicle data and the known position of the reference object. Based on this comparison, calibration data can then be determined, which is used to calibrate the environmental sensor. In this case, the measurement data is corrected or adapted using the calibration data when processing the measurement data.

[0021] To increase the reliability of predicting vehicle data during calibration, the aforementioned external detection equipment can be used, and the predicted vehicle data can be compared with the detection data from the external detection equipment. Reference object data describing the location of at least one reference object and / or route data describing the calibration route can also be transmitted to the vehicle from the external detection equipment.

[0022] The computing device for a vehicle according to the invention is designed to perform the method according to the invention and its advantageous modifications. The computing device can be incorporated into at least one control device of the vehicle, which is particularly associated with environmental sensors.

[0023] The sensor system for a vehicle according to the present invention includes a computing device according to the present invention. The sensor system also includes an environmental sensor. Furthermore, the sensor system may include a driving planning device by means of which driving planning data can be provided. It may also be proposed that the sensor system receive driving planning data from a driving planning device external to the vehicle. The computing device of the sensor system is designed to predict or anticipate vehicle data at at least one point in time based on the driving planning data. Additionally, the sensor system may include a motion sensor inside the vehicle by means of which onboard electrical network data can be provided.

[0024] The vehicle according to the invention includes a computing device according to the invention or a sensor system according to the invention. The vehicle is particularly configured as a passenger car. The vehicle can also be designed as a multi-purpose vehicle.

[0025] Another aspect of the invention relates to a computer program comprising instructions that, when executed by a computing device, cause the computing device to perform the method according to the invention and to execute its advantageous design. Furthermore, the invention relates to a computer-readable (storage) medium comprising instructions that, when executed by a computing device, cause the computing device to perform the method according to the invention and to execute its advantageous design.

[0026] Another aspect of the invention relates to a detection device for detecting the exterior of a vehicle. The detection device includes sensor devices for detecting the exterior of the vehicle during operation. Furthermore, the detection device includes a computer for determining detection data based on data from the sensor devices, the detection data describing the position and / or movement of the vehicle during operation. Additionally, the detection device includes a transmission device for transmitting the detection data. The exterior detection device may have multiple sensor devices positioned at known locations. Regions of the vehicle's exterior can be detected individually using the sensor devices. The computer can determine vehicle data based on each detected region of the exterior. For this purpose, regions of the exterior can be compared with a three-dimensional model of the vehicle.

[0027] The preferred embodiments and advantages presented with reference to the method according to the invention are accordingly applicable to computing devices according to the invention, sensor systems according to the invention, vehicles according to the invention, computer programs according to the invention, computer-readable (storage) media according to the invention, and detection devices according to the invention. Attached Figure Description

[0028] Other features of the invention are derived from the claims, the drawings, and the description of the drawings. The features and combinations thereof mentioned in the foregoing description, as well as the features and combinations thereof mentioned in the following description of the drawings and / or shown separately in the drawings, may be used not only in the combinations described separately, but also in different combinations or individually, without departing from the scope of the invention.

[0029] The invention will now be explained in more detail with reference to preferred embodiments and the accompanying drawings. Herein lies:

[0030] Figure 1 A schematic diagram of a vehicle including a sensor system with environmental sensors is shown; and

[0031] Figure 2 The image shows a vehicle maneuvering on a calibration route used to calibrate environmental sensors, with external detection equipment associated with the calibration route.

[0032] Components that are identical or have the same function are given the same reference numerals in the accompanying drawings. Detailed Implementation

[0033] Figure 1 The diagram shows a top view of a vehicle 1, which is designed herein as a passenger car. Vehicle 1 includes a sensor system 2, which in turn includes environmental sensors 4. Environmental sensors 4 can be designed as, for example, lidar sensors, laser scanners (i.e., radar sensors), cameras, or ultrasonic sensors. In principle, sensor system 2 may include multiple environmental sensors 4.

[0034] Additionally, sensor system 2 includes a computing device 3 connected to environmental sensor 4 for data transmission. The computing device 3 can be configured, for example, by means of a control device. Environmental sensor 4 can provide measurement data describing objects or reference objects 6 in the environment 5 of vehicle 1. The measurement data can be transmitted from environmental sensor 4 to computing device 3 for further processing. Furthermore, sensor system 2 includes a receiving device 7, by means of which data can be received wirelessly.

[0035] Furthermore, the sensor system 2 includes a travel planning device 15, by means of which future and at least partially automated driving of the vehicle 1 can be planned. Specifically, travel planning data describing the movement of the vehicle 1 during future at least partially automated driving can be determined by means of the travel planning device 15. The travel planning data is transmitted from the travel planning device 15 to a computing device 3. The computing device 3 can then estimate or predict vehicle data at at least one point in time during future at least partially automated driving based on the travel planning data. The vehicle data describes the position of the vehicle 1, the speed of the vehicle 1, and / or the rotation of the vehicle 1 about its vertical axis, lateral axis, and / or longitudinal axis. Control signals for controlling the vehicle 1 during at least partially automated driving can also be output by means of the travel planning device 15. Alternatively, it can be proposed that the travel planning data be determined and transmitted to the vehicle 1 by means of a travel planning device external to the vehicle.

[0036] In addition, sensor system 2 includes motion sensor 16, which provides onboard electrical network data. The onboard electrical network data also describes the position and / or movement of vehicle 1. However, due to the vehicle's onboard electrical network architecture, the onboard electrical network data is transmitted to computing device 3 with a delay or a certain waiting time.

[0037] The measurement data from environmental sensor 4 can be processed based on the predicted vehicle data. The measurement data can be processed to detect the environment 5. The measurement data and vehicle data can also be used to calibrate environmental sensor 4. This will be referenced below. Figure 2 To explain.

[0038] Figure 2 A schematic diagram is shown of a vehicle 1 maneuvering along a calibration route 8. During the maneuvering of vehicle 1 on the calibration route 8, the environmental sensors 4 of vehicle 1 should be calibrated. The calibration route 8 can be a defined section of route on which vehicle 1 moves. Here, vehicle 1 can maneuver at least partially automatically along the calibration section 8. For example, vehicle 1 can be maneuvered on the calibration route 8 after production or in a factory environment.

[0039] The reference object 6 detected by the environmental sensor 4 is associated with the calibration route 8. For this purpose, the environmental sensor 4 provides measurement data describing the relative orientation of the environmental sensor 4 with respect to the reference object 6 and / or the relative velocity between the environmental sensor 4 and the reference object 6. As described above, the measurement data can be transmitted to the computing device 3. Furthermore, data describing the installation position of the environmental sensor 4 at the vehicle 1 can be stored in the computing device 3 or in the memory of the computing device 3.

[0040] Furthermore, to calibrate the environmental sensor 4, it is necessary to know the actual relative position of vehicle 1 with respect to reference object 6 or the actual relative speed between vehicle 1 and reference object 6. Here, the travel path of vehicle 1 during maneuvering on calibration route 8 is known, as the travel path is planned by the travel planning device 15. Therefore, vehicle data such as speed, acceleration, yaw rate, and roll can be predicted with high confidence. Correction data can then be determined within vehicle 1 using computing device 3 based on the uncalibrated measurement data and vehicle data. Calibration of the environmental sensor 4 can then be performed based on the correction data.

[0041] Here, detection data provided by an external detection device 9 is also used to improve the prediction of vehicle data. The external detection device 9 includes at least one sensor device 10, by which the vehicle 1 can be continuously detected during maneuvers on the calibration route 8. In the illustrated embodiment, the external detection device 9 includes, for example, two sensor devices 10 positioned on opposite sides of the calibration route 8. Furthermore, the sensor devices 10 are held at corresponding posts 11 or stakes. The sensor devices 10 can be designed, for example, as cameras or lidar sensors.

[0042] The corresponding sensor device 10 can provide data describing various areas of the vehicle body 12. The data from the sensor device 10 can then be transmitted to the computer 13 of the external detection device 9. For example, a model of the vehicle body 12 can be stored in the memory of the computer 13. Therefore, detection data can be determined based on the data provided by the sensor device 10 and the model of the vehicle 1. The detection data describes the position and / or movement of the vehicle 1 on the calibration route 8. The detection data can then be transmitted via the transmitting device 14 of the external detection device 9 to the receiving device 7 of the vehicle 1 or the receiving device of the sensor system 2.

[0043] It can also be proposed to adapt the control signal for controlling the movement of vehicle 1 along the calibration route 8 based on the comparison between the detected data and the predicted vehicle data. Furthermore, onboard electrical network data, which also describes the movement of vehicle 1 and is provided by the vehicle's internal motion sensors 16, can be used. The onboard electrical network data can additionally be used to improve the confidence level of the vehicle data predictions.

Claims

1. A method for processing measurement data from an environmental sensor (4) of a vehicle (1) to calibrate the environmental sensor (4), comprising the following steps: Receive driving planning data describing the future movement of the vehicle (1) during at least partially automated operation of the vehicle (1); Based on the driving planning data, vehicle data describing the position and / or movement of the vehicle (1) at at least one point in time during the future movement of the vehicle (1) is predicted. Receive onboard electrical network data describing the position and / or movement of the vehicle (1) from at least one motion sensor (16) of the vehicle (1); Receive measurement data from the environmental sensor (4) at at least one time point, wherein the measurement data describes the environment (5) of the vehicle (1); and Taking into account the predicted vehicle data and the on-board electrical network data, the measurement data received for the at least one time point is processed to calibrate the environmental sensor (4).

2. The method according to claim 1, Its features are, The driving planning data is received from the driving planning device (15) of the vehicle (1).

3. The method according to claim 1, Its features are, The driving planning data is received from a driving planning device outside the vehicle.

4. The method according to any one of claims 1 to 3, Its features are, Receive detection data from an external detection device (9), wherein the detection data describes the position and / or movement of the vehicle (1), and compare the detection data with predicted vehicle data.

5. The method according to claim 4, Its features are, The comparison outputs a control signal for controlling and / or regulating the movement of the vehicle (1).

6. A computing device (3) for a vehicle (1), wherein the computing device (3) is designed to perform the method according to any one of claims 1 to 5.

7. A computer program product comprising instructions that, when executed by a computing device (3), cause the computing device (3) to perform the method according to any one of claims 1 to 5.

8. A computer-readable storage medium comprising instructions, which, when executed by a computing device (3), cause the computing device (3) to perform the method according to any one of claims 1 to 5.

Images