Method and device for mitigating the effect of a faulty sensor in a vehicle

Abstract

The invention relates to a method (100) for mitigating the effect of a faulty sensor (200) of a vehicle (70), comprising the following steps: - Providing (101) sensor data (SDO, SD1, SD2, SD3) of at least one sensor (200) of the vehicle (70); - Calculating (102) external processing results (eVE) based on the provided sensor data (SDO, SD1, SD2, SD3) of at least one sensor (200) of the vehicle (70); - Providing (103) vehicle-internal processing results (iVE) based on the sensor data (SDO, SD1, SD2, SD3) of at least one sensor (200) of the vehicle (70); - Comparing (104) the external processing results (eVE) with the vehicle-internal processing results (iVE); - Initiating (105) the mitigation of the effect of the faulty sensor (200), preferably by outputting a signal to the vehicle (70), depending on the comparison.

Description

[0001] The invention relates to a method for mitigating the effect of a faulty sensor in a vehicle, a data processing device, a vehicle, and a system. State of the art

[0002] There are already approaches to diagnosing or validating sensors via infrastructure. For example, validation is performed using "known objects" or test patterns in the vehicle's environment, as is the case when vehicles exit a fleet garage at mobility service providers. Validation via interaction with other V2V-connected vehicles is also possible. However, safety considerations often necessitate the deactivation of the sensor or sensor component when a fault is diagnosed. This can occur, for instance, with only one sensor (plus diagnostics) due to a lack of redundancy for fallback or for implementing a safety-related function. Even with two sensors, a fault usually cannot be diagnosed in a specific sensor; rather, the existing discrepancy can only be identified by comparing both sensors.In this case, it is often impossible to determine which of the two sensors is still reliable. Furthermore, a detailed interpretation of the specific impact and situational criticality of a fault is often lacking. It can happen that the failure of one sensor is not critical for the current situation and the remainder of the mission, but this assessment is difficult or even impossible to make from within the vehicle. Correction or reparameterization is also often not reliably feasible due to the aforementioned issues. Even a temporary reboot for fault diagnosis and correction is often not reliably feasible without additional external information about the associated criticality.

[0003] EP 2436 192 A1 discloses a vehicle sensor or a system comprising a control unit for determining the vehicle's condition and at least two vehicle sensors, as well as a method for operating such a system. Disclosure of the invention

[0004] Against this background, the approach presented here provides a method with the features of claim 1, a device with the features of claim 11, a vehicle with the features of claim 12, a system with the features of claim 13, a computer program with the features of claim 14 and a computer-readable medium with the features of claim 15.

[0005] The method for mitigating the effect of a faulty sensor in a vehicle comprises the steps of providing sensor data from at least one sensor of the vehicle, calculating external processing results based on the provided sensor data from the at least one sensor of the vehicle, providing vehicle-internal processing results based on the sensor data from the at least one sensor of the vehicle, comparing the external processing results with the vehicle-internal processing results, and initiating the mitigation of the effect of the faulty sensor depending on the comparison.

[0006] Preferably, mitigation is initiated by sending an output signal to the vehicle.

[0007] Providing sensor data can mean, for example, receiving data from an external vehicle device. The data can be sent to the external device by the vehicle itself. Processing results can refer to any results or information that can be calculated, extracted, or obtained from the sensor data. Providing in-vehicle processing results can mean, for example, receiving the in-vehicle processing results from the external device. The in-vehicle processing results can be sent to the external device by the vehicle itself. Alternatively, it can also mean that the in-vehicle processing results are made available to the vehicle or...Vehicle-internal facilities, such as a control unit, are provided by the vehicle. Advantages of the invention

[0008] This method enables more comprehensive diagnostics, including for sensor faults that are difficult to detect, especially using in-vehicle diagnostic tools. This enhances the safety of the vehicle and its surrounding environment. Furthermore, required diagnostic functions can potentially be partially or even completely outsourced to a cloud with potentially much higher computing power, leading to greater efficiency. Depending on the diagnostic function, this can result in savings not only in software but also in hardware. In addition, both the detection and evaluation of sensor faults take place in an independent system, increasing availability. This allows non-safety-critical or transient sensor faults to be tolerated without immediately putting the sensor, the vehicle, or any sensor-related system into a safe state. Further advantages arise from the sub-claims.

[0009] In a preferred embodiment, the sensor data is specific to at least one sensor component, in particular to preprocessing, processing, postprocessing, and / or interpreting the sensor data. This allows the sensor error to be further narrowed down and even more specific measures to be taken to mitigate the faulty sensor or sensor component.

[0010] Furthermore, it is advantageous if, in the step of calculating external processing results, these results are additionally based on external sensor data and / or data from an environmental model. External sensor data can include, for example, data from roadside units (RSUs), external traffic cameras, external radar, ultrasonic, and / or lidar sensors. Data from an environmental model can be understood as, for example, data that can be extracted from an environmental model that has already been calculated, particularly based on the external sensor data. This can significantly increase the accuracy, confidence, and / or robustness of the external processing results.

[0011] In a further embodiment, a faulty sensor or sensor component may be detected if, during the comparison step, the vehicle's internal processing results deviate from the external processing results by a predefinable or predetermined limit, preferably by more than 10%, particularly preferably by more than 5%, and most preferably by more than 2%. This allows for a simple determination of whether a faulty sensor or sensor component is present.

[0012] Furthermore, it is advantageous to precede the initiation step with a determination step, in which a comparison is made to determine whether a safety-critical fault or a non-safety-critical, particularly transient, fault is present. Knowing this information allows the measures for mitigating the impact of the faulty sensor to be adjusted accordingly.

[0013] It is particularly advantageous if the determination step is based on current and / or future traffic conditions, especially the remaining distance of the journey. This allows for a more precise and situation-dependent determination of whether a safety-critical or non-safety-critical, particularly transient, error is present. Consequently, any mitigation measures can be defined more precisely, significantly reducing, for example, the frequency of complete sensor deactivation and the associated loss of a specific driving function.

[0014] In a further embodiment, it can be provided that, during the initiation step, an output signal is issued such that a driver or the vehicle itself is informed of the presence of a fault, particularly a non-safety-critical one. This can advantageously warn the driver, allowing them to adjust their driving style accordingly. Specifically, the output signal can instruct the driver to perform a specific action, such as deactivating a driving function impaired by the faulty sensor. This can advantageously guide the driver to return the vehicle to a safe state.

[0015] In a further preferred embodiment, it can be provided that the vehicle remains operational for a predefinable or predetermined period during the initiation step. This ensures that, in the event of a fault, particularly a non-safety-critical one, the vehicle is not instantly switched to a safe state or even automatically shut down, but rather that the driver can continue to operate the vehicle autonomously for a certain period of time.

[0016] Furthermore, it can be provided that the external processing results are transmitted to the vehicle during the initiation step. This allows the vehicle to choose whether and to what extent it considers the external processing results in its internal calculations.

[0017] Preferably, the external processing results, particularly those from the vehicle, can be checked for safety integrity, confidence, reliability, and / or availability along a route. This can be achieved through additional metadata of the data or the external processing results, containing information about the ASIL and / or the confidence / uncertainty estimation and / or the QoS guarantees (latency, determinism, availability, etc.) of the external signal.

[0018] Furthermore, it may be possible to transmit external sensor data to the vehicle during the initiation step. This allows the vehicle to choose whether and to what extent it considers the external sensor data in its internal calculations.

[0019] Furthermore, it can be provided that data from an environmental model is transmitted to the vehicle during the initiation step. This allows the vehicle to choose whether and to what extent it considers data from an environmental model in its internal calculations.

[0020] In a further preferred embodiment, it can be provided that, during the initiation step, the external processing results replace the vehicle's internal processing results. This ensures that the erroneous vehicle-internal processing results are not reused for the execution of, for example, a driving function, particularly one relevant to safety. The driver can thus rely on the fact that the vehicle and its respective control systems are always supplied with the correct processing results.

[0021] In a further preferred embodiment, it can be provided that, during the initiation step, the external sensor data replaces the vehicle's internal processing results and / or sensor data. This ensures that the faulty sensor data is not used for the execution of, for example, a driving function, particularly one that is safety-relevant. The driver can therefore rely on the fact that the vehicle and its respective control systems are always supplied with the correct sensor data.

[0022] Alternatively or additionally, it can be provided that, during the initiation step, data from an environmental model replaces the vehicle's internal processing results and / or sensor data. This ensures that faulty sensor data is not used for the execution of, for example, a driving function, especially one that is safety-relevant. The driver can therefore rely on the fact that the vehicle and its respective control systems are always supplied with the correct sensor data.

[0023] The aforementioned advantages also apply accordingly to a device, particularly for data processing, that is configured to perform at least some, preferably all, process steps of the method according to one of the previously described embodiments. For example, the device can include a control unit, wherein the control unit is configured to perform at least one of the steps of one of the methods according to the previous embodiments. Preferably, the device is configured as the vehicle-external unit.

[0024] The method can be implemented, for example, in software, hardware, or a hybrid of both in the device and / or the control unit. The device and / or the control unit can include at least one evaluation unit for processing signals or data, at least one storage unit for storing signals or data, at least one interface to a sensor unit or actuator for reading sensor signals or parameters from the sensor unit or for outputting control signals to the actuator, and / or at least one communication interface for reading or outputting data embedded in a communication protocol. The evaluation unit can be, for example, a signal processor, a microcontroller, or the like, while the storage unit can be flash memory, an EPROM, or a magnetic storage device.The communication interface can be configured to read or output data wirelessly and / or via wired connections, whereby a communication interface that can read or output wired data can, for example, read this data electrically or optically from or output it into a corresponding data transmission line.

[0025] The invention also relates to a vehicle that can be connected or is connected to the previously described device via signal technology, wherein the vehicle is equipped to initiate a mitigation of an effect of the faulty sensor. This can be initiated, for example, by receiving a corresponding output signal.

[0026] Furthermore, a system comprising the device and vehicle described above is also the subject of the invention.

[0027] The invention also relates to a computer program product or computer program with program code, which may be stored on a machine-readable, in particular non-volatile, carrier or storage medium such as a semiconductor memory, a hard disk memory or an optical memory and is used for carrying out, implementing and / or controlling the steps of the method according to one of the embodiments described above, in particular when the program product or program is executed on a computer or a device according to one of the embodiments described above.

[0028] The invention also relates to a computer-readable storage medium comprising the computer program. The storage medium is, for example, designed as a data storage device such as a hard drive and / or non-volatile memory and / or a memory card. The storage medium can, for example, be integrated into the computer or a device according to one of the embodiments described above. Brief description of the drawing

[0029] Exemplary embodiments of the invention are shown schematically in the drawings and explained in more detail in the following description. The same reference numerals are used for the elements shown in the various figures that have a similar effect, thus omitting a repeated description of the elements.

[0030] They show: Fig. 1 a schematic representation of a method, a device, a vehicle, a system and a computer program for mitigating the effect of a faulty sensor of a vehicle according to exemplary embodiments; Fig. 2 a schematic representation of the system for mitigating the effect of a faulty sensor in a vehicle according to a further embodiment; and Fig. 3 A schematic representation of the system for mitigating the effect of a faulty sensor in a vehicle according to a further embodiment.

[0031] As already explained above, the present invention describes a method, a device, a vehicle, a system and a computer program that make it possible to increase the availability of a, in particular automated, driving function of a vehicle and to ensure safe operation of the vehicle.

[0032] Fig. Figure 1 illustrates a method 100 and a system 80 for mitigating the effect of a faulty sensor of a vehicle 70 according to exemplary embodiments of the invention.

[0033] According to a first process step 101, sensor data from at least one sensor of the vehicle can be provided, in particular to an external device 60. The sensor data can, for example, be specific to a driving function of the vehicle 70. The sensor data can be acquired, for example, by internal sensors of the vehicle 70, in particular by an inertial sensor, by an optical sensor, preferably a camera or a LiDAR sensor, by a radar sensor, or by an ultrasonic sensor. Subsequently, according to a second process step 102, external processing results can be calculated based on the provided sensor data from the at least one sensor of the vehicle 70. Then, according to a third process step 103, vehicle-internal processing results can be provided based on the sensor data from the at least one sensor of the vehicle 70.Then, according to a fourth process step 104, a comparison of the external processing results with the vehicle-internal processing results can be carried out, and according to a fifth process step 105, the mitigation of the effect of the faulty sensor can be initiated, preferably by outputting an output signal to the vehicle 70, depending on the comparison.

[0034] The process steps 101-105 can be performed by a device 10. The device 10 is, for example, configured as a computer and / or as a data processing device 10 and / or as the vehicle-external unit 60, and can include means for performing the steps of a process 100 according to exemplary embodiments of the invention. Furthermore, the device 10 can have a communication interface for networking, in particular wirelessly, with other devices, units, vehicles, or the like. The device 10 can also include a computer program 50 according to exemplary embodiments of the invention. When executed by a computer or the device 10, the computer program 50 can cause it(s) to perform the steps of the process 100 according to exemplary embodiments of the invention.Together, the vehicle 70 and the device 10 form a system 80 for mitigating the effect of the faulty sensor of the vehicle.

[0035] In Fig. Figure 2 shows a schematic representation of the system 80 for mitigating the effect of a faulty sensor 200 of a vehicle 70 according to a further embodiment. The system 80 comprises a device 10 configured as an external vehicle unit 60, in particular as a cloud 11, and at least one vehicle 70. The vehicle 70 has at least one sensor 200, in particular an internal vehicle unit, which in turn may comprise individual sensor components 210, 211, 212, 213. A sensor component 210 may be provided for preprocessing, a sensor component 211 for processing, a sensor component 212 for postprocessing, and / or a sensor component 213 for interpreting the sensor data. The sensor 200 may, for example, be configured as a camera and acquire image data from an environment 300, in particular the immediate vicinity, of the vehicle 70.The sensor 200 can alternatively be configured as an inertial sensor and sensing, in particular instantaneously, the pose, location, velocity, and / or acceleration of the vehicle 70. The sensor data SD0, SD1, SD2, SD3 of the sensor 200 are preferably processed along the individual sensor components 210, 211, 212, 213 and forwarded to a perception unit 220 of the vehicle 70. The perception unit 220 is configured to calculate a vehicle-internal processing result iVE based on the sensor data SD0, SD1, SD2, SD3 of the sensor 200 and to regularly forward this result to the vehicle 70 in order to implement a specific function, in particular a driving function, in the vehicle 70.The vehicle 70 also has a communication unit 230, to which the sensor data SD0, SD1, SD2, SD3 from sensor 200 can also be forwarded. This allows the communication unit 230, which is connected to the vehicle's external device 60, to send the sensor data SD0, SD1, SD2, SD3 to the vehicle's external device 60. The vehicle's external device 60 calculates an external processing result eVE based on the provided sensor data SD0, SD1, SD2, SD3. Optionally, the vehicle's external device 60 can be configured to additionally calculate the external processing result eVE based on external sensor data eSD, for example, from an external camera 20, and / or data from an environment model UM. This makes the external processing result eVE particularly accurate and reliable.

[0036] The external processing result eVE can then be forwarded via the communication unit 230 to the perception unit 220 of the vehicle 70, the perception unit 220 comparing the external processing result eVE with the vehicle's internal processing result iVE 104. In a preferred embodiment, the sensor 200 and / or one of the individual sensor components 210, 211, 212, 213 can be considered faulty if the vehicle's internal processing results iVE deviate from the external processing results eVE by a predefinable or predetermined limit value, preferably by more than 10%, particularly preferably by more than 5%, and most preferably by more than 2%.Optionally, and preferably taking into account current and / or future traffic conditions, particularly the remaining journey, it can be determined whether the fault is a safety-critical fault or a non-safety-critical fault, especially a transient fault. Depending on the result of the comparison, particularly if a fault is present in sensor 200 and / or one of the individual sensor components 210, 211, 212, 213, the perception unit 220 can initiate mitigation measures. This can be done, for example, by sending an output signal to the vehicle 70 to initiate mitigation of the effect of the faulty sensor 200. The mitigation measures can depend on whether the fault is a safety-critical fault or a non-safety-critical fault, especially a transient fault.

[0037] Alternatively, the perception unit 220 can transmit the vehicle-internal processing result iVE to the vehicle-external device 60 via the communication unit 230, and the vehicle-external device 60 can then perform the comparison 104. The vehicle-external device 60 can then transmit the result of the comparison back to the perception unit 220 via the communication unit 230. Depending on the result of the comparison, particularly if there is a fault in the sensor 200 and / or one of the individual sensor components 210, 211, 212, 213, the perception unit 220 can initiate mitigation measures. This can be done, for example, by sending an output signal to the vehicle 70 to initiate mitigation of the effect of the faulty sensor 200.

[0038] Initiating the mitigation of the effect of the faulty sensor 200 can, for example, consist of informing the driver of the vehicle 70 about the existence of a fault, particularly a non-safety-critical one, in the sensor 200 and / or one of the individual sensor components 210, 211, 212, 213. Furthermore, it can be provided that the vehicle 70 remains operational for a predetermined or specified period of time.

[0039] Alternatively or additionally, it can be provided that, during the initiation step, the external processing results eVE replace the vehicle-internal processing results iVE and / or that external sensor data eSD and / or data from the environment model UM replace the sensor data SD0, SD1, SD2, SD3, or that the sensor 200 is reparameterized. For example, in the case of a sensor 200 configured as a camera with a permanent bit flip in its signal processing chip, which leads to a mirroring of the image, this can be corrected by a subsequent software step and / or the aforementioned reparameterization.

[0040] This case is in Fig. 2 shown schematically again. Fig. 2 differs from Fig. 1. This is because the individual connection lines, which are intended to illustrate the forwarding of sensor data SD1, SD2, SD3 to the communication unit 230, are not shown for the sake of clarity. Another difference to Fig. 1 consists in the fact that the output signal is output in such a way that an intermediate result, namely the sensor data SD1 expected directly after the sensor component 211, is replaced by an external sensor data eSD1 forwarded by the vehicle-external device 60 as an external processing result eVE.

[0041] In this process, the procedure 100 can be carried out regularly, in particular according to specified or predeterminable intervals, and / or event-triggered for the diagnosis of the sensor 200 or the entire vehicle 70. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] EP 2436 192 A1

[0003]

Claims

[1] A method (100) for mitigating the effect of a faulty sensor (200) of a vehicle (70), comprising the following steps: - Providing (101) sensor data (SD0, SD1, SD2, SD3) of at least one sensor (200) of the vehicle (70); - calculating (102) external processing results (eVE) based on the provided sensor data (SDO, SD1, SD2, SD3) of the at least one sensor (200) of the vehicle (70); - Providing (103) vehicle-internal processing results (iVE) based on the sensor data (SDO, SD1, SD2, SD3) of the at least one sensor (200) of the vehicle (70); - comparing (104) the external processing results (eVE) with the vehicle-internal processing results (iVE); - initiating (105) the mitigation of the effect of the faulty sensor (200), preferably outputting an output signal to the vehicle (70), depending on the comparison. [2] Method (100) according to claim 1, characterized by that the sensor data (SDO, SD1, SD2, SD3) are specific for at least one sensor component (210, 211, 212, 213), in particular for preprocessing, processing, postprocessing and / or interpreting the sensor data (SDO, SD1, SD2, SD3) of the sensor (200). [3] Method (100) according to claim 1 or 2, characterized by that in the step of calculating (102) external processing results (eVE), the external processing results (eVE) are additionally based on external sensor data (eSD) and / or data from an environment model (UM). [4] Method (100) according to one of claims 1 to 3, characterized bythat a faulty sensor (200) or a faulty sensor component (210, 211, 212, 213) is present if, in the step of comparing (104), the vehicle-internal processing results (iVE) deviate from the external processing results (eVE) by a predeterminable or predetermined limit value, preferably by more than 10%, particularly preferably by more than 5% and most particularly preferably by more than 2%. [5] Method (100) according to one of the preceding claims, characterized by that before the initiation step (105) a determination step takes place in which it is determined on the basis of the comparison whether a safety-critical error or a non-safety-critical, in particular transient, error is present. [6] Method (100) according to claim 5, characterized by that the step of determining takes place depending on a current and / or future traffic situation, in particular a remaining journey to be completed. [7] Method (100) according to one of the preceding claims, characterized by that in the initiating step an output signal is output such that a driver of the vehicle (70) is informed about the presence of an error, in particular one that is not safety-critical. [8] Method (100) according to one of the preceding claims, characterized by that in the initiating step the vehicle (70) is still operated for a predeterminable or predetermined period of time. [9] Method (100) according to one of the preceding claims, characterized by that in the initiation step, the external processing results (eVE) and / or external sensor data (eSD) and / or data from an environment model (UM) are transmitted to the vehicle (70). [10] Method (100) according to one of the preceding claims, characterized bythat in the initiation step, the external processing results (eVE) replace the vehicle-internal processing results (iVE) and / or that external sensor data (eSD) and / or data from the environment model (UM) replace the sensor data (SDO, SD1, SD2, SD3). [11] Device (10) for data processing, in particular vehicle-external device (60), which is designed to carry out the method (100) according to one of claims 1 to 10. [12] Vehicle (70), wherein the vehicle (70) is connectable or connected by signal technology to the device (10) according to claim 11, wherein the vehicle (70) is configured to receive the output signal and, depending on the received output signal, to initiate a mitigation of an effect of the faulty sensor (200). [13] System (80) comprising a device (10) according to claim 11 and a vehicle (70) according to claim 12. [14] Computer program (50) comprising instructions which, when the computer program (50) is executed by a computer or by a device (10) according to claim 11, cause the computer or device (10) to carry out the method (100) according to one of claims 1 to 10. [15] A computer-readable medium on which the computer program (50) according to claim 14 is stored.

Images