Method and device for operating a vehicle

Abstract

Method for operating a vehicle having multiple environmental sensors (103, 105, 107, 109) for detecting the vehicle's environment, comprising the following steps: - Capturing (201) the environment of the vehicle using the environment sensors (103, 105, 107, 109), so that the environment sensors (103, 105, 107, 109) each provide raw data corresponding to the environment captured by the respective environment sensor, - Determining (203) object-related object data for each environmental sensor based on the raw data of the corresponding environmental sensor, - Fusion (205) of the respective object data of the environment sensors (103, 105, 107, 109) with each other, so that fused object data are determined, - Fusing (207) the respective raw data from the environmental sensors (103, 105, 107, 109) together, so that fused raw data are determined, - Determining (209) objects corresponding to raw object data based on the fused raw data, - comparing (211) the merged object data and the raw object data, - Control (213) at least one vehicle system depending on the comparison, wherein the driver assistance system provides a limited driver assistance function if the fused object data and the raw object data have differences that are outside a predetermined error tolerance range.

Description

[0001] The invention relates to a method and a device for operating a vehicle. The invention further relates to a computer program. State of the art

[0002] The patent application DE 101 33 945 A1 discloses a method and a device for exchanging and jointly processing object data between sensors and a processing unit, wherein position information and / or speed information and / or other object attributes of sensor objects and fusion objects are transmitted and processed.

[0003] The paper by Haberjahn, Mathias et al. (2011) “Vehicle environment detection – by a combined low and mid level fusion of a laser scanner and stereo vision”, in: IEEE Conference of Intelligent Transportation Systems, discloses a sensor system for observing the environment of a vehicle, comprising a multi-layer laser scanner and a stereo camera. Through a multi-stage fusion process, the heterogeneous sensor data can be combined at different processing levels.

[0004] Document DE 10 2010 021 591 A1 discloses a method for controlling the operation of a fully automatic driver assistance system for a motor vehicle, designed for independent vehicle control. Disclosure of the invention

[0005] The object underlying the invention can be seen as providing a method for operating a vehicle wherein the vehicle has several environmental sensors for detecting an environment of the vehicle.

[0006] The object underlying the invention can also be seen as providing a device for operating a vehicle.

[0007] The object underlying the invention can further be seen as providing a computer program.

[0008] These tasks are solved by means of the respective subject matter of independent claims 1, 7 and 9. Advantageous embodiments are the subject of dependent subclaims.

[0009] According to one aspect, a method for operating a vehicle is provided, wherein the vehicle has several environmental sensors for detecting the vehicle's environment, comprising the following steps: - Capturing the vehicle's surroundings using the environmental sensors, so that the environmental sensors each provide raw data that corresponds to the environment captured by the respective environmental sensor, - Determining objects and corresponding object data for each environmental sensor based on the raw data of the respective environmental sensor, - Fusing the respective object data from the environmental sensors together, so that fused object data is determined, - Fusing the respective raw data from the environmental sensors together, so that fused raw data is determined, - Identifying objects corresponding to raw object data based on the merged raw data, - comparing the merged object data with the raw object data, - Control of at least one vehicle system depending on the comparison.

[0010] According to another aspect, a device for operating a vehicle is provided, comprising: - multiple environmental sensors to detect the vehicle's surroundings, - wherein the environmental sensors are designed to provide raw data corresponding to the environment detected by the respective environmental sensor, - a processing unit configured to perform the steps of identifying, fusing and comparing according to the inventive method, and - a control device for controlling at least one vehicle system depending on the comparison.

[0011] Another aspect is that a computer program is provided which includes program code for carrying out the method according to the invention when the computer program is executed on a computer, in particular a processing device.

[0012] According to another aspect, a vehicle is provided which includes the device according to the invention.

[0013] The invention therefore encompasses, in particular, the concept of both performing object-based fusion of sensor objects and fusing raw data from environmental sensors to determine raw object data corresponding to objects based on the fused raw data. According to the invention, the results of these two methods—the object-based fusion and the raw data fusion followed by the determination of the raw object data—are compared, so that, for example, any errors that may be present in one of the methods or in one of the method steps can be advantageously detected. Depending on the comparison, at least one vehicle system is then advantageously controlled. Thus, redundancy is advantageously created, since object data—the fused object data and the raw object data—are determined in two different ways.Thus, decisions regarding the control of the vehicle system can be advantageously secured, and in particular better secured, in the event of a match, especially within a specific error tolerance range. If both the fused object data and the raw object data lie within a specific error tolerance range, it can generally be assumed that objects actually exist at the real locations in the vehicle's environment that correspond to this fused object data and raw object data.

[0014] An environmental sensor according to the present invention comprises, in particular, a passive and / or an active sensor and a control unit associated with this sensor, which can, for example, be called a sensor control unit. The measurement can, for example, be carried out in the sensor control unit.

[0015] According to one embodiment, the environmental sensors can be identical or, for example, different. An environmental sensor can be, for example, a video sensor, a radar sensor, an ultrasonic sensor, a laser sensor, or a lidar sensor.

[0016] According to one embodiment, the control may include controlling a warning signal device to provide a warning signal to a driver of the vehicle such that a warning signal is provided when the fused object data and the raw object data have differences that are outside a predetermined error tolerance range.

[0017] The driver is thus advantageously warned that there are deviations that lie outside the predetermined error tolerance range. He can then advantageously adjust his driving style accordingly. The warning signal can be, for example, a visual, audible, or haptic signal. In particular, several warning signals can be provided, which may be the same or, preferably, different.

[0018] According to the invention, the control comprises controlling a driver assistance system of the vehicle such that the driver assistance system provides a limited driver assistance function when the fused object data and the raw object data have differences that lie outside a predetermined error tolerance range.

[0019] Limited functionality is therefore advisable, as a driver assistance system typically relies on object data and / or raw object data for its decisions. A decision might, for example, be an answer to the following questions: Is the vehicle allowed to overtake? Is there an obstacle? Is the vehicle staying in its lane or not? However, since the discrepancies lie outside the predetermined error tolerance range, the object data and raw object data are generally no longer sufficiently reliable to safely provide the full functionality or range of functions.

[0020] For example, an automatic speed control system (also called ACC, which stands for Adaptive Cruise Control) will reduce its target speed or only allow target speeds that are lower than a predetermined target speed threshold. A driver assistance system that normally supports a driver during an overtaking maneuver will refuse to provide assistance if the differences fall outside the predetermined error tolerance range. The driver is then completely on their own and must perform the overtaking maneuver independently. However, since the object data, especially the fused object data and the raw object data, are no longer sufficiently reliable, this advantageously prevents such an overtaking assistant from failing to make decisions or intervening in vehicle operation or control in ways that could compromise safety.

[0021] In yet another embodiment, the control may include controlling a processing device in such a way that at least some of the data are discarded and recalculated if the merged object data and the raw object data have differences that lie outside a predetermined error tolerance range.

[0022] A recalculation therefore takes place, offering a new chance that this time the differences will fall within the predetermined error tolerance range, and the data can thus be considered sufficiently reliable. By discarding the data, there is also no longer any risk that the discarded data could otherwise be used for other calculations and lead to incorrect results that could, for example, compromise vehicle safety.

[0023] In yet another embodiment, the object data is determined internally within the corresponding environmental sensor, which then provides the determined object data. This internal determination can be carried out, for example, by means of an internal processing unit (i.e., within the environmental sensor).

[0024] This advantageously achieves the benefit that at least part of the processing has already been completed. Therefore, only a correspondingly smaller computing and / or storage capacity needs to be provided for the remaining processing steps.

[0025] According to a further embodiment, the environmental sensors can provide their raw data externally, so that the determination of the object data for each environmental sensor is carried out externally by the respective environmental sensor. This external determination can be performed, for example, by means of an external processing unit (i.e., external to the environmental sensor).

[0026] This advantageously achieves the benefit that even environmental sensors that cannot internally determine the object data themselves, for example because they lack the necessary computing capacity, can be used for the inventive method and / or the inventive device. Thus, older sensor models can also be used. In particular, older vehicles can be retrofitted.

[0027] According to one embodiment, the determination of object data can be carried out both internally within the environmental sensors themselves and externally by the environmental sensors. Externally, for example, by means of a processing unit provided externally by the environmental sensors. Internally, for example, by means of a processing unit provided internally within the environmental sensors.

[0028] In one embodiment, the merging process (in particular, the merging of object data and / or the merging of raw data) and / or the determination or specification of a predetermined error tolerance range may include the determination of a quality or measure. The quality or measure can, for example, be determined depending on the object's position or location. An object located at the edge of the error tolerance range, relative to its center, will have a different measure than an object located relatively closer to the center. The quality or measure indicates, in particular, how reliable a determined or calculated result or value is, since determined or calculated results or values ​​can contain errors. The quality or measure thus corresponds, in particular, to an error bar. The quality or measure may, for example, represent a probability.This means that object data, and in particular fused object data, can comprehensively and accurately make a statement about the probability Z that an object X is present at a specific location.

[0029] Embodiments of the method result from the corresponding embodiments of the device and vice versa.

[0030] Statements made in connection with the process apply analogously to the device and vice versa.

[0031] In one embodiment, the processing device may comprise several processing units for determining the object data, with each environmental sensor comprising a processing unit so that the environmental sensors can output the determined object data. These processing units can be referred to as internal processing units with respect to the environmental sensors.

[0032] A processing unit can, for example, be a control unit of the environmental sensor or be integrated into such a control unit.

[0033] In a further embodiment, the processing device may include at least one processing unit for determining the object data, which is located externally from the environmental sensors, so that the environmental sensors can output their raw data to the at least one processing unit for determining the object data. This at least one processing unit can be referred to as an external processing unit in relation to the environmental sensors.

[0034] According to one embodiment, it can be provided that several vehicle systems are controlled depending on the comparison. The vehicle systems can, for example, be identical or, preferably, different.

[0035] The invention will be explained in more detail below with reference to preferred embodiments. Here, we will show... Fig. 1 a device for operating a vehicle and Fig. 2. A flowchart of a procedure for operating a vehicle.

[0036] Fig. Figure 1 shows a device 101 for operating a vehicle (not shown).

[0037] The device 101 comprises several environmental sensors 103, 105, 107, and 109. For example, environmental sensor 103 can be a video sensor. Environmental sensor 105 can be, for example, a radar sensor. Environmental sensor 107 can be, for example, a lidar sensor. Environmental sensor 109 can be, for example, an ultrasonic sensor. The four environmental sensors 103, 105, 107, and 109 are each configured to provide raw data corresponding to the environment detected by the respective environmental sensor.

[0038] This raw data is provided to a processing unit 111, which comprises several processing units 113, 115, 117, and 119. More precisely, the raw data from the environmental sensors 103, 105, 107, and 109 are provided to processing unit 115. Processing unit 115 merges the raw data from the environmental sensors 103, 105, 107, and 109 to determine merged raw data. Based on this merged raw data, processing unit 115 determines raw object data corresponding to objects.

[0039] Within the environmental sensors 103, 105, 107, and 109 themselves, object data corresponding to objects is determined internally for each environmental sensor based on the raw data of the respective sensor. This is preferably carried out by means of respective processing units (not shown here), which are, for example, arranged in a respective sensor control unit. These processing units (not shown) are also included in the processing unit 111. This object data from the respective environmental sensors 103, 105, 107, and 109 is provided to a further processing unit 113 of the processing unit 111. The further processing unit 113 merges the respective object data from the environmental sensors 103, 105, 107, and 109 to determine merged object data.

[0040] Both the raw object data and the merged object data are provided to another processing unit 117. This unit compares the merged object data and the raw object data with each other.

[0041] The device 101 further comprises a control unit 121, which is configured to control at least one vehicle system depending on the comparison. This means, in particular, that the further processing unit 117 can provide the result of the comparison to the control unit 121. In the Fig. In the embodiment shown in Figure 1, the control device 121 is arranged externally from the processing device 109. In an embodiment not shown, for example, the control device 121 may be integrated into the processing device 111.

[0042] The processing unit 111 also includes another processing unit 119, which can perform further calculations based on the comparison. Such calculations can include, for example, merging the merged object data and the raw object data. This merging of the merged object data and the raw object data can be parameterized, i.e., parameterized merging. In particular, it can be parameterized based on the comparison, i.e., on a result of the comparison. For example, different parameters can be used during a recalculation. The merging of the merged object data and the raw object data can, for example, calculate a quality factor for the merging result. This quality factor could, for example, be a probability.This means that object data, and in particular fused object data, can comprehensively and accurately make a statement about the probability Z that an object X is present at a specific location.

[0043] In an embodiment not shown, more or fewer than four environmental sensors may be provided.

[0044] According to one embodiment, an environment model for the vehicle's surroundings can be determined based on the fused raw data, particularly using processing unit 115. In particular, a further environment model is determined based on the fused object data, particularly using processing unit 113. These two environment models are then compared, particularly using processing unit 117. This comparison of the two environment models includes, in particular, the step of comparing the fused object data and the raw object data. If the two environment models are outside a predetermined error tolerance range, a recalculation of at least one of the two environment models, preferably both, can be provided. The data underlying the environment models can, for example, be discarded.

[0045] A fault tolerance range can, for example, encompass a predetermined number of objects that are not present in both the merged object data and the raw object data. For instance, if the merged object data indicates 3 objects in the vicinity of the vehicle, but the raw object data indicates 6 objects, then three objects are not present in both the merged and raw object data. Depending on the specific value of the predetermined number, this can result in discrepancies (3 objects) that lie either outside or within the fault tolerance range (predetermined number). For example, the predetermined number could be 2. In this case, the discrepancies lie outside the predetermined fault tolerance range. Alternatively, the predetermined number could be 4. In this case, the discrepancies lie within the predetermined fault tolerance range.It should be noted that the above values ​​and object counts are for illustrative purposes only and are not binding. Other values ​​are possible depending on the specific individual case.

[0046] Fig. Figure 2 shows a flowchart of a procedure for operating a vehicle which has several environmental sensors for detecting the vehicle's surroundings.

[0047] According to step 201, the multiple environmental sensors each encompass a vehicle environment, so that each environmental sensor provides raw data corresponding to the environment detected by the respective sensor. In step 203, object data is determined for each environmental sensor based on the raw data of the corresponding sensor, where the object data corresponds to objects. In step 205, the respective object data from the environmental sensors is fused together to determine fused object data.

[0048] In step 207, the respective raw data from the environmental sensors are fused together to determine fused raw data. In step 209, corresponding raw object data is determined based on the fused raw data.

[0049] In step 211, the fused object data and the raw object data are compared, so that, according to step 213, at least one vehicle system is controlled depending on the comparison.

[0050] According to an embodiment not shown, it is possible to control several vehicle systems depending on the comparison. The vehicle systems can, for example, be identical or, preferably, different.

Claims

[1] Method for operating a vehicle having multiple environmental sensors (103, 105, 107, 109) for detecting the vehicle's environment, comprising the following steps: - Capturing (201) the environment of the vehicle using the environment sensors (103, 105, 107, 109), so that the environment sensors (103, 105, 107, 109) each provide raw data corresponding to the environment captured by the respective environment sensor, - Determining (203) object-related object data for each environmental sensor based on the raw data of the corresponding environmental sensor, - Fusion (205) of the respective object data of the environment sensors (103, 105, 107, 109) with each other, so that fused object data are determined, - Fusing (207) the respective raw data from the environmental sensors (103, 105, 107, 109) together, so that fused raw data are determined, - Determining (209) objects corresponding to raw object data based on the fused raw data, - comparing (211) the merged object data and the raw object data, - Control (213) at least one vehicle system depending on the comparison, wherein the driver assistance system provides a limited driver assistance function if the fused object data and the raw object data have differences that are outside a predetermined error tolerance range. [2] Method according to claim 1, wherein the control comprises controlling a warning signal device to provide a warning signal to a driver of the vehicle such that a warning signal is provided when the fused object data and the raw object data have differences that are outside a predetermined error tolerance range. [3] Method according to a previous claim, wherein the controlling comprises controlling a processing device (111) such that at least some of the data are discarded and recalculated if the merged object data and the raw object data have differences that are outside a predetermined error tolerance range. [4] Method according to one of the preceding claims, wherein the determination of the object data is carried out internally in the corresponding environment sensor, which accordingly provides the determined object data. [5] Method according to one of the preceding claims, wherein the environmental sensors provide their raw data externally, so that the determination of the object data for each environmental sensor is carried out externally by the respective environmental sensor. [6] Device (101) for operating a vehicle, comprising: - multiple environmental sensors to detect the vehicle's surroundings, - wherein the environmental sensors are designed to provide raw data corresponding to the environment detected by the respective environmental sensor, - a processing device (111) configured to perform the steps of identifying, merging and comparing according to any of the preceding claims, and - a control device (121) for controlling at least one vehicle system depending on the comparison, wherein the driver assistance system provides a limited driver assistance function if the fused object data and the raw object data have differences that are outside a predetermined error tolerance range. [7] Device (101) according to claim 6, wherein the processing device (111) comprises several processing units (113, 115, 117, 119) for determining the object data, wherein the environmental sensors (103, 105, 107, 109) each comprise a processing unit, so that the environmental sensors (103, 105, 107, 109) can output the determined object data. [8] Device (101) according to claim 6 or 7, wherein the processing device (111) comprises at least one processing unit for determining the object data, which is provided externally from the environmental sensors (103, 105, 107, 109), so that the environmental sensors (103, 105, 107, 109) can output their raw data to the at least one processing unit for determining the object data. [9] Computer program comprising program code for carrying out the method according to any one of claims 1 to 5, when the computer program is executed on a computer.

Images