Method for operating environmental sensors in a motor vehicle and motor vehicle

Abstract

Method for operating environmental sensors (3, 4, 15, 16, 17) in a motor vehicle (8), wherein for at least some of the environmental sensors (3, 4, 15, 16, 17) operating information describing the availability of the respective rated environmental sensor (3, 4, 15, 16, 17) and / or the reliability of the sensor data of the respective rated environmental sensor (3, 4, 15, 16, 17) is determined and provided with the sensor data, characterized in that, for determining the operating information, at least one additional information describing the current detection situation of the rated environmental sensor (3, 4, 15, 16, 17) beyond the detection capabilities of the rated environmental sensor (3, 4, 15, 16, 17) is provided by at least one sensor-external additional information source (1), wherein the at least one rated environmental sensor (3, 4, 15, 16, 17) determine the operational information using the additional information itself.

Description

[0001] The invention relates to a method for operating environmental sensors in a motor vehicle, wherein, for at least some of the environmental sensors, operating information describing the availability of the respective evaluated environmental sensor and / or the reliability of the sensor data of the respective evaluated environmental sensor is determined and provided with the sensor data. The invention also relates to a motor vehicle.

[0002] Modern vehicles utilize various environmental sensors to gather information about their surroundings, such as the presence or absence of dynamic and / or static objects and / or other situational conditions, like road surface conditions. Examples of environmental sensors include radar sensors, lidar sensors, cameras, ultrasonic sensors, and similar devices.

[0003] The sensor data acquired from environmental sensors can be evaluated and processed by various functions of vehicle systems, particularly driver assistance systems. Safety systems and automated or semi-automated driving functions, especially with regard to autonomous driving, are becoming increasingly relevant in current developments. In this context, ever higher demands are being placed on the reliability of the sensor data and the reliability of the sensor data itself, as errors must be avoided or eliminated as far as possible. Therefore, prior art has already proposed that environmental sensors provide not only their current sensor data but also current operating information specifically linked to this sensor data.The operational information should describe the availability, specifically the scope of availability, and / or the reliability of the sensor data based on the current acquisition conditions. For example, availability information in such operational information can relate to the current detection range of the environmental sensor; for instance, in the case of a radar sensor, the range up to which relevant environmental objects can still be detected. Reliability information in such operational information can relate to error values ​​or similar parameters, such as standard deviations or uncertainties.

[0004] The operational information associated with the sensor data, which is then passed on to downstream functions, is determined within the environmental sensor itself. For example, radar data can be used to determine the current detection range, such as the minimum range. Since sensor data is typically pre-processed within the environmental sensors themselves, errors apparent during measurement can be traced using error analysis or similar methods to determine reliability information. Certain situational detection errors can also manifest as specific, recognizable artifacts in the sensor data.

[0005] However, the problem here is that inherent weaknesses in the measurement principle exist in the detection of situations, leading to a reduction in the availability or quality of the sensor data. For example, a radar sensor usually cannot reliably detect heavy rain, but experiences a significant reduction in range.

[0006] DE 10 2017 101 505 A1 concerns a method for operating railway vehicles. This method uses route- or position-related information to optimize a control variable, incorporating not only route-related information but also non-route-related information, such as information characterizing the railway vehicle itself. This approach is to be integrated into a system network of the railway vehicle.

[0007] DE 10 2008 008 555 A1 relates to a method and a device for minimizing hazardous situations in vehicles, comprising a system that detects the vehicle's surroundings and evaluates them with regard to potentially arising hazardous situations. This system can detect a hazardous situation based on an analysis of the surroundings, the vehicle, and / or the driver's situation, and upon detecting a hazardous situation in the vehicle, it can trigger driver assistance functions that partially access safety systems installed in the vehicle. A redundant design of the safety systems is to be implemented.

[0008] DE 10 2007 041 121 A1 relates to a method and a device for processing sensor data for a driver assistance system of a vehicle, in which a first sensor acquires a measured value and transmits it to a processing unit, whereby an error value is assigned to the measured value. A second sensor acquires a measured value of a quantity that cannot be measured with the first sensor, wherein the error value belonging to the measured value of the first sensor is adjusted by means of the measured value of the second sensor.

[0009] The invention is based on the objective of providing a way to compensate for weaknesses inherent in the measurement principle of reliability and / or availability information determined within an environmental sensor.

[0010] To solve this problem, a method of the type mentioned at the outset provides that, for determining the operating information, at least one additional piece of information describing the current detection situation of the evaluated environmental sensor beyond the detection capabilities of the evaluated environmental sensor is provided by at least one additional information source external to the sensor, in particular the evaluated environmental sensor itself which determines the operating information.

[0011] It is therefore an idea of ​​the present invention to selectively combine information from diverse environmental sensors and data sources within the vehicle's system network using a means that determines operating information for an evaluated or to-be-evaluated environmental sensor, in order to close existing detection gaps and to improve the statements of the determining means regarding the availability and reliability of the evaluated environmental sensors based on the sensor's own basic information and the additional information. The environmental sensor itself is advantageously provided as the determining means, so that in this case the environmental sensor evaluated (by the operating information) can also be referred to as the "determining environmental sensor".In other words, environmental sensors and / or, where applicable, additional information sources mutually support each other with up-to-date knowledge, which improves the operational information in the respective environmental sensors. The additional information thus complements the basic information already available within the environmental sensor, and therefore goes beyond the detection capabilities of the environmental sensor itself.

[0012] This approach offers numerous advantages. Firstly, existing weaknesses of the respective physical measurement principles, particularly regarding the assessment of their availability and / or reliability, can be at least partially mitigated. This results in an improvement in the self-assessment of individual environmental sensors. However, relevant sensor know-how remains primarily within the respective environmental sensor; this means that the interpretation of the provided additional information, especially in conjunction with basic information acquired within the environmental sensor itself, is performed decentrally at the respective location.While it would in principle also be conceivable to determine operating information for several environmental sensors centrally externally to the environmental sensor, this is less preferred according to the invention, since the specific effects of a circumstance described by additional information on the evaluated environmental sensor can best be assessed locally there.

[0013] At least one of the additional information sources can be another environmental sensor and / or a communication device that receives the additional information from outside the vehicle, and / or a navigation device that determines the additional information from a digital map. This means that for each environmental sensor that is always or optionally available in the vehicle's system, it can be defined which additional information, potentially relevant to other environmental sensors, it can provide. The environmental sensors themselves can complement each other particularly well. However, additional information sources can also be other devices, such as communication devices and / or navigation devices.This means that digital maps (online or offline) and / or cloud / backend and / or swarm data services can ultimately also be used as a kind of "environmental sensor," as they too can provide current supplementary information describing the measurement or recording conditions. A communication device in this context could be, for example, a vehicle-to-vehicle (C2C) communication device, whereby environmental sensors from other vehicles can also act as supplementary information sources. However, a communication device could also be a communication device of the vehicle itself that allows access to the internet, for example via a mobile network, in order to retrieve useful, current supplementary information from the internet.

[0014] As already mentioned, it is preferable for environmental sensors, especially those with different measurement principles, to provide each other with additional information within the vehicle. This means that, of course, an environmental sensor that itself determines and evaluates operating information can also act as a source of additional information for another environmental sensor that determines and evaluates operating information. However, it is also particularly advantageous if the other sensor providing the additional information for a determining environmental sensor uses a different physical measurement principle than the environmental sensor that determines the operating information using the additional information.

[0015] Advantageously, the distribution of additional information within the vehicle can be described using an allocation table. Such an allocation table can be understood as a kind of "subscription," meaning that each environmental sensor that determines its own operating information can subscribe to relevant additional information from another environmental sensor or other source of additional information in order to incorporate this additional information into the calculation of its own operating information (state information). Preferably, to determine the allocation table for each evaluated environmental sensor and each potential source of additional information, it can be determined, taking into account the physical measurement principle and / or the measurement characteristics of the evaluated environmental sensor, whether and what useful additional information the potential source of additional information can provide to the evaluating environmental sensor.Therefore, it is essential to assess, based on the physical measurement principle, which influencing factors, which the environmental sensor itself cannot detect, might have a relevant impact on the operational information. Then, for example, it can be determined which of these influencing factors, i.e., additional information, can best be obtained from which source, i.e., where it is measured, estimated, and / or retrieved. Based on these findings, the aforementioned allocation table can be described.

[0016] The additional information is preferably determined by the additional information source itself. This means, for example, that if another environmental sensor is used as the additional information source, it can be provided that this sensor evaluates its own sensor data to generate the additional information and, in particular, makes it directly accessible to at least one other environmental sensor for determining its operating information. In other words, if an environmental sensor is used as the additional information source, the additional information can be determined by evaluating its sensor data. For example, current weather conditions can be determined by evaluating the images (i.e., sensor data) from a camera, such as one mounted in a camera head, and provided as additional information to other environmental sensors.

[0017] As already mentioned, it can be provided that the additional information is transmitted from the additional information source directly or via a central intermediary device to the evaluated environmental sensor that itself determines the operational information. The intermediary device, for example a central control unit, can be involved in the process of determining the additional information; however, it is preferred if the determination of the additional information is carried out exclusively by the additional information source that provides it.

[0018] In this context, it should be noted that in both of the aforementioned configurations, it may be possible to standardize the additional information or to define a specific data format for the additional information in order to enable the exchange of information between sensors from different manufacturers or from different types of additional information sources in such a way that the additional information can also be understood by the determining environmental sensor.

[0019] Advantageously, the operating information can be determined even without additional information, at least when this additional information is missing. This means that each environmental sensor should always be able to independently estimate its own state and thus determine the operating information. This allows the operating information to be determined even if the additional information fails. Furthermore, situations may arise where it is expedient or even necessary that, at least for a predefined operating phase, the operating information is determined both with and without the additional information, with both sets of operating information being provided along with the sensor data.For example, depending on the allocation of fault budgets and / or due to a safety level decomposition, such as ASIL decomposition, it may be necessary to provide both "standalone" operating information (i.e., information determined without the additional information) and operating information determined with the additional information to enable diverse paths. The more complex the functions that use the sensor data are, or the higher the degree of automation within the vehicle, the more likely such a requirement becomes. In other words, this design means that, for example, errors in the additional information can also be detected through mutual plausibility checks, and so on.

[0020] Operational information can be determined, for example, as availability information, such as current detection range, particularly for predefined objects, and / or activity information, particularly binary, and / or reliability information, such as a quality measure and / or error measure assigned to the sensor data, particularly a standard deviation, and / or sensor data quality information. Additional information can be weather information, environmental information describing the surroundings, and / or illumination information describing the ambient light levels.

[0021] The application of the present invention to a current detection range as availability information is particularly preferred. Depending on the type of influence, for example, atmospheric disturbances due to heavy rain / fog, the range of an environmental sensor is reduced from an initial value to a lower value, with such a detection range typically referring to specific, predefined objects. For example, such predefined objects can be other road users, possibly also road users of a specific type. The detection range with reference to these predefined objects then describes the distance up to which the predefined objects can still be detected by the environmental sensor.In the case of a range reduction, an environmental sensor, using operational information, informs the sensor data users about which typical objects it could currently detect and at what distance. In a particularly preferred embodiment, such a range estimate can also be broken down, for example by determining detection ranges for different predefined objects or different predefined object groups.

[0022] In a specific application, for example, the range estimation of radar and / or lidar sensors can be supported by additional information on rain, snow, and fog, possibly including an intensity estimate of the weather phenomenon. A camera and / or a rain sensor can serve as an additional information source. Other additional information sources, such as weather and / or cloud services as external communication partners of a communication device, can of course also be used. For example, a radar sensor cannot reliably detect heavy rain itself, but can advantageously utilize corresponding information obtained through image analysis of sensor data from a camera. Naturally, other applications within the scope of the present invention are also conceivable.

[0023] For example, angle error estimation in radar sensors can be improved by using information about tunnel conditions based on digital maps and / or swarm data. Another example in this context is additional information about snow dust in the air, as a radar sensor, even with integrated heating, then has to cope with slush on its surface, which can also reduce the accuracy of angle measurements.

[0024] For lidar sensors, additional information about the sun's position can help assess glare effects. Additional information about weather conditions, i.e., weather information, is relevant for a significantly larger number of environmental sensors. This weather information can also include the state of the atmosphere (current attenuation of different wavelengths). Regarding weather information, it should be noted that lidar sensors are typically particularly sensitive to fog and fine raindrops, similar to cameras, while radar sensors typically react more to large raindrops and snow, resulting in reduced availability or quality of the sensor data.

[0025] In another specific embodiment, the inventive method also enables pre-control of, for example, cameras as environmental sensors, in particular with regard to sensitivity and / or brightness and / or aperture control, when other environmental sensors and / or digital maps indicate that a tunnel entrance or exit is imminent.

[0026] This opens up a multitude of possibilities for utilizing additional information within environmental sensors that collect operational data, particularly to provide improved availability and / or reliability information. It should be noted here that availability information can also include binary activity information. For example, certain detection conditions can completely prevent an environmental sensor from measuring, such as extreme glare, contamination, or similar factors. Such availability information would be a binary signal.

[0027] In addition to the method, the present invention also relates to a motor vehicle comprising several environmental sensors and a computing system configured to carry out the method according to the invention. At least some of the environmental sensors can also form part of the computing system. In particular, it is possible for the computing system to be comprised entirely of environmental sensors. In other embodiments, however, it is also possible for the computing system to include at least one control unit of the motor vehicle and / or at least one additional data source other than an environmental sensor, for example, a communication device and / or a navigation device. All aspects relating to the method according to the invention can be applied analogously to the motor vehicle according to the invention, with which the aforementioned advantages can therefore also be obtained.

[0028] Further advantages and details of the present invention will become apparent from the exemplary embodiments described below and from the drawings. These show: Fig. 1 a flowchart of an embodiment of the method according to the invention, Fig. 2 a possible recording situation, and Fig. 3 a second recording situation to explain the method according to the invention, Fig. 4 a schematic diagram of a motor vehicle according to the invention.

[0029] Fig. Figure 1 shows a flowchart of an embodiment of a method according to the invention for operating environmental sensors in a motor vehicle. Steps S1a and S1b describe the usual operation of two environmental sensors for recording sensor data according to their respective physical measurement principles. As is generally known in the prior art, operating information for each environmental sensor is to be assigned to this measurement data. In this case, this operating information is to include availability information, for example, an availability measure, and reliability information, for example, a standard deviation and / or another error value. Since the operating information represents a type of evaluation, these environmental sensors can also be referred to as evaluated or to-be-evaluated environmental sensors.The availability information includes, as an availability measure, at least a current detection range, possibly broken down by different types of predefined objects in the vicinity of the motor vehicle that can still be detected up to this detection range.

[0030] In the procedure described here, it is provided that environmental sensors support each other with additional information, and that other additional information sources, only indicated here, can be used to provide useful additional information for determining the operational information, which goes beyond the recording capabilities of the evaluated environmental sensor that determines the operational information.

[0031] In prior art methods, the operating information was determined solely based on basic information already present in the environmental sensor itself. This basic information can include or be derived from sensor data of the environmental sensor, and of course, it can also contain or describe operating data, in particular current operating parameters of the environmental sensor, and / or operating characteristics. Taking into account the physical measurement principle of the respective environmental sensor intended to determine operating information, the procedure described here has already been developed to identify which additional information might be useful beyond the basic information and which additional information source—be it another environmental sensor and / or another additional information source—can determine and provide this additional information.These relationships were defined in an assignment table, so that each environmental sensor and each additional information source 1 knows which additional information, preferably within the additional information source 1 or the environmental sensor itself, is to be determined and to which communication partner, for example via a bus system, this additional information is to be provided, so that the corresponding determining environmental sensor can automatically determine its operating information from the basic information and the received assignment information.

[0032] Steps S2a and S2b indicate this determination of the additional information and, according to arrows 2, its corresponding exchange.

[0033] In steps S3a and S3b, the basic and supplementary information are used to determine the operational information. In steps S4a and S4b, the sensor data is then provided with the associated operational information, for example, to a downstream function of a vehicle system that uses the sensor data, such as a driver assistance system and / or safety system.

[0034] In this context, it should be noted that every environmental sensor capable of determining current operating information is also designed to determine this information autonomously. This means it can determine the operating information purely from the basic information, even without any additional information received. In some embodiments, it may also be advantageous to provide two sets of operating information: one determined solely from the basic information, and the other from the basic information and the additional information. This allows, for example, the creation of plausibility checks that also permit an assessment of the reliability of the additional information, which itself may be erroneous. In some embodiments, it may also be possible to derive operating information solely from the additional information, particularly as a further plausibility check tool.

[0035] Generally speaking, the additional information can include weather information, environmental information describing the surroundings, illumination information describing the ambient light levels, and other useful information describing the conditions for specific physical measurement principles. Operational information can include availability information, such as the current detection range, particularly for predefined objects, activity information (especially binary), and the like, as well as reliability information, such as a quality measure and / or error measure assigned to the sensor data.

[0036] Fig. Figure 2 shows a concrete example of how two different physical measurement principles can complement each other. A radar sensor 3 is shown as the first environmental sensor, and a camera 4 as the second environmental sensor. The basic detection range 5 of the radar sensor 3 is also indicated, as well as predefined objects 6 of a specific object class, for example, other road users, specifically other motor vehicles. The current detection range 19 is shown in Fig. 2 is only roughly indicated as a dashed line, with the boundary actually being more of an arc. It is currently raining heavily, as indicated by the raindrops 7. This significantly limits the detection range 19, which is to be determined as operational information, of the radar sensor 3, so that, for example, the more distant object 6 would no longer be detectable. However, the heavy rain is not detectable by the radar sensor 3, so that the basic information alone could potentially result in an unrealistically large detection range 19. The camera 4, however, can detect the heavy rain, so that corresponding additional information can be determined from its sensor data, i.e., images, through appropriate image processing, which in this case takes place within the camera 4.This additional information is provided to radar sensor 3 by camera 4, again indicated by arrow 2, so that a significantly improved estimation of the detection range 19 within radar sensor 3 can be made. It should be noted that corresponding weather information can also be obtained from other additional information sources, for example from a rain sensor and / or even from a weather service, which can be accessed, for example, via the vehicle's internet communication system.

[0037] Another concrete example is in Fig. Figure 3 is shown schematically. A motor vehicle 8 according to the invention is shown, approaching a tunnel entrance 10 of a tunnel 11 on a road 9. The motor vehicle 8 has a radar sensor 3 and a camera 4, both directed in the direction of travel of the motor vehicle 8. In addition to the detection range 19, an angular error, which increases significantly within a tunnel 11, is to be determined as operating information for the radar sensor 3. Furthermore, the detection characteristics of the camera 4 may be significantly limited in the darker tunnel, or difficulties may occur during the transition in the area of ​​the tunnel entrance 10, which are also to be represented by operating information.

[0038] To provide additional information about the upcoming tunnel 11 as environmental information, sensor data from camera 4 can be evaluated. Sensor data from radar sensor 3 can also provide an indication of the approaching tunnel 11 if its detection range is sufficient. However, additional information sources are also suitable in this case, which are not necessarily environmental sensors but function similarly with regard to providing additional information. One such source is the navigation system 12 of the vehicle 8, which can read from a digital map that the tunnel entrance 10 is approaching. Another additional information source could be a vehicle-to-vehicle communication device 13, through which corresponding information can be received from at least one other vehicle 14 that is already closer to the tunnel entrance 10. Swarm data collection is also possible here.This can also be done via a fixed server.

[0039] Fig. Figure 4 shows a schematic diagram of the motor vehicle 8 according to the invention. In addition to the camera 4 and the radar sensor 3, further radar sensors 15, a lidar sensor 16, and a rain sensor 17 are shown as environmental sensors. Their sensor data are used by at least one downstream vehicle system 18. Also shown as additional information sources are the navigation device 12 and the communication device 13, which can also enable communication, in particular via a mobile network, to the internet.

[0040] The additional information sources 1 and the environmental sensors form a calculation system for carrying out the method according to the invention.

Claims

[1] Method for operating environmental sensors (3, 4, 15, 16, 17) in a motor vehicle (8), wherein for at least some of the environmental sensors (3, 4, 15, 16, 17) operating information describing the availability of the respective evaluated environmental sensor (3, 4, 15, 16, 17) and / or the reliability of the sensor data of the respective evaluated environmental sensor (3, 4, 15, 16, 17) is determined and provided with the sensor data, characterized by , that to determine the operating information, at least one additional information describing the current detection situation of the evaluated environmental sensor (3, 4, 15, 16, 17) beyond the detection capabilities of the evaluated environmental sensor (3, 4, 15, 16, 17) is provided by at least one additional information source (1) external to the sensor, wherein the at least one evaluated environmental sensor (3, 4, 15, 16, 17) determines the operating information itself using the additional information. [2] Method according to claim 1, characterized by , that at least one of the at least one additional information source is another environmental sensor (3, 4, 15, 16, 17) and / or a communication device (13) that receives the additional information from outside the motor vehicle (8) and / or a navigation device (12) that determines the additional information from a digital map. [3] Method according to claim 1 or 2, characterized by , that the additional information is determined by the additional information source (1) itself. [4] Method according to claim 3, characterized by , that in the case of an environmental sensor (3, 4, 15, 16, 17) as an additional information source (1) the additional information is determined by evaluating its sensor data. [5] Method according to any of the preceding claims, characterized by, that the additional information is transmitted from the additional information source (1) directly or via a particularly central intermediate device to the evaluated environmental sensor (3, 4, 15, 16, 17) which determines the operational information itself. [6] Method according to any of the preceding claims, characterized by , that the operational information is determined without it, at least in the absence of additional information. [7] Method according to claim 6, characterized by , that at least for a predefined operating section, if additional information is available, the operating information is determined both with and without taking the additional information into account, whereby both operating information is provided with the sensor data. [8] Method according to any of the preceding claims, characterized by, that the operational information as availability information includes a current detection range (19), in particular for predefined objects (6), and / or in particular binary activity information and / or as reliability information a quality measure and / or error measure assigned to the sensor data, in particular a standard deviation and / or sensor data quality information. [9] Method according to any of the preceding claims, characterized by , that additional information such as weather information and / or environmental information describing the surroundings and / or illumination information describing the lighting of the surroundings is used. [10] Motor vehicle (8) comprising several environmental sensors (3, 4, 15, 16, 17) and a calculation system configured to carry out a method according to one of the preceding claims.

Images