Method and device for recognizing the surroundings of a vehicle, and computer program

Abstract

The invention relates to a method for identifying the surroundings of a vehicle (100), the vehicle being equipped with at least one surroundings sensor (104) for providing information (106, 110) about features of the surroundings. First information (106) representing at least one feature of a first feature set is evaluated in order to identify whether a first situation of the vehicle is present. Furthermore, second information (110) representing at least one feature of a second feature set is evaluated in order to identify whether a second situation of the vehicle, which is different from the first situation, is present. The first feature set and the second feature set have no common features.

Description

Technical Field

[0001] The invention is based on a device or a method for recognizing the surroundings of a vehicle. The subject matter of the invention is also a computer program. Background Art

[0002] In the context of driver assistance systems and highly automated systems, the following question is often asked: how can it be ensured that a given situation or an aspect of this situation is correctly detected? A major problem here is that the evaluation device of the surrounding sensor system cannot fully and systematically interpret all available information.

[0003] In order to determine whether the vehicle is on a highway or on a road used by the authorization determination technical system, a map can be used, for example. However, this information is usually not of sufficient safety quality, so the information important for safety should be provided by the surrounding sensor system.

[0004] It is known to use redundancy to solve safety issues. Summary of the invention

[0005] Against this background, a method, a device and a corresponding computer program are proposed according to the technical solution of the present invention, which are used to recognize the surroundings of a vehicle.

[0006] Advantageous developments and improvements can be achieved by means of the measures implemented in the technical solution of the present invention.

[0007] A method for detecting the surroundings of a vehicle is proposed, wherein the vehicle is equipped with at least one surroundings sensor for providing information about characteristics of the surroundings, wherein the method comprises the following steps:

[0008] The first information is analyzed and processed to identify whether a first situation of the vehicle exists, and the first information represents at least one feature of a first feature set; the second information is analyzed and processed to identify whether a second situation of the vehicle different from the first situation exists, and the second information represents at least one feature of a second feature set; wherein the first feature set and the second feature set have no common features.

[0009] Environmental sensors can be understood, for example, as optical sensors, ultrasonic sensors, radar sensors or lidar sensors. The information can be information obtained using at least one environmental sensor. The information can be in the form of an electrical information signal. Here, the information can be read directly by an interface of at least one environmental sensor, or the information can be obtained using an environmental signal provided by at least one environmental sensor. Features (also called indicators) can be understood, for example, as traffic signs, lane markings, lane widths, pedestrians, traffic lights, another vehicle, greenery (Bepflanzung) or buildings or other parameters suitable for characterizing the vehicle situation. The first information can, for example, indicate that a certain feature of the first feature set exists or does not exist in the surrounding environment. Similarly, the second information can indicate that a certain feature of the first feature set exists or does not exist in the surrounding environment. Therefore, the feature can also be referred to as an indicator. The first situation and the second situation can, for example, be mutually exclusive situations. Therefore, the first situation can, for example, relate to a situation on a freeway, and the second situation can, for example, relate to a situation in a residential area (Ortschaft). The first information and the second information can relate to independent of each other. Provided information. Accordingly, the features of the first feature set and the features of the second feature set may be independent features, in which case, for example, the features of the second feature set may not simply be formal negations of the features of the first feature set, but rather features that are detected independently of the features of the first feature set and are positively identified. Conversely, this may also apply to the features of the first feature set relative to the features of the second feature set. The features of the first feature set and the features of the second feature set may, for example, have been detected by the same environmental sensor or by a plurality of environmental sensors of the same type.

[0010] According to one specific embodiment, if the evaluation reveals that the first situation exists and the second situation does not exist, a confirmation signal is provided which confirms that the first situation exists.

[0011] This is based on the knowledge that a safety-critical function for situation recognition can be implemented with the aid of a first module for positive recognition of a situation and a second module for negative recognition of a situation or for positive recognition of a non-situation. A consistency check can be performed based on this recognition. Advantageously, mutually independent criteria can be considered for situation recognition, the data of which can be provided, for example, by the same sensor or sensors of the same type, such as cameras. The approach proposed here can in particular adequately guarantee machine learning methods for situation recognition.

[0012] With the help of corresponding methods, environmental information can be guaranteed and provided with appropriate security quality. Here, the given information can be obtained in two ways. In the first way, a positive identification of the aspect to be evaluated can be performed. The first module that implements this positive identification can, for example, provide a yes / no evaluation of the situation in the sense of "yes means that the situation exists." Conversely, the second module can perform a positive identification of the opposite of this aspect, as long as the opposite is described well enough. For example, if the first model concludes that "the situation under consideration exists" and the second model concludes that "the opposite of the situation does not exist", it can be considered at the system level that: the situation exists. In order to make the independence between the first module and the second module as good as possible, and in order to make the redundancy effective fundamentally, it is important that the second module does not simply implement the algorithm of the first model in a reverse manner, but analyzes information that is different from the first model.

[0013] The advantages of this scheme are that environmental information can be used in a guaranteed manner, the method can be used for machine learning methods, and the independence proof between modules can be easily performed.

[0014] The method may include the step of detecting the first information and / or the second information using at least one surroundings sensor. Furthermore, in the step of providing a confirmation signal, the confirmation signal may be used by a driver assistance system of the vehicle to solve the driving task.

[0015] According to one embodiment, in the analysis step, first information may be analyzed, which represents a feature of the first feature set that is not obtained by negating a feature of the second feature set. Additionally or alternatively, second information may be analyzed, which represents a feature of the second feature set that is not obtained by negating a feature of the first feature set. This may avoid correlation between features of the first feature set and features of the second feature set.

[0016] According to another embodiment, it can be detected in the evaluation step whether a second situation exists which excludes the first situation. The first situation can thus be detected with high reliability.

[0017] Advantageously, in the step of analyzing, first information representing features of a first feature set is analyzed, the first feature set including as features: traffic signs, traffic sign colors, multiple lanes, lane widths, roadside buildings, isolated structural road guidance, or a combination of at least two of the above features. Additionally or alternatively, second information representing features of a second feature set can be analyzed, the second feature set including as features: cross traffic, pedestrians, another traffic sign different from the traffic sign, another traffic sign color different from the traffic sign color, intersection conditions, traffic lights, or a combination of at least two of the above features. As a result, the second situation can be identified with high reliability as a situation that excludes the first situation, or vice versa.

[0018] Furthermore, the first information can be evaluated in the evaluation step in order to detect the presence or absence of the first situation based on the number and / or frequency of features of the first feature set. Additionally or alternatively, the second information can be evaluated in order to detect the presence or absence of the second situation based on the number and / or frequency of features of the second feature set. This makes it possible to simply weight the features of the first feature set or the features of the second feature set.

[0019] It is also advantageous that in the filtering step, the first information is filtered in order to obtain filtered first information. Additionally or alternatively, the second information can be filtered in order to obtain filtered second information. Accordingly, in the evaluation step, the filtered first information can be analyzed in order to identify whether the first situation exists; or additionally or alternatively, the filtered second information can be analyzed in order to identify whether the second situation exists. For example, the filtered first information can represent a smaller number of features than the first information, and the filtered second information can represent a smaller number of features than the second information. This can reduce the error rate of the method.

[0020] According to one embodiment, the first information and / or the second information can be evaluated in a filtering step to remove features that are specific for multiple applications and / or ambiguous features. Independence between the features of the first feature set and the features of the second feature set can thereby be ensured.

[0021] The method can be implemented, for example, in software or in hardware or in a hybrid form of software and hardware (for example in a control unit).

[0022] The solution proposed here also provides an apparatus, which is designed to execute, control or implement the steps of the variant of the method proposed here in a corresponding device. Moreover, this embodiment variant of the invention in the form of an apparatus can quickly and efficiently solve the task on which the invention is based.

[0023] For this purpose, the device can have at least one computing unit for processing signals or data, at least one memory unit for storing signals or data, at least one interface to a sensor or actuator for reading a sensor signal from a sensor or for outputting a data signal or a control signal to an actuator, and / or at least one communication interface for reading or outputting data embedded in a communication protocol. The computing unit can be, for example, a signal processor, a microcontroller or the like, wherein the memory unit can be, for example, a flash memory, an EPROM or a magnetic storage unit. The communication interface can be designed for wireless and / or wired reading or output of data, wherein a communication interface that can read or output wired data can, for example, electrically or optically read these data from a corresponding data transmission line or output these data to a corresponding data transmission line.

[0024] Here, the device can be understood as an electrical device that processes sensor signals and outputs control signals and / or data signals according to the sensor signals. The device can have an interface that can be constructed in hardware form and / or software form. In the hardware form, the interface can be, for example, a part of a so-called system-specific integrated circuit, which contains various functions of the device. However, it is also possible to realize that the interface is an independent integrated circuit or is at least partially composed of discrete components. In the software form, the interface can be a software module, which, for example, exists on a microcontroller together with other software modules.

[0025] In an advantageous embodiment, the vehicle is controlled by the device. For this purpose, the device can, for example, detect sensor signals, such as acceleration signals, pressure signals, steering angle signals or environmental sensor signals. The control is achieved by actuators, such as brake actuators or steering actuators or motor control devices of the vehicle. In one embodiment, the method accordingly includes the step of controlling the vehicle based on the detected surroundings.

[0026] Also advantageous is a computer program product or a computer program with a program code, which can be stored on a machine-readable carrier or storage medium, such as a semiconductor memory, hard disk memory or optical memory, and is used to execute, implement and / or control the steps of the method according to any of the above-described embodiments, in particular when the program product or the program is executed on a computer or device. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Embodiments of the present invention are shown in the drawings and are further explained in the following description. The drawings show:

[0028] Figure 1 A schematic diagram showing a vehicle having a device according to one embodiment;

[0029] Figure 2 Show Figure 1 A schematic diagram of the device in FIG.

[0030] Figure 3 A flow chart of a method according to an exemplary embodiment is shown.

[0031] In the following description of advantageous exemplary embodiments of the invention, identical or similar reference numerals are used for elements which are shown in the various figures and which act similarly, wherein a repeated description of these elements is omitted. DETAILED DESCRIPTION

[0032] Figure 1 A schematic diagram of a vehicle 100 with a device 102 according to an exemplary embodiment is shown. Device 102 for detecting the surroundings of vehicle 100, for example a control device, is coupled to an environment sensor 104. Environment sensor 104, for example a camera, is designed to detect different features of the surroundings. In this case, environment sensor 104 provides first information 106 in the form of an electrical signal via an environment sensor interface, which first information represents at least one feature of a first feature set, the features of which are associated with a first situation of vehicle 100, for example a highway situation. By way of example, Figure 1 The first information 106 in the image represents the presence of a highway sign 108 in the surroundings of the vehicle 100, which has a typical sign color for highways, such as blue. In addition, the environmental sensor 104 provides second information 110 in the form of an electrical signal via the environmental sensor interface, which second information represents at least one feature of a second feature set, the features of which are assigned to a second situation of the vehicle 100 that is different from the first situation. The first feature set and the second feature set are elementfremd. It is particularly advantageous if the features of the second feature set are assigned to a second situation that is opposite to the first situation, such as a residential area of ​​the vehicle 100. situation. Accordingly, the second information 110 represents, for example, that there is no intersection situation in the surroundings of the vehicle 100. The device 102 receives these two types of information 106, 110 from the environmental sensor 104 and analyzes these information in order to identify whether the first situation exists and further identify whether the second situation exists. According to this embodiment, the device 102 recognizes the existence of the first situation based on the first information 106, that is, based on the detected highway sign 108, and recognizes the absence of the second situation based on the second information 110, that is, based on the lack of an intersection. Therefore, the positive recognition of the absence of the second situation is redundant, through which the existence of the first situation can be confirmed with high safety quality. As a result of the analysis, the device 102 provides a confirmation signal 112 confirming the existence of the first situation.

[0033] Depending on the exemplary embodiment, the first information 106 or the second information 110 represents a plurality of features of the feature set in question, wherein the features of the first feature set and the features of the second feature set always represent mutually independent features.

[0034] According to an alternative exemplary embodiment, different environmental sensors provide information 106 , 110 . These different environmental sensors may be arranged, for example, at different locations on vehicle 100 and / or have different sensor characteristics.

[0035] According to one exemplary embodiment, confirmation signal 112 is used by driver assistance system 114 in order to solve the driving task. According to one exemplary embodiment, device 102 is part of driver assistance system 114. According to one exemplary embodiment, driver assistance system 114 is designed to provide a control signal 116 using confirmation signal 112, which is used to control a functional unit 118 of vehicle 100, such as a drive, a brake or a steering device.

[0036] Figure 2 Show Figure 1 Schematic diagram of a device 102 in FIG. According to one embodiment, the device 102 is implemented as a hardware component or a part of a hardware component of a vehicle. The device 102 includes a first module 200 for processing a first information 106 and a second module 202 for processing a second information 110 independently of the first module 200. In this case, the first module 200 starts from an uncertain state x and processes the first information 106 according to the first indicator A1 ... A n At least one feature of the first feature set of an element of the form identifies whether the first situation exists. Similarly, the second module 202 starts from an uncertain state x' and determines whether the first situation exists according to the second indicator B1...B m At least one feature of the second feature set of the element of the form identifies whether the second situation exists. For example, a "yes" answer is given for the existence of the first situation or the second situation, and a "no" answer is given for the non-existence of the first situation or the second situation. Figure 2 As shown, if a “yes” is obtained in the evaluation of first information 106 and a “no” is obtained in the evaluation of second information 110 , a confirmation signal 112 is provided by provision unit 204 of device 102 , which indicates that the first situation is plausible.

[0037] Next, according to Figure 2The solution proposed here is described by taking the following question as an example: Is the vehicle on a highway? The first module 200 evaluates this question directly based on first indicators that point to the highway. Indicators associated with the highway situation are, for example, highway signs, multiple lanes, lane width, roadside buildings, isolated structural guides, or specific signs that only appear on highways.

[0038] Advantageously, first module 200 is designed to exclude certain information during situation detection by appropriate filtering, such as detected road signs indicating the end of a motorway or navigation information if these are systematically used elsewhere or impair the independence between two models 200 , 202 .

[0039] For example, by suitable selection of training data using a machine learning method, it is ensured that the first module 200 changes from an initially uncertain state x to "yes" only when the first indicator occurs sufficiently frequently. Depending on the application in which the method is used, it is considered whether uncertain states x, x' are introduced or always only one aspect is involved in each case - for example, in the first module 200 x = no, and in the second module 202 x' = yes.

[0040] The second module 202 evaluates the question "Is the system located somewhere else?", for example, based on a suitable second indicator. Such indicators are, for example, cross traffic, pedestrians, street signs present in cities, place name signs, speed limits up to 70 km / h, intersection conditions, street sign color "yellow" or traffic lights. It is important for establishing the expected independence that the second indicator does not merely represent a negation of the first indicator.

[0041] For example, if the first indicator A j Check the existence of highway signs, then any second indicator B i It does not just mean that the highway sign does not exist. Otherwise, in this case, the independence between the two modules 200, 202 is compromised or at least difficult to verify. That is, the analysis performed by the two modules 200, 202 should be a positive identification.

[0042] The solution proposed here aims to use the first indicator and the second indicator as positive confirmation of a feature or characteristic and not to use the first indicator and the second indicator as negative confirmation of the absence of a feature or characteristic, which is equivalent to negative identification. Independence in the evaluation of the indicators can thus be advantageously ensured.

[0043] Furthermore, this recognition method allows for simple verification and easy validation based on the independence of the indicators. For example, the feature combination "pedestrian detected" / "isolated structural guidance" can be analyzed, argued and validated in an intuitive manner with respect to common error causes by the surrounding sensor system. If the method is used together with a machine learning method, this can be ensured by selecting appropriate training data.

[0044] according to Figure 2 , a reliable state is identified only when the first module 200 detects that the system is located on a highway and the second module 202 detects that the system is not located elsewhere. Advantageously, it is relatively simple to improve both modules by gaining experience. In addition to improving indicators, new indicators can also be simply added to the feature set.

[0045] Figure 3 A flow chart of a method 300 according to an embodiment is shown. Figure 2 The method 300 for identifying the surroundings of a vehicle is carried out by the device described. In this case, in step 310, two pieces of information provided by the surroundings sensor are evaluated with respect to the detected features of the surroundings in order to detect whether a first situation exists or whether a second situation exists. In an optional further step 320, a confirmation signal confirming the existence of the first situation is provided, if the evaluation of the two pieces of information shows that the first situation exists and the second situation does not exist. The second situation here particularly refers to a situation that is opposite to the first situation or excludes the first situation.

[0046] According to one exemplary embodiment, in a preceding step 330 , information is detected using one or more environmental sensors.

[0047] If an exemplary embodiment includes an “and / or” relationship between a first feature and a second feature, this can be interpreted as: the exemplary embodiment has both the first feature and the second feature according to a first exemplary embodiment, and has either only the first feature or only the second feature according to another exemplary embodiment.

Claims

1. A method (300) for identifying the surroundings of a vehicle (100), wherein: The vehicle (100) is equipped with at least one environmental sensor (104) for providing information (106, 110) about characteristics of the surrounding environment, wherein the method (300) comprises the following steps: The first information (106) is analyzed and processed (310) to identify whether a first condition of the vehicle (100) exists, the first information representing a first feature set (A1 . . . A n ), at least one feature of the vehicle (100) obtained by using the at least one environmental sensor (104); analyzing (310) second information (110) in order to identify whether a second situation of the vehicle (100) different from the first situation exists, wherein the first situation is a highway situation and the second situation is a residential area situation, and the second information represents a second feature set (B1 ... B m ), which is determined using the at least one environmental sensor (104); wherein the first feature set (A1 . . . A n ) and the second feature set (B1...B m ) do not have common features, wherein the second case is a positive identification of the opposite of the first case, wherein the second feature set (B1...B m ) does not involve the first feature set (A1...A n ), but rather refers to features that are detected independently of the features of the first feature set and are positively identified, providing (320) a confirmation signal (112), which confirms the existence of the first condition if it is determined in the analysis (310) that the first condition exists and the second condition does not exist, A step of filtering the first information (106) to obtain filtered first information; and / or, a step of filtering the second information (110) to obtain filtered second information, wherein, in the analysis and processing (310) step, the filtered first information is analyzed and processed to identify whether a first situation exists; and / or, the filtered second information is analyzed and processed to identify whether a second situation exists, wherein in the filtering step, the first information and / or the second information is analyzed and processed to remove features that undermine the independence between features of the first feature set and the second feature set, wherein in the situation identification, the detected road sign indicating the end of the highway is removed by filtering.

2. The method (300) according to claim 1, wherein: In the providing (320) step, the confirmation signal (112) is used by a driver assistance system (114) of the vehicle (100) to solve a driving task.

3. The method (300) according to claim 1 or 2, comprising the step of detecting (330) the first information (106) and / or the second information (110) using the at least one environmental sensor (104).

4. The method (300) according to claim 1 or 2, wherein in the method, in the step of analyzing (310) first information (106) is analyzed, the first information representing the first feature set (A1 ... A n ) as follows: the feature is not negated by the second feature set (B1 ... B m ) features; and / or, analyzing and processing second information (110), wherein the second information represents the second feature set (B1 ... B m ):The feature is not negated by the first feature set (A1...A n ) characteristics are sought.

5. The method (300) according to claim 1 or 2, wherein in the method, it is identified in the analysis (310) step whether there is a second situation that excludes the first situation.

6. The method (300) according to claim 1 or 2, wherein in the method, in the step of analyzing (310) first information (106) is analyzed, the first information representing a first feature set (A1 ... A n ), wherein the first feature set includes the following as features: traffic signs (108) and / or traffic sign colors and / or multiple lanes and / or lane widths and / or roadside buildings and / or isolated structural road guidance; and / or, analyzing and processing second information (110), wherein the second information represents a second feature set (B1 ... B m ), the second feature set includes the following as features: lateral traffic and / or pedestrians and / or another traffic sign different from the traffic sign (108) and / or another traffic sign color different from the traffic sign color and / or intersection conditions and / or traffic lights.

7. The method (300) according to claim 1 or 2, wherein in the method, in the step of analyzing and processing (310), the first information (106) is analyzed and processed so as to obtain a first feature set (A1 ... A n ) to identify the presence or absence of the first situation by the number and / or frequency of the features of the second feature set (B1 ... B m )’s number and / or frequency of features to identify the presence or absence of the second situation.

8. A device (102) for identifying the surroundings of a vehicle (100), the device being constructed to implement and / or control a method (300) according to any one of the preceding claims, the device comprising: at least one computing unit for processing signals or data, at least one memory unit for storing signals or data, at least one interface to a sensor or actuator for reading a sensor signal from a sensor or for outputting a data signal or a control signal to an actuator, and / or at least one communication interface for reading or outputting data embedded in a communication protocol. 9 . A computer program product, which is designed to carry out and / or control the method ( 300 ) according to claim 1 .

10. A machine-readable storage medium having stored thereon a computer program product according to claim 9.

Images