Vehicle assistance system

Abstract

Vehicle assistance system for at least partially autonomous control of a vehicle (2) with a vehicle computing unit which is configured to generate a model of the vehicle (2) in spatial relation to objects (1) detected in the sensor data on the basis of sensor data acquired by the vehicle (2) itself and / or transmitted to the vehicle (2), where the vehicle (2) is part of a vehicle fleet in a vehicle network, which further comprises an external server (4) that is in communication connection with fleet vehicles (2,3) of the vehicle fleet, wherein the fleet vehicles (2,3) are equipped with sensors for providing sensor data, comprising at least the position of the respective fleet vehicle (2,3) and objects (1) in its environment, wherein a server computing unit of the external server (4) is configured to fuse sensor data received from the fleet vehicles (2,3) in order to generate relative position data of fleet vehicles (2,3) and other objects (1) in the environment of the vehicle (2) based on the fused sensor data and to transmit this data to the vehicle (2). characterized by the fact that the vehicle computing unit is configured to determine the number of fleet vehicles (2,3) traveling along a planned route (8) of the semi- or fully autonomous vehicle (2) and to determine a confidence value for overall sensor coverage along the route (8) from their position and the detection ranges (20, 30) of their sensors; and to adjust the level of automation and / or the driving behavior of the vehicle (2) depending on the confidence value.

Description

[0001] The invention relates to a vehicle assistance system for at least partially autonomous control of a vehicle of the type defined in more detail in the preamble of claim 1.

[0002] A comparable vehicle assistance system is disclosed, for example, in DE 10 2018 210 226 A1. The vehicle assistance system described therein is located in a vehicle, which in this context is also referred to as an ego-vehicle. In this context, an ego-vehicle is understood to be a reference vehicle, for example, a vehicle that communicates and interacts with other vehicles. This ego-vehicle includes sensors for detecting its surroundings and a processing unit. This processing unit is configured to determine the relative positions of other vehicles in the vicinity of the ego-vehicle based on the sensor data and to generate a model of the vehicle in spatial relation to the other vehicles detected in the sensor data.

[0003] The disadvantage of such an assistance system is that the sensors of the self-driving vehicle have a comparatively short range, especially when adverse environmental conditions such as rain, snow, fog, or the like are present. In this case, the model used to control the self-driving vehicle, at least partially autonomously, is relatively limited in its spatial extent, which represents a significant drawback.

[0004] To overcome such a disadvantage, DE 10 2017 203 838 A1 discloses a method and a system for environmental perception. The document describes the perception of the environment by several vehicles in a fleet, which transmit their generated environmental data to a central computer for fusion. The central computer generates a comprehensive representation of the environment from the environmental data. This comprehensive representation is then transmitted back to the respective vehicle.

[0005] The object of the present invention is to provide an improved vehicle assistance system within the meaning of the preamble of claim 1, which avoids these disadvantages and is improved compared to the prior art.

[0006] According to the invention, this problem is solved by a vehicle assistance system with the features in claim 1, and in particular in the characterizing part of claim 1. Further advantageous embodiments of the vehicle assistance system are described in the dependent claims.

[0007] The vehicle assistance system, as configured according to the invention, takes advantage of the fact that vehicles are typically networked today, so that virtually every modern vehicle is part of a fleet within such a vehicle network. This vehicle network, which could also be described as a vehicle ecosystem, always includes, in addition to the vehicles in the fleet, at least one external server, which is typically operated at least indirectly by the vehicle manufacturer and is also referred to as the backend or vehicle backend. This external server communicates with the fleet vehicles.

[0008] For the purposes of this invention, a vehicle fleet is understood to be a group of vehicles that exchange data via a common standard. This could, for example, be the vehicles of a single manufacturer that communicate with each other and with an external server of the manufacturer. However, it is also conceivable that this could be the vehicles of a consortium that meet the requirements. The consortium could extend across several different vehicle manufacturers or could comprise a group of different vehicle manufacturers working together to improve road safety or similar objectives.

[0009] Each of the fleet vehicles is equipped with sensors to provide sensor data, which includes at least the position of the respective fleet vehicle and objects in its vicinity. A server processing unit on the vehicle-external server is configured to fuse the sensor data received from the fleet vehicles. Based on this fused data, relative position data of the fleet vehicles and other objects can be generated. In a particularly advantageous configuration, this can include other road users that are not part of the fleet of the ego-vehicle and / or static objects. Static objects would be, for example, buildings, traffic lights, road signs, or other obstacles that are not moving at the moment of detection and cannot be clearly identified as other road users such as vehicles or pedestrians.

[0010] The fused sensor data, which includes the relative positions of the fleet vehicles in relation to the ego vehicle and the positions of other objects detected by the fleet vehicles' sensors in the vicinity of these vehicles, is then transmitted to the ego vehicle. The ego vehicle can then generate a relatively accurate model of itself in spatial relation to other fleet vehicles and / or objects detected by them, based on a combination of the transmitted data and the data acquired by its own sensors. This enables a very long sensor range, resulting in a significant improvement, particularly in the partially or fully autonomous control of the ego vehicle.

[0011] Furthermore, the vehicle's computer unit is designed to determine the number of fleet vehicles traveling along a planned route, which may be known from the vehicle's navigation system, and to calculate a confidence value for overall sensor coverage along the route based on their positions and sensor detection ranges. This is advantageous for semi- or fully autonomous vehicles, allowing for an early assessment of the quality of the data available regarding overall sensor coverage.

[0012] Furthermore, the vehicle's computing unit is designed to adapt the driving behavior of the autonomous vehicle depending on the confidence level. This adaptation means, for example, that autonomous driving is not possible in certain areas or is only possible at reduced speed.

[0013] According to a highly advantageous embodiment of the vehicle assistance system according to the invention, the vehicle's computing unit can also be configured to directly receive sensor data from fleet vehicles located in the vicinity of the vehicle and incorporate it into the model. Fleet vehicles belonging to the same fleet as the ego vehicle can thus provide their data directly to the nearby ego vehicle without having to go through the external server. The ego vehicle then automatically incorporates this data into its model without requiring prior fusion by the external server.This can be used as an alternative or supplement and is particularly advantageous when the vehicles have a communication link with each other, but the communication link to the external server is not available or only available with poor quality and limited bandwidth due to weaknesses in the mobile network, adverse weather conditions or the like.

[0014] The sensor data can include the position of the vehicle transmitting the data on the one hand, and the objects detected by the fleet vehicle in its vicinity relative to its position on the other. With this specific content of the sensor data, a very simple and efficient transmission of this data is necessary, in which only the data required for the immediate purpose actually needs to be transmitted. This saves computing power in particular, since the data typically still needs to be transmitted in encrypted and anonymized form, so limiting the data to the absolutely necessary data offers a significant advantage in terms of communication overhead.

[0015] A further, exceptionally advantageous embodiment of the method according to the invention can further provide that the vehicle computing unit, and in particular in conjunction with the server computing unit, is configured to determine a preview horizon of the sensor data, including the fused sensor data received from the external server and / or the sensor data received from the fleet vehicles. Such a preview horizon of the total data available from the vehicle's own sensors and the sensors of other vehicles in the fleet enables a very good assessment of the situation within the model generated in the ego-vehicle and its representation, so that a very safe operation is possible through the accurate estimation of the horizon of available data.

[0016] According to a highly advantageous further development of this concept, the vehicle's computing unit, and especially in conjunction with the server computing unit, can be configured to identify sections with and without sensor coverage along the planned route. This allows for the relatively early development of a driving strategy, determining in which areas autonomous driving is efficiently possible and in which areas, due to the limited range of the vehicle's own sensors or the lack of fleet vehicles in these areas, fully autonomous operation is not possible or only possible to a limited extent, for example, with reduced speed.

[0017] In an advantageous embodiment, the vehicle assistance system according to the invention may also include a visualization device for displaying the model. In this display, the vehicle (i.e., the ego vehicle), fleet vehicles, and other objects (e.g., other vehicles and static objects) are marked differently, so that a person using the visualization device can easily distinguish their own vehicles, other fleet vehicles, other (non-fleet) vehicles, and static objects. This type of visualization allows the person driving the vehicle or riding as a passenger to gain a better understanding of how the vehicle assistance system functions.This increases confidence in the vehicle assistance system and allows a person, especially the driver, to better assess whether the system can continue to be used in the current traffic situation or whether they should take over the driving task themselves. Ultimately, this leads to increased safety.

[0018] This is particularly true if, according to an advantageous further development of this aspect of the vehicle assistance system according to the invention, the respective sensor detection areas of the depicted fleet vehicles, and thus also of the ego vehicle, which is part of the fleet, are displayed. This makes it very easy to assess which areas can be detected with regard to other road users or stationary objects, and which areas may not be detected or only with difficulty. This enables an improved assessment of the situation, so that, if necessary, the driving task can be taken over independently, which ultimately increases safety. Further advantageous embodiments of the vehicle assistance system according to the invention also result from the exemplary embodiment, which is described in more detail below with reference to the figures.

[0019] Today, many vehicles are interconnected, creating a vehicle network typically operated by a manufacturer or consortium. These vehicles, referred to as fleet vehicles, exchange data using a common standard. This allows for both direct data exchange between vehicles and exchange via an external server, the so-called vehicle backend. This backend receives data in real time, processes it with high computing power, and then sends it back to other vehicles, for example, to support them. This has a crucial impact on the performance of semi- or fully autonomous vehicles. In the following description, different terms will be used for the various types of vehicles.The first term is the so-called ego vehicle, which is depicted with cross-hatching in the diagrams and represents the user's own vehicle, forming the reference point for the described system. In addition, there are so-called fleet vehicles, which are depicted with hatching in the diagrams and belong to the same fleet, allowing them to exchange data with each other and / or with the external server. Among the other road users or objects in the vicinity of the ego vehicle and the fleet vehicles, vehicles that do not belong to the fleet are of particular interest here and are subsequently referred to as vehicles or external vehicles; these are depicted without hatching in the diagrams.

[0020] All of this will be described below using the characters.

[0021] This shows: Fig. 1 an exemplary representation of a process of the method according to the invention; Fig. 2 an exemplary representation of a horizon view as it can be visualized as a model in the ego vehicle; Fig. 3 an alternative representation analogous to the one in Fig. 2; and Fig. 4. A bird's-eye view of a planned route and the sensor coverage to be achieved along that route.

[0022] In the presentation of the Fig. Figure 1 shows various vehicles on a four-lane road with two directions of travel. Most of the vehicles are depicted without hatching; these are external vehicles that do not belong to the fleet of an ego-vehicle 2 depicted with cross-hatching. The four vehicles 1 on the far left are traveling downwards, while the vehicles in the two rightmost lanes are traveling upwards. In addition to the ego-vehicle 2 depicted with cross-hatching, the following is also shown in the illustration: Fig. 1. A hatched fleet vehicle 3 can be identified. The ego vehicle 2 and the fleet vehicle 3 now have their own sensors, which measure the environment of the respective vehicle 2, 3. They then create an environmental representation in their own coordinate system, which is shown here in the representation of the Fig. 1 in the middle, each with x, y accordingly drawn. As indicated by the connecting lines, different vehicles 1 within a detection range 20 of the sensors of the ego vehicle 2, which is shown in the representation of the Fig. Vehicle 1, shown on the right with a solid line, and fleet vehicle 3, whose detection range 30 is shown with a dashed line, are detected. All this data is then uploaded to a backend 4, which fuses the data accordingly to create a complete picture of the situation from the fused sensor data. In particular, the different coordinate systems of vehicles 2 and 3 are aligned so that the sensor data of fleet vehicle 3 can be used for ego vehicle 2 (and vice versa, since ego vehicle 2 is also part of the vehicle fleet). The aggregated and fused data is then processed by backend 4, as shown in the diagram. Fig. 1, which can be seen on the far right, is played back to the ego vehicle 2 and the fleet vehicle 3, so that these vehicles 2,3 or the vehicle computing units installed in these vehicles 2, 3 are able to provide the most accurate model possible, which can then be used for at least partially autonomous driving.

[0023] This model of vehicle 1, fleet vehicle 3, and the ego-vehicle 2, as well as other static objects in the environment (which will not be discussed further here), can now be visualized via the vehicle's computer unit in the ego-vehicle 2. An example of such a visualization, in the form of a so-called horizon view (i.e., a three-dimensional view looking towards the horizon), is shown in the representation of the Fig. 2. The horizon is marked with reference symbol 5, the ego vehicle with reference symbol 2, the two fleet vehicles present with reference symbol 3, and the other vehicles with reference symbol 1. The vehicles are moving towards the horizon 5 on a three-lane road 7. The representation can be designed in a model display on a display device 10 of the ego vehicle 2, particularly in color, so that, for example, the other vehicles 1 would be shown in gray, the fleet vehicles 3 in green, and the ego vehicle 2 in red.

[0024] A person using the Ego Vehicle 2 thus gains a good insight into the model underlying the current driving situation and can therefore assess whether the assistance system is able to master the situation or whether intervention by the person is necessary. This representation can be further improved by the fact that, as is the case in the representation of the Fig. 3. The respective detection areas 20 and 30 of the sensors, also referred to as sensor illumination areas, are shown. As shown in the illustration of the Fig. The detection area 20 of the ego vehicle 2 is shown as an oval with a solid line around it. The detection areas 30 of the two fleet vehicles 3 are each shown with dashed lines. Additionally, there may be stationary infrastructure, indicated here by a radio mast 6. This mast can also have a detection area 60, shown here with a dashed line, and can make the data it collects available to all other vehicles 2, 3 in the fleet via the backend 4, similar to the data collected by the fleet vehicles 3.

[0025] In particular, such a visualization, which not only visualizes a model of the current situation but also the underlying detection areas 20, 30 of the sensors of the ego vehicle 2 and the participating fleet vehicles 3, can strengthen confidence and assessment in the performance of the vehicle assistance system.

[0026] A further improvement in the presentation of system capabilities can be achieved, for example, by a presentation analogous to the one in Fig. The bird's-eye view shown in Figure 4 illustrates this. A planned route 8 of the ego vehicle is depicted along a winding road. On this planned route 8 are two fleet vehicles 3, the middle one traveling in the same direction as the ego vehicle 2 and the rightmost one in the opposite direction. Several other vehicles 1 are also visible. From the data obtained via the respective detection areas 20 and 30 of the sensors of the ego vehicle 2 and the sensors of the fleet vehicles 3, it is now possible to estimate along the planned route 8 which areas are covered by the sensors and which sections remain without sensor coverage. In the representation of the Fig.Section 4 of the planned route is shown with sensor coverage by the detection areas 20 and 30 of the ego vehicle 2 (solid line) and the fleet vehicles 3 (dashed line). Areas between these are cross-sharpened on the road. These indicate sections without sensor coverage. Based on this information, the journey along the planned route 8 can now be planned very effectively.

Claims

[1] Vehicle assistance system for at least partially autonomous control of a vehicle (2) with a vehicle computing unit which is configured to generate a model of the vehicle (2) in spatial relation to objects (1) detected in the sensor data on the basis of sensor data acquired by the vehicle (2) itself and / or transmitted to the vehicle (2), where the vehicle (2) is part of a vehicle fleet in a vehicle network, which further comprises an external server (4) that is in communication connection with fleet vehicles (2,3) of the vehicle fleet, wherein the fleet vehicles (2,3) are equipped with sensors for providing sensor data, comprising at least the position of the respective fleet vehicle (2,3) and objects (1) in its environment, wherein a server computing unit of the external server (4) is configured to fuse sensor data received from the fleet vehicles (2,3) in order to generate relative position data of fleet vehicles (2,3) and other objects (1) in the environment of the vehicle (2) based on the fused sensor data and to transmit this data to the vehicle (2). characterized by , that the vehicle computing unit is configured to determine the number of fleet vehicles (2,3) traveling along a planned route (8) of the semi- or fully autonomous vehicle (2) and to determine a confidence value for overall sensor coverage along the route (8) from their position and the detection ranges (20, 30) of their sensors; and to adjust the level of automation and / or the driving behavior of the vehicle (2) depending on the confidence value. [2] Vehicle assistance system according to claim 1, characterized by , that the vehicle computing unit is configured to directly receive sensor data from fleet vehicles (2,3) located in the vicinity of the vehicle (2) and to incorporate this data into the model. [3] Vehicle assistance system according to claim 1 or 2, characterized by, that the sensor data include the position of the fleet vehicle (2,3) transmitting the sensor data and the objects (1) detected by the fleet vehicle (2,3) in the environment relative to its position. [4] Vehicle assistance system according to claim 1, 2 or 3, characterized by , that the objects (1) include other road users and / or static objects. [5] Vehicle assistance system according to one of claims 1 to 4, characterized by , that the vehicle computing unit, in particular in conjunction with the server computing unit, is configured to determine a preview horizon of the sensor data including the fused sensor data received from the vehicle-external server (4) and / or the sensor data received directly from the fleet vehicles (2,3). [6] Vehicle assistance system according to any one of claims 1 to 5, characterized by, that the vehicle computing unit, in particular in conjunction with the server computing unit, is configured to identify sections with and sections without sensor coverage along the planned route (8). [7] Vehicle assistance system according to any one of claims 1 to 6, characterized by , that a display (10) is provided for the presentation of the model, wherein in the presentation the vehicle (2), fleet vehicles (2,3) and other objects (1) are marked differently, wherein in particular static and dynamic objects are marked differently. [8] Vehicle assistance system according to claim 7, characterized by , that the representation includes the respective detection range (20, 30 ) of the sensors of the fleet vehicles shown (2,3).

Images