Methods for operating a vehicle that is at least partially automated and vehicle

Abstract

The invention relates to a method for operating a vehicle (1) that is at least partially automated, wherein an in-vehicle computing unit determines a trajectory (2.1, 2.2) for the vehicle (1) by minimizing a cost function (3), the cost function (3) being composed of a weighted sum of several cost terms (4). The method according to the invention is characterized in that at least one separate cost term (4) for potholes (5) is included in the cost function (3).

Description

[0001] The invention relates to a method for operating an at least partially automated controllable vehicle according to the type defined in more detail by the preamble of claim 1 and to a correspondingly controllable vehicle according to the type defined in more detail in the preamble of claim 10.

[0002] Motion planning is an essential component of automated vehicles. These vehicles can be highly automated, fully automated, or even autonomous, particularly according to SAE Levels 3 to 5. An autonomous vehicle does not require the presence of a driver. To do so, the vehicles use environmental sensors to sense their surroundings and are thus able to generate a virtual representation of the environment. The vehicles predict the movements of surrounding agents, i.e., other road users such as pedestrians, cyclists, cars, and the like. Taking the destination into account, the computing unit responsible for controlling the vehicle is responsible for determining a travel trajectory to be selected by the vehicle. Motion planning therefore has a direct impact on the safety, efficiency, comfort, and other aspects of the automated vehicle.

[0003] A motion planning module is also called a “motion planner” and essentially consists of a cost function that represents a weighted sum of several cost terms. Where w i the weighting factor of the cost factor C i . The cost function could be: u1:T=argminu1:T∑wiCi

[0004] During movement planning, the goal is to determine those temporal inputs and 1:T to find that minimize the cost function. For u tIt is an action to be performed by the at least partially automated vehicle, also referred to as a control action, such as accelerating, decelerating, turning a specific steering angle, or the like. To calculate the cost terms, it may be necessary to convert such control actions into states. A state typically involves the position of the vehicle in the environment, for example, specified in a 2D coordinate system, the orientation of the vehicle relative to the environment, i.e., an angle specification, and a speed of travel. Using a mathematical model, control actions can then be converted into states. For this purpose, see, for example, https: / / arxiv.org / pdf / 2207.10422.pdf.

[0005] Typical cost terms of a vehicle that can be controlled at least partially automatically are driving efficiency, driving comfort, compliance with traffic regulations, compliance with safety limits, such as a necessary safety distance from other agents, or fuel consumption or vehicle load.

[0006] During optimization, the inputs and 1:T which minimize the cost function over the horizon T. An analytical solution to the problem is usually not possible. The weighting factors w i are either determined manually and adjusted accordingly or are determined data-driven in order to, for example, simulate human driving behavior as closely as possible.

[0007] At least partially automated vehicles, especially autonomous vehicles, must be able to handle obstacles appropriately. This means that obstacles must be avoided if contact with them could cause damage to the vehicle. There are also obstacles that can be touched or driven over without threatening to damage the vehicle. This is the case, for example, with potholes. Therefore, special measures must be taken to enable at least partially automated vehicles, which determine their driving trajectory based on minimizing a cost function, to handle potholes appropriately.

[0008] Algorithms for detecting potholes from a vehicle's sensor data are known from the state of the art. For example, see: https: / / towardsdatascience.com / building-a-realtime-pothole-detection-system-using-machine-learning-and-computer-vision-2e5fb2e5e746.

[0009] A method for determining a trajectory for an autonomously driving motor vehicle, as well as a control device and such a motor vehicle, are known, for example, from DE 10 2017 212 373 A1. This document describes the derivation of a control behavior of an autonomously controllable vehicle by minimizing a cost function. The autonomously controllable vehicle is also capable of avoiding potholes. This is possible by incorporating the load acting on the vehicle's occupants and the vehicle itself during travel into the cost function. Thus, the load increases when touching or driving over a pothole, which leads to an increase in the cost function. Trajectories that avoid the potholes are determined accordingly.However, the extent to which potholes affect the load must be approximated, and the estimated load can differ significantly from the actual load. This has a negative impact on trajectory planning.

[0010] Furthermore, DE 10 2021 133 744 A1 discloses methods for navigating an autonomous vehicle based on a perceived risk. Here, too, the trajectory of an autonomously controlled vehicle is determined by minimizing a cost function. A lateral distance to obstacles at the edge of the road can be included as a cost term in the cost function. Potholes can be located at the edge of the road, which accordingly lead to a reduction in the lateral distance. The cost function corresponds to the perceived risk as opposed to an objectively calculated safety.

[0011] Furthermore, DE 10 2022 112 748 A1 describes the use of arrival times and safety procedures in motion planning trajectories for autonomous vehicles.

[0012] Furthermore, WO 2021 / 127186 A1 describes methods and systems for surface modeling using so-called "ensemble machine learning prediction," trained with data derived from at least one external model. The publication describes the common procedure for training machine learning models. Using an untrained machine learning model, a target variable is determined, and a ground truth corresponding to the target variable is provided as a reference. The variables are incorporated into a cost function, which must be minimized by adjusting the parameters of the machine learning model. Once the cost function is minimized, the machine learning model is sufficiently trained.

[0013] The present invention is based on the object of specifying an improved method for operating an at least partially automated vehicle, with the aid of which it is possible to determine a trajectory for the vehicle which particularly reliably avoids touching or driving over potholes.

[0014] According to the invention, this object is achieved by a method for operating an at least partially automated controllable vehicle having the features of claim 1. Advantageous embodiments and further developments as well as a correspondingly controllable vehicle emerge from the dependent claims.

[0015] A generic method for operating an at least partially automated vehicle, wherein an internal vehicle processing unit determines a trajectory for the vehicle by minimizing a cost function, wherein the cost function is composed of a weighted sum of several cost terms, is further developed according to the invention in that at least one separate cost term for potholes is included in the cost function.

[0016] According to the invention, potholes are thus directly considered in the cost function, in contrast to solutions known from the prior art, in which potholes are either not considered at all or only indirectly, for example by deteriorating another cost term, such as a load-related degradation of driving comfort. This allows for a more differentiated control of the influence of potholes on the determination of the trajectory for the at least partially automated vehicle, which ultimately makes it possible to determine trajectories that reliably avoid potholes. In contrast to an indirect consideration of potholes in the cost function by deteriorating another cost term, it is thus possible to understand the influence that the existence of potholes in the vehicle's surroundings has on trajectory planning.

[0017] In general, it is possible to insert a single cost term into the cost function that takes into account all potholes in the vehicle's vicinity, or an individual cost term could be included in the cost function for each pothole.

[0018] The vehicle can be controlled at least partially automatically. This means that the vehicle is capable of independently assuming longitudinal and / or lateral control, at least in certain driving situations. In particular, the vehicle can be highly or even fully automated. In a particularly advantageous embodiment, the vehicle can also be controlled autonomously, so that no person driving the vehicle needs to be present to monitor the vehicle's behavior.

[0019] An advantageous development of the method according to the invention provides that the vehicle detects potholes using environmental sensors and locates them relative to the vehicle, the cost term for the potholes comprises at least one cost factor, and the current distance between the vehicle and the nearest pothole at a given time is used to determine the cost factor. As already mentioned at the beginning, sensor-based detection of potholes is generally known from the prior art. Vehicles can have a wide variety of environmental sensors, such as mono and / or stereo cameras, laser scanners such as a LiDAR, ultrasonic sensors, radar sensors, and the like. Such sensors can be used to obtain depth information and thus detect static and dynamic objects in the environment. Furthermore, the environment can be measured, which makes it possible to create a virtual representation of the environment.In particular, computer-aided analysis of camera images, also known as "computer vision," allows a differentiated assessment of the environment. For example, potholes can be detected based on characteristic visual features in camera images. Due to the known installation position of the sensors on the vehicle, the potholes can then be located relative to the vehicle, and the distance between the potholes and the vehicle can be calculated. Corresponding algorithms for pothole detection can also be AI-based, for example using deep learning. The computing unit takes into account the current distance of the potholes from the vehicle to determine the cost factor. Therefore, if the vehicle moves further during a time increment, the distance to the respective potholes also changes.

[0020] The distance to a particular pothole can be incorporated into the cost function in various ways. For example, the distance can be subjected to any mathematical operation to be considered as a cost factor in the cost function. In the simplest case, the distance can be incorporated directly into the cost function, for example, as a distance specification in meters or millimeters. The distance could also be dedimensionalized.

[0021] Preferably, the computing unit enters a bounding box for the vehicle and a bounding box for each pothole in a 2D coordinate system representing the environment from a bird's eye view, determines the Euclidean distance between the bounding box of the vehicle and the bounding box of the respective pothole for each pothole, and uses the shortest Euclidean distance to determine the cost factor. The Euclidean distance between two objects is the shortest distance, i.e. a direct connecting line between the respective objects. Determining the Euclidean distance between the vehicle and the respective potholes is particularly easy if the corresponding objects are entered into the said 2D coordinate system of the environment. The bounding box can correspond to the actual perimeter of the vehicle's border or the respective potholes.This requires particularly precise measurement of the surroundings, but also allows for the determination of a particularly precise distance measure. The cost function thus includes the distance of the vehicle to the pothole closest to the vehicle. This allows the influence of all potholes to be considered in a particularly clever and simple way, since only the most relevant pothole is relevant at any given time and is considered accordingly for trajectory planning. Furthermore, the "relevant" pothole changes automatically as soon as another pothole is closer to the vehicle.

[0022] In particular, the 2D coordinate system is the one used to convert the vehicle's control actions into states. This facilitates the performance of mathematical operations because consistent coordinate information is available.

[0023] Preferably, each bounding box is idealized as a cuboid and / or the origin of the 2D coordinate system is fixed at the center of the bounding box of the vehicle.

[0024] The bounding box is also called the "bounding box." By idealizing the bounding boxes of the vehicle and / or potholes as a cuboid, or in a 2D view as a rectangle, the requirements for the detection accuracy of the environmental sensors are reduced. Furthermore, the computational effort required to calculate the representation of the environment is reduced.

[0025] Advantageously, the 2D coordinate system also moves with the vehicle, further facilitating the calculation of the vehicle's behavior relative to its surroundings. The vehicle thus considers the environment from an egocentric perspective.

[0026] According to a further advantageous embodiment of the method according to the invention, a consideration distance for potholes is defined, and the computing unit only considers the cost factor for the pothole in the cost function if the distance between the vehicle and the nearest pothole is smaller than the consideration distance. The computing unit determines the cost factor, in particular, by subtracting the distance from the consideration distance. This makes it possible to exclude potholes that are farther from the vehicle than the consideration distance as an influencing factor for trajectory determination. The consideration distance can assume a suitable value depending on the situation, such as 0.2 m.

[0027] The cost factor for potholes is then calculated particularly advantageously according to the formula: Cost factor = consideration distance − current distance of the vehicle to the nearest pothole.

[0028] This means that at a high distance, the cost factor assumes a low value and thus has little influence on the trajectory planning, and as the distance of the vehicle to the nearest pothole decreases, it increases, whereby the influence on the trajectory calculation also increases.

[0029] A further advantageous embodiment of the method according to the invention further provides that the cost factor for a respective pothole is determined at at least two time steps, and the computing unit considers a cost factor averaged from these cost factors in the cost function. The time window considered for this purpose can be of any size and, for example, can be fixed or can be longer or shorter depending on the situation. For example, the time window can correspond to a few milliseconds or seconds, or even fractions or multiples thereof.

[0030] According to a further advantageous embodiment of the method according to the invention, the cost term for the potholes comprises at least one weighting factor, which is multiplied by the cost factor. The weighting factor can be used to adjust the extent to which potholes affect trajectory determination. The weighting factor can be determined experimentally by a team of experts or can be data-driven. In particular, the weighting factor can also be variable and thus change depending on the situation. For example, the weighting factor can also depend on another variable, such as the vehicle's speed. For example, the weighting factor could also increase with increasing driving speed.

[0031] In the simplest case, the weighting factor is "1." Thus, the cost factor is included in the cost function unchanged.

[0032] A further advantageous embodiment of the method according to the invention further provides that the cost term for potholes takes into account the depth of each pothole. While driving over a relatively shallow pothole can be carried out without any problems, the comfort for the vehicle occupants as well as the potential for damage to the vehicle or a vehicle component, such as the wheel suspension, can increase when driving over a deep pothole. Taking the pothole depth into account, even more suitable trajectories for guiding the vehicle can be determined.

[0033] Without taking the pothole depth into account, for example, a trajectory could be determined that would inevitably bypass the pothole. This could lead to sudden steering maneuvers or sudden braking and / or acceleration, which would reduce driving comfort and place stress on the vehicle structure. However, if the pothole in question is shallow, driving comfort and vehicle load would be less affected if the driving trajectory were to pass through the pothole instead of avoiding it. Accordingly, driving through a particularly deep pothole could have serious consequences, i.e., a sudden avoidance of the pothole, so that in such a case, the avoidance trajectory would still be preferred.

[0034] According to a further advantageous embodiment of the method according to the invention, in order to take the depth of a respective pothole into account in the cost term for each pothole, the computing unit subtracts the depth of the pothole, particularly in the form of a weighted depth, from the respective Euclidean distance of the respective pothole to the vehicle. This allows the pothole depth to be taken into account in the cost function in a mathematically particularly clever and thus efficient way. By selecting an individual weighting factor for the depth of the potholes, the extent of the pothole depth's influence on the cost function can also be adjusted. Here, too, the weighting factor can be determined manually by a team of experts or derived data-driven.

[0035] In a vehicle comprising environmental sensors and a computing unit configured to determine control commands taking into account a planned trajectory, the environmental sensors and the computing unit are configured, according to the invention, to execute a method described above. The vehicle can be any road vehicle, such as a car, truck, van, bus, or the like. The vehicle can have a wide variety of environmental sensors, such as the aforementioned mono or stereo cameras, LiDARS, ultrasonic sensors, radar sensors, and the like. The computing unit can be a central on-board computer or a plurality of control units of vehicle subsystems that interact as a computing unit.For example, sensor data can be evaluated by a first control unit, whereupon a second control unit calculates the trajectory, which is used by a third control unit to derive control commands for the at least partially automated controllable vehicle.

[0036] Further advantageous embodiments of the method according to the invention for operating an at least partially automated controllable vehicle and of the vehicle also emerge from the exemplary embodiments which are described in more detail below with reference to the figures.

[0037] Showing: Fig. 1 shows a schematic perspective view of a vehicle according to the invention while driving and the corresponding environmental representation generated by a computing unit of the vehicle; Fig. 2 a cost function used to determine a trajectory to be chosen; Fig. 3 two equations used to determine a separate cost term for potholes; Fig. 4 an equation for determining an average cost factor for potholes; and Fig. 5 two equations used to determine a cost term for potholes taking into account a pothole depth.

[0038] Fig. 1 shows a vehicle 1 according to the invention, which is driven at least partially automatically, in particular autonomously, on a road 11 toward a pothole 5. The vehicle 1 detects its surroundings using an environmental sensor system 6 and is thus able to detect the pothole 5 and locate it relative to the vehicle 1.

[0039] Fig. 1 also shows an environmental representation 12 generated by a computing unit (not shown in detail) of the vehicle 1 in a 2D coordinate system 7 from a bird's eye view. In the adjacent lane, there is a second pothole 5 located somewhat further away. The computing unit generates a boundary box 8.1 around the vehicle 1 and respective boundary boxes 8.2 around the potholes 5 and determines a distance between the boundary boxes 8.1 and 8.2. This distance is particularly advantageous if it is the Euclidean distance d. The pothole 5 located directly in front of the vehicle 1 is closer to the vehicle 1 than the pothole 5 in the adjacent lane, so that the computing unit determines the shortest Euclidean distance d. min of all recorded potholes 5 detected.

[0040] In addition, a first and second trajectory 2.1, 2.2 are plotted in the environment representation 12, which were calculated by the computing unit for controlling the vehicle 1. The first trajectory 2.1 is an original trajectory that leads through the pothole 5, and the second trajectory 2.2 is a trajectory that leads around the pothole 5 ahead.

[0041] It is sufficiently known for the movement planning of at least partially automated controllable vehicles 1 or autonomously controllable vehicles 1, the trajectory to be actually selected by the vehicle 1 by minimizing a Fig. 2. According to the invention, a separate cost term 4 for potholes 5 is included in this cost function 3. This allows potholes 5 to be directly considered in the cost function 3, which makes it particularly effective to avoid potholes 5. Thus, the computing unit would accordingly select the trajectory 2.2 for actually controlling the vehicle 1.

[0042] Cost function 3 comprises several cost terms 4 for a wide variety of variables, such as driving efficiency, driving comfort, compliance with traffic regulations, safety factors such as maintaining specified distances from surrounding agents, and the like. Each cost term 4 represents a product of a cost factor 4.1 and a weighting factor 4.2. In the simplest case, the weighting factor 4.2 can assume the value "1," so that the cost factor 4.1 is included "unweighted" in cost function 3. However, a larger or smaller number than "1" can also be selected as the weighting factor 4.2 for a respective cost factor 4.1.

[0043] During movement planning, the goal is to use the temporal inputs and 1:t to find that minimize the cost function.

[0044] According to the invention, the Fig. 3 shown cost factor 4.1 c pothole,tincluded in cost function 3 to directly represent potholes 5. Fig. Figure 3 shows a possible embodiment of how the cost factor 4.1 can be calculated. A consideration distance 9 can be specified, for example, 0.2 m, from which the distance of vehicle 1 to the nearest pothole 5 is subtracted. The cost factor 4.1 can optionally only be considered if the consideration distance 9 is exceeded or reached. Otherwise, i.e., if vehicle 1 is farther from the nearest pothole 5 than the consideration distance 9, the value "0" is selected for the cost factor 4.1.

[0045] In a preferred embodiment, the Euclidean distance d is chosen as the distance between the vehicle 1 and the respective potholes 5. In the equation, av represents t the boundary box 8.1 of the vehicle and potthole t jthe bounding box 8.2 of the respective pothole j at time t. This automatically determines the Euclidean distance d min to determine the cost factor 4.1, which represents the distance to the nearest pothole 5.

[0046] It is also possible to use a cost factor 4.1 instead of a current cost factor 4.1 in the equation Fig. 4 is used. For this purpose, the instantaneous cost factors 4.1 determined at the respective times t are summed and divided by the respective number of measurements.

[0047] Fig.Figure 5 illustrates an equation that allows a depth 10 of the respective potholes 5 to be taken into account in the cost function 3. The weighted depth 10 of the respective pothole 5 is subtracted from the Euclidean distance d. The depth 10 of the respective pothole j is multiplied by a separate weighting factor 13. This results in the distance: distance_offset (av t ,pothole t j ) receive.

[0048] Using the method according to the invention, potholes 5 can be considered particularly easily and directly as an influencing factor for determining corresponding trajectories 2.1, 2.2. By selecting clever weighting factors 4.2 and 13, respectively, the extent to which the depth 10 of a respective pothole 5 or the distance of the pothole 5 from the vehicle 1 affects the trajectory selection can be adjusted. This allows particularly appropriate trajectories to be determined for specific situations. Wear and tear on the vehicle components is reduced, since particularly deep potholes 5 are avoided. This shortens downtimes of the vehicle 1 and reduces maintenance costs. Due to the formulation as cost term 4, integration into existing algorithms for behavior planning is possible. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] DE 10 2017 212 373 A1

[0009] DE 10 2021 133 744 A1

[0010] DE 10 2022 112 748 A1

[0011] WO 2021 / 127186 A1

[0012] Cited non-patent literature

[0000] https: / / arxiv.org / pdf / 2207.10422.pdf

[0004] https: / / towardsdatascience.com / building-a-realtime-pothole-detection-system-using-machine-learning-and-computer-vision-2e5fb2e5e746

[0008]

Claims

[1] Method for operating an at least partially automated controllable vehicle (1), wherein an in-vehicle computing unit determines a trajectory (2.1, 2.2) for the vehicle (1) by minimizing a cost function (3), wherein the cost function (3) is composed of a weighted sum of several cost terms (4), characterized by that at least one separate cost term (4) for potholes (5) is included in the cost function (3). [2] Method according to claim 1, characterized by that the vehicle (1) detects potholes (5) by means of an environmental sensor system (6) and locates them relative to the vehicle (1), the cost term (4) for the potholes (5) comprises at least one cost factor (4.1) and the current distance between the vehicle (1) and the nearest pothole (5) at the respective time (t) is used to determine the cost factor (4.1). [3] Method according to claim 2, characterized bythat the computing unit enters a boundary box (8.1) for the vehicle (1) and a boundary box (8.2) for each pothole (5) in a 2D coordinate system (7) representing the environment from a bird's eye view, determines the Euclidean distance (d) between the boundary box (8.1) of the vehicle (1) and the boundary box (8.2) of the respective pothole (5) for each pothole (5) and calculates the shortest Euclidean distance (d min ) is used to determine the cost factor (4.1). [4] Method claim 3, characterized by that a respective boundary box (8.1, 8.2) is idealized as a cuboid and / or the origin of the 2D coordinate system (7) is fixed in the center of the boundary box (8.1) of the vehicle (1). [5] Method according to one of claims 2 to 4, characterized bythat a consideration distance (9) for potholes (5) is determined and the computing unit only takes the cost factor (4.1) for the pothole (5) into account in the cost function (3) if the distance between the vehicle (1) and the nearest pothole (5) is smaller than the consideration distance (9), wherein the computing unit determines the cost factor (4.1) in particular by subtracting the distance from the consideration distance (9). [6] Method according to one of claims 2 to 5, characterized by that the cost factor (4.1) for a respective pothole (5) is determined at least two time steps and the computing unit takes into account a cost factor (4.3) averaged from these cost factors (4.1) in the cost function (3). [7] Method according to one of claims 2 to 6, characterized bythat the cost term (4) for the potholes (5) comprises at least one weighting factor (4.2) which is multiplied by the cost factor (4.1). [8] Method according to one of claims 1 to 7, characterized by that the cost term (4) for potholes (5) takes into account a depth (10) of a respective pothole (5). [9] Method according to one of claims 3 to 7 and claim 8, characterized by in that the computing unit, in order to take into account the depth (10) of a respective pothole (5) in the cost term (4), subtracts the depth (10) of the pothole (5) from the Euclidean distance (d) for each pothole (5), in particular in the form of a weighted depth (10). [10] Vehicle (1), comprising an environmental sensor system (6) and a computing unit configured to determine control commands taking into account a planned trajectory (2.1, 2.2), characterized bythat the environmental sensor system (6) and the computing unit are set up to carry out a method according to one of claims 1 to 9.

Images