Methods for operating a vehicle that is at least partially automated and vehicle

Abstract

Method for operating a vehicle (1) that is at least partially automated, wherein an in-vehicle computing unit determines a trajectory (2.1, 2.2) for the vehicle (1) by minimizing a cost function (3), wherein the cost function (3) is composed of a weighted sum of several cost terms (4), where at least one separate cost term (4) for potholes (5) is included in the cost function (3), characterized by the fact that The vehicle (1) detects potholes (5) using an environmental sensor (6) and locates them relative to the vehicle (1), the cost term (4) for the potholes (5) includes at least one cost factor (4.1) and the distance between the vehicle (1) and the nearest pothole (5) at the respective time (t) is used to determine the cost factor (4.1).

Description

[0001] The invention relates to a method for operating a vehicle that is at least partially automated and controllable according to the type defined in more detail by the preamble of claim 1, and to a correspondingly controllable vehicle according to the type defined in more detail by the preamble of claim 9.

[0002] Motion planning is an essential component of automated vehicles. These vehicles can be highly automated, fully automated, or even autonomously controlled, particularly according to SAE levels 3 to 5. An autonomously controlled vehicle does not require a driver to be present. To achieve this, the vehicles use environmental sensors to perceive their surroundings and are thus able to generate a virtual representation of the environment. The vehicles predict the movements of nearby agents, such as other road users like pedestrians, cyclists, cars, and the like. Taking the destination into account, the vehicle's control unit is responsible for determining a trajectory for the vehicle to choose. Motion planning therefore directly impacts the safety, efficiency, comfort, and other aspects of the automated vehicle.

[0003] A module for motion planning is also known as a "motion planner" and essentially consists of a cost function that represents a weighted sum of several cost terms. Here, w i the weighting factor of cost factor C i The cost function could be: u1:T=argminu1:T∑wiCi

[0004] During movement planning, the goal is to identify those temporal inputs and 1:T to find those that minimize the cost function. At u tThis refers to an action to be performed by a vehicle that is at least partially automated, also known as a control action, such as accelerating, braking, steering at a specific angle, or the like. To calculate the cost terms, it may be necessary to convert such control actions into states. A state typically consists of the vehicle's position in its environment, for example, specified in a 2D coordinate system, the vehicle's orientation relative to its environment (i.e., an angle), and its speed. Control actions can then be converted into states using a mathematical model. For example, see: https: / / arxiv.org / pdf / 2207.10422.pdf.

[0005] Typical cost terms of a vehicle that can be controlled at least partially automatically include driving efficiency, driving comfort, compliance with traffic rules, adherence to safety limits such as a necessary safety distance to other agents, or fuel consumption or vehicle load.

[0006] During optimization, the inputs u 1:T approximates which minimize the cost function over the horizon T. An analytical solution to the problem is usually not possible. The weighting factors w i These parameters are either determined manually and adjusted accordingly, or determined in a data-driven manner, for example to replicate human driving behavior as closely as possible.

[0007] Vehicles capable of at least partial automation, and especially autonomous vehicles, must be able to deal appropriately with obstacles. This means that obstacles must be avoided if contact with them would risk damaging the vehicle. There are also obstacles that can be touched or driven over without risk of damage. This is the case, for example, with potholes. Therefore, specific measures must be taken to enable vehicles capable of at least partial automation, which determine their trajectory based on minimizing a cost function, to deal appropriately with potholes.

[0008] Algorithms for detecting potholes from a vehicle's sensor data are known from the state of the art. For example, see: https: / / towardsdatascience.com / building-a-realtime-pothole-detection-system-using-machine-learning-and-computer-vision-2e5fb2e5e746.

[0009] A method for determining a trajectory for an autonomously driving vehicle, as well as for a control unit and such a vehicle, is known, for example, from DE 10 2017 212 373 A1. This publication describes deriving the control behavior of an autonomously controlled vehicle by minimizing a cost function. The autonomously controlled vehicle is also capable of avoiding potholes. This is possible by considering the stress exerted on the vehicle's occupants and the vehicle itself during travel in the cost function. The stress increases when the vehicle touches or drives over a pothole, leading to an increase in the cost function. Accordingly, trajectories are determined that lead around the potholes.However, it is necessary to approximate the extent to which individual potholes affect the load, and the estimated load can differ significantly from the actual load. This negatively impacts trajectory planning.

[0010] Furthermore, DE 10 2021 133 744 A1 discloses methods for navigating an autonomous vehicle based on perceived risk. Here, too, the trajectory of an autonomously controlled vehicle is determined by minimizing a cost function. The cost function can include a lateral distance to obstacles at the edge of the road. Potholes may be present at the road edge, which would correspondingly reduce this lateral distance. The cost function represents the perceived risk, as opposed to an objectively calculated safety factor.

[0011] Furthermore, DE 10 2022 112 748 A1 describes the use of arrival times and safety procedures in motion planning trajectories for autonomous vehicles.

[0012] Furthermore, WO 2021 / 127186 A1 describes methods and systems for surface modeling using so-called "Ensemble Machine Learning Prediction," trained with data derived from at least one external model. The publication describes the common procedure for training machine learning models. An untrained machine learning model is used to determine a target variable, and a corresponding basic truth is provided as a reference. These variables are incorporated into a cost function, which must be minimized by adjusting the parameters of the machine learning model. Once the cost function is minimized, the machine learning model is sufficiently trained.

[0013] Furthermore, DE 11 2021 004 953 T5 discloses an electronic control device for a vehicle. The control device is capable of generating automated control commands for the vehicle, taking potholes into account.

[0014] The present invention is based on the objective of providing an improved method for operating a vehicle that is at least partially automated and with the help of which it is possible to determine a trajectory for the vehicle which particularly reliably avoids touching or driving over potholes.

[0015] According to the invention, this problem is solved by a method for operating a vehicle that is at least partially automatically controllable and has the features of claim 1. Advantageous embodiments and further developments, as well as a correspondingly controllable vehicle, are described in the dependent claims.

[0016] A generic method for operating a vehicle that is at least partially automated, wherein an in-vehicle computing unit determines a trajectory for the vehicle by minimizing a cost function, the cost function being composed of a weighted sum of several cost terms, further provides that at least one separate cost term for potholes is included in the cost function. According to the invention, the vehicle detects potholes using environmental sensors and locates them relative to the vehicle, the cost term for the potholes includes at least one cost factor, and the current distance between the vehicle and the nearest pothole is used to determine the cost factor.

[0017] According to the invention, potholes are thus directly considered in the cost function, in contrast to prior art solutions where potholes are either not considered at all or only indirectly, for example, by worsening another cost term, such as a load-related degradation of ride comfort. This allows for a more differentiated control of the influence of potholes on determining the trajectory for the at least partially automated vehicle, ultimately making it possible to determine trajectories that reliably avoid potholes. Unlike indirectly considering potholes in the cost function by worsening another cost term, it is therefore possible to understand the impact of potholes in the vehicle's vicinity on trajectory planning.

[0018] In general, it is possible to insert a single cost term into the cost function that takes into account all potholes existing in the vicinity of the vehicle, or it could also be possible to include an individual cost term for each pothole in the cost function.

[0019] The vehicle is at least partially automated. This means that the vehicle is capable of independently taking over longitudinal and / or lateral control, at least in certain driving situations. In particular, the vehicle can be highly automated or even fully automated. In a particularly advantageous embodiment, the vehicle can also be autonomously controlled, so that no driver needs to be present to monitor the vehicle's behavior.

[0020] As already described, the invention provides that the vehicle detects potholes using environmental sensors and locates them relative to the vehicle. The cost term for the potholes includes at least one cost factor, and the current distance between the vehicle and the nearest pothole is used to determine this cost factor. Sensor-based pothole detection is, as mentioned at the outset, generally known from the prior art. Vehicles can be equipped with a wide variety of environmental sensors, such as mono and / or stereo cameras, laser scanners like LiDAR, ultrasonic sensors, radar sensors, and the like. Depth information can be obtained using such sensors, thus enabling the detection of static and dynamic environmental objects. Furthermore, the environment can be measured, which makes it possible to create a virtual representation of the environment.

[0021] In particular, computer-aided analysis of camera images, also known as "computer vision," allows for a differentiated assessment of the environment. For example, potholes can be detected based on characteristic visual features in camera images. Because the sensors are known to be mounted in the vehicle, the potholes can then be located relative to the vehicle, and thus the distance between the potholes and the vehicle can be calculated. Corresponding algorithms for pothole detection can also be AI-based, for example, using deep learning. In this case, the processing unit considers the current distance of the potholes from the vehicle to determine the cost factor. Therefore, if the vehicle moves further during a time increment, the distance to the respective potholes also changes.

[0022] The distance to a particular pothole can be incorporated into the cost function in various ways. The distance can be subjected to any mathematical operation to be included as a cost factor in the cost function. In the simplest case, the distance can be directly included in the cost function, for example, as a measurement in meters or millimeters. Alternatively, the distance could be represented as a flat shape.

[0023] Preferably, the processing unit plots a boundary box for the vehicle and a boundary box for each pothole in a 2D coordinate system representing the environment from a bird's-eye view. For each pothole, it determines the Euclidean distance between the vehicle's boundary box and the respective pothole's boundary box and uses the shortest Euclidean distance to determine the cost factor. The Euclidean distance between two objects is the shortest distance, i.e., a direct line connecting the respective objects. Determining the Euclidean distance between the vehicle and the respective potholes is particularly easy if the corresponding objects are plotted in the aforementioned 2D coordinate system of the environment. The boundary box can then correspond to the actual perimeter of the vehicle's outline or the respective potholes.This requires particularly precise surveying of the surroundings, but also allows for the determination of a highly accurate distance measurement. The cost function thus incorporates the distance of the vehicle to the pothole that is closest to it. This allows the influence of all potholes to be considered in a particularly clever and simple way, since only the most relevant pothole is important at any given time and is taken into account accordingly for trajectory planning. Furthermore, the "relevant" pothole changes automatically as soon as another pothole is closer to the vehicle.

[0024] In particular, the 2D coordinate system is the coordinate system used to convert vehicle control actions into states. This simplifies the execution of mathematical operations, as consistent coordinate data is then available.

[0025] Preferably, each boundary box is ideally designed as a rectangle and / or the origin of the 2D coordinate system is fixed at the center of the vehicle's boundary box.

[0026] The boundary box is also referred to as a "bounding box". By idealizing the vehicle's boundary boxes and / or the potholes as cuboids, or as rectangles in a 2D view, the requirements for the detection accuracy of the environmental sensors are reduced. Furthermore, the computational effort required to calculate the representation of the environment is also reduced.

[0027] Advantageously, the 2D coordinate system also moves with the vehicle, which further simplifies the calculation of the vehicle's driving behavior relative to its surroundings. The environment is thus considered from the vehicle's perspective, rather than from an egocentric point of view.

[0028] According to a further advantageous embodiment of the method according to the invention, a consideration distance for potholes is defined, and the processing unit only includes the cost factor for the pothole in the cost function if the distance between the vehicle and the nearest pothole is less than the consideration distance. The processing unit determines the cost factor, in particular, by subtracting the distance from the consideration distance. Thus, it is possible to exclude potholes that are farther from the vehicle than the consideration distance as an influencing factor for trajectory determination. The consideration distance can assume a suitable value depending on the situation, such as 0.2 m.

[0029] The cost factor for the potholes can then be calculated particularly advantageously according to the formula: Cost factor = Consideration distance - current distance of the vehicle to the nearest pothole.

[0030] This means that when the distance is large, the cost factor has a low value and therefore little influence on trajectory planning, and when the distance of the vehicle to the nearest pothole decreases, it increases, which also increases the influence on the trajectory calculation.

[0031] A further advantageous embodiment of the method according to the invention provides that the cost factor for each pothole is determined at least two times, and the computing unit incorporates a cost factor averaged from these cost factors into the cost function. The time window considered for this purpose can be chosen to be arbitrarily large and, for example, be fixed or be longer or shorter depending on the situation. For example, the time window can correspond to a few milliseconds or seconds, or even fractions or multiples thereof.

[0032] According to a further advantageous embodiment of the method according to the invention, the cost term for the potholes includes at least one weighting factor, which is multiplied by the cost factor. The weighting factor allows the extent to which potholes affect trajectory determination to be adjusted. The weighting factor can be determined experimentally by a team of experts or determined using data. In particular, the weighting factor can also be variable and thus change depending on the situation. For example, the weighting factor can also depend on another variable, such as the vehicle's speed. Thus, the weighting factor could, for instance, increase with increasing speed.

[0033] In the simplest case, the weighting factor is "1". Therefore, the cost factor enters the cost function unchanged.

[0034] A further advantageous embodiment of the method according to the invention provides that the cost term for potholes takes into account the depth of each pothole. While driving over a relatively shallow pothole can be done without problems, the comfort for the vehicle occupants and the potential for damage to the vehicle or a vehicle component, such as the wheel suspension, can increase when driving over a deep pothole. Taking the pothole depth into account thus allows for the determination of even more suitable trajectories for driving the vehicle.

[0035] Without considering the pothole depth, a trajectory could be determined that inevitably leads around the pothole. This could result in jerky steering maneuvers or strong braking and / or acceleration, which would reduce driving comfort and put stress on the vehicle structure. However, if the pothole is shallow, driving comfort and vehicle stress would be less affected if the trajectory led through the pothole instead of driving around it. Conversely, driving through a particularly deep pothole could have serious consequences, so in such a case, the avoidance trajectory would still be preferred.

[0036] According to a further advantageous embodiment of the method according to the invention, the computing unit subtracts the depth of each pothole, particularly in the form of a weighted depth, from the respective Euclidean distance of the pothole to the vehicle for each pothole in the cost term. This allows the pothole depth to be considered in the cost function in a mathematically sophisticated and therefore efficient way. By selecting an individual weighting factor for the depth of the potholes, the extent to which the pothole depth influences the cost function can also be adjusted. Here, too, the weighting factor can be determined manually by a team of experts or derived from data.

[0037] In a vehicle comprising environmental sensors and a processing unit configured to determine control commands taking into account a planned trajectory, the environmental sensors and the processing unit are configured, according to the invention, to execute a method described above. The vehicle can be any road vehicle, such as a car, truck, van, bus, or the like. The vehicle can have a wide variety of environmental sensors, such as the aforementioned mono or stereo cameras, LiDAR sensors, ultrasonic sensors, radar sensors, and the like. The processing unit can be a central on-board computer or a multitude of control units from vehicle subsystems that interact as a processing unit.For example, sensor data can be evaluated by a first control unit, whereupon a second control unit calculates the trajectory, which is then used by a third control unit to derive control commands for the vehicle, which can be controlled at least partially automatically.

[0038] Further advantageous embodiments of the inventive method for operating a vehicle that is at least partially automatically controllable and of the vehicle itself also result from the exemplary embodiments which are described in more detail below with reference to the figures.

[0039] This shows: Fig. 1 a schematic perspective representation of a vehicle according to the invention during driving and the corresponding environment representation generated by a computing unit of the vehicle; Fig. 2 a cost function used to determine a trajectory to be chosen; Fig. 3 two equations used to determine a separate cost term for potholes; Fig. 4. An equation for determining an average cost factor for potholes; and Fig. 5 Two equations used to determine a cost term for potholes, taking into account pothole depth.

[0040] Fig. Figure 1 shows a vehicle 1 according to the invention, which is controlled at least partially automatically, and in particular autonomously, as it travels towards a pothole 5 on a road 11. The vehicle 1 detects its surroundings by means of an environmental sensor system 6 and is thereby able to detect the pothole 5 and locate it relative to the vehicle 1.

[0041] Fig. Figure 1 also shows an environment representation 12 generated by a processing unit of vehicle 1 (not shown in detail) in a 2D coordinate system 7 from a bird's-eye view. A second pothole 5 is located slightly further away in the adjacent lane. The processing unit generates a boundary box 8.1 around vehicle 1 and boundary boxes 8.2 around each pothole 5, and determines a distance between the boundary boxes 8.1 and 8.2. This distance is advantageously the Euclidean distance d. The pothole 5 directly in front of vehicle 1 is closer to vehicle 1 than the pothole 5 in the adjacent lane, so the processing unit calculates the shortest Euclidean distance d for this pothole 5. min Of all the potholes recorded, 5 were detected.

[0042] Furthermore, the environment representation 12 shows a first and second trajectory 2.1, 2.2, which were calculated by the computing unit for controlling vehicle 1. The first trajectory 2.1 is an original trajectory that passes through pothole 5, and the second trajectory 2.2 is a trajectory that goes around the preceding pothole 5.

[0043] It is well known that for the motion planning of at least partially automated or autonomously controlled vehicles 1, the trajectory actually to be chosen by the vehicle 1 is determined by minimizing a Fig. The cost function 3 shown in Figure 2 is to be determined. According to the invention, it is provided that a separate cost term 4 for potholes 5 is included in this cost function 3. This makes it possible to consider potholes 5 directly in the cost function 3, which makes it particularly effective to avoid potholes 5. Thus, the processing unit would accordingly select the trajectory 2.2 for the actual steering of the vehicle 1.

[0044] The cost function 3 comprises several cost terms 4 for a wide variety of factors, such as driving efficiency, driving comfort, compliance with traffic regulations, safety factors like maintaining specified distances to surrounding agents, and the like. Each cost term 4 represents the product of a cost factor 4.1 and a weighting factor 4.2. In the simplest case, the weighting factor 4.2 can take the value "1", so that the cost factor 4.1 enters the cost function 3 "unweighted". However, a value larger or smaller than "1" can also be chosen as the weighting factor 4.2 for a given cost factor 4.1.

[0045] During movement planning, the goal is to optimize the timing inputs and 1:t to find those that minimize the cost function.

[0046] According to the invention, the in Fig. 3 shown cost factor 4.1 C pothole,tincluded in the cost function 3 to directly represent potholes 5. Fig. Figure 3 shows one possible embodiment of how cost factor 4.1 can be calculated. A consideration distance 9 can be defined, for example, 0.2 m, from which the distance of vehicle 1 to the nearest pothole 5 is subtracted. Cost factor 4.1 can optionally be considered only if the consideration distance 9 is reached or not reached. Otherwise, i.e., if vehicle 1 is further from the nearest pothole 5 than the consideration distance 9, the value "0" is selected for cost factor 4.1.

[0047] In a preferred embodiment, the Euclidean distance d is chosen as the distance between the vehicle 1 and the respective potholes 5. In the equation, av represents t the vehicle's boundary box 8.1 and pothole t jThe boundary box 8.2 of the respective pothole j at time t. Thus, the Euclidean distance d is automatically determined. min to determine the cost factor 4.1, which represents the distance to the nearest pothole 5.

[0048] It is also possible to use a cost factor 4.1 instead of an instantaneous one in the equation in Fig. The average cost factor 4.3 shown in Figure 4 is used. For this purpose, the instantaneous cost factors 4.1 determined at the respective time points t are summed and divided by the respective number of measurements.

[0049] Fig.Figure 5 illustrates an equation that allows the depth 10 of each pothole 5 to be considered in the cost function 3. The weighted depth 10 of each pothole 5 is subtracted from the Euclidean distance d. The depth 10 of each pothole 5 is then multiplied by a separate weighting factor 13. This results in the distance: distance_offset (av t , pothole t j ) receive.

[0050] Using the method according to the invention, potholes 5 can be considered particularly easily and directly as an influencing factor for determining corresponding trajectories 2.1, 2.2. By selecting appropriate weighting factors 4.2 and 13, respectively, the extent to which the depth 10 of a respective pothole 5 or the distance of the pothole 5 to the vehicle 1 affects the trajectory selection can be adjusted. This allows for the determination of particularly suitable trajectories for specific situations. The wear and tear on the vehicle components is reduced, as deep potholes 5, in particular, are avoided. Thus, vehicle 1 downtime can be shortened and maintenance costs reduced. Due to its formulation as a cost term 4, integration into existing behavioral planning algorithms is possible.

Claims

[1] Method for operating a vehicle that is at least partially automated (1), wherein an in-vehicle computing unit determines a trajectory (2.1, 2.2) for the vehicle (1) by minimizing a cost function (3), wherein the cost function (3) is composed of a weighted sum of several cost terms (4), where at least one separate cost term (4) for potholes (5) is included in the cost function (3), characterized by , that The vehicle (1) detects potholes (5) using an environmental sensor (6) and locates them relative to the vehicle (1), the cost term (4) for the potholes (5) includes at least one cost factor (4.1) and the distance between the vehicle (1) and the nearest pothole (5) at the respective time (t) is used to determine the cost factor (4.1). [2] Method according to claim 1, characterized by, that the computing unit, in a 2D coordinate system (7) representing the environment from a bird's-eye view, enters a boundary box (8.1) for the vehicle (1) and a boundary box (8.2) for each pothole (5), determines the Euclidean distance (d) between the boundary box (8.1) of the vehicle (1) and the boundary box (8.2) of the respective pothole (5) for each pothole (5), and calculates the shortest Euclidean distance (d) min ) used to determine the cost factor (4.1). [3] Procedure Claim 2, characterized by , that each boundary box (8.1, 8.2) is ideally designed as a rectangle and / or the origin of the 2D coordinate system (7) is fixed at the center of the boundary box (8.1) of the vehicle (1). [4] Method according to any one of claims 1 to 3, characterized by, that a consideration distance (9) for potholes (5) is defined and the computing unit only considers the cost factor (4.1) for the pothole (5) in the cost function (3) if the distance between the vehicle (1) and the nearest pothole (5) is smaller than the consideration distance (9), wherein the computing unit determines the cost factor (4.1) in particular by subtracting the distance from the consideration distance (9). [5] Method according to any one of claims 1 to 4, characterized by , that the cost factor (4.1) for each pothole (5) is determined at least two time steps and the calculation unit takes into account a cost factor (4.3) averaged from these cost factors (4.1) in the cost function (3). [6] Method according to any one of claims 1 to 5, characterized by, that the cost term (4) for the potholes (5) includes at least one weighting factor (4.2) which is multiplied by the cost factor (4.1). [7] Method according to any one of claims 1 to 6, characterized by , that the cost term (4) for potholes (5) takes into account a depth (10) of each pothole (5). [8] Method according to any one of claims 2 to 6 and claim 7, characterized by , that the computational unit for taking into account the depth (10) of a respective pothole (5) in the cost term (4) for each pothole (5) subtracts the depth (10) of the pothole (5) from the Euclidean distance (d), in particular in the form of a weighted depth (10). [9] Vehicle (1) comprising environmental sensors (6) and a computing unit set up to determine control commands taking into account a planned trajectory (2.1, 2.2), characterized by, that the environmental sensor system (6) and the computing unit are configured to perform a method according to one of claims 1 to 8.

Images