Computer-implemented method, processing device and vehicle control device

Abstract

The invention relates to a computer-implemented method (10) for planning the behavior of a vehicle (14), a processing device (12) and a vehicle control device (62).

Description

[0001] The invention relates to a computer-implemented method for behavior planning of a vehicle according to claim 1. Furthermore, the invention relates to a processing device and a vehicle control device. State of the art

[0002] A dual-channel redundancy architecture is common in automated vehicles. This architecture comprises two processing paths, one of which uses a camera, for example, at the perception level, and the other a lidar sensor or radar sensor.

[0003] Each of these processing paths operates independently, continuously monitoring the vehicle's surroundings and processing the environmental sensor data with a mathematical model that, for example, considers the safety distance as a function of speed. The main goal is to identify potential hazards and obstacles and determine whether automatic intervention is necessary, for example, to prevent collisions or avoid driving too close.

[0004] A common characteristic of the redundancy architectures used today, especially with regard to highly automated vehicle systems, is their 1oo2 structure. This means that if one of the processing paths reports a false detection or "false positive" (FP), the system may tend to react proactively. These reactions could result in sudden braking maneuvers or, in extreme situations, even a complete stop of the vehicle.

[0005] In complex and dynamic traffic environments, such a rapid and potentially unexpected reaction can create risks. An example might be a scenario in which a vehicle stops unexpectedly at a busy intersection, increasing the risk of a side-on collision with an approaching vehicle.

[0006] Both false positive (FP) and false negative (FN) detections can lead to risk situations that occur more frequently compared to human drivers with their experience and intuition.

[0007] The 1oo2 redundancy architecture used in automated driving systems places particular emphasis on safety and is therefore designed to favor intervention to minimize potential hazards. This emphasizes the need to consider both processing paths equally, as neither is considered more dominant or reliable. The lack of a way to validate the intervention of one processing path as more trustworthy ensures that both processing paths are treated equally and fulfill their role in the overall functionality of the system. Disclosure of the invention

[0008] According to the present invention, a computer-implemented method for planning the behavior of a vehicle with the features of claim 1 is proposed. This allows a safety-critical and dangerous vehicle maneuver to be avoided. The set operating parameters can be more appropriate to the situation. The driving safety of the vehicle can be increased. Possible FP events can be identified and filtered out. Unnecessary braking or inappropriate minimal risk maneuvers can be avoided.

[0009] The vehicle can be a motor vehicle, a truck, a two-wheeled vehicle, a mobile robot, or an industrial robot. The vehicle can assume at least a semi-autonomous or autonomous operating state during driving.

[0010] The first environmental sensor can be a LIDAR sensor, a radar sensor, an ultrasonic sensor, or a camera. The second environmental sensor can have a different sensor category than the first environmental sensor, in particular a different measuring principle. The second environmental sensor can be a LIDAR sensor, a radar sensor, an ultrasonic sensor, or a camera. If the first environmental sensor is a camera, the second environmental sensor is preferably a radar sensor, LIDAR sensor, or ultrasonic sensor.

[0011] The first and / or second surrounding area of ​​the vehicle can include, preferably exclusively, the immediate surrounding area of ​​the vehicle. The second surrounding area can spatially overlap primarily with the first surrounding area.

[0012] The first and second processing can form a 1oo2 redundancy architecture.

[0013] During the first and / or second environmental sensing, raw data from the physical environment can be collected using the respective environmental sensor. The capability, reliability, and / or integrity of the first and / or second environmental sensing may depend on a range, resolution, and / or weather condition. The first and / or second environmental sensing may at least partially involve the processing of the first and / or second environmental sensor data, respectively. In this regard, ASIL (Automotive Safety Integrity Level) according to ISO 26262, which refers to the probability that internal processing is faulty due to an electrical / electronic failure, may be applicable.

[0014] The environmental object can be a building, a living being, a device, an object, another vehicle, a plant or a road surface.

[0015] The first environmental perception can process and interpret the first environmental sensor data provided by the first environmental sensor. The second environmental perception can process and interpret the second environmental sensor data provided by the second environmental sensor. This can include attempting to understand the meaning of the information contained in the first and / or second environmental sensor data about the first or second environmental region and the at least one environmental object. For example, it can be recognized whether the environmental object is a car, a pedestrian, or a bicycle and / or whether it is moving or stationary. Machine learning and computer vision techniques can be used in the first and / or second environmental perception, in particular to recognize patterns in the first or second environmental sensor data and / or to correctly classify the at least one environmental object.The goal of the first and / or second environmental perception is a reliable and accurate representation of the environment in real time. The first and / or second environmental perception can include object detection and / or object tracking of at least the environmental object.

[0016] A first environmental model can be created using the first environmental perception. A second environmental model can be created using the second environmental perception. The first and / or second environmental model can be a real-time representation of the first or second environmental area, respectively. The first and / or second environmental model can comprise an environmental map or a 3D model of the first or second environmental area, respectively. The first and / or second environmental model can contain information about stationary environmental objects (e.g., buildings, trees, roads) and / or about moving objects (e.g., other vehicles, pedestrians, animals) in the first or second environmental area, respectively.

[0017] In addition to the first and second environment models, a fusion environment model can be created from the first and / or second environment capture.

[0018] The proposed and / or planned operating parameter to be applied can be an emergency operating parameter for an emergency operating situation, which is applied in a fallback solution to maximize driving safety, for example, in a minimal risk maneuver. The planned emergency operating situation can relate to the current situation or future situations. The proposed and / or planned operating parameter to be applied can be the operating parameter to be set at an initial point in time.

[0019] The behavior planning may include or be a vehicle behavior planning of a vehicle behavior of the vehicle and / or a trajectory planning of a vehicle trajectory of the vehicle.

[0020] The action planning can be part of vehicle behavior planning and / or trajectory planning. The action planning can be vehicle behavior planning and / or trajectory planning.

[0021] The operating parameter can be a vehicle parameter, a driving operation parameter of a driving operation of the vehicle, and / or a movement parameter of the vehicle. The operating parameter can actually be adjustable and / or variable by at least one actuator. The operating parameter can be a longitudinal speed, longitudinal acceleration, lateral speed, lateral acceleration, a vehicle pose, and / or a vehicle position. The operating parameter can be a signal, in particular a visual, acoustic, and / or vibrating signal. The operating parameter can comprise a sequence of actions for several operating parameters. The proposed operating parameter can be a target value of the planned operating parameter.

[0022] The safety rules can be a responsibility-sensitive safety model (e.g. RSS for responsibility sensitive safety or TTC for Time-To-Collision), in particular a mathematical safety model, in particular the evaluation of which is comprehensible and justifiable.

[0023] The intervention measure can influence the output of the proposed operating parameter and / or the operating parameter itself. For example, the intervention measure can delay, reject, or allow the output of the proposed operating parameter. The intervention measure can change a magnitude and / or a gradient of the proposed operating parameter. The intervention measure can limit the intervention to the proposed operating parameter of the safety assessment. rule-based, for example by directly inferring from detected external and internal conditions their effect on the performance of the individual processing, for example FP in the radar due to heavy rain, heat rollers, reflective elements that can be driven over such as rails or manhole covers, ramps and others (“phantom braking”), with calculation of confidence or uncertainty metrics, for example covariance (Bayes Filter), plausibility (Possibility Theory), belief (Dempster-Shafer Theory) and their comparison against a limit value, through established rules, by at least one lookup table and / or AI-based, for example by applying Deep Neural Network, Foundation Model or other ML models establish.

[0024] The intervention measure can be integrated in the sequence between the safety assessments and the action planning.

[0025] In a preferred embodiment of the invention, it is advantageous if the state detection includes a capability detection of the detection capabilities of the first environment detection, the second environment detection, the first environment perception, and / or the second environment perception. The capability detection can include a capability assessment of the first and second environment sensors, in particular via diagnostic functions. The capability detection can electronic, electrical faults or conditions of the environmental sensors, a diagnosis of the environmental sensors, physical conditions, such as temperatures or alignment errors or obscurations, such as leaves and / or performance-limiting influences, in particular “potential triggering conditions” according to ISO 21448, such as weather conditions, restricted visibility, lighting conditions, reflections or obscurations capture.

[0026] Capability assessment can include information from the environmental sensors, the sensor electronics of the environmental sensors and / or the control units, other vehicle sensors, in particular the rain sensor, light sensor, the surrounding area map, for example with known geolocal function-limiting conditions and / or a V2X communication, for example from other road users, the traffic infrastructure, edge or cloud receive.

[0027] The capability assessment can include at least two properties of the processing, in particular of the perception system, namely the integrity (AIL according to ISO 26262) and the confidence or uncertainty (uncertainty as a metric, performance limitation according to ISO 21448).

[0028] In a preferred embodiment of the invention, it is advantageous if the intervention measure comprises intervention filtering for filtering the output suggested operating parameter. The intervention measure can be intervention filtering. The intervention measure can decide whether to output the suggested operating parameter. The intervention measure can be carried out depending on a comparison of the safety assessment of the first environmental perception and the safety assessment of the second environmental perception. If the results of the safety assessment of both processing steps agree, the suggested operating parameter can be output according to the safety assessment. If the safety assessments differ, the intervention measure can evaluate the results depending on the state detection.

[0029] The intervention measure may specifically allow, block, temporarily block or temporarily allow or modify, in particular weaken, the proposed operating parameters issued.

[0030] A preferred embodiment of the invention is advantageous in that the status detection includes error detection and / or functional limitation detection during the first environmental detection, the second environmental detection, the first environmental perception, and / or the second environmental perception. The error detection can detect an electrical and / or electronic error. The functional limitation detection can detect a functional limitation of the first environmental detection, the second environmental detection, the first environmental perception, and / or the second environmental perception.

[0031] The error detection and / or functional limitation detection can utilize diagnostic results from the first environment detection, the second environment detection, the first environment perception, and / or the second environment perception. The error detection and / or functional limitation detection can evaluate external and / or internal conditions.

[0032] In a specific embodiment of the invention, it is advantageous if the intervention measure includes a time delay for the output of the proposed operating parameter to the action planning system. The output delay can be a delay in an error response, i.e., error debouncing.

[0033] In a preferred embodiment of the invention, the state detection system gathers currently existing safety requirements for the first and / or second environmental perception, and the duration of the output delay is determined depending on the safety requirements. The duration of the output delay can be determined depending on an assessment of the current vehicle situation.

[0034] The safety requirements can result from the current scenario as functional limits that must not be exceeded, for example distances to vehicles, road edge. The safety requirements can be based on information about a vehicle condition, for example a vehicle speed or a distance to a vehicle in front, from the first environment model, the second environment model and / or the fusion environment model, from a map of the surrounding area, for example the presence of traffic lights, traffic signs on the route ahead and / or from V2X information, for example from other road users, in particular other vehicles and / or traffic infrastructure be formed.

[0035] In an advantageous embodiment of the invention, the intervention measure initiates the use of a third processing step comprising a third environmental detection, a third environmental perception, a safety assessment of the third environmental perception through application of the safety rules, and an output of the proposed operating parameter depending on the safety assessment including the third environmental perception. This allows the temporary failure of a processing step to be compensated for by temporarily bridging the third processing step. The third environmental detection can be obtained by additional vehicle sensors, for example ultrasonic sensors and / or at least one IR camera, by expansion modules connected to a vehicle interface, in particular CAN, and / or by V2X communication.Communication via V2X can take place with an external system (e.g. edge, cloud, road-side unit), which in turn can generate an environment model (e.g. with infrastructure sensors or the merging of sensors from several V2X vehicles in the vicinity).

[0036] The condition detection and / or intervention measure can be carried out in a central control unit of the vehicle.

[0037] In a specific embodiment of the invention, it is advantageous if the intervention measure includes the third processing and deactivates the first and / or second processing identified as faulty or unreliable by the status detection. This allows for a temporary override of the faulty or unreliable processing.

[0038] According to the present invention, a processing device for a vehicle is further proposed, which is configured to execute the computer-implemented method for behavior planning with at least one of the previously described features. The processing device can have at least one processor for at least partially executing the computer-implemented method.

[0039] According to the present invention, a vehicle control device for a vehicle is further proposed, which is configured to set the operating parameter to be applied using a computer-implemented method having at least one of the previously described features. The vehicle control device can be a control unit or a part of a control unit.

[0040] At least one method step of the computer-implemented method can be performed in the vehicle using edge computing and / or outside the vehicle using cloud computing. At least one method step of the computer-implemented method can be processed using machine learning. The machine learning can use a neural network. The machine learning can include reinforcement learning.

[0041] Furthermore, the invention relates to a non-transitory, computer-readable medium containing instructions, the execution of which causes at least one processor to carry out said computer-implemented method with at least one previously described feature.

[0042] Further advantages and advantageous embodiments of the invention emerge from the description of the figures and the illustrations. Character description

[0043] The invention is described in detail below with reference to the figures. They show in detail: Fig. 1: A computer-implemented method in a specific embodiment of the invention. Fig. 2: A computer-implemented method in another specific embodiment of the invention. Fig. 3: A computer-implemented method in another specific embodiment of the invention. Fig. 4: A computer-implemented method in another specific embodiment of the invention.

[0044] Fig. 1 shows a computer-implemented method in a specific embodiment of the invention. The computer-implemented method 10 for behavior planning of a vehicle can be executed by a processing device 12 of the vehicle 14 and comprises a first processing step 16 with a first environmental detection 18 of a first environmental region 22 of the vehicle 14, which includes at least one environmental object 20, with a first environmental sensor 26 generating first environmental sensor data 24 in this regard. The first environmental sensor 26 is, for example, a camera 28. Furthermore, a second processing step 30 is configured with a second environmental detection 32 of a second environmental region 34 of the vehicle 14, which at least partially encompasses the first environmental region 22, with a second environmental sensor 38 generating second environmental sensor data 36. The second environmental sensor 38 is, for example, a lidar sensor and / or radar sensor 40.The second surrounding area 34 spatially coincides in particular mainly with the first surrounding area 22.

[0045] The first and second processing 16, 30 may preferably form a two-channel redundancy architecture.

[0046] A subsequent first environmental perception 42 including creation of a first environmental model 44 from the first environmental sensor data 24 and a second environmental perception 46 including creation of a second environmental model 48 from the second environmental sensor data 36 is additionally accompanied by the creation of a fusion environmental model 50, which is created from the first and second environmental detections 18, 32.

[0047] In a subsequent safety assessment 52 of the first environmental perception 42 and a safety assessment 54 of the second environmental perception 46, a safety rule set 55 is applied. The safety rule set 55 can be a responsibility-sensitive safety model (RSS, TTC), in particular a mathematical safety model whose assessment is comprehensible and justifiable.

[0048] Depending on the safety assessments 52, 54, at least one suggested operating parameter 56 of the vehicle 14 is output. For example, the safety assessment 52 can output a suggested operating parameter 56 corresponding to braking the vehicle 14 to avoid a collision or driving too close to a preceding vehicle 14.

[0049] Subsequently, an action planning 58 is performed, which plans the operating parameter 60 to be applied, for example, the braking process, depending on the proposed operating parameter 56. The planned operating parameter 60 to be applied can be set by a vehicle control device 62.

[0050] An intervention measure 64 is integrated between the safety assessments 52, 54 and the action planning 58, which, depending on a status detection 66, specifically influences the outputted at least one proposed operating parameter 56. The status detection 66 is, in particular, a capability detection 68 of the detection capabilities of the first and second environmental sensors 18, 32 and / or the first and second environmental perception 42, 46. Information from additional vehicle sensors or from V2X communication 70 can be incorporated into the status detection 66. For example, the capability detection 68 can include a capability assessment of the first and second environmental sensors 26, 38 via diagnostic functions.

[0051] The intervention measure 64 can include an intervention filter 72 for filtering the proposed operating parameter 56 to be output. The operating parameter proposed by the safety assessments 52, 54 can be filtered. In particular, the intervention measure 64 can decide whether the operating parameter proposed by the safety assessments 52, 54 is permitted and output as the proposed operating parameter 56.

[0052] The state detection 66 can alternatively or additionally include error detection 74 and / or functional restriction detection in the first environment detection 18, the second environment detection 32, the first environment perception 42, and / or the second environment perception 46. The intervention measure 64 can alternatively or additionally include a time delay 76 for the output of the proposed operating parameter 56 to the action planning 58. The state detection 66 can obtain currently existing safety requirements, for example, from other vehicle sensors or a V2X communication 70, and the duration of the output delay 76 can be determined depending on the safety requirements.

[0053] Fig. Figure 2 shows a computer-implemented method in another specific embodiment of the invention. The computer-implemented method is similar to that of Fig. 1 except for the following differences. In case of conflicting safety assessments 52, 54, the execution in Fig. 1 additionally includes a further security assessment 78 based on the fusion environment model 50, for example, as a further processing 80 in a 2oo3 majority decision. Based on the capability assessment 68 of the three processing steps 16, 30, and 80, this decision could possibly also be made with additional weighting.

[0054] Fig. Figure 3 shows a computer-implemented method in another specific embodiment of the invention. The computer-implemented method is similar to that of Fig. 1 except for the following differences. In the case of conflicting safety assessments 52, 54 of the first and second environmental perceptions 42, 46, a risk assessment 82 of the (estimated) risks associated with the respective decision regarding the proposed operating parameter is additionally included, in particular based on the first and second environmental models 44, 48 and the fusion environmental model 50. In this case, the consideration of the relative risk may be sufficient.

[0055] For example, the risk of false detection (“false positive”, FP) could be represented as the product of the probability of false detection and the severity of the accident resulting from a reaction. And the risk of correct detection (“true positive”, TP) could be calculated as the product of the probability of correct detection and the severity of the accident resulting from a non-reaction. The probability of correct or incorrect detection results from the capability assessment of the first and second processing steps 16, 30 and the further processing steps 80 and their probabilistic results. These could, for example, be the probability of existence of an environmental object calculated from the first and second environmental sensor data 24, 36, which was used for the safety assessments 52, 54 and which could now also be made available to the intervention measure 64.

[0056] The severity of the accident results from the prediction of the first and second environmental model 44 48 (e.g. other surrounding objects) and the vehicle’s own movement (depending on the reaction or decision), i.e. on the basis of a “classical” collision estimation. Here is a first example:

[0057] If the probability for an environmental object in front of the vehicle is 0.5 and for an environmental object behind the vehicle is 0.9 and the severity of the accident if no braking is 100 with respect to the environmental object in front of the vehicle and the severity of the accident if braking is 20 with respect to the environmental object behind the vehicle, then the risk for no braking is 50 and the risk for braking is 18. With this calculated risk, the proposed operating parameter of the safety assessment 52 associated with braking is permitted by the intervention measure 64. A second example:

[0058] If the probability for an environmental object in front of the vehicle is 0.3 and for an environmental object behind the vehicle is 0.9 and the severity of the accident if no braking is 100 with respect to the environmental object in front of the vehicle and the severity of the accident if braking is 50 with respect to the environmental object behind the vehicle, then the risk for no braking is 30 and the risk for braking is 45. With this calculated risk, the proposed operating parameter of the safety assessment 52 associated with braking is not permitted by the intervention measure 64.

[0059] The intervention measure 64 also makes it possible to detect matching FP events or FN events that occur temporarily in the first and second environmental perceptions 42, 46 and to suitably influence the proposed operating parameter 56 to be output.

[0060] Fig. Figure 4 shows a computer-implemented method in another specific embodiment of the invention. The computer-implemented method is similar to that of Fig. 1 except for the following differences. The intervention measure 64 initiates the use of a third processing 84 with a third environmental detection 86, a third environmental perception 88, a safety assessment 90 of the third environmental perception 88 by applying the safety rule set 55, and an output of the proposed operating parameter depending on the safety assessment 90 including the third environmental perception 88. The third environmental detection 86 can be carried out via additional vehicle sensors, for example, ultrasonic sensors and / or at least one IR camera. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited non-patent literature

[0000] ISO 26262

[0013]

Claims

[1] Computer-implemented method (10) for behavior planning of a vehicle (14), at least comprising a first processing (16) with a first environmental detection (18) of a first environmental region (22) of the vehicle (14) including at least one environmental object (20) with a first environmental sensor (26) generating first environmental sensor data (24) in this regard, a first environmental perception (42) from the first environmental sensor data (24) and a safety assessment (52) of the first environmental perception (42) by applying a safety rule set (55), a second processing (30) with a second environmental detection (32) of a second environmental region (34) of the vehicle (14) at least partially encompassing the first environmental region (22) with a second environmental sensor (38) generating second environmental sensor data (36) in this regard, a second environmental perception (46) from the second environmental sensor data (36) and a safety assessment (54) of the second environmental perception (46) by applying the safety rule set (55), an output (92) of at least one proposed operating parameter (56) of the vehicle (14) depending on the safety evaluations (52, 54), an action plan (58) which plans the operating parameter (60) to be used depending on the proposed operating parameter (56), wherein an intervention measure (64) is integrated between at least one of the safety assessments (52, 54) and the action plan (58), which intervention measure specifically influences the output at least one proposed operating parameter (56) depending on a state detection (66). [2] Computer-implemented method (10) according to claim 1, characterized by that the state detection (66) includes a capability detection (68) of detection capabilities of the first environment detection (18), the second environment detection (32), the first environment perception (42) and / or the second environment perception (46). [3] Computer-implemented method (10) according to claim 1 or 2, characterized by that the intervention measure (64) comprises an intervention filtering (72) for filtering the proposed operating parameter (56). [4] Computer-implemented method (10) according to one of the preceding claims, characterized by that the state detection (66) includes an error detection (74) and / or functional restriction detection in the first environment detection (18), the second environment detection (32), the first environment perception (42) and / or the second environment perception (46). [5] Computer-implemented method (10) according to one of the preceding claims, characterized by that the intervention measure (64) comprises a time delay (76) of the output (92) of the proposed operating parameter (56) to the action planning (58). [6] Computer-implemented method (10) according to claim 5, characterized by that the state detection (66) obtains currently existing safety requirements for the first and / or second environmental perception (42, 46) and the duration of the output delay (76) is determined depending on the safety requirements. [7] Computer-implemented method (10) according to one of the preceding claims, characterized by that the intervention measure (64) causes the use of a third processing (84) with a third environmental detection (86), a third environmental perception (88), a safety assessment (90) of the third environmental perception (88) by applying the safety rule set (55) and an output (92) of the proposed operating parameter (56) depending on the safety assessment (90) including the third environmental perception (88). [8] Computer-implemented method (10) according to claim 7, characterized by that the intervention measure (64) includes the third processing (84) and switches off the first and / or second processing (16, 30) recognized as faulty or unreliable by the status detection (66). [9] Processing device (12) for a vehicle (14) which is arranged to carry out the computer-implemented method (10) according to one of the preceding claims. [10] Vehicle control device (62) for a vehicle (14) which is configured to set the operating parameter (60) of the vehicle (14) to be applied, as planned using a computer-implemented method (10) according to one of claims 1 to 8.

Images