Method for operating an assistance system of a vehicle and assistance system

Abstract

The invention relates to a method for operating an assistance system (10) of a vehicle, comprising the steps: Capturing sensor data using at least one sensor device of the vehicle; Generating a grid-based environmental map in which the surroundings of the vehicle are divided in multiple spatial areas; Applying a trained neural network to predict a height value for each grid cell of the environmental map; and Classifying the predicted height of each grid cell into one of several predefined height classes, wherein an uncertainty estimation is performed for each grid cell based on the entropy of the probability distribution over the height classes; and the behaviour of the assistance system is adjusted depending on the uncertainty estimation. Furthermore the invention relates to a assistance system (10), a computer program product, and a non-transitory computer-readable storage medium.

Description

Applicant’s Ref.: 2024P00908WOAttorney’s Ref.: 957441Mercedes-Benz Group AG Negi 2026-01-29METHOD FOR OPERATING AN ASSISTANCE SYSTEM OF A VEHICLE AND ASSISTANCE SYSTEMFIELD OF THE INVENTION

[0001] The invention relates to the field of automobiles. More specifically, the present invention relates to a method for operating an assistance system of a vehicle according to claim 1. Furthermore, the present invention relates to an assistance system, a corresponding computer program product, and a corresponding non-transitory computer-readable storage medium.BACKGROUND INFORMATION

[0002] The technical subject of the invention involves controlling an assistance system, such as a parking assistance system, that uses a neural network to estimate the height of spatial areas in the vehicle’s surroundings and to adjust the system's behaviour based on an uncertainty assessment of these estimations. Unlike conventional systems that detect and classify discrete objects, the present invention operates on a grid-based representation of the environment and provides a height classification for each grid cell.

[0003] From DE 102022201 112 A1, a method for object detection and uncertainty estimation is known, in which a neural network is used for object detection.

[0004] In the state of the art, the vehicle environment is often represented using so-called 2.5D occupancy grid maps. In these maps, the surroundings are divided into a regular grid of spatial cells, with each cell storing geometric information such as occupancy status and / or the estimated height of structures within that area. Such maps enable a simplified yet spatially detailed modeling of the environment and are widely used for perception and navigation tasks in automated driving.Applicant’s Ref.: 2024P00908WOAttorney’s Ref.: 957442

[0005] The objective of the invention is to provide a method for operating an assistance system that enables improved system control in order to enhance safety during the driving process.

[0006] This objective is achieved by a method with the features of claim 1.Advantageous embodiments of the inventive method are considered advantageous embodiments of the assistance system, wherein the means of the assistance system are used to perform the method steps. Furthermore, advantageous developments of the invention are described by the dependent claims, the following description, and the figures.SUMMARY OF THE INVENTION

[0007] One aspect of the invention relates to a method for operating an assistance system of a vehicle, in particular a passenger car, especially a partially or fully autonomous passenger car, comprising the steps: capturing sensor data using at least one sensor device of the vehicle, generating a grid-based representation of the vehicle environment, applying a trained neural network to estimate the height in each spatial cell of the grid, classifying the estimated height into discrete height categories, and executing an action of the assistance system depending on the height classification and the associated uncertainty.

[0008] Accordingly, sensor data is captured using at least one sensor device of the vehicle. Alternatively, a detection system or a combination of multiple sensor devices may be used. The sensor data may be provided by cameras, LIDAR, or radar, thereby comprehensively capturing the environment of the vehicle.

[0009] The sensor data is processed to generate a grid map, a 2.5D grid map for example, in which the surrounding environment is divided into regular spatial cells. Each cell contains geometric information such as the estimated height of the surface within that cell. This raster-based representation serves as input for the neural network and enables structured processing of spatial data without relying on discrete object detection.

[0010] The neural network is trained to estimate the height for each grid cell based on the sensor input. Instead of treating the height estimation purely as a regression task, the invention proposes to perform a classification of the estimated height into discreteApplicant’s Ref.: 2024P00908WOAttorney’s Ref.: 957443categories, such as “low”, “medium”, and “high”. This enables a probabilistic interpretation of the network output and forms the basis for a reliable uncertainty estimation.

[0011] To improve system control through uncertainty awareness, the invention provides that each height classification is associated with a probability distribution over the defined height categories. From this distribution, the uncertainty of the prediction for each grid cell is derived using a suitable entropy measure.

[0012] The approach thus replaces or supplements continuous height estimation (regression) with a classification task, in which each spatial cell receives a probabilistic classification into discrete height levels. For example, instead of predicting a single height value, the system might output a probability distribution such as 0.1 for "low", 0.6 for "medium", and 0.3 for "high". The shape of this distribution indicates how confident the system is in its prediction.

[0013] This reformulation from regression to classification offers the key advantage of enabling an entropy-based uncertainty estimation. The sharper the probability distribution, the more confident the prediction. A flatter distribution (high entropy) indicates uncertainty.

[0014] This enables the assistance system to adapt its behaviour depending on the confidence of the height estimation in individual grid cells. For example, the system may act more cautiously when uncertainty is high, especially in areas where elevated structures may be present.

[0015] Additional gradations or a larger number of height categories may optionally be introduced to allow for finer resolution in the classification, depending on the use case and sensor accuracy. This increases the system's flexibility and supports more differentiated responses.

[0016] For instance, the classification categories may correspond to discrete height intervals relevant to the driving context. The assistance system then interprets these categories to assess driveability, ground clearance, or potential collision risks.

[0017] The uncertainty estimation may be based on Shannon entropy, with the base of the logarithm adjusted according to the number of height classes used. This provides a mathematically sound measure of uncertainty for each prediction..Applicant’s Ref.: 2024P00908WOAttorney’s Ref.: 957444

[0019] The approach enables the system to assign different confidence levels to different areas of the environment. Grid cells with low entropy may be treated as reliable, whereas cells with high entropy may prompt caution or further analysis.

[0020] For example, the system may reduce speed or trigger a re-evaluation cycle in the presence of high uncertainty. Conversely, confident predictions can lead to assertive and efficient system actions.

[0021] Although the addition of entropy estimation increases computational complexity slightly, the overhead remains minimal (e.g., between 0.1% and 3% compared to the base model). Two architectural variants are possible: a separate classification head that predicts height categories directly, or a two-step approach where an intermediate class is predicted and refined by a delta regression head. Both approaches require only minor changes to the network structure.

[0022] Therefore, the invention offers significant advantages by enabling grid-cell-specific uncertainty estimation for height classification. This allows for precise adaptation of driving or parking strategies, increasing safety and operational robustness in autonomous or semi-autonomous driving systems.

[0023] Additionally, the invention allows for more intelligent driver interaction. Visual, acoustic, or haptic signals can be dynamically adjusted depending on the estimated certainty of the situation. In high-uncertainty scenarios, warnings may be delayed until additional sensor data is available, helping to reduce false alarms and increase user trust.

[0024] In an advantageous embodiment of the invention, it is provided that at least one warning signal is issued in case of high uncertainty in the height classification of one or more grid cells. The warning signals may be acoustic, visual, and / or haptic, thereby allowing the driver to be informed of uncertain environmental conditions in a timely and intuitive manner.

[0025] In another advantageous embodiment of the invention, it is provided that at least one emergency control signal is issued in case of high uncertainty. For example, the assistance system may automatically reduce the vehicle’s speed or delay a maneuver toApplicant’s Ref.: 2024P00908WOAttorney’s Ref.: 957445minimize the risk of collision or misinterpretation of the environment due to uncertain height estimations.

[0026] In a further advantageous embodiment of the invention, it is provided that the predicted height values are divided into at least three discrete height classes. For example, the classes "low", "medium", and "high" may be defined to categorize the height of each grid cell. This classification enables the assistance system to differentiate between flat, moderately elevated, and high terrain segments. The use of three classes offers a balance between sufficient differentiation and low model complexity, avoiding the overfitting risks that may arise from using too many classes.

[0027] In yet another advantageous embodiment of the invention, it is provided that a separate classification head is added to the neural network. This classification head is configured to predict the height class of each grid cell based on the latent representation of the sensor data. Importantly, the gradient flow from this classification head to the underlying model is blocked, so that the training of the original network remains unaffected. This enables precise and independent uncertainty estimation of the height classification using entropy measures. For example, if a grid cell is classified with nearly equal probability across the "low", "medium", and "high" classes, this indicates high entropy and therefore low certainty. In such cases, the system may respond with adaptive actions such as issuing a warning or reducing speed.

[0028] Another aspect of the invention relates to an assistance system of a vehicle, such as a parking assistance system, comprising at least one sensor device for capturing environmental data, and an electronic device including a trained neural network configured to estimate the height of each grid cell in a raster-based environment model. The neural network classifies each height estimation into predefined height classes and calculates an uncertainty value per cell based on the entropy of the classification result. A control unit executes or modifies an action of the assistance system depending on the classification and the corresponding uncertainty estimation. Additionally, the assistance system may include a display unit for issuing acoustic, visual, or haptic warning signals to the driver in response to high uncertainty in the environmental data. Furthermore, a data storage unit may be provided to log the classification and uncertainty results per grid cell for further analysis and system learning. The assistance system may also interface with internal or external data sources, such as cloud-based databases, to retrieve stored environmental information or to share current uncertainty data with other vehicles,Applicant’s Ref.: 2024P00908WOAttorney’s Ref.: 957446especially in fleet-based applications. This enables learning across multiple vehicles and contributes to more robust and cooperative environmental perception.

[0029] In other words, the invention provides an assistance system and method that enable improved control decisions based on cell-wise height classification and associated uncertainty estimation. In particular, the use of discrete classes and a separate classification head enables a flexible, scalable, and safety-oriented system architecture -especially suitable for challenging scenarios such as parking or low-speed maneuvering. Further advantages, features, and details of the invention derive from the following description of preferred embodiment as well as from the drawing. The features and feature combinations previously mentioned in the description as well as the features and feature combinations mentioned in the following description of the figure and / or shown in the figure alone can be employed not only in the respectively indicated combination but also in any other combination or taken alone without leaving the scope of the invention.BRIEF DESCRIPTION OF THE DRAWING

[0030] The novel features and characteristic of the disclosure are set forth in the appended claims. The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate exemplary embodiments and together with the description, serve to explain the disclosed principles. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The same numbers are used throughout the figures to reference like features and components. Some embodiments of system and / or methods in accordance with embodiments of the present subject matter are now described below, by way of example only, and with reference to the accompanying figures.

[0031] The drawing shows in:

[0032] Fig. 1 a diagram illustrating a possible implementation of a method for operating a parking assistance system as example for any assistance system using height classification in a raster-based environmental representation.

[0033] In the figure the same elements or elements having the same function are indicated by the same reference signs.Applicant’s Ref.: 2024P00908WOAttorney’s Ref.: 957447DETAILED DESCRIPTION

[0034] In the present document, the word "exemplary" is used herein to mean "serving as an example, instance, or illustration". Any embodiment or implementation of the present subject matter described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

[0035] While the disclosure is susceptible to various modifications and alternative forms, specific embodiments thereof have been shown by way of example in the drawing and will be described in detail below. It should be understood, however, that it is not intended to limit the disclosure to the particular forms disclosed, but on the contrary, the disclosure is to cover all modifications, equivalents, and alternatives falling within the scope of the disclosure.

[0036] The terms “comprises”, “comprising”, or any other variations thereof, are intended to cover a non-exclusive inclusion so that a setup, device or method that comprises a list of components or steps does not include only those components or steps but may include other components or steps not expressly listed or inherent to such setup or device or method. In other words, one or more elements in a system or apparatus preceded by “comprises” or “comprise” does not or do not, without more constraints, preclude the existence of other elements or additional elements in the system or method.

[0037] In the following detailed description of the embodiment of the disclosure, reference is made to the accompanying drawing that forms part hereof, and in which is shown by way of illustration a specific embodiment in which the disclosure may be practiced. This embodiment is described in sufficient detail to enable those skilled in the art to practice the disclosure, and it is to be understood that other embodiments may be utilized and that changes may be made without departing from the scope of the present disclosure. The following description is, therefore, not to be taken in a limiting sense.

[0038] Fig. 1 shows a diagram illustrating a possible implementation of a method for operating a parking assistance system 10 using height classification in a raster-based environmental representation. The method begins by capturing sensor data using at least one sensor device. This data is then processed by a trained neural network, which estimates a height value for each spatial area (grid cell) of a vehicle environment map. Each estimated height is then classified into one of several predefined height categories.Applicant’s Ref.: 2024P00908WOAttorney’s Ref.: 957448Based on this classification, the parking assistance system 10 executes an action, such as issuing a warning or adjusting the vehicle’s behaviour. To improve the performance of the parking assistance system 10, the neural network is further configured to assess the uncertainty of the height classification using the entropy of the probability distribution. The parking assistance system 10 then adjusts its behaviour depending on the level of uncertainty, ensuring safer and more reliable operation during parking maneuvers.

[0039] This process is partially illustrated by the block diagram with several interconnected components. The first block B1 represents an encoder, which processes the captured sensor data and generates a latent representation. This encoded data forms the foundation for further analysis and prediction. From block B1, the encoded data flows along two parallel paths.

[0040] In a primary path, the encoded data is sent to a second block B2, which contains a regression head. This regression head uses the latent representation to predict continuous height values for each grid cell. The output from block B2 is then passed to a third block B3, where the final height map is generated. This output may be used to control the vehicle’s actions, such as steering or speed adjustments, depending on the estimated surface structure in the environment.

[0041] In a parallel second path, the encoded data from B1 is also sent to a fourth block B4, which contains an additional classification head. This classification head assigns each grid cell a height category, such as "low", "medium", or "high", based on the estimated height. The result of this classification is then forwarded to a fifth block B5, where class entropy is calculated. Class entropy measures the uncertainty in the classification by analyzing the probability distribution of the predicted categories. A high entropy value indicates greater uncertainty, signaling that the system is unsure about the height level of a specific area.

[0042] The dual-path approach, with regression in the primary path (B1 — > B2 — > B3) and classification with uncertainty estimation in the parallel path (B1 — > B4 — > B5), allows the parking assistance system 10 to produce both precise continuous height estimations and categorized height levels, while also evaluating the reliability of those predictions. If high uncertainty is detected, the parking assistance system 10 may adapt its behaviour, for example by issuing a warning to the driver or reducing the vehicle’s speed to increase safety.Applicant’s Ref.: 2024P00908WOAttorney’s Ref.: 957449

[0043] This architecture is particularly advantageous because it combines the strengths of regression for detailed height prediction and classification for robust decision-making. By incorporating uncertainty estimation, the system can handle ambiguous or unclear sensor data in a more cautious and adaptive manner. The modular design of encoder, regression head, classification head, and entropy-based uncertainty estimation also allows for easy integration of further functionalities, such as cloud connectivity or shared learning across a vehicle fleet.

[0044] In contrast to the primary B paths, which focus on continuous regression and general uncertainty estimation, additional C paths provide a refined approach using discrete classification, delta regression, and detailed uncertainty handling.

[0045] The method in the C paths begins at a first block C1 representing an encoder, where sensor data is converted into a latent representation. From block C1, the data follows one of two paths. In the first path, data is passed to a second block C2 representing the classification head, where each grid cell is classified into a discrete height category ("low", "medium", and / or "high"). The resulting probability distribution from the third block C3 is passed to a fourth block C4 representing a delta regression module, which refines the initial classification by predicting small height deviations within the assigned category. The refined result from C4 is then passed to a fifth block C5, where the final height output is generated. This height output can be used by the parking assistance system 10 to control vehicle movement more precisely.

[0046] In the second path, after the probability distribution is generated in block C3, it is sent to a sixth block C6 for class entropy and uncertainty estimation. The parking assistance system 10 assesses the confidence of the height classification by calculating entropy. If entropy is high (indicating low confidence), the system may respond by issuing a warning or reducing vehicle speed. This allows the system to remain cautious when the reliability of the prediction is low.

[0047] By combining both B and C paths, the parking assistance system 10 achieves high accuracy and safety, enabling precise predictions and adaptive responses in real-world parking scenarios.

[0048] In summary, the invention proposes a system or parking assistance system 10 that performs height estimation per grid cell and assesses the uncertainty of thisApplicant’s Ref.: 2024P00908WOAttorney’s Ref.: 9574410classification using entropy. The system adjusts its actions depending on the assessed confidence, thereby increasing reliability and safety during operation.Applicant’s Ref.: 2024P00908WOAttorney’s Ref.: 9574411Reference sign’s list10 Parking assistance system B1-B5 BlocksC1-C6 BlocksPathPath

Claims

Applicant’s Ref.: 2024P00908WOAttorney’s Ref.: 9574412Mercedes-Benz Group AG Negi 2026-01-29CLAIMS1. A method for operating an assistance system (10) of a vehicle, comprising the steps:- Capturing sensor data using at least one sensor device of the vehicle;- Generating a grid-based environmental map in which the surroundings of the vehicle are divided in multiple spatial areas and where each grid cell stores information about the presence and properties of objects;- Applying a trained neural network to detect relevant objects, predict a height value for each grid cell of the environmental map, and classify the predicted height of each grid cell into one of several predefined height classes,characterized in that- an uncertainty estimation is performed for each grid cell based on the entropy of the probability distribution over the height classes; and- the behaviour of the assistance system is adjusted depending on the uncertainty estimation.

2. The method according to claim 1 ,characterized in thatat least one warning signal is issued in case of high uncertainty in the height classification of one or more grid cells.

3. The method according to claim 1 or 2,characterized in thatat least one emergency control signal is issued in case of high uncertainty in the height classification.Applicant’s Ref.: 2024P00908WOAttorney’s Ref.: 95744134. The method according to any of the preceding claims,characterized in thatat least three discrete height classes are defined for classifying the predicted height of each grid cell.

5. The method according to any of the preceding claims,characterized in thata separate classification head is added to predict the height class based on the latent representation of the neural network, wherein the training of the original model remains unaffected by blocking the gradient flow from the classification head.

6. An assistance system (10) of a vehicle, comprising:- at least one sensor device for capturing sensor data,- an electronic device comprising a trained neural network to predict a height value for each of multiple spatial grid cells based on the captured sensor data and to classify the predicted height into at least one of several predefined height classes, and- a control unit to execute an action of the assistance system based on the classified height classes,characterized in that- the neural network is further configured to estimate uncertainty for each grid cell based on the entropy of the probability distribution over the height classes, and - the control unit is configured to adjust the behaviour of the assistance system (10) depending on the uncertainty estimation.

7. A computer program product comprising program code means for performing a method according to claim 1 to 6.

8. A non-transitory computer-readable storage medium comprising at least the computer program product according to claim 7.

Images