System and method for self-monitoring detection of free-space detection errors in an ADAS / ADS system

Abstract

The invention relates to a system for self-monitoring detection of free space detection errors in an autonomous driving assistance system (ADAS) or an autonomous driving system (ADS) of a vehicle (10), comprising: - a map module (200) that is trained to subdivide environmental maps (220) into clusters and to create cluster data (230) that represent specific environmental features; - a detection module (300) with an AI model (350) trained to generate free space detections (320) for the cluster data (230); - a comparison module (400) designed to compare the generated free space detections (320) with real free space detections (420) acquired by sensors (50) of the vehicle (10), and to calculate a characteristic value (450) to determine deviations between the generated free space detections (320) and the real free space detections (420); - an anomaly detection module (500) trained to identify anomalies in the real free-space detections (420) by applying a threshold to the characteristic value (450); and - a control module (700) that is trained to adapt the driving strategy of the ADAS / ADS system (70) of the vehicle (10) in a safety-conscious manner based on the identified anomalies.

Description

[0001] The invention relates to a system, a method and a computer program product for the self-monitoring detection of clearance detection errors in an autonomous driver assistance system (ADAS) or an autonomous driving system (ADS) of a vehicle.

[0002] Advanced Driver Assistance Systems (ADAS) are electronic systems in vehicles that support the driver in various driving situations to increase safety and comfort. These systems use a combination of sensors such as lidar and ultrasound, cameras, radar, and software to collect and process information about the environment and the vehicle. They can then react accordingly or warn the driver.

[0003] One example is Adaptive Cruise Control (ACC), which automatically adjusts the vehicle's speed to the traffic flow to maintain a safe distance from the vehicle ahead. Lane Keeping Assist detects lane markings and gently corrects the steering to help the driver keep the vehicle in its lane. Another important system is Emergency Braking, which detects potential collisions and initiates automatic braking in an emergency to avoid or mitigate an impact.

[0004] The Parking Assist system supports the driver when parking by calculating the optimal parking path and partially or completely taking over control of the vehicle. Traffic Sign Recognition detects traffic signs such as speed limits and displays this information to the driver.

[0005] These driver assistance systems help reduce human error, a common cause of traffic accidents. By supporting the driver in critical situations, they increase overall safety and driving comfort.

[0006] Autonomous driving goes a step further than driver assistance systems by aiming to control the vehicle completely without driver intervention (SAE Level 5). The technology not only provides assistance, but also takes complete control of vehicle guidance, navigation, and road decision-making. Autonomous vehicles utilize a variety of sensors, including LIDAR, radar, cameras, and ultrasonic sensors, which enable the vehicle to sense and respond to its surroundings in real time.

[0007] An autonomous vehicle must be able to make driving decisions necessary for navigating traffic, such as detecting obstacles, obeying traffic laws, and changing lanes, without human intervention. Complex software and algorithms enable the vehicle to understand and safely navigate various driving scenarios, such as turning, overtaking, or reacting to unexpected traffic situations.

[0008] Autonomous driving is divided into different levels depending on the degree of automation. In Level 1 (driver assistance), the driver is still fully involved in the driving process, while in Level 5 (fully autonomous), the vehicle can drive without any human control.

[0009] The transition from driver assistance systems to fully autonomous driving could further increase road safety by reducing human errors that lead to accidents. However, several technical challenges remain to be overcome, such as processing complex traffic situations and, in particular, interaction with other road users.

[0010] Freespace detection plays a key role here. It is responsible for providing the vehicle with an accurate picture of its surroundings, especially those areas that are free of obstacles and therefore safe to drive through. This technology ensures that the vehicle makes the right decisions by identifying the safe area on the road and avoiding obstacles such as other vehicles, pedestrians, or buildings.

[0011] Driver assistance systems generally rely on the driver maintaining control of the vehicle. ADAS systems, such as adaptive cruise control, emergency braking, or parking assist, rely on clearance detection to ensure the vehicle can operate safely. If an error occurs during clearance detection—for example, an obstacle is not detected or a clear space is incorrectly displayed as blocked—this can mislead the driver, cause the assistance system to make incorrect decisions, or unnecessarily prompt the driver to take control of the vehicle. These errors can lead to misinterpretations of the situation, compromising the safety and effectiveness of the driver assistance system.

[0012] In autonomous driving, where the vehicle operates without human intervention, errors in clear space detection are even more critical, as the vehicle assumes sole decision-making and control responsibility. In this case, a misjudgment of the drivable areas can directly lead to collision risks or incorrect route guidance. If the vehicle mistakenly assumes that an area is drivable when it is actually blocked or dangerous, there is a risk of collisions. Conversely, errors in clear space detection can cause the vehicle to perceive a drivable area as blocked, which can lead to lane avoidance and unnecessary driving maneuvers. This misinformation can negatively influence the vehicle's decision-making processes and thus endanger the safety of the overall system.

[0013] Common approaches to detecting these errors often use supervised learning techniques. This involves training models based on large, well-annotated datasets. These datasets often contain detailed information about which areas are passable and which are not. However, such annotated data is often time-consuming and expensive to create, as it requires the manual annotation of large volumes of images or sensor data. In practice, this means that the development and improvement of open space detection systems is severely limited by the need to collect large and accurate datasets.

[0014] Another problem is that even with large datasets, errors in annotation or model development can occur, further jeopardizing the reliability of open space detection. This makes the continuous improvement process even more complex and expensive.

[0015] DE 11 2019 000 049 T5 discloses a method for detecting and classifying objects in a field of view using machine learning. Data representing the positions of detected objects in the field of view is collected. Based on these positions, one or more clusters are formed that can represent objects. The features of the clusters are determined and used as input data for machine learning models. These models calculate a confidence value that represents the probability that a cluster actually corresponds to an object represented in the field of view.

[0016] The object of the invention is to provide a method and a system for continuously monitoring and optimizing clearance detection in a vehicle with an ADAS / ADS (Advanced Driver Assistance System / Automated Driving System) system. This method reliably detects anomalies in the clearance detections of the vehicle sensors and dynamically adapts the vehicle's driving behavior to the quality of the sensor data in order to ensure safe and adaptive vehicle control. Furthermore, the invention is intended to enable continuous improvement of the detection models of an ADAS / ADS system through the use of anomaly data in order to increase the reliability and safety of automated vehicle control.

[0017] This object is achieved according to the invention with respect to a method by the features of patent claim 1, with respect to a system by the features of patent claim 7, and with respect to a computer program product by the features of patent claim 12. The further claims relate to preferred embodiments of the invention.

[0018] The present invention enables the continuous monitoring and optimization of clearance detection in a vehicle with an ADAS / ADS (Advanced Driver Assistance System / Automated Driving System) system. The system uses a self-monitoring method to detect anomalies in the real-world clearance detection of the real environment captured by the vehicle's sensors.

[0019] A central component of the invention is the use of a comparison module that compares generated open space detections with the actual open space detections to verify consistency and identify potential errors. The anomaly detection module detects significant deviations between these two data sets and flags erroneous open space detections.

[0020] These detected anomalies are used to improve system reliability and the vehicle's driving strategy. The ADAS / ADS system can dynamically adapt driving behavior based on the reliability of the sensor and map data by reducing speed or activating a safety-aware driving strategy when anomalies are detected. For larger anomalies, the ADAS / ADS system can also make a route adjustment or prompt the driver to take control.

[0021] By integrating the system according to the invention into the software architecture of an ADAS / ADS system, the vehicle is enabled to dynamically adapt to new environmental and traffic situations and to drive safely even in less well-mapped areas.

[0022] According to a first aspect, the invention provides a system for self-monitoring detection of clear space detection errors in an autonomous driver assistance system (ADAS) or an autonomous driving system (ADS) of a vehicle. The system comprises a map module configured to divide environmental maps into clusters based on environmental features such as obstacle density or structural properties using a density-based clustering algorithm (DBSCAN) and to create cluster data representing specific environmental features; a detection module with an AI model configured to generate clear space detections for the cluster data based on the mapped environmental features in order to identify areas that can be classified as safe open space;a comparison module configured to compare the generated free-space detections with real free-space detections recorded by the vehicle's sensors and to calculate a characteristic value using difference metrics such as intersection over union (IoU) or mean absolute error (MAE) to determine deviations between the generated free-space detections and the real free-space detections; an anomaly detection module configured to identify anomalies in the real free-space detections by applying a threshold to the characteristic value, whereby deviations below a certain threshold are interpreted as an indication of errors in the free-space detection of the ADAS / ADS system and can be used to adjust the system parameters of the ADAS / ADS system;a control module designed to safety-consciously adapt the driving strategy of the vehicle's ADAS / ADS system based on the identified anomalies, whereby, in the event of detected errors, measures such as reducing speed or increasing the safety distance can be activated in order to increase vehicle safety.

[0023] In a further development, an optimization module is provided which is designed to store and systematically evaluate the detected anomalies and the associated free space detection errors for the continuous adaptation and improvement of the sensor models and the classification models of the ADAS / ADS system for free space detection.

[0024] In an advantageous embodiment, the environment maps contain precise geodata as well as information about traffic infrastructure and road geometry, including lanes, intersections, sidewalks and road signs, to enable detailed mapping and clustering of the environment.

[0025] In a further embodiment, the detection module comprises a convolutional neural network (CNN) as part of the AI ​​model, which is configured to recognize specific features within the cluster data that indicate drivable open spaces and, due to its layered structure, enables greater accuracy in object detection and open space classification.

[0026] Advantageously, the anomaly detection module comprises an adaptive threshold control that dynamically adjusts the threshold based on the specific environmental conditions or vehicle speed to ensure greater sensitivity in complex traffic conditions.

[0027] In particular, the detection module is designed to detect further relevant environmental parameters in addition to the free space detections and to forward this information to the comparison module for further processing.

[0028] According to a second aspect, the invention provides a method for self-monitoring detection of clearance detection errors in an autonomous driver assistance system (ADAS) or an autonomous driving system (ADS) of a vehicle. The method comprises the following steps: - Subdividing environmental maps into clusters based on environmental features such as obstacle density or structural properties using a density-based clustering algorithm (DBSCAN) to generate cluster data representing specific environmental features; - Generating open space detections using an AI model for the cluster data based on the mapped environmental features to identify areas that can be classified as safe open space for driving; - Comparing the generated free space detections with real free space detections recorded by the vehicle's sensors and calculating a characteristic value using difference metrics, such as Intersection over Union (IoU) or Mean Absolute Error (MAE), to determine deviations between the generated free space detections and the real free space detections; - Identifying anomalies in the actual free space detections by applying a threshold to the calculated characteristic value, whereby deviations below a certain threshold are considered as an indication of errors in the free space detection of the ADAS / ADS system and are used to adapt the system parameters of the ADAS / ADS system; - Adapting the driving strategy of the vehicle's ADAS / ADS system based on the identified anomalies by activating safety-conscious measures, such as reducing speed or increasing the safety distance, when errors are detected in order to increase vehicle safety.

[0029] A further development is intended for the environmental maps to contain precise geodata as well as information on traffic infrastructure and road geometry, including lanes, intersections, sidewalks and street signs, to enable detailed mapping and clustering of the environment.

[0030] In an advantageous embodiment, the detection module comprises a convolutional neural network (CNN) as part of the AI ​​model, which is configured to recognize specific features within the cluster data that indicate drivable open spaces and, due to its layered structure, enables greater accuracy in object detection and open space classification.

[0031] In a further embodiment, the anomaly detection module comprises an adaptive threshold control that dynamically adjusts the threshold based on the specific environmental conditions or the vehicle speed to ensure greater sensitivity in complex traffic conditions.

[0032] Advantageously, the detection module detects further relevant environmental parameters in addition to the free space detections and forwards this information to the comparison module for further processing.

[0033] According to a third aspect, the invention provides a computer program product comprising executable program code configured to carry out the method according to the first aspect when executed.

[0034] The invention is explained in more detail below with reference to embodiments shown in the drawing.

[0035] It shows: Fig. 1 is a block diagram illustrating an embodiment of a system according to the invention; Fig. 2 a flow chart explaining the individual method steps of a method according to the invention; Fig. 3 is a block diagram of a computer program product according to an embodiment of the third aspect of the invention.

[0036] Additional features, aspects and advantages of the invention or embodiments thereof will become apparent from the detailed description taken in conjunction with the claims.

[0037] Fig. 1 shows a system 100 according to the invention for self-monitoring detection of clearance detection errors in an autonomous driver assistance system (ADAS) or an autonomous driving system (ADS) of a vehicle 10. The system 100 comprises a map module 200, a detection module 300, a comparison module 400, an anomaly detection module 500, and a control module 700. In addition, an optimization module 800 may be provided.

[0038] These modules work together to automatically detect clearance detection errors, thereby improving the reliability of Advanced Driver Assistance Systems (ADAS) and Automated Driving Systems (ADS). The mapping module 200, the detection module 300, the comparison module 400, and the anomaly detection module 500 can be integrated into a central processing unit, such as a cloud computing infrastructure, to perform complex calculations and analyses. Furthermore, each of these modules can be equipped with a memory unit and / or a processor to support the required data processing.

[0039] In the context of the invention, a "module" is defined as a self-contained, specialized unit of software and / or hardware components. A module is designed to perform a specific function or task and is independent and self-contained, meaning it accepts specific inputs, performs internal processing, and then delivers specific outputs or results. A module can communicate with other modules or components via interfaces. These interfaces determine how data or commands are input to the module and how results or information are output.

[0040] In the context of the invention, a "processor" can be an electronic machine or circuit. For example, it can be a central processing unit (CPU), a microprocessor, a microcontroller, an application-specific integrated circuit (ASIC), or a digital signal processor. The processor can also be virtualized, e.g., as a virtual machine or soft CPU. It can also be a programmable processor configured through configuration steps to execute the method according to the invention and the corresponding modules or sub-aspects of the invention.

[0041] A "storage unit" or "memory module" refers to volatile memory such as random access memory (RAM) or permanent memory such as a hard drive or data storage device. Removable memory modules or cloud storage solutions can also be used as a storage unit.

[0042] Cloud computing offers the ability to scale resources as needed, allowing computing power, storage space, and network resources to be flexibly adapted to changing requirements. This scalability enables cost optimization and efficient resource allocation without large investments in hardware. Furthermore, users can access applications and data from anywhere with internet access, facilitating collaboration and seamless integration across devices and locations. Cloud computing also offers flexibility in software deployment, allowing applications to be updated and deployed quickly and continuously. Cryptographic encryption methods can be used to secure the connection to the cloud infrastructure over a cellular connection.

[0043] In the context of the invention, “data” means both raw data and already processed or prepared data, such as measurement results from sensors (e.g. cameras) or results from simulations.

[0044] Furthermore, communication connections are provided for the exchange and transmission of data between the individual modules, which can be implemented in particular as a wireless communication connection, e.g. as a mobile radio connection (e.g. 4G LTE, 5G, 6G) and / or as a near-field communication connection, e.g. Bluetooth ® , Ethernet, NFC (Near Field Communication) or Wi-Fi ® are trained.

[0045] The map module 200 is a central component of the self-monitoring open space detection and works closely with various data sources and algorithms to provide precise and reliable information about the environment. Environmental maps 220 form the basis for the segmentation and analysis of a vehicle's environment 10. In the field of autonomous driving and driver assistance systems, high-definition maps (HD maps) are typically used. HD maps provide a detailed representation of the environment and are significantly more accurate than conventional navigation maps. In addition to road data, they also contain information about lanes, road markings, traffic signs, traffic lights, and often even the exact position of curbs and objects. These HD maps contain data with centimeter-level accuracy, which is crucial for high-precision applications such as autonomous driving.

[0046] HD maps are generated from various sources to provide the most complete and accurate representation of the surroundings. These sources include, in particular, Lidar (Light Detection and Ranging) data, camera data, GPS data, and map providers. Lidar data: Lidar sensors generate detailed 3D point clouds of the surroundings, providing accurate elevation information and spatial structures. High-resolution cameras capture visual information to detect objects, road markings, and traffic signs. GPS and other positioning technologies enable precise localization and orientation on the map. Map providers such as OpenStreetMap (OSM) or commercial providers such as TomTom ® : Google Maps ® or Apple Maps ® , provide maps that are further refined using machine learning methods and algorithms.

[0047] The environment maps 220 are subdivided in the map module 200 to distinguish different areas of the environment based on certain features. Typical features that can be used for clustering are: - Obstacle density: Identifies areas with high or low obstacle density. For example, sidewalks or parking lots are less dense than busy roads. - Environmental structure: Areas are differentiated based on their physical structure, e.g. narrow alleys vs. wide streets or areas with vegetation vs. areas without vegetation.

[0048] The clusters thus generated serve as the basis for free-space detection. They help to better understand the environment and minimize potential errors in free-space detection.

[0049] The DBSCAN algorithm (Density-Based Spatial Clustering of Applications with Noise) is used for clustering. DBSCAN is a database-oriented, density-based clustering algorithm that is particularly suitable for detecting clusters with varying densities.

[0050] The DBSCAN algorithm considers data points to be clusters if they are close (within a specified radius) to a certain minimum number of points. Isolated points that do not have a minimum number of neighbors are classified as "noise" and are not included in the analysis. DBSCAN is particularly suitable for environments where clusters have different shapes and densities because it is not bound to geometric shapes.

[0051] DBSCAN is therefore used to segment the environment maps 220 to detect clusters of different densities and structures and ignore irrelevant points. This is particularly useful when analyzing complex cityscapes or other urban environments where different areas can differ greatly.

[0052] After segmenting the environment maps 220, cluster data 230 is generated, which serves as a structural representation of the environment. This cluster data 230 provides valuable information about the density and distribution of obstacles in the environment, the classification of the environment into drivable and non-driftable areas, and the identification of potential open spaces, which are later further processed in the detection module 300.

[0053] For efficient processing and reuse, both the environment maps 220 and the cluster data 230 are stored in a database 250. This central storage enables quick access to previously processed environment maps 220 and facilitates the integration of the information into other modules of the system 100, such as the detection module 300 and the anomaly detection module 400.

[0054] The map module 200 forms the basis for precise and reliable clear space detection. By applying DBSCAN and utilizing HD maps from a variety of data sources, the system 100 according to the invention achieves a solid foundation for identifying erroneous clear space detections and making driving decisions safer.

[0055] The detection module 300 generates free space detections 320 based on the environmental data 220 and the previously generated cluster data 230 using an AI model 350 that is specifically designed to identify and mark drivable areas within the given environment.

[0056] The AI ​​model 350 contains at least one neural network designed as a convolutional neural network (CNN). A neural network consists of neurons arranged in multiple layers and connected to each other in various ways. A neuron is capable of receiving information at its input from outside or from another neuron, evaluating it in a specific way, and passing it on in a modified form at the neuron's output to another neuron or outputting it as the final result. Between the input neurons and the output neurons are the so-called hidden neurons. Depending on the type of network, there can be several layers of hidden neurons. They ensure the forwarding and processing of information. The output neurons ultimately provide a result and pass it on to the outside world. The arrangement and interconnection of the neurons creates different types of neural networks, such as:Feedforward networks (FFNs), recurrent networks (RNNs), or convolutional neural networks (CNNs). The networks can be trained using unsupervised or supervised learning.

[0057] The convolutional neural network (CNN) is a specific type of artificial neural network. CNNs are particularly well-suited for processing image data, making them an optimal choice for open space detection. This is because CNNs are able to detect spatial features in the data and identify patterns necessary for distinguishing between drivable and non-driftable areas.

[0058] It has multiple convolutional layers and is well-suited for machine learning and artificial intelligence (AI) applications in the field of pattern recognition. A CNN works by processing image data in several steps. In the convolutional layer, the CNN analyzes the image in small, overlapping areas (patches) and identifies important features such as edges, shapes, and textures. In each layer, specific low-level patterns are learned, which are then combined into more complex features in deeper layers. Each convolutional layer is often followed by a pooling layer (reduction layer), which reduces the amount of data to be processed and thus optimizes computation time. By grouping similar information, the core content of the image is preserved, helping the network deliver stable and robust detections.The fully connected layers summarize the extracted features and produce the final output, in this case the decision as to which areas are classified as open spaces.

[0059] A CNN is useful because it is robust to environmental variations and can detect subtle details such as edges and contours, which are crucial for distinguishing between passable and non-passable areas. This makes it particularly accurate in real-time image analysis.

[0060] The input data for the detection module 300 is the cluster data 230, which contains information about the environmental structure (e.g., map sections, obstacle density, and other environmental features). Based on this, the detection module 300 generates open space detections 320, which represent drivable areas as marked map sections. These map sections are visual representations on which the open spaces are marked accordingly and thus form the basis for identifying the drivable areas.

[0061] In addition to the visual representation of the open spaces, a linguistic representation is provided, allowing each cluster to be classified into passable and non-passable areas. For this purpose, the AI ​​model 350 contains a language model that translates the CNN results into a text format useful for further applications or for communication with other modules within an ADAS or ADS system.

[0062] A language model can be useful for providing the clear-space detections in a form that is easily accessible and understandable for the system, thus facilitating the integration of the clear-space data into a centralized, comprehensive vehicle control system. It also offers flexibility in output, as both visual and textual information are available.

[0063] The AI ​​model 350 is trained using a training dataset 370 with real detection data to achieve high accuracy in detecting open spaces. The AI ​​model 350 is trained through a process in which it is repeatedly applied to image and sensor data from real driving situations. This training data 370 already contains annotations indicating which areas are passable (open spaces) and which are not. These annotations serve as "ground truth"—the actual, correct values ​​that the AI ​​model 350 uses as a basis for comparison.

[0064] The AI ​​model 350 receives real-world environmental data and map sections divided into clusters that distinguish drivable from non-driftable areas. This training data 370 is preprocessed and optimized to enable efficient and meaningful training.

[0065] In each training step, the AI ​​model 350 makes a prediction about which areas of the image should be classified as drivable. These predictions are based on the current parameters of the neural network and reflect how well the AI ​​model 350 is trained at that point.

[0066] After the AI ​​model 350 has made a prediction about open spaces in an image, the loss is calculated - that is, the difference between the actual classification (whether an area is passable or not) and the prediction of the AI ​​model 350.

[0067] For this purpose, a binary cross-entropy loss function is used. This type of loss function was specifically developed to achieve particularly accurate results in classification tasks with only two classes (here: passable and non-passable).

[0068] The binary cross-entropy loss function calculates how well or poorly the AI ​​model 350's predictions match the actual values. It is based on the probability with which the AI ​​model 350 classifies each area as passable or impassable and compares these probabilities with the actual classifications.

[0069] High confidence in correct predictions leads to lower loss, while low or incorrect predictions increase the loss. The sum of the losses across all data points results in the total training loss.

[0070] The AI ​​model 350 is continuously adjusted to minimize this loss. An optimization algorithm (e.g., stochastic gradient descent) changes the neural network's parameters (i.e., the "weights" in the nodes) so that the predictions gradually improve. The goal of the optimization process is to enable the AI ​​model 350 to make increasingly accurate predictions as it is trained.

[0071] The result of the training is an AI model 350 that can learn to accurately detect open spaces because it has adjusted its parameters based on the calculated losses. Through this process of loss calculation and optimization, the detection performance of the AI ​​model 350 is continuously improved. The trained AI model 350 is then able to reliably detect open spaces in new, unknown environmental data and is available for real-time application in advanced driver assistance systems (ADAS) or autonomous driving systems (ADS).

[0072] The comparison module 400 is designed to check the correspondence between the open space detections 320 generated in the AI ​​model 350 and the open space detections 420 actually recorded by the vehicle 10. The goal of this comparison is to evaluate the accuracy of the model predictions and to ensure that the detected open spaces marked as passable correspond to the real conditions.

[0073] The comparison module 400 is designed to check the correspondence between the real open space detections 420 actually recorded by the vehicle 10 and the open space detections 320 generated by the AI ​​model 350. The goal of this comparison is to evaluate the timeliness and accuracy of the real open space detections 420 and to ensure that the detected open spaces marked as passable correspond to the actual conditions in the environment, i.e., the extent to which the data recorded by the sensors 50 of the vehicle 10 correspond to the real environment.

[0074] The vehicle 10 captures real-world environmental data using various sensors 50, such as lidar, radar, cameras, and ultrasonic sensors. These sensors 50 continuously provide information about the surroundings of the vehicle 10, which is then processed and analyzed to detect obstacles, lane markings, buildings, and other structures. The resulting data, referred to as real-world clear space detection 420, represents the areas that are currently passable and do not contain any obstacles. These real-world clear space detections 420 are based on the real-time inputs from the sensors 50 and are then compared with the clear space detections 320 generated by the AI ​​model 350.

[0075] The comparison between the generated open space detections 320 and the real open space detections 420 is performed at identical map positions to ensure that the data originates from the same environmental sections. The metrics Intersection over Union (IoU) and Mean Absolute Error (MAE) are typically used for the comparison and to quantify the similarity or discrepancy.

[0076] The Intersection over Union (IoU) metric measures the degree of overlap between two areas - in this case between the generated open space detections 320 and the real open space detections 420. IoU calculates the ratio of the intersection of the two areas (i.e., the area detected as drivable in both the generated open space detections 320 and the real open space detections 420) to the union of these areas (i.e., the area detected as drivable in either the generated open space detections 320 or the real open space detections 420).

[0077] An IoU value close to 1 indicates that the generated free space detections 320 and the actual free space detections 420 are nearly identical. This indicates a high accuracy of the sensors 50 and the processing software of the vehicle 10. An IoU value close to 0, on the other hand, indicates that there are significant differences between the two surfaces, indicating possible errors in the sensors 50, the model prediction, or the processing of the environmental data.

[0078] The Mean Absolute Error (MAE) calculates the mean absolute deviation between the generated free-space detections 320 and the actual free-space detections 420 at a set of data points. The MAE therefore measures the average difference between the predictions and the actual values ​​without considering the signs of the deviations, so only the magnitude of the deviations is assessed. A low MAE value indicates that the generated free-space detections 320 and the actual free-space detections 420 are close to each other on average. A high MAE indicates that the detections exhibit larger differences.

[0079] The comparison module 400 provides a characteristic value 450 as a result, which quantifies the similarity or deviation between the generated free-space detections 320 and the actual free-space detections 420. A high characteristic value 450 can indicate a high degree of similarity and thus a high degree of reliability of the sensors 50 and the processing of the environmental data, while a low characteristic value 450 indicates larger deviations. The characteristic value 450 makes it possible to continuously monitor the accuracy of the free-space detection and, in the event of deviations, to make adjustments to increase the reliability of the detection systems.

[0080] The comparison thus checks the accuracy of the real free space detection 420 of the vehicle 10 and evaluates the quality of the environmental data recorded by the vehicle sensors 50.

[0081] The anomaly detection module 500 serves to identify possible errors in the real free space detection 420 based on the calculated characteristic values ​​450. In doing so, it detects anomalous deviations between the generated free space detections 320 and the real free space detections 420. To detect significant deviations, certain threshold values ​​are set for the characteristic value 450. If, for example, the characteristic value 450 falls below a defined threshold, the corresponding real free space detection 420 is marked as anomalous free space detection 520 and classified as faulty. This faulty detection can be an indication of a possible detection error in the sensors 50 of the vehicle 10 used for environment detection.

[0082] The anomaly detection module 500 transmits the information about the detected faulty free space detections 520 to the control module 700. Furthermore, the anomaly detection module 500 can also capture details about the type and extent of the deviation and store them in the database 250. This information can be retrieved later for further processing steps and to improve the detection algorithms.

[0083] Additionally, the anomaly detection module 500 may include adaptive threshold control that dynamically adjusts the threshold based on specific environmental conditions or vehicle speed to ensure greater sensitivity in complex traffic conditions.

[0084] The control module 700 is configured to adapt the driving strategy of the vehicle 10 based on the detected anomalies. If errors in the clearance detection are detected, the control module 700 can activate a safety-conscious driving style to ensure the safety of the vehicle 10. This may include, among other things, reducing the speed or increasing the safety distance in order to appropriately respond to the detected uncertainty in the clearance detection.

[0085] In a further development, an offline application is also conceivable. This would allow the free space detection to be performed iteratively in a simulated environment to specifically improve and optimize the ADAS / ADS system. Such an offline application would enable testing of various scenarios and configurations under controlled conditions, thus continuously improving the performance and reliability of the ADAS / ADS system without affecting real-time operation.

[0086] The detection of anomalies in clear-space detection is crucial for increasing the safety and reliability of ADAS / ADS systems (Advanced Driver Assistance Systems / Automated Driving Systems). For this reason, the system 100 or method according to the invention is integrated into the software architecture of an ADAS / ADS system 70 to enable control and adaptation of the driving style depending on the quality of the clear-space detection.

[0087] Integrating anomaly detection into the main software of an ADAS / ADS system 70 enables continuous monitoring of the environment detection. This ensures that the ADAS / ADS system 70 can take immediate action when anomalies are detected that indicate potential errors in the clearance detection.

[0088] ADAS / ADS systems 70 can be designed to dynamically adapt their driving behavior to the reliability of the sensor and map data. If many erroneous clear-space detections 520 are detected, the confidence in the clear-space detection of the vehicle sensors 50 decreases due to the detected anomalies. In such cases, the ADAS / ADS system 70 could, for example, reduce the speed of the vehicle 10 or switch to a more cautious driving strategy.

[0089] Furthermore, the ADAS / ADS system 70 could perform dynamic route adjustment. If the comparison module 500 determines that the clearance detection of a drivable area is inaccurate or faulty, the planned route could be recalculated to avoid potentially dangerous areas and maximize safety.

[0090] If the confidence in detection falls below a specified threshold, the ADAS / ADS system 70 could prompt the driver to take control, depending on the level of automation (e.g., SAE Level 3). At a higher level of automation (Levels 4-5), the ADAS / ADS system 70 could switch to a safe mode or activate an evasive strategy.

[0091] In situations with lower reliability of free space detection, the ADAS / ADS system 70 could implement a safety-oriented driving strategy, such as reducing speed or increasing the safety distance, to ensure safety.

[0092] Furthermore, continuous anomaly detection provides valuable data that can be used for long-term optimization of open space detection. ADAS / ADS systems 70 could regularly use this information to improve their data-based detection models and thus increase their ability to detect complex environmental scenarios. Therefore, according to the invention, an optimization module 800 is provided, which is configured to store and systematically evaluate the detected anomalies and the associated open space detection errors for the continuous adaptation and improvement of the sensor models and the classification models of the ADAS / ADS system 70 for open space detection.

[0093] Through self-monitoring anomaly detection, the ADAS / ADS system 70 is continuously evolving. It can better adapt to new environmental and traffic conditions and specifically optimize clear-space detection. The detected erroneous clear-space detections are not only considered in isolation but also analyzed in a higher-level AI model 850 of the optimization module 800, which identifies patterns and correlations between the detection errors and the underlying sensor or environmental conditions.

[0094] This higher-level AI model 850 plays a key role in the continuous improvement of the ADAS / ADS system 70 by optimizing the sensor layout and refining the classification models. Based on the collected anomaly data, the AI ​​model 850 can make targeted adjustments that increase detection accuracy in the long term. This not only reacts to individual error sources, but also allows recurring error patterns to be identified and systematically corrected.

[0095] Through this intelligent adaptation and optimization, the ADAS / ADS system 70 is capable of dynamically adapting to changing environmental conditions and reliably detecting even complex traffic situations. The higher-level AI model 850 thus contributes to continuous improvement in open space detection and enables increasingly precise control of the vehicle 10.

[0096] It is also conceivable that incorrectly detected free spaces could be shared with other vehicles via V2V (vehicle-to-vehicle) communication. This could reduce the dependence on high-resolution maps in the vehicle's surroundings.

[0097] A particularly advantageous aspect is the possibility of adaptive driving in areas with limited map availability. When high-resolution (HD) maps are unavailable or incomplete, the ADAS / ADS system 70 can still operate reliably. This is achieved by the ADAS / ADS system 70 dynamically adapting its decisions to the actual detection accuracy and continuing to make safe decisions by comparing the generated clear-space detections 320 with the actual clear-space detections 420. In this way, the reliability of the vehicle 10 is ensured in less well-mapped areas.

[0098] In a further development of the invention, the system architecture according to the invention can also be used to detect and process additional environmental parameters in order to further increase the adaptability and safety of the ADAS / ADS system 70. By using the AI ​​model 350, the system 100 according to the invention is capable of continuously monitoring the driving environment. Thus, in addition to the free space detection 320, the detection module 300 can identify and analyze additional environmental parameters that are crucial for a safe driving strategy. The AI ​​model 350 is trained with corresponding training data 370 in order to reliably identify and analyze the relevant environmental parameters.

[0099] These additional environmental parameters include, for example, road conditions and structural properties of the surrounding area. For example, the inclination angle of the road surface can be analyzed to detect inclines and declines that can influence braking distances. Narrow sections and changes in road width are also detected, allowing the vehicle 10 to react precisely to narrow sections. Road and lane markings can also be recognized, allowing the ADAS / ADS system 70 to navigate reliably even with unclear markings or in poorly marked areas. Furthermore, the AI ​​model 350 can analyze traffic signs such as speed limits or construction site warnings and dynamically incorporates this information into the driving strategy.

[0100] All of these environmental parameters are fed to the comparison module 500, which compares them with the real-time conditions detected by the vehicle sensors 50. In this way, the system 100 according to the invention enables situation-appropriate, safety-conscious, and comfort-oriented driving through continuous self-monitoring.

[0101] In Fig. 2 shows the method steps for the self-monitoring detection of free space detection errors in an autonomous driver assistance system (ADAS) or an autonomous driving system (ADS) of a vehicle 10.

[0102] In a step S10, environmental maps 220 are divided into clusters based on environmental features such as obstacle density or structural properties using a density-based clustering algorithm (DBSCAN) to generate cluster data 230 representing specific environmental features.

[0103] In a step S20, open space detections 320 are generated by an AI model 350 for the cluster data 230 based on the mapped environmental features in order to identify areas that can be classified as safe open space for driving.

[0104] In a step S30, the generated free space detections 320 are compared with real free space detections 420 detected by sensors 50 of the vehicle 10, and a characteristic value 450 is calculated using difference metrics, such as intersection over union (IoU) or mean absolute error (MAE), to determine deviations between the generated free space detections 320 and the real free space detections 420.

[0105] In a step S40, anomalies in the real free space detections 420 are identified by applying a threshold value to the calculated characteristic value 450, wherein deviations below a certain threshold value are evaluated as an indication of errors in the free space detection of the ADAS / ADS system 70 and are used to adapt the system parameters of the ADAS / ADS system 70.

[0106] In a step S50, the driving strategy of the ADAS / ADS system 70 of the vehicle 10 is adapted based on the identified anomalies by activating safety-conscious measures, such as reducing the speed or increasing the safety distance, in order to increase vehicle safety when errors are detected.

[0107] Fig.3 schematically illustrates a computer program product 900 comprising executable program code 950 configured to perform the method according to the first aspect of the present invention.

[0108] The present invention enables the continuous monitoring and optimization of clear space detection in a vehicle with an ADAS / ADS system. The system according to the invention uses a self-monitoring method to detect anomalies in the real-world clear space detection of the real environment captured by the vehicle's sensors.

[0109] A central component of the invention is the use of a comparison module that compares generated open space detections with the actual open space detections to verify consistency and identify potential errors. The anomaly detection module detects significant deviations between these two data sets and flags erroneous open space detections.

[0110] These detected anomalies are used to improve system reliability and the vehicle's driving strategy. The ADAS / ADS system can dynamically adapt driving behavior based on the reliability of the sensor and map data by reducing speed or activating a safety-aware driving strategy when anomalies are detected. For larger anomalies, the ADAS / ADS system can also make a route adjustment or prompt the driver to take control.

[0111] By integrating the system according to the invention into the software architecture of an ADAS / ADS system, the vehicle is enabled to dynamically adapt to new environmental and traffic situations and to drive safely even in less well-mapped areas. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] DE 11 2019 000 049 T5

[0015]

Claims

[1] System for self-monitoring detection of free space detection errors in an autonomous driver assistance system (ADAS) or an autonomous driving system (ADS) of a vehicle (10), comprising: - a map module (200) configured to divide environmental maps (220) into clusters based on environmental features such as obstacle density or structural properties using a density-based clustering algorithm (DBSCAN) and to create cluster data (230) representing specific environmental features; - a detection module (300) with an AI model (350) designed to generate open space detections (320) for the cluster data (230) based on the mapped environmental features in order to identify areas that can be classified as safe open space; - a comparison module (400) designed to compare the generated free space detections (320) with real free space detections (420) recorded by sensors (50) of the vehicle (10) and to calculate a characteristic value (450) using difference metrics such as intersection over union (IoU) or mean absolute error (MAE) in order to determine deviations between the generated free space detections (320) and the real free space detections (420); - an anomaly detection module (500) designed to identify anomalies in the real free space detections (420) by applying a threshold value to the characteristic value (450), wherein deviations below a certain threshold value are evaluated as an indication of errors in the free space detection of the ADAS / ADS system (70) and can be used to adapt the system parameters of the ADAS / ADS system (70); and - a control module (700) designed to safety-consciously adapt the driving strategy of the ADAS / ADS system (70) of the vehicle (10) based on the identified anomalies, wherein, in the event of detected errors, measures such as a reduction in speed or an increase in the safety distance can be activated in order to increase vehicle safety. [2] System (100) according to claim 1, wherein an optimization module (800) is designed to store and systematically evaluate the detected anomalies and the associated free space detection errors for the continuous adaptation and improvement of the sensor models and the classification models of the ADAS / ADS system (70) for free space detection. [3] The system (100) of claim 1 or 2, wherein the environment maps (220) include precise geospatial data and information about traffic infrastructure and road geometry, including lanes, intersections, sidewalks, and road signs, to enable detailed mapping and clustering of the environment. [4] System (100) according to one of the preceding claims, wherein the detection module (300) comprises a convolutional neural network (CNN) as part of the AI ​​model (350), which is configured to recognize specific features within the cluster data (230) that indicate drivable open spaces and, due to its layered structure, enables greater accuracy in object detection and open space classification. [5] The system (100) of any preceding claim, wherein the anomaly detection module (500) comprises an adaptive threshold controller that dynamically adjusts the threshold based on the specific environmental conditions or vehicle speed to ensure greater sensitivity in complex traffic conditions. [6] System (100) according to one of the preceding claims, wherein the detection module (300) is configured to detect further relevant environmental parameters in addition to the free space detections (320) and to forward this information to the comparison module (400) for further processing. [7] Method for the self-monitoring detection of free space detection errors in an autonomous driver assistance system (ADAS) or an autonomous driving system (ADS) of a vehicle (10), comprising the method steps: - subdividing (S10) environmental maps (220) into clusters based on environmental features such as obstacle density or structural properties using a density-based clustering algorithm (DBSCAN) to generate cluster data (230) representing specific environmental features; - generating (S20) open space detections (320) by an AI model (350) for the cluster data (230) based on the mapped environmental features in order to identify areas that can be classified as safe open space; - comparing (S30) the generated free space detections (320) with real free space detections (420) recorded by sensors (50) of the vehicle (10) and calculating a characteristic value (450) using difference metrics, such as intersection over union (IoU) or mean absolute error (MAE), to determine deviations between the generated free space detections (320) and the real free space detections (420); - identifying (S40) anomalies in the real free space detections (420) by applying a threshold value to the calculated characteristic value (450), wherein deviations below a certain threshold value are evaluated as an indication of errors in the free space detection of the ADAS / ADS system (70) and are used to adapt the system parameters of the ADAS / ADS system (70); - adapting (S50) the driving strategy of the ADAS / ADS system (70) of the vehicle (10) based on the identified anomalies by activating safety-conscious measures, such as reducing the speed or increasing the safety distance, in case of detected errors in order to increase vehicle safety. [8] The method of claim 7, wherein the environment maps (220) include precise geospatial data and information about traffic infrastructure and road geometry, including lanes, intersections, sidewalks, and road signs, to enable detailed mapping and clustering of the environment. [9] The method according to claim 7 or 8, wherein the detection module (300) comprises a Convolutional Neural Network (CNN) as part of the AI ​​model (350), which is configured to detect specific features within the cluster data (230) that indicate drivable open spaces and, due to its layered structure, enables greater accuracy in object detection and open space classification. [10] The method of any one of claims 7 to 9, wherein the anomaly detection module (500) comprises an adaptive threshold control that dynamically adjusts the threshold based on the specific environmental conditions or the vehicle speed to ensure greater sensitivity in complex traffic conditions. [11] Method according to one of claims 7 to 10, wherein the detection module (300) detects further relevant environmental parameters in addition to the free space detections (320) and forwards this information to the comparison module (400) for further processing. [12] A computer program product (900) comprising an executable program code (950) configured to carry out the method according to any one of claims 7 to 11 when executed.

Images