Method and system for identifying representative route sections

Abstract

The invention relates to a method for identifying representative road sections (S1, S2, ..., Sn) for testing driver assistance functions and / or driving functions, comprising the following method steps: - Recording of sensor data (250) by at least one sensor of a sensor device of a vehicle (200) while driving on a multitude of routes with track sections (S1, S2, ..., Sn); - Generating a data set (350) for each route segment (Sj) by data fusion of the various sensor data (250) and other data (270) such as map data; - Extracting scenario attributes (A1, A2, ..., An) from each data record (350) of a route segment (Sj); - Transforming the scenario attributes (A1, A2, ..., An) of a route segment (Sj) into a multidimensional route segment vector (585); - Training an embedding (590) with a neural network (570) by embedding the segment vectors (585) into a multidimensional vector space (580) according to the degree of their similarity; - Analyzing the clusters in the embedding (590) using clustering algorithms to identify similar segments (Si, Sj); - Generating output data (750) regarding the frequency of road segments (S1, S2, ..., Sn) with specific scenario attributes (A1, A2, ..., An).

Description

[0001] The invention relates to a method, a system and a computer program product for identifying representative route sections for testing driver assistance functions and / or driving functions.

[0002] Driver assistance functions and driving functions are two different types of systems used in modern vehicles to increase safety and driving comfort. The main difference is that driver assistance functions support the driver while the driver maintains control of the vehicle, while driving functions are capable of acting autonomously and performing certain driving tasks without human intervention. One example of driving functions that can achieve a high degree of automation is autonomous vehicles.

[0003] Driver assistance functions support the driver in driving the vehicle and relieve them of certain tasks to make driving safer and more comfortable. They are usually partially automated, meaning the driver retains full responsibility for the vehicle and the driving task. Examples of driver assistance functions include parking assistance, adaptive cruise control, lane departure warning, warning functions, and automatic brake support. These functions use sensors and cameras to collect information about the vehicle's surroundings and can trigger certain actions such as braking, steering, or warnings to the driver.

[0004] Driving functions, especially for autonomous vehicles, aim to operate the vehicle autonomously in various traffic situations without human intervention. They are fully or highly automated and can perform certain driving tasks without the need for a human driver. Examples of driving functions include autonomous vehicles capable of driving autonomously and navigating various traffic situations. These functions utilize advanced sensors, cameras, maps, and artificial intelligence to perform driving tasks such as accelerating, braking, steering, and navigating in complex traffic situations without the intervention of a human driver.

[0005] Both driver assistance functions and driving functions are subjected to validation to demonstrate their performance, functionality, and reliability. The validation of a driver assistance function or a driving function each comprises a complex process for testing and verifying their performance, functionality, and safety. The goal is to ensure that the driver assistance function or driving function works as intended, meets legal requirements, and reliably supports the driver. For example, in a driver assistance system (ADAS), the individual system components, such as adaptive cruise control, lane departure warning, or collision warning, are tested both individually and in combination. The functional tests verify whether the ADAS system is working correctly, provides the driver with precise feedback, and responds appropriately to different scenarios.An important aspect of ADAS systems and driving functions is the calibration of the sensors to check their accuracy and alignment and ensure that they provide consistent and reliable data.

[0006] To validate various driver assistance functions or driving functions, such as overtaking maneuvers, test drives are conducted regularly. Various test cases are used to ensure that the driver assistance functions or driving functions function correctly under various conditions. These test cases can be simulated or performed under real-life conditions.

[0007] When testing driver assistance and driving functions for semi-autonomous or autonomous vehicles, it is important to achieve balanced and statistically representative test coverage. This means that test drives must be planned to cover a broad range of real-world driving scenarios. Inadequate planning can lead to certain scenarios being overemphasized and others neglected. If certain scenarios are overrepresented, driver assistance or driving functions may perform well in those scenarios, but this does not necessarily mean that they will function safely and reliably in all situations, making the test drive results inconclusive. Furthermore, driving many test kilometers in already well-understood scenarios is a waste of resources, as these kilometers do not provide any new insights into the behavior of the driver assistance or driving function.

[0008] Therefore, it is important to conduct testing with relevant driving scenarios and to plan the testing in such a way that each of these driving scenarios is adequately covered. This is the only way to ensure that semi-autonomous or autonomous vehicles with driver assistance and / or driving functions operate safely and reliably in a wide range of real-world environments.

[0009] DE 10 2021 203 057 A1 discloses a method for controlling a vehicle in which geometric test data and driving dynamics test data are acquired on a test track. The test data are divided into clusters, and cluster-specific driving dynamics data are defined for each cluster. Finally, the route to be traveled is clustered, and the vehicle is controlled in a section of the track according to the defined driving dynamics data of the respective cluster.

[0010] The invention is based on the object of creating possibilities for improving the planning of real test drives for semi-autonomous or autonomous vehicles with driver assistance functions and / or driving functions in order to achieve test coverage with statistically representative scenarios and thus improve the safety and reliability of vehicles with driver assistance functions and / or driving functions.

[0011] This object is achieved according to the invention with respect to a first method by the features of patent claim 1, with respect to a second method by the features of claim 4, with respect to a system by the features of patent claim 9, and with respect to a computer program product by the features of patent claim 15. The further claims relate to preferred embodiments of the invention.

[0012] By generating an embedding of route section vectors containing scenario attributes of the respective route section according to the invention, clusters of route sections that are similar to one another can be formed. This allows representatively selected routes to be analyzed as representatives of a larger overall set. This enables targeted planning of test drives in order to cover specific scenarios in a targeted manner. From the behavior of a driver assistance or driving function on a specific route section, conclusions can be drawn about the behavior on similar route sections. In particular, the method according to the invention enables a route-section-specific release of a driver assistance or driving function. By incorporating data on suitable times / weather conditions, release can also be made in a more targeted and granular manner.

[0013] According to a first aspect, the invention provides a method for identifying representative route sections for testing driver assistance functions and / or driving functions for a vehicle. The method comprises the following method steps: - Recording sensor data by at least one sensor such as a camera, a lidar system, a radar system and / or an ultrasound system of a sensor device of a vehicle when driving a plurality of routes, wherein each route comprises a plurality of route sections and wherein each route section can in turn comprise a plurality of scenarios; - Creating a data set for each route section by fusing the sensor data from different sensors and other data such as map data; - Extracting scenario attributes from each dataset of a route section; - Transforming the scenario attributes of a route section into a multidimensional route section vector; - Training an embedding with a neural network by embedding the route section vectors in a multidimensional vector space, wherein the embedding of the route section vectors is carried out according to the degree of their similarity, so that similar route sections with similar scenario attributes form clusters; - Analyzing the clusters in the embedding using clustering algorithms to identify similar route sections; - Generating output data regarding the frequency of road sections with specific scenario attributes, particularly in the form of diagrams, graphs, text messages, video sequences and / or PowerPoint presentations.

[0014] In a further development, it is provided that a scenario attribute represents a physical quantity, a chemical quantity, a torque, a rotational speed, a voltage, an amperage, an acceleration, a speed, a braking value, a direction, an angle, a radius, a location, a number, a moving object such as a motor vehicle, a person or a cyclist, an immovable object such as a building or a tree, a road configuration such as a motorway, a traffic sign, a traffic light, a tunnel, a roundabout, a turning lane, a traffic density, a topographical structure such as a gradient, a time, a temperature, a precipitation value, a weather condition and / or a season.

[0015] In an advantageous embodiment, it is provided that the scenario attributes are selected within the framework of a 5-layer model, wherein a first layer relates to the road geometry and topography, a second layer contains information about the roadside development, a third layer contains information about construction sites and temporary changes on the road, a fourth layer contains the trajectories of dynamically moving objects and a fifth layer relates to environmental influences such as weather conditions, time of day and lighting conditions as well as the road condition.

[0016] In a further embodiment, it is provided that the network architecture of the neural network is designed as a simple feed-forward network (FFN) or as a more complex model, such as in particular a convolutional neural network (CNN) or a recurrent neural network (RNN).

[0017] Advantageously, cosine similarity is used to embed the route section vectors into the vector space with regard to their similarity.

[0018] In particular, deep metric learning and / or constraint pretraining are used to train the neural network to create the embedding.

[0019] In a further embodiment, it is provided that the weights of one or more scenario attributes are adjusted during the training process.

[0020] Advantageously, hierarchical clustering algorithms such as mean-shift clustering and / or partitioning clustering algorithms such as K-Means are used as clustering algorithms.

[0021] According to a second aspect, the invention provides a system for identifying representative route sections for testing driver assistance functions and / or driving functions for a vehicle. The system comprises an input module with a database, an extraction module, a data evaluation module, and an output module. A sensor device of a vehicle is designed to record sensor data by at least one sensor such as a camera, a lidar system, a radar system, and / or an ultrasound system when driving along a plurality of routes, wherein each route comprises a plurality of route sections and wherein each route section can in turn comprise a plurality of scenarios. The input module is designed to generate a data set for each route section by data fusion of the sensor data from various sensors and other data such as map data.The extraction module is designed to extract scenario attributes from each data set of a route section. The data evaluation module comprises a software application designed to transform the scenario attributes of a route section into a multidimensional route section vector and to train an embedding with a neural network by embedding the route section vectors in a multidimensional vector space. The embedding of the route section vectors is performed according to their degree of similarity, so that similar route sections with similar scenario attributes form clusters. The software application includes clustering algorithms for analyzing the clusters in the embedding to identify similar route sections.The output module is designed to generate output data regarding the frequency of road sections with certain scenario attributes, in particular in the form of diagrams, graphics, text messages, video sequences and / or PowerPoint presentations.

[0022] In a further development, it is provided that a scenario attribute represents a physical quantity, a chemical quantity, a torque, a rotational speed, a voltage, an amperage, an acceleration, a speed, a braking value, a direction, an angle, a radius, a location, a number, a moving object such as a motor vehicle, a person or a cyclist, an immovable object such as a building or a tree, a road configuration such as a motorway, a traffic sign, a traffic light, a tunnel, a roundabout, a turning lane, a traffic density, a topographical structure such as a gradient, a time, a temperature, a precipitation value, a weather condition and / or a season.

[0023] In an advantageous embodiment, it is provided that the scenario attributes are selected within the framework of a 5-layer model, wherein a first layer relates to the road geometry and topography, a second layer contains information about the roadside development, a third layer contains information about construction sites and temporary changes on the road, a fourth layer contains the trajectories of dynamically moving objects and a fifth layer relates to environmental influences such as weather conditions, time of day and lighting conditions as well as the road condition.

[0024] Advantageously, the network architecture of the neural network (570) is designed as a simple feed-forward network (FFN) or as a more complex model, such as in particular a convolutional neural network (CNN) or a recurrent neural network (RNN).

[0025] In particular, cosine similarity is used to embed the route section vectors into the vector space with regard to their similarity.

[0026] In particular, deep metric learning and / or constraint pretraining are used to train the neural network to create the embedding.

[0027] In particular, the weights of one or more scenario attributes are adjusted during the training process.

[0028] Advantageously, hierarchical clustering algorithms such as mean-shift clustering and / or partitioning clustering algorithms such as K-Means are used as clustering algorithms.

[0029] According to a third aspect, the invention provides a computer program product comprising executable program code configured to carry out the method according to the first aspect when executed.

[0030] The invention is explained in more detail below with reference to embodiments shown in the drawing.

[0031] It shows: Fig. 1 is a block diagram illustrating an embodiment of a system according to the invention; Fig. 2 a schematic representation of an embedding according to the invention; Fig. 3 a flow chart to explain the individual method steps of a first method according to the invention; Fig. 4 a schematic representation of a computer program product.

[0032] Additional features, aspects and advantages of the invention or embodiments thereof will become apparent from the detailed description taken in conjunction with the claims.

[0033] Fig. Figure 1 shows a system 100 according to the invention for identifying representative route sections, in particular for conducting test drives to test driver assistance functions and / or driving functions of a vehicle. The system 100 comprises an input module 300 with a database 370, an extraction module 400, a data evaluation module 500, and an output module 700. The vehicle can be configured as a motor vehicle, an automated vehicle, an agricultural vehicle such as a combine harvester, a production or service robot, a watercraft, or a flying object such as a drone.

[0034] The input module 300, the extraction module 400, and the data evaluation module 500 can each be equipped with a processor and / or a memory unit. In particular, the extraction module 400 and the data evaluation module 500 can be integrated into a cloud computing infrastructure that is connected to the input module 300 via a cellular connection.

[0035] In the context of the invention, a “processor” can be, for example, a machine or an electronic circuit. A processor can in particular be a central processing unit (CPU), a microprocessor or a microcontroller, e.g. an application-specific integrated circuit or a digital signal processor, possibly in combination with a memory unit for storing program instructions. A processor can also be a virtualized processor, a virtual machine or a soft CPU. It can also be, for example, a programmable processor which is equipped with configuration steps for carrying out the aforementioned method according to the invention or is configured with configuration steps such that the programmable processor implements the inventive features of the method, the component, the modules or other aspects and / or sub-aspects of the invention.In particular, the processor can contain highly parallel computing units and powerful graphics modules.

[0036] In the context of the invention, a "storage unit" or "storage module" and the like can be understood as, for example, a volatile memory in the form of random access memory (RAM), a permanent memory such as a hard drive or a data storage device, or, for example, a removable storage module. The storage module can also be a cloud-based storage solution.

[0037] In the context of the invention, a "module" can be understood, for example, as a processor and / or a memory unit for storing program instructions. For example, the processor is specifically configured to execute or implement the program instructions of the method according to the invention or a step of the method according to the invention.

[0038] A vehicle 200 or a group of vehicles 200 travels a plurality of routes, each of which is divided into a plurality of route sections S1, S2, ..., Sn. A route section Sj of a route can be defined differently within the scope of the invention depending on requirements and context. For example, a route section can be defined by geographical coordinates that determine the start and end points of the route section. This can be done, for example, by longitude and latitude or other coordinate systems. However, a route section can also be defined by a specific number of road kilometers. Furthermore, a route section can be defined by known reference points, e.g., a section between two exits on a motorway or between two intersections in a city. Another possibility is the temporal definition, in which a route section is defined based on the travel time, e.g.as a section of road that the vehicle travels within 2 minutes. Combinations of these methods can also be used to more accurately describe and identify road sections.

[0039] The vehicle 200 is equipped with various sensors of a sensor device that record data 250 and information about the vehicle 200 and the vehicle's surroundings while traveling along the route sections S1, S2, ..., Sn of a route. The sensors comprise, in particular, one or more cameras, including rearview cameras and front cameras, which are used for parking and assisting with lane keeping, but also for detecting traffic signs and other traffic objects. Additionally, ultrasonic sensors can be used to measure the distance to obstacles. Radar sensors are used for adaptive cruise control and collision avoidance systems because they can measure the distance and speed of surrounding vehicles. Lidar (Light Detection and Ranging) sensors are increasingly used in advanced driver assistance systems and autonomous vehicles.They use laser pulses to map the environment in 3D and detect obstacles. Speed ​​sensors measure the vehicle's speed and enable the control of cruise control systems such as cruise control and ABS (anti-lock braking system). Other sensors include speed sensors, steering angle sensors, accelerometers that measure the vehicle's acceleration in various directions, gyro sensors that measure the vehicle's rotational movement, humidity sensors, and temperature sensors.

[0040] Within the scope of the present invention, the recorded data 250 from the vehicle 200 or from a plurality of vehicles (vehicle swarm data) is transmitted to the input module 300 or stored in the database 370. Vehicle swarm data is data collected and exchanged by a group of networked vehicles. This data encompasses a wide range of information collected by sensors and communication systems in the vehicles and sent via wireless connections to central servers or other vehicles.

[0041] The data 250 thus comprises data from different sensor types, which are integrated and fused, particularly as part of a sensor fusion, to obtain a complete picture of the vehicle's surroundings in a route section Sj. Artificial intelligence algorithms can be used to utilize the strengths of the individual sensors and compensate for their weaknesses. The result is a comprehensive perception of the respective route section Sj. The input module 300 and / or the database 370 contain a correspondingly configured software application for performing such a sensor data fusion.

[0042] In addition, the input module 300 can retrieve further data 270, in particular map data, from the database 370 and merge it with the sensor data 250. Map data or cartographic data comprises a variety of geographical and topographical information in digital form that is used to represent geographical areas, roads, buildings, and other geographical objects. Map data is used in various applications and services, including navigation software and online mapping applications. Map data contains information about geographical features such as rivers, lakes, mountains, forests, roads, and buildings linked to coordinates (longitude and latitude). In particular, map data contains information about road networks. Furthermore, map data can contain detailed information about buildings such as location, elevation, type of use, and number of floors.Additional information may relate to the type of land use, such as residential, commercial, agricultural, or natural land. Map data can be obtained from various sources, including companies such as Google Maps, Apple Maps, and OpenStreetMap.

[0043] The additional data 270 may also contain traffic information, e.g., typical traffic situations at a specific time and location. In addition, further information, such as statistical weather data containing information such as temperature, precipitation, wind speed, and humidity, may be assigned to a route section Sj.

[0044] Overall, the input module 300 generates a data set 350 for a route section Sj from the data 250 of the sensors, map data and the additional data 270, which represents the most complete image possible of the respective route section Sj.

[0045] The data sets 350 for a plurality of route sections S1, S2, ..., Sn are transferred from the input module 300 to the extraction module 400. The extraction module 400 comprises a software application 450 that extracts the relevant scenario attributes A1, A2, ..., An from the data set 350 for a route section Sj.

[0046] A route section Sj can comprise various scenarios that occur sequentially or simultaneously on the route section Sj. A scenario is a specific traffic situation used to test and evaluate the performance and behavior of vehicles with driver assistance functions and / or driving functions in various real or potential traffic environments. Scenarios cover a wide range of traffic situations, including normal driving situations such as driving on motorways, country roads, and inner-city streets, as well as special traffic situations such as crossing intersections, parking and reversing, overtaking other vehicles, and driving in construction zones or in difficult weather conditions. Furthermore, the scenarios include critical situations such as sudden braking, evasive maneuvers, reacting to obstacles on the roadway, or unexpected behavior of other road users.

[0047] A scenario can be characterized by various scenario attributes A1, A2, ..., An. A scenario attribute Aj is, for example, a physical quantity, a chemical quantity, a torque, a rotational speed, a voltage, a current, an acceleration, a speed, a braking value, a direction, an angle, a radius, a location, a number, a moving object such as a motor vehicle, a person or a cyclist, an immovable object such as a building or a tree, a road configuration such as a motorway, a traffic sign, a traffic light, a tunnel, a roundabout, a turning lane, a traffic density, a topographical structure such as a gradient, a time, a temperature, a precipitation value, a weather condition and / or a season.

[0048] Within the scope of the present invention, scenario attributes thus characterize properties and characteristics of a scenario.

[0049] The selection of the scenario attributes A1, A2, ..., An relevant for the description of a route section Sj is carried out in particular within the framework of a 5-layer model with the following layers: - The first layer relates to road geometry and topography. This includes the description of the various road types, such as highways, rural roads, and inner-city roads, and their characteristics. It also includes information on road geometry such as road type, shape, and layout, including curves, intersections, number of lanes, junctions, roundabouts, and topographic elevation information, as well as the representation of the connections and relationships between different roads and traffic routes. - The second layer contains information about roadside development, i.e. information about traffic signs, traffic lights and road markings, as well as information about buildings, shops, parking lots and other roadside features that may affect traffic. - The third layer includes information about construction sites and temporary changes on the road, such as detours and alternative routes. - The fourth layer includes the trajectories of dynamically moving objects, i.e. the description of the behavior and movement of the ego vehicle as well as other vehicles on the road, including cars, trucks, motorcycles and bicycles, as well as pedestrians, athletes, children, animals and other objects that can move on the road, such as trains and trams. - The fifth layer refers to environmental factors such as weather conditions (factors such as rain, snow, fog and their impact on road conditions and visibility), time of day and light conditions (taking into account changes in light conditions during the day, twilight and night), road surface and condition (information on the condition of the road surface, including bumps, potholes and road repairs).

[0050] This five-layer model enables a detailed analysis and description of scenarios and considers the most important factors and influencing variables that are important in the development and implementation of driver assistance systems and driving functions. It serves as a basis for the development and testing of semi-autonomous and autonomous vehicles, as well as for the simulation of traffic situations in different environments. The layers and the information they contain can be further refined according to specific requirements and objectives.

[0051] The software application 450 for extracting the scenario attributes A1, A2, ..., An from the data set 350 includes artificial intelligence algorithms such as classification and extraction algorithms. In particular, deep neural networks such as convolutional neural networks (CNNs) for images and recurrent neural networks (RNNs) for sequences are used. These neural networks are trained with suitable test data sets.

[0052] Neural networks are often cited as the prototype of AI algorithms. Neural networks are also interesting because they can be combined, thus increasing learning success. Such coupled neural networks are also referred to as deep learning. A neural network consists of neurons arranged in multiple layers and connected to each other in various ways. A neuron is capable of receiving information at its input from outside or from another neuron, evaluating the information in a specific way, and forwarding it in a modified form to another neuron at the neuron output or outputting it as the final result. Hidden neurons are arranged between the input neurons and output neurons. Depending on the network type, there may be multiple layers of hidden neurons. They ensure the forwarding and processing of the information.Output neurons ultimately deliver a result and transmit it to the outside world. The arrangement and connection of neurons creates different types of neural networks, such as feedforward neural networks (FNNs), recurrent neural networks (RNNs), or convolutional neural networks (CNNs).

[0053] CNNs were specifically designed for image processing and use layers of convolutional operations, pooling, and activation functions to extract features from images. They are widely used for tasks such as image classification, object detection, and segmentation.

[0054] By extracting the scenario attributes A1, A2, ..., An from the data sets 350, the relevant information is extracted from the raw data. The extracted scenario attributes A1, A2, ..., An are then forwarded to the data analysis module 500.

[0055] The data evaluation module 500 comprises a software application 550 configured to train an embedding 590 of the extracted scenario attributes A1, A2, ..., An using a neural network 570. For this purpose, the extracted scenario attributes A1, A2, ..., An of a route section Sj are transformed by the software application 550 into a multidimensional route section vector 585 located in a multidimensional vector space 580. Representing the scenario attributes A1, A2, ..., An of a route section Sj as a route section vector 585 enables data processing by the neural network 570.

[0056] The neural network 570 is trained with the plurality of route section vectors 585 for the plurality of route sections S1, S2, ..., Sn to generate an embedding 590 of the route section vectors 585. Training an embedding 590 with the neural network 570 refers to the process of embedding the vectors 585 in the multidimensional vector space 580. The embedding 590 of the route sections S1, S2, ..., Sn is thus the representation of the scenario attributes A1, A2, ..., An for each of the route sections S1, S2, ..., Sn in the multidimensional vector space 580. The embedding of the route section vectors 585 is performed according to their degree of similarity, so that similar route sections Si, Sj are close to one another in this vector space 580. Similar route sections Si, Sj contain similar scenario attributes A1, A2, ..., An and thus also have a similar vector representation. Fig. 2 shows a schematic representation of an embedding 590.

[0057] For embedding the segment vectors 585 in terms of their similarity, cosine similarity or cosine distance is used. Cosine similarity is a metric for quantifying the similarity between two vectors in a multidimensional space and measures the cosine of the angle between two vectors. The closer the cosine similarity value is to 1, the more similar the vectors are; the closer it is to -1, the more different they are. A value of 0 means that the vectors are orthogonal to each other and therefore have no similarity.

[0058] Depending on the type of data and the specific task, the network architecture of the neural network 570 can be a simple feed-forward network (FFN) or a more complex model such as a convolutional neural network (CNN) or a recurrent neural network (RNN). In addition, an embedding layer is added to the network architecture, which generates the embeddings for the track segments S1, S2, ..., Sn to be learned. Furthermore, the size of the embedding vectors 585 and other hyperparameters are defined. The neural network 580 is now trained to perform a clustering of the track segments S1, S2, ..., Sn.

[0059] During the training process, the weightings of one or more scenario attributes A1, A2, ..., An can be adjusted in the embedding layer. This allows for the creation of an embedding that delivers optimized results for specific tasks related to a particular driver assistance or driving function.

[0060] After the embedding 590 has been created, appropriate metrics can be used to check whether the quality and accuracy are satisfactory.

[0061] Various methods such as deep metric learning or constraint pretraining can be used to train the neural network 570 to create the embedding 590.

[0062] Deep metric learning is a method that focuses on learning distance or similarity measures between data points. This approach aims to develop models that represent data in a space where similar data points are close together and dissimilar data points are far apart. By creating a model of the similarity relationships between data points, similar objects can be identified in the data sets 350. An important part of deep metric learning is the transformation of the data sets 350 into the embedding space, where the similarity relationships between the individual data points of the different route segments S1, S2, ..., SN become more clearly visible.

[0063] Contrastive pretraining refers to a model training method in which a model is trained to learn differences (contrasts) between data points. The model learns to bring similar data points closer together (similar code) and separate dissimilar data points (dissimilar code). During training, the model learns to represent the data in an embedding space. In this space, similar data points are close together, while dissimilar data points are farther apart. After an initial training phase, the model can also be fine-tuned. For example, the model can be further refined for a specific driver assistance or driving function by adding additional layers or neurons. The particular advantage of contrastive pretraining is that it can also be trained with unlabeled datasets.

[0064] Clusters in an embedding 590 are determined by applying clustering algorithms to the embedding vectors 585. Since similar vectors 585 are close to each other in the vector space 580, similar track sections Si, Sj can be identified and organized into groups or clusters by clustering these vectors 585.

[0065] Clustering algorithms are machine learning and data analysis techniques used to organize similar data points into groups or clusters. Clusters are groups of data points that are more similar to each other than data points in other groups. There are different types of clustering algorithms, which are further divided into hierarchical and partitioning clustering algorithms.

[0066] Hierarchical clustering algorithms organize data into a hierarchical tree structure of clusters. They often begin with each data point as a separate cluster and then gradually group clusters until the desired grouping is achieved. An example of hierarchical clustering is mean-shift clustering, a method that uses density estimates to find cluster centers and assigns data points to the nearest centers.

[0067] Partitioning clustering algorithms divide the data into a specified number of clusters and attempt to find an optimal distribution of the data within the clusters. K-Means, for example, is a partitioning clustering algorithm in which K clusters are specified and the data points are assigned to the nearest cluster centers. The selection of the appropriate clustering algorithm also depends on the specific requirements of the respective driver assistance or driving function.

[0068] The selected clustering algorithm is now applied to the embedding data and assigns the vectors 585 to the corresponding clusters based on their similarities in the embedding space 580.

[0069] The clustering results can be visualized and evaluated using an additional software application. The visualization shows how the route segments S1, S2, ..., Sn are arranged within the clusters. Various evaluation metrics can be used to assess the quality of the clusters.

[0070] The clusters found can now be interpreted. The clusters resulting from embedding 590 provide information about the frequency of certain road sections S1, S2, ..., Sn, their properties, and the frequency of co-occurrence of certain scenario attributes A1, A2, ..., An. In this way, representative road sections Si, Sj can be found for test drives to test a driver assistance function or a driving function, which can cover the widest possible spectrum of scenarios. This enables optimized test route planning based on representative road sections S1, S2, ..., Sn, which can reduce the size of the test set. Furthermore, conclusions can be drawn from previously traveled road sections about as yet unknown road sections with similar properties.

[0071] The results of the clustering in the form of visualizations and evaluations can be displayed by an output module 700 as output data 750. The output module 700 can be located, in particular, on a computer or a mobile device. The output data 750 can be output, in particular, in the form of a report and can contain diagrams, graphics, text messages, video sequences, PowerPoint presentations, etc.

[0072] In Fig. 3 shows the process steps of a method for identifying representative route sections S1, S2, ..., Sn for testing driver assistance functions and / or driving functions for a vehicle.

[0073] In a step S10, sensor data 250 are recorded by at least one sensor such as a camera, a lidar system, a radar system and / or an ultrasound system of a sensor device of a vehicle 200 while driving along a plurality of routes, wherein each route comprises a plurality of route sections S1, S2, ..., Sn and wherein each route section Sj can in turn comprise a plurality of scenarios.

[0074] In a step S20, a data set 350 is generated for each route section Sj by data fusion of the sensor data 250 of different sensors and further data 270, such as map data.

[0075] In a step S30, scenario attributes A1, A2, ..., An are extracted from each data set 350 of a route section Sj.

[0076] In a step S40, the scenario attributes A1, A2, ..., An of a route section Sj are transformed into a multidimensional route section vector 585.

[0077] In a step S50, an embedding 590 is trained with a neural network 570 by embedding the route section vectors 585 into a multidimensional vector space 580, wherein the embedding of the route section vectors 585 is carried out according to the degree of their similarity, so that similar route sections S1, S2, ..., Sn with similar scenario attributes A1, A2, ..., An form clusters.

[0078] In a step S60, the clusters in the embedding 590 are analyzed using clustering algorithms to identify similar route sections Si, Sj.

[0079] In a step S70, output data 750 relating to the frequency of road sections S1, S2, ..., Sn with specific scenario attributes A1, A2, ..., An are generated, in particular in the form of diagrams, graphics, text messages, video sequences and / or PowerPoint presentations.

[0080] Fig.Figure 4 schematically illustrates a computer program product 900 comprising executable program code 950 configured to perform the method according to the first aspect of the present invention when executed.

[0081] By generating an embedding of route section vectors containing scenario attributes of the respective route section Sj according to the invention, clusters of route sections S1, S2, ..., Sn that are similar to one another can be formed. Representatively selected routes Si, Sj can thus be analyzed as representatives of a larger overall set. This enables targeted planning of test drives in order to cover specific scenarios in a targeted manner. From the behavior of a driver assistance or driving function on a specific route section Sj, conclusions can be drawn about the behavior on similar route sections. In particular, the method according to the invention enables a route-section-specific release of a driver assistance or driving function. By including data on suitable times / weather conditions, release can also be more targeted and granular. Reference symbol 100 systems 200 vehicles 250 sensor data 270 additional data 300 input module 350 data sets 370 database 400 Extraction Module 450 software application 500 data analysis module 550 software application 570 neural network 580 Vector space 585 Route section vector 590 Embedding 700 output model 750 Report 900 computer program product 950 program code QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] DE 10 2021 203 057 A1

[0009]

Claims

[1] Method for identifying representative route sections (S1, S2, ..., Sn) for testing driver assistance functions and / or driving functions for a vehicle, comprising the following process steps: - recording (S10) sensor data (250) by at least one sensor such as a camera, a lidar system, a radar system and / or an ultrasound system of a sensor device of a vehicle (200) when driving along a plurality of routes, wherein each route comprises a plurality of route sections (S1, S2, ..., Sn) and wherein each route section (Sj) can in turn comprise a plurality of scenarios; - generating (S20) a data set (350) for each route section (Sj) by data fusion of the sensor data (250) of various sensors and further data (270) such as map data; - extracting (S30) scenario attributes (A1, A2, ..., An) from each data set (350) of a route section (Sj); - transforming (S40) the scenario attributes (A1, A2, ..., An) of a route section (Sj) into a multidimensional route section vector (585); - training (S50) an embedding (590) with a neural network (570) by embedding the route section vectors (585) in a multidimensional vector space (580), wherein the embedding of the route section vectors (585) is carried out according to the degree of their similarity, so that similar route sections (S1, S2, ..., Sn) with similar scenario attributes (A1, A2, ..., An) form clusters; - analyzing (S60) the clusters in the embedding (590) using clustering algorithms to identify similar route sections (Si, Sj); - generating (S70) output data (750) relating to the frequency of road sections (S1, S2, ..., Sn) with specific scenario attributes (A1, A2, ..., An), in particular in the form of diagrams, graphics, text messages, video sequences and / or PowerPoint presentations. [2] The method of claim 1, wherein a scenario attribute (Aj) represents a physical quantity, a chemical quantity, a torque, a rotational speed, a voltage, a current, an acceleration, a speed, a braking value, a direction, an angle, a radius, a location, a number, a moving object such as a motor vehicle, a person or a cyclist, an immovable object such as a building or a tree, a road configuration such as a motorway, a traffic sign, a traffic light, a tunnel, a roundabout, a turning lane, a traffic density, a topographical structure such as a gradient, a time, a temperature, a precipitation value, a weather condition and / or a season. [3] Method according to claim 1 or 2, wherein the scenario attributes (A1, A2, ..., An) are selected within the framework of a 5-layer model, wherein a first layer relates to the road geometry and topography, a second layer contains information about the roadside development, a third layer contains information about construction sites and temporary changes on the road, a fourth layer contains the trajectories of dynamically moving objects and a fifth layer relates to environmental influences such as weather conditions, time of day and lighting conditions as well as the road condition. [4] Method according to one of claims 1 to 3, wherein the network architecture of the neural network (570) is designed as a simple feed-forward network (FFN) or as a more complex model, such as in particular a convolutional neural network (CNN) or a recurrent neural network (RNN). [5] Method according to one of claims 1 to 4, wherein the cosine similarity is used to embed the route section vectors (585) in the vector space (580) with regard to their similarity. [6] Method according to one of claims 1 to 5, wherein deep metric learning and / or constraint pretraining are used for training the neural network (570) to create the embedding (590). [7] Method according to one of claims 1 to 6, wherein during the training process the weights of one or more scenario attributes (A1, A2, ..., An) are adjusted. [8] Method according to one of claims 1 to 7, wherein hierarchical clustering algorithms such as in particular mean-shift clustering and / or partitioning clustering algorithms such as in particular K-Means are used as clustering algorithms. [9] System (100) for identifying representative route sections (S1, S2, ..., Sn) for testing driver assistance functions and / or driving functions for a vehicle, comprising an input module (300) with a database (370), an extraction module (400), a data evaluation module (500) and an output module (700), wherein a sensor device of a vehicle (200) is designed to record sensor data (250) by at least one sensor such as a camera, a lidar system, a radar system and / or an ultrasound system when driving along a plurality of routes, wherein each route comprises a plurality of route sections (S1, S2, ..., Sn), and wherein each route section (Sj) can in turn comprise a plurality of scenarios; wherein the input module (300) is designed to generate a data set (350) for each route section (Sj) by data fusion of the sensor data (250) of the various sensors and with further data (270) such as map data; wherein the extraction module (400) is designed to extract scenario attributes (A1, A2, ..., An) from each data set (350) of a route section (Sj); wherein the data evaluation module (500) comprises a software application (550) designed to transform the scenario attributes (A1, A2, ..., An) of a route section (Sj) into a multidimensional route section vector (585) and for training an embedding (590) with a neural network (570) by embedding the route section vectors (585) in a multidimensional vector space (580), wherein the embedding of the route section vectors (585) is carried out according to the degree of their similarity, so that similar route sections (S1, S2, ..., Sn) with similar scenario attributes (A1, A2, ..., An) form clusters, wherein the software application (550) comprises clustering algorithms for analyzing the clusters in the embedding (590) in order to identify similar route sections (Si, Sj); and wherein the output module (700) is designed to generate output data (750) relating to the frequency of road sections (S1, S2, ..., Sn) with certain scenario attributes (A1, A2, ..., An), in particular in the form of diagrams, graphics, text messages, video sequences and / or PowerPoint presentations. [10] The system (100) of claim 9, wherein a scenario attribute (Aj) represents a physical quantity, a chemical quantity, a torque, a rotational speed, a voltage, a current, an acceleration, a speed, a braking value, a direction, an angle, a radius, a location, a number, a moving object such as a motor vehicle, a person or a cyclist, an immovable object such as a building or a tree, a road configuration such as a motorway, a traffic sign, a traffic light, a tunnel, a roundabout, a turning lane, a traffic density, a topographical structure such as a gradient, a time, a temperature, a precipitation value, a weather condition and / or a season. [11] System (100) according to claim 9 or 10, wherein the scenario attributes (A1, A2, ..., An) are selected within the framework of a 5-layer model, wherein a first layer relates to the road geometry and topography, a second layer contains information about the roadside development, a third layer contains information about construction sites and temporary changes on the road, a fourth layer contains the trajectories of dynamically moving objects and a fifth layer relates to environmental influences such as weather conditions, time of day and lighting conditions as well as the road condition. [12] System (100) according to one of claims 9 to 11, wherein the network architecture of the neural network (570) is designed as a simple feed-forward network (FFN) or as a more complex model, such as in particular a convolutional neural network (CNN) or a recurrent neural network (RNN). [13] System (100) according to one of claims 9 to 12, wherein the cosine similarity is used to embed the route section vectors (585) in the vector space (580) with regard to their similarity; and / or wherein deep metric learning and / or constraint pretraining are used to train the neural network (570) to create the embedding (590); and / or wherein the weightings of one or more scenario attributes (A1, A2, ..., An) are adjusted during the training process. [14] System (100) according to one of claims 9 to 13, wherein hierarchical clustering algorithms such as in particular mean-shift clustering and / or partitioning clustering algorithms such as in particular K-Means are used as clustering algorithms. [15] A computer program product (900) comprising an executable program code (950) configured to carry out the method according to any one of claims 1 to 8 when executed.

Images