Methods for estimating the severity of driving function errors for a motor vehicle

Abstract

A method for estimating the severity of driving function errors for a motor vehicle comprises the following procedural steps: Providing navigation data. Providing environmental attributes. Processing the navigation data, environmental attributes, and vehicle information using a pre-trained neural network. Predicting the occurrence of hazards at at least one location in the navigation data using the neural network. Issuing a warning about a hazard to a vehicle user and / or marking the hazard in the navigation data. A detected hazard is extrapolated to the entire navigation dataset, and the corresponding predicted hazards are marked.

Description

[0001] Severity estimation of driving function errors for a motor vehicle.

[0002] In today's modern motor vehicles, safety systems play an increasingly important role in protecting both vehicle occupants and other road users. A key challenge is the early detection and prediction of potential hazards to prevent accidents and increase road safety.

[0003] This invention describes an innovative system for the automated prediction of hazard areas to improve the state of the art. This invention not only improves the vehicle's responsiveness but also makes a significant contribution to accident prevention and improving general road safety.

[0004] For this purpose, the invention provides a method for driver profile recognition for a motor vehicle according to claim 1, a method for training a neural network according to claim 5, a motor vehicle for carrying out the method according to claim 9, and a computer program product according to claim 10. Advantageous embodiments can be found in the subclaims and the description.

[0005] The invention relates to a method for estimating the severity of driving function errors for a motor vehicle, comprising the following method steps: - Providing navigation data; - Providing environmental attributes; - Processing the navigation data and environmental attributes by a pre-trained neural network; - Prediction of the occurrence of hazards by the neural network at at least one position in the navigation data; - Issuing a warning about the prediction of a hazard to a vehicle user and / or Marking the danger area in the navigation data.

[0006] Prediction refers to the prediction of hazardous locations. In other words, the pre-trained neural network is trained to recognize when a hazardous situation exists given a certain constellation of parameters. This information is then extrapolated to the entire navigation data. This allows hazardous locations with similar parameters to be predicted.

[0007] In an advantageous further development, the warning can be given when planning a route and / or when the motor vehicle approaches a danger area.

[0008] In an advantageous development, the navigation data in the motor vehicle can refer to information used by a navigation system to assist a vehicle user in finding a planned route. The navigation data can include map material, where map material includes digitized maps with streets, paths, geographical features, and places of interest. Additionally, POIs (points of interest) such as gas stations, restaurants, hospitals, and tourist attractions can be included.

[0009] In an advantageous further development, the navigation data can include traffic information. The traffic information can include real-time data on road conditions, such as traffic jams, accidents, or construction sites. In addition, the navigation data can include GPS data from satellites that represent the precise position of the vehicle.

[0010] Alternatively or additionally, route information may include instructions on turn-by-turn directions and / or distances and / or alternative route options. Navigation data may also include speed limits.

[0011] In an advantageous further development, a motor vehicle user can select a route using a navigation system. This route selection can then become part of the navigation data and be processed by the pre-trained neural network. From this, a potential hazard on the selected route can be predicted.

[0012] In a further development, the environmental attributes include road topologies and / or a road type and / or a number of lanes and / or a road condition and / or weather conditions and / or driver information and / or temperatures.

[0013] In an advantageous development, the neural network can be implemented as an AI module, or the AI ​​module can comprise a neural network. The AI ​​module or neural network can be part of a data processing device. For this purpose, the AI ​​module or neural network can comprise at least one input node. The input node is the point at which the input data is entered into the module. The input data can then be processed and analyzed by the AI ​​module or neural network to make decisions or predictions, particularly regarding a hazard location.

[0014] In an advantageous development, the AI ​​module or neural network comprises at least one output node. The output node of the AI ​​module or neural network is the point at which a predetermined action can be triggered based on the result of the processing and analysis of the input data. At the output node, a warning about a predicted hazard location can be issued to a vehicle user and / or at least one marker about a predicted hazard location can be generated in the navigation data.

[0015] In an advantageous further development, the neural network (NN) can be a feed-forward NN (FFNN), a graph NN (GNN) or a convolutional NN (CNN).

[0016] A feed-forward neural network (FFNN) is a fundamental architecture in artificial intelligence in which information flows in only one direction, from input to output. It consists of an input layer, hidden layers, and an output layer. Neurons are connected by weighted connections that are adjusted during training. Activation functions such as sigmoid, ReLU, or tanh can help learn nonlinear patterns. The goal of an FFNN is to optimize the weights so that the error between the predicted and actual outputs can be minimized.

[0017] A graph neural network (GNN) is designed to process data in the form of graphs consisting of nodes and edges. Nodes represent entities such as navigation data or environmental attributes, and edges represent the relationships between these nodes. In a GNN, nodes exchange information with their neighbors to update their features, which occurs in multiple layers. The aggregation and transformation of the features are performed using weighted functions, whose weights can be learned during the training process. Activation functions are applied to model nonlinear relationships. GNNs are used where they can effectively model complex dependencies and relationships.

[0018] A convolutional neural network (CNN) is a deep neural network specifically designed for processing image and video data. It uses convolution operations, in which small filters are passed over the input image to detect local features such as edges and textures. A typical CNN consists of convolutional layers that extract patterns in the image, pooling layers that reduce spatial resolution, and fully connected layers that aggregate features and produce the final output. This architecture enables CNNs to capture translational invariants and learn complex feature representations. CNNs are particularly effective in image processing and are used for applications such as image recognition, object tracking, and face recognition. They have revolutionized computer vision research by enabling deep hierarchical feature representations.

[0019] In a further development, the pre-trained neural network is used to process the environmental attributes to predict a hazard location at at least one position in the navigation data. The prediction is performed based on at least one similar environmental attribute and / or the navigation data of an identified hazard location, and / or at least one relationship between at least two environmental attributes and / or navigation data, or a combination thereof. Here, a further probable hazard location is predicted based on an already identified hazard location.

[0020] The similarity may, for example, refer to certain road geometries and / or weather conditions or a combination thereof.

[0021] In other words, in an advantageous refinement, the neural network receives all geographically relevant input parameters (navigation data, environmental attributes) as input parameters and predicts an error occurrence (1 or 0) by feature selection from the (navigation data, environmental attributes). Subsequently, the relevant input parameters related to the hazard location can be selected, which primarily contribute to the neural network's prediction.

[0022] In a further development, the predicted hazard location is marked in the navigation data. This involves extrapolating a hazard location identified by the trained neural network to another position in the navigation data. These extrapolated positions can have similar environmental attributes and / or navigation data. The corresponding relationships can be recognized by the trained neural network. This allows further hazard locations in the navigation data to be predicted based on a known hazard location.

[0023] In a further development, an active assistance system is designed to execute a predetermined action when approaching a predicted hazard location. The action is determined by an estimate of the severity of the predicted hazard location. Alternatively or additionally, a predicted hazard location can also be taken into account during route planning, and a predetermined action can be initiated.

[0024] In an advantageous further development, the prediction can not only indicate the general occurrence of a hazard, but alternatively or additionally, the severity of the hazard can also be estimated.

[0025] Severity estimation is about how serious an error in an assistance system is and what measures should be taken to prevent this error from occurring.

[0026] In a beneficial further development, scenarios of varying severity can be identified. Based on these scenarios or the resulting severity estimate, an action for the vehicle can be derived.

[0027] In an advantageous further development, a predetermined action of the motor vehicle can be carried out based on the severity estimation.

[0028] In an advantageous further development, a modified route can be suggested and / or the control of the motor vehicle can be transferred to a driver and / or a warning can be issued to the driver and / or an assistance system can be deactivated in whole or in part.

[0029] The assistance systems in question may in particular be assistance systems that allow the driver to at least partially automate certain functions of the motor vehicle.

[0030] In an advantageous further development, this is a lane keeping assistant and / or a distance cruise control and / or a function for autonomous driving.

[0031] The following illustration of possible scenarios with a corresponding severity estimate is purely exemplary: - Bridge height too low on planned route → severity level 10, as 100% accident → other route. - Rain on country road on planned route → lane not recognized → severity level 5 → full vehicle control handed over to driver. - Situation of 3-on-1 lane on planned route → function not able to handle this → severity level 4 → temporary handover of vehicle control until this point is passed.

[0032] For example, a dangerous area can be predicted if the environmental attributes are snow on a selected route and the road gradient is greater than 5%. The neural network could indicate an increased risk of slipping in the navigation data, particularly on the selected route, or recommend the use of snow chains if necessary.

[0033] The hazards can then be predicted in the navigation data. This means that a single identified hazard can be extrapolated to the entire navigation data. For example, all four-lane highways are recorded in the navigation data. Alternatively or additionally, their frequency can be returned. Combinations, such as four-lane highways with winding sections, can also be recorded.

[0034] The invention also relates to a method for training a neural network for estimating the severity of driving function errors for a motor vehicle, comprising the following method steps: - Providing navigation data; - Providing environmental attributes; - Providing vehicle information; - Processing of navigation data, environmental attributes and vehicle information by the neural network; - Correlating the vehicle information with the navigation data and the environmental attributes by the neural network, whereby a danger point is predicted if at least one parameter of the vehicle information is exceeded or undershot.

[0035] The neural network or the neural network trained after completion of the above procedure can be used in one of the above further developments.

[0036] For this purpose, the neural network can be trained using vehicle information, environmental attributes, and navigation data. The neural network can receive the vehicle information transmitted by the motor vehicle to a data processing device.

[0037] In an advantageous development, the vehicle information contains information about the motor vehicle. The information can include steering interventions and / or speed and / or sensor data and / or pedal actuation. The vehicle information can be correlated with environmental attributes and / or navigation data.

[0038] A modern motor vehicle can possess a wealth of sensor data that can be used to support driver assistance systems, improve safety, and optimize vehicle performance. This includes data from radar, lidar, and ultrasonic sensors, which are used to detect obstacles and assess the vehicle's surroundings. Camera sensors capture visual information that is used for lane detection, traffic sign recognition, and assistance systems such as lane keeping assist or adaptive cruise control.

[0039] Pressure sensors monitor tire pressure, while infrared sensors monitor vehicle occupants and enhance automatic climate control. Position data from GPS sensors ensure precise navigation functions. Accelerometers and gyroscopes detect changes in motion and support stability control systems such as ESP (Electronic Stability Program). Other sensors, such as steering angle sensors, rain sensors, and temperature sensors, also contribute to comprehensive monitoring and control of the vehicle.

[0040] Additionally, mass flow sensors and oxygen sensors collect information about engine operation and emissions, while lidar sensors and radar provide highly precise distance measurement, which is essential for autonomous driving functions. Together, this sensor data provides a comprehensive picture of the vehicle's status and environment, making modern vehicles safer, more efficient, and more comfortable.

[0041] Alternatively or additionally, the vehicle information may include information about whether an assistance system of the vehicle was active. This makes it possible to detect whether intervention by the vehicle user was necessary to correct a malfunction of an assistance system.

[0042] A possible malfunction of an assistance system could be incorrect recognition of traffic signs. The vehicle does not recognize a stop sign or recognizes it incorrectly. Alternatively or additionally, there may be a misinterpretation of lanes. The vehicle changes lanes unexpectedly or drives over lane markings (possibly due to road topology). Alternatively or additionally, there may be inadequate braking behavior. The vehicle brakes too late or too hard in a certain traffic situation. Alternatively or additionally, there may be problems with obstacle detection. The vehicle does not recognize an obstacle or reacts incorrectly to it (e.g. incorrect AEB activation). Alternatively or additionally, there may be incorrect behavior in complex traffic situations. The vehicle reacts incorrectly to right-of-way rules or unpredictable movements of other road users (possibly due to topology).Alternatively, there may be a problem with traffic flow adaptation. The vehicle is not adapting correctly to the traffic flow, e.g., driving too slowly or too fast. Alternatively, or additionally, there may be a lack of adaptation to roadworks. The vehicle does not recognize roadworks or reacts incorrectly to changed lanes and traffic signals. In an advantageous further development, an active action by the vehicle user, i.e. the vehicle information, can be correlated by the neural network with the navigation data and environmental attributes.

[0043] This allows situations to be identified in which assistance systems must or should be supported by the vehicle user. Alternatively or additionally, general danger points can be detected.

[0044] In an advantageous development, a test data set for training the neural network can be generated by performing test maneuvers. The test data set can include navigation data and / or environmental attributes and / or vehicle information. Alternatively or additionally, a motor vehicle can be configured to collect data during regular operation on public roads and transmit the collected data to a data processing device. The data can be transmitted wirelessly to a cloud processing device.

[0045] The term "test maneuver" describes a series of driving situations that can be simulated on a vehicle test bench or by a test driver. From these, the test data set can then be created. For example, by feeding sensor data into the vehicle, a reaction from vehicle systems such as the distance sensor, lane departure warning system, or brake assist can be provoked, and the response of the vehicle systems can be evaluated.

[0046] This process can be used to pre-train an AI module or a neural network.

[0047] Neural networks or AI systems can be pre-trained to improve the efficiency and effectiveness of subsequent training processes and to optimize performance on different tasks. A key reason for this is transfer learning, in which the neural network or AI system, pre-trained on a large, general dataset, is adapted to specific tasks. This allows previously learned general features to be reused, saving time and resources because the neural network or AI system does not have to be retrained from scratch. Furthermore, pre-training significantly shortens training time because the neural network or AI system already has a solid foundation and only requires fine-tuning with specific data. Pre-trained neural networks or AI systemsAI systems often perform better on new tasks because they have already learned a variety of basic features and patterns, which is especially helpful for tasks with limited data. They offer greater stability during training because the previously learned weights reduce the risk of overfitting and poor convergence, thus providing a better starting point.

[0048] By leveraging powerful hardware such as large cloud systems and large datasets for pretraining, resource utilization can be optimized, and fine-tuning for specific tasks is less computationally intensive. Overall, pretraining neural networks and AI systems contributes to maximizing the efficiency and effectiveness of the learning process, reducing training costs and significantly improving performance on a wide range of specific tasks.

[0049] In an advantageous further development, the training data generated in the motor vehicle and / or in the cloud system can be made available to other motor vehicles.

[0050] In an advantageous embodiment, the test maneuvers or the intensity of the test maneuvers can be selected depending on the vehicle information.

[0051] In an advantageous further development, an AutoML function can be used to improve the above method for training the neural network. AutoML (Automated Machine Learning) refers to the automation of the essential processes of machine learning of the neural network, including data preparation, model selection, hyperparameter tuning, and model evaluation.

[0052] AutoML can be designed to integrate these processes into an automated pipeline to facilitate the use of machine learning or neural networks for users without in-depth expertise, while simultaneously providing efficiency gains for experts. The process can begin with data preparation, which involves cleaning, transforming, and applying feature engineering to the data to make it suitable for modeling.

[0053] The best algorithms can then be automatically selected based on the data properties and the problem type. Hyperparameter optimization can then be performed using techniques such as grid search or Bayesian optimization. Finally, at least one neural network is trained on the prepared data.

[0054] Finally, at least one neural network can be evaluated using suitable metrics, and the best model is prepared for further use. Additional options such as model export for further use of the trained neural network and versioning can be provided. These automated steps can minimize manual effort, reduce sources of error, ensure reproducible results, and enable scalable application of machine learning.

[0055] In a further development, the environmental attributes and navigation data are correlated with the occurrence of vehicle information that is identified with the hazardous situation, and at least one parameter of the neural network is optimized or detected to predict the hazardous situation.

[0056] In other words, when a hazard occurs, the parameter(s) in the environmental attributes or the navigation data that are relevant for the corresponding hazard can be identified from a multitude of parameters.

[0057] In an advantageous further development, the neural network or the AI ​​module is subsequently designed to determine and mark a danger point based on the at least one identified parameter in the navigation data and / or environmental attributes of the selected route.

[0058] In an advantageous further development, the at least one parameter can comprise the change in the number of lanes and / or the road geometry and / or a change in the number of lanes.

[0059] In other words, the neural network or AI module can compare the position at which a hazard was detected with the navigation data or a selected route. This could, for example, narrow the road from three to two lanes. At the same time, the right lane could be eliminated.

[0060] The invention also relates to a motor vehicle with a control / processing device for carrying out the method according to at least one of the above developments.

[0061] In an advantageous further development, the communication interface is arranged to the left of the steering wheel of a left-hand drive vehicle or to the right of the steering wheel of a right-hand drive vehicle.

[0062] The invention also relates to a computer program product with program code means for carrying out the above methods according to one of the above developments, when it runs on a control / processing device or is stored on a computer-readable data carrier.

[0063] In an advantageous development, the control / processing device can comprise at least one microcontroller and / or FPGA and / or processor configured to perform the aforementioned tasks. Alternatively or additionally, the processing or training of the neural network or AI module can be performed in a cloud.

[0064] In particular, the control / processing device can be implemented in a cloud. The control / processing device and the data processing device can receive data sets. The data processing device can comprise an AI module and / or at least one neural network.

[0065] In an advantageous further training, the training can be carried out in batch mode.

[0066] Batch-based training is a common method for training neural networks, allowing large training data sets to be divided into smaller, equally sized subsets, called batches. This method offers a compromise between training on the entire dataset in a single iteration (full-batch gradient descent) and using individual data points per iteration (stochastic gradient descent).

[0067] In a beneficial further development, training can include the following steps: First, the entire training data is divided into smaller batches. The size of these batches, the so-called batch size, is a hyperparametric value that varies depending on the implementation and available computing resources and can be, for example, 32, 64, or 128.

[0068] For each batch, the data points can be passed through the neural network to calculate the neural network outputs. The error (also called loss) for each data point in the batch can then be determined and aggregated into a single error value. This aggregated error is then used to calculate the gradients of the network's weights and biases using a backpropagation method.

[0069] Using the calculated gradients, the network's weights and biases can be updated to minimize the error value. This process of error aggregation, gradient calculation, and weight update is repeated for each batch until all batches have been run through once, which corresponds to a training epoch. The neural network can typically be trained over many epochs to achieve optimal performance.

[0070] Batch-based training can offer the advantage of more stable convergence of the training process compared to stochastic gradient descent, and reduces computational intensity and memory costs compared to full-batch gradient descent. Furthermore, it can enable the efficient use of modern hardware such as GPUs and TPUs, which can operate in parallel, significantly accelerating training.

[0071] The invention will now be described purely by way of example with reference to the drawings. Fig. 1 a method for estimating the severity of driving function errors for a motor vehicle; and Fig. 2 a motor vehicle with a control / processing device for carrying out the method; Fig. 3 a graphical representation of the procedure for estimating the severity of driving function errors for a motor vehicle with an extrapolation to the entire navigation data.

[0072] Fig. 1 shows a method for estimating the severity of driving function errors for a motor vehicle 100, comprising the following method steps: - Providing navigation data 210, S1; - Providing environmental attributes 220, S2; - processing the navigation data 210, the environmental attributes 220 and the vehicle information by a pre-trained neural network 230, S3; - Prediction of the occurrence of danger spots by the neural network 230 at at least one position in the navigation data 210, S4; - Issue a warning about the prediction of a danger point 212 to a Vehicle user and / or marking the predicted danger point 212 in the navigation data 210, S5.

[0073] Fig. 2 shows a motor vehicle 100 with a control / processing device 110 for executing a method for estimating the severity of driving function errors.

[0074] Fig.Figure 3 shows a graphical representation of the method for estimating the severity of driving function errors for a motor vehicle 100 with extrapolation to the entire navigation data. The figure shows a graphical representation of navigation data 210 in the form of a world map. The navigation data is provided with environmental attributes 220 in the form of a three-lane road, in which only the center lane is continued. The road narrows from three to one lane. A hazard 211 was detected at this position in the navigation data 210. The detection of hazard 211 can, for example, have occurred through active steering intervention by a vehicle user when the lane change assistant is malfunctioning.

[0075] The detection of the danger point 211 can be achieved by analyzing the associated vehicle information. Active steering intervention can be detected, for example, by a steering impulse via a specific steering wheel rotation angle or by an intervention of the ESP system in the vehicle. The information is fed to the neural network 230 and analyzed by it, thereby determining at least one parameter correlating with the danger point 211.

[0076] The neural network 230 then analyzes the navigation data 210 with the associated environmental attributes 220 and searches for comparable situations that, with a certain probability, cause a similar hazard. In doing so, the neural network 230 can distinguish between different road topologies. A narrowing to the right lane can be considered uncritical, in contrast to a narrowing to the middle lane.

[0077] The corresponding predicted hazard locations 212 are then marked in the navigation data 210. In other words, a detected hazard location 211 is extrapolated to the entire navigation data 210, and the corresponding predicted hazard locations 212 are marked.

[0078] The invention is not limited to the described embodiments. Within the scope of the invention, all described and / or illustrated features can be combined with one another in any way, unless otherwise stated. Reference symbol S1-S5 process steps 100 motor vehicles 110 Control / processing device 210 Navigation data 211 Danger zone 212 Predicted danger point 220 environmental attributes 230 Neural Network

Claims

[1] Method for estimating the severity of driving function errors for a motor vehicle (100), comprising the following method steps: - Providing navigation data (210), (S1); - Providing environmental attributes (220), (S2); - processing the navigation data (210) and the environmental attributes (220) by a pre-trained neural network (230) (S3); - Prediction of the occurrence of a hazard point (212) by the neural network (230) at at least one position in the navigation data (210), (S4). - issuing a warning about the prediction of the danger point (212) to a vehicle user and / or marking the predicted danger point (212) in the navigation data (220), (S5). [2] Method for estimating the severity of driving function errors for a motor vehicle (100) according to claim 1, - Use of the pre-trained neural network (230) in the processing of the environmental attributes (220) and / or the navigation data (210) to predict the danger point (212) at at least one position in the navigation data (210), wherein the prediction based on at least one similar environmental attribute (220) and / or navigation data (210) of a known hazardous situation, and / or at least one relationship between at least two environmental attributes (220) and / or navigation data (210) of an already identified hazard location (211) is used to predict a further probable hazard location (212). [3] Method for estimating the severity of driving function errors for a motor vehicle (100) according to claim 1 or 2, - marking the predicted danger point (112) in the navigation data (210). [4] Method for estimating the severity of driving function errors for a motor vehicle (100) according to one of the preceding claims, characterized by , that an active assistance system is designed to carry out a predetermined action when approaching a predicted danger point (112), wherein the action is determined by a severity estimate of the predicted hazard location. [5] Method for estimating the severity of driving function errors for a motor vehicle (100) according to one of the preceding claims, characterized by that the environmental attributes (220) include road topologies and / or road type and / or number of lanes and / or road condition and / or weather conditions and / or driver information and / or temperatures. [6] Method for training a neural network (230) for estimating the severity of driving function errors for a motor vehicle (100), comprising the following method steps: - Providing navigation data (210); - Providing environmental attributes (230); - Providing vehicle information; - processing the navigation data (210), the environmental attributes (220) and the vehicle information; - correlating the vehicle information with the navigation data (210) and the environmental attributes (220), wherein a danger point (112) is predicted if at least one parameter of the vehicle information is exceeded or undershot. [7] Method for training a neural network according to claim 5, characterized by , that the environmental attributes (220) and navigation data (210) are correlated with the occurrence of vehicle information identified with the hazard location (111) and at least one parameter of the neural network (230) is optimized for predicting the danger point (111). [8] Method for training a neural network (230) according to claim 5 or 6, characterized by that the input information for the neural network (230), such as navigation data (210) and / or the environmental attributes (220) and / or the vehicle information, is classified based on a relevance assessment and the input information to the neural network (230) is pre-filtered based on its relevance. [9] Motor vehicle (100) with a control / processing device for carrying out the method according to one of claims 1 to 4. [10] Computer program product with program code means for carrying out the method for driver profile recognition for a motor vehicle (100) according to one of claims 1 to 4, when it runs on a control / processing device (110) or is stored on a computer-readable data carrier.

Images