Method, system and computer program product for evaluating a driving situation for the predictive control of an automated driving function

Abstract

Method for evaluating a driving situation for the predictive control of an automated driving function of a vehicle (10), comprising the following procedural steps: - Recording (S10) sensor and image data (20) from a camera and sensor device (12) of the vehicle (10), wherein the image and sensor data (20) depict the environment and objects in the environment of the vehicle (10) to determine a current driving situation at a first time (t1); - Transferring (S20) the acquired sensor and image data (20) and / or further historical data from at least one database (40) to a data analysis facility (30), wherein the data analysis facility (30) comprises an analysis system (300) which uses algorithms from the field of artificial intelligence (AI) and machine image analysis; - Determine (S30) at least one future driving situation at a second time point (t2) in the data analysis unit (40) using the analysis system (300); - Classifying (S40) at least one future driving situation with regard to the probability of occurrence and / or other characteristics, conditions or safety categories; - Evaluating (S50) the classified future driving situation and assigning an evaluation index (70); - Feeding (S60) the rating index (70) to a vehicle assistance module (80); - Control (S70) at least one automated driving function (85) of the driver assistance module (80) with the rating index (70); - wherein the data analysis facility (30) is connected to at least one database (40) and / or a cloud computing infrastructure (45) via a communication link, - where the communication link is designed as a mobile communication link, and - wherein the data analysis device (30) and / or the camera and sensor device (12) is / are equipped with radio modules of the 5G standard; and - wherein the rating index (70) has at least three control states which can be represented as “1 - no response”, “2 - active response” and “3 - adaptive response”.

Description

[0001] The invention relates to a method, a system and a computer program product for evaluating a driving situation for the predictive control of an automated driving function.

[0002] When a motor vehicle is driven by a person, the driver observes very closely and intuitively whether there are any potential hazards on the road. In particular, vehicles ahead are closely observed to gather various pieces of information, such as their speed or whether an overtaking maneuver is planned. The road surface and layout are also observed to adjust driving behavior accordingly. A skilled human driver performs these observations intuitively while driving and is often unaware of how they process the information and interpret it in relation to a potential hazard.

[0003] Vehicles are increasingly equipped with driver assistance systems and highly automated driving functions to relieve the driver or to enable autonomous driving. In partially and autonomously driving vehicles, camera systems and sensors are used to gather information about the vehicle's surroundings. The development of highly automated driving (HAD) is therefore accompanied by increased demands on vehicle sensor systems for acquiring suitable sensor data, particularly image data. Furthermore, the acquired sensor data must be carefully interpreted to draw the correct conclusions regarding potential hazardous situations.

[0004] An example of a driver assistance system with highly automated driving functions is adaptive cruise control (ACC) in a motor vehicle, which uses the distance to a vehicle ahead as a control variable for the vehicle's speed. Such a system is also known as adaptive cruise control. ACC uses a sensor to determine the position and speed of the vehicle ahead and adjusts the engine and brake control accordingly.

[0005] However, current driver assistance systems and driving functions only inadequately consider the course of a vehicle's lane due to the terrain's topography, such as inclines and curves, as well as the actual traffic situation, such as vehicles merging into the lane. Merging vehicles are either ignored, or the system reacts to their potential merging. Current driver assistance systems thus only have two states: either they anticipate a possible event, such as a merging vehicle, or they ignore it. An intermediate state with only a minor adjustment of the driving function, such as reduced throttle input and therefore only a slight reduction in the vehicle's speed due to drag torque, is not yet provided for.

[0006] German patent application DE 10 2017 206 862 A1 discloses a driving system for the automated driving of a motor vehicle. The driving system is configured to detect an actual traffic situation in the vicinity of the motor vehicle with at least one other road user and to consider at least two alternative courses of action based on the actual traffic situation for a first time step. For each of these courses of action, at least one possible future traffic situation is determined, the possible future traffic situations are evaluated, and one of the courses of action is selected based on this evaluation.

[0007] DE 10 2020 201 016 A1 discloses a method for providing at least one trajectory for an automated vehicle, wherein future behavior is estimated based on environmental data of the vehicle using a prediction device and the estimated future behavior is assigned to one of at least two evaluation categories.

[0008] German patent DE 10 2017 114 876 A1 discloses a method for avoiding a potential collision between a vehicle and another road user. The future trajectories of both the vehicle and the other road user are predicted, and along each predicted trajectory, at least two probability zones with different time intervals are determined. All probability zones with equal time intervals are compared, and escalation measures are initiated if an overlap is detected.

[0009] German patent DE 10 2020 113 611 A1 discloses a method and a safety system for safeguarding an automated driving function, wherein a temporal sequence of data used by a vehicle assistance system for the automated vehicle function is recorded. Based on the recorded data sequence, an automatic prediction is made as to whether the assistance system can perform the vehicle function within a predetermined time period. If this is not expected to be the case, a corresponding message is automatically issued to the vehicle operator.

[0010] The object underlying the invention is to create a method, a system and a computer program product for evaluating a driving situation for the predictive control of an automated driving function for a vehicle, which is characterized by high reliability, safety and accuracy and is easy to implement.

[0011] According to the present invention, a method, a system and a computer program product are proposed by which a driving situation is safely detected and evaluated, enabling predictive control of an automated driving function that ensures a high level of safety.

[0012] This problem is solved according to the invention with respect to a method by the features of claim 1, with respect to a system by the features of claim 7, and with respect to a computer program product by the features of claim 10. The further claims relate to preferred embodiments of the invention.

[0013] According to a first aspect, the invention provides a method for evaluating a driving situation for the predictive control of an automated driving function of a vehicle. The method comprises the following process steps: - Recording sensor and image data from a camera and sensor device of the vehicle, wherein the image and sensor data depict the environment and objects in the vicinity of the vehicle to determine a current driving situation at a first time t1; - Passing on the acquired sensor and image data and / or other historical data from at least one database to a data analysis facility, wherein the data analysis facility includes an analysis system which uses algorithms from the field of artificial intelligence (AI) and machine image analysis; - Determining at least one future driving situation at a second time t2 in the data analysis facility using the analysis system; - Classifying at least one future driving situation with regard to the probability of occurrence and / or other characteristics, conditions or safety categories; - Evaluating the classified future driving situation and assigning a rating index; - Feeding the rating index to a vehicle assistance module; - Control at least one automated driving function of the driver assistance module using the rating index.

[0014] The data analysis device is connected to at least one database and / or a cloud computing infrastructure via a communication link, whereby the communication link is a mobile network connection and the data analysis device and / or the camera and sensor device are equipped with 5G standard radio modules. Furthermore, the evaluation index has at least three control states, which can be represented as "1 - no reaction", "2 - active reaction", and "3 - adaptive reaction".

[0015] In an advantageous embodiment, the analysis system comprises a detection module for recognizing the current driving situation at a first time t1, a classification module for classifying the current driving situation and / or at least one future driving situation, a prediction module for calculating at least one future driving situation at a second time t2 and for estimating a probability value for the occurrence of the future driving situation, and an evaluation module for evaluating the future driving situation using evaluation criteria.

[0016] In particular, the recognition module includes at least one encoder module for extracting image features from the sensor and image data and at least one decoder module for evaluating the extracted image features.

[0017] In a further embodiment, the camera and sensor device comprises at least one camera and / or at least one LIDAR sensor (Light Detection and Ranging) for optical distance and speed measurement and / or an ultrasonic sensor and / or a radar sensor (Radio Detection and Ranging) and is designed for a fast recording frequency.

[0018] In an advantageous embodiment, it is provided that algorithms from the field of artificial intelligence (AI) and machine image analysis, specifically machine learning algorithms, preferably deep learning with deep neural networks and / or convolutional neural networks and / or reinforcement learning agents (LV), are used to analyze and evaluate a driving situation using the acquired image and sensor data and / or other historical data.

[0019] In an advantageous embodiment, it is provided that at least one driving function is designed as lane keeping assistance or brake support or autonomous driving function.

[0020] According to a second aspect, the invention provides a system for evaluating a driving situation for the predictive control of an automated driving function of a vehicle. The system comprises a camera and sensor device configured to record sensor and image data, wherein the image and sensor data depict the environment and objects in the vicinity of the vehicle in order to determine a current driving situation at a first time t1;a data analysis facility comprising an analysis system which uses and is trained to determine at least one future driving situation at a second time t2 from the image and sensor data and other historical data, to classify at least one future driving situation with regard to the probability of occurrence and / or other characteristics, states or safety categories, to evaluate the classified future driving situation and to assign it an evaluation index;and a vehicle assistance module trained to control at least one automated driving function using the rating index. The data analysis unit is connected to at least one database and / or cloud computing infrastructure via a communication link, the communication link being a mobile network connection, and the data analysis unit and / or the camera and sensor unit being equipped with 5G standard radio modules. Furthermore, the rating index has at least three control states, which can be represented as "1 - no reaction", "2 - active reaction", and "3 - adaptive reaction".

[0021] In an advantageous embodiment, the analysis system comprises a detection module for recognizing the current driving situation at a first time t1, a classification module for classifying the current driving situation and / or at least one future driving situation, a prediction module for calculating at least one future driving situation at a second time t2 and for estimating a probability value for the occurrence of the future driving situation, and an evaluation module for evaluating the future driving situation using evaluation criteria.

[0022] In a further development, algorithms from the field of artificial intelligence (AI) and machine image analysis, specifically machine learning algorithms, preferably deep learning with deep neural networks and / or convolutional neural networks and / or reinforcement learning agents (LV), are used to analyze and evaluate a driving situation using the captured image and sensor data and / or other data.

[0023] According to a third aspect, the invention provides a computer program product comprising an executable program code configured to perform the method according to the first aspect when executed.

[0024] The invention will now be explained in more detail with reference to exemplary embodiments shown in the drawing.

[0025] This shows: Fig. 1 a schematic representation of a first embodiment of the system according to the invention; Fig. 2 a schematic representation of an analysis system according to the invention; Fig. 3 a flowchart to explain the individual process steps of the process according to the invention; Fig. 4 a block diagram of a computer program product according to an embodiment of the third aspect of the invention.

[0026] Additional features, aspects and advantages of the invention or its embodiments become apparent from the detailed description in conjunction with the claims.

[0027] Fig. Figure 1 shows a first embodiment of a system 100 for evaluating a driving situation for controlling an automated driving function for a vehicle 10. The vehicle 10 can be, for example, a motor vehicle, an autonomously driving vehicle, an agricultural vehicle such as a combine harvester, a robot in production or in service and care facilities, or a watercraft or an aircraft such as a drone.

[0028] The vehicle 10 is equipped with a camera and sensor system 12 that records the surroundings and objects in the vicinity of the vehicle 10 and generates sensor and image data 20. The camera and sensor system 12 records the sensor and image data 20 within a recording area that captures the surroundings of the vehicle 10 as comprehensively as possible. In particular, image and sensor data 20 are captured from the lane ahead, but the area to the side and rear of the vehicle 10 are also important, for example, to monitor overtaking maneuvers by other road users on multi-lane roads. Other moving objects such as vehicles, pedestrians, cyclists, motorcyclists, etc., or stationary objects such as houses, road signs, fences, trees, etc., may be located in the vicinity of the vehicle 10.The camera and sensor unit 12 transmits the recorded sensor and image data 20 to a data analysis unit 30 for further processing.

[0029] The camera and sensor assembly 12 includes, in particular, at least one RGB camera in the visible range with the primary colors blue, green, and red. It may also additionally include at least one UV camera in the ultraviolet range and / or at least one IR camera in the infrared range. The cameras, which differ in their recording spectrum, can thus depict different lighting conditions within the recording area.

[0030] The recording frequency of the camera and sensor unit 12 is designed for high vehicle speeds 10 and can record sensor and image data 20 at a high image recording frequency. Furthermore, the camera and sensor unit 12 can be equipped with a microphone for capturing acoustic signals. This allows tire noise or engine noise to be recorded, providing information about the road surface and road conditions.

[0031] Furthermore, the camera and sensor unit 12 can be configured to automatically start the image acquisition process when a significant change in area occurs within its recording range, for example, when clear changes in the road surface are visible. This enables a selective data acquisition process, and only relevant sensor and image data 20 are processed by the data analysis unit 30. This allows for more efficient use of computing resources.

[0032] In particular, it is planned to use at least one weatherproof action camera, which can be positioned on the exterior of the vehicle 10. Action cameras have wide-angle fisheye lenses, making it possible to achieve a visible radius of approximately 180°. This allows the road ahead 20 to be comprehensively captured. Action cameras can typically record videos in Full HD (1920 x 1080 pixels), but Ultra HD or 4K action cameras (at least 3840 x 2160 pixels) can also be used, resulting in a significant improvement in image quality. The frame rate is typically 60 frames per second in 4K and up to 240 frames per second in Full HD. An integrated image stabilizer may also be included. Furthermore, action cameras are often equipped with an integrated microphone.Furthermore, differential signal processing methods can be used to selectively block out background noise.

[0033] Furthermore, the camera and sensor system can include 12 sensors such as at least one LIDAR sensor (Light Detection and Ranging) for optical distance and speed measurement and / or at least one ultrasonic sensor and / or at least one radar sensor (Radio Detection and Ranging).

[0034] The sensor and image data 20 acquired by the camera and sensor unit 12 are transmitted to the data analysis unit 30 via communication links such as a CAN bus system (Controller Area Network). Wireless connections can also be used. A wireless communication link is typically a mobile network connection and / or a near-field communication connection such as Bluetooth. ®, Ethernet, NFC (near field communication) or Wi-Fi® trained.

[0035] Furthermore, a GPS connection 14 is advantageously provided to determine the geographical location of the vehicle 10 and to assign it to the recorded sensor and image data 20.

[0036] Furthermore, the data analysis unit 30 can access one or more additional databases 40. Database 40 may, for example, store classification parameters for analyzing the recorded image and sensor data 20, or further historical data and images and / or key performance indicators. Target variables and target values ​​that define a safety standard may also be stored in database 40. Cartographic and topographic data may also be stored. Additionally, a user interface 50 may be provided for inputting further data and for displaying the calculation results generated by the data analysis unit 30. In particular, the user interface 50 is designed as a display with a touchscreen.

[0037] Based on the determined geographical location, further data about the topographical structure of the roadway on which vehicle 10 is located can be retrieved from database 40 by the data analysis unit 30. This topographical structure could, for example, be the crest of a hill, so that the road descending behind the crest is not visible to the driver, and thus he cannot perceive oncoming traffic. Or it could be a sharp bend that should only be negotiated at a reduced speed.

[0038] In particular, the route can be entered into a navigation device so that the data for this route can be retrieved from a database 40.

[0039] Additional data, such as weather data for the current location of vehicle 10 and for a planned route, can be retrieved from database 40 or another data facility, as weather conditions like rain, fog, or snow can influence driving behavior. Further historical data can include information on traffic volume.

[0040] The data analysis device 30 preferably comprises a memory module 32 and a processor 34, which processes the sensor and image data 20 by means of an analysis system 300. The processor 34, or another processor, is also configured to control the camera and sensor device 12. The data analysis device 30 can be integrated into the vehicle 10 or configured as a cloud-based solution. In particular, the data analysis device 30 can be connected via a communication link to a cloud computing infrastructure 45 or another computing unit. The database 40 can also be integrated into the cloud computing infrastructure 45.

[0041] In the context of the invention, a "processor" can be understood to mean, for example, a machine or an electronic circuit. In particular, a processor can be a central processing unit (CPU), a microprocessor, or a microcontroller, such as an application-specific integrated circuit or a digital signal processor, possibly in combination with a memory unit for storing program instructions, etc. A processor can also be understood to be a virtualized processor, a virtual machine, or a soft CPU.For example, it can also be a programmable processor equipped with configuration steps for executing the aforementioned method according to the invention, or configured with configuration steps such that the programmable processor realizes the features of the method, the component, the modules, or other aspects and / or partial aspects of the invention. In particular, the processor 34 can contain highly parallel processing units and powerful graphics modules. The processor 34 can also advantageously use AI hardware acceleration such as a Coral Dev Board to enable real-time processing. This is a microcomputer with a tensor processing unit (TPU), which allows a pre-trained software application to evaluate up to 70 images per second.

[0042] In the context of the invention, a "storage unit" or "storage module" and the like can refer, for example, to volatile memory in the form of random-access memory (RAM), persistent storage such as a hard drive or data carrier, or, for example, a removable storage module. The storage module can also be a cloud-based storage solution.

[0043] In the context of the invention, a "module" can be understood to mean, for example, a processor and / or a memory unit for storing program instructions. For example, the processor is specifically configured to execute the program instructions in such a way that the processor and / or the control unit performs functions to implement or realize the method according to the invention or a step thereof.

[0044] In the context of the invention, "data" means both raw data and already processed data from the measurement results of the camera and sensor device 12 and / or data stored in the database 40 and / or the cloud computing infrastructure 45.

[0045] The term "database" refers to both a storage algorithm and the hardware in the form of a storage unit. In particular, the database is designed as a cloud computing infrastructure.45

[0046] The communication link between the data analysis facility 30 and the database 40 or cloud computing infrastructure 45 is in particular a mobile communication link and / or a near field communication link such as Bluetooth. ® , Ethernet, NFC (near field communication) or Wi-Fi® trained.

[0047] In particular, the data analysis unit 30 and / or the camera and sensor unit 12 are equipped with 5G-standard mobile communication modules. 5G is the fifth-generation mobile communication standard and, compared to the 4G mobile communication standard, is characterized by higher data rates of up to 10 Gbit / sec, the use of higher frequency ranges such as 2100, 2600, or 3600 megahertz, increased frequency capacity, and thus increased data throughput and real-time data transmission, as up to one million devices per square kilometer can be addressed simultaneously. Latency times range from a few milliseconds to less than 1 ms, enabling real-time transmission of data and calculation results. The sensor and image data 20 acquired by the camera and sensor unit 12 can be sent in real time to the cloud computing infrastructure 45, where the analysis of the sensor and image data 20 is performed.The analysis and calculation results can be sent back to the data analysis unit 30 or another control module in the vehicle 10.

[0048] This data transmission speed is required if cloud-based solutions are to be used for processing the sensor and image data 20. Cloud-based solutions offer the advantage of high and therefore fast computing power. Cryptographic encryption methods are specifically intended to protect the connection to the cloud computing infrastructure 45 via a mobile network connection. Cryptographic encryption methods can also be used for the connection between the camera and sensor unit 12 and the data analysis unit 30.

[0049] The data analysis unit 30 can be a separate unit with regard to its hardware configuration and is in particular located in the vehicle 10, but it can also rely on other hardware and software components in the vehicle 10 or in the cloud computing infrastructure 45 for its operation.

[0050] The analysis system 300 comprises artificial intelligence and machine image analysis algorithms to select and classify the sensor and image data 20. Advantageously, the analysis system 300 uses machine learning algorithms, preferably deep learning with, for example, convolutional neural networks and / or reinforcement learning agents (LV) to analyze and evaluate a future driving situation using the acquired image and sensor data 20.

[0051] A neural network consists of neurons arranged in multiple layers and interconnected in various ways. A neuron can receive information at its input from outside or from another neuron, process the information in a specific manner, and then forward it in a modified form to another neuron at its output, or output it as a final result. Hidden neurons are located between the input and output neurons. Depending on the network type, there can be several layers of hidden neurons. They are responsible for the transmission and processing of information. Output neurons ultimately deliver a result and transmit it to the outside world. The arrangement and interconnection of the neurons give rise to different types of neural networks, such as feedforward networks, recurrent neural networks, and convolutional neural networks.The networks can be trained through unsupervised or supervised learning.

[0052] The Convolutional Neural Network (CNN) is a special type of artificial neural network. It possesses multiple layers of convolution and is well-suited for machine learning and artificial intelligence (AI) applications in the field of image and speech recognition. The functionality of a CNN is partly modeled on biological processes, and its structure is comparable to the visual cortex of the brain. Training of a CNN is typically supervised. Conventional neural networks consist of fully or partially meshed neurons in multiple layers. However, these structures reach their limits when processing images, as a number of inputs corresponding to the number of pixels would be required. The CNN is composed of various layers and is fundamentally a partially locally meshed neural feedforward network.The individual layers of a CNN are the convolutional layer, the pooling layer, and the fully meshed layer. The convolutional layer is the actual folding layer and is capable of recognizing and extracting individual features from the input data. In image processing, these can be features such as lines, edges, or specific shapes. The input data is processed in the form of tensors, such as a matrix or vectors. The pooling layer, also called the subsampling layer, condenses and reduces the resolution of the recognized features using appropriate filtering functions. The reduced data volume increases the computation speed.

[0053] The Convolutional Neural Network (CNN) therefore offers numerous advantages over conventional non-convolutional neural networks. It is suitable for machine learning and artificial intelligence applications with large amounts of input data, such as image recognition. The network operates reliably and is insensitive to distortions or other optical alterations. The CNN can process images taken under varying lighting conditions and from different perspectives, yet still recognize the typical features of an image. Because the CNN is divided into several local, partially meshed layers, it requires significantly less memory than fully meshed neural networks. The convolutional layers drastically reduce memory requirements. The training time of the Convolutional Neural Network is also significantly reduced. With the use of modern graphics processing units (GPUs), CNNs can be trained very efficiently.

[0054] A reinforcement learning agent (LV) selects a specific state s i ∈ S from a set of available states for at least one action a i ∈ A from a set of available actions. For the selected action a i The agent receives a reward, which can be positive, neutral, or negative. The states s i ∈ S the agent receives from a state module that processes data from various sensors and data sources and assigns states s to these processed data. i ∈ S, which the LV agent can access.

[0055] Fig. Figure 2 shows the analysis system 300 according to the invention, which comprises a recognition module 310, a classification module 320, a prediction module 330 and an evaluation module 340.

[0056] In the recognition module 310, the acquired sensor and image data 20 are processed to recognize the environment and the objects within it and to determine the current driving situation at a first time point t1. Specifically, the sensor and image data 20 are processed using a convolutional neural network. A driving situation at a first time point t1 thus represents various objects at different locations in a three-dimensional space relative to the current position of the vehicle 10. However, the objects and the vehicle 10 move at different speeds, so a future driving situation at a second time point t2 will differ from the current driving situation.

[0057] The 310 recognition module can, in turn, include at least one encoder module for extracting image features. The extracted image features are then passed on to at least one decoder module for evaluation. This reduces the computing power requirements for processing sensor and image data with regard to specific characteristics such as object detection and depth estimation for accurate positioning.

[0058] In the classification module 320, the detected objects are classified according to various characteristics, states, and / or safety categories, in conjunction with other data, particularly from database 40. For example, a road layout can be classified as safe or dangerous based on its topography and assigned a safety level from low to high. Similarly, the driving behavior of another vehicle can be classified as safe or considered a safety risk based on its speed. These characteristics, states, and / or safety categories were preferably predefined, transmitted to the classification module 320, and implemented during a training phase of a neural network.

[0059] The identified and classified driving situation is processed in prediction module 330 with regard to predicting a future driving situation at time t2. Based on a prediction of the change in the spatial position of the various objects in the vicinity of the vehicle 10, prediction module 330 calculates at least one future driving situation at a second time t2. The shorter the time interval between t1 and t2, the more accurate the prediction of the future driving situation. The future driving situation at a second time t2 is therefore weighted with a probability of the actual event occurring, i.e., the future driving situation. In particular, prediction module 330 can calculate several future driving situations at a second time t2, each of which is assigned a probability of actual occurrence.The prediction module 330 can also calculate at least one future driving situation at each of several time points ti in a time sequence, each of which is assigned a probability of occurrence. These various future driving situations at a time point t2 and / or at further time points ti can then be classified by the classification module 320 according to characteristics, states, and / or safety categories. Thus, a first future driving situation might have a high probability of occurrence and be assigned a low safety level. A second future driving situation, on the other hand, might have a low probability of occurrence but be assigned a high safety level.

[0060] The prediction module 330 calculates at least one future driving situation, which is then evaluated by the assessment module 340. The evaluation criteria weigh the probability of occurrence against other characteristics, conditions, and safety categories. Since there are a number of ethical questions in the field of automated driving to ensure the safety of all road users, ethical criteria, such as those developed by an ethics committee for automated driving, can also be incorporated into the evaluation criteria.

[0061] Based on the evaluation of at least one future driving situation at time t2 and / or at another time ti, the evaluation module 340 outputs an evaluation index of 70. The evaluation index 70 corresponds to a control state for a driver assistance module 80, whereby in particular three control states are provided, which can be designated as "1 - no reaction", "2 - active reaction" and "3 - adaptive reaction".

[0062] In the first control state “1 - no reaction”, there is no change in driving behavior, as the probability of a critical future event occurring is low and / or the resulting danger and / or reduced driving comfort can be accepted.

[0063] In the second control state “2 - active reaction”, there is an active change in driving behavior because the probability of a critical future event occurring is high and / or the probability of a critical future event occurring is low, but the hazard potential of the event is very high, and therefore the resulting danger and / or reduced driving comfort must be counteracted in order to ensure safe and / or comfortable driving when the future driving situation occurs.

[0064] In the third control state, "3 - Adaptive Response," the driving behavior is moderately adjusted, for example, by withholding throttle input and slightly reducing speed, as the probability of a critical future event occurring is in the medium range. This moderate adjustment of driving behavior can reduce the risk and / or improve driving comfort. If the event does not occur, this third control state is more optimal than the reactive behavior of the first control state.

[0065] In a further development, it may also be provided that, in addition to the three described control states, a continuum of several control states is provided between the first control state “1” - no reaction and the second control state “2” - active reaction”, each of which enables a predictive individual reaction to a specific future situation.

[0066] The rating index 70 is thus forwarded to the driver assistance module 80 and integrated into at least one automated driving function 85 and / or transmitted to the user interface 50. A driving function 85 is, for example, lane keeping assistance, brake support, or an autonomous driving function. With an automated driving function 85, the vehicle speed can, for example, be automatically reduced or automatic braking can be performed by issuing corresponding control commands to control units 14 of the vehicle 10, such as the engine and / or the brakes. In addition, the seat belts can be automatically tightened.

[0067] Additionally, recommendations for action or warnings can be displayed to the driver of vehicle 10 via user interface 50. For example, a warning tone or a visual indicator can be issued via user interface 50, depending on the rating index 70, to draw the driver's attention to a particular traffic situation that requires increased attention.

[0068] Since the calculation of the rating index 70 must be performed in real time for the timely control of the driving function 85, the processing speed is crucial for the numerous calculations in the various modules of the analysis system 300. To enable real-time calculations, the use of the cloud computing infrastructure 45 is therefore advantageous, as it ensures rapid computation. Furthermore, a 5G mobile connection is advantageous for communication between the data analysis unit 30 and the cloud computing infrastructure 45, as this allows for real-time data transmission.

[0069] In a further development, it may also be provided that at least some of the objects in the vicinity of the vehicle 10 also send their recorded data to the cloud computing infrastructure 45, so that the cloud computing infrastructure 45 can simulate the current driving situation as well as future driving situations from this data of the objects and the data of the vehicle 10.

[0070] By taking into account expected, estimated, or known future events, such as inclines or curves due to the topography, or vehicles merging into the lane, safe and comfortable driving is made possible. Future driving situations are calculated using predictive algorithms and weighted with a probability of occurrence. From this, an evaluation index for the future driving situation is calculated, enabling predictive control of the vehicle's driving functions. In particular, three control states are proposed that allow for better adaptation of the driving functions to a future driving situation. The driving behavior with regard to safety, comfort, and performance can thus be fine-tuned, significantly improving confidence in automatic driving functions and, consequently, the driving experience.

[0071] In Fig. Section 3 describes the procedural steps for evaluating a driving situation for the predictive control of an automated driving function.

[0072] In step S10, sensor and image data 20 are recorded by a camera and sensor device 12 of a vehicle 10, wherein the image and sensor data 20 depict the environment and objects in the vicinity of the vehicle 10.

[0073] In step S20, the sensor and image data 20 and / or other historical data from a database 40 are passed to a data analysis facility 30, wherein the data analysis facility 30 comprises an analysis system 300 which uses algorithms from the field of artificial intelligence (AI) and machine image analysis.

[0074] In step S30, at least one future driving situation is determined from the sensor and image data 20 in the data analysis unit 40 using the analysis system 300.

[0075] In step S40, at least one future driving situation is classified with regard to the probability of occurrence and / or other characteristics, conditions or safety categories.

[0076] In step S50, the classified future driving situation is assessed and assigned a rating index of 70.

[0077] In step S60, the rating index 70 is fed to a vehicle assistance module 80.

[0078] In step S70, an automated driving function is controlled using the evaluation index 70 of the future driving situation.

[0079] Fig. Figure 4 schematically represents a computer program product 400 comprising an executable program code 450 configured to execute the method according to the first aspect of the present invention when executed.

[0080] The method and system 100 according to the present invention thus enable the reliable, real-time prediction of the probability of a critical future driving situation occurring. Through a finely tuned assessment of the future driving situation, the control parameters of automated driving functions can, in turn, be adjusted more precisely, thereby increasing overall safety and driving comfort. Reference sign 10 vehicles 12 Camera and sensor setup 14 control devices 20 sensor and image data 30 Data analysis facility 32 memory module 34 processor 40 database 45 Cloud Computing Infrastructure 50 User interface 70 rating index 80 Driver assistance module 85 Driving function 100 System 300 analysis system 310 Recognition module 320 Classification module 330 Prediction module 340 Assessment Module 400 computer program product 450 program code

Claims

[1] Method for evaluating a driving situation for the predictive control of an automated driving function of a vehicle (10), comprising the following procedure steps: - Recording (S10) sensor and image data (20) from a camera and sensor device (12) of the vehicle (10), wherein the image and sensor data (20) depict the environment and objects in the environment of the vehicle (10) to determine a current driving situation at a first time (t1); - Transferring (S20) the acquired sensor and image data (20) and / or further historical data from at least one database (40) to a data analysis facility (30), wherein the data analysis facility (30) comprises an analysis system (300) which uses algorithms from the field of artificial intelligence (AI) and machine image analysis; - Determine (S30) at least one future driving situation at a second time point (t2) in the data analysis unit (40) using the analysis system (300); - Classifying (S40) at least one future driving situation with regard to the probability of occurrence and / or other characteristics, conditions or safety categories; - Evaluating (S50) the classified future driving situation and assigning an evaluation index (70); - Feeding (S60) the rating index (70) to a vehicle assistance module (80); - Control (S70) at least one automated driving function (85) of the driver assistance module (80) with the rating index (70); - wherein the data analysis facility (30) is connected to at least one database (40) and / or a cloud computing infrastructure (45) via a communication link, - where the communication link is designed as a mobile communication link, and - wherein the data analysis device (30) and / or the camera and sensor device (12) is / are equipped with radio modules of the 5G standard; and - wherein the rating index (70) has at least three control states which can be represented as “1 - no response”, “2 - active response” and “3 - adaptive response”. [2] Method according to claim 1, wherein the analysis system (300) comprises a detection module (310) for detecting the current driving situation at a first time (t1), a classification module (320) for classifying the current driving situation and / or at least one future driving situation, a prediction module (330) for calculating at least one future driving situation at a second time (t2) and for estimating a probability value for the occurrence of the future driving situation, and an evaluation module (340) for evaluating the future driving situation using evaluation criteria. [3] Method according to claim 2, wherein the recognition module (310) comprises at least one encoder module for extracting image features from the sensor and image data (20) and at least one decoder module for evaluating the extracted image features. [4] Method according to one of claims 1 to 3, wherein the camera and sensor device (12) comprises at least one camera and / or at least one such as a LIDAR sensor (Light Detection and Ranging) for optical distance and speed measurement and / or an ultrasonic sensor and / or a radar sensor (Radio Detection and Ranging) and is configured for a fast recording frequency. [5] Method according to one of claims 1 to 4, wherein algorithms from the field of artificial intelligence (AI) and machine image analysis are algorithms from the field of machine learning, preferably deep learning with deep neural networks and / or convolutional neural networks and / or reinforcement learning agents (LV) for analyzing and evaluating a driving situation using the acquired image and sensor data (20) and / or other historical data. [6] Method according to any one of claims 1 to 5, wherein a driving function (85) is configured as lane keeping assistance or brake support or autonomous driving function. [7] System (100) for evaluating a driving situation for the predictive control of an automated driving function of a vehicle (10), comprising a camera and sensor device (12) configured to record sensor and image data (20), wherein the image and sensor data (20) depict the environment and objects in the vicinity of the vehicle (10) for determining a current driving situation at a first time (t1);a data analysis device (30) comprising an analysis system (300) which uses and is trained to determine at least one future driving situation at a second time (t2) from the image and sensor data (20) and other historical data, to classify at least one future driving situation with regard to the probability of occurrence and / or other features, states or safety categories, to evaluate the classified future driving situation and to assign it an evaluation index (70); and a vehicle assistance module (80) which is trained to control at least one automated driving function (85) with the evaluation index (70);wherein the data analysis device (30) is connected to at least one database (40) and / or a cloud computing infrastructure (45) via a communication link, wherein the communication link is configured as a mobile communication link and the data analysis device (30) and / or the camera and sensor device (12) is / are equipped with radio modules of the 5G standard; and wherein the evaluation index (70) has at least three control states which can be represented as “1 - no response”, “2 - active response” and “3 - adaptive response”.; [8] System (100) according to claim 7, wherein the analysis system (300) comprises a detection module (310) for detecting the current driving situation at a first time (t1), a classification module (320) for classifying the current driving situation and / or at least one future driving situation, a prediction module (330) for calculating at least one future driving situation at a second time (t2) and for estimating a probability value for the occurrence of the future driving situation, and an evaluation module (340) for evaluating the future driving situation using evaluation criteria. [9] System (100) according to claim 7 or 8, wherein algorithms from the field of artificial intelligence (AI) and machine image analysis are algorithms from the field of machine learning, preferably deep learning with deep neural networks and / or convolutional neural networks and / or reinforcement learning agents (LV) for analyzing and evaluating a driving situation using the acquired image and sensor data (20) and / or other data. [10] Computer program product (400) comprising an executable program code (450) configured to perform the method according to any one of claims 1 to 6 when executed.

Images