Support system and computer-implemented method using prediction with human factor

Abstract

To provide a computer-implemented method for assisting an agent in moving in a dynamic environment where at least one another agent exists.SOLUTION: A method includes the steps of: acquiring sensor information related to an environment of an agent; predicting at least one action of at least one of the agent and at least one another agent on the basis of the acquired sensor information; determining at least one human factor related to the at least one predicted action; adapting the at least one predicted action on the basis of the determined human factor; determining whether or not a communication based on the at least one adapted action is useful for the agent or an overall traffic goal, such as safety; and if it is determined that the communication is useful, generating a signal on the basis of the at least one adapted action and outputting the generated signal to the agent.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to the general field of assistance systems in dynamic environments based on prediction of future behavior of mobile agents. In particular, a computer-implemented method for assisting agents operating in dynamic environments is proposed. [Background technology]

[0002] The field of automated driver assistance in road traffic environments corresponds to a particular field of application, where a mobile agent moves with the assistance of an assistance system in a highly dynamic environment in which there are several other mobile agents.

[0003] The agents may be traffic participants, including but not limited to pedestrians, bicyclists, motorcyclists, motorized vehicles, such as cars, trucks and buses in a traffic scenario of a road traffic environment.

[0004] Other examples of agents moving in dynamically changing scenarios include airborne vehicles, such as manned aircraft, helicopters, unmanned aerial vehicles moving in an airspace environment, or surface vehicles in a marine environment.

[0005] Current advanced driver assistance systems in road traffic environments use predictions of future behaviors of other vehicles in the environment of a vehicle (subject vehicle) that operates autonomously with the assistance of the driver assistance system. For example, in patent U.S. Pat. No. 8,903,588 (B2), a driver assistance system is disclosed that predicts future movement behaviors of target objects based on sensor data acquired by at least one sensor that physically senses the environment of the subject vehicle. The system calculates multiple movement behavior options for an agent in the environment of the subject vehicle sensed by the sensor. In a context-based prediction step, the system uses a set of classifiers, each classifier estimating the probability that the sensed agent will perform a certain movement behavior at a certain time. The system uses the physical predictions to validate the movement behavior options by comparing the measured points with the trajectory of a situation model, and determines at least one trajectory that indicates at least one future behavior of the agent. The system may estimate at least one future position of the agent based on the determined at least one trajectory, and output information representative of the estimated future position as a basis for performing a driving operation of the autonomous driving of the subject vehicle or for assisting the driver of the subject vehicle by outputting a signal, such as a warning signal.

[0006] Driver assistance systems may include an adaptive cruise control system (ACC), which provides longitudinal control of the host vehicle based on the speed set by the host vehicle's driver, the perceived distance from the host vehicle to other vehicles, and the respective speeds of the other vehicles.

[0007] Nevertheless, known ACC systems lack the ability to deal with various driver personalities, especially in road traffic environments where human drivers operating various vehicles or people traveling on foot still dominate. With respect to ACC, human drivers may react differently depending on their personal characteristics and mental states, and may react early or late to changing situations. Human drivers may perform actions such as sudden lane changes to cut in or out in front of the vehicle, thereby suddenly changing the safety distance between the vehicle and other vehicles.

[0008] Current driver assistance systems generally fail to take into account the driver's state and may result in erroneous warnings or insufficient actuation of the accelerator, brake, or vehicle steering controls.

[0009] European patent EP1544070B1 discloses a driver assistance system capable of estimating the driver's intention with a confidence index by providing multiple virtual operators. The estimated driver's intention is used to modify a risk value or other output generated based on the risk value. EP1544070B1 provides a prediction of the discrete lateral behavior of the other vehicle in a lane change scenario and determines whether the other vehicle will or will not perform a lane change. The improved assistance of EP1544070B1 includes modifying the actuation of the accelerator pedal or braking system of the host vehicle based on the estimated driver's intention and does not consider driver warning or risk communication scenarios. Summary of the Invention [Means for solving the problem]

[0010] These and further problems are advantageously addressed by a computer implemented method according to independent claim 1. The dependent claims define further preferred embodiments.

[0011] A computer-implemented method assists an agent moving in a dynamic environment, with at least one other agent being present in the agent's environment. The method includes acquiring sensor information about the agent's environment, predicting at least one behavior of at least one of the agent and the at least one other agent based on the acquired sensor information, determining at least one human factor associated with the predicted at least one behavior, adapting the predicted at least one behavior based on the determined human factor, and determining whether communication of a consequence based on the adapted at least one behavior is beneficial to the operation of the agent. Such a consequence may be a collision or any other type of event resulting from the predicted behavior. If the method determines that communication is beneficial to the agent, it generates a signal based on the adapted at least one behavior and communicates at least one of the consequences of the predicted behavior or the predicted behavior to the agent based on the generated signal. The method outputs a communication to the agent according to the generated signal. Such a communication may be a warning informing the agent of a collision or other event that is a result of the predicted behavior or the predicted behavior itself.

[0012] An agent is a person that participates in a dynamic environment by acting, moving, or at least being present, or by operating any type of land, sea, or air vehicle to move within the dynamic environment. The resulting movement of the agent or the vehicle operated by the agent defines the behavior.

[0013] The human factor according to the present invention defines the individual aspects of a particular agent currently participating in a dynamic environment, making it possible to tailor predicted behavior to the individual, e.g., the current state of the agent.

[0014] Determining whether communication of at least one of a predicted behavior consequence or one of the predicted behaviors based on the adapted at least one behavior is beneficial to operation of the agent may include determining whether communication based on the adapted at least one behavior is beneficial to the agent or an overall traffic goal, e.g., safety.

[0015] Aspects and implementations of the present disclosure are described in the following description of specific embodiments in conjunction with the accompanying drawings. [Brief description of the drawings]

[0016] [Figure 1] 1 is a flowchart illustrating a process of a human factors assistance method for assisting an agent's actions, according to one embodiment. [Diagram 2] FIG. 1 is a simplified block diagram illustrating a process structure of a human factors assistance method for assisting an agent's behavior, according to one embodiment. [Diagram 3] 1 is an example of a system architecture configured to implement an embodiment of a human factors assistance method for assisting an agent's actions. [Figure 4] FIG. 1 is a diagram illustrating an application example of a support method using human factors for supporting the behavior of an agent in a road traffic environment. [Diagram 5] 1 is a diagram of a scenario in a road traffic environment, in which a support method for supporting the actions of an agent according to the prior art is applied; [Figure 6] FIG. 1 illustrates a further application of the human factors assistance method for assisting the behavior of an agent in a road traffic environment. [Figure 7] FIG. 1 illustrates an alternative application of the human factors assistance method for assisting the behavior of an agent in a road traffic environment. [Figure 8] FIG. 1 illustrates a further embodiment of the assistance method using human factors for assisting the operation of an agent in a road traffic environment, in which the human factors are used to perform an impact assessment. [Figure 9]FIG. 1 illustrates a further embodiment of the human factor assistance method for assisting the operation of an agent in a road traffic environment in an alternative scenario, using human factors for impact assessment. [Figure 10] FIG. 1 illustrates a further embodiment of the human factor assistance method for assisting the operation of an agent in a road traffic environment in an alternative scenario, using human factors for impact assessment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] In the description of the drawings, the same reference numbers are used for the same or corresponding elements in different figures. In the description of the drawings, detailed descriptions of the same reference numbers in different figures are omitted, where possible, without negatively affecting the comprehension.

[0018] The computer-implemented method provides the ability to predict the physical behavior of an agent and adapt the trajectory based on human factors in long-term predictions. Current driver assistance systems do not take into account the state of the agent and therefore may cause warning errors, but the computer-implemented method has the ability to consider human factors that affect the predictions. The method improves the driver assistance system by using human factors to improve predictions of the agent's behavior.

[0019] The human factors may include information about a human state, e.g., a human physical or mental state that influences the human's behavior when operating in an environment. The human mental state may include an emotional state, a human concentration, a human distraction, e.g., distraction from a primary driving task, or similar human factors. The human operating in an environment may include, among other things, how the human operates the vehicle or how the human perceives the environment, including, e.g., information output to the human via a human-machine interface. The agent may be a physical agent, e.g., a human operating a vehicle, or a physical agent itself. For example, human emotions may be inferred by performing an analysis of visual features in images taken of the human's face. Some emotions are correlated with more aggressive human behavior. And aggressive behavior increases the likelihood that the human driver will react in a way that reduces safety in a traffic scenario.

[0020] A computer-implemented method according to one embodiment includes determining at least one event involving an agent based on the predicted at least one behavior after adaptation and estimating a risk associated with signaling information regarding the determined at least one event to the agent. The method further includes adapting the adapted behavior further based on the estimated risk associated with signaling information regarding the determined at least one event to the agent, determining whether the estimated risk associated with signaling information regarding the determined at least one event to the agent is reduced based on the further adapted behavior, and generating and outputting a signal based on the further adapted at least one behavior if determining that the estimated risk is reduced.

[0021] Estimating the risk associated with signaling (communicating) information regarding at least one behavior includes estimating (determining) the impact that outputting information regarding at least one behavior, for example a warning of a predicted collision, has on a behavior or a human decision-making regarding the further development of the current scenario in the environment.

[0022] Thus, the signalling of information to the assisted person or agent takes into account specific human behavioural aspects and facilitates the safe progression of the current traffic scenario.

[0023] The method allows to estimate the risk and, in case of high risk, to evaluate the impact of the communication on the driver using human factors. If the communication is considered beneficial, the method proceeds to send an event signal to the own agent or to another agent.

[0024] Thus, the amount of information sent to the agent is limited to information that the agent actually benefits from, the provision of unnecessary information is avoided, and the output of annoying information is avoided, improving the acceptance of the assistance system.

[0025] A computer-implemented method according to one embodiment includes estimating a risk associated with signaling information regarding the at least one determined event includes evaluating an impact of signaling information regarding the at least one trajectory.

[0026] Thus, the assisted agent only obtains information about the predicted future travel path and travel-related information.

[0027] A computer-implemented method according to one embodiment includes estimating a risk associated with signaling information about at least one determined event, including predicting a human response to the information output in the signal, and enabling generation and output of the signal including the information if the estimated risk is determined to be decreased based on the predicted human response, and disabling generation and output of the signal including the information if the estimated risk is determined to be increased based on the predicted human response.

[0028] Thus, the output of information whose communication may increase risks in foreseen scenarios in a dynamic environment is suppressed, increasing the safety of the future evolution of traffic scenarios.

[0029] A computer-implemented method according to one embodiment includes the determined at least one human factor including information about at least one of the agent, at least one other agent, or a subset or all of the other agents in the agent's environment.

[0030] A computer-implemented method according to one embodiment includes adapting at least one predicted behavior of the agent, at least one other agent, or at least all other agents in the agent's environment based on the determined human factors.

[0031] A computer-implemented method according to one embodiment includes adapting the adapted behavior further based on an estimated risk associated with signaling information regarding the determined at least one event to at least one of the agent, at least one other agent, or all other agents in the agent's environment.

[0032] The method can be easily adapted from improving ACC systems integrated into advanced driver assistance systems that assist the driver of the own vehicle to a combined and comprehensive analysis of human factors provided by multiple humans operating in the same dynamic environment.

[0033] A computer-implemented method according to one embodiment includes updating a human model based on a determined response of at least one of the agent and at least one other agent based on predicted behavior in a previous processing cycle.

[0034] The method makes it possible to improve the human model used to determine the human factors based on previous experience in a learning process integrated into the application phase of the assistance system and without the need to (re)enter a training phase.

[0035] A computer-implemented method according to one embodiment includes determining a weight associated with the human factor based on a certainty of the human factor, and determining an impact of the human factor on the predicted behavior based on the determined weight.

[0036] The weighting of human factors allows for efficient handling of multiple human factors within the assistance system and for managing the influence of individual human factors on the assistance system's predictions.

[0037] According to one embodiment, the computer-implemented method, in determining at least one human factor associated with at least one predicted behavior, includes determining a combination of human factor information associated with the agent and human factor information associated with at least one other agent to determine an overall uncertainty associated with the at least one human factor.

[0038] Thus, the assistance system considers human factors of multiple agents, each of which affects the uncertainty of at least one human factor, and refines the further processing of the determined human factors, the uncertainty for multiple human factors of multiple agents, so that the processing requirements are relaxed.

[0039] A computer-implemented method according to one embodiment includes determining whether communication based on the adapted at least one behavior is beneficial to the agent and at least one other agent, including determining whether it is beneficial to output communication to the agent and at least one other agent at different times, and if determining that communication at different time steps is beneficial, generating and outputting a generated signal to the agent and at least one other agent at different times.

[0040] Communicating information to different agents at different times may increase security in future evolutions of current scenarios of dynamic environments, and this embodiment makes it computationally efficient to benefit from this increased security.

[0041] A computer-implemented method according to one embodiment includes determining whether communication based on the adapted at least one behavior is beneficial to the agent and at least one other agent, including determining whether outputting the communication to the agent will have a negative impact on the at least one other agent.

[0042] Thus, the interdependence of the information communicated to different agents is recognised and may be used to increase the safety of future evolutions of current traffic scenarios.

[0043] A computer-implemented method according to one embodiment includes determining whether communication based on the adapted at least one behavior is beneficial to the agent and at least one other agent, which includes predicting and simulating behavior variations of the agent and the at least one other agent in response to a plurality of communication candidates, determining, for each communication candidate, an effect of outputting a communication to the agent or the at least one other agent, and selecting a communication candidate or a combination of communication candidates based on the determined effect of the communication candidates.

[0044] Thus, multiple behavioral and communication options offered by the assistance system may be analyzed taking into account human factors in order to select an appropriate option with particular suitability for execution by a human agent.

[0045] A computer-implemented method according to one embodiment includes communicating the generated signal to at least one other agent for output via a human-machine interface of the at least one other agent.

[0046] The method is suitable for application in assistance systems incorporating V2X communication for improved safety in road traffic scenarios.

[0047] A computer-implemented method according to one embodiment includes predicting at least one behavior of at least one of the agent or at least one other agent based on acquired sensor information using a physical prediction for a first time into the future, predicting the at least one behavior for a second time into the future, and adapting the predicted at least one behavior based on determined human factors. The first time is shorter than the second time. In one example, the first time ranges from 0 to 2 seconds, and the second time includes a time greater than 2 seconds. In alternative embodiments and application scenarios, a threshold value larger than the cited 2 seconds separating the first time including the physical prediction from the second time including adapting the predicted at least one behavior based on determined human factors may be advantageous. This is particularly true in embodiments that may require a larger threshold value between the first time and the second time for the physical prediction before any, e.g., less reliable, human factors are taken into account.

[0048] The method is particularly advantageous for improving long-term prediction of behavior in assistance systems, since the human component that is introduced with human factors to adapt the physical predictions provides particularly advantageous results for prediction periods that extend long into the future. For example, adapting the predicted behavior based on human factors information refines predictions for prediction periods where humans may or may not act depending on the evolution of the current scenario.

[0049] A computer-implemented method according to one embodiment includes: the physical prediction includes longitudinal behavior including at least one of a constant speed, a constant deceleration or constant acceleration, and a delayed constant acceleration; the physical prediction includes lateral behavior including at least one of moving at a constant turning angle, moving along a map path, changing lanes, and taking a left path, a straight path, or a right path at an intersection; the physical prediction includes environmental parameters, in particular a road slope angle, a road curvature radius; predicting the at least one behavior and adapting the predicted at least one behavior includes longitudinal behavior including at least one of a behavior change based on a driver model and a behavior change based on a driver state. and in particular, the behavior change includes at least one of a change in a deceleration or acceleration value, a change from a constant speed to a constant acceleration at a future time, and a change from a constant acceleration to a constant speed at a future time; and in particular, the behavior change includes a delay based on a determined driver state, the determined driver state including one of an attentive state, a drowsy state, a startled state, a distracted state, an experienced driver state, an unskilled state, and a disoriented state; and predicting the at least one behavior and adapting the predicted at least one behavior includes lateral behavior including a lane change, a left, straight, or right turn at an intersection, and weaving around a center line or the average path of the agent.

[0050] The decision to change lanes in a road traffic scenario may be related to human factors. As an example, a startled driver, which corresponds to a driver in a determined startled state (startled mental state), may make abrupt lane changes in a particular traffic scenario, unlike a driver determined to be in a non-startled state (non-startled mental state) in the same traffic scenario. As a result, the startled driver approaches other vehicles traveling closely in adjacent lanes. At an intersection, the human factors included in the human factor information may change the driving style of the person driving the vehicle. At an intersection, the driving style of how a left turn, right turn, or straight is performed by the driver may differ depending on the determined human factors. For example, a driver in a drowsy state may be more likely to drive outside the boundary line of his lane while turning left at an intersection. As a result, adapting the prediction based on the determined human state of the driver can improve the quality of the physical prediction. This assistance method can be easily incorporated into various assistance systems currently used in automotive scenarios.

[0051] A computer-implemented method according to one embodiment includes estimating risk, where estimating risk involves signaling information regarding the determined at least one event to an agent, and further involves signaling information regarding the determined at least one event to at least one other agent, and if it is determined that outputting the signal and the further signal reduces the estimated risk, generating and outputting a signal to the agent and generating and outputting a further signal to the at least one other agent.

[0052] Thus, the method is applicable to scenarios involving multiple agents and their associated respective human factors.

[0053] A computer-implemented method according to one embodiment includes executing the computer-implemented method by at least one processor co-located with the agent.

[0054] A computer-implemented method according to one embodiment includes performing the computer-implemented method in a distributed manner by at least two processors of the agent, at least one other agent, and at least one server that is remote from the agent and the at least one other agent.

[0055] The method steps of integrating information and behavior prediction may be performed locally, for example in the computational resources of the agent, or by a central instance having fast access to data from multiple traffic parties, environmental sensors, databases and means for selectively distributing communication signals between traffic parties based on the generated outputs.

[0056] For example, in some embodiments, vehicle sensors may acquire user and vehicle data, which the vehicle transmits to an external computational node, e.g., a remote server. The computational node may receive data from various additional sources and generate a more comprehensive model of the traffic scenario than would be possible for the individual agents involved in the local traffic scenario.

[0057] A computer-implemented method according to one embodiment includes transmitting information regarding the human factors and associated effects of the human factors on the predicted behavior to at least one of at least one other agent and at least one remote server.

[0058] A computer-implemented method according to one embodiment includes executing the computer-implemented method within an advanced driver assistance system of a host vehicle.

[0059] A computer-implemented method according to one embodiment includes performing at least some steps of the computer-implemented method by a remote server remote to the agent.

[0060] Thus, the computer-implemented method may implement selectivity based on usefulness, i.e., the external computing node may selectively distribute information to relevant agents, e.g., vehicles and devices, for which the information is determined to be useful. The external computing node may also consider that some computing and communication between agents may be performed locally.

[0061] The presence of another vehicle closely following the own vehicle in the same lane (close-following vehicle) is relevant for safety. Nevertheless, if the driver of the own vehicle tends to react to the notification about the presence of the close-following vehicle by increasing his / her own speed (speeding up), it is not necessarily useful to inform the driver of the own vehicle of the presence of the close-following vehicle. In this case, the human factor including the "tendency to react by speeding up" becomes an individual and statistically determined human factor, which may disable the sending of information by the assistance system to the own driver in the scenario described here. In contrast, another driver of the own vehicle who "tends to react to the notification about the close-following vehicle by increasing the safe distance to another vehicle traveling in the same lane and in the same direction as the own vehicle (the leading vehicle)" in order to obtain a smoother braking margin to reduce the risk of a rear impact collision will use this information in a beneficial way, and the assistance system should act accordingly. If human factor information about what individual drivers "tend to do" is not available, the average driver response may be considered as human factor information in some embodiments or scenarios.

[0062] Thus, the computer-implemented method may implement a prohibition function, where in case of risk of conflict between the local conclusion reached by the agent and the global conclusion derived by the external computational node, either the local agent or the external computational node may prohibit the propagation of a particular notification from the other source to prevent a predicted negative impact from occurring. The authority in such cases may depend on the respective trust measures. For example, if the predicted negative impact of a particular notification by the external computational node on a particular agent has a higher trust than the predicted positive impact by the local computational node, for example on the agent itself, the external computational node may prohibit notifications from the local computational node, and vice versa.

[0063] According to one embodiment of the computer-implemented method, the determined event is a predicted collision involving the agent or a near collision involving the agent.

[0064] According to one embodiment of the computer-implemented method, the at least one behavior includes at least one trajectory of at least one of the agent or the at least one other agent.

[0065] According to one embodiment of the computer-implemented method, the agent and the at least one other agent include at least one of a pedestrian, a bicyclist, a motorcyclist, and a driver of a road vehicle in a road traffic environment.

[0066] Humans operating vehicles in a dynamically changing road traffic environment may benefit from a computer-implemented method that provides specific predictions of the road traffic environment that take into account human factors relevant to the adaptation of predictions and the evaluation of events. Thus, the quality of predictions and the assistance provided by driver assistance systems is improved. Acceptance of assistance by users of assistance systems implementing this method is increased.

[0067] FIG. 1 is a flow chart illustrating steps of a human factors forecasting method for supporting agent behavior, according to one embodiment.

[0068] The process steps S1-S11 of the computer-implemented method illustrated by the flow chart of Fig. 1 may be applied in an advanced driver assistance system to assist a human to operate a vehicle in a traffic environment. The block diagram of Fig. 2 shows a schematic structure of the processing modules of the computer-implemented method, and Fig. 3 shows the structural elements and considerations of the assistance system 1 for assisting an agent to move in a dynamic traffic environment. The agent and at least one other agent include at least one of a pedestrian, a cyclist, a motorcyclist, and a driver of a road vehicle in the road traffic environment.

[0069] The computer-implemented method begins by acquiring sensor information about the agent's environment in steps S1, S2, and S3. In particular, in step S1, the method monitors the agent's environment. In step S2, the method monitors the agent, e.g., acquiring sensor information about an ego-vehicle. In step S3, the method performs human monitoring, e.g., including monitoring a human driver of the ego-vehicle.

[0070] The method may perform steps S1, S2, and S3 in parallel. In addition, the method may perform steps S1, S2, and S3 periodically, for example at least once for each processing cycle of the computer-implemented method. In addition, the computer-implemented method may perform steps S1, S2, and S3 continuously during execution of the method.

[0071] In step S4, the method continues with predicting at least one behavior of at least one of the agent or the at least one other agent based on the acquired sensor information. In particular, step S4 may include making a physics prediction to predict a behavior of the agent and at least one other agent in the agent's environment based on environmental information acquired by the at least one environmental sensor 2 and information relating to the agent acquired by the at least one agent sensor 3.

[0072] In step S5, the method determines at least one human factor associated with the at least one predicted behavior based on the acquired sensor information. In particular, step S5 may include determining at least one human factor associated with the at least one predicted behavior based on obtained human information of the agent acquired by at least one human monitoring sensor. The human monitoring sensor may be a driver sensor 2 adapted to determine at least one parameter (human state parameter) for determining a human state associated with influencing human behavior in the environment.

[0073] The at least one determined human factor includes information about at least one of the agent, at least one other agent, and a subset or all of the other agents in the agent's environment.

[0074] In step S6, following steps S4 and S5, the method continues by adapting at least one predicted behavior of the agent, at least one other agent, or at least one of all other agents in the agent's environment based on the determined human factors.

[0075] In step S6, the method adapts the at least one predicted behavior of step 4 based on the determined human factors of step S5.

[0076] Step S6 may include determining a weight associated with the determined human factor based on the certainty of the determined human factor, and determining an impact of the determined human factor on the predicted behavior based on the determined weight.

[0077] 1, the computer-implemented method of one embodiment may proceed directly from adapting predictions based on human factors in step S5 to step S11. In step S11, if the method determines that communication would be beneficial to the agent, it generates a signal based on the adapted at least one behavior. The method then outputs the generated signal to the agent to assist the agent in operating within the environment.

[0078] The computer-implemented method of the preferred embodiment further includes steps S7 to S10.

[0079] Step S6 is followed by step S7, where the method determines at least one event involving the agent based on the at least one predicted behavior after the adaptation of step S6. If the method determines in step S7 that there are no relevant events in the environment based on the adapted predictions, the processing of the method ends. If the method determines in step S7 that there is at least one relevant event based on the adapted predictions, the processing proceeds to step S8.

[0080] In step S8, the method estimates a risk associated with signaling information about the determined at least one event to the agent, in particular estimating the risk includes determining the impact of signaling information generated based on the adapted prediction to the agent based on the human factors determined in step S5.

[0081] In step S8, estimating a risk associated with signaling information about the determined at least one event may include evaluating the impact of signaling information about the at least one trajectory to the agent.

[0082] Estimating a risk associated with signaling information regarding the determined at least one event may include predicting a human reaction to the information output in the signal to the agent.

[0083] In step S8, the method may determine whether communication based on the adapted at least one behavior is beneficial to the agent and the at least one other agent, which may include determining whether outputting the communication to the agent will have a negative impact on the at least one other agent.

[0084] In step S8, determining whether communication based on the adapted at least one behavior is beneficial for the agent and the at least one other agent includes predicting and simulating behavior variations of the agent and the at least one other agent in response to a plurality of communication candidates. For each communication candidate, the method then determines an effect of outputting that communication candidate to the agent or the at least one other agent, and selects a communication candidate or a combination of communication candidates based on the determined effect of the communication candidate.

[0085] When evaluating the alerts or assistance of multiple agents in the environment, the method may perform additional functions.

[0086] The agent warnings may be made at different times, for example, first warning vehicle A so that vehicle A acts on vehicle B and vehicle A has sufficient distance to vehicle B, and then warning vehicle B of vehicle A for a possible future interaction such as lane change. This allows to eliminate the negative impact of warnings on one or more drivers of the vehicles. Additionally or alternatively, the method may check whether warnings of multiple agents are beneficial, for example, whether the modality of warning vehicle A negatively affects vehicle B. This may be the case when A is a pedestrian and is warned through audio. Other pedestrians may be distracted by the warning of pedestrian A, but should receive their own warning to mitigate a possible collision. The evaluation may be made by predicting and simulating various variations of the predicted behavior and checking the impact of the aggregate warning for all simulated variations of the predicted behavior. The best combination of warnings is applied to the vehicles in the driving situation to reduce the overall driving situation risk of the current scenario in the traffic environment.

[0087] Evaluating the warning or assistance of the multiple agents is based on simulating predicted behavior variations representative of possible traffic developments, the range of possibilities being expanded by considering human factors for different variations or possibilities of communicating a risk with a warning signal to the multiple agents or not communicating a risk to at least some of the agents. Some human factors affect the distribution of reaction times, others affect the safety margin and lane change probability in the predicted future development of the traffic scenario. The method may select among the simulated variations of predicted behavior and output a signal to the multiple agents by performing an optimization process that minimizes a risk index that considers the accident probability and potential severity for each combination of predicted behaviors.

[0088] In step S9, the method continues by further adapting the predicted behavior based on an estimated risk associated with signaling information about the determined at least one event to the agent.

[0089] Step S9 includes further adapting the predicted behavior based on the estimated risk and further based on the determined human factors, which may include adapting the predicted at least one behavior of the agent, the at least one other agent, or at least one of all other agents in the agent's environment.

[0090] Step S9 is followed by step S10, in which the method determines whether the estimated risk associated with signaling information about the determined at least one event to the agent is reduced based on this further adapted behavior. If step S10 determines that the risk is reduced ("YES"), the method proceeds to step S11.

[0091] If step S10 determines that the estimated risk has decreased, then in step S11 the method generates and outputs a signal based on the at least one further adapted behavior. In particular, since the determined signaling impact of step S8 indicated that the signaling is favorable for the future development of the scenario in the environment, the method may communicate the determined relevant event to the agent.

[0092] If step S10 determines that the risk has increased ("NO"), the method may end for that particular agent. In particular, if the processing according to the method indicates that the signaling based on the adapted behavior, based on the determined signaling impact (estimated risk) of step S8, is detrimental to the future development of the environmental scenario.

[0093] The method may include enabling generation and output of the information-containing signal if a decrease in the estimated risk is determined based on a predicted human response, and disabling generation and output of the information-containing signal if an increase in the estimated risk is determined based on a predicted human response to the signal notification.

[0094] The method may include, in step S8, determining whether communication based on the adapted at least one behavior is beneficial to the agent and the at least one other agent includes determining whether it is beneficial to output the communication to the agent and the at least one other agent at different times. If determining that communication at different time steps is beneficial, the method proceeds to step S11, generating and outputting the generated signal to the agent and the at least one other agent at different times.

[0095] In step S11, the method may include communicating the generated signal to at least one other agent for output via a human-machine interface of the at least one other agent. In step S8, estimating risk includes estimating a risk associated with signaling information about the determined at least one event to the agent, and estimating a risk further associated with signaling information about the determined at least one event to the at least one other agent. And, in step S11, signaling the relevant event includes generating and outputting a signal to the agent, and generating and outputting a further signal to the at least one other agent if it is determined that outputting the signal and the further signal would reduce the estimated risk.

[0096] The method may further include updating the human model based on a determined reaction of at least one of the agent and the at least one other agent based on a predicted behavior in a previous processing cycle after signaling the relevant event to the agent. FIG. 1 illustrates the updating of the human model by link AA between steps S11-S5. The method may determine the reaction based on sensor information from the environment, the agent, or the human. Updating the human model based on the determined reaction of at least one of the agent and the at least one other agent may be based on a predicted behavior after adaptation obtained from a previous processing cycle (iteration) of the method.

[0097] FIG. 2 shows a simplified block diagram illustrating the processing structure of a method for forecasting using human factors to support the behavior of an agent, according to one embodiment.

[0098] A driver assistance system 1 configured to perform a computer-implemented method according to an embodiment includes a prediction module 7 .

[0099] The prediction module 7 makes physics predictions based on the vehicle and environmental information 5 provided by the environmental sensors 2 and the agent sensors 3 .

[0100] The environmental sensor 2 may include a sensor group including a plurality of sensors including a camera, a RADAR sensor, a LIDAR sensor, and an acoustic sensor for acquiring sensor information including environmental information regarding the agent's environment.

[0101] The agent sensors 3 may include sensors that acquire position data, orientation data, and movement data of an agent assisted by the assistance system 1.

[0102] The driver sensors 4 may include sensors that obtain human information 6 about the human operator of the agent assisted by the assistance system.

[0103] The prediction module 7 may perform physics prediction to generate predicted longitudinal behavior including at least one of constant velocity (CV), constant deceleration or constant acceleration (CA), and delayed constant acceleration of the agent. The physics prediction module 7 predicts physical behavior of the agent and at least one other agent based on vehicle information and environmental information 5 provided by the environmental sensors 2 and the agent sensors 3. The predicted behavior may include at least one predicted trajectory of the agent.

[0104] The physical predictions may include predicted lateral behavior including at least one of moving at a constant turn angle, moving along a map path, changing lanes, and taking a left path, a straight path, or a right path at an intersection.

[0105] The physical predictions may include predicted environmental parameters, in particular road slope, road curvature radius, etc. The environmental parameters may be known from map data and the timing may be based on predictions.

[0106] The driving assistance system of this embodiment further includes a human model determination module 8. The human model determination module 8 determines a human factor 10 based on the human information 6 provided by the driver sensor 4, and provides the determined human factor 10 to the (first) predictive adaptation module 9 and the event signal notification impact determination module 12. The human model determination module 8 may determine the human factor 10 based on the human information 6 obtained through access to stored information, for example, record information about a particular human driver. The human model determination module 8 may determine the human factor 10 based on the human information 6 generated and stored in the preceding step of human modeling the driver. The human model determination module 8 may determine the human factor 10 based on the human information 6 obtained by V2X communication through the network N.

[0107] The prediction adaptation module 9 adapts the predicted behavior provided by the prediction module 7. The adapting of the at least one predicted behavior by the prediction adaptation module 9 may include adapting a longitudinal behavior including at least one of a behavior change based on a driver model or a behavior change based on a driver state, and the adapting of the at least one predicted behavior includes a lateral behavior including a lane change, a left turn, a straight turn, or a right turn at an intersection, and swerving around a center line.

[0108] In particular, the behavior change may include at least one of a change in deceleration or acceleration value, a change from a constant velocity to a constant acceleration at a future time, and a change from a constant acceleration to a constant velocity at a future time.

[0109] Alternatively or additionally, the behavior change may include a delay based on a determined driver state, the determined driver state including, for example, one of an attentive state, a drowsy state, a startled state, a distracted state, an experienced driver state, an unskilled state, and a disoriented state.

[0110] The predictive adaptation module 9 provides the predicted behavior after adaptation to the relevant events determination module 11 .

[0111] The relevant event determination module 11 determines at least one relevant event in the agent's environment involving the agent and at least one other agent based on the predicted behavior after adaptation provided by the predictive adaptation module 9.

[0112] The event signal notification impact determination module 12 obtains at least one related event determined by the related event determination module 11, and estimates an impact of a signal notification for each determined related event. The event signal notification impact determination module 12 may determine the impact of a signal notification based on human factors 10 provided by the human model determination module 8 for the agent only.

[0113] Additionally or alternatively, the event signal notification impact determination module 12 determines the impact of the signal notification on the agent and other agents based on the human factors 10 provided by the human model determination module 8 .

[0114] Based on the determined signalling influence on the predicted behaviour of the agent and possibly other agents, the assistance system 1 may further adapt the adapted predicted behaviour provided by the predictive adaptation module 9 in a (second) predictive adaptation module 13 based on the determined event signalling influence.

[0115] The assistance system may perform these processes iteratively in the associated event determination module 11, the event signal notification impact determination module 12, and the predictive adaptation module 13, as shown by the feedback loop 14 from the predictive adaptation module 13 to the associated event determination module 11.

[0116] The prediction adaptation module 13 provides the further adapted behavior to the event signal generation module 15, which generates an output event signal 16 based on the further adapted prediction and controls the output of the output event signal 16 via at least one output device 28 of the agent.

[0117] In a particular embodiment of the computer-implemented method, processing proceeds directly from adapting the prediction in the prediction adaptation module 9 to event signal generation of the output event signal 16 without determining at least one event and analyzing the impact of each determined associated event in the associated event determination module 11, the event signal notification impact determination module 12 and the second prediction adaptation module 13.

[0118] The prediction module 7, the human model determination module 8, the predictive adaptation module 9, the related event determination module 11, the event signal notification impact determination module 1, the predictive adaptation module 13, and the event signal generation module may be implemented as software running on at least one computer or processor having associated memory. The at least one processor having associated memory may include, for example, a processor 21 and a memory 22 of the host vehicle A.

[0119] The computer-implemented method may be used for various driver assistance systems in the automotive field based on predicting the behavior of an agent. Such driver assistance systems may include, for example, systems and methods based on the application of risk maps, such as US Patent Application Publication No. 2020 / 0231149(A1) for priority-based planning for intersections. The method assists driving of an ego-vehicle and includes the steps of searching for a priority relationship between the ego-vehicle (ego-agent) and at least one other traffic actor (other agent) involved in a traffic situation, selecting a prediction model for the at least one traffic actor according to the priority relationship, predicting finally one hypothetical future trajectory for the ego-vehicle, predicting at least one hypothetical future trajectory for the at least one traffic actor based on the selected prediction model, and calculating a behavior-related score for the ego-vehicle based on the calculated hypothetical future trajectory.

[0120] A further driver assistance system that may incorporate a computer-implemented method according to one embodiment is disclosed in U.S. Patent Application Publication No. 2022 / 316897(A1), which shows a system for visualizing an individual's risk space.

[0121] The prediction module 7 may be incorporated in the advanced driver assistance system disclosed in US Pat. No. 8,903,588 (B2) to perform prediction, and computationally predicts future movement behavior of at least one object by generating sensor data by at least one sensor that physically senses the environment of the host vehicle, and calculating multiple movement behavior options for the object sensed by the sensor. The context-based prediction step uses a set of classifiers, each classifier estimating the probability that the sensed object will perform a certain movement behavior at a certain time. The method also includes verifying the movement behavior options according to the physical prediction by comparing the measured points with the trajectory of the situation model, determining at least one trajectory indicating at least one possible behavior of the traffic participants, estimating at least one future position of the traffic participants based on the at least one trajectory, and outputting a signal representative of the estimated future position.

[0122] US Patent No. 9,969,388 (B2) discloses the calculation of multiple movement behavior options for a target object sensed by a sensor. The calculation includes predicting the movement behavior of the target object by applying a context-based prediction using at least one indirect indicator and / or a combination of indicators derived from the sensor data. In the context-based prediction, a probability that the target object will perform a certain movement behavior is estimated. A future position of the target object is estimated and a signal representative of the estimated future position is output. At least one historical indicator for one movement behavior option is generated for the current time using at least one indicator of the indirect indicators at a time in the past.

[0123] The prediction module 7 may perform prediction based on the disclosure US Pat. No. 9,620,008 (B2), using global scene context for adaptive prediction.

[0124] US Patent No. 9,308,919(B2) also discloses useful information related to a driver assistance system including a prediction subsystem in a vehicle. US Patent No. 9,308,919(B2) includes steps of accepting a set of base environmental representations, assigning a set of base reliability estimates, assigning weights to the base reliability estimates, calculating a weighted composite reliability estimate for the composite environmental representation, and providing the weighted composite reliability estimate as an input for evaluating predictions based on the composite environmental representation.

[0125] EP2840007B1 relates to a driver assistance system for a vehicle, comprising generating a decision signal by a first evaluation of sensor data acquired by a sensor means, generating an activation signal for an activation means when the decision signal exceeds a signal threshold, generating an interrupt decision signal based on a second evaluation, stabilizing the activation signal in time, deciding based on the decision interrupt signal whether to interrupt the stabilization of the activation signal, and interrupting the stabilization of the activation signal if it is decided to interrupt the stabilization of the activation signal.

[0126] EP 2 845 779 B1 relates to generating a consistent behavior of a predictive advanced driver assistance system, comprising generating a decision signal by a first evaluation of sensor data acquired by a sensor means, generating an activation signal for an activation means when the decision signal exceeds a signal threshold, generating an interrupt decision signal based on a second evaluation, stabilizing the activation signal in time, deciding whether to interrupt the stabilization of the activation signal based on the decision interrupt signal, and interrupting the stabilization of the activation signal if it is decided to interrupt the stabilization of the activation signal.

[0127] U.S. Patent No. 10,625,776 (B2) discloses a reliability estimation for a predictive driver assistance system based on plausibility rules and relates to driver assistance techniques for active vehicle control, which may be used in combination with the present disclosure.

[0128] 3 shows various examples of system architectures suitable for implementing embodiments of a method for predicting using human factors to assist the behavior of an agent. FIG. 3 shows system architectures suitable for implementing various embodiments of a driver assistance system for assisting a driver to operate an ego-vehicle A in a road traffic environment.

[0129] A computer-implemented method for assisting an agent according to one embodiment may be implemented as software executed by at least one processor 21 of the ego-vehicle A.

[0130] In an alternative embodiment, the computer implemented method for assisting an agent is implemented as software executed by processors of at least two of the vehicles A, B in a road traffic environment.

[0131] In yet another alternative embodiment, the computer-implemented method is executed in a distributed manner on processors 21 of the agent (own vehicle A), at least one other agent (other vehicle B), and at least one server that is remote from the agent and the at least one other agent.

[0132] The servers 31, 32 may include a server 31 having a human-machine interface that allows access to, control and modification of the processing of the computer-implemented method according to one embodiment and other elements of the driver assistance system.

[0133] Each server may include one or more processors, e.g., a central processing unit (CPU), a signal processor, and associated memory, including, e.g., read only memory (RAM), read only memory (ROM), and disk data storage, flash memory, to name a few known structural elements.

[0134] The servers 31, 32 may include at least one server 32 configured to control and obtain information from traffic infrastructure equipment located in the road traffic environment, such as traffic lights or cameras 34 monitoring the road traffic environment.

[0135] Apart from the distribution of processing of the computer-implemented method, the driver assistance system 1 of the ego vehicle A may obtain sensor information about the environment of the ego vehicle A from its own sensors 25, 26 or from sensors external to the ego vehicle A via the network N and the communication interface 26 of the ego vehicle A. In Fig. 3, the sensors arranged external to the ego vehicle A include a camera 34 and may also include sensors of a sensor group of the other vehicle B. The sensors may also include a navigation system, for example a navigation receiver of a Global Navigation Satellite System (GNSS) providing navigation services using a number of navigation satellites 35.

[0136] Vehicle A includes a number of sensors 25, 26 for monitoring the environment of vehicle A, monitoring the driver of vehicle A, and providing performance information of vehicle A.

[0137] The information obtained from the environment of the host vehicle may include information regarding the position, orientation, speed, acceleration or deceleration of the host vehicle A and another vehicle B present in the environment of the host vehicle A.

[0138] The information about the host vehicle A may include information about the position, orientation, speed, acceleration or deceleration, throttle position, steering angle, gear selection, and motor parameters, such as the rotational speed or torque value of the drive motor 24.

[0139] The information about the driver of the host vehicle A may include information obtained by a camera or microphone monitoring the interior, including the driver, of the host vehicle A. The information about the driver of the host vehicle A may include any information that enables the driver assistance system 1 to determine a driver state, with the driver state being one particular example of a human factor used in a computer-implemented method according to one embodiment.

[0140] An agent, e.g., the own agent or another agent, may include a human having a personal computing device 29, e.g., a smart device such as a smartphone, that communicates with other elements in the own agent's environment via the network N. For example, the own agent may be a human user with a personal computing device 29.

[0141] The ego-vehicle A includes a communication interface 26. The communication interface 26 enables wireless communication between the ego-vehicle A and other agents and with the infrastructure of the road traffic environment. In particular, the communication interface may enable direct communication with other vehicles B in the ego-vehicle A's environment, which are also equipped with respective communication interfaces.

[0142] The communication interface 26 may enable communication via a network N with other elements of the road traffic environment. The other elements may include other vehicles B, and servers 31, 32.

[0143] The communication interface 26 may enable communication via the network N with humans, such as pedestrians and cyclists, who have personal devices 29, such as smartphones, that are also part of the road traffic environment.

[0144] The communication interface 26 may enable communication incorporating the vehicle A into a V2X system, for example by sending and receiving signals to and from other vehicles and traffic infrastructure.

[0145] The host vehicle A in FIG. 3 further includes a data storage device 22 (memory 22).

[0146] The ego-vehicle A of FIG. 3 further comprises an output module 28. The output module 28 may comprise means for outputting information to the driver of the ego-vehicle A via different modalities, in particular visual, acoustic or haptic. The output module 28 forms part of the human-machine interface of the driver assistance system 1 of the ego-vehicle A. The output module 28 may comprise a head-up display for providing information to the driver of the ego-vehicle without the driver having to divert his attention from the road traffic environment. The output module 28 may comprise at least one of at least one loudspeaker for outputting an acoustic signal, at least one display for outputting a visual signal or haptic output means for outputting a haptic signal to the driver of the ego-vehicle A. The processor 21 may change the modality of the output signal to the driver. Alternatively or additionally, the processor 21 may adapt the strength, e.g. the output intensity, of the signal conveying information to the driver of the ego-vehicle A.

[0147] FIG. 4 shows a first example of the application of the prediction method using human factors to support the behavior of an agent in a specific road traffic environment.

[0148] The road traffic scenario shown in the upper part of Figure 4 is a vehicle moving at a constant speed V along the road lanes. A The vehicle A is moving at a speed of 0.01 m. The vehicle B is moving ahead of the vehicle A in the direction of travel of the vehicle A at a speed of 0.01 m. B Vehicle A is traveling in the same lane and in the same direction as vehicle B. The distance between vehicle A and vehicle B at time t = 0 is d 0 The vehicle B decelerates at a constant rate to the first speed V B = 50km / h to second speed V B =Slow down to 30km / h.

[0149] A driver assistance system assisting the driver of vehicle A analyses the traffic scenario of Fig. 4 and predicts the future development of this traffic scenario based on information acquired by sensors 2, 3 from an environment corresponding to the traffic scenario illustrated in Fig. 4. The driver assistance system not only makes physical predictions based on the acquired information about the environment, but also determines human factors, which are illustrated in Fig. 4 by the human states "attention driver", "drowsy driver" and "startled driver".

[0150] The method shown in FIG. 4 predicts at least one behavior of at least one of the agent or at least one other agent based on acquired sensor information using physical predictions over a first time into the future, predicts the at least one behavior over a second time into the future, and adapts the predicted at least one behavior based on determined human factors.

[0151] Forecasts using physical forecasts are short-term forecasts. Forecasts using adapted forecasts based on human factors are typically long-term forecasts.

[0152] Distinguishing between short-term and long-term predictions allows separating time periods with different importance, ensuring that unreliable information is hardly or preferably not included in the prediction of critical events, which are events predicted to occur within a particularly short time period. In the short term, the assisted driver has no or only a short time to react to the event to avoid the predicted critical event occurring. Human factors information may be less reliable due to the difficulty of determining the mental state, which is not completely clear and whose impact on the situation may vary greatly between individual people. Therefore, less reliable human factors information is most suitable for adjusting the medium-term and long-term predictions of future behavior and the development of the current scenario in a dynamic traffic environment.

[0153] An implementation of the method may preferably use a single prediction process with changes based on determined human factors after a certain point in the prediction. This point or threshold separates short-term prediction from long-term prediction. Splitting the prediction into two time periods is preferred because, as mentioned above, mental states are not completely evident and their impact on situations may vary significantly between individuals and for one individual at different times, so the determined human factors information and the adapted prediction based thereon may be less reliable than the physical prediction alone. Such less reliable information is therefore preferably used to adjust the medium to long-term prediction when implementing the method. The medium to long-term prediction may be thought of as adjusting the underlying physical prediction beyond a threshold time from the current time, beyond which the use of human factors information provides good or acceptable results for a given embodiment and scenario. The threshold time may be, for example, 2 seconds.

[0154] In the particular example shown in FIG. 4, the first period ranges from 0 to 2 seconds, and the second period begins at 2 seconds and extends to 10 seconds.

[0155] The center part of Figure 4 shows the predicted velocity V A The curve characteristics of the vehicle A are shown. The driver assistance system that assists the driver of the vehicle A uses a physical prediction process for short-term prediction. The short-term prediction is based on the motion parameter values ​​and in the illustrated example extends from the current time t=0 to 2 seconds into the future.

[0156] The long-term prediction is based on motion parameter values ​​and adapts the physics prediction based on determined human factors, and in the illustrated example begins at a future time t=2 seconds and extends 10 seconds into the future.

[0157] In the short-term prediction, the driving assistance system determines, in particular, the trajectories of vehicles A and B, their respective speeds along the trajectories V A and V B The predicted trajectory and velocity V A and V BBased on this, the driver assistance system predicts a linearly decreasing distance d for future times during the prediction period. In the road traffic scenario shown in Fig. 4, the driver assistance system concludes that the predicted distance d is sufficient as a safety distance between vehicles A and B during the prediction period of the short-term prediction, and therefore no imminent risk of a collision event involving vehicles A and B is determined in Fig. 4.

[0158] In the long term, driver assistance systems will be A Let the speed of vehicle B be V B 4. Thus, in the road traffic scenario shown in Fig. 4, the driver assistance system concludes that during the prediction period of the long-term prediction, the predicted distance d may again be sufficient as a safety distance between vehicle A and vehicle B, and therefore, in the long-term prediction as in the short-term prediction, no imminent risk of a collision event involving vehicles A and B is determined in Fig. 4.

[0159] As a result, the driver assistance system may forgo generating and outputting information, including warnings of impending collisions.

[0160] Figure 5 shows a scenario in a road traffic environment, where a prior art prediction method for supporting the behavior of an agent is applied: Vehicle A is the ego vehicle, whose driver is supported by a prior art driver support system.

[0161] The driver assistance system of FIG. 5 may be a distance warning system or may be part of an adaptive cruise control system.

[0162] The road traffic scenario shown in the upper part of Figure 5 is a vehicle moving at a constant speed V along the lanes of a road. A The vehicle A is moving at a speed of 0.01 m. The vehicle B is moving ahead of the vehicle A in the direction of travel of the vehicle A at a speed of 0.01 m. B At time t = 0, vehicle A and vehicle B are traveling in the same lane and in the same direction. The distance between vehicle A and vehicle B is d 0 It is.

[0163] Vehicle B decelerates at a constant rate to a first speed V B = 50km / h to second speed V B =Slow down to 30km / h.

[0164] The driver assistance system, which assists the driver of the vehicle A, analyzes the traffic scenario of Fig. 5 on the basis of information acquired by the sensors 2, 3 from the environment corresponding to the traffic scenario illustrated in Fig. 5 and predicts the future evolution of this traffic scenario. The central part of Fig. 5 shows the predicted speed V for a prediction period. A , V B and the curve characteristics of the distance d. The driving assistance system is particularly A and V B The predicted trajectory and velocity V A and V B Based on this, the driver assistance system predicts a linearly decreasing distance d for future times during the prediction period. In the illustrated road traffic scenario of FIG. 5, the driver assistance system concludes that during the prediction period, the predicted distance d reaches 0 m, resulting in a collision involving vehicle A hitting the rear end of vehicle B. As a result, the driver assistance system generates and outputs the information "crash in x seconds".

[0165] The driver assistance system may communicate the generated and output information to the driver of vehicle A via a human-machine interface. Alternatively or additionally, the driver assistance system may control an actuator, for example apply the brakes or adapt the accelerator setting of vehicle A to slow down vehicle A to avoid a predicted crash event.

[0166] In current driver assistance systems, the predicted evolution of a traffic scenario is calculated by ignoring the human factor and only by kinematic relations and measurable physical parameters, for example the speed V in the example of Figure 5. A , V B, and the distance,d,. In particular, in long-term forecasts (long-term forecasts), which, unlike short-term forecasts, span a period of,e.g., more than 2 seconds, the human factor can,significantly influence the future evolution of the current traffic,scenario.

[0167] FIG. 6 illustrates an application of a computer-implemented methodology that applies human factors prediction to support the behavior of an agent in a road traffic environment.

[0168] The driver assistance system of figure 6 may be a distance warning system or part of an adaptive cruise control system. Vehicle A is the ego vehicle, the driver of which is assisted by a driver assistance system according to the prior art.

[0169] The road traffic scenario shown in the upper part of Figure 6 is a vehicle moving at a constant speed V along the lanes of a road. A The vehicle A is moving at a speed of 0.01 m. The vehicle B is moving ahead of the vehicle A in the direction of travel of the vehicle A at a speed of 0.01 m. B At time t = 0, vehicle A and vehicle B are traveling in the same lane and in the same direction. The distance between vehicle A and vehicle B is d 0 The vehicle B decelerates at a constant rate to the first speed V B = 50km / h to second speed V B =Slow down to 30km / h.

[0170] A driver assistance system assisting the driver of vehicle A analyzes the traffic scenario of Fig. 6 based on information acquired by sensors 2, 3 from an environment corresponding to the traffic scenario depicted in Fig. 6 and predicts the future development of this traffic scenario. The prediction according to one embodiment of the computer-implemented method comprises a physical prediction over a first prediction period. The prediction according to this embodiment of the computer-implemented method comprises a physical prediction adapted based on human factors over a time range from the first prediction period to a second prediction period. The second prediction period is beyond the first prediction period.

[0171] 6 assumes that the human factor for adapting the physical prediction is the driver state of the driver of the ego-vehicle A. In particular, the driver of the ego-vehicle A has the driver state of an attentive driver.

[0172] The central part of Figure 6 shows the predicted velocity V A , V B and the curve characteristics of the distance d. A driving assistance system that uses physical predictions to make predictions is particularly concerned with the trajectories of vehicles A and B, their respective velocities V along the trajectories, A and V B As shown in the center of FIG. 6, the driver assistance system predicts a constant speed V A The prediction is based on the physical prediction of the distance d between the vehicle A and the other vehicle B, and the speed V of the other vehicle B is B However, the distance d over the first prediction period remains greater than 0 m, and therefore the driver assistance system concludes, based on the short-term prediction including the physical prediction, that there is no risk of a collision event occurring over the first prediction period.

[0173] The driver assistance system performs a long-term prediction for the time between the first prediction period and the second prediction period based on a driver state of the driver of the own vehicle A, which is an attentive driver. The attentive driver is considered to be able to react in time to avoid a collision event between the own vehicle A and the other vehicle B. In detail, the assistance system determines whether the driver should early stop the own vehicle A from reaching a constant speed V by, for example, braking the own vehicle A. A Let's consider changing from 0 to 100%. The predicted trajectory and speed V A and V B Based on this, the driving assistance system calculates a speed V of the host vehicle A for a time between the first prediction period and the second prediction period, and a future time within the second prediction period. A is the speed of vehicle B, V BFor the remainder of the future times during the second prediction period, the long-term prediction is that the distance d will remain constant, but the velocity V A and V B 6 is equal to the other vehicle B, the driver assistance system concludes that the predicted distance d will not reach 0 m during the second prediction period corresponding to the long-term prediction in the illustrated road traffic scenario of FIG. 6. As a result, considering the human factor of an attentive driver of the own vehicle A, the driver assistance system does not generate and output the information of "crash in x seconds" because the driver of the own vehicle A can react in time and avoid a collision event involving the own vehicle A and the other vehicle B. Therefore, unnecessary warnings to the driver of the own vehicle A are avoided by the computer-implemented method according to this embodiment.

[0174] In contrast, current driver assistance systems predict the evolution of a traffic scenario by ignoring the human element and only predicting the motion relations and measurable physical parameters, such as the speed V in the example of Figure 6. A , V B , and the distance,d,. In particular, in long-term forecasts (long-term forecasts), which, as opposed to short-term forecasts, span a period of,e.g., more than 2 seconds, the human factor has a large influence on the future evolution of the current traffic scenario.

[0175] FIG. 7 shows an alternative scenario of application of the computer-implemented method, applying human factor prediction to support the behavior of an agent in a road traffic environment.

[0176] The driver assistance system of figure 7 is a distance warning system or part of an adaptive cruise control system, as described with reference to figure 6. Vehicle A is the ego vehicle, the driver of which is assisted by a driver assistance system according to the prior art.

[0177] The illustrated road traffic scenario in the upper part of Fig. 7 corresponds to the traffic scenario in Fig. 5 and Fig. 6, in which the vehicle moves at a constant speed V along the lanes of the road. AThe vehicle A is moving at a speed of 0.01 m. The vehicle B is moving ahead of the vehicle A in the direction of travel of the vehicle A at a speed of 0.01 m. B At time t = 0, vehicle A and vehicle B are traveling in the same lane and in the same direction. The distance between vehicle A and vehicle B is d 0 The vehicle B decelerates at a constant rate to the first speed V B = 50km / h to second speed V B =Slow down to 30km / h.

[0178] The traffic scenario in Fig. 7 includes a human factor for adapting the physical prediction as a driver state of the driver of the host vehicle A, similar to Fig. 6. In contrast to the scenario in Fig. 6, the driver of the host vehicle A has a driver state of a drowsy driver.

[0179] A driver assistance system assisting the driver of vehicle A analyzes the traffic scenario of Fig. 7 based on information acquired by sensors 2, 3 from an environment corresponding to the traffic scenario depicted in Fig. 7 and predicts the future development of this traffic scenario. The prediction according to one embodiment of the computer-implemented method comprises a physical prediction over a first prediction period. The prediction according to this embodiment of the computer-implemented method comprises a physical prediction adapted based on human factors over a time range from the first prediction period to a second prediction period. The second prediction period is beyond the first prediction period.

[0180] 7 assumes that the human factor for adapting the physical prediction is the driver state of the driver of the ego-vehicle A. In particular, the driver of the ego-vehicle A has the driver state of an attentive driver.

[0181] The center part of Figure 7 shows the predicted velocity V A , V B and the curve characteristics of the distance d. A driving assistance system that uses physical predictions to make predictions is particularly concerned with the trajectories of vehicles A and B, their respective velocities V along the trajectories, A and V B In the center of FIG. 7, the driver assistance system predicts the constant speed V ABased on the physical prediction, the prediction is made until the first prediction period has elapsed, and the distance d between the host vehicle A and the other vehicle B is calculated based on the speed V of the other vehicle B. B However, the distance d over the first prediction period remains greater than 0 m, and therefore the driver assistance system concludes, based on the short-term prediction including the physical prediction, that there is no risk of a collision event occurring over the first prediction period.

[0182] The driver assistance system performs a long-term prediction for a time period between a first prediction period and a second prediction period based on the driver state of the driver of the host vehicle A being a drowsy driver. It is considered that the drowsy driver cannot react in time to avoid a collision event between the host vehicle A and the other vehicle B. In detail, the assistance system predicts that the driver will delay the acceleration of the host vehicle A to a constant speed V by, for example, applying the brakes of the host vehicle A. A Let's consider changing from 0 to 100%. The predicted trajectory and speed V A and V B Based on this, the driving assistance system calculates a speed V of the host vehicle A for a time between the first prediction period and the second prediction period, and a future time within the second prediction period. A is the speed of vehicle B, V B The driver assistance system predicts the distance d further decreasing for future time until it is equal to . The driver assistance system concludes that in the illustrated road traffic scenario of FIG. 7, the predicted distance d will reach 0 m during the second prediction period corresponding to the long-term prediction. As a result, considering the human factor of the drowsy driver of the own vehicle A, the driver of the own vehicle A reacts too late and cannot prevent the distance d from reaching 0, so the driver assistance system generates and outputs the information of "crash in x seconds". Thus, the assistance system predicts that in the long-term evolution of the traffic scenario of FIG. 7, a collision event involving the own vehicle A and the other vehicle B is likely to occur. Therefore, the computer-implemented method according to this embodiment generates and outputs a respective warning to the driver of the own vehicle A to give the driver enough time to prevent the predicted collision event despite the driver's drowsy state.

[0183] In contrast, current driver assistance systems predict the progression of a traffic scenario by ignoring the human element and only predicting the motion relations and measurable physical parameters, such as the speed V in the example of Figure 7. A , V B , and the distance,d,.

[0184] FIG. 8 illustrates an embodiment of a computer-implemented method for assessing the impact of using human factors in a forecasting method that uses human factors to support the operation of an agent in a road traffic environment.

[0185] The driver assistance system of Figure 8 is a distance warning system or part of an adaptive cruise control system as described with reference to Figures 6 and 7. Vehicle A is an ego vehicle, the driver of which is assisted by a driver assistance system implementing a computer-implemented method according to an embodiment.

[0186] The road traffic scenario shown in the upper part of Fig. 8 corresponds to the traffic scenarios in Figs. 5, 6 and 7, in which the vehicle moves at a constant speed V along the lanes of the road. A The vehicle A is moving at a speed of 0.01 m. The vehicle B is moving ahead of the vehicle A in the direction of travel of the vehicle A at a speed of 0.01 m. B At time t = 0, vehicle A and vehicle B are traveling in the same lane and in the same direction. The distance between vehicle A and vehicle B is d 0 The vehicle B decelerates at a constant rate to the first speed V B = 50km / h to second speed V B =Slow down to 30km / h.

[0187] The traffic scenario in Fig. 8 includes a human factor for adapting the physical prediction as the driver state of the driver of the host vehicle A, similar to Fig. 7. In contrast to the scenario in Fig. 6, the driver of the host vehicle A has a driver state of a drowsy driver.

[0188] The driver of the own vehicle A is classified as being a drowsy driver or in a drowsy driver state. Both the own vehicle and the other vehicle B of FIG. 8 are equipped with the communication module 26 described with reference to FIG. 3. The driver assistance system of the own vehicle A may transmit a communication signal 36 to the other vehicle B. The communication signal may include a warning of a predicted collision event involving the own vehicle A and the other vehicle B. The driver assistance system of the vehicle A may output a warning to the driver of the own vehicle A via a human-machine interface, for example, the output module 28. The other vehicle B may also include a driver assistance system, which is configured to generate and output a corresponding warning to the driver of the other vehicle B based on the received communication signal 36 communicated by the assistance system of the own vehicle A.

[0189] As explained with reference to Fig. 7, the assistance system predicts that in the long-term evolution of the traffic scenario of Fig. 9, a collision event involving the own vehicle A and the other vehicle B is likely to occur. Therefore, the computer-implemented method according to this embodiment generates and outputs a respective warning to the driver of the own vehicle A in order to give the driver sufficient time to prevent the predicted collision event despite the driver being in a drowsy state.

[0190] Furthermore, the driver assistance system of FIG. 9 estimates the influence of the warning on the driver of the own vehicle A and the driver of the other vehicle B.

[0191] In particular, the driver assistance system estimates the positive impact of outputting a warning signal to the driver of the host vehicle A because it is assumed that the driver of the host vehicle A will brake after noticing the warning. The host vehicle A is expected to slow down in response to outputting a warning to the drowsy driver, although it will be slow to brake.

[0192] The driver assistance system estimates the negative impact of outputting a warning signal to the driver of the other vehicle B, since it is assumed that the driver of the other vehicle B will be surprised by a sudden warning resulting from communicating a predicted collision event from behind that is not predicted by its own driver assistance system. The other vehicle B is expected to decelerate more strongly in response to the output of the warning to the driver. The driver of the other vehicle B may brake longer due to the warning, or may change from braking to braking with an increased deceleration value, since the output warning surprises the driver of the other vehicle B.

[0193] Therefore, the driver assistance system decides to generate and output a warning to the driver of the own vehicle A to mitigate the risk of a predicted collision event involving the own vehicle A and the other vehicle B from a long-term prediction using physical predictions together with predicted behavior after human factors based adaptation. The assistance system of FIG. 8 applies a computer-implemented method to estimate the impact of the warning on both drivers including the own vehicle A and the other vehicle B, both of which are involved in a predicted collision event in a shared road traffic environment.

[0194] Generally, it is beneficial to communicate the generated signal or event to another agent. In one embodiment, the human model 8 is configured to determine the human's current mental state as well as to create and update an individual driver profile of the human driver of the vehicle. The assistance system may share and update the generated driver profile with other participants of the V2X network. In a particular scenario, the assistance system identifies that the first vehicle is driving closely behind another (second) vehicle driving in the same lane in the same direction ahead of the first vehicle (close following). The assistance system may alert the driver of the first vehicle that the distance to the second vehicle is close. However, due to manual driving, the assistance system cannot guarantee that the driver of the first vehicle will accordingly improve his driving behavior and increase the distance to the second vehicle to a level generally accepted as a safe distance. In this particular scenario, the assistance system may further improve traffic safety by having the other vehicles partially compensate for the decrease in safety of the overall traffic scenario due to the small distance between the first and second vehicles by increasing the safety margin in their respective predicted behavior. However, the reaction of other human drivers in such a driving scenario involving vehicles that do not maintain a sufficient safety distance may vary and is not uniform. Some human drivers react to the information about a vehicle that is following the own vehicle at a sufficient distance from a safety point of view (close-following vehicle) by accelerating and possibly shortening the distance to the other vehicle traveling in front of the own vehicle, bringing themselves below the safety distance and thus also becoming a close-following vehicle. Therefore, outputting information about a close-following vehicle to such a driver is likely to decrease the overall traffic safety. Still other drivers may react to receiving information about the close-following other vehicle by increasing the distance to the vehicle traveling in front of their own vehicle. Behaving in this way allows the driver to brake more slowly if he detects an event ahead. As a result, the close-following vehicle behind the host vehicle has more time to react if the host vehicle suddenly brakes.As a result, communicating information about specific human factors, mental states, and individual driver profiles in certain embodiments of the communication signal 36 improves road safety, especially in combination with adapting the predicted behavior of the assistance system in accordance with embodiments of the computer-implemented method. The decision to deliver information, or alternatively not deliver information, therefore becomes dependent on knowledge of the typical reactions of individual drivers, which may initially be based on a predefined average behavior of human drivers and may later be updated and refined as more data is acquired about the individual human factors of the driver. In addition to this human driver model derived from past driving behavior, the current mental state may also be considered in the human factors information.

[0195] In contrast to the application scenario shown in FIG. 8, FIG. 9 shows an application scenario in which only the impact of the warning output on the driver of the host vehicle A is estimated and taken into account.

[0196] Fig. 9 shows an embodiment of a computer-implemented method for using human factors in predictions to assist the behavior of an agent in a road traffic environment for impact assessment. The assistance system in Fig. 9 is a distance warning system or part of an adaptive cruise control system, as described with reference to Figs. 6, 7 and 8. Vehicle A is an ego vehicle, whose driver is assisted by a driver assistance system implementing a computer-implemented method according to an embodiment.

[0197] The road traffic scenario shown in the upper part of Fig. 9 corresponds to the traffic scenarios in Figs. 5, 6, 7, and 8, in which the vehicle moves at a constant speed V along the lanes of the road. A The vehicle A is moving at a speed of 0.01 m. The vehicle B is moving ahead of the vehicle A in the direction of travel of the vehicle A at a speed of 0.01 m. B At time t = 0, vehicle A and vehicle B are traveling in the same lane and in the same direction. The distance between vehicle A and vehicle B is d 0 The assistance system detects that the other vehicle B is decelerating at a constant rate to the first speed V B = 50km / h to second speed VB =Predicted to slow down to 30km / h.

[0198] The traffic scenario of FIG. 9 includes a human factor for adapting the physical predictions as a driver state of the driver of the ego-vehicle A, similar to FIG. 8. Similar to the scenario of FIG. 8, the driver of the ego-vehicle A is determined to have a driver state of a drowsy driver. The driver of the ego-vehicle A is classified as being a drowsy driver or being in a drowsy driver state. In contrast to the embodiment of FIG. 8, the driver assistance system of the ego-vehicle A may transmit a communication signal 36 to the other vehicle B. The communication signal may include a warning of a predicted collision event involving the ego-vehicle A and the other vehicle B. The driver assistance system of the ego-vehicle A may output a warning to the driver of the ego-vehicle A via a human-machine interface, for example, the output module 28.

[0199] As explained with reference to Figures 7 and 8, the assistance system predicts that in the long-term evolution of the traffic scenario of Figure 9, a collision event involving the own vehicle A and the other vehicle B is likely to occur. Therefore, the computer-implemented method according to this embodiment generates and outputs a respective warning to the driver of the own vehicle A in order to give the driver sufficient time to prevent the predicted collision event despite the driver being in a drowsy state.

[0200] Furthermore, the assistance system estimates the effect of outputting a warning of a predicted crash event to the driver of host vehicle A.

[0201] In particular, the driver assistance system estimates the positive impact of outputting a warning signal to the driver of the own vehicle A because the driver of the own vehicle A is assumed to brake after noticing the warning of the predicted collision. The own vehicle A is expected to slow down in response to outputting the warning to the drowsy driver, although it is slow to brake. The prediction of the behavior of the own vehicle A and the other vehicle B in the center part of FIG. 9 reflects the impact of delayed deceleration of the own vehicle A in response to the output warning of a predicted collision involving the own vehicle A and the other vehicle B.

[0202] The driver assistance system estimates the negative impact of outputting a warning signal to the driver of the other vehicle B, since it is assumed that the driver of the other vehicle B will be surprised by a sudden warning resulting from communicating a predicted collision event from behind that is not predicted by its own driver assistance system. The other vehicle B is expected to decelerate more strongly in response to the output of the warning to the driver. The driver of the other vehicle B may brake longer due to the warning, or may change from braking to braking with an increased deceleration value, since the output warning surprises the driver of the other vehicle B.

[0203] Thus, the driver assistance system decides to generate and output a warning to the driver of the own vehicle A to mitigate the risk of a predicted collision event involving the own vehicle A and the other vehicle B from the long-term prediction using the physical prediction together with the predicted behavior after human factors based adaptation. The assistance system of FIG. 9 applies a computer-implemented method of estimating the impact of the warning of the driver of the own vehicle A, for example by outputting an acoustic warning via a loudspeaker to the driver of the own vehicle A involved in the predicted collision event in the shared road traffic environment. Despite the slow reaction time of the driver of the own vehicle A due to the driver's slow reaction to the output warning via the human-machine interface of the own vehicle A, the probability of the predicted collision event occurring is advantageously low due to the long-term prediction including the human factors based adaptation.

[0204] FIG. 10 shows a second embodiment of the impact assessment using the human factor of the prediction method using the human factor to support the operation of an agent in a road traffic environment in an alternative scenario to that of FIGS. 8 and 9.

[0205] Figure 10 shows an embodiment of a computer-implemented method for using human factors in predictions to assist the behavior of an agent in a road traffic environment for impact assessment. The driver assistance system of Figure 10 is a distance warning system or part of an adaptive cruise control system as described with reference to Figures 6, 7, 8 and 9. Vehicle A is an ego vehicle, whose driver is assisted by a driver assistance system implementing a computer-implemented method according to an embodiment.

[0206] The road traffic scenario shown in the upper part of Fig. 10 corresponds to the traffic scenarios in Figs. 5, 6, 7, 8 and 9, in which the vehicle moves at a constant speed V along the lanes of the road. A The vehicle A is moving at a speed of 0.01 m. The vehicle B is moving ahead of the vehicle A in the direction of travel of the vehicle A at a speed of 0.01 m. B Vehicle A is traveling in the same lane and in the same direction as vehicle B. The distance between vehicle A and vehicle B at time t = 0 is d 0 The assistance system detects that the other vehicle B is decelerating at a constant rate to the first speed V B = 50km / h to second speed V B =Predicted to slow down to 30km / h.

[0207] The traffic scenario of FIG. 10 includes a human factor for adapting the physical predictions as a driver state of the driver of the ego-vehicle A, similar to FIG. 8 and FIG. 9. Similar to the scenarios of FIG. 8 and FIG. 9, the driver of the ego-vehicle A is determined to have a driver state of a drowsy driver. The driver of the ego-vehicle A is classified as being a drowsy driver or being in a drowsy driver state. Similar to the scenario of FIG. 9, the driver assistance system of the ego-vehicle A has the capability to transmit a communication signal 36 to the other vehicle B. The communication signal may include a warning of a predicted collision event involving the ego-vehicle A and the other vehicle B. The driver assistance system of the ego-vehicle A may output the warning to the driver of the ego-vehicle A via a human-machine interface, for example, the output module 28.

[0208] As explained with reference to Figures 7, 8 and 9, the assistance system predicts that in the long-term evolution of the traffic scenario of Figure 10, a collision event involving the own vehicle A and the other vehicle B is likely to occur. Therefore, the computer-implemented method according to this embodiment generates and outputs a respective warning to the driver of the own vehicle A in order to give the driver sufficient time to prevent the predicted collision event despite the driver being in a drowsy state.

[0209] Furthermore, the assistance system estimates the effect of outputting a warning of a predicted crash event to the driver of host vehicle A.

[0210] In particular, the driver assistance system estimates the positive impact of outputting a warning signal to the driver of the own vehicle A because the driver of the own vehicle A is assumed to brake after noticing the warning of the predicted collision. The own vehicle A is expected to slow down in response to outputting the warning to the drowsy driver, although it is slow to brake. The prediction of the behavior of the own vehicle A and the other vehicle B in the center part of FIG. 10 reflects the impact of delayed deceleration of the own vehicle A in response to the output warning of a predicted collision involving the own vehicle A and the other vehicle B.

[0211] The driver assistance system estimates the negative impact of outputting a warning signal to the driver of the other vehicle B, since it is assumed that the driver of the other vehicle B will be surprised by a sudden warning resulting from communicating a predicted collision event from behind that is not predicted by its own driver assistance system. The other vehicle B is expected to decelerate more strongly in response to the output of the warning to the driver. The driver of the other vehicle B may brake longer due to the warning, or may change from braking to braking with an increased deceleration value, since the output warning surprises the driver of the other vehicle B.

[0212] Therefore, the driver assistance system decides to generate and output a warning only to the driver of the own vehicle A in order to mitigate the risk of a predicted collision event involving the own vehicle A and the other vehicle B from a long-term prediction using physical predictions together with the predicted behavior after human factors based adaptation. In contrast to the embodiment of the computer-implemented method in the traffic scenario of Figure 8, the assistance system further decides to alert only the driver of the own vehicle A and not to communicate information to vehicle B due to the predicted negative impact of alerting the driver of the other vehicle B.

[0213] In detail, the driving assistance system evaluates the effect of outputting a warning to the driver of the host vehicle A and the effect of outputting a warning to the driver of the other vehicle B based on human factors.

[0214] Based on the assessed impact of outputting a warning to the driver of the own vehicle A and the assessed impact of outputting a warning to the driver of the other vehicle B, which are based on human factors, the driving assistance system decides to alert an agent whose reaction is likely to have a positive impact on the predicted development of the traffic scenario.

[0215] Based on the assessed impact of outputting a warning to the driver of the own vehicle A and the assessed impact of outputting a warning to the driver of the other vehicle B, which are based on human factors, the driving assistance system may decide to adapt the intensity and modality of the output warning via the output module 28 depending on the estimated probability of a predicted collision event (likelihood of collision).

[0216] The improved prediction process according to one embodiment of a computer-implemented method using adaptation of predictions based on human factors may be performed for both the own agent (own vehicle A) and the other agent (other vehicle B).

[0217] 10 applies a computer-implemented method for estimating the impact of a warning of a driver of an ego vehicle A, for example by outputting an acoustic warning via a loudspeaker to a driver of the ego vehicle A involved in a predicted crash event in a shared road traffic environment. Despite a slow reaction time of the driver of the ego vehicle A due to the driver's slow reaction to the output warning via the human-machine interface of the ego vehicle A, the probability of the predicted crash event occurring is advantageously low due to the long-term prediction including human factors based adaptation.

[0218] All the features described above or shown in the figures can be combined with each other in any advantageous manner within the scope of the present disclosure. In the detailed description of the embodiments, numerous specific details have been set forth in order to provide a thorough understanding of the invention as defined in the claims. It is clear that it is possible to practice the invention as claimed without including all the specific details.

[0219] In the specification and claims, the phrase "at least one of A and B" can be substituted for the phrase "A and / or B" and vice versa, since they are used interchangeably. The phrase "A and / or B" means "A, or B, or A and B."

Claims

1. 1. A computer-implemented method for assisting an agent in operating within a dynamic environment, wherein at least one other agent is present in the environment, the method comprising: acquiring sensor information about the environment of the agent; predicting at least one behavior of at least one of the agent and the at least one other agent based on the acquired sensor information; determining at least one human factor associated with the at least one predicted behavior; adapting the at least one predicted behavior based on the determined human factors; and determining whether communication of at least one of the predicted behavior consequences and one of the predicted behaviors based on the adapted at least one behavior is beneficial to operation of the agent; if determining that the communication would be beneficial, generating a signal based on the adapted at least one behavior and communicating at least one of the consequences of the predicted behavior or the predicted behavior to the agent based on the generated signal; A method comprising:

2. The method further comprising: determining at least one event involving the agent based on the adapted at least one behavior; and estimating a risk associated with signaling information regarding the determined at least one event to the agent; and adapting the adapted behavior further based on the estimated risk associated with signaling the information regarding the determined at least one event to the agent; determining whether the estimated risk associated with signaling the information regarding the determined at least one event to the agent is reduced based on the further adapted behavior; generating and outputting the signal based on the at least one further adapted behavior if the estimated risk is determined to have decreased; The computer-implemented method of claim 1 , comprising:

3. The method further comprising: estimating the risk associated with signaling the information regarding the determined at least one event, including evaluating the impact of signaling the information regarding the at least one trajectory. The computer-implemented method of claim 2 , comprising:

4. The method further comprising: estimating the risk associated with signaling the information regarding the determined at least one event includes predicting a human response to the information output in the signal; enabling generation and output of said signal containing said information when determining said estimated risk reduction based on said predicted human response; disabling the generation and output of the signal containing the information when determining an increase in the estimated risk based on the predicted human response; The computer-implemented method of claim 2 , comprising:

5. 2. The computer-implemented method of claim 1 , wherein the determined at least one human factor includes information about at least one of the agent, the at least one other agent, a subset or all of other agents in the agent's environment.

6. 2. The computer-implemented method of claim 1 , wherein the method includes adapting at least one predicted behavior of at least one of the agent, the at least one other agent, or all other agents in the agent's environment based on the determined human factors.

7. 2. The computer-implemented method of claim 1, wherein the method includes adapting the adapted behavior further based on the estimated risk associated with signaling the information regarding the determined at least one event to at least one of the agent, the at least one other agent, or all other agents in the agent's environment.

8. The method further comprising: updating a human model based on a determined response of at least one of the agent and the at least one other agent based on the predicted behavior in a previous processing cycle; The computer-implemented method of claim 1 , comprising:

9. The method further comprising: determining a weight associated with the human factor based on the certainty of the human factor, and determining an impact of the human factor on the predicted behavior based on the determined weight; The computer-implemented method of claim 1 , comprising:

10. The method further comprising: determining at least one human factor associated with the at least one predicted behavior includes determining a combination of human factor information associated with the agent and human factor information associated with the at least one other agent to determine an overall uncertainty associated with the at least one human factor. The computer-implemented method of claim 1 , comprising:

11. The method further comprising: determining whether the communication based on the adapted at least one behavior is beneficial to the agent and the at least one other agent includes determining whether it is beneficial to output the communication to the agent and the at least one other agent at different times; generating and outputting the generated signal to the agent and the at least one other agent at different times if the communication at the different times is beneficial; The computer-implemented method of claim 1 , comprising:

12. The method further comprising: determining whether the communication based on the adapted at least one behavior is beneficial to the agent and the at least one other agent includes determining whether output of the communication to the agent will have a negative impact on the at least one other agent. The computer-implemented method of claim 1 , comprising:

13. The method further comprising: determining whether the communication based on the adapted at least one behavior is beneficial for the agent and the at least one other agent includes predicting and simulating behavior variations of the agent and the at least one other agent in response to a plurality of communication candidates; determining, for each communication candidate, an effect of outputting the communication to the agent or the at least one other agent; and selecting the communication candidate or a combination of communication candidates based on the determined effect of the communication candidates. The computer-implemented method of claim 1 , comprising:

14. 2. The computer-implemented method of claim 1, wherein the method includes communicating the generated signal to the at least one other agent for output via a human-machine interface of the at least one other agent.

15. the method comprising predicting, for a first time into the future, the at least one behavior of at least one of the agent or the at least one other agent based on the acquired sensor information using physical predictions, predicting the at least one behavior for a second time into the future, and adapting the predicted at least one behavior based on the determined human factors; The computer-implemented method of claim 1 , wherein the first time period is less than the second time period.

16. The method further comprising: the physical predictions include longitudinal behavior including at least one of a constant velocity, a constant deceleration or a constant acceleration, and a delayed constant acceleration; the physical predictions include lateral behavior including at least one of: moving through a constant turn angle, moving along a map path, changing lanes, and taking a left path, a straight path, or a right path at an intersection; The physical prediction includes environmental parameters including road slope and road curvature radius; predicting the at least one behavior and adapting the predicted at least one behavior includes a longitudinal behavior including at least one of a behavior change based on a driver model and a behavior change based on a driver state; the behavior change includes at least one of a change in deceleration or acceleration value, a change from a constant velocity to a constant acceleration at a future time, and a change from a constant acceleration to a constant velocity at the future time; the behavior change includes a delay based on a determined driver state, the determined driver state including one of an attentive state, a drowsy state, a startled state, a distracted state, an experienced driver state, an unskilled state, and a disoriented state; The predicting at least one behavior and adapting the predicted at least one behavior includes lateral behaviors including changing lanes, taking a left path, a straight path, or a right path at an intersection, and swerving around a center line.

20. The computer-implemented method of claim 15, comprising:

17. The method further comprising: estimating the risk includes estimating the risk with signaling information about the determined at least one event to the agent, and further with signaling the information about the determined at least one event to the at least one other agent; generating and outputting said signal to said agent and generating and outputting said further signal to said at least one other agent if it is determined that outputting said signal and a further signal reduces said estimated risk. The computer-implemented method of claim 1 , comprising:

18. The method further comprising: executing said computer-implemented method by at least one processor co-located with said agent; The computer-implemented method of claim 1 , comprising:

19. The method further comprising:

2. The computer-implemented method of claim 1, comprising performing the computer-implemented method in a distributed manner by at least two processors of the agent, the at least one other agent, and at least one server remote from the agent and the at least one other agent.

20. The method further comprising: transmitting information regarding the human factors and associated effects of the human factors on the predicted behavior to at least one of the at least one other agent and at least one remote server. The computer-implemented method of claim 1 , comprising:

21. The method further comprising: Executing the computer-implemented method within an advanced driver assistance system of the host vehicle. The computer-implemented method of claim 1 , comprising:

22. The method further comprising: The computer-implemented method of claim 1 , further comprising performing at least some steps of the computer-implemented method by a remote server remote to the agent.

23. The computer-implemented method of claim 1 , wherein the determined event is a predicted collision involving the agent or a near collision involving the agent.

24. The computer-implemented method of claim 1 , wherein the at least one behavior comprises at least one trajectory of at least one of the agent or the at least one other agent.

25. The computer-implemented method of claim 1 , wherein the agent and the at least one other agent include at least one of a pedestrian, a bicyclist, a motorcyclist, and a driver of a road vehicle in a road traffic environment.

Images