System and method for predicting driver behavior

Abstract

The present invention relates, in general, to the technical fields of autonomous driving and advanced driver assistance. More specifically, the present invention relates to preventing collisions and dangerous driving situations. The present invention may relate to a method for predicting driver behavior using at least one computing device. The method may include the steps of: receiving a plurality of motion data elements characterizing the motion of at least one vehicle in at least one specific driving situation; constructing a behavior model representing expected driver behavior in the at least one specific driving situation based on the plurality of motion data elements; and inferring the behavior model for at least one incoming motion data element to predict expected driver behavior in the specific driving situation.

Description

Technical Field

[0001] (Cross - reference to related applications) This application claims the priority of U.S. Provisional Patent Application No. 63 / 454,685, filed on March 26, 2023, the content of which is hereby incorporated by reference in its entirety into this specification.

[0002] The present invention generally relates to the technical field of autonomous driving and advanced driver assistance. More specifically, the present invention relates to preventing the occurrence of collisions and dangerous driving situations.

Background Art

[0003] As is known in the art, an Advanced Driver Assistance System (ADAS) represents a group of electronic and computer - implemented technologies that assist a driver in various driving modes. ADAS uses multiple input modules such as sensors and cameras to detect nearby obstacles or driver errors and respond accordingly. Most of the technologies used for driver assistance purposes are often applied to autonomous driving systems and vice versa.

[0004] The main purpose of using ADAS is to enhance driving safety by, for example, warning the driver about errors and malfunctions of various vehicle components via a user interface, or by providing respective control signals (steering, acceleration, braking, etc.) for controlling the driving, thereby automating, adapting, and improving various aspects of vehicle technology. The safety functions of such systems may also assist in performing safety protection functions, automating lighting control, providing adaptive cruise control, incorporating satellite navigation and traffic warnings, warning the driver about the possibility of obstacles, lane departure and lane centering assistance, etc. Thereby, ADAS helps to avoid crashes and collisions.

[0005] It is known that most road collisions and crashes are caused by human error, which is often triggered by inappropriate driving behavior (speeding, violation of traffic rules (e.g., driving the wrong way), aggressive driving, reckless driving, drunk driving, etc.). Although many technologies are known today to eliminate human error in driving, this aspect remains an active research topic. [Overview of the project] [Problems that the invention aims to solve]

[0006] Therefore, there is a need for systems and methods that predict driver behavior, which will lead to improvements in advanced driver assistance and autonomous driving technologies by reducing the risk of collisions caused by human error. [Means for solving the problem]

[0007] In general embodiments, the present invention may relate to a method for predicting driver behavior using at least one computing device. The method may include the steps of: receiving a plurality of motion data elements characterizing the motion of at least one vehicle in at least one specific driving situation; constructing a behavior model representing expected driver behavior in the at least one specific driving situation based on the plurality of motion data elements; and inferring the behavior model for at least one incoming motion data element to predict expected driver behavior in the specific driving situation.

[0008] In another general embodiment, the present invention may relate to a method for predicting the motion of a vehicle using at least one computing device, the method comprising: receiving a plurality of geolocation data elements representing the geolocation of at least one vehicle, each geolocation data element being assigned a global timestamp corresponding to the time of determination of the respective geolocation, the time of reception of the respective geolocation data element, and a reception timestamp; calculating a plurality of extrapolated geolocation data elements based on (i) the respective geolocation of the plurality of geolocation data elements, (ii) the respective global timestamp, and (iii) the respective reception timestamp of the respective geolocation data element; calculating at least one incoming motion data element representing the velocity and direction of motion between the plurality of extrapolated geolocation locations; and inferring a pre-trained machine learning (ML)-based model on the at least one incoming motion data element to predict an outcome motion data element representing the expected motion of the at least one vehicle.

[0009] In yet another general embodiment, the present invention may relate to a system for predicting driver behavior, the system comprising a non-temporary memory device storing a module of instruction code, and at least one processor associated with the memory device and configured to execute the module of instruction code, wherein, when the module of instruction code is executed, the at least one processor is configured to: receive a plurality of motion data elements characterizing the motion of at least one vehicle in at least one specific driving situation; construct a behavior model representing expected driver behavior in the at least one specific driving situation based on the plurality of motion data elements; and infer the behavior model for at least one incoming motion data element to predict expected driver behavior in the specific driving situation.

[0010] In some embodiments, the at least one specific driving condition may be predefined by a plurality of motion scenarios. The expected driver behavior may be predefined by a plurality of expected driver decisions, each corresponding to following a specific motion scenario among the plurality of motion scenarios. The step of inferring the behavior model may include inferring the behavior model for the at least one incoming motion data element to predict the occurrence of a specific driver decision among the plurality of expected driver decisions.

[0011] In some embodiments, each of the plurality of motion scenarios may be represented as a sequence of each of the plurality of motion data elements.

[0012] In some embodiments, the behavior model may be a machine learning (ML) based model, and the steps of constructing the behavior model are machine learning (ML) based, and the steps of constructing the behavior model may include: analyzing the plurality of motion data elements to determine a sequence of motion data elements of the plurality of motion data elements representing the plurality of motion scenarios; forming a plurality of decision data elements each representing a plurality of expected driver decisions corresponding to following a particular motion scenario among the plurality of motion scenarios; and training the behavior model based on the plurality of decision data elements to (a) receive the incoming motion data element; (b) calculate the probability that a particular driver decision among the plurality of expected driver decisions will occur based on the incoming motion data element; and (c) predict the occurrence of the particular driver decision based on the probability.

[0013] In some embodiments, the step of receiving the plurality of motion data elements may include the step of receiving a plurality of motion data elements characterizing the motion of a plurality of vehicles in the at least one specific driving condition, the method further including the step of analyzing the plurality of decision data elements to obtain a baseline profile data element representing a baseline distribution of the plurality of expected driver decisions for the at least one specific motion scenario among the plurality of motion scenarios, and the step of analyzing at least one incoming motion data element of the at least one vehicle in relation to the baseline distribution to obtain a vehicle-specific profile data element representing a deviation of one or more driver decisions of each vehicle from following the at least one specific motion scenario.

[0014] In some embodiments, the method may further include the step of receiving the vehicle-specific profile data elements of the at least one vehicle, and the step of inferring the behavior model may further include the step of inferring the behavior model with respect to (a) the at least one incoming motion data element and (b) the vehicle-specific profile data elements to predict the occurrence of the particular driver decision among the plurality of predicted driver decisions.

[0015] In some embodiments, the predicted driver determination may be represented by at least one consequence motion data element that characterizes the predicted motion of at least one vehicle in at least one specific driving situation.

[0016] In some embodiments, the step of constructing the behavioral model may be performed by at least one server computing device, and the step of inferring the behavioral model may be performed by at least one client computing device communicatively connected to the at least one server computing device.

[0017] In some embodiments, the at least one client computing device is associated with a first vehicle, and the method may further include: determining the geographic location of the first vehicle by the at least one client computing device; obtaining a segment of the behavior model representing a geographic area surrounding the geographic location of the first vehicle from the at least one server computing device by the at least one client computing device; obtaining at least one second motion data element corresponding to the geographic location of a second vehicle within the geographic area from the at least one server computing device by the at least one client computing device; and inferring the segment of the behavior model with respect to the at least one second motion data element by the at least one client computing device to predict the occurrence of the specific driver determination for the second vehicle.

[0018] In some embodiments, the specific driver determination of the second vehicle may be represented by at least one second consequence motion data element that characterizes the expected motion of the second vehicle in at least one specific driving situation within the geographical area, and the method may further include the step of calculating the expected motion trajectory of the second vehicle based on the at least one second consequence motion data element of the second vehicle.

[0019] In some embodiments, the predicted trajectory may be calculated as a Bézier curve.

[0020] In some embodiments, the method may further include: the steps of: having the at least one client computing device infer the segment of the behavior model for the at least one first incoming motion data element to predict the occurrence of the particular driver determination of the first vehicle, represented by at least one first resulting motion data element that characterizes the expected motion of the first vehicle in at least one particular driving situation within the geographical area; the steps of: having the at least one client computing device calculate the expected motion trajectory of the first vehicle based on the at least one first resulting motion data element; having the at least one client computing device calculate the risk of collision between the first vehicle and the second vehicle based on the expected motion trajectories of the first vehicle and the second vehicle; and, if the calculated risk of collision exceeds a predetermined threshold, providing a collision warning via the user interface of the client computing device.

[0021] In some embodiments, the step of calculating the predicted motion trajectory includes the step of iteratively inferring the segment of the behavior model for at least one resulting motion data element calculated in a preceding iteration by the at least one client computing device to predict a sequence of driver decisions for each of the respective vehicles, represented as a sequence of resulting motion data elements, where the resulting motion data element of each iteration represents the motion of each of the vehicles at a future point in time preceding the motion data element of a subsequent iteration.

[0022] In some embodiments, each of the sequences of driver decisions is associated with the probability that each of the driver decisions occurs, and the step of calculating the expected motion trajectory further includes the step of calculating a decreasing probability path data element that represents the probability of following the expected motion trajectory, based on (i) the sequence of resulting motion data elements and (ii) the probability that each of the driver decisions occurs.

[0023] In some embodiments, each of the plurality of motion data elements may represent at least one of the following: (a) the geographical location of the at least one vehicle, (b) the velocity of the at least one vehicle, (c) the acceleration of the at least one vehicle, and (d) the direction of motion of the at least one vehicle.

[0024] In some embodiments, the method may further include the steps of receiving a plurality of geographic location data elements representing a plurality of geographic locations of the at least one vehicle, and calculating each of the plurality of motion data elements as a motion vector characterizing the motion of the at least one vehicle between the plurality of geographic locations, based on the plurality of geographic location data elements.

[0025] In some embodiments, the method may further include: receiving a plurality of geolocation data elements representing each of a plurality of geolocations of the at least one vehicle, wherein each of the plurality of geolocation data elements is assigned a global timestamp corresponding to the time of determination of each geolocation and the time of reception of each of the geolocation data elements, and a reception timestamp; calculating an extrapolated geolocation of the at least one vehicle based on (i) each of the plurality of geolocations, (ii) each global timestamp and (iii) each reception timestamp of the plurality of geolocation data elements; and calculating the at least one incoming motion data element as a motion vector based further on the extrapolated geolocation.

[0026] In some embodiments, the method further comprises receiving a plurality of geographical location data elements representing the geographical locations of a plurality of vehicles, calculating, based on the plurality of geographical location data elements, a plurality of motion data elements representing the speed and direction of the movement of each vehicle among the plurality of vehicles between respective geographical locations, analyzing the plurality of motion data elements to determine a sequence of motion data elements of the plurality of motion data elements representing a plurality of motion scenarios in at least one specific driving situation, forming a plurality of decision data elements respectively representing a plurality of predicted driver decisions each corresponding to following a specific motion scenario among the plurality of motion scenarios, training an ML-based model based on the plurality of decision data elements to (a) receive the incoming motion data elements, (b) calculate the probability of occurrence of a specific driver decision among the plurality of predicted driver decisions, (c) predict the occurrence of the specific driver decision among the plurality of predicted driver decisions based on the calculated probability, and (d) calculate the resultant motion data elements characterizing the predicted motion of the at least one vehicle in the at least one specific driving situation based on the predicted occurrence of the specific driver decision.

[0027] In some embodiments, the at least one specific driving situation may be predefined by a plurality of motion scenarios, the predicted driver behavior may be predefined by a plurality of predicted driver decisions each corresponding to following a specific motion scenario among the plurality of motion scenarios, and the at least one processor may be configured to further infer the behavior model by inferring the behavior model with respect to the at least one incoming motion data element to predict the occurrence of a specific driver decision among the plurality of predicted driver decisions.

[0028] In some embodiments, the behavior model may be a machine learning (ML)-based model, and the at least one processor analyzes the plurality of motion data elements to determine a sequence of motion data elements of the plurality of motion data elements representing the plurality of motion scenarios, forms a plurality of decision data elements respectively representing a plurality of predicted driver decisions corresponding to following a specific motion scenario among the plurality of motion scenarios, and trains the behavior model based on the plurality of decision data elements to (a) receive the incoming motion data element, (b) calculate a probability that a specific driver decision among the plurality of predicted driver decisions occurs based on the incoming motion data element, and (c) predict the occurrence of the specific driver decision based on the probability, and may be configured to construct the behavior model by performing the steps.

[0029] In some embodiments, the plurality of motion data elements may characterize the motion of a plurality of vehicles in at least one specific driving situation, and the at least one processor further analyzes the plurality of decision data elements to obtain a baseline profile data element representing a baseline distribution of the plurality of predicted driver decisions regarding the at least one specific motion scenario among the plurality of motion scenarios, and analyzes at least one incoming motion data element of the at least one vehicle in relation to the baseline distribution to obtain a vehicle-specific profile data element representing a deviation of one or more driver decisions of the respective vehicle from following the at least one specific motion scenario, and may be further configured to perform the steps.

[0030] In some embodiments, the at least one processor may be further configured to perform the steps of: receiving the vehicle-specific profile data elements of the at least one vehicle; and further inferring the behavior model by inferring the behavior model with respect to (a) the at least one incoming motion data element and (b) the vehicle-specific profile data elements, thereby predicting the occurrence of the specific driver decision among the plurality of predicted driver decisions.

[0031] In some embodiments, the at least one processor may comprise at least one first processor associated with at least one server computing device and at least one second processor associated with at least one client computing device communicably connected to the at least one server computing device, wherein the at least one processor configured to construct the behavioral model may be the at least one first processor, and the at least one processor configured to infer the behavioral model may be the at least one second processor.

[0032] In some embodiments, the at least one client computing device may be associated with a first vehicle, and the at least one second processor may be further configured to perform the steps of: determining the geographic location of the first vehicle; obtaining from the at least one server computing device a segment of the behavior model representing a geographic area surrounding the geographic location of the first vehicle; obtaining from the at least one server computing device at least one second motion data element corresponding to the geographic location of the second vehicle within the geographic area; and inferring the behavior model by inferring the segment of the behavior model against the at least one second motion data element to predict the occurrence of the specific driver determination for the second vehicle.

[0033] In some embodiments, the specific driver determination of the second vehicle may be represented by at least one second consequence motion data element that characterizes the expected motion of the second vehicle in at least one specific driving situation within the geographical area, and the at least one second processor may be further configured to perform the step of calculating the expected motion trajectory of the second vehicle based on the at least one second consequence motion data element of the second vehicle.

[0034] In some embodiments, the at least one second processor may be further configured to perform the steps of: acquiring the at least one first incoming motion data element characterizing the current motion of the first vehicle; inferring the segments of the behavior model with respect to the at least one first incoming motion data element to predict the occurrence of the particular driver decision of the first vehicle, which is represented by at least one first resulting motion data element characterizing the expected motion of the first vehicle in at least one particular driving situation within the geographical area; calculating the expected motion trajectory of the first vehicle based on the at least one first resulting motion data element; calculating the risk of collision between the first vehicle and the second vehicle based on the expected motion trajectories of the first vehicle and the second vehicle; and providing a collision warning via the user interface (UI) of the at least one client computing device if the calculated collision risk exceeds a predetermined threshold.

[0035] In some embodiments, the at least one second processor may be configured to calculate the predicted motion trajectory by performing the steps of iteratively inferring the segment of the behavior model for at least one resulting motion data element computed in a preceding iteration to predict a sequence of driver decisions for each of the respective vehicles, which are represented as a sequence of resulting motion data elements, wherein the resulting motion data element of each iteration represents the motion of each of the vehicles at a future point in time preceding the motion data element of a subsequent iteration.

[0036] In some embodiments, the sequence of each driver decision may be associated with the probability that each of the driver decisions occurs, and the at least one second processor may be configured to perform the step of further calculating the predicted trajectory by calculating a decreasing probability path data element representing the probability of following the predicted trajectory based on (i) the sequence of resulting motion data elements and (ii) the probability that each of the driver decisions occurs.

[0037] In some embodiments, the at least one second processor may be further configured to calculate the predicted trajectory as a Bézier curve.

[0038] In some embodiments, the at least one processor may be further configured to: receive a plurality of geographic location data elements representing a plurality of geographic locations of the at least one vehicle; and calculate each of the plurality of motion data elements as a motion vector characterizing the motion of the at least one vehicle between the plurality of geographic locations, based on the plurality of geographic location data elements.

[0039] In some embodiments, the at least one processor may be further configured to: receive a plurality of geolocation data elements representing each of a plurality of geolocations of the at least one vehicle, wherein each of the plurality of geolocation data elements is assigned a global timestamp corresponding to the time of determination of each geolocation and the time of reception of each geolocation data element, and a reception timestamp; calculate an extrapolated geolocation of the at least one vehicle based on (i) each of the plurality of geolocations, (ii) each global timestamp and (iii) each reception timestamp of the plurality of geolocation data elements; and further calculate the at least one incoming motion data element as a motion vector based on the extrapolated geolocation. [Brief explanation of the drawing]

[0040] The subject matter considered to be the present invention is specifically pointed out and explicitly claimed in the concluding portion of the specification. However, the present invention, along with its object, features, and advantages, as well as with respect to both its organization and method of operation, can be best understood by referring to the following detailed description when read together with the accompanying drawings.

[0041] [Figure 1] This block diagram shows a computing device that may be included in a system for predicting driver behavior, according to several embodiments.

[0042] [Figure 2] This is a schematic diagram illustrating the concept of the present invention relating to providing collision warnings via a UI, according to several embodiments.

[0043] [Figure 3A] This is a schematic diagram illustrating the concept of the present invention relating to predicting driving decisions that follow specific motion scenarios. [Figure 3B] This is a schematic diagram illustrating the concept of the present invention relating to predicting driving decisions that follow specific motion scenarios.

[0044] [Figure 4A] This block diagram shows a client computing device for a system for predicting driver behavior, according to several embodiments.

[0045] [Figure 4B] A block diagram shows a client computing device for a system for predicting driver behavior, according to several alternative embodiments.

[0046] [Figure 4C] This block diagram shows a server computing device for a system for predicting driver behavior, according to several embodiments.

[0047] [Figure 5A] This flowchart illustrates a method for predicting driver behavior according to several embodiments.

[0048] [Figure 5B] This flowchart illustrates a method for predicting vehicle motion according to several embodiments.

[0049] For the sake of brevity and clarity, please understand that the elements shown in the diagrams are not necessarily drawn to scale. For example, the dimensions of some elements may be exaggerated relative to others for clarity. Furthermore, where appropriate, reference numbers may be repeated between diagrams to indicate corresponding or similar elements. [Modes for carrying out the invention]

[0050] Those skilled in the art will understand that the present invention can be embodied in other specific forms without departing from its spirit or essential features. Therefore, the embodiments described herein should be considered illustrative and not limiting in any respect to the invention as described herein. Accordingly, the scope of the invention is indicated not by the foregoing description but by the appended claims, and therefore, all modifications that fall within the meaning and scope of the equivalents of the claims are intended to be included within the claims.

[0051] The following detailed description includes numerous specific details to provide a complete understanding of the invention. However, those skilled in the art will understand that the invention can be carried out without these specific details. In other examples, well-known methods, procedures, and components are not described in detail so as not to obscure the invention. Some features or elements described in relation to one embodiment may be combined with features or elements described in relation to another embodiment. For clarity, the discussion of the same or similar features or elements may not be repeated.

[0052] Embodiments of the present invention are not limited thereto, but discussions using terms such as “process,” “calculate,” “calculate,” “determine,” “establish,” “analyze,” “check,” “select,” “choose,” “omit,” and “train” may refer to the operation and / or process of a computer, computing platform, computing system, or other electronic computing device that operates and / or converts data represented as physical (e.g., electronic) quantities in the computer’s registers and / or memory to other data represented as similar physical quantities in the computer’s registers and / or memory, or in other non-temporary storage media of information capable of storing instructions for performing operations and / or processes.

[0053] Embodiments of the present invention are not limited in this respect, but the terms “plurality” and “a plurality” as used herein may include, for example, “multiple” or “two or more.” The terms “plurality” or “a plurality” may be used throughout this specification to describe two or more components, devices, elements, units, parameters, etc. The term “set,” as used herein, may include one or more items.

[0054] Unless expressly stated otherwise, embodiments of the methods described herein are not restricted to any particular order or sequence. Furthermore, some embodiments of the methods described or some of their elements may occur or be performed simultaneously, at the same time, together, or repeatedly.

[0055] In embodiments of the present invention, some steps of the claimed method may be performed using a machine learning (ML) based model. The ML-based model may be configured or “trained” for a specific task, such as classification or regression.

[0056] In some embodiments, the ML-based model may be an artificial neural network (ANN).

[0057] A neural network (NN) or artificial neural network (ANN), for example, a neural network implementing machine learning (ML) or artificial intelligence (AI) functions, may also refer to an information processing paradigm that may include nodes called neurons, which are organized into layers along with links between neurons. Links may transfer signals between neurons or be associated with weights. An NN may be configured or trained for a specific task, such as pattern recognition or classification. Training an NN for a specific task may involve adjusting these weights based on examples. Each neuron in an intermediate or final layer may receive an input signal, such as a weighted sum of output signals from other neurons, and may process the input signal using a linear or nonlinear function (e.g., an activation function). The results of the input and intermediate layers may be transferred to other neurons, and the results of the output layer may be provided as the output of the NN. Typically, neurons and links in an NN are represented by mathematical constructs such as activation functions and matrices of data elements and weights. A processor, such as a CPU or graphics processing unit (GPU), or dedicated hardware device may perform the relevant calculations.

[0058] Those skilled in the art will see that various ML-based models can be implemented without departing from the essence of the present invention. It should also be understood that in some embodiments, the ML-based model may be a single ML-based model or a set (ensemble) of ML-based models that collectively achieve the same functionality as a single model. Therefore, given the scope of the present invention, the above-described modifications should be considered equivalent.

[0059] In the context of this specification, the term “driving situation” should be considered in the broadest possible sense. This may refer to any specific situation that may occur during the process of driving a vehicle and in which the driver may be required to decide how to act. For example, a driving situation may include choosing a particular route at an intersection (e.g., turning left, turning right, or continuing straight), passing through a particular segment of the road, overtaking another vehicle, parking, etc. It should also be understood that, depending on the embodiment of the invention, a “driving situation” may be referred to as a specific geographical location (e.g., a particular intersection, a segment of the road, etc.) or may be general, and may combine all similar cases regardless of those geographical locations.

[0060] Therefore, the general term "driver behavior" or the more specific term "driver decision" should be understood as the way each driver behaves, or the decisions each driver must make when entering each driving situation. For example, driver decisions or behaviors may include deciding whether to change direction or continue straight, whether to accelerate or decelerate when passing through a particular segment of the road, or whether to overtake another vehicle when passing through a particular segment of the road and / or at a particular speed. However, the terms "driver behavior" and "driver decision" should not be confused with behavior or decisions relating to taking any action that is not related to the process of controlling the vehicle while driving it.

[0061] As can be seen, this specification proposes collecting vehicle motion data (e.g., motion data elements that can be a combination of geographical location, speed, acceleration, direction of motion, etc.) in at least one specific driving situation (e.g., an intersection), and then constructing a behavioral model that represents driver behavior (e.g., driver decisions, i.e., specific driving actions) in this situation. This situation may be predefined by a plurality of motion scenarios (e.g., (a) turning around, or (b) driving straight).

[0062] Each of the multiple motion scenarios can be represented as a sequence of motion data elements. In this context, "sequence of motion data elements" means a sequence of motion data elements, each of which corresponds to a different phase of the vehicle's motion within each scenario.

[0063] For example, the "turn right" scenario may be represented by n motion data elements, starting with a motion data element indicating deceleration as approaching the intersection, then several motion data elements representing the action of turning itself (e.g., changing direction of motion), and then ending with a motion data element indicating acceleration without further changing direction of motion. The "go straight" scenario may now be represented by m motion data elements, each of which may indicate gradual acceleration without changing direction of motion.

[0064] Therefore, as can be seen from the provided examples, each sequence of motion data elements may clearly represent the “behavioral signature” of each scenario, and consequently, the “signature” of each “expected driver decision.” For example, if the received motion data elements (or sequences of motion data elements) in the vicinity of an intersection show a certain degree of probability deceleration, it can be predicted (e.g., based on known mathematical methods) that the subsequent expected driver decision will be to change direction rather than continue straight.

[0065] Therefore, as described in detail herein, applying such behavioral models may provide reliable predictions of driver behavior, and thus may be used as a valuable tool for advanced driver assistance systems and automated driving systems to provide warnings or control signals in the event of dangerous road conditions, inappropriate driver behavior, etc., thereby reducing the risk of collisions caused by human error.

[0066] Furthermore, the present invention may have various embodiments relating to the construction (training) of behavior models. In particular, in some embodiments, behavior models may be trained separately based on the motion data elements of each vehicle. In such embodiments, each vehicle, and therefore each particular driver, may have its own vehicle-specific profile, which describes how each particular vehicle (driver) behaves in a particular driving situation. This may allow for the evaluation of the specificity of each driver's driving characteristics (peculiarities) to improve the efficiency of collision avoidance.

[0067] In other embodiments, the behavior model may be trained based on motion data elements from multiple vehicles. In such embodiments, a baseline profile may be calculated, as described in further detail herein.

[0068] Furthermore, in yet another embodiment, the approaches described above may be used in combination. In particular, the method may include the calculation of a baseline profile, and then the calculation of a vehicle-specific profile relative to the baseline profile. In such a case, each specific driver behavior may be evaluated relative to the baseline behavior, thereby identifying drivers with inappropriate driving behavior, and other drivers located near such potentially dangerous drivers may be correspondingly warned.

[0069] In this specification, the term “behavioral model” refers to a mathematical model of multiple driving conditions (in some embodiments, a machine learning-based model), each of which is represented by multiple motion scenarios, each of which is then represented by multiple motion data elements, and then by motion parameters such as (a) the geographical location of at least one vehicle, (b) the velocity of at least one vehicle, (c) the acceleration of at least one vehicle, and (d) the direction of motion of at least one vehicle. In some embodiments, the behavioral model may be geographically oriented, for example, and therefore may be segmented by geographical regions surrounding a desired geographical location. Those skilled in the art will understand that the “behavioral models” described herein can be constructed (or, in the case of machine learning, trained) and further applied (inferred) using mathematical (e.g., machine learning-based) methods known in the art. The present invention should not be considered limited to any particular method for constructing such behavioral models.

[0070] The various types of calculations described in this application (e.g., calculation of the probability of a particular driver determination occurring among several predicted driver determinations, calculation of the predicted trajectory of a vehicle, calculation of the risk of collision between vehicles, calculation of an extrapolated geographical location, etc.) may be performed, for example, using mathematical methods and techniques that are obvious to those skilled in the art and are known to the general knowledge in the art, based on the respective inputs shown in this disclosure.

[0071] Another important aspect of the present invention that contributes to the above-mentioned technical improvements lies in the purpose and necessity of relying on the prediction of driver behavior and driver decisions. In particular, if we consider hypothetical systems and methods that do not provide predictions of driver behavior and driver decisions and rely exclusively on current motion data received from the vehicle (e.g., their geographical location, speed, direction of motion, etc.), such systems and methods would not be effective in preventing collisions. Clearly, driving is a dynamic process, and each situation that occurs can change very quickly, especially when the driving speed is relatively high. Furthermore, if such a system or method requires receiving information from a server via a network, network latency should also be taken into account. On the other hand, drivers need some time to react to warnings. Thus, time is a crucial factor in avoiding the risk of collisions.

[0072] This invention addresses this problem by applying a behavioral model to predict the behavior and decisions of a certain driver and giving another driver time to react by warning them in advance (or providing them with corresponding control signals). Furthermore, this invention contributes to the improvement of advanced driver assistance and autonomous driving technologies by mitigating network latency issues.

[0073] Referring now to Figure 1, this figure is a block diagram showing a computing device that may be included in one embodiment of a system for predicting driver behavior, according to several embodiments.

[0074] Computing device 1 may include, for example, a processor or controller 2 which may be a central processing unit (CPU) processor, a chip, or any suitable computing or computing device, an operating system 3, a memory device 4, instruction code 5, a storage system 6, an input device 7, and an output device 8. Processor 2 (or, optionally, one or more controllers or processors across multiple units or devices) may be configured to perform the methods described herein and / or to perform or act as various modules, units, etc. Multiple computing devices 1 may be included in a system according to embodiments of the present invention, and one or more computing devices 1 may act as components of that system.

[0075] Operating System 3 may be, or may include, any code segment (for example, similar to instruction code 5 described herein) designed and / or configured to perform tasks involving the coordination, scheduling, arbitration, supervision, control, or other management of the operation of computing device 1, such as scheduling the execution of a software program or task, or enabling communication between a software program or other module or unit. Operating System 3 may be a commercially available operating system. It should be noted that Operating System 3 may be an optional component, and for example, in some embodiments, the system may include computing devices that do not require or include Operating System 3.

[0076] The memory device 4 may be, for example, random access memory (RAM), read-only memory (ROM), dynamic RAM (DRAM), synchronous DRAM (SD-RAM), double data rate (DDR) memory chip, flash memory, volatile memory, non-volatile memory, cache memory, buffer, short-term memory unit, long-term memory unit, or other suitable memory unit or storage unit, or may include them. The memory device 4 may be, or include, several different possible memory units. The memory device 4 may be a non-temporary readable medium of a computer or processor, or a non-temporary storage medium of a computer (e.g., RAM). In one embodiment, a non-temporary storage medium such as the memory device 4, a hard disk drive, or another storage device may store instructions or code, and when the instructions or code are executed by a processor, the processor may cause the processor to perform the methods described herein.

[0077] Instruction code 5 may be any executable code, such as an application, program, process, task, or script. Instruction code 5 may optionally be executed by a processor or controller 2 under the control of the operating system 3. For example, instruction code 5 may be a standalone application or API module that can be configured to calculate the prediction of driver behavior or the occurrence of a specific driver decision, as further described herein. For clarity, although a single item of instruction code 5 is shown in Figure 1, systems according to some embodiments of the present invention may include multiple executable code segments or modules similar to instruction code 5 that can be loaded into a memory device 4 and cause the processor 2 to execute the method described herein.

[0078] The storage system 6 may be, for example, a flash memory known in the art, a memory located inside or incorporated into a microcontroller or chip known in the art, a hard disk drive, a CD recordable (CD-R) drive, a Blu-ray disc (BD), a Universal Serial Bus (USB) device, or other suitable removable and / or fixed storage unit, or may include them. Various types of input and output data may be stored in the storage system 6, or loaded from the storage system 6 to the memory device 4, where they may be processed by the processor or controller 2. In some embodiments, some of the components shown in Figure 1 may be omitted. For example, the memory device 4 may be a non-volatile memory having the storage capacity of the storage system 6. Thus, although shown as a separate component, the storage system 6 may be embedded in or included in the memory device 4.

[0079] Input device 7 may be or include any suitable input device, component, or system, such as a detachable keyboard or keypad, mouse, etc. Output device 8 may include one or more (potentially detachable) displays or monitors, speakers, and / or any other suitable output devices. Any applicable input / output (I / O) devices may be connected to computing device 1 as shown by blocks 7 and 8. For example, a wired or wireless network interface card (NIC), a Universal Serial Bus (USB) device, or an external hard drive may be included in input device 7 and / or output device 8. It should be noted that any suitable number of input devices 7 and output devices 8 may be operably connected to computing device 1 as shown by blocks 7 and 8.

[0080] Systems according to some embodiments of the present invention may include, but are not limited to, a plurality of central processing units (CPUs) or any other suitable multipurpose or specific processor or controller (for example, similar to element 2), a plurality of input units, a plurality of output units, a plurality of memory units, and a plurality of storage units.

[0081] Referring now to Figure 2, this figure shows a schematic diagram of the concept of the present invention relating to providing collision warnings via a UI, according to several embodiments.

[0082] As can be seen from the figure, according to the concept of the present invention, the driver of a particular vehicle (e.g., the first vehicle 100) may be provided with collision warnings (e.g., warnings 111 and 112) via the user interface (UI) 110 of a client computing device 30 associated with the vehicle (e.g., installed in the vehicle).

[0083] Collision warnings 111 and 112 may be provided as a result of the detection and predicted behavior of the driver of another vehicle (e.g., a second vehicle 200) located within a geographical area surrounding the geographical location of the first vehicle.

[0084] For example, in the case of warning 111, the system may calculate a baseline profile of driver behavior in the segment of road on which both the first vehicle 100 and the second vehicle 200 are currently traveling. The system may then calculate a vehicle-specific profile for each vehicle relative to the baseline profile, representing the deviation of each vehicle's driver decision with respect to following at least one specific driving scenario (e.g., a scenario of traversing the segment of road shown on map 111A provided via UI 110). The system may then detect that the deviation of driver decision by the vehicle-specific profile of the second vehicle 200 substantially deviates from the deviation of the baseline profile (e.g., the driver of the second vehicle 200 suddenly stops their vehicle and begins to change direction). Thus, the driver of the first vehicle 100 may be informed in advance and advised to slow down on the way to the geographic location where the inappropriate driver behavior was detected (e.g., as shown in message 111B provided via UI 110). The driver of the first vehicle 100 may optionally be notified of the distance to the vehicle indicated as having inappropriate behavior.

[0085] However, the scope of the present invention is not limited to the detection of inappropriate behavior. In another example, in the case of warning 112, the system may use predictions of future driver behavior and driver decisions based on historical data (multiple motion data elements) received and stored from multiple vehicles that have passed through the same segment of the road or the same intersection (e.g., intersection 112A' shown on map 112A provided via UI 110) that the first vehicle 100 and the second vehicle 200 are currently approaching from different sides. Thus, for example, according to such predictions, the system can detect that neither driver is attempting to slow down their vehicle before crossing intersection 112A'. Furthermore, the system may be configured to calculate predicted motion trajectories 101 and 201 for each of the vehicles 100 and 200 based on each of the multiple motion data elements representing each sequence of predicted driver decisions. The system may be configured to calculate the risk of collision between vehicle 100 and vehicle 200 based on the predicted motion trajectories 101 and 201 for each of the vehicles 100 and 200. Therefore, the warning 112, including message 112B provided via UI 110, may be issued when the calculated collision risk exceeds a predetermined threshold.

[0086] The motion data elements may include data relating to the geographical location, speed, acceleration, and direction of motion of each vehicle. Therefore, in the context of this specification, the term “trajectory” should be understood to refer not only to data elements indicating the direction or path of motion, but also to elements having this “path” augmented with speed and / or acceleration and / or precise geographical location. Furthermore, the predicted motion trajectory data elements may be augmented with decreasing probability paths representing the probability of following the predicted motion trajectory within its different segments. Thus, the system may be configured to calculate the risk of collision between vehicle 100 and vehicle 200 by, for example, calculating the probability that trajectories 101 and 201 intersect (e.g., at the same time).

[0087] It should also be understood that orbits 101 and 201 are schematically shown in Figure 2 not to provide examples of the orbits themselves, but to aid in understanding how warning 112 is formed.

[0088] Referring here to Figures 3A and 3B, the concept of the present invention relating to predicting driving decisions according to a specific motion scenario is schematically illustrated.

[0089] As described above, in some embodiments, a specific driving condition (e.g., driving condition 300) may be predefined by a plurality of motion scenarios (e.g., motion scenario 310 shown in Figure 3A and motion scenario 320 shown in Figure 3B). Therefore, in some embodiments, the expected driver behavior may be predefined by a plurality of expected driver decisions, each corresponding to following a specific motion scenario among the plurality of motion scenarios (e.g., motion scenarios 310 and 320).

[0090] Figures 3A and 3B illustrate two examples of motion scenarios 310 and 320 that may occur in a specific driving situation (e.g., driving situation 300). As can be seen from the figures, situation 300 represents a case of two consecutive turns. According to motion scenario 310, each driver makes a turn at the first turn but does not make a turn at the second turn and continues to drive straight. According to motion scenario 320, each driver decides to make a turn at both the first and second turns.

[0091] In some embodiments, each of the motion scenarios 310 and 320 may be represented as a sequence of corresponding motion data elements 311, 312, 313, 314, 315 and 321, 322, 323, 324, 325, and 326. Each motion data element 311-315 and 321-326 is shown in the figure as a velocity vector representing the geographical position, velocity (e.g., represented as the length of each vector), and direction of motion (e.g., represented as the orientation of each vector) of the respective vehicle.

[0092] As can be seen from the figure, motion data elements 311 and 312 are equal to their respective motion data elements 321 and 322, and therefore do not represent any difference between motion scenarios 310 and 320 at this stage. However, starting from motion data elements 313 and 323, the difference is clearly visible. According to motion scenario 310, the driver decides not to turn at the second and to continue going straight, so does not decelerate their vehicle before the second. Therefore, as shown in the figure, motion data elements 313, 314, and 315 indicate that the vehicle's speed is gradually increasing (each following motion data element is longer than the preceding element).

[0093] Motion data elements 323 and 324 show the same direction of motion as elements 313-315, but the vehicle speed gradually decreases, which is typical action before a change of direction. Elements 325 and 326 show the change in direction of motion and the increase in speed after a change of direction.

[0094] Furthermore, the respective motion trajectories 316 and 327 may be calculated based on motion data elements 311-315 and 321-326.

[0095] As can be seen from the provided examples of motion scenarios 310 and 320, each of the multiple motion data elements may represent a strong basis for reliable prediction of expected driver behavior, particularly specific driver decisions.

[0096] Referring now to Figures 4A, 4B, and 4C, these figures show a system 10 for predicting driver behavior, which includes at least one client computing device 30 that is communicatively connected to a server computing device 40, according to several embodiments.

[0097] According to some embodiments of the present invention, system 10 may be implemented as a software module, a hardware module, or any combination thereof. For example, system 10 may be a computing device such as element 1 in Figure 1, or may include such a device. Furthermore, system 10 may be adapted to execute one or more modules of instruction code (e.g., element 5 in Figure 1) to request, receive, analyze, calculate, and generate various data.

[0098] As will be described in more detail herein, system 10 may be adapted to execute one or more modules of instruction code (e.g., element 5 in Figure 1) to perform the steps of the claimed method.

[0099] As shown in Figures 4A, 4B, and 4C, the arrows may represent the flow of one or more data elements to, from, and / or between modules or elements of system 10. Some arrows are omitted in Figures 4A, 4B, and 4C for clarity.

[0100] As shown in Figure 4A, the client computing device 10 may be associated with the first vehicle 100, or, for example, installed inside the first vehicle 100. The client computing device 10 may be communicably connected to a vehicle motion sensor 20, which includes a Global Positioning System (GPS) 21, an accelerometer (or gyroscope) 22, a speed sensor 23, and a timestamp module 24.

[0101] In some embodiments, the client computing device 10 may be configured to receive geolocation data elements 21A' representing the respective geolocations 21A of the first vehicle 100. The client computing device 10 may be further configured to receive acceleration values ​​22A from an accelerometer 22. The client computing device 10 may be further configured to receive speed values ​​23A from a speed sensor 23. The client computing device 10 may be further configured to receive a global timestamp 24A from a timestamp module 24 indicating the time of determination of each parameter (e.g., geolocation 21A, acceleration value 22A, speed value 23A) by the sensors 20.

[0102] In some embodiments, the client computing device 10 may include a motion data element generation module 31. The motion data element generation module 31 may be configured to aggregate data received from the sensor 20 and form motion data elements (e.g., incoming motion data elements 31A) that characterize the motion of the first vehicle 100. The motion data elements 31A may represent the geographical location, velocity, acceleration, and direction of motion of the first vehicle 100, and each may be assigned a global timestamp 24A representing the time when each measurement is taken.

[0103] In some embodiments, the motion data element generation module 31 may be configured to calculate the direction of motion based on a pair of consecutive geographical locations 21A, for example, as the direction of movement from a paired first geographical location 21A to a second geographical location. In some alternative embodiments, the motion data element generation module 31 may be configured to calculate the direction of motion based on acceleration values ​​22A from an accelerometer 22 (for example, if a 3-axis accelerometer sensor is used). It should be understood that the above examples of motion direction calculation are non-exclusive and different methods may be used within the scope of the present invention.

[0104] The client computing device 30 may be further configured to obtain from the server computing device 40 a segment 45' of a behavioral model 44' representing a geographical area surrounding the geographical location 21A of the first vehicle 100. The client computing device 30 may be further configured to obtain from the server computing device 40 a motion data element (e.g., an incoming motion data element 41A of the other vehicle) corresponding to the geographical location of a second vehicle 200 within the same geographical area.

[0105] The client computing device 30 may be further configured to predict the occurrence of a specific driver determination 10A' of the first vehicle 100, which is represented by a consequence motion data element (e.g., consequence motion data element 10A) that characterizes the expected motion of the first vehicle 100 in at least one specific driving situation (e.g., the driving situations 300 shown in Figures 3A and 3B) within each geographical area, and to predict the expected driver behavior. The client computing device 30 may further be configured to infer a segment 45' of a behavior model 44' for motion data elements corresponding to the geographical location of the second vehicle 200 (e.g., incoming motion data elements 41A of other vehicles) and to predict the occurrence of a specific driver determination 10A'' of the second vehicle 200, which is represented by a consequence motion data element (e.g., consequence motion data element 10A) that characterizes the expected motion of the second vehicle 100 in at least one specific driving situation (e.g., driving situations 300 shown in Figures 3A and 3B) within each geographical area.

[0106] The client computing device 30 may further include a trajectory calculation module 32. The trajectory calculation module 32 may be further configured to receive consequent motion data elements (e.g., consequent motion data element 10A) of the first vehicle 100 and the second vehicle 200. The trajectory calculation module 32 may be further configured to calculate the predicted trajectory 32A' of the first vehicle 100 based on at least one consequent motion data element (e.g., consequent motion data element 10A) of the first vehicle 100. The trajectory calculation module 32 may be further configured to calculate the predicted trajectory 32A'' of the second vehicle 200 based on at least one consequent motion data element (e.g., consequent motion data element 10A) of the second vehicle 200.

[0107] In some embodiments, the trajectory calculation module 32 may be further configured to calculate the predicted trajectory (e.g., trajectory 32A' or 32A'') as a Bézier curve.

[0108] In some embodiments, the client computing device 30 may be further configured to predict a sequence of driver decisions for each vehicle (e.g., decision 10A' for vehicle 100 or decision 10A'', represented as a sequence of result motion data elements 10A, where each result motion data element 10A represents the motion of each vehicle 100 or 200 at a future point in time preceding that of a subsequent iteration.

[0109] In some embodiments, the client computing device 30 may be configured to calculate the probabilities of the driver decisions 10A' and 10A'' for the drivers of the first vehicle 100 and the second vehicle 200 occurring, respectively, by inferring a segment 45' of the behavior model 44'. The trajectory calculation module 32 may be further configured to calculate, with respect to the first vehicle 100 and the second vehicle 200, decreasing probability path data elements 32B' and 32B'' representing the probabilities of following predicted motion trajectories 32A' and 32A'', respectively, based on (i) the respective sequences of the resulting motion data elements 10A for the respective vehicles 100 and 200 and (ii) the probabilities of the respective driver decisions 10A' and 10A'' occurring.

[0110] In some embodiments, the client computing device 30 may further include a collision risk analysis module 33. The collision risk analysis module 33 may be configured to receive data representing predicted motion trajectories 32A' and 32A'' and optionally decreasing probability paths 23B' and 32B''. The collision risk analysis module 33 may be further configured to calculate the risk (e.g., probability) of a collision between the first vehicle 100 and the second vehicle 200 based on the predicted motion trajectories 32A' and 32A'' and optionally based on decreasing probability paths 23B' and 32B'' for the first vehicle 100 and the second vehicle 200, respectively.

[0111] The client computing device 30 may further include a user interface (UI) module 34. The UI module 34 may be configured to receive data relating to the risk of collision 33A. The UI module 34 may be further configured to provide collision warnings 34A (e.g., the same as warnings 111 and 112 shown in Figure 2) via the UI to each driver (e.g., the driver of the first vehicle 100) when the calculated risk of collision exceeds a predetermined threshold.

[0112] It should be understood that providing collision warning 34A to the driver is provided only as a non-exclusive example of how predicted expected driver behavior or specific driver decisions (e.g., driver decisions 10A' or 10A'') may be further used to reduce the risk of collisions, particularly collisions caused by human error.

[0113] For example, in some alternative embodiments, the first vehicle 100 may be an autonomous vehicle. In such a case, the client computing device 30 may be further configured to apply respective control signals to avoid collisions (for example, by slowing, accelerating, and turning each vehicle) based on at least one of the following: (i) a predicted driver determination (e.g., driver determination 10A''), (ii) a resulting motion data element 10A, (iii) a motion trajectory 32A'', and (iv) a decreasing probability path 32B'' of other vehicles (e.g., a second vehicle 200) within a geographical area surrounding the geographical location 21A of the first vehicle 100.

[0114] Furthermore, in some alternative embodiments, the client computing device 30 may be further configured to calculate multiple collision risks (e.g., collision risk 33A) for multiple motion scenarios (e.g., motion scenario 310 or 320). The client computing device 30 may be further configured to select and apply a “driver” decision to follow the motion scenario (e.g., motion scenario 310 or 320) associated with the lowest collision risk (e.g., collision risk 33A).

[0115] Referring now to Figure 4B, an alternative embodiment of the client computing device 30 is provided.

[0116] Since most of the embodiments provided are similar to the embodiments shown in Figure 4A, only the different embodiments will be described with reference to Figure 4B.

[0117] As can be seen from the figure, in the provided embodiment, only the geographical location 21A and global timestamp 24A are provided from the vehicle motion sensor 20. Therefore, in such an embodiment, the motion data element generation module 31 may be further configured to receive a plurality of geographical location data elements 21A' representing each of the plurality of geographical locations 21A of the first vehicle 100. The motion data element generation module 31 may be further configured to calculate each motion data element 31A as a motion vector (e.g., velocity vector) characterizing the motion (e.g., velocity and direction) of the first vehicle 100 between the plurality of geographical locations 21A, based on the plurality of geographical location data elements 21A'.

[0118] In some embodiments, the motion data element generation module 31 may be further configured to receive from a server computing device 40 a plurality of geolocation data elements 21A''' representing each of a plurality of geolocations 21A'' of another vehicle (e.g., a second vehicle 200), each of which is assigned a global timestamp 24A' and a received timestamp 24A'', corresponding to the time of determination of each geolocation 21A'' and the time of reception of each geolocation data element 21A'''. The motion data element generation module 31 may be further configured to calculate an extrapolated geolocation 21B'' of at least one vehicle (e.g., a second vehicle 200) based on (i) each of the plurality of geolocations 21A'', (ii) each of the global timestamps 24A' and (iii) each of the received timestamps 24A'' of the plurality of geolocation data elements 21A'''.

[0119] In some embodiments, the motion data element generation module 31 may be further configured to calculate the incoming motion data element 31A as a motion vector (e.g., a velocity vector) based on a further extrapolated geographical location 21B''.

[0120] As can be seen from the figure, the embodiment provided in Figure 4B may have aspects that contribute to the technical effects described above in addition, and in particular this embodiment may provide an additional improvement in negating the risk of collision. Such an improvement may be provided by taking into account the time of reception (reception timestamp 24A'') in combination with each global timestamp 24A', thereby disabling network latency (e.g., the network communicating between the server computing device 40 and the client computing device 30) and correcting the geographical location of each vehicle (e.g., vehicle 200) accordingly.

[0121] Referring now to Figure 4C, we see a server computing device 40 of a system 10 for predicting driver behavior, according to several embodiments.

[0122] As can be seen from the figures, in some embodiments, the server computing device 40 may include a motion data element generation module 41. The motion data element generation module 41 may be similar to or the same as the motion data element generation module 31 described with reference to Figures 4A and 4B.

[0123] The motion data element generation module 41 may be configured to receive global timestamps 24A and 24A' and a plurality of geographic location data elements 21A'' representing a plurality of geographic locations 21A'' for each of the plurality of vehicles (e.g., vehicles 100 and 200) from each client computing device 30. The motion data element generation module 41 may be further configured to calculate motion data elements 41A for the plurality of vehicles (in the same manner as described with respect to the motion data element generation module 31 with reference to Figures 4A and 4B). The motion data elements 41A may characterize the motion of the plurality of vehicles (e.g., vehicles 100 and 200) in at least one specific driving condition (e.g., driving condition 300).

[0124] In some embodiments, the server computing device 40 may further include a driving condition analysis module 42. The driving condition analysis module 42 may be configured to receive motion data elements 41A. The driving condition analysis module 42 may be further configured to analyze a plurality of motion data elements 41A to determine a sequence of motion data elements 41A (e.g., sequences 311-315 and 321-326, as shown in Figure 2) that represents a plurality of motion scenarios 42A'' (e.g., motion scenarios 310 and 320, as shown in Figure 2).

[0125] The driving situation analysis module 42 may be further configured to form a plurality of decision data elements 42A', each representing a plurality of predicted driver decisions (e.g., driver decisions 10A' and 10A'') that correspond to following a specific motion scenario of a plurality of motion scenarios 42A'' (e.g., motion scenarios 310 and 320 as shown in Figure 2).

[0126] In some embodiments, the server computing device 40 may further include a training module 44. The training module 44 may be configured to construct a behavior model 44' representing expected driver behavior in at least one specific driving situation (e.g., driving situation 300) based on a plurality of motion data elements 41A. In some embodiments, the behavior model 44' may be a machine learning (ML) based model. Thus, in some embodiments, the training module 44 may be further configured to construct a behavior model 44' by training it based on a plurality of decision data elements 42A' to (a) receive incoming motion data elements (e.g., motion data elements 31A or 41A), (b) calculate the probability that a particular driver decision will occur among a plurality of expected driver decisions (e.g., decisions 10A' and 10A'') based on the incoming motion data elements (e.g., motion data elements 31A or 41A), and (c) predict the occurrence of a particular driver decision based on the calculated probability.

[0127] It should be understood that the ML-based model may be based on any known machine learning and artificial intelligence techniques (e.g., artificial neural networks, linear regression, decision tree regression, random forest, KNN models, support vector machines (SVM)) (or combinations thereof) that are commonly used for classification, clustering, regression, and other tasks that may be relevant to the objectives of the present invention. Accordingly, the scope of the present invention is not limited to any particular embodiment of the ML-based model, and it is implied herein that it will be obvious to those skilled in the art which techniques may be applied in the form of the resulting motion data element 10A to train the ML-based model to predict expected driver behavior (e.g., driver decision 10A' or 10A'').

[0128] In some embodiments, the server computing device 40 may further include a segmentation module 45. The segmentation module 45 may be configured to receive geolocation data elements 21A''' of a plurality of vehicles (e.g., vehicles 100 and 200) and segment the behavior model 44' to obtain a segment 45' of the behavior model 44' representing the geographic area surrounding the geolocation 21A'' of each vehicle (e.g., vehicles 100 and 200). The server computing device 40 may be further configured to transmit the segment 45' to the client computing devices 30 of each vehicle (e.g., vehicles 100 and 200).

[0129] In some embodiments, the server computing device 40 may further include a vehicle profile analysis module 43. The vehicle profile analysis module 43 may be configured to analyze a plurality of decision data elements 42A' to obtain a baseline profile data element 43A' representing a baseline distribution of a plurality of expected driver decisions (e.g., driver decisions 10A' and 10A'') for a particular motion scenario 42A'' among a plurality of motion scenarios 42A''.

[0130] In some embodiments, the vehicle profile analysis module 43 may be further configured to analyze incoming motion data elements (e.g., motion data element 41A) of a particular vehicle (e.g., vehicle 100 or 200) in relation to a baseline profile data element 43A' that represents a deviation of one or more driver decisions (e.g., decisions 10A' and 10A'') for each vehicle (e.g., vehicle 100 or 200) with respect to following at least one specific motion scenario (e.g., motion scenario 310 or 320 shown in Figures 3A and 3B) in order to obtain a vehicle-specific profile data element 43A''.

[0131] Now, let's refer back to Figure 4A.

[0132] In some embodiments, the client computing device 30 may be configured to receive vehicle-specific profile data elements 43A'' for either the vehicle associated with the device 30 (e.g., vehicle 100) or another vehicle (e.g., vehicle 200).

[0133] The client computing device 30 may further be configured to infer a behavior model 44', in particular a segment 45' of the behavior model 44', for each of the incoming motion data elements 31A or 41A, and for each of the vehicle-specific profile data elements 43A'' of each vehicle (e.g., the same vehicle, e.g., the first vehicle 100, or another vehicle, e.g., the second vehicle 200), in order to predict the occurrence of a particular driver decision among a plurality of expected driver decisions (e.g., driver decisions 10A' and 10A'').

[0134] Referring to Figure 5A, a flowchart is presented showing a method for predicting driver behavior using at least one processor, according to several embodiments.

[0135] As shown in step S1005, at least one processor (e.g., processor 2 in Figure 1) may perform the reception of a plurality of motion data elements (e.g., motion data elements 31A or 41A) that characterize the motion of at least one vehicle (e.g., vehicle 100 or 200) in at least one specific driving condition (e.g., driving condition 300). Step S1005 may be performed by a motion data element generation module 31 (as described with reference to Figures 4A to 4C).

[0136] As shown in step S1010, at least one processor (e.g., processor 2 in Figure 1) may perform the construction of a behavior model (e.g., behavior model 44') that represents the expected driver behavior (e.g., driver decision 10A' or 10A'') in at least one specific driving situation (e.g., driving situation 300) based on a plurality of motion data elements (e.g., motion data elements 31A or 41A). Step S1010 may also be performed by the driving situation analysis module 42 and the training module 44 (as described with reference to Figure 4C).

[0137] As shown in step S1015, at least one processor (e.g., processor 2 in Figure 1) may perform inference of a behavior model (e.g., behavior model 44') on at least one incoming motion data element (e.g., motion data element 31A or 41A) to predict expected driver behavior (e.g., driver decision 10A' or 10A'') in a specific driving situation (e.g., driving situation 300). Step S1015 may be performed by a client computing device 30 (as described with reference to Figures 4A and 4B).

[0138] Referring now to Figure 5B, a flowchart is presented showing a method for predicting vehicle motion using at least one processor, according to several embodiments.

[0139] As shown in step S2005, at least one processor (e.g., processor 2 in Figure 1) may perform the reception of a plurality of geolocation data elements (e.g., geolocation data element 21A'') representing the geolocation (e.g., geolocation 21A'') of at least one vehicle (e.g., vehicles 100 and 200), each geolocation data element (e.g., geolocation data element 21A''') is assigned a global timestamp (e.g., global timestamp 24A' or 24A) corresponding to the time of determination of the respective geolocation (e.g., geolocation 21A'') and the time of reception of the respective geolocation data element (e.g., geolocation data element 21A'''), and a reception timestamp (e.g., reception timestamp 24A''). Step S2005 may be performed by the motion data element generation module 31 (as described with reference to Figures 4B to 4C).

[0140] As shown in step S2010, at least one processor (e.g., processor 2 in Figure 1) may perform the calculation of a plurality of extrapolated geographic locations (e.g., extrapolated geographic locations 21B'') based on (i) the geographic location (e.g., geographic location 21A'') of a plurality of geographic location data elements (e.g., geographic location data element 21A''), (ii) the global timestamp (e.g., global timestamp 24A'') of a plurality of geographic location data elements (e.g., geographic location data element 21A''). Step S2010 may be performed by the motion data element generation module 31 (as described with reference to Figure 4B).

[0141] As shown in step S2015, at least one processor (e.g., processor 2 in Figure 1) may perform the calculation of at least one incoming motion data element (e.g., incoming motion data element 31A) that represents the velocity and direction of motion between a plurality of extrapolated geographical locations (e.g., extrapolated geographical location 21B''). Step S2015 may be performed by the motion data element generation module 31 (as described with reference to Figure 4B).

[0142] As shown in step S2020, at least one processor (e.g., processor 2 in Figure 1) may perform inference on at least one incoming motion data element (e.g., incoming motion data element 31A) using a pre-trained machine learning (ML) based model (e.g., behavior model 44' or segment 45' of behavior model 44') to predict a consequence motion data element (e.g., consequence motion data element 10A) representing the expected motion of at least one vehicle (e.g., vehicle 100 or 200). Step S2020 may be performed by a client computing device 30 (as described with reference to Figure 4B).

[0143] As can be seen from the description provided, the present invention provides a system and method for predicting driver behavior that provides improvements in the technology of advanced driver assistance and autonomous driving by reducing the risk of collisions caused by human error.

[0144] Unless expressly stated otherwise, the embodiments of the methods described herein are not restricted to any particular order or sequence. Furthermore, all formulas described herein are intended merely as examples, and other or different formulas may be used. In addition, some of the embodiments or elements thereof of the described methods may occur or be performed simultaneously.

[0145] While some features of the present invention have been illustrated and described herein, those skilled in the art will be able to conceive of many modifications, substitutions, alterations, and equivalents. Therefore, it should be understood that the appended claims are intended to cover all such modifications and alterations that fall within the true spirit of the invention.

[0146] We have presented various embodiments. Each of these embodiments may, of course, include features from other embodiments presented, and embodiments not specifically described may also include various features described herein.

Claims

1. A method for predicting driver behavior using at least one computing device, The steps include receiving a plurality of motion data elements that characterize the motion of at least one vehicle in at least one specific driving condition, The steps include constructing a behavior model that represents the expected driver behavior in at least one specific driving situation based on the plurality of motion data elements, The steps include: inferring the behavior model for at least one incoming motion data element to predict the expected driver behavior in at least one specific driving situation; Methods that include...

2. The aforementioned at least one specific driving condition is predefined by a plurality of motion scenarios, The predicted driver behavior is predefined by a plurality of predicted driver decisions, each corresponding to following a specific motion scenario among the plurality of motion scenarios. The method according to claim 1, wherein the step of inferring the behavior model includes the step of inferring the behavior model for at least one incoming motion data element to predict the occurrence of a specific driver decision among the plurality of predicted driver decisions.

3. The method according to claim 2, wherein each of the plurality of motion scenarios is represented as a sequence of motion data elements of the plurality of motion data elements.

4. The behavioral model is a machine learning (ML) based model, and the steps for constructing the behavioral model are as follows: The steps include analyzing the plurality of motion data elements to determine the sequence of motion data elements representing the plurality of motion scenarios, The steps include forming a plurality of decision data elements, each representing a plurality of predicted driver decisions corresponding to following a specific motion scenario among the plurality of motion scenarios, The steps include: training the behavior model based on the plurality of decision data elements to (a) receive the incoming motion data element, (b) calculate the probability that a specific driver decision will occur among the plurality of predicted driver decisions based on the incoming motion data element, and (c) predict the occurrence of the specific driver decision based on the probability; The method according to any one of claims 1 to 3, including

5. The step of receiving the plurality of motion data elements includes the step of receiving a plurality of motion data elements that characterize the motion of a plurality of vehicles in at least one specific driving condition, and the method is The steps include analyzing the plurality of decision data elements to obtain a baseline profile data element representing the baseline distribution of the plurality of predicted driver decisions for at least one specific motion scenario among the plurality of motion scenarios, The steps include: analyzing at least one incoming motion data element of the at least one vehicle in relation to the baseline distribution to obtain vehicle-specific profile data elements representing the deviation of one or more driver decisions of each vehicle from following the at least one specific motion scenario; The method according to claim 4, further comprising:

6. step of receiving the vehicle-specific profile data elements of the at least one vehicle. It further includes, The method according to claim 5, wherein the step of inferring the behavior model further includes the step of inferring the behavior model with respect to (a) the at least one incoming motion data element and (b) the vehicle-specific profile data element to predict the occurrence of the specific driver decision among the plurality of predicted driver decisions.

7. The method according to any one of claims 1 to 6, wherein the predicted driver determination is represented by at least one consequence motion data element that characterizes the predicted motion of at least one vehicle in at least one specific driving situation.

8. The method according to any one of claims 1 to 7, wherein the step of constructing the behavioral model is performed by at least one server computing device, and the step of inferring the behavioral model is performed by at least one client computing device communicatively connected to the at least one server computing device.

9. The at least one client computing device is associated with the first vehicle, and the method is The steps include determining the geographical location of the first vehicle using at least one client computing device, The steps include: obtaining a segment of the behavior model representing the geographical area surrounding the geographical location of the first vehicle from the at least one server computing device using the at least one client computing device; The steps include: obtaining from the at least one server computing device, using the at least one client computing device, at least one second motion data element corresponding to the geographical location of the second vehicle within the geographical area; The steps include: using the at least one client computing device to infer the segment of the behavior model with respect to the at least one second motion data element to predict the occurrence of the specific driver determination for the second vehicle; The method according to claim 8, further comprising:

10. The determination of the specific driver of the second vehicle is represented by at least one second consequence motion data element that characterizes the expected motion of the second vehicle in at least one specific driving situation within the geographical area, The method according to claim 9, further comprising the step of calculating a predicted motion trajectory of the second vehicle based on the at least one second resulting motion data element of the second vehicle.

11. The steps include receiving the at least one first incoming motion data element characterizing the current motion of the first vehicle by the at least one client computing device, The steps include: using the at least one client computing device to infer the segment of the behavior model with respect to the at least one first incoming motion data element, and predicting the occurrence of the specific driver determination of the first vehicle, which is represented by at least one first resulting motion data element that characterizes the expected motion of the first vehicle in at least one specific driving situation within the geographical area; The steps include: calculating the predicted trajectory of the first vehicle based on the at least one first resulting motion data element using the at least one client computing device; The steps include: calculating the risk of collision between the first vehicle and the second vehicle based on the predicted motion trajectories of the first vehicle and the second vehicle using at least one client computing device; The steps include: providing a collision warning via the user interface (UI) of the client computing device when the calculated collision risk exceeds a predetermined threshold; The method according to claim 10, further comprising:

12. The method according to claim 10 or 11, wherein the step of calculating the predicted motion trajectory includes the step of iteratively inferring the segment of the behavior model for at least one resulting motion data element calculated in a preceding iteration by the at least one client computing device to predict a sequence of driver decisions for each of the respective vehicles, which are represented as a sequence of resulting motion data elements, the resulting motion data element of each iteration representing the motion of each of the vehicles at a future point in time preceding the motion data element of a subsequent iteration.

13. Each of the aforementioned driver decision sequences is associated with the probability of each of the aforementioned driver decisions occurring. The method of claim 12, wherein the step of calculating the predicted motion trajectory further includes the step of calculating a decreasing probability path data element representing the probability of following the predicted motion trajectory, based on (i) the sequence of the resulting motion data elements and (ii) the probability of each of the respective driver decisions occurring.

14. The method according to any one of claims 10 to 13, wherein the predicted trajectory is calculated as a Bézier curve.

15. The method according to any one of claims 1 to 14, wherein each of the plurality of motion data elements represents at least one of (a) the geographical location of the at least one vehicle, (b) the speed of the at least one vehicle, (c) the acceleration of the at least one vehicle, and (d) the direction of motion of the at least one vehicle.

16. The steps include receiving a plurality of geographic location data elements representing a plurality of geographic locations of each of the at least one vehicle, A step of calculating each of the motion data elements of the plurality of motion data elements as a motion vector that characterizes the motion of the at least one vehicle between the plurality of geographic locations, based on the plurality of geographic location data elements. The method according to any one of claims 1 to 15, further comprising:

17. A step of receiving a plurality of geographic location data elements representing a plurality of geographic locations of at least one vehicle, wherein each of the plurality of geographic location data elements is assigned a global timestamp corresponding to the time of determination of each geographic location and the time of reception of each geographic location data element, and a reception timestamp. A step of calculating the extrapolated geographic location of the at least one vehicle based on (i) each of the multiple geographic locations, (ii) each global timestamp, and (iii) each received timestamp of the multiple geographic location data elements, The steps include: calculating the at least one incoming motion data element as a motion vector based on the extrapolated geographic location; The method according to any one of claims 1 to 16, further comprising:

18. A method for predicting the motion of a vehicle using at least one computing device, A step of receiving a plurality of geolocation data elements representing the geolocation of at least one vehicle, wherein each geolocation data element is assigned a global timestamp corresponding to the time of determination of the respective geolocation, the time of reception of the respective geolocation data element, and a reception timestamp. The steps include: calculating a plurality of extrapolated geographic locations based on (i) the geographic location of each of the plurality of geographic location data elements, (ii) the global timestamp of each of the plurality of geographic location data elements, and (iii) the received timestamp of each of the plurality of geographic location data elements; A step of calculating at least one incoming motion data element representing the velocity and direction of motion between a plurality of extrapolated geographic locations, The steps include: inferring a pre-trained machine learning (ML) based model from the at least one incoming motion data element to predict a consequence motion data element representing the expected motion of the at least one vehicle; Methods that include...

19. The steps include receiving multiple geographic location data elements representing the geographic locations of multiple vehicles, A step of calculating a plurality of motion data elements that represent the speed and direction of the motion of each of the plurality of vehicles between each geographic location, based on the plurality of geographic location data elements. The steps include analyzing the plurality of motion data elements to determine the sequence of motion data elements representing a plurality of motion scenarios in at least one specific driving condition, The steps include forming a plurality of decision data elements, each representing a plurality of predicted driver decisions corresponding to following a specific motion scenario among the plurality of motion scenarios, The steps include: training an ML-based model based on the plurality of decision data elements to (a) receive the incoming motion data element, (b) calculate the probability that each of the plurality of predicted driver decisions will occur, (c) predict the occurrence of a specific driver decision among the plurality of predicted driver decisions based on the calculated probabilities, and (d) calculating the resulting motion data element that characterizes the predicted motion of the at least one vehicle in the at least one specific driving situation based on the predicted occurrence of the specific driver decision; The method according to claim 18, further comprising:

20. A system for predicting driver behavior, comprising a non-temporary memory device storing instruction code modules, and at least one processor associated with the memory device and configured to execute the instruction code modules, wherein when the instruction code modules are executed, the at least one processor The steps include receiving a plurality of motion data elements that characterize the motion of at least one vehicle in at least one specific driving condition, The steps include constructing a behavior model that represents the expected driver behavior in at least one specific driving situation based on the plurality of motion data elements, The steps include: inferring the behavior model for at least one incoming motion data element to predict the expected driver behavior in at least one specific driving situation; A system configured to perform the following actions.

21. The aforementioned at least one specific driving condition is predefined by a plurality of motion scenarios, The predicted driver behavior is predefined by a plurality of predicted driver decisions, each corresponding to following a specific motion scenario among the plurality of motion scenarios. The system according to claim 20, wherein the at least one processor is configured to further infer the behavior model by inferring the behavior model on the at least one incoming motion data element to predict the occurrence of a specific driver decision among the plurality of predicted driver decisions.

22. The system according to claim 21, wherein each of the plurality of motion scenarios is represented as a sequence of motion data elements of the plurality of motion data elements.

23. The behavioral model is a machine learning (ML) based model, and the at least one processor is The steps include analyzing the plurality of motion data elements to determine the sequence of motion data elements representing the plurality of motion scenarios, The steps include forming a plurality of decision data elements, each representing a plurality of predicted driver decisions corresponding to following a specific motion scenario among the plurality of motion scenarios, The steps include: training the behavior model based on the plurality of decision data elements to (a) receive the incoming motion data element, (b) calculate the probability that a specific driver decision will occur among the plurality of predicted driver decisions based on the incoming motion data element, and (c) predict the occurrence of the specific driver decision based on the probability; The system according to any one of claims 20 to 22, configured to construct the behavioral model by performing the above.

24. The aforementioned plurality of motion data elements characterize the motion of the plurality of vehicles in the at least one specific driving situation. The aforementioned at least one processor is The steps include: analyzing the plurality of decision data elements to obtain a baseline profile data element representing the baseline distribution of the plurality of predicted driver decisions for at least one specific motion scenario among the plurality of motion scenarios; The steps include: analyzing at least one incoming motion data element of the at least one vehicle in relation to the baseline distribution to obtain vehicle-specific profile data elements representing the deviation of one or more driver decisions of each vehicle from following the at least one specific motion scenario; The system according to claim 23, further configured to perform the following:

25. The aforementioned at least one processor is The steps include receiving the vehicle-specific profile data elements of at least one of the vehicles, (a) inferring the behavior model with respect to at least one incoming motion data element and (b) the vehicle-specific profile data element, thereby further inferring the behavior model to predict the occurrence of the specific driver decision among the plurality of predicted driver decisions; The system according to claim 24, further configured to perform the following:

26. The system according to any one of claims 20 to 25, wherein the predicted driver determination is represented by at least one consequence motion data element that characterizes the predicted motion of at least one vehicle in at least one specific driving situation.

27. The system according to any one of claims 20 to 26, wherein the at least one processor comprises at least one first processor associated with at least one server computing device and at least one second processor associated with at least one client computing device communicably connected to the at least one server computing device, the at least one processor configured to construct the behavioral model being the at least one first processor, and the at least one processor configured to infer the behavioral model being the at least one second processor.

28. The at least one client computing device is associated with a first vehicle, and the at least one second processor is The steps include determining the geographical location of the first vehicle, The steps include obtaining a segment of the behavior model representing the geographical area surrounding the geographical location of the first vehicle from at least one server computing device, and The steps include obtaining at least one second motion data element corresponding to the geographical location of the second vehicle within the geographical area from the at least one server computing device, The steps include inferring the behavior model by inferring the segment of the behavior model for the at least one second motion data element, thereby predicting the occurrence of the specific driver determination for the second vehicle, The system according to claim 27, further configured to perform the following:

29. The determination of the specific driver of the second vehicle is represented by at least one second consequence action data element that characterizes the expected motion of the second vehicle in at least one specific driving situation within the geographical area, The system according to claim 28, wherein the at least one second processor is further configured to perform the step of calculating a predicted motion trajectory of the second vehicle based on the at least one second resulting motion data element of the second vehicle.

30. The at least one second processor is The steps include receiving the at least one first incoming motion data element that characterizes the current motion of the first vehicle, Steps include: inferring the segment of the behavior model with respect to the at least one first incoming action data element to predict the occurrence of the specific driver determination of the first vehicle, which is represented by at least one first resulting motion data element that characterizes the expected motion of the first vehicle in at least one specific driving situation within the geographical area; A step of calculating the predicted motion trajectory of the first vehicle based on the at least one first resulting motion data element, A step of calculating the collision risk between the first vehicle and the second vehicle based on the predicted motion trajectories of the first vehicle and the second vehicle, The steps include providing a collision warning via the user interface (UI) of at least one client computing device when the calculated collision risk exceeds a predetermined threshold, The system according to claim 29, further configured to perform the following:

31. The at least one second processor is Steps include: iteratively inferring the segments of the behavior model for each of the at least one resulting motion data elements computed in a preceding iteration to predict a sequence of driver decisions for each of the vehicles, represented as a sequence of resulting motion data elements, wherein each resulting motion data element in each iteration represents the motion of each of the vehicles at a future point in time preceding the motion data elements in a subsequent iteration; The system according to claim 29 or 30, configured to calculate the predicted motion trajectory by performing the following.

32. Each of the aforementioned driver decision sequences is associated with the probability of each of the aforementioned driver decisions occurring. The system according to claim 31, wherein the at least one second processor is configured to perform the step of further calculating the predicted motion trajectory by performing the step of calculating a decreasing probability path data element representing the probability of following the predicted motion trajectory based on (i) the sequence of the resulting motion data elements and (ii) the probability of each of the respective driver decisions occurring.

33. The system according to any one of claims 29 to 32, wherein the at least one second processor is further configured to calculate the predicted trajectory as a Bézier curve.

34. The method according to any one of claims 1 to 14, wherein each of the plurality of motion data elements represents at least one of (a) the geographical location of the at least one vehicle, (b) the speed of the at least one vehicle, (c) the acceleration of the at least one vehicle, and (d) the direction of motion of the at least one vehicle.

35. The aforementioned at least one processor is The steps include receiving a plurality of geographic location data elements representing a plurality of geographic locations of each of the at least one vehicle, A step of calculating each of the motion data elements of the plurality of motion data elements as a motion vector that characterizes the motion of the at least one vehicle between the plurality of geographic locations, based on the plurality of geographic location data elements. The system according to any one of claims 20 to 34, further configured to perform the following:

36. The aforementioned at least one processor is A step of receiving a plurality of geographic location data elements representing a plurality of geographic locations of at least one vehicle, wherein each of the plurality of geographic location data elements is assigned a global timestamp corresponding to the time of determination of each geographic location and the time of reception of each geographic location data element, and a reception timestamp. A step of calculating the extrapolated geographic location of the at least one vehicle based on (i) each of the multiple geographic locations, (ii) each global timestamp and (iii) each received timestamp of the multiple geographic location data elements, The steps include: calculating the at least one incoming motion data element as a motion vector based on the extrapolated geographic location; The system according to any one of claims 20 to 35, further configured to perform the following:

Images