SYSTEM AND METHOD FOR MITIGATING THE RISK OF OPERATING A VEHICLE USING A CONTEXTUALIZED RISK FACTOR

Abstract

A method for mitigating the risk of operating a vehicle using a contextualized risk factor involves receiving an initial input from multiple first sensors inside the vehicle and a second input from multiple second sensors outside the vehicle, determining a set of internal vehicle state factors based on the initial input, and weighting the internal vehicle state factors, the second input, and the identity of an in-vehicle user to generate the contextualized risk factor. The user's identity and behavior are recorded in a user profile, which is added to a user profile database. If the contextualized risk factor exceeds a calibrated safety threshold, the vehicle may take an indirect and / or direct action.The vehicle can also send a warning about the presence of an unsafe driving environment to a third party.

Description

INTRODUCTION

[0001] The present disclosure relates generally to mitigating the risk of operating a vehicle, in particular by weighting several factors both inside and outside the vehicle, generating a contextualized risk factor in relation to a driving environment, and enabling the vehicle to take an indirect and / or direct action based on the contextualized risk factor.

[0002] Vehicles employ a variety of systems and techniques to mitigate the risks of operating a vehicle, including the use of sensors that analyze the vehicle's surroundings and automatic emergency braking that can lessen or even prevent a potential accident. However, with the advent of sophisticated emotional, mental, and behavioral analysis, it is now possible to consider not only the environment surrounding the vehicle when employing risk mitigation techniques, but also the behavioral patterns of the driver and passengers within the vehicle.

[0003] Thus, while current risk mitigation systems in vehicles achieve their intended purpose, there is a need for a new and improved system and procedure for mitigating the risk of operating a vehicle using a contextualized risk factor to accurately and effectively weigh multiple factors from inside and outside the vehicle that the driver of a vehicle may encounter during the course of operating the vehicle, in such a way that an appropriate mitigation action can be taken. SUMMARY

[0004] According to several aspects, a method for mitigating the risk of operating a vehicle is provided using context-dependent information related to a vehicle user and an operating state of the vehicle. The method may include receiving an initial input from multiple initial sensors located in the vehicle, where the initial input specifies an internal state of the vehicle. The method may further include determining a user's identity based on the initial input and a user profile in a user profile database. The method may also include determining a set of internal vehicle state factors based on the initial input and the user's identity.The method may further include receiving a second input from multiple secondary sensors mounted on the vehicle, the second input indicating the vehicle's operating state. The method may further include determining a contextualized risk factor based on the vehicle's internal state factors and the second input. The method may further include comparing the contextualized risk factor with a calibrated safety threshold. The method may further include performing at least one action when the contextualized risk factor exceeds the calibrated safety threshold, in order to mitigate the risk of operating the vehicle.

[0005] In an additional aspect of the present disclosure, receiving the first input may further include receiving the first input using at least one microphone, determining that the first input is human speech, generating a transcript of the first input, analyzing the first input using a sentiment analysis program, and analyzing the first input using a speech recognition program.

[0006] In another aspect of the present disclosure, determining the user's identity may further include creating the user's profile based on the analysis of the initial input by the speech recognition program if comparing the initial input with each entry in the user profile database does not result in a match. Determining the user's identity may further include adding the user profile to the user profile database. Determining the user's identity may further include assigning the set of internal vehicle condition factors to the user in the user profile database.

[0007] In an additional aspect of the present disclosure, determining the set of internal vehicle state factors may further include generating an assessment of a user's mental and emotional state based on the recording of the initial input and the analysis of the initial input by the mood analysis program. Determining the set of internal vehicle state factors may further include determining when excessive interference by a third party occurs, based on the recording of the initial input. Determining the set of internal vehicle state factors may further include determining the number of users in the vehicle based on the number of users identified by the speech recognition program.Determining the set of internal vehicle condition factors may further include determining when the user's language is affected, based on the analysis of the initial input by the sentiment analysis program and the analysis of the initial input by the speech recognition program.

[0008] In another aspect of the present disclosure, receiving the first input may further comprise receiving the first input using at least one camera and classifying the user as either a driver or a passenger based on the user's position in the vehicle.

[0009] In an additional aspect of this disclosure, determining the user's identity may further include analyzing the initial input using a facial recognition program. Determining the user's identity may further include creating the user's profile based on the facial recognition program's analysis of the initial input if comparing the initial input with each entry in the user profile database does not result in a match. Determining the user's identity may further include adding the user profile to the user profile database. Determining the user's identity may further include mapping the internal vehicle condition factors and the second input to the user in the user profile database.

[0010] In another aspect of the present disclosure, determining the set of internal vehicle state factors may further include generating an assessment of a user's mental and emotional state based on the tracked activity of the driver. Determining the set of internal vehicle state factors may further include generating an assessment of a user's mental and emotional state based on the tracked activity of a passenger. Determining the set of internal vehicle state factors may further include determining the number of passengers in the vehicle based on the tracked activity of at least one passenger. Determining the set of internal vehicle state factors may further include determining when excessive disturbance occurs based on the tracked activity of the driver.Determining the set of internal vehicle condition factors may further include determining when excessive interference from a third party occurs, based on the tracked activity of at least one passenger.

[0011] In an additional aspect of the present disclosure, receiving the first input may further include using at least one eye-tracking device to receive the first input, determining when the pupils of the user's eyes are in a dilated or non-dilated state, and determining the direction of the user's gaze.

[0012] In another aspect of the present disclosure, determining the set of internal vehicle condition factors may further include generating an assessment of a user's mental and emotional state based on when the user's eyes are in an expanded state, and determining when excessive disturbance occurs based on the direction of the user's gaze.

[0013] In an additional aspect of the present disclosure, the second input may further include a current speed of the vehicle, a current acceleration of the vehicle, a detected speed limit of a roadway on which the vehicle is operated, an ambient temperature of the environment in which the vehicle is operated, a precipitation intensity of the environment in which the vehicle is operated, a time of day of the environment in which the vehicle is operated, and a location of the vehicle.

[0014] In another aspect of the present disclosure, the method may further include receiving a third input from a server, wherein the third input specifies the environment surrounding the vehicle, which includes weather information of the environment in which the vehicle is operated and traffic information of the environment in which the vehicle is operated.

[0015] In an additional aspect of the present disclosure, determining the contextualized risk factor may further include weighting the set of internal vehicle condition factors, the second input and the third input based on the user's identity and generating the contextualized risk factor.

[0016] In another aspect of this disclosure, taking at least one action when the contextualized risk factor is above the calibrated safety threshold may further include taking an indirect action if the contextualized risk factor is more than any first margin above the calibrated safety threshold. Taking at least one action when the contextualized risk factor is above the calibrated safety threshold may further include taking the indirect action if the contextualized risk factor remains relatively larger than the calibrated safety threshold over a period of time.Taking at least one action when the contextualized risk factor exceeds the calibrated safety threshold may further include taking direct action if the contextualized risk factor remains above the calibrated safety threshold by more than any first margin over a period of time. Taking at least one action when the contextualized risk factor exceeds the calibrated safety threshold may further include taking direct action if the contextualized risk factor exceeds the calibrated safety threshold by more than any second margin, where any second margin exceeds any first margin.

[0017] In an additional aspect of the present disclosure, the indirect measure may include playing a relaxing piece of music via a sound system in the vehicle and warning the user of the risky operating condition of the vehicle.

[0018] In another aspect of the present disclosure, the direct measure may also include a temporary restriction of the vehicle's speed and a temporary restriction of the vehicle's acceleration.

[0019] In an additional aspect of the present disclosure, carrying out the direct measure may further include analyzing the second input and determining, based on the second input, that an environment in which the vehicle is operated is safe for limiting the vehicle's agility.

[0020] In another aspect of the present disclosure, performing at least one action when the contextualized risk factor is above the calibrated safety threshold may also include warning a third party regarding the risky operating condition of the vehicle.

[0021] In an additional aspect of the present disclosure, the calibrated safety threshold can be set based on the internal vehicle condition factors, the second input, the third input, and the user's identity.

[0022] In another aspect of the present disclosure, a method for mitigating the risk of operating a vehicle is provided using context-dependent information related to a user of the vehicle and an operating state of the vehicle. The method may include receiving an initial input from several initial sensors arranged in the vehicle, wherein the initial input specifies an internal state of the vehicle. The method may further include determining the identity of a user based on the initial input and a user profile in a user profile database. The method may also include determining a set of internal vehicle state factors of the internal state of the vehicle based on the initial input and the user's identity.The method may further include receiving a second input from multiple secondary sensors mounted on the vehicle, the second input indicating the vehicle's operating state. The method may further include determining a contextualized risk factor based on the vehicle's internal state factors and the second input. The method may further include recording the user's behavior, based on the vehicle's internal state factors and the second input, into multiple historical / heuristic data points associated with the user profile. The method may further include comparing the contextualized risk factor with a calibrated safety threshold. The method may further include performing at least one action if the contextualized risk factor exceeds the calibrated safety threshold, in order to mitigate the risk of operating the vehicle.The procedure may further include determining that the at least one action is an indirect measure if the contextualized risk factor is more than any first margin above the calibrated safety threshold or if the contextualized risk factor is continuously relatively larger than the calibrated safety threshold over a period of time, wherein the indirect measure includes playing a relaxing piece of music through a sound system in the vehicle and warning the user about the risky operating condition of the vehicle.The procedure may further include determining that the at least one action is a direct measure if an environment in which the vehicle is operated is safe to restrict the vehicle's agility and either the contextualized risk factor is continuously greater than any first margin above the calibrated safety threshold over a period of time, or the contextualized risk factor is greater than any second margin above the calibrated safety threshold, where any second margin is greater than any first margin, and the direct measure includes temporarily restricting the vehicle's speed and temporarily restricting the vehicle's acceleration.

[0023] In an additional aspect of the present disclosure, a method for mitigating the risk of operating a vehicle is provided using context-dependent information related to a user of the vehicle and an operating state of the vehicle. The method may include receiving an initial input from several initial sensors arranged in the vehicle, wherein the initial input specifies an internal state of the vehicle. The method may further include determining the identity of a user based on the initial input and a user profile in a user profile database. The method may also include determining a set of internal vehicle state factors of the internal state of the vehicle based on the initial input and the user's identity.The method may further include receiving a second input from multiple secondary sensors mounted on the vehicle, the second input indicating the vehicle's operating state. The method may further include determining a contextualized risk factor based on the vehicle's internal state factors and the second input. The method may further include recording the user's behavior, based on the vehicle's internal state factors and the second input, into multiple historical / heuristic data points associated with the user profile. The method may further include comparing the contextualized risk factor with a calibrated safety threshold. The method may further include performing at least one action if the contextualized risk factor exceeds the calibrated safety threshold, in order to mitigate the risk of operating the vehicle.The procedure may also include warning a third party regarding the existence of an unsafe driving environment for the vehicle if the contextualized risk factor is more than any margin above the calibrated safety threshold.

[0024] Further areas of application will become apparent from the description provided here. It should be understood that the description and specific examples serve only for illustration and are not intended to limit the scope of this disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings described here serve only for illustration and are not intended to limit the scope of the present disclosure in any way; they show: Fig. 1 a schematic diagram of a system for mitigating the risk of operating a vehicle using context-dependent information related to a user of the vehicle and an operating state of the vehicle, according to an exemplary embodiment; Fig. 2 a diagram of weighting a set of internal vehicle condition factors, a second input and a third input to determine a contextualized risk factor, according to an exemplary embodiment; Fig. 3A a diagram for determining the set of internal vehicle condition factors based on data received from multiple microphones, according to an exemplary embodiment; Fig. 3B a diagram for determining the set of internal vehicle condition factors based on data received from multiple cameras, according to an exemplary embodiment; Fig. 3C a diagram for determining the set of internal vehicle condition factors based on data received from an eye-tracking device, according to an exemplary embodiment; Fig. 4 a diagram of a method for determining which of the following measures the vehicle will take, either a direct or an indirect measure, based on the contextualized risk factor, according to an exemplary embodiment; Fig. 5. A flowchart of a procedure for determining when a warning message should be provided to a third party, based on the contextualized risk factor according to an exemplary embodiment; and Fig. 6. A flowchart of a procedure for adding a user profile to a user profile database according to an exemplary embodiment. DETAILED DESCRIPTION

[0026] The following description is merely exemplary and is not intended to limit the present disclosure, application or uses.

[0027] With reference to Fig. Figure 1 is a schematic diagram of a system for mitigating the risk of operating a vehicle using context-dependent information related to a vehicle user and an operating state of the vehicle, generally indicated by reference numeral 10. The system 10 generally includes a vehicle 12, a user 14, and a server 16.

[0028] Vehicle 12 is a land vehicle, such as a passenger car, truck, etc., which can be operated by user 14 or by an autonomous driving module. Vehicle 12 can exhibit various levels of driving automation, including Level 5, Level 4, Level 3, and Level 2 automation. For example, a Level 5 system indicates "full automation," which refers to the full-time capability of an automated driving system to handle aspects of the dynamic driving task under multiple road and environmental conditions that can be managed by a human driver. A Level 4 system indicates "high automation," which refers to the driving-mode-specific capability of an automated driving system to handle aspects of the dynamic driving task, even if a human driver does not respond appropriately to a request to intervene.In Level 3 vehicles, the vehicle systems perform the entire Dynamic Driving Task (DDT) within their designated operating zone. The vehicle operator is only expected to be responsible for the DDT evasive maneuver when the vehicle essentially "requests" the driver to take over if something goes wrong or the vehicle is about to leave the operating zone. In Level 2 vehicles, systems provide steering, braking / acceleration assistance, lane centering, and adaptive cruise control. However, even when these systems are activated, the vehicle operator must remain at the steering wheel and continuously monitor the automated features. The vehicle may contain various actuator devices (not shown) used to implement the levels of automation described above.The actuator devices control one or more vehicle features, which include, but are not limited to, a propulsion system, a transmission system, a steering system, and a braking system (not shown). In various embodiments, the vehicle features may further include interior and / or exterior vehicle features such as, but not limited to, doors, a trunk, and passenger compartment features such as air conditioning, music, lighting, etc. In the particular example shown in . Fig. 1 is provided, the vehicle 12 comprises a controller 18, a display device 20, several first sensors 22 and several second sensors 24.

[0029] The controller 18 is a non-generalised electronic control device comprising a pre-programmed digital computer or processor 26, a memory 28, a transceiver 30, and multiple input and output ports 32. The processor 26 can be a custom-made or commercially available processor, a central processing unit (CPU), a graphics processing unit (GPU), an auxiliary processor among several processors associated with the controller 18, a semiconductor-based microprocessor (in the form of a microchip or chipset), a microprocessor, a combination thereof, or generally, an instruction-executing device. The memory 28 is used to store data such as control logic, software applications, instructions, computer code, data, lookup tables, etc. The memory 28 contains any type of medium accessible by a computer, such as...A read-only memory (ROM), a read / write memory (RAM), a hard disk drive, a compact data carrier (CD), a digital video data carrier (DVD), or any other type of storage. A "non-transient" computer-readable medium excludes wired, wireless, optical, or other communication links that carry transient, electrical, or other signals. A non-transient computer-readable medium includes media on which data can be permanently stored and media on which data can be stored and later overwritten, such as a rewritable optical disc or an erasable storage device. Computer code includes any type of program code, which includes source code, object code, and executable code. The processor 26 is configured to execute the code or instructions.

[0030] The transceiver 30 is configured to communicate wirelessly with a hotspot using Wi-Fi protocols under IEEE 802.11x standards. The transceiver 30 is also configured to communicate wirelessly using cellular data communication under GSMA standards such as SGP.02, SGP.22, SGP.32, and the like. Accordingly, the vehicle 12 may further contain an embedded universal integrated circuit card (eUICC) configured to store at least one cellular connectivity configuration profile, such as an embedded subscriber identity module (eSIM) profile. The transceiver 30 is also configured to communicate via a personal network (e.g., Bluetooth), near-field communication (NFC), and / or any additional type of radio frequency communication.

[0031] The multiple input / output ports 32 receive incoming data from the multiple first sensors 22 and the multiple second sensors 24 and communicate the incoming data to the processor 26. The multiple input / output ports 32 also receive outgoing data from the processor 26 and communicate the outgoing data to the multiple first sensors 22 and the multiple second sensors 24. The multiple input / output ports 32 are configured to communicate wirelessly with the multiple first sensors 22 and the multiple second sensors 24 via the transmit / receive device 30, and are also configured to communicate with the multiple first sensors 22 and the multiple second sensors 24 via a wired connection of a universal serial bus (USB connection).

[0032] The controller 18 can also contain one or more applications. An application is a software program configured to perform a specific function or set of functions. Applications can include one or more computer programs, software components, sets of instructions, procedures, functions, objects, classes, instances, related data, or a portion thereof adapted for implementation in suitable machine-readable program code. Applications can be stored in memory 28 or in additional or separate memory.

[0033] The controller 18 communicates electrically with the multiple first sensors 22 and the multiple second sensors 24. In one exemplary embodiment, the electrical communication is established using, for example, a CAN network, a FLEXRAY network, a local area network (e.g., Wi-Fi, Ethernet, and the like), a serial peripheral interface network (SPI network), or the like. It is understood that various additional wired and wireless techniques and communication protocols for communicating with the controller 18 are within the scope of this disclosure.

[0034] The display device 20 is a screen located in the vehicle 12, which has a human-machine interface. It displays data received by the multiple first sensors 22 and the multiple second sensors 24, and allows the user 14 to configure the vehicle 12 and run the applications contained in the controller 18. The display device 20 is an optional feature, meaning that it is not required for the proper use or functionality of the vehicle 12, the multiple first sensors 22, the multiple second sensors 24, or any other parts of the system 10.

[0035] The multiple first sensors 22 are arranged in the vehicle 12 and are used to acquire a first input. The first input comprises information relevant to an internal driving environment in the vehicle 12. In an exemplary embodiment, the multiple first sensors 22 include multiple microphones 36, multiple cameras 38, and an eye-tracking device 40.

[0036] The multiple microphones 36 are arranged throughout the vehicle 12 and receive audio data from inside the vehicle 12. The audio data is then communicated to the processor 26 for analysis and to determine the presence of the internal driving environment in the vehicle 12. It should be noted that only one microphone 36 can be used without deviating from the scope of this disclosure.

[0037] The multiple cameras 38 are arranged throughout the vehicle 12 and receive optical data from inside the vehicle 12. This optical data is then communicated to the processor 26 for analysis and to determine the presence of the internal driving environment in the vehicle 12. It should be noted that only one camera 38 can be used without deviating from the scope of this disclosure.

[0038] The eye-tracking device 40 is located in front of a driver's seat (not shown) of the vehicle 12 and is positioned in such a way that the eye-tracking device 40 can clearly see the eyes of the user 14, who is seated in the driver's seat of the vehicle 12. The eye-tracking device 40 receives eye-tracking data from the user 14, who is seated in the driver's seat of the vehicle 12. The eye-tracking data is then communicated to the processor 26 for analysis and to determine the presence of the internal driving environment in the vehicle 12.

[0039] The multiple secondary sensors 24 are mounted on the vehicle 12 and are used to acquire a secondary input 42. The secondary input 42 contains information relevant to an external driving environment surrounding the vehicle 12. In an exemplary environment, the multiple secondary sensors 24 include a speedometer 44, an accelerometer 46, a thermometer 48, a global navigation satellite system (GNSS) 50, a clock 52, a speed limit sensor 54, and a precipitation intensity sensor 56.

[0040] The speedometer 44 is used to provide data indicating the current speed of the vehicle 12. In non-restrictive examples, the speedometer 44 can be a mechanical speedometer that uses a magnetic field to induce the rotation of a speedometer to determine the speed of the vehicle 12, or an electronic speedometer that uses pulse generation to determine the speed of the vehicle 12.

[0041] The current speed of vehicle 12 is then communicated to processor 26 to be analyzed and to determine the current state of the external driving environment surrounding vehicle 12.

[0042] The accelerometer 46 is used to provide data indicating the current acceleration of the vehicle 12. In non-restrictive examples, the accelerometer 46 can be a piezoelectric accelerometer, a piezoresistive accelerometer, or a capacitive accelerometer. The current acceleration of the vehicle 12 is then communicated to the processor 26 for analysis and to determine the current state of the external driving environment surrounding the vehicle 12.

[0043] The thermometer 48 is used to provide data indicating the current temperature of the environment surrounding the vehicle 12. In non-restrictive examples, the thermometer 48 can be a liquid-in-glass thermometer, a bimetallic strip thermometer, an electronic thermometer, or an infrared thermometer. The current temperature is then communicated to the processor 26 for analysis to determine the current state of the external driving environment surrounding the vehicle 12.

[0044] The GNSS 50 is used to determine the geographic location of the vehicle 12. In an exemplary embodiment, the GNSS 50 is a global positioning system (GPS). In a non-limiting example, the GPS includes a GPS receiver antenna (not shown) and a GPS control unit (not shown) in electrical communication with the GPS receiver antenna. The GPS receiver antenna receives signals from multiple satellites, and the GPS control unit calculates the geographic location of the vehicle 12 based on the signals received by the GPS receiver antenna. In an exemplary embodiment, the GNSS 50 additionally includes a map. The map contains information about infrastructure such as municipal boundaries, roads, railways, sidewalks, buildings, and the like. Therefore, the geographic location of the vehicle 12 is contextualized using the map information.In one non-restrictive example, the map is retrieved from a remote source using a wireless connection. In another non-restrictive example, the map is stored in a database of the GNSS 50. It is understood that various additional types of satellite-based radio navigation systems, such as Galileo, Globalnaya Navigazionnaya Sputnikvaya Sistema (GLONASS), and the BeiDou Navigation Satellite System (BDS), are within the scope of this disclosure. It is also understood that the GNSS 50 can be integrated with the controller 18 (e.g., on the same circuit board as the controller 18 or otherwise forming a section of the controller 18) without exceeding the scope of this disclosure. The geographic location of the vehicle 12 is then communicated to the processor 26 for analysis and to determine the current state of the external driving environment surrounding the vehicle 12.

[0045] Clock 52 is used to provide data indicating the current time of day in the environment surrounding vehicle 12. This current time is then communicated to processor 26 for analysis to determine the current state of the external driving environment surrounding vehicle 12.

[0046] The sensor 54 for detected speed limits is used to provide data on a posted legal speed limit for a lane on which the vehicle 12 is currently traveling. In an exemplary non-restrictive embodiment, data on the posted legal speed limit is received by the GNSS 50. The posted legal speed limit is then communicated to the processor 26 for analysis and to determine the current state of the external driving environment surrounding the vehicle 12.

[0047] The precipitation intensity sensor 56 is used to provide data indicating the presence, type, and amount of precipitation (precipitation data) in the environment surrounding the vehicle 12. In an exemplary non-limiting embodiment, the precipitation intensity sensor 56 uses infrared light to determine the amount of precipitation. It can be determined that less precipitation is present in the environment surrounding the vehicle 12 when a large amount of the emitted infrared light is reflected back into the precipitation intensity sensor 56. Similarly, it can be determined that more precipitation is present in the environment surrounding the vehicle 12 when a small amount of the emitted infrared light is reflected back into the precipitation intensity sensor 56, because the precipitation scatters the infrared light in several different directions.Non-restrictive examples of precipitation include rain and snow. The precipitation data is then communicated to processor 26 for analysis to determine the current state of the external driving environment surrounding vehicle 12.

[0048] User 14 is a person who is in vehicle 12 while vehicle 12 is in operation. User 14 can be a driver of vehicle 12 or a passenger in vehicle 12. User 14 is identified as a driver if they are in the driver's seat of vehicle 12. User 14 is identified as a passenger if they are not in the driver's seat of vehicle 12. Vehicle 12 determines the identity of user 14 using the multiple first sensors 22 and records the identity as a user profile in a user profile database 58 in memory 28.

[0049] Server 16 can be a computing device (which, for example, contains one or more controllers, where controllers contain one or more processors and one or more memories programmed to perform operations and execute instructions). Furthermore, Server 16 can be accessed via a network (e.g., the Internet or any other large-scale network). Using the transmit / receive device 30, the server can communicate a third input 60 to the vehicle 12. The third input 60 contains further information relevant to the external driving environment surrounding the vehicle 12, as described in more detail below.

[0050] With reference to Fig. Figure 2 shows a diagram of the weighting of a set of internal vehicle condition factors 62, the second input 42, and the third input 60 to determine a contextualized risk factor 64. The set of internal vehicle condition factors 62, the second input 42, and the third input 60 are provided to a weighting algorithm 66, which weights the internal vehicle condition factors 62, the second input 42, and the third input 60 with respect to the identity of the user 14 to generate the contextualized risk factor 64.

[0051] The internal vehicle condition factors 62 use the data received in the initial input, after it has been analyzed by the processor 26, as a basis and indicate the internal driving environment in the vehicle. The internal vehicle condition factors 62 include the following: an assessment 68 of the user's mental and emotional state, a factor 70 of excessive disturbance, a factor 72 of the user's affected speech, and a factor 74 of the number of passengers.

[0052] With reference to Fig. Figure 3A shows a diagram for determining the set of internal vehicle condition factors 62 based on data received by the multiple microphones 36. The initial input can include three data types: audio data, optical data, and eye-tracking data. The multiple microphones 36 receive data, including the audio data, which is then communicated to and analyzed by the processor 26. The analysis of the audio data performed by the processor 26 includes a speech recognition program 76 of the user 14 and a mood analysis program 78 of the user 14, as well as the generation of a speech transcript 80 of the audio data. The eye-tracking device 40 receives data, including the eye-tracking data, which is then communicated to and analyzed by the processor 26. It should be noted that the analysis of the audio data is performed on each user 14 in the vehicle 12, regardless of the number of users in the vehicle 12.

[0053] The speech recognition program 76 is a program executed by the processor 26 that enables the vehicle 12 to determine the identity of the user 14 based on the unique language of the user 14. The speech recognition program 76 can also be used to provide a reference for the language of the user 14 against which it can be measured when the language of the user 14 is being manipulated. In an exemplary non-restrictive embodiment, the speech recognition program 76 can be implemented using machine learning models, deep learning models, and / or other techniques that employ feature extraction (e.g., Mel frequency cepstral coefficients (MFCCs), linear predictive coding (LPC), etc.), acoustic modeling (e.g., hidden Markov models (HMMs), deep neural networks (DNNs), etc.), or speaker embedding (e.g., i-vectors, x-vectors, etc.).) and sequence-to-sequence models. In one example, if two users, a driver and a passenger, are present in vehicle 12, the speech recognition program 76 enables the vehicle to distinguish between the driver user and the passenger user based on the differences between the unique speech features and characteristics of the driver user and the passenger user, which include pitch, timbre, formant frequencies, speech rate, intensity, speech onset time (VOT), harmonic / noise ratio (HNR), sentence rhythm, and more.

[0054] The mood analysis program 78 of user 14 is a program executed by the processor 26 that enables the vehicle 12 to determine the mood of user 14 using the user's speech. The mood analysis program 78 can also be used to provide an emotional reference state of user 14 against which changes in the user's emotional state can be measured. In an exemplary non-restrictive embodiment, the mood analysis program 78 can be performed using lexicon-based methods, machine learning models, or deep learning models. In conjunction with the speech recognition program 76, the mood analysis program 78 then records the user's speech and assigns it to the corresponding user profile of user 14 in the user profile database 58.In one example, if a user is present in vehicle 12, the mood analysis program 78 can determine the emotional tone behind the driver-user's speech and words, which includes whether the driver-user is happy, sad, angry, anxious, surprised, worried, tense, etc.

[0055] The speech transcript 80 is a program that uses the audio data received by the multiple microphones 36 and converts the audio data into words, which are recorded in a file that can be read later. In an exemplary non-limiting embodiment, the speech transcript 80 can be generated by an automated speech recognition algorithm (ASR), machine learning models, or deep learning models. In conjunction with the speech recognition program 76, the speech transcript is assigned to the corresponding user profile of the user 14 in the user profile database 58.

[0056] With reference to Fig. Figure 3B shows a diagram for determining the set of internal vehicle state factors 62 based on data received from the multiple cameras 38. The multiple cameras 38 receive data, including optical data, which is then communicated to and analyzed by the processor 26. The optical data analysis performed by the processor 26 includes a driver activity tracking device 82 and a passenger activity tracking device 84. It should be noted that the optical data analysis is performed on each user 14 in the vehicle 12, regardless of the number of users in the vehicle 12.

[0057] The driver activity tracking device 82 is a program that uses the optical data received by the multiple cameras 38 to monitor the behavior of a driver user. The driver user's behavior is then recorded and assigned to the corresponding driver user profile in the user profile database 58. For example, if a driver user and a passenger user are in the vehicle 12, and the driver user frequently takes their hands off the steering wheel of the vehicle 12 to make gestures while speaking to the passenger user, the driver user's behavior is recorded and assigned to a driver user profile in the user profile database 58.

[0058] The passenger activity tracking device 84 is a program that uses the optical data received by the multiple cameras 38 to monitor the behavior of a passenger user. The passenger user's behavior is then recorded and assigned to the corresponding passenger user profile in the user profile database 58. In an example where a driver user and a passenger user are in the vehicle 12, and the passenger user frequently touches the driver user while the driver user is operating the vehicle 12, the passenger user's behavior is recorded and assigned to the passenger user's profile in the user profile database 58.

[0059] With reference to Fig. Figure 3C shows a diagram for determining the set of internal vehicle condition factors 62 based on data received from the eye-tracking device 40. The analysis of the eye-tracking data, performed by the processor 26, includes a pupil dilation detector 86 and a gaze-tracking device 88. It should also be noted that the analysis of the eye-tracking data is only performed on the user 14 if the user 14 is identified as a driver.

[0060] The Pupil Dilation Detector 86 is a program that uses the eye-tracking data received by the eye-tracking device 40 and detects the emotional state of User 14 based on the size of User 14's pupils. For example, dilated User 14 pupils may indicate that User 14 is excited or has a heightened sense of attention. Conversely, constricted User 14 pupils may indicate that User 14 is calm, bored, or tired. User 14's emotional state is then recorded and assigned to the corresponding User 14 profile in the User Profile Database 58.

[0061] The eye-tracking device 88 is a program that uses the eye-tracking data received by the eye-tracking device 40 and detects the gaze path of user 14. This can help determine whether user 14, who is a driver, is keeping their attention on a lane on which vehicle 12 is traveling while user 14 is operating vehicle 12. In one example, where user 14 frequently keeps their eyes on the lane, this behavior is recorded and assigned to the corresponding user profile of user 14 in the user profile database 58. In another example, where user 14 frequently takes their eyes off the lane, this behavior is recorded and assigned to the corresponding user profile of user 14 in the user profile database 58.

[0062] With reference to Fig. 3A, Fig. 3B and Fig. 3C provides the assessment of the user's mental and emotional state 68, data regarding the user's mental and emotional state 14, such as excitement, partner pressure, rudeness, and other emotional states. The assessment of the user's mental and emotional state 68 is determined from the mood analysis program 78, the speech transcript 80, the driver activity tracking device 82, the passenger activity tracking device 84, and the pupil dilation detector 86.

[0063] A factor of 70 for excessive interference detects the presence of interference within the vehicle 12 that can increase the potential for an internally hazardous driving environment. In a non-restrictive example, a factor of 70 for excessive interference could be the presence of a rear-seat passenger in the vehicle 12, where a passenger user provides an unsolicited recommendation to a driver user regarding how the vehicle 12 should be operated. In another non-restrictive example, a factor of 70 for excessive interference could be frequent use of a device (e.g., a mobile phone, tablet, smartwatch, etc.) by a driver user while the vehicle 12 is in operation. In further non-restrictive examples, a factor of 70 for excessive interference could be loud noises originating from inside or outside the vehicle 12, a driver user frequently using controls within the vehicle 12 (e.g.,A driver-user who adjusts a radio, climate control system, navigation system, etc., reaches for items (e.g., a mobile phone, handbag, etc.) in another section of the vehicle while operating the vehicle, consumes food or beverages while operating the vehicle, engages in personal grooming (e.g., combing hair, applying makeup, etc.) while operating the vehicle, and exhibits distracting behavior towards a driver-user (e.g., speaking loudly, making exaggerated gestures towards the driver-user, etc.). The excessive disturbance factor is determined from the voice recorder, driver activity tracking device, passenger activity tracking device, and eye-tracking device.

[0064] The User's Influenced Speech Factor 72 detects whether User 14's speech has been influenced to such a degree that it deviates significantly from the reference assigned to User 14 by Speech Recognition Program 76 and / or the emotional reference state assigned to User 14 by Sensation Analysis Program 78. In a non-restrictive example, influenced speech might mean that User 14's speech is slurred compared to the reference. In other non-restrictive examples, influenced speech might mean that User 14's speech is significantly slower or significantly faster than the reference. The User's Influenced Speech Factor 72 is determined from Speech Recognition Program 76 and Sensation Analysis Program 78.

[0065] The factor 74 of the number of passengers detects the number of passenger users in the vehicle 12. The factor 74 of the number of passengers is determined from the speech recognition program 76 and the passenger activity tracking device 84.

[0066] Returning to Fig. 2 is the second input 42, the data received by the multiple second sensors 24. The second input 42 contains the current speed 104 of the vehicle, the current acceleration 106 of the vehicle, the current temperature 108 of the environment surrounding the vehicle, the geographical location 110 of the vehicle, the current time of day 112 of the environment surrounding the vehicle, the posted legal speed limit 114 of the roadway on which the vehicle is traveling, and the precipitation data 116 in relation to the environment surrounding the vehicle.

[0067] The third input 60 is data received from multiple third-party sources via server 16. The third input 60 contains weather condition data 90 and traffic condition data 92.

[0068] Weather condition data 90 is data regarding the current weather conditions of the environment surrounding vehicle 12, including forecasts of the start time and duration of sunshine, clouds, rain, snow, hail, thunderstorms, blizzards, tornadoes, hurricanes, flooding, fog, etc. Weather condition data 90 is provided by a weather service via server 16.

[0069] The traffic condition data 92 is data concerning the current traffic conditions of the environment surrounding vehicle 12, including a non-restrictive list containing the number of vehicles traveling on a roadway over a given period, the speed of the vehicles traveling on the roadway over that period, road closures, lane closures, the presence of roadworks on the roadway, the presence of another vehicle on the shoulder of the roadway, and the presence of a vehicle accident on the roadway. The traffic condition data 92 is provided by a traffic service via server 16.

[0070] With reference to Fig. Figure 4 is a flowchart of a procedure for determining which at least one of several indirect actions or at least one of several direct actions the vehicle 12 will take, based on the contextualized risk factor 64, generally specified by reference numeral 200. In this example, it is assumed that the user 14 is a driver operating the vehicle 12. The procedure 200 begins in step 202 by determining the contextualized risk factor 64 based on the internal vehicle condition factors 62, the second input 42, and the third input 60, with respect to several historical / heuristic data points 98 associated with the user profile of user 14.

[0071] The contextualized risk factor 64 is a dynamic value determined by weighting the internal vehicle state factors 62, the second input 42, and the third input 60 with respect to the identity of the users in the vehicle 12 using the weighting algorithm 66. When the internal vehicle state factors 62, the second input 42, and the third input 60 are received, the processor 26 detects recurring trends and correlations between the user 14, the emotional state of the user 14, and the driving behavior of the user 14, which are recorded in the multiple historical / heuristic data points 98 associated with the user profile of the user 14. The processor 26 then determines and correlates the internal vehicle state factors 62, the second input 42, and the third input 60 and their respective thresholds that lead to the creation of an unsafe driving environment.The respective thresholds of the internal vehicle condition factors 62, the second input 42 and the third input 60 can then be checked using subsequent unsafe driving environments to verify and declare the respective thresholds valid in order to ensure accuracy in their ability to detect the unsafe driving environment.

[0072] In step 204, the procedure 200 records the behavior of user 14 into the multiple historical / heuristic data points 98 associated with user 14's user profile. These multiple historical / heuristic data points 98 are used to refine and adjust thresholds for determining whether any of the internal vehicle state factors 62, the second input 42, and the third input 60 contribute to the presence of the unsafe driving environment, enabling more accurate and effective identification of behaviors contributing to the unsafe driving environment. The multiple historical / heuristic data points 98 can also be used to develop and improve mitigation strategies to reduce the presence of the unsafe driving environment, which are discussed further below. If there is more than one user in vehicle 12, step 204 is repeated for each user in vehicle 12.

[0073] In an example where a driver user and a passenger user are in vehicle 12, and the passenger user exhibits distracting behavior (i.e., touching the driver user) while the driver user is operating vehicle 12, the behavior of the driver user and the passenger user 14 is observed by the multiple secondary sensors 24 and assigned to the corresponding user profiles of the driver user and the passenger user. If the driver user's ability to operate the vehicle remains unaffected by the passenger user's behavior (e.g., despite the passenger user's behavior, the driver user keeps both hands on the steering wheel of vehicle 12, the driver user's gaze remains on the road, the speech recognition program 76 does not detect that the driver user is exhibiting affected speech, etc.), the system will then process the information.The internal vehicle condition factors 62 will not receive a significant weight when the contextualized risk factor 64 is determined. The driver-user's behavior is also recorded in the multiple historical / heuristic data points 98 associated with the driver-user's user profile. This means that if a future driving scenario occurs in which the same driver-user and the same passenger-user exhibit similar behavior, the internal vehicle condition factors 62 will be given a lower weight when the contextualized risk factor 64 is determined. This means that if the passenger-user's behavior is repeated in the future and the driver-user's behavior remains the same, the driver-user's behavior will not be considered risky for the vehicle's driving environment 12.

[0074] Conversely, in another example where a driver user and a passenger user are in the vehicle 12, and the passenger user frequently touches the driver user while the driver user is operating the vehicle 12, then, if the driver user's ability to operate the vehicle is affected by the passenger user's behavior (e.g., the driver user takes their hands off the steering wheel of the vehicle 12 because of the passenger user's behavior, the driver user's gaze is not on the road, the speech recognition program 76 detects that the driver user is using affected speech, etc.), the internal vehicle state factors 62 are given significant weight when the contextualized risk factor 64 is determined. The driver user's behavior is also recorded in the several historical / heuristic data points 98 that are assigned to the driver user's user profile.This means that if a future driving scenario occurs in which the same driver-user and the same passenger-user exhibit similar behavior, the internal vehicle state factors 62 will be given more weight when the contextualized risk factor 64 is determined. Furthermore, if the passenger-user's behavior is repeated in the future and the driver-user's behavior remains the same, the driver-user's behavior may still be determined to be risky to the vehicle's driving environment 12. The driver-user's behavior may be determined to be increasingly risky after each subsequent similar driving situation if the driver-user's ability to operate the vehicle is more strongly affected by the passenger-user's behavior (e.g.,(due to the behavior of the passenger user, the driver user takes their hands off the steering wheel of the vehicle 12 for a significant period of time, the driver user's gaze is not on the road for a significant period of time, the speech recognition program 76 detects that the driver user is exhibiting impaired speech for a significant period of time, etc.). This means that, due to each subsequent similar driving situation, the internal vehicle condition factors 62 are given significantly more weight when the contextualized risk factor 64 is determined.

[0075] It should be noted that the internal vehicle state factors 62 are also weighted based on the identity of the passenger user. In an example where a driver user, a first passenger user, and a second passenger user are present, if the driver user's ability to operate remains unaffected by the distracting behavior of the first passenger user, but the driver user's ability to operate is affected by the distracting behavior of the second passenger user, then the internal vehicle state factors 62 are given a greater weight when the second passenger user exhibits distracting behavior, while the internal vehicle state factors 62 are given a lesser weight when the first passenger user exhibits distracting behavior. This applies if the first passenger user and the second passenger user exhibit distracting behavior at different times, even though the distracting behavior is the same.The behavior of the first passenger user and the second passenger is also recorded in the multiple historical / heuristic data points 98 that are assigned to the user profile of the first passenger and the user profile of the second passenger, respectively, which means that the internal vehicle state factors 62 can then be dynamically weighted on the basis of the past behavior of the driver user, the first passenger user and the second passenger user when a future driving situation occurs.

[0076] Furthermore, it should be noted that, regardless of the driving situation, the internal vehicle state factors 62, the second input 42, and the third input 60 are recorded in the multiple historical / heuristic data points 98 for each respective user in the vehicle 12. This means that when a future driving situation occurs, the internal vehicle state factors 62, the second input 42, and the third input 60 can be dynamically weighted based on the past behavior of each respective user in the vehicle 12. The procedure 200 then proceeds to step 206.

[0077] In step 206, procedure 200 compares the contextualized risk factor 64 with a calibrated safety threshold. If the contextualized risk factor 64 is more than any first margin above the calibrated safety threshold, procedure 200 then proceeds to step 208.

[0078] The calibrated safety threshold is a dynamic value against which the contextualized risk factor 64 is compared to determine whether the vehicle 12 must take one of several indirect or one of several direct actions based on the existence of the unsafe driving environment. The calibrated safety threshold is determined based on the respective thresholds of the internal vehicle condition factors 62, the second input 42, and the third input 60 with respect to the several historical / heuristic data points 98 in the user profile of the user 14.

[0079] In step 208, procedure 200 causes vehicle 12 to perform at least one of the several indirect actions. After vehicle 12 has performed at least one of the several indirect actions, if the contextualized risk factor 64 remains above the calibrated safety threshold by more than any first margin over a period of time, procedure 200 can repeat step 208. If the contextualized risk factor 64 falls below the calibrated safety threshold, procedure 200 restarts in step 202.

[0080] Returning to step 208, if the contextualized risk factor 64 continues to exceed the calibrated safety threshold by more than any first margin over a period of time, or if it exceeds the calibrated safety threshold by more than any second margin at any given time, procedure 200 will proceed to step 210. It should be noted that any second margin is greater than any first margin.

[0081] The multiple indirect measures are several possible actions that the vehicle 12 can take after the contextualized risk factor 64 exceeds the calibrated safety threshold by more than any first margin, either to reduce the risk of operating the vehicle 12 in the unsafe driving environment or to reduce the presence of the unsafe driving environment. In an exemplary non-restrictive embodiment, the multiple indirect measures include providing safety alerts and warnings to the user 14 through the systems in the vehicle 12 (e.g., at the display device 20, via a plurality of loudspeakers, etc.) about which of the internal vehicle state factors 62, the second input 42, and the third input 60 contribute to the detection of the unsafe driving environment and how, if possible, the user 14 can reduce the risk of operating the vehicle 12.In another exemplary non-restrictive embodiment, the multiple indirect measures include playing calming and relaxing sounds or music through the multiple loudspeakers in the vehicle 12 when the assessment of the user's mental and emotional state 68 by the user 14 detects emotions shown that may contribute to an unsafe driving environment (e.g. anger, tension, fear, etc.).

[0082] In step 210, the procedure 200 uses the multiple first sensors 22, the multiple second sensors 24, and the third input 60 to monitor the environment surrounding the vehicle 12. The procedure 200 then proceeds to step 212.

[0083] In step 212, based on the second input 42 and the third input 60, vehicle 12 determines whether the environment surrounding vehicle 12 is suitable for limiting its agility without exacerbating the unsafe nature of the driving environment. For example, if vehicle 12 is traveling on a straight road without adverse driving conditions (e.g., rough weather, traffic, roadworks, etc.), vehicle 12 will determine that the environment surrounding vehicle 12 is suitable for limiting its agility. Vehicle 12's agility is its ability to accelerate and maneuver within an environment.In another example, if vehicle 12 is driving on a winding road and the vehicle detects negative driving conditions, vehicle 12 will not restrict the agility of vehicle 12 despite the contextualized risk factor 64 until vehicle 12 determines that taking at least one of the several direct measures will not increase the nature of the unsafe driving environment created by the negative driving conditions.

[0084] The multiple direct measures are several possible actions that the vehicle 12 can take after the contextualized risk factor 64 is more than any second margin above the calibrated safety threshold, or if the contextualized risk factor 64 remains more than any first margin above the calibrated safety threshold after the vehicle has taken at least one of the multiple indirect measures to reduce the risk of operating the vehicle 12 in the unsafe driving environment or to reduce the presence of the unsafe driving environment. In an exemplary non-restrictive embodiment, the multiple direct measures include temporarily limiting the maximum possible speed of the vehicle 12 for a period of time.In another exemplary non-restrictive embodiment, the multiple direct measures include temporarily limiting the possible maximum acceleration of the vehicle 12 for a period of time. If the vehicle 12 determines that the environment surrounding it is suitable to limit its agility, the procedure 200 can proceed to step 214.

[0085] In step 214, procedure 200 temporarily restricts the maximum possible speed of vehicle 12 for a period of time until the contextualized risk factor 64 falls below the calibrated safety threshold. Procedure 200 then restarts in step 202.

[0086] Returning to step 212, if the vehicle 12 determines that the environment surrounding the vehicle 12 is suitable to limit the agility of the vehicle 12, the procedure 200 can proceed to step 216.

[0087] In step 216, procedure 200 temporarily limits the maximum possible acceleration of vehicle 12 for a period of time until the contextualized risk factor 64 falls below the calibrated safety threshold. Procedure 200 then restarts in step 202.

[0088] Returning to step 206, if the contextualized risk factor 64 is more than any second margin above the calibrated safety threshold, procedure 200 proceeds to step 210.

[0089] With reference to Fig. Section 5 is a flowchart of a procedure for determining when to provide an alert to a third party, based on the contextualized risk factor generally indicated by reference numeral 300. In this example, user 14 is assumed to be a driver operating vehicle 12. It should also be acknowledged that, while in this example vehicle 12 performs procedure 300, procedure 300 can be initiated by any device wearable by user 14, including a watch, bracelet, locket, etc.

[0090] Procedure 300 begins in step 302 by determining the contextualized risk factor 64 based on the internal vehicle condition factors 62, the second input 42, and the third input 60 with respect to the multiple historical / heuristic data points 98 associated with the user profile of user 14. If there is more than one user in vehicle 12, step 302 is repeated for each user in vehicle 12. Procedure 300 then proceeds to step 304.

[0091] In step 304, procedure 300 records the behavior of user 14 in the multiple historical / heuristic data points 98 that are assigned to user 14's user profile. If there is more than one user in vehicle 12, step 304 is repeated for each user in vehicle 12. Procedure 300 then proceeds to step 306.

[0092] In step 306, procedure 300 generates a contextualized driving behavior report of user 14 and uses it in conjunction with a third party. The contextualized driving behavior report contains the multiple historical / heuristic data points 98 associated with user 14's user profile. The third party is a person or group that either has a relationship with user 14 (e.g., a parent, guardian, administrator, or employer of user 14) or an interest in the maintenance of vehicle 12 (i.e., an owner of vehicle 12). Procedure 300 then proceeds to step 308.

[0093] In step 308, procedure 300 compares the contextualized risk factor 64 with the calibrated safety threshold to determine whether a third party should be warned of the presence of the unsafe driving environment. If the contextualized risk factor 64 is no more than one-two margin above the calibrated safety threshold, the third party is not warned of the presence of the unsafe driving environment, and procedure 300 returns to step 302. If the contextualized risk factor 64 is more than one-two margin above the calibrated safety threshold, procedure 300 then proceeds to step 310.

[0094] In step 310, procedure 300 warns the third instance of the presence of an unsafe driving environment. Procedure 300 then restarts in step 302.

[0095] Returning to step 308, if the contextualized risk factor 64 is no more than any second margin above the calibrated safety threshold, procedure 300 restarts in step 302.

[0096] With reference to Fig. Section 6 is a flowchart of a method for adding a user profile to a user profile database, generally specified by reference numeral 400. The method 400 begins in step 402 by receiving identity data from the first multiple sensors 22, which enable the vehicle 12 to determine the identity of the user 14. In one exemplary non-limiting embodiment, the identity data includes data received by the speech recognition program 76. In another exemplary non-limiting embodiment, the identity data includes data received by a face recognition system 102.

[0097] The facial recognition 102 is a program executed by the processor 26 that enables the vehicle 12 to determine the identity of the user 14 based on the user 14's unique face. In an exemplary non-restrictive embodiment, the facial recognition 102 can be performed using machine learning models, deep learning models, or other techniques including convolutional neural networks (CNNs), DeepFace, OpenFace, FaceNet, and DLib.In one example, if two users, a driver and a passenger, are present in the vehicle 12, the facial recognition 102 enables the vehicle to distinguish between the driver user and the passenger user based on the differences between their unique facial features and characteristics, which include facial shape, nose shape, nose position, eye shape, eye color, distance between the eyes, mouth and lip contours, cheekbone structure, jaw structure and chin shape, distance from the forehead to the chin, and more. The procedure 400 then proceeds to step 404.

[0098] In step 404, procedure 400 then determines whether the user profile of user 14 exists in the user profile database 58 by determining, based on the identity data received by the multiple first sensors 22, whether a user profile exists in the user profile database 58 that matches the speech recognition program 76 and / or the facial recognition 102 of user 14. If the speech recognition program 76 and / or the facial recognition 102 matches a user profile found in the user profile database 58, procedure 400 then proceeds to step 406.

[0099] In step 406, procedure 400 retrieves the multiple historical / heuristic data points 98 from the user profile assigned to user 14 in order to consider these data points during the weighting of the internal vehicle condition factors 62, the second input 42, and the third input 60 to determine the contextualized risk factor 64. Procedure 400 then proceeds to step 408.

[0100] In step 408, the procedure 400 assigns the behavior of user 14 to the corresponding user profile and records the behavior in the several historical / heuristic data points 98 from the corresponding user profile.

[0101] Returning to step 404, if the speech recognition program 76 and / or the face recognition program 102 does not match a user profile found in the user profile database 58, the procedure 400 proceeds to step 410.

[0102] In step 410, procedure 400 creates a user profile based on the identity data received by the first multiple sensors 22 and adds the user profile to the user profile database 58. Procedure 400 then proceeds to step 412.

[0103] In step 412, the procedure 400 assigns the behavior of user 14 to the corresponding user profile and records the behavior in the several historical / heuristic data points 98 from the corresponding user profile.

[0104] The contextualized risk factor 64 of the present disclosure offers several advantages. These include mitigating the unsafe driving environment due to distracting factors both inside and outside the vehicle 12, proactive safety measures that allow the vehicle 12 to intervene before the risk of driving in the unsafe driving environment increases, providing a contextualized driving behavior report to a third party, enabling the third party to address unsafe behavior exhibited by the user 14, and a streamlined procedure for monitoring the emotional and mental health of the user 14 by providing an assessment of the user's mental and emotional state, which can enable the user 14 to better understand their mental health.

[0105] The description of the present revelation is merely exemplary, and it is intended that variations that do not deviate from the main content of the present revelation remain within its scope. Such variations should not be considered a deviation from the idea and scope of the present revelation.

Claims

[1] Method for mitigating the risk of operating a vehicle using context-dependent information relating to a user of the vehicle and an operating state of the vehicle, the method comprising: Receiving an initial input from several first sensors located in the vehicle, wherein the first input specifies an internal state in the vehicle; Determining a user's identity based on the initial input and a user profile in a user profile database; Determining a set of internal vehicle state factors of the internal state in the vehicle based on the initial input and the user's identity; Receiving a second input from multiple second sensors mounted on the vehicle, the second input indicating the vehicle's operating state; Determining a contextualized risk factor based on the internal vehicle condition factors and the second input; Comparing the contextualized risk factor with a calibrated safety threshold; and Perform at least one action when the contextualized risk factor is above the calibrated safety threshold to mitigate the risk of operating the vehicle. [2] The method of claim 1, wherein receiving the first input further comprises: Receiving the initial input using at least one microphone; Determine that the first input is human language; Generating a transcript of the first input; Analyzing the initial input using a sentiment analysis program; and Analyzing the initial input using a speech recognition program. [3] Method according to claim 2, wherein determining the identity of the user further comprises: Creating the user profile based on the analysis of the first input by the speech recognition program if comparing the first input with each entry in the user profile database does not result in a match; Adding the user profile to the user profile database; and Assigning the set of internal vehicle condition factors to the user in the user profile database. [4] Method according to claim 2, wherein determining the set of internal vehicle condition factors comprises: Generating an assessment of a user's mental and emotional state based on the recording of the initial input and the analysis of the initial input by the mood analysis program; Determine when there is an excessive disturbance of a third party, based on the transcript of the initial submission; Determining the number of users in the vehicle based on the number of users identified by the speech recognition program; and Determine when the user's language is influenced, based on the analysis of the first input by the sentiment analysis program and the analysis of the first input by the speech recognition program. [5] The method of claim 1, wherein receiving the first input further comprises: Receiving the initial input using at least one camera; and Classifying the user as either a driver or a passenger based on the user's position in the vehicle. [6] Method according to claim 5, wherein determining the identity of the user further comprises: Analyzing the first input using a facial recognition program; Creating the user profile based on the analysis of the first input by the facial recognition program if comparing the first input with each entry in the user profile database does not result in a match; Adding the user profile to the user profile database; and Assigning the internal vehicle condition factors and the second input to the user in the user profile database. [7] Method according to claim 5, wherein determining the set of internal vehicle condition factors comprises: Generating an assessment of a user's mental and emotional state based on the tracked activity of the driver; Generating an assessment of a user's mental and emotional state based on the tracked activity of the passenger; Determining the number of passengers in the vehicle based on the tracked activity of at least one passenger; Determine when excessive disturbance occurs, based on the tracked activity of the driver; and Determine when excessive interference by a third party occurs, based on the tracked activity of at least one passenger. [8] The method of claim 1, wherein receiving the first input further comprises: Use at least one eye-tracking device to receive the initial input; Determine when the user's pupils are in a dilated or undilated state; and Determining the direction of the user's gaze. [9] Method according to claim 8, wherein determining the set of internal vehicle condition factors comprises: Generating an assessment of a user's mental and emotional state based on when the user's eyes are in an expanded state; and Determine when excessive disturbance occurs based on the direction of the user's gaze. [10] Method according to claim 1, wherein the second input further comprises: the current speed of the vehicle; a current acceleration of the vehicle; a detected speed limit on a roadway on which the vehicle is operated; an ambient temperature of the environment in which the vehicle is operated; a precipitation intensity of the environment in which the vehicle is operated; a time of day in the environment in which the vehicle is operated; and a location of the vehicle.

Images