A human-computer interaction method, system and readable storage medium

Abstract

This invention discloses a system and method for improving driver safety. The method may include: acquiring current data of a vehicle driver and each passenger in the vehicle who is a potential driver from one or more sensors; further, acquiring historical data of the vehicle driver and historical data of each passenger who is a potential driver from one or more databases; analyzing the current and historical data of the vehicle driver and each passenger who is a potential driver to identify preferred candidates for driving the vehicle, wherein the analysis is performed using a computer-implemented model, such as a computer-implemented neural network; and then outputting a recommendation for the candidates to drive the vehicle. Feedback information may be obtained and used to update the model.

Description

Technical Field

[0001] This disclosure generally relates to methods and systems for improving driver safety, and in particular, to methods and systems for identifying preferred candidates for driving vehicles. Background Technology

[0002] Over the years, countless innovations have been made to improve driver safety. These include seat belts, airbags, safety glass, and impact buffers, all of which help protect drivers in the event of a collision. Other types of innovations attempt to reduce the likelihood of a collision in the first place, such as anti-lock braking systems (ABS) and collision avoidance systems. More recently, some automakers have introduced attention assist systems that warn drivers when excessive fatigue or injury is detected. Despite these innovations, there is still a desire to further improve driver safety by further reducing the probability of collisions. Summary of the Invention

[0003] According to one aspect of this disclosure, a method includes: acquiring current data of a vehicle driver and current data of each of one or more passengers in the vehicle who are potential drivers, from one or more sensors. The method further includes: acquiring historical data of the vehicle driver and historical data of each of the one or more passengers in the vehicle who are potential drivers, from one or more databases. Furthermore, the method includes: analyzing the current data and historical data of the vehicle driver and the current data and historical data of each of the one or more passengers in the vehicle who are potential drivers, to identify preferred candidates for driving the vehicle, wherein the analysis is performed using one or more computer-implemented models. The one or more such computer-implemented models may be computer-implemented neural networks. The method further includes: outputting a suggestion that the preferred candidates drive the vehicle.

[0004] Optionally, in any of the foregoing aspects, the method further includes: obtaining feedback data indicating whether the recommendation is followed; and updating at least one of the one or more computer-implemented models based on the feedback data.

[0005] Optionally, in any of the foregoing aspects, the current data of the vehicle driver includes one or more of the following: current image data, current video data, current audio data, current heart rate data, current body temperature data, current driving speed data, or current driving performance data.

[0006] Optionally, in any of the foregoing aspects, the current data of each of the one or more passengers in the vehicle as a potential driver includes one or more of the following: current image data, current video data, current audio data, current heart rate data, or current body temperature data.

[0007] Optionally, in any of the foregoing aspects, the historical data of the vehicle driver and each of the one or more passengers of the vehicle as potential drivers includes one or more of the following: historical driving record data, historical driving performance data, driver's license type data, historical driving time data, or historical rest time data.

[0008] Optionally, in any of the foregoing aspects, the analysis of the current data and the historical data of the vehicle driver and the current data and the historical data of each of the one or more passengers of the vehicle as a potential driver includes: quantifying one or more of the following of the driver and each of the one or more passengers of the vehicle as a potential driver: mental state, emotional state, fatigue state, or alertness state.

[0009] According to another aspect of this disclosure, a system includes one or more sensor interfaces, one or more database interfaces, and one or more processors. The one or more sensor interfaces are used to acquire current data of a vehicle driver and current data of each of one or more passengers in the vehicle who are potential drivers, from one or more sensors. The one or more database interfaces are used to acquire historical data of the vehicle driver and historical data of each of the one or more passengers in the vehicle who are potential drivers, from one or more databases. The one or more processors are used to analyze the current data and historical data of the vehicle driver and the current data and historical data of each of the one or more passengers in the vehicle who are potential drivers, thereby identifying a preferred candidate to drive the vehicle. The processors are also used to output a recommendation that the preferred candidate drive the vehicle.

[0010] Optionally, in any of the foregoing aspects, the one or more processors implement one or more computer-implemented models for analyzing the current and historical data of the vehicle driver and the current and historical data of each of the one or more passengers of the vehicle as potential drivers, thereby identifying preferred candidates to drive the vehicle.

[0011] Optionally, in any of the foregoing aspects, the one or more processors are further configured to: acquire feedback data indicating whether the recommendation is followed; and update at least one of the one or more computer-implemented models based on the feedback data.

[0012] Optionally, in any of the foregoing aspects, the current data of the vehicle driver analyzed by the one or more processors includes one or more of the following: current image data, current video data, current audio data, current heart rate data, current body temperature data, current driving speed data, or current driving performance data.

[0013] Optionally, in any of the foregoing aspects, the current data of each of the one or more passengers in the vehicle who is a potential driver, analyzed by the one or more processors, includes one or more of the following: current image data, current video data, current audio data, current heart rate data, or current body temperature data.

[0014] Optionally, in any of the foregoing aspects, the historical data of each of the vehicle driver and the one or more passengers of the vehicle as potential drivers analyzed by the one or more processors includes one or more of the following: historical driving record data, historical driving performance data, driver's license type data, historical driving time data, or historical rest time data.

[0015] Optionally, in any of the foregoing aspects, in order to analyze the current data and historical data of the vehicle driver and the current data and historical data of each of the one or more passengers of the vehicle as a potential driver, the one or more processors are configured to: quantify one or more of the following of the driver and each of the one or more passengers of the vehicle as a potential driver: mental state, emotional state, fatigue state, or alertness state.

[0016] Optionally, in any of the foregoing aspects, the one or more sensors include one or more cameras for acquiring current image data of the vehicle driver and current image data of each of the one or more passengers in the vehicle who are potential drivers. The one or more database interfaces are used to acquire historical image data of the vehicle driver and historical image data of each of the one or more passengers in the vehicle who are potential drivers from one or more databases. Furthermore, the one or more processors are used to analyze the current image data and historical image data of the vehicle driver and the current image data and historical image data of each of the one or more passengers in the vehicle who are potential drivers to identify preferred candidates for driving the vehicle.

[0017] According to another aspect of the invention, a non-transitory computer-readable medium stores computer instructions that, when executed by one or more processors, cause the one or more processors to perform the following steps: acquiring current data of a vehicle driver and current data of each of one or more passengers in the vehicle who are potential drivers from one or more sensors; acquiring historical data of the vehicle driver and historical data of each of the one or more passengers in the vehicle who are potential drivers from one or more databases; analyzing the current data and historical data of the vehicle driver and the current data and historical data of each of the one or more passengers in the vehicle who are potential drivers to identify a preferred candidate to drive the vehicle; and outputting a suggestion that the preferred candidate drive the vehicle. The non-transitory computer-readable medium may also store computer instructions that, when executed by one or more processors, cause the one or more processors to perform additional steps of the method outlined above and further described in detail below.

[0018] This summary is provided to introduce, in a simplified form, a series of concepts further described in the detailed embodiments. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to help determine the scope of the claimed subject matter. The claimed subject matter is not limited to embodiments that address any or all the shortcomings pointed out in the background art. Attached Figure Description

[0019] Various aspects of this disclosure are shown by way of example and are not limited by the accompanying drawings, in which the same reference numerals denote elements.

[0020] Figure 1 A driver recommendation system according to an embodiment of the present technology is shown;

[0021] Figure 2 Provided according to one embodiment of the present technology Figure 1 A block diagram showing additional details of the driver suggestion system introduced in the document;

[0022] Figure 3A and Figure 3B A flowchart is shown to describe a method for recommending preferred candidates for driving vehicles according to certain embodiments of the present technology;

[0023] Figure 4 It shows what can be provided Figure 1 The driver recommendation system uses various types of data (or subsets thereof) to recommend preferred candidates for driving vehicles;

[0024] Figure 5 It is used to describe Figure 3A A flowchart with additional details of one of the steps described;

[0025] Figure 6 It is a flowchart used to describe how rewards and / or feedback are quantified;

[0026] Figure 7 It is a block diagram used to describe additional details of how to detect the emotional state, fatigue state, mental state, and alertness state of the vehicle driver and each passenger who is a potential driver. Detailed Implementation

[0027] This technical embodiment relates to methods and systems for analyzing current and historical data of a vehicle driver and current and historical data of each of one or more passengers in the vehicle who are potential drivers, to identify and recommend preferred candidates to drive the vehicle. Such embodiments can be used to improve the safety of the driver and passengers of the vehicle, as well as the safety of drivers and passengers of other vehicles.

[0028] It should be understood that embodiments of this disclosure may be implemented in many different forms, and the scope of the claims should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of embodiments of the invention to those skilled in the art. In fact, this disclosure is intended to cover alternatives, modifications, and equivalents of these embodiments included within the spirit and scope of this disclosure as defined by the appended claims. Furthermore, in the following detailed description of embodiments of this disclosure, many specific details are set forth in order to provide a thorough understanding. However, it will be apparent to those skilled in the art that embodiments of this disclosure may be practiced without these specific details.

[0029] Figure 1 A driver suggestion system 110 according to an embodiment of the present technology is illustrated. As shown, the driver suggestion system 110 is installed or otherwise included within a vehicle 101, which also includes a cabin where a driver and one or more passengers can sit. The driver suggestion system 110 or one or more portions thereof may be implemented by an in-cabin computer system and / or a mobile computing device, such as, but not limited to, a smartphone, tablet, laptop, and / or similar mobile computing device.

[0030] According to certain embodiments of the present technology, the driver recommendation system 110 acquires current data of the vehicle driver and current data of each of the one or more passengers in the vehicle who are potential drivers from one or more sensors. The driver recommendation system 110 also acquires historical data of the vehicle driver and historical data of each of the one or more passengers in the vehicle who are potential drivers from one or more databases. The driver recommendation system 110 analyzes the current and historical data of the vehicle driver and the current and historical data of each of the one or more passengers in the vehicle who are potential drivers to identify preferred candidates for driving the vehicle, wherein the analysis is performed using one or more computer-implemented neural networks and / or certain other computer-implemented models. Furthermore, the driver recommendation system 110 outputs a recommendation for the preferred candidates to drive the vehicle.

[0031] like Figure 1 As shown, the driver suggestion system 110 is communicatively coupled to a driver capture subsystem 102 and a passenger capture subsystem 103. The driver capture subsystem 102 can be used to acquire current data of the driver of the vehicle 101. The passenger capture subsystem 103 can be used to acquire current data of one or more passengers of the vehicle. In other words, the driver capture subsystem 102 and the passenger capture subsystem 103 may include sensors for acquiring current data of the vehicle driver and current data of each passenger of the vehicle who is a potential driver. (Referring below...) Figure 2 In further detail, such sensors may include one or more cameras, microphones, and / or physiological sensors.

[0032] Although Figure 1 Only one passenger capture subsystem 103 is shown, but the vehicle 101 may include multiple passenger capture subsystems 103. Figure 1 As shown, the driver capture subsystem 102 and the passenger capture subsystem 103 may be located outside the driver suggestion system 110, or, depending on the specific implementation, may be included as part of the driver suggestion system 110. (See below for reference.) Figure 2 Additional details are described for the driver capture subsystem 102 and the passenger capture subsystem 103 according to certain embodiments of the present technology.

[0033] Still referencing Figure 1The driver suggestion system 110 is also shown communicatively coupled to various types of vehicle-related sensors 105 included within the vehicle 101. These sensors 105 may include, but are not limited to, speedometers, global positioning system (GPS) receivers, and clocks. The driver suggestion system 110 is also shown communicatively coupled to one or more communication networks 130, which provide access to one or more databases 140 and / or other types of databases. The databases 140 and / or other types of databases may store historical data of the vehicle's driver and historical data of each of one or more passengers in the vehicle who are potential drivers. Examples of such historical data include, but are not limited to, historical driving record data, historical driving performance data, driver's license type data, historical driving time data, and / or historical rest time data. Such historical data may be stored in a local database or other database located within the vehicle. However, it is more likely that the historical data is stored in one or more databases 140 or other databases located further away from the vehicle 101. Therefore, such databases 140 or other databases may be communicatively coupled to the driver suggestion system via one or more communication networks 130.

[0034] The communication network 130 may include a data network, a wireless network, a telephone network, or any combination thereof. It is contemplated that the data network may be any local area network (LAN), metropolitan area network (MAN), wide area network (WAN), public data network (e.g., the Internet), short-range wireless network, or any other suitable packet-switched network. Furthermore, the wireless network may be, for example, a cellular network, and may employ various technologies, including enhanced data rate for global evolution (EDGE), general packet radio service (GPRS), global system for mobile communications (GSM), Internet protocol multimedia subsystem (IMS), universal mobile telecommunications system (UMTS), and any other suitable wireless medium, such as worldwide interoperability for microwave access (WiMAX), Long Term Evolution (LTE) networks, code division multiple access (CDMA), wideband code division multiple access (WCDMA), wireless fidelity (Wi-Fi), wireless LAN (WLAN), Bluetooth, Internet Protocol (IP) data transmission, satellite, mobile ad-hoc network (MANET), and any combination thereof. The communication network 130 can provide the driver suggestion system 110 with the database 140 and / or the other databases.

[0035] Now refer to Figure 2 Additional details are described for the driver suggestion system 110 according to certain embodiments of the present technology. References Figure 2As shown, the driver suggestion system 110 includes one or more processors 208, an input / output (I / O) interface 210, a memory 212, a display 214, and a communication device 216. As also shown, the driver suggestion system 110 further includes a driver analyzer 215, a driver profile 217, a machine learning engine 219, a passenger analyzer 225, a passenger profile 227, and a database 218. For example, the communication device 216 can provide communication between the driver suggestion system 110 and one or more external databases via one or more communication networks 130. The display 214 can display instructions and / or suggestions to the driver of the vehicle 101 and one or more passengers of the vehicle 101. Additionally or alternatively, such instructions and / or suggestions can be provided by the audio speaker 220 of the driver suggestion system 110.

[0036] Despite Figure 2 While shown as separate boxes, the driver analyzer 215 and the passenger analyzer 225 can be implemented by one or more processors 208 of the driver suggestion system 110. The driver analyzer 215 can detect and quantify various states of the driver of the vehicle 101, including but not limited to the driver's mental state, emotional state, fatigue state, and / or alertness state. Similarly, the passenger analyzer 225 can detect and quantify various states of the passengers of the vehicle 101, including but not limited to the passengers' mental state, emotional state, fatigue state, and / or alertness state. Depending on the specific implementation, the passenger analyzer 225 can perform such analysis on all passengers in the vehicle or only on one or more passengers who have been identified as potential drivers. The driver analyzer 215 can perform analysis based on the current data of the vehicle's driver and the vehicle's driver's historical data. Similarly, the passenger analyzer 225 can perform analysis based on the current data of the vehicle's passengers and the vehicle's passengers' historical data.

[0037] As described above, historical data that can be stored for the driver of the vehicle and each passenger of the vehicle as a potential driver may include historical driving record data, historical driving performance data, driver's license type data, historical driving time data, and / or historical rest time data. For example, the historical driving record data may be generated and maintained by one or more government agencies and / or private institutions or companies that track driver's licenses and / or vehicle registration information. Examples of government agencies that may generate and maintain the historical driving record data include motor vehicle departments, motor vehicle bureaus, public safety departments, transportation departments, and / or similar government agencies. Private insurance companies may also generate and maintain the historical driving record data. For example, the historical driving record data may include information about the frequency and type of traffic violations committed by a licensed driver. These agencies and / or companies may also generate and maintain the driver's license type data, which may indicate the specific type of vehicle an individual is permitted to drive, such as passenger vehicles, commercial vehicles, semi-trailers, and / or similar vehicles, and whether the person holds a full driver's license or only a learner's permit. Such data may also specify whether an individual's driver's license is suspended or has been suspended in the past. The historical driving performance data can be specified, for example, the frequency with which an individual drives a vehicle and the level of safety they demonstrate while driving.

[0038] The historical driving record data and / or the historical driving performance data can be used to predict an individual's future driving performance. For example, assuming the first person was involved in only one collision last year, while the second person was involved in two collisions last year, without knowing more information, it can be inferred that the first person is a safer driver than the second person. However, if it is known that the first person drives only one hour per week, while the second person drives ten hours per week, then inferring that the second person is a safer driver than the first person would be more accurate, because the second person has far fewer collisions per hour than the first person. The historical driving record data and the historical driving performance data can also specify specific types of driving conditions under which a particular person is considered to be in good or poor driving condition. These are just a few examples and do not encompass all situations.

[0039] The historical driving time data can indicate the frequency and duration of driving by a particular person in each specific time period (e.g., daily). Alternatively or additionally, the historical driving time data can indicate how recently and / or for how long a particular person has driven in one or more recent time periods (e.g., the past hour, the past four hours, the past eight hours… the past day, the past two days, etc.). For example, the historical driving time data can be obtained from one or more wearable devices (e.g., watches) and / or mobile computing devices that people typically wear and / or carry (e.g., smartphones). Alternatively or additionally, the historical driving time data is obtained from a computer system within the cockpit. In some embodiments, for example, the driver and / or passenger can input historical driving data via the I / O interface 210 in response to questions posed to the driver and / or passenger.

[0040] The historical driving record data and / or the historical driving performance data may also include information about the driver's seniority, experience type, traffic accident history, items in the employment record, history of speeding and / or other traffic violations, and the number, severity, and duration of various previously identified driving behaviors. As described above, the driver recommendation system 110 may collect and store driving profiles (e.g., profile 217) for one or more specific drivers as part of the driver's history. For example, the driver's profile may also include the driver's acceleration or deceleration speed, normal speed under specific driving conditions (e.g., night driving, driving in rain or snow), frequency of the driver's sharp turns or crossing the center line, etc. In one embodiment, the driver history information may be retrieved from an external database, such as the database of the driver's insurance company.

[0041] The driving history may also include information about one or more driving routes in which the driver of vehicle 101 or the driver of other vehicles was fatigued while driving. In some embodiments, the information about the one or more driving routes may also include road profiles, road topology, the number of parking spots associated with the driving route, the number of curves associated with the driving route, etc.

[0042] The historical rest time data can indicate the frequency and duration of a particular person's usual sleep or rest during specific time periods (e.g., daily). Alternatively or additionally, the historical rest time data can indicate how long a particular person has rested and / or for how long during one or more recent time periods (e.g., the past hour, the past four hours, the past eight hours… the past day, the past two days, etc.). For example, the historical rest time data can be obtained from one or more wearable devices (e.g., watches) and / or mobile computing devices that people typically wear and / or carry (e.g., smartphones). Alternatively or additionally, the historical rest time data is obtained from an in-cabin computer system. In some embodiments, for example, the driver and / or passenger can input the historical rest time data via the I / O interface 210 in response to questions posed to the driver and / or passenger.

[0043] Figure 2 Additional details of the driver capture subsystem 102 and the passenger capture subsystem 103 are also shown. (Still referencing...) Figure 2 As shown, the driver capture subsystem 102 includes one or more cameras 205, one or more microphones 206, and one or more physiological sensors 204. Similarly, the passenger capture subsystem 103 includes one or more cameras 205, one or more microphones 206, and one or more physiological sensors 204. As described above, the driver capture subsystem 102 and the passenger capture subsystem 103 may be located outside the driver suggestion system 110, or, depending on the specific implementation, may be included as part of the driver suggestion system 110.

[0044] The input / output interface 210 allows information to be presented to the user and / or other components or devices using various input / output devices. Examples of input devices include a keyboard, microphone, touchscreen (e.g., including a capacitive sensor or other sensor for detecting physical touch), camera (e.g., employing visible or non-visible wavelengths, such as infrared frequencies, to recognize movement as a gesture not involving touch), etc. Examples of output devices include display device 214, speaker, etc. In some embodiments, the I / O interface 210 receives current driver data and current passenger data from the driver capture subsystem 102 and the passenger capture subsystem 103. The I / O interface 210 can provide an interface between the sensors of the capture subsystems 102 and 103 and the driver suggestion system 110, and / or may use an additional sensor interface 207.

[0045] The driver capture subsystem 102 can be used to monitor and identify driver behavior based on driver data captured using one or more of the camera 205, the microphone 206, and / or the physiological sensor 204. In one embodiment, the camera 205 is positioned to capture images and movements of the driver's head and face, while in other embodiments, images and movements of the driver's torso and / or the driver's limbs and hands are also captured. Using such image data, the driver analyzer 215 can detect driver behavior captured by the camera 205 to detect, for example, specific facial expressions, gazes, blinks, and / or body postures, which may include head postures, nodding, and / or similar body postures. One or more cameras 205 may be RGB cameras, RGB-D cameras, and / or near-infrared (NIR) cameras including image sensors (e.g., CMOS image sensors), which may be used to capture multiple two-dimensional RGB, RGB-D, and / or NIR images per second (e.g., 30 images per second). At least one camera may be a depth camera that generates depth images, for example, reconstructing 3D structures on point clouds or similar objects using structured light and / or time-of-flight (TOF) sensors. These are just examples of some types of cameras and images that can be used. Other types of cameras and images may also be used and are also within the scope of the embodiments described herein.

[0046] One or more microphones 206 of the driver capture subsystem 102 can be used to capture one or more audio signals of the driver captured by the microphones 206. The audio can be analyzed to detect various characteristics that may vary with the driver's state. Examples of such audio characteristics include slurred speech or snoring, which may indicate that the driver is fatigued or drowsy.

[0047] The components of the driver capture subsystem 102 (e.g., physiological sensors, cameras, microphones, etc.) may all be located within the vehicle cabin, or they may be located in different areas of the vehicle cabin. In another example, a single component of the capture subsystem 102 may be part of another component or device. For example, the camera 205 and the microphone 206 (and the display 214) may be part of a smartphone or tablet (not shown) placed in the vehicle cabin, while one or more physiological sensors 204 may be part of a wearable device and / or located in one or more different locations within the vehicle cabin. The physiological sensors 204 may include, but are not limited to, heart rate sensors, body temperature sensors, blood pressure sensors, and / or similar physiological sensors. One or more such sensors may be integrated into the driver's seat.

[0048] Similarly, the components of the passenger capture subsystem 103 (e.g., physiological sensors, cameras, microphones, etc.) may all be located within the vehicle cabin, or they may be located in different areas of the vehicle cabin. Furthermore, a single component of the capture subsystem 103 may be part of another component or device. Each passenger may have a separate passenger capture subsystem 103, or multiple passengers may use one subsystem 103. The driver capture subsystem 102 and the passenger capture subsystem 103 may also share certain components (e.g., microphones and / or cameras), and / or multiple passenger capture subsystems 103 may share certain components.

[0049] The driver analyzer 215 may be associated with a driver profile 217, which models driver behavior and is further explained below, and together may detect one or more attributes indicating the manner in which the driver behaves in the vehicle 101. Driver behavior may include, for example, the driver's normal blink rate, gaze direction, head, limb and / or torso position and / or other driver movements and activities or driver voice or audio. When the driver's behavior deviates significantly from the profile 217, this may indicate unstable or abnormal driver behavior that suggests a safety hazard to the vehicle and that its occupants are at increased risk. For example, a rapid blink rate or awkward gaze direction may indicate that the driver is fatigued. In another example, snoring or slurred speech may indicate that the driver is about to fall asleep or has already fallen asleep. As further explained below, real-time evaluation of driver behavior in conjunction with historical and current data or information enables the system 110 to provide real-time feedback (e.g., a suggestion for a passenger to take over driving) in response to the detection of dangerous driver behavior.

[0050] In some embodiments, the driver behavior can be compared to an individual profile or a set of driver profiles, or to a “threshold” profile, where each “threshold” profile can serve as a baseline for the comparison. The comparison can be used to determine whether a particular driver’s actions (e.g., facial expressions, movements, and / or voice) constitute acceptable (e.g., fatigue is not or likely is not a problem) or unacceptable (e.g., fatigue may or is a problem) driver behavior. For example, an algorithm can be used to determine whether the driver’s eye-closing captured by the capture subsystem 102 is more likely to be associated with acceptable driver behavior (e.g., opening and blinking at a specified behavior threshold) or unacceptable driver behavior (e.g., eye-closing time exceeding a driver behavior threshold).

[0051] While driver images, motion, and / or audio can be captured for analysis to identify driver behavior, additional information can also be captured and collected. This additional information may include, but is not limited to, driver heart rate, body temperature, and / or blood pressure. Historical data of the vehicle's driver can also be accessed and used to determine whether the driver's behavior is normal. The identified behavior can be used to generate an initial or baseline driver profile and to generate historical profiles, wherein data is collected over a predetermined time period. In one embodiment, the additional information includes the context or environment in which the driver is currently driving the vehicle 101. For example, the context or environment applicable to driving the vehicle can be defined as the current set of conditions under which the driver is driving. Such conditions may include the length of time the driver has been on the road (e.g., current driving hours), the time since the driver last went to sleep, weather conditions, temperature, direction of travel, sun position, road conditions, date and time of day / night, traffic conditions, and / or any other current state related to and / or that may affect the driver's behavior, including a historical set of any of the above conditions. This information can be collected for the driver of the vehicle and one or more passengers who are potential drivers.

[0052] In some embodiments, the detected driver behavior is associated with fatigue, drowsiness, or lethargy. These driver behaviors can be indicated by actions such as excessive yawning, blinking, and / or nodding. In some embodiments, the driver behavior is voluntary (the opposite of fatigue-related behavior), such as texting, emailing, using a mobile phone or tablet, eating, reading, talking, etc. For example, movement / position consistent with driver and device interaction can be indicated by the driver's gaze direction, eye and / or retinal movements (e.g., back-and-forth movement from the road to the device, finger / hand movements consistent with touch or keyboard interaction, head turning (e.g., alternating between looking towards the road and towards the device)), etc.

[0053] In some embodiments, the driver analyzer 215 may include hardware running software and / or firmware for driver behavior detection. In some embodiments, the driver analyzer 215 may be implemented as part of an onboard computer or processor 208 of a vehicle (not shown). In other embodiments, captured driver images and associated audio data, along with optionally at least some additional information, are transmitted from the vehicle to an external computer or processor via a network (not shown). In some embodiments, some portions of the driver behavior detection, such as motion detection, image processing and / or behavior recognition, classification, and detection, may be implemented by the driver behavior analyzer 215 in the vehicle 101, while other portions of the driver analysis, such as the analysis of processed data, may be implemented by an external processor via a network.

[0054] The input / output interface 210 presents information to the driver and / or one or more passengers who may be the driver. Examples of input devices include a keyboard, microphone, touchscreen (e.g., including a capacitive sensor or other sensor for detecting physical touch), camera (e.g., employing visible or non-visible wavelengths, such as infrared frequencies, to recognize movement as a gesture not involving touch), etc. Examples of output devices include one or more display devices 214 and / or speakers 220, etc.

[0055] In some embodiments, the driver identifies themselves to the driver suggestion system 110 using the I / O interface 210, thereby enabling the driver suggestion system 110 to access the driver's historical data. Alternatively, the driver suggestion system 110 may use facial recognition technology or similar methods to identify the driver, thereby enabling the driver suggestion system 110 to access the driver's historical data. More specifically, one or more cameras 205 may be used to acquire image data of the driver, and this image data may be compared with image data stored in a database or similar source to identify the driver.

[0056] One or more passengers in the vehicle can also identify themselves to the driver suggestion system 110 using the I / O interface 210, thereby enabling the driver suggestion system 110 to obtain the passenger's historical data and determine whether the passenger is a potential driver. The passenger or the driver can use the I / O interface 210 to specify which passengers are potential drivers. Passengers in the vehicle can be identified using facial recognition technology, etc., enabling the driver suggestion system 110 to determine whether they are potential drivers and obtain the passenger's historical data. More specifically, one or more cameras 205 can be used to acquire image data of each passenger, and this image data can be compared with image data stored in a database, etc., to identify the passenger. Once the passenger is identified using image data and / or input by the driver or the passenger, the driver suggestion system 110 can determine whether the passenger is a potential driver. This determination can be based on whether the passenger possesses an appropriate and valid license to drive the vehicle in which they are located. In some embodiments, the driver and / or the passenger can use the I / O interface 210 to specify one or more passengers as potential drivers. Furthermore, one or more passengers can be allowed to choose not to be considered as potential drivers.

[0057] In some embodiments, the I / O interface 210 (and / or interface 207) receives current data of the vehicle driver, which may include, for example, current image data, current video data, current heart rate data, current body temperature data, current driving speed data, and / or current driving performance data, but is not limited thereto. The current image and / or video data may be obtained from one or more cameras 205 of the driver capture subsystem 102. The current heart rate data and the current body temperature data may be obtained from one or more physiological sensors 204 of the driver capture subsystem 102. Such physiological sensors 204 may be contained in a wearable device (e.g., a device worn on the wrist) that can wirelessly communicate with the driver suggestion system 110. Such physiological sensors 204 may also be built into the vehicle, for example, in the driver's seat.

[0058] Similarly, in some embodiments, the I / O interface 210 (and / or interface 207) receives current data of one or more passengers in the vehicle, which may include, for example, current image data, current video data, current heart rate data, current body temperature data, current driving speed data, and / or current driving performance data, but is not limited thereto. The current image and / or video data may be obtained from one or more cameras 205 of the passenger capture subsystem 103. The current heart rate data and the current body temperature data may be obtained from one or more physiological sensors 204 of the passenger capture subsystem 103. Such physiological sensors 204 may be included in a device worn by the passenger (e.g., a device worn on the wrist) that can wirelessly communicate with the driver suggestion system 110. Such physiological sensors 204 may also be built into the vehicle, for example, in the passenger seat. The I / O interface 210, communication device 216, and / or interface 209 may provide one or more database interfaces that retrieve historical data of the vehicle driver and historical data of each of the one or more passengers in the vehicle who are potential drivers from one or more databases.

[0059] Despite Figure 2Shown as separate boxes, but the driver analyzer 215 and the passenger analyzer 225 can be implemented by one or more processors 208 of the driver suggestion system 110. In some embodiments, the driver analyzer 215, or more specifically, the processor 208, extracts and measures the number of times the driver closes their eyes, and can determine the driver's eye-closing ratio over a period of time using one or more eye-closing measurements and the driver profile 217. The eye-closing ratio can indicate the ratio of the number of times the driver closes their eyes measured in the current time period to the number of times the driver closes their eyes retrieved from the driver profile 217 in a baseline state. The eye-closing ratio can then be normalized, and the processor 208 can determine a percentage eye-closing value, and this process can be repeated over a period of time. The measured values ​​can then be compared with values ​​stored in the driver profile 217 to determine whether the measured number of eye-closings indicates that the driver of the vehicle 101 is fatigued or drowsy. In one embodiment, the processor 208 continues to measure until the vehicle stops or the driver otherwise completes driving, such as by leaving the vehicle 101.

[0060] The data collected by the driver capture subsystem 102 and the passenger capture subsystem 103 may be stored in database 218, memory 212, or any combination thereof. In one embodiment, the collected data originates from one or more sources outside the vehicle 101. The stored driver information may be data related to driver behavior and safety, such as information captured by the capture subsystem 103. In one embodiment, the data stored in database 218 may be a collection of data collected for the vehicle 101 and / or other vehicles. In one example, the collected data may provide current data collected in real time and may include current profile data as well as other data that is relevant and useful to driver behavior and safety. In another embodiment, the collected data may provide historical data or information, such as driving history, driver fatigue data, historical profile data, and other data that is relevant and useful to driver behavior and safety collected in the past and / or over a period of time. In one embodiment, the driving history may include a portion of the stored profile 217. In another embodiment, the collected data may be current data of one or more drivers currently driving on a specific route or within a geographic area that provides real-time indication of the collected data.

[0061] In some embodiments, the memory 212 may store executable instructions of the processor 208 and the machine learning engine 219, as well as programs or applications (not shown) that can be loaded and executed by the processor 208. The machine learning engine 219 may include executable code of the processor 208 stored in the memory 212 and select one or more machine learning models, such as behavioral models, stored in the memory 212 (or the database 218). The behavioral models may be developed based on acceptable and unacceptable driver behaviors (e.g., driver body position) using current and historical data collected from the vehicle 101 and / or other vehicles, and any other information related to driver behavior or history. Models of acceptable and unacceptable driver behaviors (e.g., driver body position) may be developed using known and conventional machine learning and deep learning techniques, such as convolutional neural networks (CNNs), as described in more detail below.

[0062] Figure 3A and Figure 3B A flowchart is shown for describing a method for recommending preferred candidates for driving vehicles, such as vehicle 101, according to certain embodiments of the present technology. See also Figure 3A Step 302 involves obtaining current data of the vehicle driver, and step 304 involves obtaining current data of one or more passengers of the vehicle who are potential drivers. Figure 3A As shown, steps 302 and 304 can be executed in parallel. Alternatively, these steps can be executed sequentially, i.e., one after another; for example, step 302 can be executed before step 304, or vice versa. See also... Figure 3A Step 306 involves obtaining historical data of the vehicle's driver, and step 308 involves obtaining historical data of one or more passengers of the vehicle who are potential drivers. Figure 3A As shown, steps 306 and 308 can be executed in parallel. Alternatively, these steps can be executed sequentially, i.e., one after another; for example, step 306 can be executed before step 308, or vice versa. The order of steps 302 and 306, and the order of steps 304 and 308, can also be reversed. Example types of current and historical data of the driver and passenger that can be obtained have been discussed in detail above, and will be discussed in more detail below, for example, referring to... Figure 4 .

[0063] Still referencing Figure 3AStep 312 involves analyzing the current and historical data of the vehicle driver and the current and historical data of each passenger in the vehicle who is a potential driver, thereby identifying preferred candidates to drive the vehicle. According to some embodiments, the analysis is performed using a computer-implemented neural network and / or some other type of computer-implemented model. According to some embodiments of the present technology, the following will be combined with… Figure 5 Additional details on how to perform step 312 are discussed. After identifying the preferred candidate to drive the vehicle in step 312, step 314 involves outputting a recommendation for the preferred candidate to drive the vehicle. See again... Figure 2 The suggestion can be output via the display 214 and / or the speaker 220. In some embodiments, the suggestion may typically be to suggest that a specific passenger take over driving the vehicle. In other embodiments, the rationale for the suggestion may be explained. For example, the suggestion may be to suggest that a specific passenger take over driving the vehicle because the driver appears very tired and / or distracted, and the specific passenger appears well-rested and attentive. For a more specific example, the driver may be given an audio output saying, “You look tired; it would be safer for a passenger to take over driving.” Alternatively, one of the passengers may be given an audio output saying, “The driver is tired; you look like the best person to drive the vehicle now. Would you like to switch?” These are just a few example types of outputs that may occur in step 314. It is also possible that there is no suitable candidate to drive the vehicle; in this case, the system may simply suggest that the vehicle driver find a rest stop so that the people in the vehicle can rest, etc.

[0064] According to some embodiments, if it is determined that it is safe for the current driver to continue driving (e.g., if the quantified result exceeds a certain specified threshold), one of the passengers will not be advised to take over driving, even if the analysis indicates that the passenger may be a safer driver. In other words, for example, the analysis result in step 312 can be labeled as positive or negative for the driver and each passenger based on a comparison with a threshold. If the driver's result is positive, the driver recommendation system 110 may advise the driver to continue driving, or provide no recommendation, as the status quo should be maintained. However, if the driver's result is negative, and one or more passengers have positive results, the driver recommendation system 110 may advise one of the passengers to take over driving. Otherwise, the driver recommendation system 110 may advise all occupants of the vehicle to take a break.

[0065] Now for reference Figure 3B The flowchart in Figure 3B The flowchart in is Figure 3AThis is a continuation of the flowchart in step 314. Step 316 involves obtaining feedback data indicating whether the recommendation has been followed (the output of step 314), and step 318 involves updating the computer-implemented neural network and / or some other type of computer-implemented model (for performing the analysis in step 312) based on the feedback data. The feedback may simply indicate whether the recommendation output in step 314 has been followed. The driver or passenger may input the feedback via the I / O interface 210. Alternatively, the driver capture subsystem 102 may capture image data indicating whether the vehicle driver has been replaced by one of the passengers, thus indicating that the vehicle has a new driver.

[0066] If the suggestion output in step 314 is for a specific passenger to take over driving the vehicle 101, the feedback obtained in step 316 can indicate whether the specific passenger has taken over driving the vehicle. If the suggestion is followed, the driver suggestion system 110 can infer that its suggestion is accurate and that the driver and the specific passenger are likely to follow such suggestions. If the suggestion is not followed, the driver suggestion system 110 can infer that its suggestion may be inaccurate and / or that the driver and the specific passenger are unlikely to follow such suggestions. In the event of non-following the suggestion, the driver suggestion system 110 can ask the driver and / or the specific passenger (identified as the preferred passenger to drive the vehicle) one or more questions via the display 214, etc., to receive specific feedback on why the suggestion was not followed. These reasons may include, for example, that the driver believes they can drive, and / or that the specific passenger does not want to drive the vehicle and / or other specific reasons. Such feedback information can be used to update stored historical data of the driver and the specific passenger. As described above, this feedback information can also be used to update the model in step 312 that performs the analysis, thereby potentially changing how the analysis will be performed in the future.

[0067] Figure 4 It shows what can be provided to (and thus obtained) in Figure 1 The driver recommendation system 110 incorporates various types of data, which it uses (or portions thereof) to recommend preferred candidates for driving vehicles. For example... Figure 4As shown and as described above, such data may include the driver's current data, the current data of one or more passengers acting as potential drivers, and historical data of the driver and each passenger acting as a potential driver. As mentioned above, the driver's current data may include, but is not limited to, current image data, current video data, current audio data, current heart rate data, current body temperature data, current driving speed data, and / or current driving performance data. The current data of each of the one or more passengers may include similar types of data, but may not include current driving speed data and current driving performance data, because the passenger is not currently driving the vehicle. As mentioned above, the historical data of the driver and each passenger may include, for example, historical driving record data, historical driving performance data, driver's license type data, historical driving time data, historical rest time data, and / or similar data.

[0068] Now refer to Figure 5 The description above refers to certain embodiments of the present technology. Figure 3A Additional details regarding the analysis performed in step 312 are provided below. Along Figure 5 The analysis of the driver is shown on the left side without a specific order. This analysis may include quantifying the driver's mental state in step 502, quantifying the driver's emotional state in step 504, quantifying the driver's fatigue state in step 506, and quantifying the driver's alertness in step 508. Similarly, the analysis may also be shown without a specific order along... Figure 5 In the examples of steps 512, 514, 516, and 518 shown on the right, the state of one or more passengers is quantified. The mental state of each of the driver and the one or more passengers can indicate, for example, whether they are mentally sound, but is not limited to this. The emotional state of each of the driver and the one or more passengers can indicate, for example, whether they are angry or happy, but is not limited to this. The fatigue state of each of the driver and the one or more passengers can indicate, for example, whether they are tired or fully awake, but is not limited to this. The alertness state of each of the driver and the one or more passengers can indicate, for example, whether they are distracted or focused, but is not limited to this. Other types of states that can be quantified for the driver and the passengers can also be physical health, for example, whether they are healthy or sick. The quantization performed in steps 502 to 508 and 512 to 518 can assign a first value (e.g., 1) to states conducive to safe driving and a second value (e.g., -1) to states detrimental to safe driving. In other words, the quantization can be binary. Alternatively, the quantization can have more granularity than just binary, with each quantized state having more than two possible values.

[0069] Still referencing Figure 5 Step 520 involves identifying preferred candidates based on quantified states corresponding to the driver and the passengers who are potential drivers. According to some embodiments, step 520 may involve summing the quantified values ​​determined in steps 502-508 and the instances in steps 512-518 (performed for each passenger who is a potential driver), and identifying which person (driver or passenger) corresponds to the largest sum. In performing the summation, the various states can be weighed equally. Alternatively, some states may have different weights than others. For example, fatigue may have the highest weight, while another state may have the lowest weight. Other variations are also possible, within the scope of the embodiments described herein.

[0070] In some embodiments, quantized information vectors are used to quantize the states of the driver and passengers. Assume... Let m be the information vector of the potential driver. For the feedback of the potential driver m, For the result of the potential driver m, then Similarly, suppose It is the driver's information vector. This is the driver's feedback. If it is the result of the driver, then .if The driver suggestion system 110 can suggest that the driver continue driving. If The driver recommendation system 110 can recommend a driver replacement. When this occurs, if there is... The system recommends the passenger with the highest status value as the preferred candidate to drive the vehicle. If no such passenger exists... The system suggests that everyone find a parking lot or rest stop to rest. This functionality can be achieved using various machine learning methods, such as logistic regression, support vector machines, neural networks, or hidden Markov models. In some embodiments, the outcome for the driver and the outcome for each potential driver can be determined separately.

[0071] In some embodiments, to increase the probability that the driver and the passenger will follow the recommendations, a reward may be offered to the driver and / or one or more passengers whenever a recommendation is followed. Such rewards may be coupons from certain companies, cashback rewards, vehicle insurance discounts, and / or similar incentives.

[0072] Figure 6This is a flowchart describing how rewards and / or feedback are quantified. In step 602 (which may be the same as step 314), a suggestion is provided to the driver. In a separate instance of step 602, suggestions may also be provided to one or more passengers. In an instance of step 604, an indication of whether the suggestion has been followed is received. In step 606, it is determined whether the suggestion has been followed. If the suggestion has been followed, then in step 608, the reward provided to the person who followed the suggestion may be increased. Alternatively or additionally, in step 608, positive feedback may be provided to the driver suggestion system 110 indicating that a particular suggestion has been followed, and therefore a particular person may follow the suggestion again in the future. If the suggestion has not been followed, then in step 610, the reward provided to the person who did not follow the suggestion may be reduced. Alternatively or additionally, in step 610, negative feedback may be provided to the driver suggestion system 110 indicating that a particular suggestion has not been followed, and therefore a particular person may not follow the suggestion in the future.

[0073] Figure 7 It is a block diagram used to describe additional details of how to detect the emotional state, fatigue state, mental state, and alertness state of a person (e.g., the driver and each passenger who is a potential driver). Figure 7 The box in the middle can be Figure 2 This refers to a portion of the driver analyzer 215 and / or the passenger analyzer 225 described herein. More generally, Figure 7 The boxes in the text may be part of the driver suggestion system 110.

[0074] See Figure 7 , Figure 7 The face detection block 710 (also known as face detector 710) and the facial expression recognition block 712 (also known as facial expression recognizer 712) are shown. Figure 7 The diagram also shows a skeleton detection block 714 (also known as a skeleton detector 714) and a pose recognition block 716 (also known as a pose recognizer 716). Figure 7 As shown, the face detector 710 and the skeleton detector 714 receive video signal 702. The video signal 702 may be, for example, a video signal of the driver (or passenger) received from the camera 205 of the driver capture subsystem 102 (or the passenger capture subsystem 103). Figure 7 The diagram also shows an audio signal processing block 724 (also known as an audio signal processor 724 or an audio signal analyzer 724) and another signal processing block 744. The audio signal 722 may be, for example, an audio signal of the driver (or passenger) received from the microphone 206 of the driver capture subsystem 102 (or the passenger capture subsystem 103).

[0075] According to some embodiments, the face detector 710 can detect faces within an image and also detect facial features within the image. The face detector 710 can use developed (or to be developed) computer vision techniques to detect such facial features. For example, a Hue-Saturation-Value (HSV) color model or some other computer vision technique can be used to detect faces in an image; a feature detection model or some other computer vision technique can be used to identify facial features, such as, but not limited to, eyes, nose, lips, chin, cheeks, eyebrows, forehead, and / or similar facial features. Feature detection can also be used to detect wrinkles in specific facial regions, such as wrinkles on the forehead, sides of the mouth, and / or around the eyes. In some embodiments, bounding boxes can be used to identify faces and their facial features. Some features to be identified may be contained within other features, such as a user's eyes. In this case, consecutive bounding boxes can be used to first identify the contained features (e.g., the face) and then identify the included features (e.g., each eye in a pair of eyes). In other embodiments, a single bounding box can be used to identify each distinct feature. In some embodiments, one or more algorithm libraries, such as the OpenCV (http: / / opencv.willowgarage.com / wiki / ) computer vision library and / or the Dlib algorithm library (http: / / dlib.net / ), can be used to recognize these facial features and generate bounding boxes. In some embodiments, the bounding boxes do not have to be rectangular; they can be other shapes, such as, but not limited to, ellipses. In some embodiments, machine learning techniques (e.g., boosting) can be used to improve the confidence in the detection of facial features (e.g., eyes, nose, lips, etc.). More generally, deep neural networks (DNNs) and / or other computer models can be trained using datasets to detect facial features from images, and the trained DNNs and / or other computer models can therefore be used for facial feature recognition.

[0076] Once the face detector 710 identifies (also known as detects) a facial feature, the facial expression recognizer 712 can determine a person's facial expression. Generally, the human face is composed of different parts, such as the chin, mouth, eyes, and nose, as mentioned above. The shape, structure, and size of these facial features vary with changes in facial expressions. Furthermore, wrinkles at specific facial locations may change with certain facial expressions. For example, the shape of a person's eyes and mouth can be used to distinguish different facial expressions, as can wrinkles on a person's forehead and / or other features. This is at least in part based on the detected facial expressions of the person.

[0077] The skeleton detector 714 can use a skeleton detection model or some other computer vision technique to identify human body parts and joints, such as, but not limited to, arms, hands, elbows, wrists, and / or similar human body parts and joints. The pose recognizer 716 can detect specific poses, such as whether a person is holding the steering wheel of a vehicle with both hands while driving, or whether a person is raising one arm and making a fist while driving. Deep neural networks (DNNs) and / or other computer models can be trained using datasets to detect human poses from images; the trained DNNs and / or other computer models can therefore be used for pose recognition.

[0078] Once the skeleton detector 714 detects a human body part in the image, the pose recognizer 716 can determine the person's pose. Generally, the human body consists of different parts such as the head, neck, torso, upper arms, elbows, forearms, wrists, and hands. Under specific poses, the overall and relative positions and orientations of these parts may change. For example, when a person is driving a vehicle, they will often place both hands on the steering wheel, but if they are angry, they might raise one arm and clench their fist because another driver caused them to stop, make a sharp turn, and / or similar actions.

[0079] As mentioned above, in Figure 7 In this embodiment, the audio signal analyzer 724 receives an audio signal 722. The audio signal 722 may be, for example, the audio signal of the driver (or passenger). The audio signal analyzer 724 can analyze the audio signal 722 to detect various characteristics of the audio signal 722 that can vary with a person's emotional state, fatigue, mental state, and / or alertness. Examples of such audio characteristics include pitch, vibrato, and tone variation. Pitch is related to the frequency of the signal and can therefore be quantified as a frequency. Variations in the pitch of a person's voice are generally associated with a person's arousal state, or more generally, their emotional state. For example, an increase in pitch is generally associated with a high arousal state, such as anger, joy, or fear, while a decrease in pitch is generally associated with a low arousal state, such as sadness or calmness. Vibrato is a periodic modulation of the pitch (e.g., the fundamental frequency) of a person's voice, occurring at a given rate and depth. Vibrato is also associated with shaking, which is often related to changes in mood. An increase in pitch fluctuations results in vibrato, which, for example, can indicate an increase in the degree of happiness, pain, or fear. Pitch variation is a rapid modification of pitch at the beginning of each sentence, raising it several semitones above the target pitch but quickly decaying back to the normal value. The use of pitch variation leads to increased pitch variation, which is associated with high emotional intensity and positive valence.

[0080] The outputs of the face detector 710, the facial expression recognizer 710, the posture recognizer 716, and the audio signal processor 724 can be provided to the emotion detector 750, the fatigue detector 752, the mental state detector 754, and the alertness detector 756. Each of the emotion detector 750, the fatigue detector 752, the mental state detector 754, and the alertness detector 756 can be implemented using one or more processors (e.g., 108). The detectors 750, 752, 754, and 756 can also receive one or more physiological signals 742 after selective processing by the signal processor 744. As described above, such physiological signals can provide information about heart rate, body temperature, blood pressure, and / or similar physiological signals. Such physiological information can be used to train one or more deep neural networks (DNNs) and / or other computer models to determine how this information and / or changes therein relate to emotional state, fatigue state, mental state, and / or alertness.

[0081] Each of the detectors 750, 752, 754, and 756 can use one or more look-uptable (LUT) tables to quantify the subject's emotional state, fatigue state, mental state, and alertness state. Furthermore, the physiological signal 742 can be provided to one or more such detectors in the physiological sensors 204, examples of which have been discussed above, and such signals can be processed by the signal processor 744 before being provided to one or more of the detectors 750, 752, 754, and 756. The output of the facial expression analyzer 712 can specify one or more facial expression features of a person determined based on the human video signal 702, the output of the pose recognizer 716 can specify one or more body postures of a person determined based on the human video signal 702, and the output of the audio signal analyzer 724 specifies one or more audio features determined based on the audio signal 722. Conversely, or in addition to using a LUT, one or more of the detectors 750, 752, 754, and 756 may be implemented by one or more deep neural networks (DNNs) and / or one or more other computer models trained on training data, including facial expression training data, body posture training data, speech training data, and / or other training data. One or more of the detectors 750, 752, 754, and 756 may analyze the driver's (and one or more passengers') blinking, gaze, head posture, hand posture, voice features, and / or similar characteristics to quantify the person's emotional state, fatigue state, mental state, and alertness state, respectively.

[0082] The outputs of various detectors 750, 752, 754, and 756 are shown as being provided to a preferred driver selector 760. The preferred driver selector 760 may be implemented using one or more processors (e.g., 108) implementing a DNN and / or one or more computer models for identifying preferred drivers based on various conditions detected by the various detectors 750, 752, 754, and 756.

[0083] Sometimes, arguments arise during relaxed and enjoyable journeys regarding who should or shouldn't drive the vehicle. This can lead to quarrels between friends and / or family members and potentially traffic accidents. Embodiments of this technology can be used to avoid such arguments, for example, if people agree to follow suggestions generated by the system described herein before the journey begins. Furthermore, if the driver and all passengers are drowsy or in an unfit state to drive, the system described herein can advise all occupants to rest. This prevents one person from pretending to be capable of taking responsibility for driving.

[0084] Embodiments of this technology are particularly useful when strangers agree to share a rental vehicle. For example, suppose an app and / or service arranges for strangers to rent a car together, and they are all valid drivers. In this situation, these strangers are unaware of each other's driving abilities. Embodiments of this technology can be used to determine who should drive the rental vehicle first and when one of the passengers should take over driving. Through these embodiments, drivers can be smoothly selected in a way that satisfies everyone, avoiding the awkwardness of having to decide for themselves who should drive and for how long.

[0085] Embodiments of this technology can also be used to match drivers with similar or complementary abilities and experience using driving history to create a group of truck drivers. For example, if some long-haul truck drivers have experience driving during the day while others have experience driving at night, embodiments of this technology can be used to match such drivers and / or recommend when one of the truck drivers should be driving while the others rest, or vice versa.

[0086] Some embodiments of the technology described herein can be implemented using hardware, software, or a combination of both. The software used is stored on one or more processor-readable storage devices described above to program one or more processors to perform the functions described herein. The processor-readable storage device may include computer-readable media, such as volatile and non-volatile media, removable and non-removable media. By way of example and not limitation, the computer-readable medium may include computer-readable storage media and communication media. The computer-readable storage medium may be implemented in any method or technique for storing information such as computer-readable instructions, data structures, program modules, or other data. Examples of computer-readable storage media include RAM, ROM, EEPROM, flash memory or other storage technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic tape cassette, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible by a computer. Computer-readable media do not include propagated signals, modulated signals, or transient signals.

[0087] Communication media typically embody computer-readable instructions, data structures, program modules, or other data in a data signal propagated by a carrier wave or other transmission mechanism, a modulated data signal, or a transient data signal, and include any information transmission medium. The term "modulated data signal" refers to a signal whose one or more characteristics are set or altered in a manner used to encode information in the signal. By way of example and not limitation, the communication medium includes wired media (e.g., wired networks or direct wired connections) and wireless media (e.g., RF and other wireless media). Any combination of the above is also included within the scope of computer-readable media.

[0088] In alternative embodiments, some or all of the software may be replaced with dedicated hardware logic components. For example, but not limited to, illustrative types of hardware logic components that may be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SOCs), complex programmable logic devices (CPLDs), dedicated computers, etc. In one embodiment, software implementing one or more embodiments (stored on a storage device) is used to program one or more processors. The one or more processors may communicate with one or more computer-readable media / storage devices, peripherals, and / or communication interfaces.

[0089] It should be understood that the present invention can be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that the subject matter will be thorough and complete, and will fully convey the disclosure to those skilled in the art. In fact, the subject matter is intended to cover alternatives, modifications, and equivalents to these embodiments that are included within the spirit and scope of the disclosure of the subject matter as defined by the appended claims. Furthermore, numerous specific details are set forth in the following detailed description of the subject matter in order to provide a thorough understanding of the subject matter. However, it will be apparent to those skilled in the art that the subject matter claimed can be practiced without such specific details.

[0090] Various aspects of this disclosure have been described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to generate a machine, such that the instructions, which execute via the processor of said computer or other programmable instruction execution apparatus, create mechanisms for implementing the functions / actions specified in the flowcharts and / or block diagrams.

[0091] The description presented in this disclosure is for illustrative and descriptive purposes and is not intended to be exhaustive or limited to the forms disclosed. Various modifications and alterations will be apparent to those skilled in the art without departing from the scope and spirit of the invention. The various aspects of the invention were chosen and described in order to better explain the principles and practical application of the invention and to enable those skilled in the art to understand the various modifications to the invention suitable for the particular intended use.

[0092] This disclosure has been described in conjunction with various embodiments. However, other variations and modifications to the disclosed embodiments can be understood and implemented by studying the drawings, the disclosure, and the appended claims, and such variations and modifications should be interpreted as being included in the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" does not exclude a plurality.

[0093] For the purposes of this document, it should be noted that the dimensions of the various features depicted in the figures are not necessarily drawn to scale.

[0094] For the purposes of this document, the description may refer to "one embodiment," "some embodiments," or "another embodiment" to describe different or the same embodiments.

[0095] For the purposes of this document, a connection can be direct or indirect (e.g., via one or more other parts). In some cases, when a component is said to be connected to or coupled to another component, that component can be directly connected to the other component or indirectly connected to the other component via an intermediate component. When a component is said to be directly connected to another component, then there is no intermediate component between that component and said other component. If two devices are directly or indirectly connected, they are in "communication," so they can transmit electrical signals to each other.

[0096] For the purposes of this document, the term “based on” may be understood as “at least partially based on”.

[0097] For the purposes of this document, the use of numerical terms such as “first” object, “second” object, and “third” object without additional context may not imply an order of objects, but may be used to identify different objects for distinction.

[0098] The detailed description above has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the inventive subject matter to the precise forms disclosed. Many modifications and variations are possible in accordance with the foregoing teachings. The various embodiments selected and described are intended to better explain the principles of the disclosed technology and its practical application, thereby enabling those skilled in the art to better utilize the techniques of the various embodiments and various modifications suited to the intended particular use.

[0099] The scope is intended to be defined by the appended claims.

[0100] Although the subject matter has been described in language specific to structural features and / or method actions, it should be understood that the subject matter defined in the claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are disclosed as exemplary forms for implementing the claims.

Claims

1. A human-machine interaction method, characterized in that, include: Acquire current data of the vehicle driver and current data of each of the vehicle’s one or more passengers who are potential drivers from one or more sensors; Historical data of the vehicle's driver and historical data of each of the one or more passengers in the vehicle who are potential drivers are obtained from one or more databases. Analyzing the current and historical data of the vehicle driver and the current and historical data of each of the one or more passengers in the vehicle who are potential drivers, thereby identifying preferred candidates more favorable to the safe driving of the vehicle, wherein the analysis is performed using one or more computer-implemented models; and Output a suggestion for the preferred candidate to drive the vehicle; The current data of the vehicle driver includes one or more of the following: current image data; current video data; current audio data; current heart rate data; current body temperature data; current driving speed data; or current driving performance data; The current data of each of the one or more passengers in the vehicle that is a potential driver includes one or more of the following: current image data; current video data; current audio data; current heart rate data; and current body temperature data; The historical data of the vehicle driver and each of the one or more passengers of the vehicle as potential drivers includes one or more of the following: historical driving record data; historical driving performance data; driver's license type data; historical driving time data; or historical rest time data.

2. The human-machine interaction method of claim 1, wherein, Also includes: Obtain feedback data indicating whether the recommendations have been followed; as well as At least one of the one or more computer-implemented models is updated based on the feedback data.

3. The human-computer interaction method according to claim 2, characterized in that, If the feedback data indicates that the recommendation has not been followed, the method further includes: Ask one or more questions to the vehicle driver and / or the preferred candidate.

4. The human-computer interaction method according to claim 2, characterized in that, If the feedback data indicates that the recommendation has not been followed, the method further includes: Obtain the reason why the driver and / or the preferred candidate did not accept the recommendation.

5. The interaction method described above, the human-computer interaction method according to claim 1 or 2, characterized in that, The analysis of the current and historical data of the vehicle driver and the current and historical data of each of the one or more passengers of the vehicle as a potential driver includes: quantifying one or more of the following for the driver and each of the one or more passengers of the vehicle as a potential driver: Mental state; Emotional state; Fatigue state; or A state of alertness.

6. The human-computer interaction method according to claim 1 or 2, characterized in that, The analysis of the current and historical data of the vehicle driver and the current and historical data of each of the one or more passengers in the vehicle who are potential drivers further includes: It was determined that there was no suitable preferred candidate to drive the vehicle; Provide suggestions for taking a break.

7. A human-computer interaction system, characterized in that, include: One or more sensor interfaces for acquiring current data of the vehicle driver and current data of each of one or more passengers in the vehicle who are potential drivers from one or more sensors. One or more database interfaces for retrieving historical data of the vehicle driver and historical data of each of the one or more passengers of the vehicle who are potential drivers from one or more databases; as well as One or more processors, for: Analyzing the current and historical data of the vehicle driver and the current and historical data of each of the one or more passengers in the vehicle who are potential drivers, thereby identifying preferred candidates that are more conducive to the safe driving of the vehicle; and Output a suggestion for the preferred candidate to drive the vehicle; The current data of the vehicle driver includes one or more of the following: current image data; current video data; current audio data; current heart rate data; current body temperature data; current driving speed data; or current driving performance data; The current data of each of the one or more passengers in the vehicle that is a potential driver includes one or more of the following: current image data; current video data; current audio data; current heart rate data; and current body temperature data; The historical data of the vehicle driver and each of the one or more passengers of the vehicle as potential drivers includes one or more of the following: historical driving record data; historical driving performance data; driver's license type data; historical driving time data; or historical rest time data.

8. The human-computer interaction system according to claim 7, characterized in that, The one or more processors implement one or more computer-implemented models for analyzing the current and historical data of the vehicle driver and the current and historical data of each of the one or more passengers of the vehicle as potential drivers, thereby identifying preferred candidates to drive the vehicle.

9. The human-computer interaction system according to claim 7 or 8, characterized in that, The one or more processors are also used for: Obtain feedback data indicating whether the recommendations have been followed; and At least one of the one or more computer-implemented models is updated based on the feedback data.

10. The human-computer interaction system according to claim 9, characterized in that, The one or more processors are also used for: If the feedback data indicates that the recommendations have not been followed, one or more questions shall be asked to the vehicle driver and / or the preferred candidate.

11. The human-computer interaction system according to claim 9, characterized in that, The one or more processors are also used for: If the feedback data indicates that the recommendation has not been followed, determine the reason why the driver and / or the preferred candidate did not accept the recommendation.

12. The human-computer interaction system according to claim 7 or 8, characterized in that, In order to analyze the current and historical data of the vehicle driver and the current and historical data of each of the one or more passengers of the vehicle as a potential driver, the one or more processors are configured to: quantify one or more of the following for the driver and each of the one or more passengers of the vehicle as a potential driver: Mental state; Emotional state; Fatigue state; or A state of alertness.

13. The human-computer interaction system according to claim 7 or 8, characterized in that, The one or more processors are further configured to: analyze the current data and historical data of the vehicle driver, and the current data and historical data of each of the one or more passengers of the vehicle who are potential drivers. If it is determined that there is no suitable preferred candidate to drive the vehicle, a rest recommendation is output.

14. The human-computer interaction system according to claim 7 or 8, characterized in that: The one or more sensors include one or more cameras for acquiring current image data of the vehicle driver and current image data of each of the one or more passengers in the vehicle who are potential drivers. The one or more database interfaces are used to obtain historical image data of the vehicle driver and historical image data of each of the one or more passengers of the vehicle who are potential drivers from one or more databases. The one or more processors are used for: The preferred candidate to drive the vehicle is identified by analyzing the current image data and the historical image data of the vehicle driver, as well as the current image data and the historical image data of each of the one or more passengers in the vehicle who are potential drivers.

15. A non-transitory computer-readable medium storing computer instructions, wherein, When the computer instructions are executed by one or more processors, the one or more processors perform a method comprising the following steps: Acquire current data of the vehicle driver and current data of each of the vehicle’s one or more passengers who are potential drivers from one or more sensors; Historical data of the vehicle's driver and historical data of each of the one or more passengers in the vehicle who are potential drivers are obtained from one or more databases. Analyzing the current and historical data of the vehicle driver and the current and historical data of each of the one or more passengers in the vehicle who are potential drivers, thereby identifying preferred candidates that are more conducive to the safe driving of the vehicle; and Output a suggestion for the preferred candidate to drive the vehicle; The current data of the vehicle driver includes one or more of the following: current image data; current video data; current audio data; current heart rate data; current body temperature data; current driving speed data; or current driving performance data; The current data of each of the one or more passengers in the vehicle that is a potential driver includes one or more of the following: current image data; current video data; current audio data; current heart rate data; and current body temperature data; The historical data of the vehicle driver and each of the one or more passengers of the vehicle as potential drivers includes one or more of the following: historical driving record data; historical driving performance data; driver's license type data; historical driving time data; or historical rest time data.

16. The non-transitory computer-readable medium according to claim 15, characterized in that, The method further includes: Obtain feedback data indicating whether the recommendations have been followed; and The one or more computer-implemented models used to perform the analysis are updated based on the feedback data.

17. The non-transitory computer-readable medium according to claim 16, characterized in that, The method further includes: If the feedback data indicates that the recommendations have not been followed, one or more questions shall be asked to the vehicle driver and / or the preferred candidate.

18. The non-transitory computer-readable medium according to claim 16, characterized in that, The method further includes: If the feedback data indicates that the recommendation has not been followed, determine the reason why the driver and / or the preferred candidate did not accept the recommendation.

19. The non-transitory computer-readable medium according to claim 15 or 16, characterized in that, The analysis of the current and historical data of the vehicle driver and the current and historical data of each of the one or more passengers of the vehicle as a potential driver includes: quantifying one or more of the following for the driver and each of the one or more passengers of the vehicle as a potential driver: Mental state; Emotional state; Fatigue state; or A state of alertness.

20. The non-transitory computer-readable medium according to claim 15 or 16, characterized in that, The method further includes: Analyze the current and historical data of the vehicle driver, as well as the current and historical data of each of the one or more passengers in the vehicle who are potential drivers. If it is determined that there is no suitable preferred candidate to drive the vehicle, a rest recommendation is output.

Images