Method for fleet management and driver analysis based on driver and scene monitoring and understanding

Abstract

Methods and systems for fleet management for driver analysis of a vehicle are provided. In one embodiment, a disclosed system includes one or more vehicle sensors, one or more transceivers, and one or more processors. The one or more vehicle sensors are configured to collect sensor data relating to a cognitive load of a driver of the vehicle. The one or more transceivers are configured to transmit sensor data via a wireless communication network to a remote server physically remote from the plurality of vehicles. The one or more processors are configured to at least facilitate: aggregating the internal data and the external data for all of the plurality of vehicles in the fleet, generating aggregated fleet data; and providing one or more control actions for a plurality of vehicles of the fleet based on the aggregated fleet data.

Description

TECHNICAL FIELD

[0001] The technical field is generally related to the collection of data, including using data collected from a fleet of vehicles and their drivers. BACKGROUND

[0002] A fleet of vehicles can encounter many potential situations, including external events as well as internal events, such as situations that can affect a driver’s cognitive load and / or other parameters related to vehicles in the fleet. However, existing technology can not always be optimal, for example in terms of using data about potential situations for managing and controlling vehicles in the fleet.

[0003] Accordingly, it is desirable to provide improved methods and systems for generating and utilizing data related to vehicles in a fleet and their surroundings, including for managing and controlling the fleet. SUMMARY

[0004] According to an exemplary embodiment, a method is provided, comprising: collecting, via one or more vehicle sensors, sensor data related to a cognitive load of a driver of a vehicle; transmitting, via one or more transceivers, the sensor data to a remote server physically remote from the plurality of vehicles via a wireless communication network; and determining, via one or more processors, the cognitive load for the driver based on the sensor data; and providing, based on the cognitive load, one or more actions related to the driver of the vehicle in accordance with instructions provided by the one or more processors.

[0005] Further in exemplary embodiments, the step of collecting sensor data includes collecting, via one or more internal vehicle sensors, internal data related to conditions within a cabin of each of the plurality of vehicles of the fleet that can affect the cognitive load; collecting, via one or more external vehicle sensors, external data related to conditions outside the cabin of each of the plurality of vehicles of the fleet that can affect the cognitive load; the step of transmitting sensor data includes transmitting the internal data and the external data to the remote server via the wireless communication network; and the method further includes aggregating, via the one or more processors, the internal data and the external data for all of the plurality of vehicles in the fleet, generating aggregated fleet data; and providing, based on the aggregated fleet data, one or more driver-related control actions for the plurality of vehicles of the fleet in accordance with instructions provided by the one or more processors.

[0006] Further in example embodiments, the method further comprises: determining, via the one or more processors, one or more external events related to the cognitive load for each of the plurality of vehicles in the fleet; determining, via the one or more processors, one or more internal conditions related to the cognitive load for each of the plurality of vehicles in the fleet, including internal distracting things likely to cause tension, distraction, or both, in the driver; and for each of the plurality of vehicles in the fleet, correlating, via the one or more processors, the one or more internal conditions with the one or more external events.

[0007] Further in example embodiments, the method further comprises: obtaining, via one or more input sensors, anonymous feedback from the driver of each of the plurality of vehicles of the fleet, including via driver interaction with a display screen, verbal feedback from the driver, or both; wherein the correlating is further performed via the one or more processors using the anonymous feedback.

[0008] Further in example embodiments, the method further comprises: correlating, via the one or more processors, the one or more external events with the cognitive load based on respective timestamps provided to the one or more external events and to the cognitive load via the one or more processors.

[0009] Further in example embodiments, the cognitive load is calculated using internal data obtained via one or more biometric sensors for the driver.

[0010] Further in example embodiments, the cognitive load is further calculated using operational data obtained via one or more vehicle sensors related to operation of the driver of a respective one of the plurality of vehicles of the fleet.

[0011] Further in example embodiments, the method further comprises providing, via the display screen, a display for an administrator of the fleet in accordance with instructions provided by the one or more processors, the display including information about the one or more external events and the cognitive load, including a mapping of the one or more external events with the cognitive load along one or more routes taken by the plurality of vehicles in the fleet, and wherein the display includes a plurality of cognitive load markers corresponding to respective regions of interest along the one or more routes and their corresponding cognitive loads.

[0012] Further in example embodiments, the method further comprises: determining, via the processor, based on the sensor data, whether the cognitive load of the driver was triggered by an internal condition within the vehicle or, alternatively, by an external condition outside the vehicle; and providing, via instructions provided by the processor, a notification about the cognitive load and one or more causes thereof, including whether the cognitive load was deemed to be caused by an external condition or an internal condition.

[0013] In another example embodiment, a system is provided that includes one or more vehicle sensors, one or more transceivers, and one or more processors. The one or more vehicle sensors are configured to collect sensor data related to a cognitive load of a driver of a vehicle. The one or more transceivers are configured to transmit the sensor data to a remote server physically remote from the plurality of vehicles via a wireless communication network. The one or more processors are configured to facilitate at least: aggregating internal data and external data for all of the plurality of vehicles in a fleet of vehicles to generate aggregated fleet data; and providing one or more control actions for the plurality of vehicles of the fleet based on the aggregated fleet data.

[0014] Further in example embodiments, the one or more sensors include: one or more internal vehicle sensors configured to collect internal data about conditions within a cabin of each of the plurality of vehicles of the fleet that can impact the cognitive load; one or more external vehicle sensors configured to collect external data about conditions outside the cabin of each of the plurality of vehicles of the fleet that can impact the cognitive load; the one or more transceivers are configured to transmit the internal data and the external data to the remote server via the wireless communication network; and the one or more processors are configured to facilitate at least: aggregating the internal data and the external data for all of the plurality of vehicles in the fleet to generate the aggregated fleet data; and providing one or more driver-related control actions for the plurality of vehicles of the fleet based on the aggregated fleet data via instructions provided by the one or more processors.

[0015] Further in example embodiments, the one or more processors are further configured to facilitate at least: determining one or more external events related to the cognitive load for each of the plurality of vehicles in the fleet; determining one or more internal conditions related to the cognitive load for each of the plurality of vehicles in the fleet; and correlating the one or more internal conditions with the one or more external events for each of the plurality of vehicles in the fleet.

[0016] Further in example embodiments, the system further includes one or more input sensors configured to obtain anonymous feedback from the driver of each of the plurality of vehicles of the fleet, including via interaction of the driver with a display screen, verbal feedback from the driver, or both; and the one or more processors are configured to further use the anonymous feedback to perform the correlating.

[0017] Further in example embodiments, the one or more processors are further configured to facilitate at least correlating the one or more external events with the cognitive load based on respective timestamps provided by the one or more processors to the one or more external events and to the cognitive load.

[0018] Further in example embodiments, the one or more processors are further configured to compute the cognitive load using internal data obtained via one or more biometric sensors for the driver.

[0019] Further in example embodiments, the one or more processors are further configured to compute the cognitive load using operational data obtained via the one or more vehicle sensors relating to operation of the driver of the respective one of the plurality of vehicles of the fleet.

[0020] Further in example embodiments, the system further comprises a display screen; and the one or more processors are further configured to facilitate at least providing, via the display screen, a display for an administrator of the fleet comprising information about the one or more external events and the cognitive load, including a mapping of the one or more external events and the cognitive load along the one or more routes taken by the plurality of vehicles in the fleet, in accordance with the instructions provided by the one or more processors.

[0021] Further in example embodiments, the display comprises information about the one or more external events and the cognitive load, including a mapping of the one or more external events and the cognitive load along the one or more routes taken by the plurality of vehicles in the fleet.

[0022] Further in example embodiments, the processors are further configured to facilitate at least: determining, based on the sensor data, whether the cognitive load of the driver is triggered by an internal situation within the vehicle or, alternatively, by an external situation outside the vehicle; and providing a notification about the cognitive load and one or more causes thereof, including whether the cognitive load is deemed to be caused by an external situation or an internal situation.

[0023] In another example embodiment, a system is provided, the system comprising a fleet comprising a plurality of vehicles, each of the plurality of vehicles comprising: one or more internal vehicle sensors configured to collect internal data about situations within a cabin of the vehicle relating to a cognitive load of a driver of the vehicle; one or more external vehicle sensors configured to collect external data about situations outside the cabin of the vehicle relating to the cognitive load of the driver of the vehicle; one or more transceivers configured to transmit the internal data and the external data via a wireless communication network; and a remote server physically remote from the plurality of vehicles and comprising a transceiver configured to receive the internal data and the external data from each of the plurality of vehicles of the fleet via the wireless communication network; and one or more processors configured to facilitate at least: aggregating the internal data and the external data for all of the plurality of vehicles in the fleet, generating aggregated fleet data; and providing one or more driver-related control actions for the plurality of vehicles of the fleet based on the aggregated fleet data and the cognitive load. BRIEF DESCRIPTION OF DRAWINGS

[0024] The present disclosure will be described below in connection with the following drawings, in which like numbers indicate like elements, and in which:

[0025] Figure 1 is a functional block diagram of a system according to exemplary embodiments, the system comprising: a fleet of vehicles, representing a fleet of vehicles; and a remote server configured for generating and aggregating data from the entire fleet relating to the vehicles in the fleet and their external environment, and which can be used via the remote server to manage and control the vehicles in the fleet;

[0026] Figure 2 is a flowchart of a process for generating and aggregating data from a fleet of vehicles and their external environment, and which can be used to manage and control the vehicles in the fleet, and which can be implemented in connection with Figure 1 the system according to exemplary embodiments;

[0027] Figure 3 and Figure 4 provides an exemplary illustration of a step of the process of Figure 2 according to exemplary embodiments (i.e., obtaining driver feedback); and

[0028] Figure 5 provides an exemplary illustration of another step of the process of Figure 2 according to exemplary embodiments (i.e., providing a display for a fleet administrator). DETAILED DESCRIPTION

[0029] The following detailed description is merely exemplary in nature and is not intended to limit the disclosure or the application and uses of it. Furthermore, there is no intention to be bound by any theory of operation presented in the preceding background or the following detailed description.

[0030] Figure 1 is illustrated. In various embodiments, and as described below, the system 10 includes a fleet of vehicles 100 and a remote server 170. In various embodiments, the depicted vehicles 100 are represented as representative vehicles of the fleet of vehicles 100. Further in various embodiments, the system 10 provides for generating and aggregating data from the entire fleet relating to the vehicles 100 in the fleet and their external environment, and which can be used via the remote server 170 to manage and control the vehicles 100 in the fleet. In various embodiments, the system 10 performs these tasks according to the process 200 of Figure 2 and its Figures 3-5 implementations according to exemplary embodiments.

[0031] Specifically, as described in further detail below, in various embodiments, in addition to the characteristics of the driver of vehicle 100 (e.g., vital signs, other biometric information, camera images, and / or other information related to the driver, their actions, and their cognitive load), vehicle 100 collects information related to external events surrounding vehicle 100 (e.g., construction areas, school zones, weather conditions, road conditions, traffic conditions, etc.) and the operation of vehicle 100 (e.g., speed, acceleration, steering, etc.). Furthermore, in various embodiments, and as described in further detail below, system 10 processes various types of data, aggregates data from all vehicles 100 across the convoy, correlates the data with external events faced by the vehicles, maps the data along the routes taken by the vehicles 100 in the convoy, and sends the data to remote server 170, where the data is used to manage and control the vehicles 100 in the convoy. In various embodiments, the collection, processing, and use of data in this manner are via… Figure 1 One or more processors described herein (e.g., Figure 1 The processor 142 of the vehicle 100 and / or the processor 173 of the remote server 170 are used to execute this.

[0032] In various embodiments, vehicle 100 refers to one of several different vehicles 100 in a convoy of vehicles 100 operating on roads or other paths (collectively referred to herein as "roads"). Although in Figure 1 The image depicts a single vehicle 100, but it should be understood that system 10 may include any number of vehicles 100 working together and cooperating with remote server 170 to perform tasks. Figure 2 The process described in 200 and Figures 3-5 The implementation of this will be further described below. Additionally, while the singular term "vehicles" may sometimes be used, it will be understood that this refers to any number of different vehicles (e.g., in a convoy, or otherwise in System 10 and in the execution of...). Figure 2 Process 200 and Figures 3-5 They are used together in the implementation methods.

[0033] In various embodiments, each vehicle 100 includes an automobile. Vehicle 100 can be any of several different types of automobiles, such as, for example, a sedan, van, truck, or sport utility vehicle (SUV), and in some embodiments can be two-wheel drive (2WD) (i.e., rear-wheel drive or front-wheel drive), four-wheel drive (4WD), or all-wheel drive (AWD) and / or various other types of vehicles. In some embodiments, vehicle 100 may also include motorcycles or other vehicles, such as aircraft, spacecraft, boats, etc., and / or one or more other types of mobile platforms (e.g., robots and / or other mobile platforms).

[0034] In certain embodiments, some of the vehicles 100 (in a fleet) can be operated in whole or in part by human drivers, while other vehicles 100 can include autonomous or semi-autonomous vehicles, e.g., where vehicle control (including acceleration, deceleration, braking, and / or steering) is automatically planned and executed in whole or in part by the control system 102. Additionally, in certain embodiments, some vehicles 100 can be operated by humans at certain times and via automatic control at other times. Further in various embodiments, some of the vehicles 100 include automatic functionality via computer models that are trained using data generated and processed via the system 10 after protecting the privacy thereof.

[0035] In the depicted embodiment, the vehicle 100 includes a body 104 disposed on a chassis 116. The body 104 substantially encloses other components of the vehicle 100. The body 104 and the chassis 116 can collectively form a frame. The vehicle 100 also includes a plurality of wheels 112. The wheels 112 are each rotatably coupled to the chassis 116 near a respective corner of the body 104 to facilitate movement of the vehicle 100. In one embodiment, the vehicle 100 includes four wheels 112, although this can vary in other embodiments (e.g., for trucks and certain other vehicles).

[0036] The drive system 110 is mounted on the chassis 116 and drives the wheels 112, e.g., via axles 114. The drive system 110 preferably includes a propulsion system. In certain example embodiments, the drive system 110 includes an internal combustion engine and / or an electric motor / generator coupled with its transmission. In certain embodiments, the drive system 110 can vary, and / or two or more drive systems 110 can be used. By way of example, the vehicle 100 can also incorporate any one or combination of several different types of propulsion systems, such as gasoline- or diesel-fueled combustion engines, "flexible fuel vehicle" (FFV) engines (i.e., using a mixture of gasoline and alcohol), engines fueled by gaseous compounds (e.g., hydrogen and / or natural gas), combustion / electric motor hybrid engines, and electric motors.

[0037] As noted above, in certain embodiments, e.g., in certain situations, the vehicle 100 includes one or more functions that can be automatically controlled via the control system 102. In certain embodiments, some of the vehicles 100 can be operated by human drivers, while other vehicles 100 can be assisted and autonomous driving vehicles for automatic control of the drive system 110 and / or other vehicle components.

[0038] As Figure 1As depicted, in various embodiments, vehicle 100 also includes a braking system 106 and a steering system 108. In example embodiments, braking system 106 controls braking of vehicle 100 using braking components that are controlled via input provided by a driver (e.g., in certain embodiments via a brake pedal) and / or automatically controlled via control system 102. Also in example embodiments, steering system 108 controls steering of vehicle 100 via steering components (e.g., a steering wheel 109 that is part of a steering column coupled to wheel axles 114 and / or wheels 112) that are controlled via input provided by a driver (e.g., in certain embodiments via steering wheel 109) and / or automatically controlled via control system 102.

[0039] In Figure 1 In the illustrated embodiment, in certain embodiments, control system 102 is coupled to braking system 106, steering system 108, and drive system 110. In various embodiments, control system 102 at least facilitates generation, processing, and transmission of data for vehicle 100 (e.g., including external data related to conditions surrounding vehicle 100, internal data regarding a driver or passenger of vehicle 100, and operational data regarding operation and movement of vehicle 100).

[0040] As Figure 1 As depicted, in various embodiments, control system 102 includes a sensor array 120, a display 124 (e.g., including a display screen), a transceiver 126, and a controller 140.

[0041] In various embodiments, the sensor array 120 obtains sensor data for generating data. In various embodiments, the sensor array 120 includes one or more internal sensors 130 (such as internal cameras, biometric sensors, etc.) for monitoring within the cabin of the vehicle 100 (e.g., for monitoring a driver of the vehicle 100) and for receiving feedback from the driver of the vehicle 100 (e.g., via one or more input sensors such as touchscreens, knobs, microphones, one or more sensors of a smartphone and / or other electronic devices of the driver, etc.). Also in certain embodiments, the sensor array 120 can also include one or more other external sensors 132 (e.g., external radar, sonar, lidar, GPS and / or other satellite-based sensors, and / or other navigation sensors) for monitoring outside of the vehicle 100 (e.g., for monitoring weather conditions, road conditions, construction zones, school zones, traffic conditions, etc. of the environment surrounding the vehicle). Additionally, in certain embodiments, the sensor array 120 also includes one or more vehicle sensors 134 related to movement and operation of the vehicle 100 (e.g., vehicle positioning sensors, speed sensors, accelerometers, brake sensors, steering sensors, etc.).

[0042] In various embodiments, the vehicle 100 also includes a transceiver 126. In various embodiments, the transceiver 126 communicates with the remote server 170 with respect to data (including sensor data from the various sensors of the sensor array 120) and / or processing thereof.

[0043] In certain embodiments, the display 124 provides information for the driver and / or other passengers of the vehicle 100, e.g., related to feedback to be obtained by the driver and / or other passengers of the vehicle 100, and / or for communicating instructions and / or controls from the remote server 170, including those related to operation of the vehicle 100.

[0044] In various embodiments, the controller 140 is coupled to the sensor array 120 as well as the display 124 and the transceiver 126 (e.g., including for obtaining sensor data, processing sensor data, and sending sensor data to the remote server 170, as well as for receiving and implementing instructions from the remote server 170 for managing the vehicle 100 and / or for a fleet of vehicles 100).

[0045] In various embodiments, the controller 140 includes a computer system, and includes a processor 142, a memory 144, an interface 146, a storage device 148, and a computer bus 149. In various embodiments, the controller (or computer system) 140 obtains sensor data from the sensor array 120, and in certain embodiments, obtains additional information via the transceiver 126 (e.g., in certain embodiments, regarding weather, traffic, and / or road conditions as can also be obtained from one or more third party sources). In various embodiments, the controller 140 processes the data, including sensor data related to external conditions from the vehicle 100, internal conditions for the vehicle 100 (including conditions that can cause tension, distraction, or both, for the driver), and operation of the vehicle 100, and correlates the different types of data, and provides for transmission to a remote server 170 for management and control of the vehicle 100 in a fleet. In various embodiments, the controller 140 provides these and other functions according to the processes and implementations depicted in Figures 2-5 and as further described in connection therewith below.

[0046] In various embodiments, the controller 140 (and in certain embodiments, the control system 102 itself) is disposed within the body 104 of the vehicle 100. In one embodiment, the control system 102 is mounted on the chassis 116. In certain embodiments, the controller 140 and / or the control system 102 and / or one or more components thereof can be disposed outside of the body 104, for example on a remote server, in the cloud, or in other devices in which image processing is performed remotely. In certain embodiments, the controller 140 of the vehicle 100 also performs functions in cooperation with the remote server 170, as further described below.

[0047] It will be appreciated that the controller 140 can otherwise differ from the embodiments depicted in Figure 1 For example, the controller 140 can be coupled to or can otherwise utilize one or more remote computer systems and / or other control systems, for example as part of one or more of the above-described vehicle 100 devices and systems.

[0048] In the depicted embodiment, the computer system of controller 140 includes a processor 142, a memory 144, an interface 146, a storage device 148, and a bus 149. Processor 142 performs the computational and control functions of controller 140 and can comprise any type of processor or multiple processors, a single integrated circuit such as a microprocessor, or any suitable number of integrated circuit devices and / or circuit boards that cooperate to effect the functions of a processing unit. During operation, processor 142 executes one or more programs 150 contained in memory 144 and, as such, controls the general operation of controller 140 and the computer system of controller 140 (typically in conjunction with programs 150, such as Figures 2-5 the processes described herein, such as

[0049] Memory 144 can be any suitable type of memory. For example, memory 144 can include various types of dynamic random access memory (DRAM), such as SDRAM, various types of static RAM (SRAM), and various types of non-volatile memory (PROM, EPROM, and flash memory). In some examples, memory 144 is on and / or co-located with processor 142 on the same computer chip. In the depicted embodiment, memory 144 stores programs 150 described above, as well as one or more databases 155 (e.g., relating to data) and other stored values 156 (e.g., thresholds for processing of data as set forth in the processes and implementations depicted in Figures 2-5 and further described below in connection therewith, according to exemplary embodiments).

[0050] Bus 149 serves to transmit program code, data, status, and other information or signals between the various components of the computer system of controller 140. Interface 146 allows communication, for example, from system drives and / or another computer system to the computer system of controller 140, and can be implemented using any suitable method and apparatus. In one embodiment, interface 146 obtains various data from sensor array 120 and / or navigation system 122. Interface 146 can include one or more network interfaces that communicate with other systems or components. Interface 146 can also include one or more network interfaces that communicate with a technician, and / or one or more storage interfaces that connect to storage devices, such as storage device 148.

[0051] Storage device 148 can be any suitable type of storage device, including various different types of direct access storage and / or other memory devices. In one exemplary embodiment, storage device 148 includes a program product from which memory 144 can receive programs 150 that, when executed by processor 142, perform Figures 2-5one or more embodiments of processes and implementations as described further below in connection therewith. In another example embodiment, the program product can be directly stored in and / or otherwise accessed by the memory 144 and / or secondary storage devices (e.g., disk 157), such as referenced hereinafter.

[0052] The bus 149 can be any suitable physical or logical means of connecting computing system components. This includes, but is not limited to, direct hard-wired connections, fiber optics, infrared, and wireless bus technology. During operation, the program 150 is stored in the memory 144 and executed by the processor 142.

[0053] It will be appreciated that, while this example embodiment is described in the context of a fully functioning computer system, those skilled in the art will recognize the mechanisms of the present disclosure are capable of being distributed as a program product in one or more types of non-transitory computer-readable signal-bearing media having stored thereon instructions for execution by a computer processor such as the processor 142, such as a non-transitory computer-readable medium bearing those instructions for performing implemented functions described herein. Such program product can take any data le format and be deployed to be accessed by a computer system using any data transfer technology, such as communication network transmission, optical, electrical, olr magnetic transmission. A non-exhaustive list of such computer-readable signal-bearing media includes computer system RAM, read only memory (ROM), volatile memory, non-volatile memory, removable storage, optical storage, and any other tangible medium which is used to store and / or carry computer Figure 1 It will be appreciated that the computer system of the controller 140 can be coupled to or otherwise utilize one or more remote computer systems and / or other control systems in the depicted embodiment, for example.

[0054] With continued reference to Figure 1 In various embodiments, the vehicles 100 and the remote server 170 communicate via one or more communication networks 160. In various embodiments, the communication networks 160 can include one or more wireless communication networks (e.g., satellite-based, cellular, and / or any number of other different types of wireless communication networks).

[0055] Further in various embodiments, the remote server 170 is disposed remotely from each of the vehicles 100 (e.g., in a fleet) or at a significant physical distance from each of the vehicles 100. In various embodiments, as Figure 1The depicted remote server 170 includes one or more transceivers 172, a processor 173, and computer memory 174 storing a map database 176 and stored values 178, as well as a display 177 for displaying data to one or more fleet managers. In various embodiments, the transceivers 172 are used to communicate with the vehicles 100, including with respect to data and processing thereof, as well as for management and control thereof. As Figure 1 The depicted transceivers 172, processor 173, memory 174, database 176, stored values 178, and display 177 are similar or identical to corresponding features of the vehicles 100 (e.g., with respect to their transceivers 126, processor 142, memory 144, database 155, stored values 156, and display 124). Moreover, in certain embodiments, the processor 173 processes or facilitates processing of data from various vehicles 100 in a fleet, including aggregation and mapping of data, and likewise utilizes data to manage and control vehicles 100 in a fleet (e.g., for managing their operations, deliveries, routes, timing, etc.) in various embodiments (e.g., as further described below in connection with Figures 2-5 processes and implementations).

[0056] Figure 2 is a flowchart of a process 200 for generating and aggregating data from a fleet of vehicles and their external environment, and which can be used to manage and control vehicles in the fleet, according to exemplary embodiments. Figure 2 The process 200 of Figure 2 and Figure 3 and Figure 4 which depict illustrative examples of steps of obtaining driver feedback during the process 200, and Figure 5 which depict illustrative examples of steps of providing displays to fleet managers according to the process 200, are described in greater detail.

[0057] With continuing reference to Figure 2 In various embodiments, the process 200 begins at step 202. In various embodiments, the process 200 begins when a vehicle 100 begins operation. It will be understood that the steps of the process 200 are performed with respect to each of the vehicles 100 in a fleet, in exemplary embodiments.

[0058] In various embodiments, external data is collected (step 204). In various embodiments, the external data is collected via Figure 1external conditions (e.g., as obtained via one or more cameras, GPS systems, radar, lidar, and / or other external sensors 132 regarding road conditions, weather conditions, traffic conditions, construction zones, school zones, and / or other conditions regarding the road on which the vehicle 100 is traveling and / or other conditions external to the vehicle 100).

[0059] In various embodiments, internal data is collected (step 205). In various embodiments, the internal data is collected via Figure 1 internal sensors 130 of the vehicle 100, and is related to internal conditions for the vehicle 100 (including regarding the driver of the vehicle 100) (e.g., as obtained via one or more cameras, biometric sensors such as body sensors, wearable smart devices, and / or other sensors that obtain vital signs, other biometric information, photographs, and / or other information related to the driver of the vehicle 100), and is likely to affect the cognitive load of the driver (e.g., by causing potential stress and / or distraction of the driver).

[0060] In various embodiments, operational data is collected (step 206). In various embodiments, the operational data is collected via Figure 1 vehicle sensors 134 of the vehicle 100, and is related to movement and operation of the vehicle 100 (including steering, turning, speed, acceleration, deceleration, etc.) (e.g., as obtained via one or more speed sensors, accelerometers, steering sensors, and / or other vehicle sensors 134).

[0061] In various embodiments, a determination is made as to whether an external event has occurred (step 208), the external event including an external event that is likely to affect the cognitive load of the driver (e.g., by potentially contributing to stress and / or distraction of the driver). In various embodiments, the one or more processors (such as the processors 142 of the vehicle 100 and / or the processors 173 of the remote server 170) analyze the data (particularly the external data of step 205) when a determination is made that any external event has occurred and / or is occurring with respect to the vehicle 100 while the vehicle 100 is being operated, and that the external event is likely to affect the cognitive load of the driver (e.g., by causing potential stress and / or distraction of the driver). By way of example, such external events would occur outside of the cabin of the vehicle 100, and would include construction zones, school zones, heavy traffic, road congestion, traffic slowdowns, adverse weather conditions (e.g., rain, sleet, snow, hail, etc.), adverse road and / or driving conditions (such as narrow roads, winding roads, slippery surfaces, etc.), the presence of emergency vehicles, and / or one or more other such external events, among other possible external events.

[0062] In various embodiments, if it is determined in step 208 that one or more external events have occurred, a timestamp is made (step 210). In various embodiments, one or more processors, such as processor 142 of vehicle 100 and / or processor 173 of remote server 170, provide the timestamp for the external event and save the timestamp in computer memory, such as memory 144 of vehicle 100 and / or memory 174 of remote server 170, for further reference.

[0063] Further in various embodiments, after the timestamp is created in step 210, an evaluation is made regarding one or more internal conditions (step 212). In various embodiments, during step 212, a calculation is performed regarding the cognitive load on the driver of vehicle 100. In various embodiments, one or more processors, such as processor 142 of vehicle 100 and / or processor 173 of remote server 170, calculate the cognitive load based on sensor data, particularly the internal data of step 205 and the operational data of step 206. For example, in certain embodiments, the internal data of step 205 is used to estimate or calculate the cognitive load based on the driver's vital signs, such as heart rate, perspiration, and / or blood pressure (e.g., as obtained via one or more biometric sensors), and / or the driver's actions, such as rapid movements of the arms, head, etc. (e.g., as obtained via one or more cameras and / or other sensors). In certain embodiments, the cognitive load can also be estimated or calculated based on the driver's operation of the vehicle, such as reflected via speed, acceleration, deceleration, turns or steering, etc. obtained by vehicle sensors 134. Figure 1

[0064] In various embodiments, process 200 then proceeds to step 220, e.g., as described in further more detail below, at which time feedback is obtained from the driver.

[0065] Referring back to step 208, if instead it is determined that no external event is detected, then in various embodiments, process 200 instead proceeds to step 214. In various embodiments, a determination is made as to whether a high cognitive load is predicted for the driver. In various embodiments, this is predicted via one or more processors, such as processor 142 of vehicle 100 and / or processor 173, based on a comparison of the cognitive load (e.g., calculated in a similar manner as described above for step 212) to a predetermined cognitive load threshold (e.g., stored as one or more stored values 156 and / or 178 in memory 144 and / or 174 of vehicle 100 and / or remote server 170, respectively). Figure 1 Figure 1

[0066] ​​​In various embodiments, if it is determined in step 214 that high cognitive load is not predicted, the process returns to step 204 and continues to collect data in new iterations of steps 204-206. In various embodiments, steps 204-214 are thereafter repeated in various new iterations until a determination is made in an iteration of step 214 that high cognitive load is predicted.

[0067] In various embodiments, once a determination is made during an iteration of step 214 that high cognitive load is predicted, a timestamp is generated (step 216). In various embodiments, during step 216, one or more processors, such as processor 142 of vehicle 100 and / or processor 173 of remote server 170, provide a timestamp for the high cognitive load prediction, and save the timestamp in computer memory, such as memory 144 of vehicle 100 and / or memory 174 of remote server 170, for further reference.

[0068] Further in various embodiments, after step 216, a determination is subsequently made as to whether any external events having the same timestamp have been detected or reported (step 218). Specifically, in various embodiments, during step 218, a determination is made as to whether any external events have a timestamp that is the same or overlapping or sufficiently close (i.e., within a predetermined threshold amount of time) to the timestamp for the high cognitive load of step 216. In various embodiments, when such matching timestamps are found, these timestamps are used to correlate external and internal data together, such as with respect to correlating external events with high cognitive load, etc.

[0069] Additionally, in certain embodiments, external data regarding external events can be used to identify events outside of the fleet, for identifying events that are likely to cause cognitive load to rise among the fleet drivers, and thus to proactively avoid those events in the same manner as the internal identification process. For example, in certain embodiments, event data and analysis related to one fleet can be leveraged relative to other fleets, for example to take similar actions that have been successfully implemented in connection with similar events encountered by other vehicle fleets, etc.

[0070] In various embodiments, the process then proceeds to step 220 described above, in order to obtain driver feedback.

[0071] In various embodiments, during step 220, driver feedback is obtained. Specifically, in various embodiments, driver feedback confirming, confirming, correcting, providing an explanation for, and / or providing other information related to the applicable external and / or internal events (e.g., as determined with respect to steps 208-212 and / or 214-218) is obtained. In various embodiments, the driver is provided a notification to confirm, reject, and / or provide additional details (such as the type of external event, the cause of the high cognitive load, etc.). Also in various embodiments, the driver feedback is subsequently obtained via Figure 1 one or more internal sensors 130 (e.g., input sensors of the vehicle 100 and / or input sensors of the driver’s smartphone and / or other portable electronic device). In various embodiments, the feedback is provided in an anonymous manner such that the feedback will only be used in aggregate and will not be used against a specific driver. In certain embodiments, the driver feedback can be provided via the driver’s interaction with a display screen of the vehicle and / or one or more other devices such as the user’s smartphone and / or other portable electronic device. In certain other embodiments, the feedback can be provided verbally by the user (e.g., by the driver), for example, as determined and collected via one or more microphones of the vehicle 100 and / or one or more microphones of the user’s smartphone and / or other electronic device.

[0072] Additionally, in certain embodiments, by providing appropriate feedback, the drivers in the fleet are also able to identify themselves situations that they feel impacted their performance (e.g., via a screen interface, through voice feedback (prompted or unprompted) to the vehicle’s AI / VA (voice assistant), etc.). Accordingly, in certain embodiments, when a situation arises in which the driver of one of the vehicles of the fleet requests to leave voice feedback even though no corresponding event was previously detected, the system can use the same approach (e.g., by evaluating fleet vehicle and non-fleet vehicle data and external data) to identify any conditions that can be related to the driver’s feedback for use in identifying future events.

[0073] Additionally, in various embodiments, conditions observed at a particular location and timestamp can subsequently be used to model and avoid similar conditions in other locations and times, for example, by relying (in whole or in part) on analysis of external data related to vehicle and driver performance and cognitive load. Accordingly, for example, where the vehicles of a fleet have observed and responded to a number of events that do indeed impact the cognitive load of their drivers while on the job (e.g., driving), that fleet can then begin to proactively identify and avoid those conditions throughout their route and shift plans. In various embodiments, this will result in an improved and increased level of confidence in the external factors to be identified and avoided.

[0074] refer to Figure 3 and Figure 4 Exemplary illustrations are provided regarding obtaining driver feedback according to exemplary embodiments.

[0075] First, such as Figure 3 The depicted display 300 provides a first display 300 for providing an initial inquiry or notification to the driver. In various embodiments, the display 300 includes a map showing the current trajectory 301 of the vehicle 100, a radius of interest 302 around the vehicle 100, and other nearby locations along the map. In various embodiments, a notification 304 is provided requesting feedback. In exemplary embodiments, the notification 304 provides a message similar to, “Would you like to provide anonymous feedback?” or similar statements. Furthermore, in various embodiments, the display 300 includes additional information such as location and local temperature 306, and estimated time of arrival to the destination 308. In the depicted embodiments, in response to the notification 304, the driver can tap the notification 304 to provide feedback, or alternatively, in some embodiments, may also potentially tap a “decline” button 310 or a “maybe later” button 312. In some embodiments, potential situations 314 (e.g., regarding near misses and / or other situations) are also depicted in the display 300.

[0076] Next, as Figure 4 As depicted, in response to an inquiry from the first display 300, a second display 400 provides feedback for the driver. Specifically, in an exemplary embodiment, the second display 400 responds to the driver's click... Figure 3 A 304 notification appears after feedback is provided. Figure 4 The description, in some embodiments, depicts a new notification 404 for driver feedback. Specifically, in various embodiments, the new notification 404 provides the driver with the opportunity to choose one of several possible scenarios 405 (e.g., explanations) for a condition. For example, in an example where high cognitive load is identified, possible scenarios 405 may include, as well as others: (a) high traffic volume (406); (b) sensory fatigue (407); (c) construction (408); and / or (d) personal reasons (409). In various embodiments, the driver chooses one of these scenarios 405 to provide feedback. It will be understood that in other embodiments, the new notification 404 and / or scenario 405 may vary. For example, in some embodiments, the new notification 404 may indicate an external event, and scenario 405 may include (a) a school zone; (b) a construction zone; (c) adverse weather conditions; (d) peak-hour traffic, etc., as well as other possible scenarios in various embodiments.

[0077] Return to referenceFigure 2 The data is recorded (step 222). In various embodiments, all of the data and determinations of steps 204-220 (including but not limited to the external data of step 204, the internal data of step 205, the operational data of step 206, the external events and timestamps of steps 208-210, the cognitive load of steps 210-212, the timestamps of steps 216-218, and the feedback of step 220) are collected with respect to each of the vehicles 100 in the fleet and with respect to each of the locations in which the vehicles 100 travel. In various embodiments, the recorded data further includes correlations between the external events and the internal events, including but not limited to correlations and relationships between the external events and the cognitive load, including as described above. In various embodiments, all of the data is stored in computer memory (such as memory 144 and / or memory 174 of Figure 1 for further use, including for controlling and managing the vehicles 100 in the fleet. Figure 1

[0078] In various embodiments, a notification about the data is provided (step 224). Specifically, in various embodiments, all of the data from the vehicles 100 (including the sensor data, the external events, the cognitive load, the correlations therebetween, and the related data) is sent to the remote server 170 (e.g., using the communication network 160 via the transceiver 126 of the vehicles 100). In various embodiments, this (along with the other steps of the process 200) is performed by each of the vehicles 100 in the fleet.

[0079] ​In various embodiments, the data is aggregated and mapped (step 226). Specifically, in various embodiments, the data is aggregated among each of the vehicles 100 in the fleet of vehicles, and is mapped corresponding to each of the locations in which the vehicles 100 were traveling when the data was collected. Accordingly, in various embodiments, the aggregated and mapped data is generated for different vehicles 100 in the fleet of vehicles, in a similar manner, with respect to various external conditions and / or other internal conditions at particular geographic locations that can be impacting cognitive load, and / or for different vehicles that can be facing the same or similar cognitive load impacts from the same or similar conditions regardless of geographic location. For example, in various embodiments, it can be determined that different vehicles are facing external conditions at the same location, and / or are facing similar internal or external locations regardless of geographic location, etc., that impact the cognitive load of the respective vehicles 100 in a similar manner, etc. In certain other embodiments, the cognitive load can be aggregated based on cognitive load impacts and proximate causes from various external events (even if the events are not at the same time and / or place) for later determination of responsive actions, etc. In various embodiments, the data aggregation and mapping is performed by one or more processors, such as the processor 173 of Figure 1

[0080] In various cases, the aggregated and mapped data is displayed (step 227). Specifically, in various embodiments, the aggregated and mapped data is displayed for one or more administrators of the fleet of vehicles 100. In certain embodiments, the display is provided via the display 177 according to instructions provided by the processor 173 of Figure 1

[0081] Referring to Figure 5 , according to exemplary embodiments, an exemplary illustration of a display 500 corresponding to the display step 227 of the process 200 of Figure 2 Figure 5 ​​​As depicted, in example embodiments, the display 500 includes a direction of travel 501 of the fleet, and various regions of interest 502 along the direction of travel. In various embodiments, the size of the various regions of interest 502 can correspond to a magnitude of a respective cognitive load of the driver of the vehicle 100 that is traveling with respect to the particular region of interest 502 (e.g., where a larger circle represents a larger cognitive load, etc.). Further in various embodiments, one particular region of interest 503 is highlighted, and a notification 510 is provided with respect to the particular region of interest 503. In example embodiments, the notification 510 provides a notification (e.g., "elevated cognitive load detected") and a situation 511 related to the elevated cognitive load. For example, in one illustrative embodiment, the situation can include the following, among others: (a) heavy pedestrian activity (512); (b) a construction zone (513); (c) difficulty focusing on driving during a peak traffic period (516), etc.

[0082] Referring back to Figure 2 In various embodiments, one or more actions are taken (step 228). In various embodiments, based on the display of step 227, one or more fleet administrators can take one or more actions in step 228 to implement control and management of the vehicles 100 in the fleet. For example, in various embodiments, the delivery location, route, and / or timing can be changed. In various other embodiments, a different driver can be utilized, a different rest interval can be provided to the driver, and / or the driver can be replaced with a new driver after a predetermined amount of time with elevated cognitive load, etc., among other possible actions. In certain embodiments, the actions are implemented via transmission of instructions corresponding to the actions to the individual vehicles 100. In various embodiments, the instructions can be received via the transceiver 126 of the vehicle 100 and displayed on the display 124 of the vehicle 100 (e.g., on its display screen). In certain embodiments, the instructions are subsequently implemented via the driver of the vehicle 100, including with respect to operation and movement of the vehicle 100 and further with respect to the route and timing associated therewith, including with respect to operation and control of the braking system 106, steering system 108, and / or drive system 110 of the vehicle 100 by the driver. In certain embodiments, the instructions can be implemented automatically via the control system 102 of the vehicle 100, including via automatic control of the braking system 106, steering system 108, and / or drive system 110 of the vehicle 100 in accordance with instructions provided via the processor 142 of the vehicle 100.

[0083] Further in various embodiments, the actions of step 228, and previous steps related to cognitive load and external and internal events, also include a determination as to whether the cognitive load of the driver was triggered by an internal or external situation. In various embodiments, this is performed by one or more processors based on sensor data, including internal and external sensor data, and including feedback provided by the driver. In various embodiments, the actions also include one or more real-time notifications regarding the cognitive load and its cause(s) (including notifications to the driver and fleet manager, as appropriate), including whether the cognitive load is deemed to have been caused by an external or internal factor or situation (and, in certain embodiments, identifying the specific cause(s) of the cognitive load).

[0084] In various embodiments, the process 200 then terminates at 230.

[0085] Accordingly, methods, systems, and vehicles for collecting, aggregating, mapping, and utilizing data related to a fleet of vehicles are provided. In various embodiments, external data, internal data, and operational data from vehicles are collected and correlated with respect to external events (e.g., traffic conditions, weather, construction areas, etc.) and internal events (e.g., an elevated cognitive load of a driver of a vehicle). In various embodiments, this data is aggregated and mapped and used by one or more fleet managers to control vehicles in a fleet.

[0086] It will be understood that the systems, vehicles, and methods can differ from those depicted in the drawings and described herein. For example, Figure 1 the system 10 (including Figure 1 the vehicle 100 and the remote server 170 and components thereof) of Figure 1 the system depicted in FIG. 1 can differ from Figures 2-5 the processes and implementations of the steps can differ from those depicted in the figures, and / or various steps can occur simultaneously and / or in a different order than depicted in the figures.

[0087] While at least one exemplary embodiment has been presented in the foregoing detailed description, it should be appreciated that a vast number of variations exist. It should also be appreciated that the exemplary embodiment or exemplary embodiments are only examples, and are not intended to limit the scope, applicability or configuration of the disclosure in any way. Rather, the foregoing detailed description will provide those skilled in the art with a convenient road map for implementing an exemplary embodiment or exemplary embodiments. It should be understood that various changes can be made in the function and arrangement of elements without departing from the scope of the disclosure as set forth in the appended claims and the legal equivalents thereof, which can be practiced or implemented by various features and combinations.

Claims

1. A method comprising: Sensor data related to the driver's cognitive load is collected via one or more vehicle sensors; The sensor data is transmitted via one or more transceivers to a remote server physically located away from multiple vehicles via a wireless communication network. as well as The cognitive load for the driver is determined based on the sensor data via one or more processors; as well as Based on instructions provided by the one or more processors, one or more actions by the driver of the vehicle are provided according to the cognitive load.

2. The method according to claim 1, wherein: The steps for collecting the sensor data include: Internal data about the conditions inside the cabin of each of the multiple vehicles in the fleet that may affect the cognitive load is collected via one or more internal vehicle sensors. External data about the conditions outside the cockpit of each of the plurality of vehicles in the fleet that may affect the cognitive load is collected via one or more external vehicle sensors; The step of sending the sensor data includes sending the internal data and the external data to the remote server via the wireless communication network; and The method further includes: Aggregated fleet data is generated by aggregating the internal and external data for all vehicles in the fleet via the one or more processors; and Based on the aggregated fleet data, instructions provided by the one or more processors are used to provide one or more driver-related control actions for the plurality of vehicles in the fleet.

3. The method according to claim 2, further comprising: One or more external events related to the cognitive load for each of the plurality of vehicles in the fleet are determined via the one or more processors; The processor determines one or more internal conditions related to the cognitive load for each of the plurality of vehicles in the fleet, including internal distractions that are likely to cause the driver tension, distraction, or both. as well as For each of the plurality of vehicles in the fleet, the one or more internal conditions are correlated with the one or more external events via the one or more processors.

4. The method according to claim 3, further comprising: Anonymous feedback from the driver of each of the plurality of vehicles in the fleet is obtained via one or more input sensors, including via the driver’s interaction with a display screen, verbal feedback from the driver, or both; The correlation is performed via the one or more processors using the anonymous feedback.

5. The method according to claim 3, further comprising: Based on the corresponding timestamps provided to the one or more external events and the cognitive load via the one or more processors, the one or more external events are correlated with the cognitive load via the one or more processors.

6. The method of claim 3, wherein the cognitive load is calculated using internal data obtained via one or more biometric sensors for the driver.

7. The method of claim 6, wherein the cognitive load is further calculated using operational data obtained via one or more vehicle sensors relating to the operation of the driver of a corresponding one of the plurality of vehicles in the fleet.

8. The method according to claim 5, further comprising: According to instructions provided by the one or more processors, a display is provided to the fleet administrator via a display screen, the display including information about the one or more external events and the cognitive load, including a mapping of the one or more external events to the cognitive load along one or more routes taken by the plurality of vehicles in the fleet, and wherein the display includes a plurality of cognitive load markers corresponding to respective regions of interest along the one or more routes and their corresponding cognitive loads.

9. The method according to claim 1, wherein the method further comprises: The processor determines, based on the sensor data, whether the driver's cognitive load is triggered by internal conditions inside the vehicle or, alternatively, by external conditions outside the vehicle. as well as The processor provides instructions to provide notifications about the cognitive load and one or more of its causes, including whether the cognitive load is considered to be caused by external or internal conditions.

10. A system comprising: One or more vehicle sensors are configured to collect sensor data relating to the cognitive load of the vehicle's driver; One or more transceivers are configured to transmit the sensor data to a remote server physically located away from multiple vehicles via a wireless communication network; as well as One or more processors are configured to at least facilitate: Aggregates internal and external data from multiple vehicles in a fleet to generate aggregated fleet data; and Based on the aggregated fleet data, one or more control actions are provided for the plurality of vehicles in the fleet.

Images