Driver state guidance device and driver state guidance method

Abstract

This application discloses a driver state guidance device and a driver state guidance method. The driver state guidance device includes: a state recognition unit (201) that identifies the current state of the driver of the vehicle; a target estimation unit (202) that estimates the target state to which the driver's state is guided; a route prediction unit (203) that provides a predicted route for the vehicle; a situation prediction unit (206) that predicts the situation in the predicted route predicted by the route prediction unit; a planning unit (207) that plans stimuli to be used in the predicted route to guide the driver's state to the target state as a stimulus plan; and a stimulus control unit (209) that provides stimuli to the driver according to the stimulus plan determined by the planning unit in the predicted route.

Description

Technical Field

[0001] This disclosure generally relates to driver status guidance devices and driver status guidance methods. Background Technology

[0002] Patent Document 1 (Japanese Unexamined Patent Publication No. 2006-247255) discloses a technique for presenting / providing various types of stimulation based on mental and physical stress levels calculated from a driver's activity history. Patent Document 1 describes increasing the duration of stimulation based on an increase in mental and physical stress levels, the stimulation being activated immediately after the operation of the vehicle equipment is initiated, for example, just after the moment the driver initiates driving operations or similar circumstances.

[0003] However, the technology disclosed in Patent Document 1 does not take into account the changes in conditions along the way (i.e., during the journey or on the route) from the time of boarding / entering the vehicle to the time of arriving at the destination.

[0004] Therefore, it may be impossible to provide the driver with the appropriate type of stimulation based on changes in the situation along the way. Summary of the Invention

[0005] The purpose of this disclosure is to provide a driver state guidance device and a method for guiding a driver state, which can provide appropriate types of stimuli as stimuli to guide / lead the driver state to a preferred state based on changes in the vehicle's on-road conditions.

[0006] The above objectives can be achieved by a combination of features described in the technical solutions of this disclosure, and other technical solutions define other advantageous embodiments of this disclosure.

[0007] As one implementation, the reference numerals in parentheses described in the paragraphs indicate the correspondence with the specific devices described in the embodiments described later, and do not limit the technical scope of this disclosure.

[0008] To achieve the above objectives, the driver status guidance device of this disclosure is a driver status guidance device that can be used in a vehicle, and the driver status guidance device includes:

[0009] A state recognition unit identifies the current state of the vehicle's driver;

[0010] The target estimation unit estimates the target state to which the driver's state is guided.

[0011] Route prediction unit, which provides predicted routes for vehicles;

[0012] The situation prediction unit predicts the situation along the predicted route;

[0013] The planning unit plans stimuli to be used on the predicted route to guide the driver's current state to the target state based on: (i) the current state identified by the state recognition unit, (ii) the target state estimated by the target estimation unit, and (iii) the situation on the predicted route predicted by the situation prediction unit; and

[0014] The stimulation control unit provides stimulation to the driver according to the stimulation plan.

[0015] To achieve the above objectives, the driver state guidance method of this disclosure is a method that can be used in a vehicle and executed by at least one processor, the driver state guidance method comprising:

[0016] The state recognition step identifies the current state of the vehicle's driver;

[0017] The target estimation step estimates the target state, which is the state to which the driver's current state is guided.

[0018] The route prediction step provides a predicted route for the vehicle;

[0019] The situation prediction step predicts the situation along the predicted route;

[0020] The planning steps plan stimuli to be used in the predicted route to guide the driver's current state to the target state as a stimulus plan, based on: (i) the current state identified by the state identification step, (ii) the target state estimated by the target estimation step, and (iii) the predicted situation; and

[0021] Stimulus control steps, which provide stimulation to the driver according to the stimulus plan.

[0022] Based on the above, in addition to the identified driver state and the target state to which the driver state is guided, the situation in the predicted route is also considered to determine the stimulus plan for guiding the current state to the target state, and the stimulus plan for providing stimulation to the driver is suitable (e.g., better reflects) the changes in the situation.

[0023] Furthermore, in the predicted route, the driver is stimulated according to the stimulus plan (i.e., the driver is provided with stimulation), so as a stimulus to guide the driver's state to a favorable state, the stimulus best suited to the situation (or best suited to changes in the situation) can be provided. Attached Figure Description

[0024] The purpose, features, and advantages of this disclosure will become more apparent from the following detailed description with reference to the accompanying drawings, in which:

[0025] Figure 1This is a schematic diagram of the configuration of vehicle system 1;

[0026] Figure 2 This is a schematic diagram of the HCU (Human Machine Interface Control Unit).

[0027] Figure 3 It is a diagram of an emotional loop (ring) model representing psychological states;

[0028] Figure 4 It is a diagram showing the confirmed stimulus plan;

[0029] Figure 5 This is a diagram illustrating how plan-related information is displayed.

[0030] Figure 6 This is an example image of icons representing mental states;

[0031] Figure 7 This is a diagram illustrating how information about the driver's current state and the types of stimuli currently activated, as well as information about the ideal state and planned stimuli to be activated;

[0032] Figure 8 This is another example of how a diagram can display information about the driver's current state and the type of stimulus currently being activated, as well as information about the ideal state and the planned stimulus to be activated;

[0033] Figure 9 This is an example diagram showing how preview information is displayed;

[0034] Figure 10 This is a flowchart of the driver state guidance process in the HCU;

[0035] Figure 11 This is a flowchart of the planning-related processes in the HCU;

[0036] Figure 12 This is a diagram illustrating an example of an HCU configuration;

[0037] Figure 13 This is an example image showing historical information;

[0038] Figure 14 This is a diagram illustrating how information about the driver's current state and the type of stimulus currently activated, as well as information about the driver's past states and the stimuli activated in the past;

[0039] Figure 15 This is a diagram illustrating how information about the driver's current state and the type of stimulus currently activated, as well as information about the driver's past states and the stimuli activated in the past;

[0040] Figure 16This is a diagram illustrating how prediction results are displayed using a state prediction unit;

[0041] Figure 17 It is a graph of stimulus assessment with ratings related to the stimulus; and

[0042] Figure 18 This is a diagram related to stimulus evaluation for preferred stimuli. Detailed Implementation

[0043] Several embodiments of this disclosure will be described with reference to the accompanying drawings. For ease of description, portions that have the same function as those shown in the drawings and are used in the description of the various embodiments may be designated by the same reference numerals and their description may be omitted. Descriptions of other embodiments may refer to portions given the same reference numerals.

[0044] (First Implementation)

[0045] <Vehicle System 1 Configuration>

[0046] The first embodiment of this disclosure will now be described with reference to the accompanying drawings. Figure 1 The vehicle system 1 shown is for an automobile (hereinafter referred to as the vehicle) and includes an HMI (Human-Machine Interface) system 2, an air conditioning system 3, a short-range communication module (hereinafter referred to as NFCM) 4, a wide-area communication module (hereinafter referred to as WACM) 5, a locator 6, a map database (hereinafter referred to as DB) 7, vehicle status sensors 8, and a data collection device 9. It is assumed that the HMI system 2, air conditioning system 3, NFCM 4, WACM 5, locator 6, map DB 7, vehicle status sensors 8, and data collection device 9 are connected to, for example, an in-vehicle LAN. The vehicle equipped with vehicle system 1 is referred to as this vehicle in the following description.

[0047] The air conditioning system 3 is the vehicle's heating / cooling system. The air conditioning system 3 obtains air conditioning request information from the HCU 20, including air conditioning-related settings set by the vehicle's occupants, which will be described later. Then, based on the obtained air conditioning request information, it adjusts the temperature, airflow, odor, etc., in the vehicle's passenger compartment. The air conditioning system 3 includes an air conditioning control ECU 30, an air conditioning unit 31, and an aromatherapy unit 32.

[0048] The air conditioning unit 31 generates hot air and cold air (hereinafter referred to as air conditioning air). The air conditioning air is supplied to the passenger compartment from an outlet located, for example, in the dashboard of this vehicle. The fragrance unit 32 has beads or the like (hereinafter referred to as the impregnation material) impregnated with aromatic oils, such as essential oils containing aromatic components. The airflow generated by the air conditioning unit 31 then bypasses the impregnation material to supply fragrance / odor to the interior of the vehicle. Additionally, the fragrance unit 32 may atomize the aromatic oils. In this case, the aromatic components atomized by the fragrance unit 32 may mix with the airflow generated by the air conditioning unit 31 and may be supplied to the interior of the vehicle. The air conditioning unit 31 provides stimulation to the occupants of this vehicle via wind or as wind. Furthermore, due to (i.e., what may be perceived as) the temperature difference of the air conditioning air, the air conditioning unit 31 provides heating / cooling stimulation to the occupants of this vehicle. That is, the air conditioning unit 31 provides tactile stimulation.

[0049] The aroma unit 32 provides stimulation to the occupants of the vehicle through fragrance. In other words, the aroma unit 32 provides olfactory stimulation. Both the air conditioning unit 31 and the aroma unit 32 are devices that present stimulation. The air conditioning control ECU 30 is an electronic control device that controls the operation of the air conditioning unit 31 and the aroma unit 32. The air conditioning control ECU 30 is connected to the air conditioning unit 31 and the aroma unit 32.

[0050] NFCM 4 is a communication module used to perform short-range wireless communication. When a communication connection is established with a mobile terminal of a passenger in the vehicle, NFCM 4 performs short-range wireless communication with the mobile terminal. Short-range wireless communication is, for example, wireless communication where the maximum communication range is limited to tens of meters. Wireless communication compatible with Bluetooth Low Energy (BLE) can be used as short-range wireless communication. Examples of mobile terminals include multi-function mobile phones and wearable devices. WACM 5 transmits information to and receives information from a center outside the vehicle via wireless communication. In other words, WACM 5 performs wide-area communication.

[0051] The locator 6 includes a GNSS (Global Navigation Satellite System) receiver and inertial sensors. The GNSS receiver receives positioning signals from multiple artificial satellites. For example, the inertial sensors include gyroscope sensors and accelerometers. The locator 6 determines the vehicle's current position sequentially by combining the positioning signals received by the GNSS receiver and the measurements from the inertial sensors. The vehicle position can be represented by coordinates such as latitude and longitude. The vehicle position can also be measured using the distance traveled, obtained sequentially from signals output from a vehicle speed sensor installed on the vehicle.

[0052] Map DB 7 is a non-volatile memory that stores map data. Map data includes, for example, link data, node data, and road attributes. Link data consists of data regarding a unique identifier for the link, link length indicating its length, link direction, link travel time, link shape information, coordinates of the start and end nodes of the link, and road attributes. Road attributes include road name, road type, road width, number of lanes, speed limit, etc. Node data, on the other hand, consists of data regarding a node ID with a unique identifier for each node on the map, node coordinates, node name, node type, link ID describing the links connected to the node, intersection type, etc. Map data can be obtained from outside the vehicle using WACM 5.

[0053] Vehicle status sensor 8 is a set of sensors used to detect the status of the vehicle, such as its driving status and operating status. Vehicle status sensor 8 includes a vehicle speed sensor, steering angle sensor, acceleration / deceleration sensor, accelerator position sensor, brake switch, and turn signal switch. The vehicle speed sensor is configured to detect the vehicle's speed. The steering angle sensor detects the steering angle of the vehicle's steering wheel. The acceleration / deceleration sensor detects the vehicle's acceleration / deceleration. The accelerator position sensor detects the opening of the vehicle's accelerator pedal. The brake switch outputs a signal based on whether the vehicle's brake pedal is depressed. The turn signal switch detects the vehicle's turn signal and illuminates the lights. Vehicle status sensor 8 outputs the detection results to the vehicle's LAN. Note that the detection results of vehicle status sensor 8 can also be output to the vehicle's LAN via an ECU installed on the vehicle.

[0054] Data collection device 9 collects information about the vehicle's driving history (hereinafter, driving history-related information). Examples of driving history information include route information, congestion level information, and driving load information. Route information can be time-series data of the vehicle's location, sequentially located by locator 6 while the vehicle is in motion. That is, it can be a data set where the vehicle's location and the time when the location was obtained are linked / associated. Therefore, route information also includes driving time information. Preferably, information about "one day of the week" for which relevant data is collected is linked to the route information.

[0055] Regarding information on congestion levels, data collection device 9 can be configured to sequentially identify and collect congestion levels based on, for example, the vehicle's speed while it is traveling. The vehicle speed can be determined using the detection results of the vehicle speed sensor in vehicle state sensor 8. For example, traffic congestion can be considered when (i) the vehicle speed is equal to or less than a threshold used to classify low speed and (ii) the vehicle is traveling over a specific time period or longer. Congestion levels can be determined in two levels: whether congestion is observed. Furthermore, congestion levels can be divided into three or more levels based on the duration of the congestion. Information on congestion levels can be associated with the vehicle's location, time, day of the week, etc., from which the congestion level is obtained.

[0056] Regarding driving load information, data collection device 9 can sequentially identify driving load based on the frequency of driving operations of the vehicle while it is in motion, such as accelerator operation, braking operation, steering operation, and turn signal lighting operation, and can collect the identified driving load. Driving load should be higher when the driver's driving tasks are heavy. Driving tasks include, for example, steering, acceleration / deceleration, and scene monitoring. Driving load is designated as higher when the frequency of driving operations increases. Driving load can be divided into two levels based on whether the load is high. The frequency of driving operations can be identified based on the degree of change in detection results from steering angle sensors, acceleration sensors, accelerator position sensors, brake pedal force sensors, etc. Driving load can be determined to be increased when the frequency of changes in the driver's facial orientation detected by the indoor camera 21 (described later) increases. This is because driving load is considered high when facial orientation changes drastically during scene monitoring. Driving load can also be identified as high when the distance between the vehicle and the vehicle in front of it can be identified by the scene monitoring sensors, as the distance between vehicles is sequentially identified as becoming shorter. Driving load information can also be linked to vehicle location, time, day of the week, etc., from which the vehicle's driving load is obtained.

[0057] The driving history information collected by data collection device 9 may be associated with identification information used to identify each driver. This driving history information collected by data collection device 9 can be uploaded to the center via WACM 5. In this way, time, information about driving routes associated with a day of the week, information about congestion levels, and information about driving load are accumulated at the center as the driver's personal historical data.

[0058] HMI System 2 acquires information about the driver of the vehicle. HMI System 2 receives input from the driver of the vehicle. HMI System 2 presents stimuli to the driver of the vehicle. These stimuli also include the provision of information. Detailed information about HMI System 2 will be described below.

[0059] <Configuration of HMI System 2>

[0060] HMI System 2 includes a HCU (Human Machine Interface Control Unit) 20, an in-vehicle camera 21, a biosensor 22, lighting equipment 23, a display device 24, an audio output device 25, and a user input device 26. The HCU is a device or unit that stimulates a person's senses visually, auditorily, or otherwise to control the information content provided / presented to the person.

[0061] The in-vehicle camera 21 captures a predetermined range inside the vehicle of the present vehicle. The in-vehicle camera 21 captures an image that at least includes the driver's seat of the present vehicle. The in-vehicle camera 21 consists of, for example, a near-infrared light source, a near-infrared camera, a control unit for controlling them, etc. The in-vehicle camera 21 captures the driver irradiated with the near-infrared light of the near-infrared light source through the near-infrared camera. The control unit performs image analysis processing on the image captured by the near-infrared camera. The control unit analyzes the captured image and detects the feature amounts of the driver's face. The control unit can detect the face direction, wakefulness level, etc. of the driver based on the detected features of the driver's face. For example, the wakefulness level can be detected by the degree of opening and closing of the eyelids.

[0062] The biosensor 22 measures the biometric information of the driver. The biosensor 22 sequentially outputs the measured biometric information to the HCU 20. The biosensor 22 can be provided in the present vehicle. The biosensor 22 can be configured to be provided in a wearable device worn by the driver. When the biosensor 22 is provided in the present vehicle, it can be provided, for example, in the steering wheel, seat, etc. When the biosensor 22 is provided in the wearable device, the HCU 20 can be configured to obtain the measurement result of the biosensor 22 via, for example, NFCM 4. Examples of the biometric information measured by the biosensor 22 include respiration, pulse, heartbeat, etc. Note that the biosensor 22 can be configured to measure biometric information other than respiration, pulse, and heartbeat. For example, the biosensor 22 can measure brain waves, heart rate variability, sweating, body temperature, blood pressure, skin conductance, etc.

[0063] The lighting equipment 23 is provided at a position visible to the driver and provides stimulation to the driver by emitting light. That is, the lighting equipment 23 provides visual stimulation. The lighting equipment 23 is a presentation device that presents stimulation. A light-emitting device such as an LED can be used as the lighting equipment 23. Preferably, the lighting equipment 23 can switch the color of light emission. In the lighting equipment 23, the timing of turning on and off and the color of light emission are controlled by the HCU 20.

[0064] Display device 24 displays information. Display device 24 is positioned in a location visible to the driver and provides stimulation to the driver via a display. That is, display device 24 provides visual stimulation. Display device 24 is a presentation device for presenting stimulation. Display device 24 displays at least images. In addition to images, display device 24 can also display text, etc. The display of display device 24 is controlled by HCU 20.

[0065] Display devices 24 can be, for example, instrument cluster MID (Multi-Information Display), CID (Center Information Display), navigation device displays, and HUD (Head-Up Display). An instrument cluster MID is a display device located in front of the driver's seat inside the vehicle. For example, an instrument cluster MID can be configured to be located on the dashboard. A CID is a display device located in the center of the vehicle's dashboard. A HUD is located, for example, in the dashboard inside the vehicle. The HUD projects a display image formed by a projector onto a projection area defined by the windshield as a projection element. The light of the image reflected from the windshield to the interior of the vehicle (i.e., towards the driver) is perceived by the driver sitting in the driver's seat. Therefore, the driver can visually recognize a virtual image of the display image formed in front of the windshield by superimposing it onto a portion of the foreground (i.e., superimposing it onto a live image). Instead of the windshield, the HUD can be configured to project the display image onto a combiner located in front of the driver's seat.

[0066] Audio output device 25 provides stimulation to the driver through sound. That is, audio output device 25 provides auditory stimulation. Audio output device 25 is a presentation device for presenting stimuli. Examples of sounds output from audio output device 25 include music, ambient sounds, etc. Music can include background music (BGM). Ambient sounds can be sounds that reproduce a specific environment. Audio output device 25 can be, for example, an audio speaker that outputs audio / speech.

[0067] User input device 26 receives input from the driver. User input device 26 can be an operating device that receives operational input from the driver. The operating device can be a mechanical switch or touch panel integrated with display device 24. User input device 26 is not limited to operating devices that receive operational input; it can be any device that receives input from the user. For example, it can be a voice input device that receives voice commands from the user.

[0068] The HCU 20 mainly consists of a microcomputer, which includes a processor, a memory, I / O, and a bus connecting them. The HCU 20 executes various processes by executing a control program stored in the memory, such as processes related to guiding the driver's state (hereinafter referred to as driver state guiding related processes). The HCU 20 corresponds to a driver state guiding device. The above-mentioned memory is a non-transitory tangible storage medium that stores non-transitory computer-readable programs and data. In addition, the non-transitory tangible storage medium is implemented by a semiconductor memory, a magnetic disk, etc. The details of the driver state guiding related processes in the HCU 20 will be described below.

[0069] <Configuration of HCU 20>

[0070] Subsequently, reference will be made to Figure 2 Describe the schematic configuration of the HCU 20.

[0071] As Figure 2 shown, the HCU 20 has a state recognition unit 201, a target estimation unit 202, a route prediction unit 203, a presentation control unit 204, a route selection unit 205, a situation prediction unit 206, a planning unit 207, a plan selection unit 208, and a stimulus control unit 209, which are set as functional blocks. In addition, the process of executing these functional blocks by a computer corresponds to executing a driver state guiding method. Note that part or all of the functions executed by the HCU 20 can be configured (i.e., implemented) by one or more ICs, etc., that is, can be set as hardware logic. In addition, part or all of the functional blocks of the HCU 20 can be implemented by a combination of hardware and software executed by a processor.

[0072] The state recognition unit 201 recognizes the state of the driver of the vehicle. The process in the state recognition unit 201 corresponds to a state recognition step. As Figure 3 shown, the state described in this embodiment is classified by a ring model representing a mental state on two axes, namely, the comfort-discomfort axis and the active-inactive axis. In other words, the state in this embodiment is assumed to be a mental state. The ring model is the so-called Russell's circumplex model. The active-inactive axis can be interpreted as an axis of wakefulness and sedation. Note that in Russell's emotion ring (circumplex) model, tiredness and drowsiness are also regarded as mental states. Hereinafter, as mental states classified by the ring model, Figure 3 shown, "irritation", "sadness", "boredom", "tiredness", "drowsiness", "excitement", "happiness", and "relaxation" are provided as examples.

[0073] "Annoyance" is a mental state characterized by both high levels of discomfort and high levels of activity. In this embodiment, for example, "annoyance" is assumed to be included in a state of stress. "Bored," "fatigued," and "drowsy" are mental states characterized by high levels of discomfort but the lowest levels of activity. Among "bored," "fatigued," and "drowsy," activity decreases in the order of "boredom" > "fatigue" > "drowsy." In this embodiment, "boredom" is, for example, included in a state of inattentiveness. It should be noted that "fatigued" can also be included in a state of inattentiveness. In this embodiment, "drowsy" is also included in a state of inattentiveness. "Excitement" is a mental state involving both high levels of comfort and high levels of activity. "Happiness" is a mental state involving high levels of comfort and a mental state where the activity / inactivity axis is neither high nor low. "Relaxation" is a mental state involving high levels of comfort but a mental state where the activity / inactivity axis is low. "Annoyance," "boredom," "fatigue," and "drowsy" are uncomfortable mental states. "Excitement," "happiness," and "relaxation" are comfortable mental states.

[0074] In this embodiment, "annoyance," "sadness," "excitement," "happiness," and "relaxation" are provided as examples of mental states, but the interpretation is not necessarily limited to these states. For example, it can be configured to involve other mental states such as "fear," "disgust," and "surprise." "Fear" is a mental state involving both high discomfort and high activity, and is more active than "annoyance." "Disgust" is a mental state involving both high discomfort and high activity, and is more uncomfortable than "annoyance." In the examples of this embodiment, "annoyance," "boredom," "fatigue," and "drowsiness" correspond to specific states. "Fear" and "disgust" correspond to uncomfortable states other than specific states. "Annoyance," "boredom," "fatigue," and "drowsiness" are also uncomfortable states. "Excitement," "happiness," and "relaxation" correspond to comfortable states.

[0075] The state recognition unit 201 can identify the driver's state from the detection results of the indoor camera 21. For example, it can identify a "drowsy" state from the level of alertness. Additionally, it can identify the driver's state based on whether facial features corresponding to the driver's state are recognized. The state recognition unit 201 can also identify the driver's state from the driver's biometric information measured by the biosensor 22. For example, it can identify the driver's state based on whether the amount of biometric information corresponding to the driver's state is recognized.

[0076] The state recognition unit 201 can also identify the driver's state from the detection results of the vehicle state sensor 8. For example, the driver's state can be identified based on whether characteristic driving behaviors corresponding to the driver's state are detected. When the driver's biometric information can be pre-measured using the driver's portable terminal, the state recognition unit 201 can identify the driver's state from the biometric information obtained from the portable terminal via NFCM 4. Note that if the state recognition unit 201 can estimate the driver's state from the biometric information using the portable terminal, the state recognition unit 201 can identify the driver's state by obtaining the estimation result via NFCM 4.

[0077] The target estimation unit 202 estimates the driver's intended state (hereinafter referred to as the target state). The process in the target estimation unit 202 corresponds to the target estimation step. When the driver state identified by the state recognition unit 201 is an uncomfortable state, the target estimation unit 202 can estimate the target state to guide the driver to a comfortable state. When the driver state identified by the state recognition unit 201 is neither an uncomfortable nor a comfortable state, the target estimation unit 202 can estimate the target state to guide the driver to a comfortable state. When the driver state identified by the state recognition unit 201 is a comfortable state, the target estimation unit 202 may estimate the target state to maintain the comfortable state.

[0078] The target estimation unit 202 may estimate one of the comfort states as the target state based on the driver's schedule, destination, etc. The driver's schedule can be obtained from the schedule application on the driver's portable terminal. If a destination is input to the navigation device, this destination can be obtained. When the route prediction unit 203, described later, predicts the destination, this destination can be obtained as the destination predicted by the route prediction unit 203. For example, if a family birthday is set as the driver's upcoming plan, the "excitement" state can be estimated as the target state. For example, if a meeting is set on the driver's schedule immediately, the "relaxation" state can be estimated as the target state. For example, if the driver has no plans and the destination is home, the "happiness" state can be estimated as the target state. For example, if the destination is the workplace, the "excitement" state can be estimated as the target state.

[0079] Route prediction unit 203 predicts the route for the vehicle. The process in route prediction unit 203 corresponds to the route prediction step. Route prediction unit 203 may predict a set of location coordinates to be used as the route and the travel time of the route. When route prediction unit 203 has a destination input to the navigation device, route prediction unit 203 can predict a route connecting the current vehicle position determined by locator 6 to the destination as the route for the vehicle. In this case, multiple route candidates can be predicted under various conditions such as time priority, distance priority, etc. The travel time of the route can be predicted using the link travel time stored in map DB 7.

[0080] Route prediction unit 203 can predict destinations and routes based on the driver's personal historical data acquired from the center via WACM 5. In this case, routes can be predicted by identifying high-frequency destinations and routes in the personal historical data based on the current vehicle location, current time, and day of the week as located / measured by locator 6. In this case, multiple route candidates can be predicted.

[0081] The travel time of a route can be predicted by using the time of each vehicle location associated with information about the route in the driver's personal historical data.

[0082] Preferably, when the actual route traveled by the vehicle deviates from the predicted route, the route prediction unit 203 re-predicts the vehicle's route. For example, deviation from the predicted route could occur when the input link is different from the link in the predicted route. Furthermore, it is preferable that the route prediction unit 203 predicts multiple candidate routes as the vehicle's route. In this way, the driver can select the desired route.

[0083] The presentation control unit 204 causes the display device 24 or the audio output device 25 to present information. In this embodiment, for example, the case of presenting information from the display device 24 is described below. The presentation control unit 204 presents information about a stimulus plan for providing stimulation to the driver, which is determined by the planning unit 207, described later.

[0084] Preferably, the presentation control unit 204 presents at least information about the multiple candidate routes predicted by the route prediction unit 203 as planning-related information. For example, the information about the multiple candidate routes can be displayed on the screen of a navigation device, such as a monitor or CID. As a display mode, the candidate routes can be displayed on a map, or a list of candidate routes can be displayed. When the presentation control unit 204 presents information about the multiple candidate routes, the driver performs a selection input to select a route from the multiple candidate routes based on the presented candidate route information. The presentation control unit 204 can provide a prompt for the selection input (e.g., a message can be displayed).

[0085] The route selection unit 205 selects a route from multiple candidate routes predicted by the route prediction unit 203 based on the selection input received from the driver. For example, the selection input can be made by configuring the touch panel to have a touch screen that receives touch operations on the screen, while simultaneously displaying an image on the navigation device's display, CID, etc.

[0086] Situation prediction unit 206 predicts the conditions of the predicted route predicted by route prediction unit 203 (hereinafter referred to as the conditions en route). The process in situation prediction unit 206 corresponds to the situation prediction step. When a predicted route is selected from multiple candidate routes by route selection unit 205, situation prediction unit 206 predicts the conditions of the predicted route (e.g., predicting the conditions to be encountered en route to the destination). Situation prediction unit 206 predicts at least one of the following as the conditions en route: (i) the driver's driving load becomes high, (ii) the degree of traffic congestion, and (iii) monotonous driving. For example, situation prediction unit 206 predicts the conditions of each of multiple regions in the predicted route. The regions mentioned herein may be regions divided by using categories such as links and nodes.

[0087] Situation prediction unit 206 may predict situations where the driver's driving load is high as a condition during the journey, based on the road type of the road along the predicted route. The road type of the predicted route can be identified by comparing map data stored in map DB 7 with the predicted route. Situation prediction unit 206 may predict situations where the driver's driving load is high in areas where the road type is identified as narrow streets or residential roads. This is because driving load is estimated to be high on narrow streets and residential roads due to the time and effort spent monitoring the vehicle's surroundings. Situation prediction unit 206 may also identify situations where the driver's driving load is high from the driver's personal history data obtained from the center via WACM 5. Situation prediction unit 206 can predict situations where the driver's driving load is high in areas recorded in the personal history data. By using personal history data, the conditions during the journey for each individual driver can be predicted more accurately.

[0088] Situation prediction unit 206 may predict the level of congestion as a situation along the route based on congestion information obtained from the center via WACM 5. Situation prediction unit 206 may identify the level of congestion for each link along the predicted route by comparing the congestion information of each link with the predicted route. Situation prediction unit 206 may predict that the predicted route will be congested in areas with congestion levels equal to or higher than a predetermined value. Situation prediction unit 206 may identify the level of congestion from the driver's personal historical data obtained from the center via WACM 5. Situation prediction unit 206 may predict congestion in areas with high congestion levels in the personal historical data. By using personal historical data, the situation along the route for each individual driver can be predicted more accurately. Congestion predicted as a situation along the route corresponds to situations that worsen the driver's condition. This is because congestion is estimated to lead / guide the driver to a state of "annoyance".

[0089] The situation prediction unit 206 may predict driving monotony as a condition along the route from map data of the predicted route. The situation prediction unit 206 may predict driving monotony based on the road type along the predicted route. The situation prediction unit 206 may predict that driving is monotonous in areas of certain road types (e.g., highways, expressways, etc.). Furthermore, the situation prediction unit 206 may predict that driving is monotonous in areas where roads extend as straight paths of a predetermined distance or longer. The situation prediction unit 206 can identify driving monotony from the driver's personal history data acquired from the center via WACM 5. The situation prediction unit 206 may predict that driving is monotonous in areas where driving load remains consistently low for a predetermined distance or longer in the personal history data. By using personal history data, the condition along the route for each individual driver can be predicted more accurately. Monotonous driving predicted as a condition along the route corresponds to conditions that worsen the driver's condition. This is because such a condition of driving monotony leads the driver to a state of "boredom" and "drowsiness."

[0090] When using personal historical data, the situation prediction unit 206 may create a map of driving load and congestion levels sorted by location, day of the week, and time of day. Referring to this map, driving load and congestion levels along the predicted route can then be identified. Because the day of the week and time of day are associated with personal historical data, driving load and congestion levels can be identified not only by location but also by day of the week and time of day. Therefore, the situation prediction unit 206 can predict high driving load, congested, and monotonous driving conditions not only by location but also by day of the week and time of day.

[0091] The situation prediction unit 206 can predict situations that change the driver's state, other than those that worsen the driver's state, as situations along the route. For example, the situation prediction unit 206 may predict that the driver's state will change to a "relaxed" state in a coastal area of ​​the predicted route.

[0092] When the route prediction unit 203 re-predicts the route of the vehicle, the situation prediction unit 206 may predict the situation along the re-predicted route. When the route prediction unit 203 predicts multiple candidate routes, the situation prediction unit 206 predicts the situation along the predicted route by using the route selected from the candidate routes by the route selection unit 205.

[0093] In addition to (i) the driver state identified by the state recognition unit 201 and (ii) the target state estimated by the target estimation unit 202, the planning unit 207 also determines a stimulus plan to be used along the predicted route to guide the driver to the target state based on the conditions on the predicted route predicted by the situation prediction unit 206. The process in the planning unit 207 corresponds to the planning decision step. The driver state identified by the state recognition unit 201 is referred to as the current state below. The stimulus plan can be provided as a schedule, including, for example, the type of stimulus to be provided, the timing of stimulus provision, the time allocation of stimulus provision, etc.

[0094] In the following text, reference will be made to Figure 4 Describe the determination of the stimulus plan.

[0095] Figure 4 This is a diagram used to explain an example of a stimulus plan.

[0096] Figure 4 An example is shown where the current state is identified as "fatigued" and the target state is estimated as "excited".

[0097] exist Figure 4 In the example, traffic congestion, high load, and monotonous operation are assumed to be conditions along the way (also known as conditions in the route).

[0098] Planning unit 207 can determine the ideal state (hereinafter referred to as the ideal state) of the driver en route on the predicted route based on (i) the current state identified by state identification unit 201 and (ii) the target state estimated by target estimation unit 202. Planning unit 207 can determine the ideal state by referring to a pre-associated correspondence between the combination of (i) the current state and the target state and (ii) the ideal state. The ideal state can preferably be set as the state through which the current state should ideally / preferably transition to the target state, and can also be set as a state in between, i.e., given the target state, this state can be specified hereinafter as the via state. Figure 4 In the example, the ideal state was identified as "relaxed" in the first half and "excited" in the second half. Figure 4 In this system, "relaxation" is represented by D, and "excitement" is represented by E. The time allocation ratio between the ideal state and the target state can be arbitrarily set. Furthermore, the target state can actually be used as the ideal state.

[0099] Planning unit 207 predicts changes in the driver's state along the predicted route based on the conditions predicted by situation prediction unit 206. For example, in an area where the situation along the route is predicted to be congested, the state can be predicted to change to "annoyed" (or "frustrated"). Figure 4 In region where the situation is predicted to be high load, a change in state to a low receptivity state (hereinafter referred to as the low receptivity state) can be predicted. Figure 4 (B in the text). In areas where the driving conditions are predicted to be monotonous, a change in state to "boredom" or "drowsiness" can be predicted. Figure 4 (C in the middle).

[0100] Planning unit 207 can determine a provisional plan that provides, for example, an estimated stimulus to guide / lead to the determined ideal state for the portion of the predicted route where no predicted state change occurs. Figure 4 As shown in F, for the portion where no state change is anticipated and "relaxation" is determined as the ideal state, a tentative plan for providing stimuli estimated to guide / lead the user to the "relaxed" state can be determined. Below are examples of stimuli for guiding / leading the user to the "relaxed" state. The period for dimming and brightening the lighting of the lighting device 23 can be set with a long period, such as 8 seconds or longer. The lighting device 23 can emit light in a color estimated to have a relaxing effect. The aroma unit 32 may generate an aroma estimated to have a relaxing effect via the air conditioning control ECU 30. Slow-tempo music may be output from the audio output device 25.

[0101] In addition, such as Figure 4 As shown in Figure I, for the portion where there is no state change and "excitement" is determined as the ideal state, a tentative plan is established to provide stimuli estimated to guide / lead the user to the state of "excitement." Below is an example of a stimulus to guide / lead the user to "excitement." Fast-paced music can be output from audio output device 25.

[0102] Planning unit 207 can determine a provisional plan that provides stimuli based on the anticipated changes in the route's condition for the portion of the forecast. Planning unit 207 determines a provisional plan that provides stimuli estimated to improve the driver's condition for the portion of the forecast that will worsen the driver's condition. For example, for the portion forecasting "annoyance" due to traffic congestion, such as... Figure 4 As shown in G, a provisional plan is determined for providing stimuli estimated to improve the "annoyance" state. Examples of stimuli estimated to improve the "annoyance" state include the following.

[0103] The fragrance unit 32 may generate a fragrance estimated to have a calming effect via the air conditioning control ECU 30.

[0104] The lighting cycle of the lighting device 23 can be set to dim and brighten at a slow cycle, such as 6 seconds or longer.

[0105] In addition, such as Figure 4 As shown in section J, in the section anticipating "boredom" and "drowsiness" due to monotonous driving, a provisional plan is determined for providing stimuli estimated to improve absent-mindedness and / or drowsiness. The following are examples of stimuli estimated to improve absent-mindedness and / or drowsiness: A refreshing aroma can be generated from the fragrance unit 32 via the air conditioning control ECU 30. Cool air can be generated from the air conditioning unit 31 via the air conditioning control ECU 30. Fast-paced music can be output from the audio output device 25.

[0106] Preferably, the planning unit 207 determines a provisional plan for suppressing stimuli in the portion of the predicted route where the situation prediction unit 206 predicts the driver's driving load will be high. That is, as in... Figure 4 In the H section, for the portion anticipating a low receptivity state due to high load, a provisional plan of not providing stimulation can be determined. This is because providing stimulation in a low receptivity state is useless. Note that for the portion anticipating high driving load, a provisional plan of stimulation inhibition can be determined from the viewpoint of reducing the impact on driving operation. In this case, in addition to the configuration of not providing stimulation, stimulation inhibition can also be achieved by a configuration of reducing the intensity of the stimulation.

[0107] Preferably, the planning unit 207 determines the stimulus plan based on the provisional plan by utilizing the fact that the situation prediction unit 206 can predict the situation along the route, making the stimulus mode more comfortable for the driver.

[0108] Preferably, the planning unit 207 determines the stimulus plan by (i) omitting stimuli scheduled to have a duration less than a threshold, and (ii) replacing such stimuli with stimuli preceding or following them. The threshold mentioned here can be a value that makes the driver feel the frequency of stimulus switching. Based on the above, stimuli will not be switched frequently, thus sparing the driver from such switching annoyance; stimuli will not be changed frequently, and the driver will not feel annoyed. Figure 4 In the example, since the duration of the F stimulus between the G and H stimuli is short, the F stimulus is replaced by the G and H stimuli respectively, thus determining the stimulus plan (see [reference]). Figure 4 (Example). In cases where stimuli below a threshold are omitted or replaced with stimuli preceding or following them, the stimulus plan can be determined by setting the preceding stimulus to gradually decrease and the following stimulus to gradually increase.

[0109] Preferably, for sections of the predicted route that are predicted by the situation prediction unit 206 to have conditions that worsen the driver's condition, the planning unit 207 determines to proactively activate a stimulus plan that is estimated to improve / worsen the condition before entering such sections. Based on the above, guiding the driver's condition in a comfortable direction becomes easier compared to activating improvement after the condition has worsened. Figure 4 In the example, the stimulus plan was identified as occurring before entering Section C, which was expected to make the driver feel “bored” and / or “drowsy” (see in...). Figure 4 The FI in the bottom row was initially estimated to be an improvement in the deterioration of the stimulus J.

[0110] This embodiment illustrates a configuration in which the planning unit 207 determines a stimulus plan based on a provisional plan after deciding on the provisional plan; however, this disclosure is not necessarily limited to this configuration. For example, the planning unit 207 may determine the provisional plan as the stimulus plan.

[0111] Preferably, the planning unit 207 determines multiple candidate routes as incentive plans for a predicted route. In this way, the driver has the freedom to choose his / her preferred incentive plan. Preferably, the presentation control unit 204 presents at least information about the multiple candidate routes determined by the planning unit 207 as plan-related information. For example, information about the multiple candidate routes can be displayed on the display of a navigation device, the screen of a CID, etc. As a display mode, each candidate route can be displayed as shown in the image below. Figure 4The diagram shows the type, sequence, and time allocation of stimuli. When the presentation control unit 204 presents information about multiple candidate programs, the driver inputs a selection input to choose a stimulus program from the multiple candidate programs based on the presented candidate program information. The presentation control unit 204 can provide a presentation prompting the selection input (e.g., it can display a message).

[0112] The presentation control unit 204 can overlay on top of the map the following plan-related information: (i) the predicted route, (ii) the type of stimulus (hereinafter, the type of activating stimulus) to be provided to the driver along the predicted route according to the plan determined by the planning unit 207, and (iii) the driver's ideal state to which the provided stimulus will guide the driver. In this way, the driver can easily and intuitively identify the plan, namely the type of activating stimulus and the ideal state along the predicted route.

[0113] Here, by using Figure 5 This describes an example of how to display plan-related information. Plan-related information can also be specified as scheduling information. Figure 5 This is an example of displaying scheduling information overlaid on an electronic map on the screen of display device 24. Figure 5 The icon IMIc in the diagram represents information about the ideal state. Figure 5 The line SR in the diagram represents the currently selected predicted route. Figure 5 The line NSR in the diagram represents the currently unselected predicted route. The selection of the predicted route can be performed as input to the planning selection unit 208, as described later. Figure 5 The different line types representing the predicted routes correspond to different types of activating stimuli. More specifically, line type FA shows a stimulus estimated to improve the "annoyance" state. Furthermore, line type RA shows a stimulus estimated to guide / induce a "relaxation" state, line type EA shows a stimulus estimated to induce a "high" state, line type AA shows a stimulus estimated to improve the "absent-minded" state / drowsiness, and line type AR shows an inhibitory stimulus. Note that the same representations are used in the accompanying figures.

[0114] like Figure 5 As shown, for the currently selected prediction route, the types of stimuli that will be activated along the route are illustrated. Note that the types of activated stimuli can also be represented using color. Furthermore, for the currently selected prediction route, the ideal state to be achieved along the route is indicated by icons. Ideal states can be categorized into types such as "neutral," "happy," "excited," "relaxed," "sad," and "angry." "Neutral" is a state that cannot be further divided into any other state. Figure 6As shown, the various types of ideal states are represented by icons that mimic different facial expressions. Figure 6 The two distinct icons representing the two ideal unpleasant states, sadness and annoyance, imply that they may have / induce an impact on the safety of driving operations. For example, when a stimulus induces a state of sadness, the aggression caused by high-intensity annoyance may be reduced, i.e., it may be guided towards normal / safe driving. Furthermore, a high-intensity state of sadness that results in less agile driving operations can be improved by guiding the driver into an annoyance state. However, guiding the driver into an unpleasant state may be less acceptable to the driver; therefore, this ideal state can be excluded from the unpleasant category in the actual operation of driver state guidance devices.

[0115] The plan selection unit 208 selects a candidate plan from multiple candidate plans based on the selection input received from the driver. Then, the plan unit 207 determines the selected candidate plan as the stimulus plan. For example, the selection input can be made by configuring the touch panel to have a touch screen that receives touch operations on the screen while simultaneously displaying an image on the navigation device's display, CID, etc.

[0116] After the stimulus plan is determined by the planning unit 207, the presentation control unit 204 can present, for example, the following plan-related information: The predicted route and travel time predicted by the route prediction unit 203 can be presented as plan-related information. The predicted route may include the destination. The conditions along the way predicted by the situation prediction unit 206 can be presented as plan-related information. For example, sections corresponding to traffic congestion, high load, or monotonous driving can be displayed on a map. The current state identified by the state recognition unit 201 can be presented as plan-related information. The type of stimulus being provided or the plan to be provided, the content of the stimulus, and the duration of the stimulus can be presented as plan-related information.

[0117] Note that the plan-related information is not limited to the configuration to be displayed on display device 24. The plan-related information can be output via voice from audio output device 25. Alternatively, the plan-related information can be transmitted to the driver's portable terminal via NFCM 4 and displayed on the portable terminal's screen. For example, the plan-related information can be presented when user input device 26 or the driver's portable terminal receives an input requesting the presentation of plan-related information.

[0118] When the route prediction unit 203 re-predicts the vehicle's route, the planning unit 207 may re-determine the stimulus plan based on the conditions along the re-predicted route. When the presentation control unit 204 re-determines the stimulus plan through the planning unit 207, it is preferable that the presentation control unit 204 at least presents the changes in the stimulus plan resulting from the re-determination as plan-related information. For example, the display device 24 may display which stimulus was changed and how it was changed, or the audio output device 25 may output speech.

[0119] In the section where the situation prediction unit 206 predicts the situation in which the driver's state should change from a specific state, when the state recognition unit 201 identifies the actual state as different from that specific state, the planning unit 207 may preferably redetermine the stimulus plan for the driver, i.e., use stimuli that are suitable for the driver and the actual identified state by the state recognition unit 201.

[0120] Based on the above, the driver's state can be more accurately guided / led to an optimal state by stimulating the driver according to the actual identified driver state. For example, when the driver's state is not "annoyed" in a section predicted to have traffic congestion, the stimulus plan can be redefined to switch from a plan to improve the annoyed state to a plan to guide the user to a "relaxed" state.

[0121] The stimulus control unit 209 stimulates the driver according to the stimulus plan determined by the planning unit 207 while traveling along a predicted route used to determine the stimulus plan. The process of the stimulus control unit 209 corresponds to the stimulus control steps. "Along the predicted route" refers to the route the vehicle travels along the predicted route. The stimulus control unit 209 selects the type of stimulus according to the stimulus plan, providing either a stimulating or inhibiting stimulus. When multiple candidate plans are determined by the planning unit 207, the stimulus control unit 209 stimulates the driver according to the plan selected from the candidate plans by the plan selection unit 208.

[0122] Regardless of the stimulus plan determined by the planning unit 207, the stimulus control unit 209 may preferably stimulate the driver along a predicted route used to determine the stimulus plan by a stimulus estimated to improve the specific state, based on a specific state identified by the state recognition unit 201. As mentioned above, the specific state is an unpleasant state, such as "annoyance," "boredom," "fatigue," and "drowsiness."

[0123] Based on the above, by prioritizing improvements to specific states deemed to have a greater adverse impact on driving, the adverse effects of those states on driving can be further suppressed. When a specific state is identified by the state recognition unit 201, it is preferable to immediately improve the specific state by providing a stimulus estimated to be effective in improving the specific state within a short period of time.

[0124] When the stimulus control unit 209 generates a stimulus estimated to improve a specific state, and after the state identified by the state recognition unit 201 has improved from the specific state by such stimulus, the stimulus control unit 209 may prioritize providing “unguided” stimuli, which are stimuli estimated not to guide the driver’s state back to the specific state, regardless of how preferably the stimulus plan determined by the planning unit 207 is based on (i) unguided stimuli and (ii) the fact that there is a stimulus mismatch according to the stimulus plan determined by the planning unit 207.

[0125] Based on the above, it is possible to prevent a specific state from immediately reverting to its previous state after improvement / exit. Unguided stimuli can be based on the type of the specific state. For example, when the specific state is "drowsy," improvement can be achieved through refreshing scents / aromas, and unguided stimuli can be provided through the output of fast-paced music.

[0126] It should be noted that the process of providing unguided stimuli in a prioritized manner can be configured to be limited to a predetermined period of time following improvement from a particular state. The predetermined period of time mentioned herein can be set arbitrarily.

[0127] In the event that the stimulus does not match the stimulus in the stimulus plan determined by the planning unit 207 along the predicted route used to determine the stimulus plan, the stimulus control unit 209 preferably provides stimuli from the following having a higher priority order according to a predetermined priority order: (i) stimuli according to the stimulus plan determined by the planning unit 207, and (ii) stimuli that do not match such stimuli, i.e., whichever has a higher priority. The predetermined priority order may be, for example, stimuli from the highest priority, stimuli corresponding to drowsiness, stimuli corresponding to absent-mindedness, stimuli corresponding to stress, stimuli corresponding to discomfort other than a specific state, and stimuli corresponding to comfort.

[0128] Based on the above, by providing high-priority stimuli first, responses with higher priority can be given priority. For example, in this way, in response to a specific state, stimuli corresponding to drowsiness are provided in a prioritized manner compared to stimuli corresponding to an inattentive state.

[0129] The presentation control unit 204 may preferably present on the display device 24 information regarding: (i) the driver's state identified by the state recognition unit 201, and (ii) the type of stimulus provided at the time of such identification of the driver's state (i.e., the currently activated stimulus type). In this way, the driver can easily recognize his / her current state and the stimulus provided to him / her. Furthermore, in addition to information regarding the driver's state and the currently activated stimulus type, information regarding: (i) the type of stimulus that should currently be provided to the driver according to the stimulus plan already determined by the planning unit 207 (i.e., the planned activation stimulus), and (ii) the driver's ideal state to be guided by this stimulus. In this way, the driver can easily recognize the difference between the plan and reality regarding the driver's state and the activated stimulus type. The presentation control unit 204 may display on the display unit 24 (e.g., on the same screen) in a comparable manner: (a) information regarding the current driver state and the currently activated stimulus type, and (b) information regarding the ideal state and the planned activation stimulus type. Note that "during travel on the predicted route" includes situations described later where the vehicle temporarily stops while its power switch is on.

[0130] Reference Figure 7 and Figure 8 This illustrates an example of how to display (a) information about the current driver state and the currently activated stimulus type, and (b) information about the ideal state and the planned stimulus type. First, by using... Figure 7 This shows an example of information being displayed overlaid on an electronic map. Figure 7 Point CP in the diagram represents the current position of this vehicle. Figure 7 The graphical representation of PU in the diagram indicates information about the current driver state and the type of stimulus currently being activated. Figure 7 The PMIc icon in the diagram represents information about the current driver status. Figure 7 The graphical CTS in the image represents the type of stimulus currently being activated. For example... Figure 7 As shown, information about the current driver status and the type of stimulus currently activated can be displayed as a pop-up graphic associated with the vehicle's current location. The current driver status can be displayed as... Figure 7 The PMIc icon is displayed. The currently activated stimulus type can be shown as... Figure 7 The graphical CTS in the image includes the name of the currently activated stimulus type and a graphic representing that stimulus type. For example... Figure 7 As shown, the ideal state can be overlaid on the electronic map along the predicted route. The type of stimulus to be activated can be represented by using different types of lines overlaid along the predicted route. Different types of lines can be represented in conjunction with... Figure 5They are categorized in the same way.

[0131] Now, by using Figure 8 This describes an example where information (a) regarding the current driver state and the currently activated stimulus type, and (b) regarding the ideal state and the planned stimulus type, are not shown overlapping on an electronic map. In this case, information (a) regarding the current driver state and the currently activated stimulus type, and information (b) regarding the ideal state and the planned stimulus type, can both be displayed in front of the driver, for example, by a head-up display (HUD). Graph Pr represents an example of information (a) regarding the current driver state and the currently activated stimulus type. Graph Sc represents an example of information (b) regarding the ideal state and the planned stimulus type. Both the current driver state and the ideal state can be represented by icons, such as... Figure 8 As shown in the image. The currently activated stimulus type can be displayed as... Figure 8 The graph Pr in the image includes the name of the currently activated stimulus type and a graph representing that type. The planned stimulus type can be displayed as... Figure 8 The graph Sc in the diagram includes the name of the stimulus type to be activated and a graph representing the stimulus type to be activated.

[0132] The presentation control unit 204 may preferably present preview information about the next stimulus according to the stimulus plan determined by the planning unit 207, while the currently active stimulus is still provided by the stimulus control unit 209. The next stimulus according to the stimulus plan determined by the planning unit 207 is a different stimulus from the currently active stimulus (i.e., a different type of stimulus). In this way, if the next stimulus in the stimulus plan is not the driver's preferred stimulus, the driver can or is able to change the next stimulus. The timing of providing the preview information may be at or near the end of the currently active stimulus. For example, the preview information may be provided 10 seconds before the end of the currently active stimulus. The preview information may be presented on the display unit 24 by a display. The preview information may also be presented by sound / voice output from the audio output device 25. The presentation control unit 204 may identify the next stimulus based on the stimulus plan determined by the planning unit 207 and the current position of the vehicle measured by the locator 6.

[0133] In the following text, by using Figure 9 Here's an example of how to display preview information. Preview information can be displayed using a HUD, a display unit on a navigation device, or a CID, etc. The preview information can be set as an image representing the next stimulus, such as... Figure 9 As shown in the image. Before switching to the next stimulus, the remaining time of the currently active stimulus can also be displayed as a preview, such as... Figure 9As shown. Information about the type of the next stimulus can also be displayed as preview information. Figure 9 The graphic NTS in is an example of preview information of the next stimulus, including the name and type of the next stimulus.

[0134] <Driver state guidance-related processes in HCU 20>

[0135] Subsequently, reference will be made to Figure 10 to describe an example of the flow of the driver state guidance-related processes in HCU 20 with reference to the flowchart of. Figure 10 The flowchart of can be configured such that, for example, when the switch for starting the internal combustion engine or the motor generator of the vehicle (hereinafter, the power switch) is turned on, the power of HCU20 is also turned on and this process is started. Additionally, in the case where the on / off setting for the function for executing the driver state guidance-related processes can be switched via the user input device 26, the condition for the function on for executing the driver state guidance-related processes can also be added to the condition for starting the process.

[0136] First, in step S1, the state recognition unit 201 recognizes the current driver state of the vehicle. In step S2, the plan-related process is executed, and the process continues to step S3. Here, reference will be made to Figure 11 to describe an example of the flow of the plan-related process with reference to the flowchart of.

[0137] In step S201, the target estimation unit 202 estimates the target state of the driver. In step S202, the route prediction unit 203 predicts the route of the vehicle. In Figure 5 the example of, the route prediction unit 203 is described as predicting multiple candidate routes. In step S203, the presentation control unit 204 presents information about the multiple candidate routes predicted in S202. In step S204, the route selection unit 205 selects one route from the multiple candidate routes predicted in S202 based on the selection input received from the driver. Note that in S202, the route prediction unit 203 can be configured to narrow down to one route and make a prediction. In this case, the processes of S203 to S204 may be omitted.

[0138] In step S205, the situation prediction unit 206 predicts the situation on the way of the predicted route selected in S204. If in S202 the route prediction unit 203 is configured to narrow down the prediction to one route, then in step S205, the situation on the way of the predicted route predicted in S202 is predicted. In step S206, the plan unit 207 determines the stimulus plan based on the driver state recognized in S1, the target state estimated in S201, and the situation on the way of the predicted route predicted in S205. In Figure 5In the example, it is assumed that for a predicted route, multiple candidate plans are determined.

[0139] In step S207, the presentation control unit 204 presents information on the multiple candidate plans determined in S206.

[0140] In step S208, the plan selection unit 208 selects a candidate plan from the multiple candidate plans determined in S206 based on the selection input received from the driver.

[0141] In step S209, planning unit 207 determines the candidate plan selected in S208 as the stimulus plan and continues to step S3.

[0142] Note that in S206, planning unit 207 can select one plan from multiple candidate plans. In this case, the processes from S207 to S208 can be omitted, and in S209, the stimulus plan determined in S206 can be effectively determined as the stimulus plan.

[0143] return Figure 10 In step S3, the stimulus control unit 209 begins to provide stimuli to the driver according to the stimulus plan determined in S209. Stimuli will be provided to the driver along the predicted route according to the stimulus plan determined in S209. For example, in congested sections, stimuli estimated to improve "annoyance" are provided. In monotonous driving sections, stimuli estimated to improve inattentiveness and drowsiness are provided. In high-load sections, inhibitory stimuli are provided. For sections where state changes are not predicted, stimuli that guide / lead the user to an ideal state in that section are provided.

[0144] In step S4, the status recognition unit 201 identifies the current driver status of the vehicle.

[0145] In step S5, if the stimulus according to the stimulus plan determined in S209 differs from the stimulus according to the current state identified in S4 (Yes in S5), the process continues to step S6. On the other hand, if there is no difference (No in S5), the process continues to step S8. In other words, the stimulus will continue according to the stimulus plan.

[0146] The stimulus required for the current state can be determined by the stimulus control unit 209 by referring to a pre-associated correspondence between driver states and stimuli. For example, for an uncomfortable state, a stimulus can be determined to eliminate the discomfort. For a comfortable state, a stimulus can be determined to maintain the comfort.

[0147] In step S6, if the stimulus identified in S4 is a priority stimulus over the stimulus according to the stimulus plan determined in S209 (Yes in S6), the process continues to step S7. On the other hand, if the stimulus is not a priority stimulus (No in S6), the process continues to step S8. In other words, the stimulus according to the stimulus plan will continue. The stimulus control unit 209 can determine the priority (i.e., priority order) based on the aforementioned priority (i.e., priority order).

[0148] In step S7, the stimulation control unit 209 provides the stimulus according to the current state identified in S4 in a manner that takes precedence over the stimulus according to the stimulation plan determined in S209.

[0149] In step S8, if the vehicle's travel route deviates from the predicted route used to determine the stimulus plan in S209 (Yes in S8), the process continues to step S9. Conversely, if the vehicle's travel route does not deviate from the predicted route (No in S8), the process continues to step S12. In other words, the stimulus plan will continue. Whether the vehicle's travel route deviates from the predicted route can be determined, for example, by the route prediction unit 203 using the vehicle position determined by the locator 6.

[0150] In step S9, a re-planning process is performed. This re-planning process may be the same as the planning process described above. During the re-planning process, the target state estimated during the planning process may be reused, or the target state may be re-estimated, for example, by re-executing the planning process.

[0151] In step S10, the presentation control unit 204 presents the changes in the stimulus plan caused by the rescheduling of the stimulus plan.

[0152] In step S11, the stimulation control unit 209 begins to stimulate the driver according to the stimulation plan redefined in the replanning process of S9.

[0153] In step S12, if it is the end sequence of the driver state guidance related process (yes in S12), the driver state guidance related process ends. On the other hand, if it is not the end sequence of the driver state guidance related process (no in S12), the process returns to S4 and repeats the process.

[0154] As an example of the termination sequence of a driver-state-guided process, the following can be considered: The vehicle's power switch has been turned off, the destination has been reached, or the function used to perform the driver-state-guided process has been switched off.

[0155] <Overview of the First Embodiment>

[0156] According to the configuration of the first embodiment, in addition to considering the identified current driver state and the driver's target state, a stimulation plan for stimulating the driver during the journey is determined based on the situation on the predicted route, so as to guide / lead the driver state to the target state. Therefore, for example, a stimulation plan / type that is more suitable for the changes in the situation during the vehicle's driving can be changed as needed and provided. In addition, during the journey on the predicted route, according to the determined plan, the driver is stimulated (i.e., a stimulus is provided to the driver), so as a stimulus for guiding the driver state to a favorable state, a stimulus that is most suitable for the situation (or most suitable for the change in the situation) can be provided.

[0157] According to the configuration of the first embodiment, the stimulation of the driver's five senses can be activated or inhibited according to the situation and plan without interfering with driving.

[0158] In addition, when the situation changes, the stimulation is rearranged so that the stimulation type can also be provided according to the changes in the situation during the vehicle's driving.

[0159] (Second Embodiment)

[0160] The present disclosure is not limited to the above configuration in the first embodiment. That is, the configuration in the second embodiment can also be adopted. Hereinafter, the configuration of the second embodiment is shown as an example in reference to the drawings. Except for the HCU 20, the in-vehicle system 1 in the second embodiment has the same configuration as the first embodiment, and the HCU 20 is replaced by the HCU 20a.

[0161] <Configuration of HCU 20a>

[0162] By using Figure 12 , the configuration of the HCU 20a is described. As the Figure 12 shown functional block, the HCU 20a includes a state recognition unit 201, a presentation control unit 204a, a planning unit 207a, a stimulation control unit 209, a recording process unit 210, a history storage unit 211, a mode switching unit 212, a state prediction unit 213, and an evaluation unit 214. From Figure 12Functional blocks unrelated to processes different from the first embodiment are omitted from the diagram. That is, as functional blocks, HCU 20a includes a target estimation unit 202, a route prediction unit 203, a route selection unit 205, a situation prediction unit 206, and a plan selection unit 208. Except for having a presentation control unit 204a and a planning unit 207a, and except for having a recording process unit 210, a history storage unit 211, a mode switching unit 212, a state prediction unit 213, and an estimation unit 214, HCU 20a is identical to HUC 20 in the first embodiment. HCU 20a also corresponds to a driver state guidance device. Furthermore, the process of executing these functional blocks via a computer corresponds to executing a driver state guidance method.

[0163] The recording process unit 210 stores in the history storage unit 211 the vehicle's past travel locations (hereinafter, location history), stimuli provided at those locations (hereinafter, stimulus history), and driver states at those locations (hereinafter, driver state history). The stimuli mentioned here refer to those provided to the driver by the stimulus control unit 209 according to the stimulus plan determined by the planning unit 207. The recording process unit 210 can identify the vehicle's travel location based on the vehicle's current location measured by the locator 6. The identified location can then be stored as location history. The recording process unit 210 may store stimuli provided at those locations by the stimulus control unit 209 as stimulus history for those locations. The recording process unit 210 may store driver states identified by the state recognition unit 201 as driver state history for those locations. The recording process unit 210 can then associate these histories in the history storage unit 211. The recording process unit 210 can store histories in trip order. A trip is recorded as the period of vehicle travel from the moment the vehicle's power switch is turned on to the moment it is turned off. The historical storage unit 211 can be implemented by using non-volatile memory or the like.

[0164] The mode switching unit 212 switches the display mode based on driver input from the user input device 26. The driver input can be a selection of one of the display modes. For example, the display modes may include normal mode, history mode, plan comparison mode, past comparison mode, and prediction mode. Normal mode is the default display mode. In normal mode, such as... Figure 7 As shown, this displays information related to the current vehicle driving plan. The history mode is a display mode that presents past data. (See image for reference.) Figure 7 and Figure 8As shown, the plan comparison mode is a display mode in which information about the current driver state and the type of stimulus currently activated, as well as information about the ideal state and the planned stimulus to be activated (hereinafter, the plan comparison display). The plan comparison display can be executed in normal mode, but in this example, it is configured to be executed when the plan comparison mode is selected. The past comparison mode is a display mode in which information about the current driver state and the type of stimulus currently activated, as well as information about the driver state and stimuli activated in the past history, is displayed. The prediction mode is a display mode in which predictions about changes in the driver state for stimulus provision and stimulus non-provision scenarios are displayed. Note that the display modes switchable by the mode switching unit 212 can be a subset of the above-described display modes.

[0165] State prediction unit 213 predicts changes in driver state in the following situations: (i) when stimulation is provided according to a stimulus plan determined by planning unit 207, and (ii) when stimulation is not provided. State prediction unit 213 can predict the ideal state determined by planning unit 207 as a change in driver state in a stimulus provision situation where stimulation is provided according to a stimulus plan determined by planning unit 207. As mentioned above, the ideal state includes a portion of the state and a portion of the target state. State prediction unit 213 may predict changes in driver state in a stimulus-not-provided situation based on the situation predicted by situation prediction unit 206. For example, at the moment when monotonous driving is predicted as an en route situation, a change in driver state to boredom and drowsiness can be predicted. In other cases, for example, at the moment when traffic congestion is anticipated as an en route situation, a change in driver state to annoyance can be predicted.

[0166] The presentation control unit 204a preferably displays historical information stored in the history storage unit 211 in an overlay manner, i.e., on top of the map, based on input received from the driver by the user input device 26. In this way, the driver can confirm and / or evaluate past history. When the history mode is selected, the presentation control unit 204a displays the historical information. The user input device 26 corresponds to the input receiving unit.

[0167] Historical information is displayed by using Figure 13 As described in the example, historical information is displayed on top of an electronic map on the screen of display device 24. Figure 13 The icon HMIc in the diagram represents information about the driver's status history. A set of historical locations for a single trip represents the route taken during that trip. Information about stimulus history is shown as... Figure 13 Different line types in the text. For example... Figure 13As shown, past driver states and previously provided stimuli are displayed in relation to each other along the route of a single trip. The presentation control unit 204a can simultaneously display historical information from multiple trips. For example, as... Figure 13 As shown, historical information can be displayed as a stack or layer from trip to trip. The history display at the front of the stack can be switched based on input from the driver, which can be received by the user input device 26.

[0168] In addition to displaying information about the driver's state identified by the state recognition unit 201 at this location, and the type of stimulus provided at this moment—that is, the type of stimulus currently active at the moment the driver's state was identified—the presentation control unit 204a may preferably display information about the type of stimulus and the driver's state at a specific location along a predetermined route during the vehicle's travel, based on the history stored in the history storage unit 211. In this way, the driver can easily recognize the driver's state and the type of stimulus activated at the same location in the current trip and past trips. That is, the differences between them (i.e., between the driver's state / stimulus type in the current trip and past trips) are easily understood. Hereinafter, the type of stimulus provided at the same location in the past is designated as the type of stimulus activated in the past. Furthermore, the driver's state at the same location in the past is designated as the past driver state. When the past comparison mode is selected, in addition to the current driver state and the currently active stimulus type, the presentation control unit 204a also displays the past driver state and the past active stimulus type. The presentation control unit 204a can display, in a comparison-enabled manner, (a) information about the current driver state and the currently activated stimulus type and (b) information about the past driver state and the past activated stimulus type.

[0169] The presentation control unit 204a preferably filters past locations from the travel history stored in the history storage unit 211 for trips with the same starting point. In this way, the driver can easily recognize differences in driver state / activation stimuli between current and past trips with similar routes. Therefore, the driver can easily compare similar situations in current and past trips to find differences in driver state and the type of activated stimuli. Furthermore, the presentation control unit 204a can also filter the history stored in the history storage unit 211 to find trips with the same route from the starting point to the destination. In this way, the driver can easily recognize differences in driver state / activation stimuli between current and past trips with the same route. Therefore, the driver can easily compare very similar situations in current and past trips to find differences in driver state and the type of activated stimuli.

[0170] By using Figure 14 and Figure 15 As an example, this describes how to display (a) information about the current driver state and the currently activated stimulus type, and (b) information about past driver states and past activated stimulus types. First, in... Figure 14 The image shows information from past history and current situations overlaid on an electronic map. Figure 14 Point CP in the diagram represents the current position of this vehicle. Figure 14 The graphical representation of PU indicates information about the current driver state and the type of stimulus currently being activated. The icon PMIc indicates information about the current driver state. The graphical representation of CTS indicates information about the type of stimulus currently being activated. (Example...) Figure 14 As shown, information about the current driver status and the type of stimulus currently activated can be displayed as a pop-up graphic associated with the vehicle's current location. The current driver status can be displayed as... Figure 14 The PMIc icon is displayed. The currently activated stimulus type can be shown as... Figure 14 The graphical CTS in the image includes the name of the currently activated stimulus type and a graphic representing that stimulus type. Past driver states can be displayed at the top of the electronic map along a route represented by a set of historical locations, as shown in [image / description]. Figure 14 The HMIc icon represents the type of stimulus that was activated in the past. This can be represented by lines of different types, shown overlapping along a route consisting of a set of historical locations. The different types of lines can be represented with... Figure 5 They are categorized in the same way.

[0171] Next, refer to Figure 15 This describes how, in the absence of an electronic map, information (a) regarding the current driver state and the type of stimulus currently activated, and (b) regarding past driver states and the types of stimuli previously activated, can be displayed in a non-overlapping manner. Displaying (a) information regarding the current driver state and the type of stimulus currently activated, and (b) information regarding past driver states and the types of stimuli previously activated, can be achieved, for example, by projecting it into the space in front of the driver using a HUD, i.e., onto the windshield. Graph Pr represents an example of (a) information regarding the current driver state and the type of stimulus currently activated. Graph Hi represents an example of (b) information regarding past driver states and the types of stimuli previously activated. Both the current and past driver states can be represented by icons, such as... Figure 15 As shown in the diagram. The currently activated stimulus type can be represented by its name and a graph, as illustrated by [example image / image]. Figure 15 The example of the graph Pr in the diagram illustrates this. The types of stimuli that were previously activated can be represented by using the names and graphs of the previously activated stimulus types, as shown by... Figure 15 The example of the graphic Hi in the image is shown.

[0172] The presentation control unit 204a preferably displays the prediction results predicted by the state prediction unit 213 on the display device 24. In this way, the driver can predictively recognize changes in the driver's state regarding (i) a stimulus provision situation in which a stimulus is provided according to the stimulus plan and (ii) a stimulus not provided. When the prediction mode is selected, the presentation control unit 204a displays the prediction results from the state prediction unit 213.

[0173] Now, by using Figure 16 This describes how the prediction results of the state prediction unit 213 are displayed. The presentation control unit 204a may present the transition of the driver's state from the current driver state for both stimulus-provided and stimulus-unprovided scenarios. More practically, as... Figure 16 As shown, driver states can be represented using icons that indicate facial expressions. Furthermore, facial expression icons can be arranged chronologically based on predictions of driver state transitions, separately for situations where stimuli are provided and situations where stimuli are not provided.

[0174] Evaluation unit 214 evaluates at least one of the following based on input received from the driver by user input device 26: (i) the stimulus currently being provided to the driver, (ii) the stimulus planned to be provided to the driver, and (iii) past stimuli provided to the driver. The input for evaluation can be, for example, input for selecting a stimulus to be evaluated (hereinafter, stimulus selection input) and input for evaluating the selected stimulus (hereinafter, stimulus evaluation input). User input device 26 for receiving stimulus selection input can be a touch panel. The stimulus selection input can then be provided as a touch operation on the screen of display device 24, thereby indicating one of the active stimulus types displayed thereon as information (e.g., as a button). That is, for example, on the displayed information, i.e. on... Figure 7 , Figure 8 , Figure 14 and Figure 15 A touch operation on one of the active stimulus types shown can be picked up as a selection input for the stimulus currently being provided to the driver. Alternatively, it can be displayed on the information, i.e., on... Figure 5 A touch operation on one of the activated stimulus types shown can be picked up as a selection input for the stimulus to be provided to the driver. Furthermore, in the untraveled portion of the route, on the displayed information, i.e. Figure 7 A touch operation on one of the activated stimulus types shown can be picked up as a selection input for the stimulus to be provided to the driver. Furthermore, in the displayed information, i.e. Figure 9The touch operation on the next stimulus shown can also be picked up as a selection input for the stimulus to be provided to the driver. Alternatively, it can be on the displayed information, i.e., on Figure 13 , Figure 14 Touch operations on one of the previously activated stimulus types shown can be picked up as a selection input for stimuli previously provided to the driver. Furthermore, in the already traveled portion of the route, on the displayed information, i.e.... Figure 7 Touch operations on one of the activated stimulus types shown can be picked up as a selection input for stimuli previously provided to the driver.

[0175] For example, stimulus assessment input can be used as Figure 17 The scoring input shown is received. In this case, evaluation unit 214 can evaluate stimuli with high score counts as high-evaluation stimuli. Evaluation unit 214 can evaluate stimuli with reference to similar situations along the route, where such stimuli are identified as being provided in the stimulus plan. Furthermore, the stimulus evaluation input can be used to allow the driver / user to select from, for example... Figure 18 The preferred selection option is chosen from the selection options shown to change the stimulus. In this case, the evaluation unit 214 can improve the evaluation of the selected (i.e., preferred) stimulus and decrease the evaluation of the stimulus replaced by the preferred stimulus.

[0176] Planning unit 207a can preferably make it easy to pick up stimuli highly evaluated by evaluation unit 214 and incorporate them into the stimulus plan, and can preferably make it difficult to incorporate stimuli poorly evaluated by evaluation unit 214 into the stimulus plan. In this way, stimulus selection can be adjusted / improved to reflect the driver's preferences.

[0177] (Third Implementation)

[0178] The stimuli for guiding / leading the driver's state are not limited to those described in the first embodiment. For example, other types of stimuli, such as vibrations from a vibrator, can be configured to guide / lead the driver's state.

[0179] (Fourth Implementation)

[0180] In the above embodiment, HCU 20 is configured to perform driver state guidance related processes. However, this disclosure is not necessarily limited to this configuration. For example, driver state guidance related processes may be performed by HCU 20 and other ECUs, or may be performed by other ECUs.

[0181] It should be noted that this disclosure is not limited to the above-described embodiments, and various modifications can be made within the scope indicated in the claims. Furthermore, embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of this disclosure.

[0182] Alternatively, the control unit and methods described in this disclosure can be implemented by a dedicated computer configured with a processor, the processor being programmed to perform one or more functions embodied by a computer program.

[0183] Alternatively, the apparatus / device and methods described in this disclosure may be implemented using dedicated hardware logic circuitry.

[0184] Alternatively, the apparatus and methods described in this disclosure may be implemented by one or more dedicated computers configured by a combination of a processor that executes computer programs and one or more hardware logic circuits.

[0185] Furthermore, computer programs can be stored as program products in a computer-readable, non-transitory tangible storage medium as instructions to be executed by a computer.

[0186] (The technical ideas disclosed in this disclosure)

[0187] This disclosure discloses a plurality of technical ideas, listed consecutively in number. A portion of each of the technical ideas selectively depends on a prior idea in multiple dependent forms. Furthermore, a portion of each of the technical ideas depends on a prior idea in multiple dependent forms referencing another multiple dependent form. These multiple dependent forms define the plurality of technical ideas.

[0188] Technical Idea 1

[0189] A driver status guidance device that can be used in a vehicle includes:

[0190] A status recognition unit that identifies the driver's status in the vehicle;

[0191] The target estimation unit estimates the target state to which the driver's state is guided.

[0192] Route prediction unit, which provides predicted routes for vehicles;

[0193] The situation prediction unit predicts the situation along the predicted route, which is predicted by the route prediction unit.

[0194] The planning unit plans stimuli to be used along the predicted route to guide the driver's state to the target state, based on: (i) the state identified by the state recognition unit, (ii) the target state estimated by the target estimation unit, and (iii) the situation along the predicted route predicted by the situation prediction unit; and

[0195] The stimulus control unit provides the driver with stimuli based on a stimulus plan determined by the planning unit along the predicted route.

[0196] Technical Idea 2

[0197] The driver state guidance device according to technical concept 1 may further include the following: That is, the situation prediction unit predicts high-load situations where the driver's driving load is high, and...

[0198] The planning unit determines the stimulus plan for suppressing the stimulus based on the portion of the predicted route that is in the high-load condition predicted by the prediction unit.

[0199] Technical Idea 3

[0200] The driver state guidance device according to technical concept 1 or 2 may also include the following: That is, the planning unit determines a stimulus plan in which a stimulus of short duration less than a threshold is arranged to replace a stimulus preceding or following the short-duration stimulus.

[0201] Technical Idea 4

[0202] The driver state guidance device according to any one of technical concepts 1 to 3 may further include the following: That is, a situation prediction unit predicts the deterioration of the driver's state, and

[0203] For the portion of the predicted route that forecasts a worsening situation, the planning unit determines the following stimulus plan: (i) to provide improvement stimulus to improve the worsening situation, and (ii) to begin providing improvement stimulus before entering this portion of the predicted route.

[0204] Technical Idea 5

[0205] The driver state guidance device according to any one of technical concepts 1 to 4 may further include the following: that is, the situation prediction unit predicts at least one of high load conditions, congestion levels, and monotonous driving.

[0206] Technical Idea 6

[0207] The driver state guidance device according to any one of technical concepts 1 to 5 may further include the following: That is, the presentation control unit presents planning-related information regarding a stimulus plan determined by the planning unit for providing stimulation to the driver.

[0208] Technical Ideas 7

[0209] The driver state guidance device according to technical concept 6 may also include the following: That is, the presentation control unit presents planning-related information at least as a display on a display device, and the display of planning-related information overlaid on a map at least partially includes: (a) a predicted route, (b) the type of stimulus provided to the driver along the predicted route according to the stimulus plan determined by the planning unit, and (c) the ideal state of the driver to which the provided stimulus guides the driver.

[0210] Technical Ideas 8

[0211] The driver state guidance device according to technical concept 6 or 7 may also include the following: That is, the presentation control unit presents at least the planning-related information as a display on the display device. The display of the planning-related information includes, in addition to (B) information about (c) the stimuli planned by the planning unit to be provided to the driver according to the stimulus plan determined by the planning unit when identifying the driver's state, and (d) the ideal state of the driver to be guided / induced by the planned stimuli, (A) information about (a) the driver's state identified by the state recognition unit, and (b) the type of stimulus provided to the driver when identifying the driver's state during the vehicle's travel along the predicted route.

[0212] Technical Idea 9

[0213] The driver state guidance device according to any one of technical concepts 6 to 8 may further include the following. That is, the driver state guidance device also includes a history storage unit that stores the history of: (i) the vehicle's past travel positions, (ii) the stimuli provided at each of the past travel positions, and (iii) the driver's state at each of the past travel positions.

[0214] The presentation control unit presents information at least as a display on a display device. The information displayed overlaid on the map is about historical information stored in the history storage unit and selected based on input received by the input receiving unit that receives input from the driver.

[0215] Technical Ideas 10

[0216] The driver state guidance device according to any one of technical concepts 6 to 9 may further include the following. That is, the driver state guidance device also includes a history storage unit that stores the history of: (i) the vehicle's past travel positions, (ii) the stimuli provided at each of the past travel positions, and (iii) the driver's state at each of the past travel positions.

[0217] The presentation control unit presents information at least as a display on a display device, in addition to (B) information about the type of stimulus and the driver’s state at each of the same historical locations stored in the historical storage unit, the information also includes (A) information about (a) the driver’s state as identified by the state recognition unit, and (b) information about the type of stimulus provided to the driver when the driver’s state is identified during the vehicle’s travel along the predicted route.

[0218] Technical Idea 11

[0219] The driver state guidance device according to any one of technical concepts 6 to 10 may further include the following: That is, the presentation control unit presents preview information while the stimulus control unit continues to provide stimulation to the driver; the preview information is presented as a preview of the next stimulus immediately following the current stimulus according to the stimulus plan determined by the planning unit.

[0220] Technical Ideas 12

[0221] The driver state guidance device according to any one of technical concepts 6 to 11 may further include the following. That is, the driver state guidance device further includes an evaluation unit that evaluates at least one of the following based on input received from the driver by the input receiving unit: (i) the stimulus currently being provided to the driver, (ii) the stimulus planned to be provided to the driver, and (iii) past stimuli provided to the driver in the past, and...

[0222] The higher the assessment of a stimulus by the assessment unit, the easier it will be for the planning unit to pick up and incorporate that stimulus into the stimulus plan.

[0223] The lower the assessment unit's rating of a stimulus, the more difficult it is for that stimulus to be picked up and incorporated into the stimulus program.

[0224] Technical Ideas 13

[0225] The driver state guidance device according to any one of technical ideas 6 to 12 may further include the following. That is, the driver state guidance device further includes a state prediction unit that predicts changes in the driver's state in the following situations: (i) a stimulus provision situation in which a stimulus is provided according to a stimulus plan determined by a planning unit, and (ii) a stimulus not provided situation, and the presentation control unit presents the information at least as a display on a display device, the display of the information being the prediction result of the state prediction unit.

[0226] Technical Idea 14

[0227] The driver state guidance device according to any one of technical concepts 6 to 13 may further include the following: That is, the route prediction unit re-predicts the vehicle's route when the actual route traveled by the vehicle deviates from the predicted route.

[0228] The situation prediction unit predicts the conditions along the re-predicted route when the route prediction unit provides the vehicle with the re-predicted route.

[0229] The planning unit re-determines the stimulus plan for stimulating the driver when the route prediction unit provides the vehicle with a re-predicted route, and

[0230] When the planning unit redefines the stimulus plan for stimulating the driver, the presentation control unit shall at least present the changes in the stimulus plan caused by the redefinition of the stimulus plan as plan-related information.

[0231] Technical Ideas 15

[0232] The driver state guidance device according to any one of technical concepts 6 to 14 may further include the following: That is, the route prediction unit predicts multiple candidate routes as the vehicle's route.

[0233] The presentation control unit presents at least the information of candidate routes predicted by the route prediction unit as planning-related information.

[0234] A route selection unit is provided that allows the user to select one of multiple candidate routes predicted by a route prediction unit based on selection input received from the driver.

[0235] The situation prediction unit predicts the situation along the predicted route, which is selected by the route selection unit as one of the multiple candidate routes predicted by the route prediction unit.

[0236] Technical Ideas 16

[0237] The driver state guidance device according to any one of technical concepts 6 to 15 may further include the following: That is, the planning unit determines multiple candidate plans for a predicted route as stimulus plans for stimulating the driver.

[0238] The presentation control unit presents at least the information of the candidate plans determined by the planning unit as plan-related information.

[0239] The driver status guidance device also includes a plan selection unit, which selects one of several candidate plans determined by the planning unit based on selection input received from the driver.

[0240] The stimulus control unit provides the driver with a stimulus along the predicted route based on a stimulus plan selected by the plan selection unit from multiple candidate plans determined by the plan selection unit.

[0241] Technical Ideas 17

[0242] The driver state guidance device according to any one of technical concepts 1 to 16 may further include the following: That is, a situation prediction unit predicts the change in the driver state from the driver state before the change to the driver state after the change, and...

[0243] In the case of a changed driver state, the planning unit re-determines the stimulus plan for using stimuli based on the actual changed driver state identified by the state recognition unit for the predicted route regarding the changes predicted by its state prediction unit.

[0244] Technical Ideas 18

[0245] The driver state guidance device according to any one of technical concepts 1 to 17 may further include the following: That is, the state recognition unit identifies a state including a specific state as the driver state, wherein the specific state is at least one of drowsy state, inattentive state, and stressed state, and

[0246] Regardless of the stimulus plan determined by the planning unit, the stimulus control unit applies stimuli to the driver on the predicted route / along the route based on the specific state identified by the state recognition unit, which are estimated to improve the specific state.

[0247] Technical Ideas 19

[0248] The driver state guidance device according to technical concept 18 may further include the following: That is, when the stimulus control unit induces a stimulus estimated to improve a specific state, and after the state identified by the state recognition unit has been improved from the specific state by such stimulus, regardless of the stimulus plan determined by the planning unit, based on the fact that (i) there is a mismatch between the unguided stimulus and (ii) the stimulus according to the stimulus plan determined by the planning unit, the stimulus control unit provides a prioritized "unguided" stimulus along the predicted route, which is a stimulus estimated not to guide the driver state back to the specific state.

[0249] Technical Ideas 20

[0250] The driver state guidance device according to technical concept 18 may further include the following: That is, the state recognition unit will recognize the driver state as selected from the following states: (i) a specific state that is at least one of drowsy state, inattentive state, and stressed state; and (ii) other states besides the specific state, including uncomfortable state and comfortable state.

[0251] If a stimulus that does not match the stimulus plan determined by the planning unit is requested along the predicted route, the stimulus control unit provides a higher priority stimulus selected from the following according to a predetermined priority order: (i) the stimulus according to the stimulus plan determined by the planning unit, and (ii) the stimulus that does not match such a stimulus. The predetermined priority order of the stimuli, in descending order of priority, is: stimuli corresponding to drowsiness, stimuli corresponding to absent-mindedness, stimuli corresponding to stress, stimuli corresponding to discomfort, and stimuli corresponding to comfort.

[0252] Technical Ideas 21

[0253] A method for guiding the state of a driver that can be used in a vehicle, the method being executable by at least one processor, the method comprising:

[0254] The state recognition step identifies the state of the vehicle's driver;

[0255] The target state estimation step estimates the target state, which is the state to which the driver is guided.

[0256] The route prediction step provides a predicted route for the vehicle;

[0257] The situation prediction step involves predicting the situation of drivers and vehicles along the predicted route given by the route prediction step.

[0258] The planning steps plan stimuli to be used along the predicted route to guide the driver's state to the target state, based on: (i) the driver's state identified by the state identification step, (ii) the target state estimated by the target state estimation step, and (iii) the situation along the predicted route predicted by the situation prediction step; and

[0259] The stimulus control step provides the driver with stimuli according to the stimulus plan determined by the planning step during the vehicle's journey along the predicted route.

Claims

1. A driver status guidance device that can be used in a vehicle, comprising: A state recognition unit that identifies the current state of the driver of the vehicle; A target estimation unit that estimates the target state of the driver; A route prediction unit that provides a predicted route for the vehicle; The situation prediction unit predicts the situation in the predicted route; A planning unit that plans a stimulus plan based on the following, the stimulus plan including stimuli to be used on the predicted route to guide the driver from the current state to the target state: (i) The current state identified by the state recognition unit, (ii) the target state estimated by the target estimation unit, and (iii) The situation predicted by the situation prediction unit; and Stimulation control unit, which provides stimulation according to the stimulation plan.

2. The driver status guidance device according to claim 1, wherein, The situation prediction unit predicts the high-load situation where the driver's driving load is high, and The planning unit plans a stimulation program for the partial inhibitory stimulation associated with the high-load condition for the predicted route.

3. The driver status guidance device according to claim 1, wherein, The planning unit plans the following stimulus program, in which the first stimulus, arranged to have a short duration less than a threshold, is replaced by the following: (i) A second stimulus placed before a stimulus of short duration, or (ii) A third stimulus that is placed after a stimulus of short duration.

4. The driver status guidance device according to claim 1, wherein, The situation prediction unit predicts the deterioration of the driver's condition, and For the portion of the predicted route associated with the deterioration, the planning unit plans the following stimulus measures: (i) Provide improving stimuli to improve the worsening situation, and (ii) The improvement stimulus is provided before entering the portion of the predicted route associated with the deterioration.

5. The driver status guidance device according to claim 1, wherein, The situation prediction unit predicts at least one of the following: (i) High load conditions, (ii) the degree of congestion, and (iii) Monotonous driving.

6. The driver status guidance device according to claim 1, further comprising: The presentation control unit displays plan-related information about the stimulus program.

7. The driver status guidance device according to claim 6, wherein, The presentation control unit at least presents the plan-related information as a display on a display device, and the display of the plan-related information overlaid on the map at least partially includes: (a) The predicted route, (b) The type of stimulus provided to the driver along the predicted route according to the stimulus plan determined by the planning unit, and (c) The driver is guided to the driver’s ideal state by the provided stimulus.

8. The driver status guidance device according to claim 6, wherein, (i) The presentation control unit at least presents the plan-related information as a display on the display device. (ii) The display of the plan-related information includes: (a) The current state of the driver as identified by the state recognition unit. (b) The type of stimulus provided to the driver when the driver's state is identified during the vehicle's travel along the predicted route. (c) When identifying the driver's state, the stimulus planned by the planning unit to be provided to the driver according to the stimulus plan determined by the planning unit, and (d) The ideal state of the driver to be guided / induced by planned stimuli.

9. The driver status guidance device according to claim 6, further comprising: Historical storage units store the history of the following items: (i) The past travel location of the vehicle, (ii) the stimuli provided at each of the previously traveled locations, and (iii) The state of the driver at each of the previously traveled locations, wherein, The presentation control unit presents information at least as a display on a display device, and the display of the information, which is overlaid on a map, is about the historical information stored in the history storage unit and selected based on the input received by the input receiving unit that receives the driver's input.

10. The driver status guidance device according to claim 6, further comprising: Historical storage units store the history of the following items: (i) The past travel location of the vehicle, (ii) the stimuli provided at each of the previously traveled locations, and (iii) The state of the driver at each of the previously traveled locations, wherein, The presentation control unit presents information at least as a display on a display device, the display of which includes: (a) The state of the driver as identified by the state recognition unit. (b) The type of stimulus provided to the driver when the driver's state is identified during the vehicle's travel along the predicted route. (c) the type of stimulus, and (d) The state of the driver at each of the same historical locations stored in the historical storage unit.

11. The driver status guidance device according to claim 6, wherein, The presentation control unit presents preview information while still providing stimulation to the driver from the stimulus control unit, wherein the preview information is presented as a preview of the next stimulus immediately following the current stimulus according to the stimulus plan determined by the planning unit.

12. The driver status guidance device according to claim 6, further comprising: An evaluation unit that evaluates at least one of the following based on input from the driver received by an input receiving unit that receives input from the driver: (i) The stimulus currently being provided to the driver, (ii) The stimuli to be provided to the driver, and (iii) Past stimuli provided to the driver in the past, wherein, The higher the evaluation of the stimulus by the evaluation unit, the easier it is for the planning unit to pick up and incorporate the stimulus into the stimulus plan. The lower the evaluation of a stimulus by the evaluation unit, the more difficult it is for that stimulus to be picked up and incorporated into the stimulus program.

13. The driver status guidance device according to claim 6, further comprising: A state prediction unit that predicts changes in the driver's state with respect to the following conditions: (i) wherein the stimulus delivery situation is provided according to the stimulus plan determined by the planning unit, and (ii) Situation where no stimulus is provided, wherein, The presentation control unit presents information at least as a display on a display device, and the display of the information is the prediction result of the state prediction unit.

14. The driver status guidance device according to claim 6, wherein, The route prediction unit re-predicts the vehicle's route when the actual route traveled by the vehicle deviates from the predicted route. The situation prediction unit predicts the situation in the re-predicted route when the route prediction unit provides the re-predicted route for the vehicle. The planning unit replans the stimulus plan for stimulating the driver when the route prediction unit provides a re-predicted route for the vehicle, and The presentation control unit at least presents the changes to the stimulus plan caused by the rescheduling of the stimulus plan as relevant information about the plan.

15. The driver status guidance device according to claim 6, further comprising: The route selection unit, in which, The route prediction unit predicts candidate routes as the vehicle's route. The presentation control unit presents at least the information of the candidate routes as plan-related information. The route selection unit selects one of the candidate routes predicted by the route prediction unit based on the selection input received from the driver, and The situation prediction unit predicts the situation in the selected route.

16. The driver status guidance device according to claim 6, further comprising: Plan to select units, where, The planning unit determines candidate stimulus plans for stimulating the driver based on a predicted route. The presentation control unit presents at least the information of the candidate stimulus plans as plan-related information. The plan selection unit selects one of the candidate stimulus plans based on the selection input received from the driver, and The stimulus control unit provides the driver with stimuli along the predicted route based on the selected candidate stimulus plan.

17. The driver status guidance device according to claim 1, wherein, The situation prediction unit predicts the changes in the driver's state from the state before the change to the state after the change, and The planning unit replans the stimulus plan based on the changed driver state actually identified by the state recognition unit for a portion of the predicted route.

18. The driver status guidance device according to claim 1, wherein, The state recognition unit identifies a state including a specific state as the driver's state, wherein the specific state is at least one of the following: (i) drowsy state, (ii) inattentive state, and (iii) stressed state. Regardless of the stimulus plan devised by the planning unit, the stimulus control unit applies stimuli to the driver that are estimated to improve the specific state.

19. The driver status guidance device according to claim 18, in, When the stimulus control unit induces a stimulus estimated to improve the specific state, and after the state identified by the state recognition unit has been improved from the specific state by such stimulus, the stimulus control unit provides unguided stimuli in a prioritized manner in the predicted path. Wherein, regardless of the stimulus plan determined by the planning unit, because the unguided stimulus does not match the stimulus previously planned according to the previous stimulus plan, the unguided stimulus is estimated not to guide the driver state back to the specific state.

20. The driver status guidance device according to claim 18, wherein, The state recognition unit will identify the driver's state from the following selected states: (i) the specific state being at least one of the drowsy state, the absent-minded state, and the stressed state, and (ii) Other states besides the specific states described above, including uncomfortable and comfortable states, and In the event that the stimulus does not match the stimulus in the stimulus plan determined by the planning unit in the predicted route, the stimulus control unit provides a higher priority stimulus selected from the following according to a predetermined priority order: (i) the stimulus according to the stimulus plan, and (ii) the stimulus not according to the stimulus plan, wherein the predetermined priority order of the stimuli is in descending order of priority: (a) Stimuli corresponding to drowsiness, (b) stimuli corresponding to the aforementioned absent-minded state, (c) A stimulus corresponding to the aforementioned stressful state, (d) The stimulus corresponding to the aforementioned discomfort, and (e) Stimuli corresponding to the comfort state.

21. A method for guiding the state of a driver, usable in a vehicle, the method being executable by at least one processor, the method comprising: The state recognition step identifies the current state of the driver of the vehicle; The target state estimation step estimates the target state, which is the state to which the driver's current state is guided. The route prediction step provides a predicted route for the vehicle; The situation prediction step predicts the situation in the predicted route; The planning steps involve planning stimuli to be used in the predicted route to guide the driver's current state to the target state as a stimulus plan, based on the following: (i) The driver's state identified by the state recognition step, (ii) the target state estimated by the target state estimation step, and (iii) The situation on the predicted route predicted by the situation prediction step; and The stimulus control step provides stimulation to the driver according to the stimulus plan planned by the planning step.

Images