Vehicle auxiliary device and vehicle auxiliary method

Abstract

By having an important factor estimation unit (212) that estimates an important factor of stress of the driver of the host vehicle, a recommended route decision unit (203) that decides a recommended route including a plurality of route sections that is estimated to be effective for alleviation of stress in correspondence with the important factor of stress estimated by the important factor estimation unit (212), and a prompt control unit (204) that prompts the driver toward the recommended route decided by the recommended route decision unit (203), it is possible to more accurately improve the stress of the driver of the host vehicle.

Description

[0001] Cross-references of related applications

[0002] This application is based on Japanese Patent Application No. 2020-204478, filed in Japan on December 9, 2020, and is incorporated herein by reference in its entirety. Technical Field

[0003] This disclosure relates to auxiliary devices and methods for vehicles. Background Technology

[0004] Techniques for checking the state of a vehicle driver are known. For example, Patent Document 1 discloses a technique for estimating the emotion of an individual based on the recognition results of facial expressions.

[0005] Patent Document 1: Japanese Patent No. 6467965

[0006] There is a possibility that the driver's mental state may affect the vehicle's operation. Therefore, it is preferable to improve the driver's mental state when it deteriorates. Summary of the Invention

[0007] One object of this disclosure is to provide a vehicle assistance device and a vehicle assistance method that can more accurately improve the mental state of a vehicle driver when the driver's mental state deteriorates.

[0008] The reference numerals in parentheses in the claims indicate the correspondence between specific units described in the embodiments described later as an example, and do not limit the technical scope of this disclosure.

[0009] To achieve the above objectives, the vehicle assistance device disclosed herein includes: a mental state estimation unit that estimates a mental state, wherein the mental state is at least one of the degree of deterioration of the driver's mental state and important factors contributing to the deterioration of the driver's mental state; a route recommendation unit that determines a recommended route comprising multiple road segments corresponding to the mental state estimated by the mental state estimation unit; and a prompt control unit that prompts the driver with the recommended route determined by the route recommendation unit.

[0010] To achieve the above objectives, the vehicle assistance method of this disclosure includes the following steps performed by at least one processor: a mental state estimation step, estimating a mental state, wherein the mental state is at least one of the degree of deterioration of the driver's mental state and important factors of the deterioration of the driver's mental state; a recommended route determination step, determining a recommended route comprising multiple road segments corresponding to the mental state estimated by the mental state estimation step; and a prompt control step, prompting the driver to the recommended route determined by the recommended route determination step.

[0011] Accordingly, it is possible to suggest recommended paths to the driver based on at least one of the factors related to the degree of deterioration in the driver's mental state and the key factors contributing to this deterioration. This allows for the provision of more detailed suggestions than simply whether a deterioration exists, including the degree of deterioration and the key factors contributing to it, to improve the driver's mental state. As a result, when a driver's mental state deteriorates, it is possible to improve their mental state with greater precision. Attached Figure Description

[0012] Figure 1 This is a diagram illustrating an example of the general structure of a driver assistance system 1.

[0013] Figure 2 This is a diagram illustrating an example of the general structure of HCU20.

[0014] Figure 3 This is a flowchart illustrating an example of the state improvement association processing flow in HCU20.

[0015] Figure 4 This is a diagram illustrating an example of the general structure of HCU20a.

[0016] Figure 5 This is a flowchart illustrating an example of the state improvement association processing flow in HCU20a.

[0017] Figure 6 This is a diagram illustrating an example of the general structure of HCU20b. Detailed Implementation

[0018] Referring to the accompanying drawings, several embodiments disclosed herein will be described. Furthermore, for ease of explanation, sometimes the same reference numerals are used between several embodiments for parts having the same function as those shown in the figures used in the description so far, and their descriptions are omitted. Parts with the same reference numerals can be referred to in the description of other embodiments.

[0019] (Implementation Method 1)

[0020] <Brief Structure of Driver Assistance System 1>

[0021] Hereinafter, this embodiment will be described using the accompanying drawings. Figure 1 The driving assistance system 1 shown can be used in a car (hereinafter referred to as a vehicle). Driving assistance system 1 includes an HMI (Human Machine Interface) system 2, a communication module 3, a locator 4, a map database (hereinafter referred to as a map DB) 5, a vehicle control ECU 6, a surrounding monitoring sensor 7, an autonomous driving ECU 8, and a vehicle status sensor 9. The HMI system 2, communication module 3, locator 4, map DB 5, vehicle control ECU 6, autonomous driving ECU 8, and vehicle status sensor 9 are connected, for example, to the vehicle's LAN. Hereinafter, the vehicle using driving assistance system 1 will be referred to as this vehicle.

[0022] Communication module 3 includes a short-range communication unit 31 and a wide-area communication unit 32. The short-range communication unit 31 performs short-range wireless communication. Examples of short-range wireless communication include Bluetooth Low Energy (BLE) and UWB (Ultra Wide Band) communication. Wi-Fi and ZigBee can also be used. The short-range communication unit 31 communicates with mobile terminals via short-range wireless communication. Examples of mobile terminals include wearable devices worn by passengers in the vehicle and multi-functional mobile phones carried by passengers in the vehicle. The wide-area communication unit 32 communicates with a central server via a public communication network. Examples of public communication networks include mobile phone networks and the Internet. The wide-area communication unit 32 can also be configured to communicate with the central server via a roadside unit. The wide-area communication unit 32 obtains traffic information from the central server and outputs it to the vehicle's LAN.

[0023] The locator 4 includes a GNSS (Global Navigation Satellite System) receiver and inertial sensors. The GNSS receiver receives positioning signals from multiple artificial satellites. The inertial sensors include, for example, gyroscope sensors and accelerometer sensors. The locator 4 sequentially measures the vehicle's position by combining the positioning signals received by the GNSS receiver and the measurement results from the inertial sensors. Alternatively, the vehicle position measurement can also utilize the travel distance calculated based on signals sequentially output from a vehicle speed sensor mounted on the vehicle.

[0024] Map DB5 stores guidance map data and high-precision map data. Map DB5 is, for example, a non-volatile memory. The guidance map data and high-precision map data may also be stored in separate memories. The guidance map data and high-precision map data may also be obtained from a central server outside the vehicle using the wide area communication unit 32. Alternatively, if the vehicle is not an autonomous vehicle capable of autonomous driving, high-precision map data may not be included in Map DB5.

[0025] Navigation data is the map data used by navigation functions that provide route guidance. As an example, it's used to search for recommended routes to the destination and provide route guidance along those routes. Navigation data represents roads used by vehicles through nodes and road segments. Nodes are points on the map where roads intersect, branch, and merge. Road segments connect nodes. A road segment represents a section of road. Road segment data consists of a unique identifier for the road segment, the segment length, the segment direction, the shape information of the segment, the coordinates of the starting and ending nodes, and various road attribute data. Road attributes include road name, road type, road width, number of lanes, and speed limit. On the other hand, node data consists of a node ID (with a unique identifier attached to each node on the map), node coordinates, node name, node type, connecting segment ID (describing the road segment ID connected to the node), and intersection type data.

[0026] High-precision map data is map data used for vehicle driving control. It is more detailed than navigation map data. High-precision map data may include, for example, 3D maps composed of point groups representing road shapes and features of structures.

[0027] The vehicle control ECU6 is an electronic control device that performs acceleration / deceleration control and / or steering control of the vehicle. The vehicle control ECU6 includes a steering control ECU, a power unit control ECU, and a braking ECU. The steering control ECU performs steering control. The power unit control ECU performs acceleration / deceleration control. The braking ECU performs deceleration control. The vehicle control ECU6 acquires detection signals from various sensors installed in the vehicle, such as the accelerator position sensor, brake travel sensor, steering angle sensor, and vehicle speed sensor, and outputs control signals to various driving control devices such as the electronic throttle valve, brake actuator, and EPS (Electric Power Steering) motor.

[0028] The surrounding monitoring sensor 7 detects moving objects such as pedestrians and other vehicles, as well as stationary objects such as fallen objects on the road, and other objects around the vehicle. In addition, it detects road markings such as lane markings around the vehicle. The surrounding monitoring sensor 7 can be, for example, a surrounding monitoring camera that captures images of a defined area around the vehicle, millimeter-wave radar, sonar, or LIDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging) sensors that transmit detection waves to a defined area around the vehicle. The surrounding monitoring camera sequentially outputs the captured images as sensing information to the autonomous driving ECU 8. Multiple cameras can also be configured to capture images of the entire area around the vehicle. Sensors that transmit detection waves, such as sonar, millimeter-wave radar, and LIDAR, sequentially output the scan results based on the received signals obtained after receiving reflected waves from obstacles as sensing information to the autonomous driving ECU 8. The surrounding monitoring sensor 7 detects the environmental conditions around the vehicle.

[0029] The autonomous driving ECU8 performs autonomous driving functions by acting as an agent for the driver's driving operations through the vehicle control ECU6. The degree of autonomous driving within this function (hereinafter referred to as the automation level) may have multiple levels, as defined by SAE. Automation levels are, for example, classified as levels 0 to 5 in the SAE definition.

[0030] Level 0 is the level where the system does not intervene and the driver performs all driving tasks. Driving tasks include, for example, steering and acceleration / deceleration. Level 0 is equivalent to manual driving. Level 1 is the level where the system assists with either steering or acceleration / deceleration. Level 2 is the level where the system assists with both steering and acceleration / deceleration. Levels 1 and 2 are equivalent to driver assistance systems.

[0031] Level 3 is a level where the system can perform all driving tasks in specific locations such as highways, with the driver taking over driving duties in emergencies. Level 3 requires the driver to respond quickly to requests for driver substitution from the system. Level 3 is equivalent to conditional automated driving. Level 4 is a level where the system can perform all driving tasks except in specific situations such as roads that the system cannot handle or extreme environments. Level 4 is equivalent to highly automated driving. Level 5 is a level where the system can perform all driving tasks in all environments. Level 5 is equivalent to fully automated driving. Levels 3 through 5 are equivalent to automated driving.

[0032] In Embodiment 1, the case where the vehicle is capable of Level 3 or higher autonomous driving will be described as an example. It is also possible to switch between levels of automation. In Embodiment 1, the case where it is possible to switch between Level 3 or higher autonomous driving and Level 0 manual driving will be described later. Furthermore, the vehicle may also be a vehicle that cannot perform Level 3 or higher autonomous driving. In this case, instead of searching for a recommended route through the autonomous driving ECU 8, a recommended route can be searched using the navigation device used by the vehicle.

[0033] The autonomous driving ECU 8 identifies the vehicle's driving environment based on the vehicle's position obtained from the locator 4, high-precision map data obtained from the map DB5, and detection results from the surrounding monitoring sensors 7. For example, based on the detection results from the surrounding monitoring sensors 7, it identifies the shape and movement of objects around the vehicle, or the shape of signs around the vehicle. Furthermore, by combining this data with the vehicle's position and the high-precision map data, it generates a virtual space that recreates the actual driving environment in three dimensions.

[0034] Furthermore, the autonomous driving ECU8 generates a driving plan based on the identified driving environment, enabling the vehicle to drive automatically using the autonomous driving function. This driving plan includes both medium- to long-term and short-term plans. In the medium- to long-term driving plan, a path is generated to take the vehicle to a designated destination. This path consists of multiple road segments. The autonomous driving ECU8 generates this path in the same way as the navigation function's path search. For example, this path search can be performed using cost calculations based on the Dijkstra method. In the Dijkstra-based cost calculation, the segment costs of road segments that satisfy search conditions such as distance priority and time priority are set relatively low. Furthermore, paths with lower segment costs are recommended. When search conditions are received from the HCU20 (described later), the autonomous driving ECU8 searches for recommended paths that satisfy the search conditions from the HCU20 and returns them to the HCU20.

[0035] In short-term driving plans, the autonomous driving ECU8 uses the generated virtual space surrounding the vehicle to generate a predetermined driving trajectory for driving according to medium- to long-term driving plans. Specifically, it determines the execution of steering maneuvers for lane changes, acceleration and deceleration for speed adjustments, and steering and braking for obstacle avoidance. Furthermore, based on the generated driving plan, the autonomous driving ECU8, in cooperation with the vehicle control ECU6, performs acceleration and deceleration control and / or steering control of the vehicle, thereby achieving autonomous driving.

[0036] Vehicle status sensor 9 is a sensor group used to detect the vehicle's driving status, operating status, and other vehicle statuses. Vehicle status sensor 9 includes a vehicle speed sensor to detect the vehicle's speed, a steering control sensor to detect the steering wheel's steering angle, an accelerator position sensor to detect the accelerator pedal's opening, and a brake pedal's depressurization level and travel sensor. Vehicle status sensor 9 outputs the detection results to the vehicle's LAN. Alternatively, the detection results from vehicle status sensor 9 can be configured to output to the vehicle's LAN via the vehicle's ECU.

[0037] HMI System 2 includes an HCU (Human Machine Interface Control Unit) 20, an indoor camera 21, a biosensor 22, a sound output device 23, a display device 24, a microphone 25, operating devices 26, and a status improvement device 27. HMI System 2 accepts input from the driver. HMI System 2 monitors the status of passengers in the vehicle, including the driver. HMI System 2 provides information to the driver.

[0038] The interior camera 21 captures images of a defined area inside the vehicle's passenger compartment. Alternatively, it can be configured to use a Driver Status Monitor (DSM) that monitors the vehicle's driver as the interior camera 21. It can also be configured to use a camera that captures images of the passenger side and rear seats. Information from the images captured by the interior camera 21 is sequentially output to the HCU 20.

[0039] The biosensor 22 measures the driver's biometric information and outputs the measured biometric information sequentially to the HCU 20. The biosensor 22 can be configured as part of the vehicle. Alternatively, the biosensor 22 can be configured as a wearable device worn by passengers such as the driver. If the biosensor 22 is installed in a wearable device worn by the driver, the HCU 20 can acquire the measurement results from the biosensor 22 via the proximity communication unit 31.

[0040] As a biosensor 22, a pulse wave sensor capable of measuring pulse waves can be included. Pulse wave sensors can be photoelectric pulse wave sensors, impedance pulse wave sensors, etc. Pulse wave sensors can be either contact or non-contact sensors. As a biosensor 22, a body temperature sensor capable of measuring body temperature can be included. Body temperature sensors can be IR sensors, etc. As a biosensor 22, an exhalation sensor capable of measuring exhaled breath can be included. Exhalation sensors can be gas sensors, etc. As a biosensor 22, a body composition sensor capable of measuring body composition can be included. As a body composition sensor, a body composition analyzer is used that flows a microcurrent through the body and estimates body fat percentage, muscle mass, and body water content based on its resistance value. Biosensors 22 that require contact with the passenger's body for measurement can be configured to be installed in the steering wheel, driver's seat, etc., of the vehicle.

[0041] Furthermore, it can also be configured as a biological sensor 22 that uses biological information other than measuring pulse waves, body temperature, respiration, and body composition. For example, sensors that measure respiration, brain waves, heart rate, heart rate fluctuations, sweating, blood pressure, and skin conductance can be listed.

[0042] The sound output device 23 provides information prompts by outputting sound. The sound output device 23 outputs sound according to the instructions of the HCU 20. The sound output device 23 can include a speaker, etc. The display device 24 provides information prompts by displaying images and text. The display device 24 displays images and text according to the instructions of the HCU 20. The display device 24 is configured with its display surface facing the interior of the vehicle's passenger compartment. For example, the display device 24 is configured with its display surface located in front of the driver's seat. The display device 24 can be a CID (Center Information Display), a navigation device display, a head-up display (hereinafter referred to as HUD), etc.

[0043] Microphone 25 collects the sounds emitted by the passengers in the vehicle, converts them into electro-acoustic signals, and outputs them to HCU 20. Microphone 25 can be configured to easily collect the driver's voice, for example, by being mounted on the top of the steering column cover or on the sun visor on the driver's side. Alternatively, microphone 25 can be configured to be installed for each seat, allowing for the differentiation of the voices of passengers in each seat. In this case, a zoom microphone with reduced directional focus can be used as microphone 25 for each seat. Microphone 25 can also accept voice-based command input from the user.

[0044] The operating device 26 receives input from the driver. The operating device 26 can be a mechanical switch or a touch switch integrated with the display device 24. Examples of mechanical switches include turn signals located on the spokes of a steering wheel. The microphone 25 and the operating device 26 can also be referred to as input devices that receive input from the driver.

[0045] The state improvement device 27 is a device that provides the driver with stimulation to improve their mental state from a deterioration. The state improvement device 27 is installed in this vehicle. Examples of a deterioration in mental state include stress, anxiety, worry, and lamentation. Stress can also be referred to as a state of tension. Worry can be further subdivided into unease and fear. Unease can be a mental state of worry without a specific object. Fear can be a mental state of worry with a specific object. In this embodiment, the case of stress as the object of mental state deterioration will be described as an example.

[0046] The state-enhancing device 27 provides the driver with stress-relieving stimulation. For example, the state-enhancing device 27 can be a device that sprays odor components estimated to reduce stress. This device can be implemented by combining an air conditioning unit and an aromatherapy unit. The state-enhancing device 27 can also be a device that alleviates stress by providing the driver with warm stimulation to improve blood circulation and relieve stiffness. In this case, it can be implemented by a heater or blower installed on the back of the driver's seat. To improve the driver's blood circulation, it is preferable to configure it to repeatedly heat or cool the driver's neck and shoulders. The state-enhancing device 27 can also be a device that alleviates stress by guiding the driver's breathing to a breathing pattern estimated to be relaxing. In this case, it can be configured to guide deep breathing by using the tightening of the seat belt. In addition, the state-enhancing device 27 can also be a device that alleviates stress by emitting light estimated to relax the driver. It can be illuminated by LEDs or the like. The state-enhancing device 27 can also be a device that alleviates stress by playing music that the driver likes. In this case, a sound output device 23 can also be used as the state-enhancing device 27.

[0047] As for the state improvement device 27, it can be any device that generates a stimulus that is estimated to alleviate stress, and can be configured in ways other than those described above. Furthermore, even if the target is something other than stress, such as a deterioration in mental state, the state improvement device 27 can be configured to improve the mental state by providing a stimulus that relaxes the driver.

[0048] The HCU20 is primarily composed of a microcomputer equipped with a processor, memory, I / O, and a bus connecting them. It executes various processes, including those related to improving the driver's mental state (hereinafter referred to as state improvement-related processing), by executing control programs stored in memory. This HCU20 is equivalent to a vehicle assistance device. The memory referred to here is a non-transitory tangible storage medium that stores programs and data that can be read by a computer. Furthermore, non-transitory tangible storage media can be implemented using semiconductor memory or disks, etc.

[0049] <Brief Structure of HCU20>

[0050] Next, use Figure 2 This section provides a general overview of the HCU20's structure. The HCU20 addresses state improvement correlation processing, as follows... Figure 2 The HCU20 includes an estimation unit 201, a search instruction unit 202, a recommended path determination unit 203, a prompt control unit 204, a sound recognition unit 205, a selection unit 206, and a stimulus control unit 207 as functional modules. Furthermore, some or all of the functions performed by the HCU20 can be configured in hardware using one or more ICs. Alternatively, some or all of the functional modules of the HCU20 can be implemented through a combination of processor-based software execution and hardware components. Processing the various functional modules of the HCU20 via a computer is equivalent to performing a vehicle assistance method.

[0051] The estimation unit 201 performs estimations related to the state of the passengers in the vehicle. The estimation unit 201 includes a driver state estimation unit 211, an important factor estimation unit 212, and a passenger state estimation unit 213 as sub-functional modules.

[0052] The driver state estimation unit 211 estimates the degree of deterioration in the driver's mental state. In this embodiment, the driver state estimation unit 211 estimates the degree of driver stress. The driver state estimation unit 211 is equivalent to the mental state estimation unit. The processing in this driver state estimation unit 211 is equivalent to the mental state estimation process. The driver state estimation unit 211 can estimate the degree of stress in two stages, namely "present" and "absent," or it can estimate the degree in three or more stages.

[0053] The driver state estimation unit 211 can estimate the driver's stress level based on an image of the driver taken by the peripheral monitoring camera in the peripheral monitoring sensor 7 before the driver gets into the vehicle. For example, the driver's stress level can be estimated based on the driver's posture and gait in the image.

[0054] The driver state estimation unit 211 can also estimate the driver's stress level based on images of the driver captured by the indoor camera 21. For example, the driver's stress level can be estimated based on the driver's facial expression in the image.

[0055] The driver state estimation unit 211 can also estimate the driver's stress level based on the driver's pulse wave measured by the pulse wave sensor in the biosensor 22. In this case, the driver state estimation unit 211 estimates the degree of sympathetic nerve dominance in the autonomic nervous system by analyzing the frequency of the pulse wave, and estimates the degree of sympathetic nerve dominance as the degree of stress.

[0056] The driver state estimation unit 211 can also estimate the driver's stress level based on the driver's exhalation measured by the exhalation sensor in the biosensor 22. In this case, the amount of cortisol secreted as a stress-determining substance is estimated based on the exhalation, and the level of cortisol secretion is estimated as the stress level.

[0057] The driver state estimation unit 211 can also estimate the level of driver stress based on the operating state detected by the vehicle state sensor 9. In this case, the higher the frequency of rapid acceleration, rapid braking, and rapid steering, the higher the estimated level of stress. Here, "rapid" can refer to a change in the amount of time per unit of time that is above a threshold.

[0058] The driver state estimation unit 211 can also estimate the driver's stress level based on the driver's voice collected by the microphone 25. In this case, the driver's voice frequency increases when they are nervous, and the stress level can be estimated based on the driver's voice. Alternatively, the stress level can also be estimated based on the speed of voice production.

[0059] The important factor estimation unit 212 estimates the important factors contributing to the deterioration of the driver's mental state. In this embodiment, the important factor estimation unit 212 estimates the important factors contributing to the driver's stress. The important factor estimation unit 212 is also equivalent to the mental state estimation unit. The processing in this important factor estimation unit 212 is also equivalent to the mental state estimation process. To suppress unnecessary processing, it is preferable that the important factor estimation unit 212 is configured to estimate the important factors of stress only when the level of stress estimated by the driver state estimation unit 211 is above a threshold. The threshold mentioned here can refer to a value that distinguishes the presence or absence of stress.

[0060] The critical factor estimation unit 212 can estimate the critical factors of pressure based on input information such as the vehicle's surrounding environment, the driver's previous and subsequent schedules, the driver's physical condition, and the driver's operating status. It can estimate the critical factors of pressure based on only a portion of the aforementioned input information, or it can estimate them based on a combination of multiple types of input information. As an example, the critical factors of pressure can be estimated by referring to a pre-established correspondence between input information and critical factors of pressure. A lookup table can be provided as an example of such a correspondence. Alternatively, a machine learning device that takes the aforementioned input information as input and outputs critical factors of pressure can be used to estimate the critical factors of pressure based on the input information.

[0061] The surrounding environment of the vehicle can be detected by the surrounding monitoring sensor 7. Examples of the surrounding environment include the number of pedestrians, road type, road width, and traffic congestion level. A large number of pedestrians increases the driving load, potentially becoming a significant factor of stress. Similarly, driving on highways increases the driving load during manual driving, and narrow roads also increase the driving load, potentially becoming a significant factor of stress. For backlighting, glare can be a significant factor of stress. Heavy traffic congestion can also be a significant factor of stress.

[0062] The driver's schedule can be obtained from the schedule application on the mobile terminal carried by some drivers via the near-field communication unit 31. As an example of a schedule, the tasks performed before and after the work can be listed. If the previous task involved desk work, eye strain may be a significant factor contributing to stress. If the driver has just returned home after work, the increased mental fatigue resulting from the end of the workday may also be a significant factor contributing to stress.

[0063] The driver's physical condition can be detected by means of peripheral monitoring sensors 7, indoor cameras 21, and biosensors 22. Examples of physical conditions include physical fatigue and stiffness. Physical fatigue can be a significant factor in stress. Physical fatigue can be estimated based on the driver's posture and gait before boarding, captured by the peripheral monitoring camera in peripheral monitoring sensor 7. Alternatively, physical fatigue can be estimated based on the driver's facial expressions captured by indoor camera 21. Stiffness can also be a significant factor in stress. For example, stiffness can be estimated based on a decrease in temperature in the shoulders, neck, etc., measured by a body temperature sensor.

[0064] The driver's operating status can be detected by the vehicle status sensor 9. Examples of operating status include frequent acceleration, frequent braking, and frequent steering. "Frequent" here can refer to the number of times this occurs per unit of time exceeding a threshold. When the operating status includes frequent acceleration, frequent braking, and frequent steering, the driving load caused by these frequent driving operations can become a significant factor of stress.

[0065] The passenger status estimation unit 213 estimates the status of passengers other than the driver (hereinafter referred to as passengers). The passenger status estimation unit 213 can estimate the passenger's fatigue, sleep, poor physical condition, etc. The passenger status estimation unit 213 can estimate the passenger's fatigue, sleep, and poor physical condition based on images of the passengers captured by the in-seat camera 21. For example, fatigue and poor physical condition can be estimated based on the passenger's facial expression in the image. For example, sleep can be estimated based on the passenger's eye opening degree in the image. The passenger status estimation unit 213 can also estimate the passenger's poor physical condition based on biometric information measured by the biometric sensor 22 installed in the seat. Furthermore, the passenger status estimation unit 213 can also be configured to estimate states other than fatigue, sleep, and poor physical condition. Additionally, the passenger status estimation unit 213 can also estimate the type of passenger. The passenger status estimation unit 213 can estimate the type of passenger based on images of the passengers captured by the in-seat camera 21. As for the types of passengers, it is possible to differentiate and estimate the status of the elderly, children, etc. In addition, the passenger status estimation unit 213 can also make estimations based on gender, etc.

[0066] The search instruction unit 202 sends an instruction to the autonomous driving ECU 8 to search for recommended routes corresponding to the important factors of mental state deterioration estimated by the important factor estimation unit 212. In this embodiment, the search instruction unit 202 determines the search conditions for prioritizing the search of routes estimated to be effective in alleviating stress based on the estimated important factors of stress. Then, the search instruction unit 202 sends an instruction to the autonomous driving ECU 8 to search for recommended routes using these search conditions. The search conditions are determined by referring to a correspondence established in advance between each important factor of stress and the search conditions for prioritizing the search of routes estimated to be effective in alleviating stress. As an example of a correspondence, a lookup table can be provided. The search conditions can also be referred to as conditions for reducing road segment costs.

[0067] When the search instruction unit 202 estimates that a large number of pedestrians in the surrounding area is a significant factor causing pressure, it can determine search criteria that prioritize routes with fewer pedestrians. For example, it can prioritize sections of dedicated motor vehicle lanes. Alternatively, it can prioritize sections of roads away from facilities and residences. When the search instruction unit 202 estimates that high traffic volume on highways is a significant factor causing pressure, it can determine search criteria that prioritize general roads. For example, it can prioritize sections of general roads. When the search instruction unit 202 estimates that narrow road width is a significant factor causing pressure, it can determine search criteria that prioritize routes with wider roads. For example, it can determine search criteria that prioritize wider road sections. When the search instruction unit 202 estimates that traffic congestion is a significant factor causing pressure, it can determine search criteria that prioritize routes with less traffic congestion. For example, it can determine search criteria that prioritize road sections with lower levels of traffic congestion.

[0068] When the search instruction unit 202 estimates that eye fatigue from previous desk work is a significant factor of stress, it prioritizes searching for routes estimated to allow the eyes to rest. For example, it can prioritize routes that pass through forests, coastlines, or other locations estimated to easily alleviate eye fatigue. Additionally, it can prioritize routes estimated to minimize eye strain. Routes with lower speed limits can be listed as examples of routes estimated to minimize eye strain. When mental fatigue after work is a significant factor of stress, the search instruction unit 202 prioritizes searching for routes that pass through places or scenery estimated to provide relaxation. For example, it can prioritize routes that pass through forests, coastlines, or other locations estimated to provide relaxation. Additionally, it can prioritize routes that pass through shops favored by the driver. The driver's preferred shops can be estimated based on search history obtained from the driver's mobile terminal via the short-range communication unit 31.

[0069] If physical fatigue is considered a significant factor of stress, the search instruction unit 202 can prioritize searching for routes that are estimated to alleviate physical fatigue. For example, it can prioritize search conditions that allow for more advanced autonomous driving at Level 3 or higher. Furthermore, it can prioritize search conditions that pass through service areas. If stiffness is considered a significant factor of stress, the search instruction unit 202 can prioritize search conditions that pass through public bathhouses.

[0070] Alternatively, the search instruction unit 202 can also use a machine learning machine that takes stress-related factors as input and outputs search conditions prioritizing paths estimated to be effective in alleviating stress, to determine search conditions based on the estimated stress-related factors. In this machine learning, it is preferable to learn sequentially the phenomenon when the driver's stress level, estimated by the driver state estimation unit 211, actually decreases. For example, if stress is alleviated when approaching a specific shop, the machine can learn to output search conditions prioritizing paths through that shop and shops of a similar type. Similarly, if stress is alleviated when passing through a place with a specific view, the machine can learn to output search conditions prioritizing paths through that place and places with similar views. The machine learning can also further refine the search conditions by adding other environmental factors such as time period and traffic congestion level to the input. In this case, the search instruction unit 202, in addition to the estimated important factors of pressure, also takes factors such as time period and environment as inputs and determines the search conditions through the learner.

[0071] The preferred search instruction unit 202 sends an instruction to the autonomous driving ECU 8 to search for a recommended path corresponding to the passenger's state estimated by the passenger state estimation unit 213. As an example, the search instruction unit 202 determines the search conditions based on the estimated passenger state. Then, the search instruction unit 202 sends an instruction to the autonomous driving ECU 8 to search for a recommended path using those search conditions. The search conditions are determined by referring to a correspondence established in advance between each passenger state and the search conditions. As an example of the correspondence, a lookup table can be provided. For example, if the search instruction unit 202 estimates that the passenger's physical condition is poor, it can determine a search condition that prioritizes searching for paths with fewer curves. For example, it can determine a search condition that prioritizes straight road segments. Accordingly, a recommended path that prevents the passenger's condition from worsening can be proposed.

[0072] More preferably, the search instruction unit 202 sends an instruction to the autonomous driving ECU 8 indicating the recommended route corresponding to the type and state of the passenger estimated by the passenger state estimation unit 213. In this case, the search conditions can be determined by referring to a correspondence established in advance between each combination of passenger type and state and the search conditions. As an example of the correspondence, a lookup table can be provided. For example, if the search instruction unit 202 estimates that the passenger type is elderly and the passenger state is fatigued, it can decide to prioritize searching for routes with fresh air where the vehicle windows can be opened. For example, it can decide to prioritize searching for routes through forests, parks, coastlines, etc. If map data is provided with information on areas with fresh air, it can also decide to prioritize searching for routes in areas with fresh air using that information. In addition, if the search instruction unit 202 estimates that the passenger type is a child and the passenger state is sleeping, it can decide to prioritize searching for routes that are longer. For example, it can decide to prioritize searching for routes with longer sections. Based on this, it is possible to propose the desired path corresponding to the combination of the types and states of the fellow passengers.

[0073] If there are no passengers in the vehicle and the passenger status estimation unit 213 cannot estimate the type and status of the passengers, the process of instructing the search for a recommended route corresponding to the passenger status is not performed. Preferably, the search instruction unit 202 is configured such that if the driver's stress level estimated by the driver status estimation unit 211 is above a predetermined threshold, the search instruction unit 202 does not instruct the search for a recommended route corresponding to the passenger status, but instead instructs the search for a recommended route corresponding to a significant factor of the driver's stress. On the other hand, preferably, the search instruction unit 202 is configured such that if the driver's stress level estimated by the driver status estimation unit 211 is below a predetermined threshold, the search instruction unit 202 does not instruct the search for a recommended route corresponding to a significant factor of the driver's stress, but instead instructs the search for a recommended route corresponding to the passenger status. The predetermined threshold can be a value that distinguishes between the presence and absence of stress. Accordingly, when the driver is under stress, a route estimated to alleviate that stress can be proposed to suppress its impact on driving, and when the driver is not under stress and the impact on driving is minimal, a desired route corresponding to the passenger status can be proposed for comfortable driving for the passenger.

[0074] Furthermore, the system can be configured to search for a recommended path in the autonomous driving ECU 8 that satisfies both the instruction to search for a recommended path corresponding to important factors of driver stress and the instruction to search for a recommended path corresponding to the state of the passenger. In this case, the search instruction unit 202 can also, if the driver stress level estimated by the driver state estimation unit 211 is above a predetermined threshold, give a higher priority to the search condition that prioritizes searching for a recommended path corresponding to important factors of driver stress (hereinafter referred to as the driver consideration search condition) than to the search condition that prioritizes searching for a recommended path corresponding to the state of the passenger (hereinafter referred to as the passenger consideration search condition). Specifically, it can give an instruction to increase the reduction in road segment costs compared to the passenger consideration search condition. Additionally, the search instruction unit 202 can also, if the driver stress level estimated by the driver state estimation unit 211 is below a predetermined threshold, give a higher priority to the passenger consideration search condition than to the driver consideration search condition. Specifically, it can give an instruction to increase the reduction in road segment costs compared to the driver consideration search condition.

[0075] In the autonomous driving ECU 8, a recommended path that satisfies the search conditions sent from the search instruction unit 202 is searched and returned to the HCU 20. In other words, the autonomous driving ECU 8 searches for recommended paths corresponding to the important factors of mental state deterioration estimated by the important factor estimation unit 212. In addition, the autonomous driving ECU 8 also searches for recommended paths corresponding to at least one of the passenger's type and state estimated by the passenger state estimation unit 213. In this case, the autonomous driving ECU 8 can prioritize time and distance when searching for recommended paths that satisfy the search conditions sent from the search instruction unit 202. The autonomous driving ECU 8 can search for recommended paths starting from the current vehicle position measured by the locator 4. The autonomous driving ECU 8 can search for recommended paths with the location set by the input of the operating device 26, etc., as the destination. In the autonomous driving ECU 8, if an instruction is received to increase the reduction of the road segment cost of one of the search conditions for driver consideration and passenger consideration, the road segment cost of that search condition can be set to be smaller. The autonomous driving ECU 8 can search for multiple paths with smaller road segment cost values ​​as recommended paths. Then, the recommended path is returned to HCU20. The recommended path searched by the autonomous driving ECU8 is a path that includes multiple road segments.

[0076] The recommended route determination unit 203 determines the recommended route as the path searched by the autonomous driving ECU 8 according to the instructions of the search instruction unit 202. In other words, the recommended route determination unit 203 determines the recommended route corresponding to the important factors of deterioration of mental state estimated by the important factor estimation unit 212. In addition, the recommended route determination unit 203 also determines the recommended route based on the type and at least one state of the passenger estimated by the passenger state estimation unit 213. The processing in this recommended route determination unit 203 is equivalent to the recommended route determination process.

[0077] In the search instruction unit 202, if the driver's stress level estimated by the driver state estimation unit 211 is above a predetermined threshold, the instruction will not be given to search for a recommended path corresponding to the passenger's state, but rather to search for a recommended path corresponding to important factors of the driver's stress. The recommended path determination unit 203, when the driver's stress level estimated by the driver state estimation unit 211 is above the predetermined threshold, will prioritize a recommended path corresponding to important factors of stress estimated by the important factor estimation unit 212, compared to the passenger's state estimated by the passenger state estimation unit 213. Conversely, if the driver's stress level estimated by the driver state estimation unit 211 is below the predetermined threshold, the recommended path determination unit 203 will prioritize a recommended path corresponding to the passenger's state estimated by the passenger state estimation unit 213, compared to the important factors of stress estimated by the important factor estimation unit 212.

[0078] The prompt control unit 204 provides the driver with a prompt regarding the recommended route determined by the recommended route determination unit 203. The processing within this prompt control unit 204 corresponds to a prompt control procedure. The prompt control unit 204 can prompt the driver with a recommended route by displaying the recommended route on the display device 24. As an example, the recommended route can be displayed on a map in a distinguishable manner. If multiple recommended routes are determined, all recommended routes can be displayed. Alternatively, the prompt control unit 204 can also prompt the driver with text displayed on the display device 24 for each recommended route, showing the transit points up to the destination of that recommended route.

[0079] When providing route guidance, the prompt control unit 204 may also prompt for routes within the route guidance in addition to the recommended route. In this case, it is preferable to prompt for routes within the route guidance in a manner that can be distinguished from them. Here, "paths within the route guidance" refers to a recommended route searched without considering the degree of deterioration in the driver's mental state or its importance. This route will be referred to hereafter as a "state-unconsidered route." Preferably, when prompting for a recommended route, the prompt control unit 204 also inquires whether the guidance for the recommended route is necessary.

[0080] In addition, the prompt control unit 204 can also provide path guidance along the recommended path determined by the recommended path determination unit 203. Path guidance can be provided by displaying instructions for driving along the recommended path on the display device 24 or by outputting sound from the sound output device 23.

[0081] The voice recognition unit 205 performs voice recognition on the sound collected by the microphone 25 to identify the content of the passenger's voice. The selection unit 206 selects whether or not the guidance of the recommended route determined by the recommended route determination unit 203 is needed based on the input received from the driver through the microphone 25 or the operating device 26. If the selection unit 206 receives input indicating that the guidance of the recommended route is needed through the microphone 25 or the operating device 26, it selects the route that is needed. If multiple recommended routes are provided, it receives input indicating that the guidance of one of the recommended routes is needed and selects the route that is needed. On the other hand, if the selection unit 206 receives input indicating that the guidance of the recommended route is not needed through the microphone 25 or the operating device 26, it selects the route that is not needed. The selection unit 206 may also select the route that is not needed if no input indicating that the guidance of the recommended route is needed is received within a constant time from the time the recommended route is provided or the time the question of whether the guidance of the recommended route is needed is raised. The selection unit 206 can determine whether input related to the need for guidance of the recommended path has been received through the microphone 25 based on the result of voice recognition in the voice recognition unit 205.

[0082] The stimulation control unit 207 controls the operation of the state improvement device 27. Preferably, the stimulation control unit 207 automatically starts the operation of the state improvement device 27 when the guide that does not require a recommended path is selected by the selection unit 206. On the other hand, the stimulation control unit 207 may be configured not to automatically start the operation of the state improvement device 27 when the guide that requires a recommended path is selected by the selection unit 206. The stimulation control unit 207 may also start the operation of the state improvement device 27 even when the guide that requires a recommended path is selected by the selection unit 206, if it receives input through the microphone 25 or the operating device 26 instructing the operation of the state improvement device 27.

[0083] If the selection unit 206 selects a path that does not require a recommended path, the prompt control unit 204 will not guide the user to the recommended path. If path guidance is being implemented without considering the path, the path guidance in that state will continue. If path guidance is not being implemented, the state of not implementing path guidance will continue. On the other hand, if the selection unit 204 selects a path that requires a recommended path, the prompt control unit 204 will guide the user to that recommended path.

[0084] <State Improvement Association Processing in HCU20>

[0085] Here, using Figure 3 The flowchart illustrates an example of the state improvement correlation processing flow in HCU20. It is configured to begin, for example, when the switch (hereinafter referred to as the power switch) used to start the vehicle's internal combustion engine or electric generator is turned on. Figure 3 The flowchart is sufficient. In addition, in the case where the driver's state is estimated based on images captured by surrounding surveillance cameras, the system is configured to begin when the driver approaches the parked vehicle. Figure 3 The flowchart is sufficient. The HCU20 can estimate the driver's approach to the vehicle based on factors such as the strength of the radio waves exchanged between the electronic key carried by the driver and the vehicle side.

[0086] First, in step S1, the estimation unit 201 performs an estimation related to the state of the passengers in the vehicle. In S1, the driver state estimation unit 211 estimates the degree of stress on the driver. In S1, the passenger state estimation unit 213 estimates the state of the passenger.

[0087] In step S2, if the driver's stress level estimated in S1 is above a predetermined threshold (S2: Yes), proceed to step S3. On the other hand, if the driver's stress level estimated in S1 is below the predetermined threshold (S2: No), proceed to step S5.

[0088] In step S3, the importance factor estimation unit 212 estimates the importance factors of the driver's stress. In step S4, the search instruction unit 202 decides to prioritize searching for search conditions that correspond to the importance factors of stress estimated in S3 and are estimated to be effective in alleviating stress, and moves to step S7.

[0089] In step S5, if there is a passenger in the vehicle (S5: Yes), proceed to step S6. Conversely, if there is no passenger in the vehicle (S5: No), proceed to step S14. The passenger status estimation unit 213 can determine whether there is a passenger in the vehicle based on whether the type and status of the passenger can be estimated. In step S6, the search instruction unit 202 determines the search conditions corresponding to at least one of the passenger types and statuses estimated in S1, and proceeds to step S7.

[0090] In step S7, the search instruction unit 202 sends an instruction to the autonomous driving ECU 8 to search for a recommended path based on the search conditions determined in S4 or S6. In step S8, the recommended path determination unit 203 obtains the recommended path searched by the autonomous driving ECU 8 based on the search conditions sent in S7. Then, the recommended path determination unit 203 determines the obtained recommended path as the recommended path. In step S9, the prompt control unit 204 prompts the driver with the recommended path determined in S8.

[0091] In step S10, if the selection unit 206 selects a path that requires guidance (S10: Yes), proceed to step S11. Conversely, if the selection unit 206 selects a path that does not require guidance (S10: No), proceed to step S12. In step S11, the prompting control unit 204 guides the user along the path that the selection unit 206 selected, and proceed to step S13. In S11, the stimulus control unit 207 prevents the automatic activation of the state improvement device 27. Conversely, in step S12, the stimulus control unit 207 automatically activates the state improvement device 27, and proceed to step S13. In S12, the prompting control unit 204 prevents the guidance along the recommended path determined in S8.

[0092] In step S13, if a constant time has elapsed since the recommended route was suggested in S9 (S13: Yes), proceed to step S14. Conversely, if no constant time has elapsed since the recommended route was suggested in S9 (S13: No), the process in S13 is repeated. The constant time mentioned here can be arbitrarily set. As an example, the frequency of recommended route suggestions resulting from the repetition of the process can be set to a frequency estimated to be one that will not bore the driver. Alternatively, the travel distance can be used instead of time.

[0093] In step S14, if the end time of the state improvement association process is set (S14: Yes), the state improvement association process ends. Conversely, if the end time of the state improvement association process is not set (S14: No), the process returns to S1 and repeats. An example of the end time of the state improvement association process could be the power switch being turned off.

[0094] <Summary of Implementation Method 1>

[0095] According to the configuration of Embodiment 1, it is possible to suggest to the driver a path to alleviate stress that corresponds to more detailed factors of stress than simply the presence or absence of stress. Specifically, as follows: Even when a driver is feeling stressed, there are cases where the effective path to alleviate stress differs depending on the factors of stress. For example, for stress where eye fatigue from desk work is a significant factor, while driving on a recommended route with less traffic congestion can alleviate it, driving on a recommended route that is estimated to allow for easier eye rest is more likely to alleviate it. Thus, by suggesting recommended routes that are estimated to alleviate stress based on the factors of stress and by guiding drivers to these recommended routes, stress can be alleviated with greater precision. As a result, when a driver's mental state deteriorates, their mental state can be improved with greater precision.

[0096] Furthermore, according to the configuration of Embodiment 1, even if the driver does not wish to be guided along a recommended path corresponding to a significant factor of stress, stress can be alleviated by automatically starting the operation of the state improvement device 27.

[0097] In Implementation 1, the case of deterioration of mental state due to stress was described, but for deterioration of mental state other than stress, it is also possible to improve the mental state more accurately by suggesting recommended paths corresponding to the important factors of deterioration of mental state.

[0098] (Implementation Method 2)

[0099] In Embodiment 1, a recommended path corresponding to the deterioration of mental state is shown, but it is not necessarily limited to this. For example, it may be configured to provide a recommended path corresponding to three or more stages of deterioration of mental state (hereinafter, Embodiment 2). Hereinafter, an example of Embodiment 2 will be described using figures. The driving assistance system 1 of Embodiment 2 is the same as the driving assistance system 1 of Embodiment 1, except that it includes HCU20a instead of HCU20.

[0100] <Brief Structure of HCU20a>

[0101] Here, using Figure 4 A brief description of the HCU20a structure is provided. HCU20a relates to state improvement and related processing, such as... Figure 4 The HCU20a shown includes an estimation unit 201a, a search instruction unit 202a, a recommended path determination unit 203, a prompt control unit 204, a voice recognition unit 205, a selection unit 206, and a stimulus control unit 207 as functional modules. The HCU20a is the same as the HCU20 in Embodiment 1, except that it includes an estimation unit 201a and a search instruction unit 202a instead of the estimation unit 201a and the search instruction unit 202a. This HCU20a also corresponds to a vehicle assistance device. Furthermore, executing the processing of each functional module of the HCU20a via a computer is equivalent to executing a vehicle assistance method.

[0102] The estimation unit 201a includes a driver state estimation unit 211a and a passenger state estimation unit 213 as sub-functional modules. The estimation unit 201a is the same as the estimation unit 201 in Embodiment 1, except that it has a driver state estimation unit 211a instead of the driver state estimation unit 211 and does not have an important factor estimation unit 212.

[0103] The driver state estimation unit 211a estimates the degree of deterioration of the driver's mental state to three or more stages. The driver state estimation unit 211a is the same as the driver state estimation unit 211 in Embodiment 1, except that it is limited to estimating the degree of deterioration of mental state to three or more stages. In this embodiment, the driver state estimation unit 211a estimating the driver's stress level will be described as an example. The driver state estimation unit 211a is also equivalent to the mental state estimation unit. The processing in this driver state estimation unit 211a is also equivalent to the mental state estimation process.

[0104] The search instruction unit 202a sends an instruction to the autonomous driving ECU 8 to search for a recommended path corresponding to three or more stages of deterioration in mental state estimated by the driver state estimation unit 211a. In this embodiment, the search instruction unit 202a determines search conditions (hereinafter referred to as mitigation search conditions) based on the estimated stress level of three or more stages, prioritizing the search for paths estimated to be effective in mitigating stress. Then, the search instruction unit 202a sends an instruction to the autonomous driving ECU 8 to search for a recommended path using these search conditions.

[0105] The search instruction unit 202a can determine more types of mitigation search conditions based on the degree of deterioration of the driver's mental state estimated by the driver state estimation unit 211a. Examples of mitigation countermeasures include the following.

[0106] As one type of mitigation search condition, it is possible to list search criteria that prioritize routes with less traffic congestion. For example, a search criterion that prioritizes road segments with lower traffic congestion levels is possible. This is because less traffic congestion is considered more effective in easing stress. Another type of mitigation search condition is to prioritize routes with fewer pedestrians. This search criterion may prioritize segments of dedicated car lanes. This is because fewer pedestrians mean less attention to pedestrians flying out, and is therefore considered more effective in easing stress. Finally, another type of mitigation search condition is to prioritize routes with lower speed limits. This search criterion may prioritize segments of roads with lower speed limits. This is because driving at lower speeds allows for more leisurely travel, and is therefore considered more effective in easing stress.

[0107] As one type of stress-relieving search condition, search conditions that prioritize searching for routes via service areas can be listed. This search condition could be, for example, prioritizing road segments near service areas. This is because it is estimated that resting at service areas can alleviate stress. As another type of stress-relieving search condition, search conditions that prioritize searching for routes that allow for continued Level 3 or higher autonomous driving can be listed. This search condition could be, prioritizing road segments that allow for continued Level 3 or higher autonomous driving. This is because it is estimated that stress can be relieved by freeing oneself from driving. As another type of stress-relieving search condition, search conditions that prioritize searching for routes passing through places or scenery estimated to offer relaxation can be listed. This search condition could be, for example, prioritizing road segments that pass through forests, coastlines, or other places estimated to offer relaxation. As another type of stress-relieving search condition, search conditions that prioritize searching for routes passing through shops favored by the driver can be listed. This search condition could be, for example, prioritizing road segments that pass through shops favored by the driver. The driver's preferred shops can be estimated based on search history obtained via the driver's mobile terminal acquired by the short-range communication unit 31.

[0108] The search instruction unit 202a determines more types of mitigation search conditions based on the estimated increase in driver stress level, and sends an instruction to the autonomous driving ECU 8 to search a recommended path using the determined mitigation search conditions. For example, it can add one more type of search condition to the combination of mitigation search conditions as the stress level increases by one stage. The combination of mitigation search conditions corresponding to the stress level can be a pre-set fixed combination or a combination learned through machine learning. For learning the combination of mitigation search conditions, for example, it can randomly change the combination of mitigation search conditions according to the stress level, while learning the combination that has a higher mitigation effect on the stress level estimated by the driver state estimation unit 211a.

[0109] Alternatively, the search instruction unit 202a may use a learner that has undergone machine learning, taking the degree of pressure as input and outputting search conditions that prioritize searching paths estimated to be effective in alleviating pressure, to determine the combination of search conditions based on the estimated degree of pressure. In this machine learning, as described in Embodiment 1, it is preferable to sequentially learn the phenomenon when the degree of driver pressure estimated by the driver state estimation unit 211a actually decreases.

[0110] The search instruction unit 202a is the same as described in Embodiment 1, and preferably sends an instruction to the autonomous driving ECU 8 to search for a recommended path corresponding to the passenger's state estimated by the passenger state estimation unit 213. In other words, the search instruction unit 202a is preferably configured, as in Embodiment 1, to not instruct the search for a recommended path corresponding to the passenger's state if the driver's stress level estimated by the driver state estimation unit 211a is above a predetermined threshold, but instead instruct the search for a recommended path corresponding to important factors of the driver's stress. On the other hand, the search instruction unit 202a is preferably configured to not instruct the search for a recommended path corresponding to important factors of the driver's stress if the driver's stress level estimated by the driver state estimation unit 211a is below a predetermined threshold, but instead instruct the search for a recommended path corresponding to the passenger's state. Furthermore, as in Embodiment 1, it can also be configured to allow the autonomous driving ECU 8 to search for a recommended path that satisfies both the instruction to search for a recommended path corresponding to important factors of the driver's stress and the instruction to search for a recommended path corresponding to the passenger's state.

[0111] In the autonomous driving ECU 8, as described in Embodiment 1, a recommended path that satisfies the search conditions sent from the search instruction unit 202a is searched and returned to the HCU 20. In other words, the autonomous driving ECU 8 searches for a recommended path corresponding to the degree of deterioration of the driver's mental state estimated by the driver state estimation unit 211a. Additionally, the autonomous driving ECU 8 also searches for a recommended path corresponding to at least one of the passenger's type and state estimated by the passenger state estimation unit 213. In this case, the autonomous driving ECU 8 prioritizes time and distance when searching for a recommended path that satisfies the search conditions sent from the search instruction unit 202a.

[0112] The recommended path determination unit 203 determines the recommended path searched by the autonomous driving ECU 8 according to the instructions of the search instruction unit 202a as the recommended path. In other words, the recommended path determination unit 203 determines the recommended path corresponding to three or more stages of deterioration of the driver's mental state as estimated by the driver state estimation unit 211a. As the degree of deterioration of the driver's mental state estimated by the driver state estimation unit 211a increases, the recommended path determination unit 203 determines the path that satisfies more types of mitigation search conditions as the recommended path.

[0113] <State Improvement Association Processing in HCU20a>

[0114] Here, using Figure 5 The flowchart illustrates an example of the state improvement association processing flow in HCU20a. Figure 5 The flowchart is also configured to be similar to Figure 3 You can start the flowchart in the same way.

[0115] First, in step S21, the estimation unit 201a performs an estimation related to the state of the passengers in the vehicle. In S21, the driver state estimation unit 211a estimates the degree of stress experienced by the driver at three or more stages. In S21, the passenger state estimation unit 213 estimates the state of the passenger.

[0116] In step S22, similar to S2, if the driver's stress level estimated in S21 is above a predetermined threshold (S22: Yes), proceed to step S23. On the other hand, if the driver's stress level estimated in S21 is below the predetermined threshold (S22: No), proceed to step S24.

[0117] In step S23, the search instruction unit 202a determines search conditions that correspond to the level of pressure estimated in S21, prioritizing the search for paths estimated to be effective in mitigating pressure, and moves to step S26. In S23, the number of determined search conditions increases as the level of pressure increases.

[0118] In step S24, similar to S5, if there is a passenger in the vehicle (S24: Yes), proceed to step S25. On the other hand, if there is no passenger in the vehicle (S24: No), proceed to step S33. In step S25, the search instruction unit 202a determines the search conditions corresponding to at least one of the types and states of the passenger estimated in S21, and proceeds to step S26.

[0119] In step S26, the search instruction unit 202a sends an instruction to the autonomous driving ECU 8 to search for a recommended path based on the search conditions determined in S23 or S25. Steps S27 to S32 are performed in the same manner as steps S8 to S13. In step S33, if the end time of the state improvement association process is set (S33: Yes), the state improvement association process ends. Otherwise, if the end time of the state improvement association process is not set (S33: No), the process returns to S21 and repeats.

[0120] <Summary of Implementation Method 2>

[0121] According to the configuration of Embodiment 2, the driver can be prompted with a path to alleviate pressure that corresponds to three or more levels of pressure, more detailed than simply the presence or absence of pressure. Specifically, as follows: Even when the driver experiences pressure, there are cases where the effective path for relieving pressure varies depending on the level of pressure. In contrast, according to the configuration of Embodiment 2, a recommended path estimated to alleviate the pressure is provided based on three or more levels of pressure. Therefore, pressure can be alleviated with greater precision. As a result, when the driver's mental state deteriorates, their mental state can be improved with greater precision.

[0122] Furthermore, according to the configuration of Implementation Method 2, as the driver's stress level increases, a recommended path is determined and proposed that satisfies more types of stress-relief search conditions. Accordingly, the possibility of more accurately alleviating stress increases when the driver's stress is higher and more important factors contribute to the stress.

[0123] Furthermore, in the configuration of Embodiment 2, even if the driver does not wish to be guided along a recommended path that corresponds to a significant factor of stress, stress can be alleviated by automatically activating the state improvement device 27.

[0124] In Implementation 2, the case of mental state deterioration due to stress is also described. However, for mental state deterioration other than stress, it is also configured to improve the mental state more accurately by providing a recommended path corresponding to three or more stages of mental state deterioration.

[0125] (Implementation Method 3)

[0126] Alternatively, it can be configured as in Embodiment 3 below. Hereinafter, an example of Embodiment 3 will be described using figures. The driving assistance system 1 of Embodiment 3 is the same as the driving assistance system 1 of Embodiment 1, except that it includes HCU20b instead of HCU20.

[0127] <Brief Structure of HCU20b>

[0128] Here, using Figure 6 A brief description of the HCU20b structure is provided. The HCU20b relates to state improvement and related processing, such as... Figure 6 The HCU20b shown includes an estimation unit 201b, a search instruction unit 202b, a recommended path determination unit 203, a prompt control unit 204, a voice recognition unit 205, a selection unit 206, and a stimulus control unit 207 as functional modules. The HCU20b is identical to the HCU20 of Embodiment 1, except that it includes an estimation unit 201b and a search instruction unit 202b instead of the estimation unit 201 and the search instruction unit 202. This HCU20b also corresponds to a vehicle assistance device. Furthermore, executing the processing of each functional module of the HCU20b via a computer is equivalent to executing a vehicle assistance method.

[0129] The estimation unit 201b includes a driver state estimation unit 211, an important factor estimation unit 212, and a passenger state estimation unit 213b as sub-functional modules. The estimation unit 201b is identical to the estimation unit 201 in Embodiment 1, except that it includes a passenger state estimation unit 213b instead of the passenger state estimation unit 213. In this embodiment, the example of the driver state estimation unit 211 estimating the driver's stress level will be used for explanation.

[0130] The passenger status estimation unit 213b estimates the type and status of the passenger. Except for the limitation of estimating the type and status of the passenger, the passenger status estimation unit 213b is the same as the passenger status estimation unit 213 in Embodiment 1.

[0131] The search instruction unit 202b can also send instructions to the autonomous driving ECU 8 to search for a recommended route corresponding to the passenger status estimated by the passenger status estimation unit 213. The search instruction unit 202b is the same as the search instruction unit 202 in Embodiment 1, except that it prioritizes which passenger to determine the search conditions.

[0132] The search instruction unit 202b switches between the passenger type estimated by the passenger status estimation unit 213b and the recommended path search criteria that prioritize either the driver's stress factor estimated by the importance factor estimation unit 212 or the passenger status estimated by the passenger status estimation unit 213b. Furthermore, the search criteria can be configured to pre-set the passenger status as the priority for each passenger type. For example, fatigue could be prioritized for elderly passengers, and sleep could be prioritized for children.

[0133] As an example, the search instruction unit 202b can be configured to prioritize children > elderly > driver. For example, if the driver's stress level is estimated above a threshold by the driver state estimation unit 211, the elderly person's fatigue is estimated by the passenger state estimation unit 213b, and the child's sleep is estimated by the passenger state estimation unit 213b, then the search condition for prioritizing the search of recommended routes corresponding to the child's sleep can be determined. For example, the search condition for prioritizing the search of detour routes can be determined. Furthermore, if the driver's stress level is estimated above a threshold by the driver state estimation unit 211, the elderly person's fatigue is estimated by the passenger state estimation unit 213b, and the child's sleep is not estimated by the passenger state estimation unit 213b, then the search condition for prioritizing the search of recommended routes corresponding to the elderly person's fatigue can be determined. For example, the search condition for prioritizing the search of routes with fresh air where the vehicle windows can be opened can be determined. If the driver's stress level is estimated above the threshold by the driver state estimation unit 211, the elderly person's fatigue is not estimated by the passenger state estimation unit 213b, and the child's sleep is not estimated by the passenger state estimation unit 213b, then the search criteria for prioritizing the search of paths estimated to be effective in relieving driver stress can be determined.

[0134] The recommended route determination unit 203 determines the recommended route as the recommended route searched by the autonomous driving ECU 8 according to the instructions of the search instruction unit 202b. In other words, based on the type of passenger estimated by the passenger state estimation unit 213b, it switches the priority between the important factors of driver stress estimated by the important factor estimation unit 212 and the passenger state estimated by the passenger state estimation unit 213b to determine which of the following recommended routes corresponds to.

[0135] Based on the above configuration, recommended routes corresponding to the states of passengers who should be prioritized can be proposed first. Furthermore, the configuration of Implementation 3 can also be combined with the configuration of Implementation 2 instead of combining with the configuration of Implementation 1.

[0136] (Implementation Method 4)

[0137] In embodiment 3, a configuration is shown that switches the recommended route based on the type of passenger, determining which passenger's route is prioritized; however, this is not a limitation. For example, the configuration could also switch the recommended route based on input received from the driver via microphone 25 or operating device 26. Accordingly, the driver can select which passenger's recommended route is prioritized.

[0138] (Implementation Method 5)

[0139] In the above embodiment, a configuration is shown that proposes a recommended path corresponding to the state of the passenger, but it is not necessarily limited to this. For example, it may be configured not to propose a recommended path corresponding to the state of the passenger. In this case, the configuration may be such that the estimation units 201, 201a, and 201b do not have passenger state estimation units 213 and 213b.

[0140] (Implementation Method 6)

[0141] In the above embodiment, the search instruction units 202, 202a, and 202b are shown to instruct the autonomous driving ECU 8 to search for recommended paths, but this is not necessarily limited to this configuration. For example, the HCUs 20, 20a, and 20b may also be configured to have a function module for searching recommended paths, and the recommended paths may be searched on the HCUs 20, 20a, and 20b side. In this case, the recommended path determination unit 203 is configured to determine the recommended path by searching for recommended paths. The search instruction units 202, 202a, and 202b send the search conditions to the recommended path determination unit 203 to search for recommended paths. Alternatively, the HCUs 20, 20a, and 20b may not have search instruction units 202, 202a, and 202b, but the recommended path determination unit 203 may have both the functions of the search instruction units 202, 202a, and 202b and the function of searching recommended paths.

[0142] (Implementation Method 7)

[0143] In the above embodiment, a configuration is shown in which the operation of the state improvement device 27 is automatically initiated when the selection unit 206 selects a guide that does not require a recommended path, but this is not necessarily the case. For example, it may be configured so that the operation of the state improvement device 27 is not automatically initiated even when the selection unit 206 selects a guide that does not require a recommended path. Alternatively, it may be configured so that the state improvement device 27 is not installed in the vehicle and the stimulation control unit 207 is not provided in the HCU 20, 20a, and 20b.

[0144] Furthermore, this disclosure is not limited to the embodiments described above, and various modifications can be made within the scope of the claims. Embodiments obtained by appropriately combining the disclosed technical units of different embodiments are also included within the technical scope of this disclosure. Additionally, the control unit and method described in this disclosure can be implemented using a dedicated computer configured to execute one or more functions embodied in a computer program. Alternatively, the apparatus and method described in this disclosure can be implemented using dedicated hardware logic circuits. Alternatively, the apparatus and method described in this disclosure can be implemented using one or more dedicated computers composed of a processor executing a computer program and one or more hardware logic circuits. Furthermore, the computer program can also be stored as instructions executable by a computer on a non-transferable tangible recording medium readable by a computer.

Claims

1. An auxiliary device for a vehicle, wherein, have: The mental state estimation unit estimates the mental state, which is at least one of the degree of deterioration of the driver's mental state and important factors of the deterioration of the driver's mental state. The recommended route determination unit determines a recommended route, which includes multiple road segments, corresponding to the mental association state estimated by the aforementioned mental association state estimation unit. The prompt control unit prompts the driver to follow the recommended route determined by the recommended route determination unit; The selection unit selects whether or not the driver needs the recommended route determined by the recommended route determination unit, based on the input received from the driver via the input device. as well as The stimulus control unit controls the operation of a state improvement device installed in the vehicle that provides the driver with stimuli to improve their mental state from deterioration. The aforementioned prompt control unit can also guide users along the recommended path. If the selection unit selects that the recommended path is not needed, the prompt control unit does not guide the recommended path, and the stimulus control unit automatically starts the operation of the state improvement device. On the other hand, if the selection unit selects that the recommended path is needed, the stimulus control unit does not automatically start the operation of the state improvement device, and the prompt control unit guides the recommended path.

2. The vehicle auxiliary device according to claim 1, wherein, The aforementioned mental state estimation unit at least estimates the significant factors contributing to the deterioration of the driver's mental state as the aforementioned mental state. The aforementioned recommended path determination unit determines the aforementioned recommended path that corresponds to the important factors of the deterioration of the driver's mental state estimated by the aforementioned mental state estimation unit, and is estimated to improve the driver's mental state.

3. The vehicle auxiliary device according to claim 1, wherein, The aforementioned mental state estimation unit at least estimates the degree of deterioration in the driver's mental state as the aforementioned mental state. The aforementioned recommended path determination unit determines the aforementioned recommended path that corresponds to three or more stages of the deterioration of the driver's mental state as estimated by the aforementioned mental state estimation unit, and is estimated to improve the driver's mental state.

4. The vehicle auxiliary device according to claim 3, wherein, The search criteria estimated to be pathways to improve the aforementioned mental state fall into several categories. The aforementioned recommended path determination unit determines the path that satisfies more types of the aforementioned search conditions as the recommended path based on the increased degree of deterioration in the driver's mental state as estimated by the aforementioned mental state estimation unit.

5. The vehicle auxiliary device according to claim 1, wherein, The aforementioned deterioration of mental state is due to stress.

6. The vehicle auxiliary device according to any one of claims 1 to 5, wherein, It has a passenger status estimation unit that estimates the status of passengers other than the driver of the aforementioned vehicle, i.e., fellow passengers. The aforementioned recommended route determination unit also makes the recommended route determination based on the state of the passenger estimated by the aforementioned passenger state estimation unit.

7. The vehicle auxiliary device according to claim 6, wherein, The aforementioned mental state estimation unit at least estimates the degree of deterioration in the driver's mental state as the aforementioned mental state. If the degree of deterioration of the driver's mental state, as estimated by the aforementioned mental state estimation unit, is above a predetermined threshold, the recommended route determination unit decides to prioritize the recommended route corresponding to the mental state estimated by the aforementioned mental state estimation unit, compared to the state of the passenger estimated by the aforementioned passenger state estimation unit. On the other hand, if the degree of deterioration of the driver's mental state, as estimated by the aforementioned mental state estimation unit, is below a predetermined threshold, the recommended route determination unit decides to prioritize the recommended route corresponding to the state of the passenger estimated by the aforementioned passenger state estimation unit, compared to the mental state estimated by the aforementioned mental state estimation unit.

8. The vehicle auxiliary device according to claim 6, wherein, The aforementioned passenger status estimation department also estimates the types of the aforementioned passengers. The recommended route determination unit switches between the recommended route corresponding to the type of passenger estimated by the passenger status estimation unit and the passenger status estimated by the passenger status estimation unit, based on the type of passenger estimated by the passenger status estimation unit.

9. An auxiliary method for vehicles, wherein, Includes the following steps performed by at least one processor: The mental state estimation process estimates the mental state, which is at least one of the degree of deterioration of the driver's mental state and the important factors of the deterioration of the driver's mental state. The recommended path determination process determines a recommended path containing multiple road segments corresponding to the mental association state estimated through the aforementioned mental association state estimation process. The prompt control process guides the driver to the recommended path determined by the recommended path determination process. The selection process involves determining whether or not guidance via the recommended path determined by the recommended path selection process is needed, based on input received from the driver via the input device. as well as The stimulus control process controls the operation of a state improvement device installed in the vehicle that provides the driver with stimuli to improve their mental state from deterioration. The aforementioned prompt control process can also guide users to the recommended path. If the selection process indicates that the recommended path is not required, the recommended path will not be guided in the prompt control process, and the operation of the state improvement device will start automatically in the stimulus control process. On the other hand, if the selection process indicates that the recommended path is required, the operation of the state improvement device will not start automatically in the stimulus control process, and the recommended path will be guided in the prompt control process.

10. An auxiliary device for a vehicle, wherein, have: The mental state estimation unit estimates the mental state, which is at least one of the degree of deterioration of the driver's mental state and important factors of the deterioration of the driver's mental state. The recommended route determination unit determines a recommended route, which includes multiple road segments, corresponding to the mental association state estimated by the aforementioned mental association state estimation unit. The prompt control unit prompts the driver to follow the recommended route determined by the recommended route determination unit; as well as The passenger status estimation unit estimates the status of passengers other than the driver of the aforementioned vehicle, i.e., the passengers. The aforementioned route recommendation unit also determines the recommended route based on the passenger status estimated by the aforementioned passenger status estimation unit. The aforementioned mental state estimation unit at least estimates the degree of deterioration in the driver's mental state as the aforementioned mental state. If the degree of deterioration of the driver's mental state, as estimated by the aforementioned mental state estimation unit, is above a predetermined threshold, the recommended route determination unit decides to prioritize the recommended route corresponding to the mental state estimated by the aforementioned mental state estimation unit, compared to the state of the passenger estimated by the aforementioned passenger state estimation unit. On the other hand, if the degree of deterioration of the driver's mental state, as estimated by the aforementioned mental state estimation unit, is below a predetermined threshold, the recommended route determination unit decides to prioritize the recommended route corresponding to the state of the passenger estimated by the aforementioned passenger state estimation unit, compared to the mental state estimated by the aforementioned mental state estimation unit.

11. The vehicle auxiliary device according to claim 10, wherein, The aforementioned mental state estimation unit at least estimates the significant factors contributing to the deterioration of the driver's mental state as the aforementioned mental state. The aforementioned recommended path determination unit determines the aforementioned recommended path that corresponds to the important factors of the deterioration of the driver's mental state estimated by the aforementioned mental state estimation unit, and is estimated to improve the driver's mental state.

12. The vehicle auxiliary device according to claim 10 or 11, wherein, The aforementioned deterioration of mental state is due to stress.

13. An auxiliary device for a vehicle, wherein, have: The mental state estimation unit estimates the mental state, which is at least one of the degree of deterioration of the driver's mental state and important factors of the deterioration of the driver's mental state. The recommended route determination unit determines a recommended route, which includes multiple road segments, corresponding to the mental association state estimated by the aforementioned mental association state estimation unit. as well as The prompt control unit prompts the driver to follow the recommended route determined by the recommended route determination unit. The aforementioned mental state estimation unit at least estimates the degree of deterioration in the driver's mental state as the aforementioned mental state. The aforementioned recommended path determination unit determines the aforementioned recommended path that corresponds to three or more stages of the deterioration of the driver's mental state as estimated by the aforementioned mental state estimation unit, and is estimated to improve the driver's mental state.

14. An auxiliary method for vehicles, wherein, Includes the following steps performed by at least one processor: The mental state estimation process estimates the mental state, which is at least one of the degree of deterioration of the driver's mental state and the important factors of the deterioration of the driver's mental state. The recommended path determination process determines a recommended path containing multiple road segments corresponding to the mental association state estimated through the aforementioned mental association state estimation process. The prompt control process guides the driver to the recommended path determined by the recommended path determination process. as well as The passenger status estimation process estimates the status of passengers other than the driver of the aforementioned vehicle, i.e., the passengers. In the aforementioned route recommendation process, the recommended route is also determined based on the passenger status estimated in the aforementioned passenger status estimation process. In the aforementioned mental state estimation process, at least the degree of deterioration of the driver's mental state is estimated as the aforementioned mental state. If the degree of deterioration of the driver's mental state estimated by the aforementioned mental state estimation process is above a predetermined threshold, the recommended route determination process prioritizes the recommended route corresponding to the mental state estimated by the aforementioned mental state estimation process, compared to the state of the passenger estimated by the aforementioned passenger state estimation process. On the other hand, if the degree of deterioration of the driver's mental state estimated by the aforementioned mental state estimation process is less than a predetermined threshold, the recommended route determination process prioritizes the recommended route corresponding to the state of the passenger estimated by the aforementioned passenger state estimation process, compared to the mental state estimated by the aforementioned mental state estimation process.

15. An auxiliary method for vehicles, wherein, Includes the following steps performed by at least one processor: The mental state estimation process estimates the mental state, which is at least one of the degree of deterioration of the driver's mental state and the important factors of the deterioration of the driver's mental state. The recommended path determination process determines a recommended path containing multiple road segments corresponding to the mental association state estimated through the aforementioned mental association state estimation process. as well as The control process prompts the driver with the recommended route determined by the recommended route determination process. In the aforementioned mental state estimation process, at least the degree of deterioration of the driver's mental state is estimated as the aforementioned mental state. In the above-mentioned recommended path determination process, the recommended path that is estimated to improve the mental state is determined, which corresponds to the degree of deterioration of the driver's mental state estimated by the above-mentioned mental state estimation process in three or more stages.

Images