Route guidance method and route guidance device

Abstract

To provide a route guidance method that can suppress the decrease in safety associated with voice guidance. [Solution] The determination unit 9 acquires the status of the driver operating the vehicle and determines the driver's margin of safety while driving based on the acquired status of the driver. The voice guidance control determination unit 13 controls the voice guidance based on the determined driver's margin of safety. This makes it possible to provide route guidance in a manner that corresponds to the driver's margin of safety. Therefore, for example, if the driver's margin of safety is low, the voice guidance can be controlled to minimize the decrease in margin of safety, thereby suppressing the decrease in safety associated with voice guidance.

Description

Technical Field

[0001] The present disclosure relates to a route guidance method and a route guidance device.

Background Art

[0002] Conventionally, a route guidance method navigation device that provides voice guidance regarding information on a route along which a vehicle should travel has been proposed (see, for example, Patent Document 1). In the navigation device described in Patent Document 1, based on vehicle information, map data, and the position of the own vehicle, it is determined whether the driving situation requires the driver to pay attention to the surrounding traffic situation (for example, when changing lanes), and if it is determined that such a driving situation exists, the playback timing of the voice guidance is shifted.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] <00​​​​​​​​​A route guidance method according to one aspect of the present disclosure is a route guidance method that provides voice guidance regarding information about the route a vehicle should take, and includes acquiring the state of the driver operating the vehicle, determining the driver's margin of safety while driving based on the acquired driver's state, and controlling the voice guidance based on the determined driver's margin of safety.

[0006] Furthermore, a route guidance device according to one aspect of the present disclosure is a route guidance device that provides voice guidance regarding information about the route a vehicle should take, and comprises: a driving state detection unit that detects the state of the driver operating the vehicle; a determination unit that determines the driver's margin of safety while driving based on the driver's state detected by the driving state detection unit; and a control unit that controls voice guidance based on the driver's margin of safety determined by the determination unit. [Effects of the Invention]

[0007] This disclosure provides a route guidance method and a route guidance device that can suppress the decrease in safety associated with voice guidance. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows a schematic configuration of a route guidance device according to the first embodiment. [Figure 2] This is a schematic diagram showing the different regions of the driver's brain. [Figure 3] This table shows the relationship between the driver's level of composure and the timing of their speech. [Figure 4] This diagram shows the various functions that the controller performs. [Figure 5] This is a flowchart showing the process for determining the margin of safety. [Figure 6] This flowchart shows the details of the attention state estimation process. [Figure 7] This flowchart shows the process for estimating the required attention state. [Figure 8] This figure shows a table illustrating the relationship between the driver's attention state, required attention state, and margin level. [Figure 9] This is a flowchart showing the contents of the speech control process. [Figure 10] A table showing the relationship between the driver's margin and the speech rate. [Figure 11] A table showing the relationship between the driver's margin and the way of speaking the message. [Figure 12] A table showing the relationship between the driver's margin and the speech content. [Figure 13] A diagram showing the schematic configuration of the route guidance device according to the second embodiment. [Figure 14] A diagram showing each function realized by the controller of the second embodiment. [Figure 15] A flowchart showing the content of the speech control process of the second embodiment. [Figure 16] A diagram showing each function realized by the controller of the modified example. [Figure 17] A diagram showing each function realized by the controller of the modified example.

Mode for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Note that each drawing is schematic and may be different from the actual one. In addition, the embodiments of the present disclosure shown below are examples of devices and methods for embodying the technical idea of the present disclosure, and the technical idea of the present disclosure does not specify the structure, arrangement, etc. of the components as follows. The technical idea of the present disclosure can be variously modified within the technical scope defined by the claims described in the claims.

[0010] (First Embodiment) In the first embodiment, as shown in FIG. 1, a case where the route guidance method and the route guidance device of the present disclosure are applied to a route guidance device 1 that performs voice guidance and display guidance on information on the route that the vehicle should travel is illustrated. FIG. 1 is a diagram showing the schematic configuration of the route guidance device 1 according to the first embodiment. The route guidance device 1 includes an external environment detection unit 2, a host vehicle state detection unit 3, a driver state detection unit 4, a human machine interface (HMI) 5, a navigation device 6, a voice guidance control DB 7, and a controller 8.

[0011] The external environment detection unit 2 detects various information about the external environment around the vehicle C (hereinafter referred to as "external environment information"). For example, the external environment detection unit 2 detects moving objects such as pedestrians, other vehicles, and bicycles around the vehicle C, and stationary objects such as traffic lights, traffic signs, road markings, curbs on the road shoulder, guardrails, etc., and lines on the road surface (stop lines, lane boundary lines, lane division lines, etc.). The external environment detection unit 2 detects the relative position and distance from these detection targets and the direction in which the detection targets exist as the external environment of the vehicle C, and outputs the detected external environment information to the controller 8 as external environment information. As the external environment detection unit 2, for example, a plurality of different types of object sensors for detecting detection targets around the vehicle C, such as a camera, a laser radar, a millimeter wave radar, and a LIDAR (Light Detection and Ranging), can be employed. The camera detects an image around the vehicle C. That is, it captures an image of the external environment outside the vehicle. The detection result is output to the controller 8.

[0012] The host vehicle state detection unit 3 detects various vehicle state information (hereinafter referred to as "vehicle information") obtained from the vehicle C. As the host vehicle state detection unit 3, for example, a vehicle speed sensor for detecting the vehicle speed of the vehicle C, a wheel speed sensor for detecting the rotational speed of the tires of the vehicle C, a three-axis acceleration sensor for detecting the acceleration and deceleration in the three-axis directions of the vehicle C, a steering angle sensor for detecting the steering angle of the steering wheel, a steering angle sensor for detecting the steering angle of the steering wheel, a shift position sensor for detecting the shift position of a shift lever or the like (for example, parking (P), drive (D), reverse (R), etc.), a gyro sensor for detecting the angular velocity of the vehicle C, a yaw rate sensor for detecting the yaw rate of the vehicle C, an accelerator sensor for detecting the accelerator opening of the vehicle C, and a brake sensor for detecting the brake operation amount of the vehicle C can be employed. The detection result is output to the controller 8.

[0013] The driver state detection unit 4 detects various information (driver state information) about the state of the driver of vehicle C. For example, the driver state detection unit 4 detects the driver's brain activity, the driver's heart rate, the driver's sweating, the direction of the driver's gaze, the speed and range of gaze movement, the driver's facial expression, the driver's vehicle operation information (timing, amount, and frequency of steering and acceleration / deceleration), and vehicle speed. The detection results are output to the controller 8. In the first embodiment, a case in which a brain activity sensor that detects the driver's brain activity is used as the driver state detection unit 4 will be described. As a brain activity sensor, for example, an electroencephalogram (EEG) sensor, a magnetoencephalogram (MEG) sensor, a sensor that measures brain activity using functional magnetic resonance imaging (fMRI), or a sensor that measures brain activity using functional near-infrared spectroscopy (fNIRS) can be used.

[0014] The brain activity sensor used in the first embodiment detects activity in multiple regions of interest (ROIs) of the driver's brain and generates a brain activity signal indicating the activity levels of these regions. Examples of brain activity signals include fMRI data, fNIRS data, EEG (ElectroEncephaloGraphy) data, and MEG (MagnetoEncephaloGraphy) data. In the first embodiment, the case in which time-series data of the BOLD (Blood Oxygenation Level Dependent) signal obtained by fMRI is used as the brain activity signal will be described. Furthermore, in the first embodiment, the case in which the regions of interest are set to each of the following regions (1) to (6) will be described, as shown in Figure 2. Figure 2 is a schematic diagram showing each region of the driver's brain. (1) Left parietal-temporal area (l-PTA) (2) Right parietal-temporal area (r-PTA) (3) Left frontal area (l-FA) (4) Right frontal area r-FA (5) The medial parietal area (m-PA), which is the area inside the crown of the head. (6) The medial prefrontal area (m-PFA), which is an internal region of the frontal lobe.

[0015] In Figure 2, the medial frontal m-PFA overlaps with the left surface frontal l-FA and the right surface frontal r-FA. However, the medial frontal m-PFA is an internal brain region, while the left surface frontal l-FA and the right surface frontal r-FA are surface brain regions; therefore, these regions do not actually overlap. Similarly, the medial parietal m-PA is an internal brain region, while the left surface parietal or temporal l-PTA and the right surface parietal or temporal r-PTA are surface brain regions; therefore, these regions do not actually overlap. Furthermore, the parietal, temporal, and frontal regions may be selected as the parietal association cortex, temporal gyrus, and frontal gyrus, respectively. Additionally, the medial parietal region may be selected as the posterior cingulate cortex, and the medial frontal region may be selected as the anterior cingulate cortex or insular cortex.

[0016] HMI5 is an interface device that exchanges information between the controller 8 and the driver of vehicle C. For example, HMI5 can be a display device that shows image information visible to the driver of vehicle C, a speaker that outputs audio information that the driver can hear, and controls that the driver can operate (e.g., buttons, switches, levers, dials, touch panels). The navigation device 6 detects the current position of vehicle C using a positioning device (not shown), acquires map data for the detected current position, sets a target driving route to the destination entered by the driver, and provides route guidance to the driver according to the target driving route. In route guidance, the navigation device 6 provides voice guidance and display guidance for information on the route that vehicle C should take (hereinafter referred to as "route information") and various other information. Examples of route information include guidance on the direction of travel (e.g., guidance on turning right or left at intersections, guidance on lanes and side roads), and information such as "Continue straight for **km ahead." Examples of other information include refresh guidance, headlight activation guidance, and railroad crossing guidance. In voice guidance, route information is output by voice from the speaker of the HMI 5. In display guidance, route information is displayed as an image on the display of the HMI 5. Voice guidance can be switched on and off by the driver operating the navigation device 6. In other words, when turned ON, the output of route information and other information via voice guidance is permitted, and when turned OFF, the output of route information and other information via voice guidance is prohibited. The display guidance is always ON, and the display of route information and other information via display guidance is always permitted. Furthermore, the navigation device 6 can control (change) the voice guidance according to commands from the controller 8.

[0017] The voice guidance control DB7 stores control rules that are referenced in the control of voice guidance based on the driver's margin of safety, as described later. For example, as shown in Figure 3, the voice guidance control DB7 stores a table showing the relationship between the driver's margin of safety and the timing of speech. The voice guidance control DB7 can employ storage means such as a small-capacity high-speed memory (e.g., cache memory) configured using SRAM (Static Random Access Memory), or a fixed disk device such as an HDD (Hard Disk Drive) or SSD (Solid State Drive). Margin of safety can be defined as the amount of information processing resources available for cognition and judgment when performing a driving task. When the margin of safety is high, the driver can pay appropriate attention to the external environment and make appropriate cognition and judgments. On the other hand, when the margin of safety is low, it becomes difficult for the driver to pay appropriate attention to the external environment and to make appropriate cognition and judgments. For example, a state of low margin of safety is when the driver pays excessive attention to a specific matter, leaving no margin to pay attention to other matters.

[0018] The controller 8 comprises a processor 8a and peripheral components such as a storage device 8b for storing computer programs and the like. For example, the processor 8a can be a CPU (Central Processing Unit) or an MPU. For example, the storage device 8b can be a semiconductor storage device, a magnetic storage device, or an optical storage device. The storage device 8b may also include registers, cache memory, and memory such as ROM and RAM used as main memory. Each function of the controller 8 described below is realized, for example, by the processor 8a executing a computer program stored in the storage device 8b.

[0019] Next, we will explain each function of the controller 8 in detail. As shown in Figure 4, the controller 8 implements the functions of the determination unit 9 (including the attention state estimation unit 10, the required attention state estimation unit 11, and the margin determination unit 12) and the voice guidance control determination unit 13 (broadly speaking, the "control unit"). Figure 4 is a diagram showing the functions implemented by the controller 8. The attention state estimation unit 10 performs a margin determination process to determine the driver's margin of safety while driving, based on driver information. The margin determination process is performed every predetermined time (for example, 10 minutes). That is, the margin of safety is updated every predetermined time. In the margin determination process, an attention state estimation process is performed to estimate the driver's attention state based on the presence or absence of the first functional connectivity C1, the second functional connectivity C2, and the third functional connectivity C3 (S101 in Figure 5). In the following description, the driver's attention state estimated based on the functional connectivity between brain activity in multiple brain regions will be referred to as the "driver's attention state". Figure 5 is a flowchart showing the contents of the margin determination process.

[0020] In the attention state estimation process, the attention state estimation unit 10 acquires brain activity signals from multiple regions of interest in the driver's brain from the driver state detection unit 4 (S201 in Figure 6). That is, it acquires the state of the driver operating vehicle C. Figure 6 is a flowchart showing the contents of the attention state estimation process. Subsequently, the attention state estimation unit 10 determines the presence or absence of functional connectivity (functional connectivity or functional brain connectivity), which is the connectivity of brain activity between these regions of interest, based on the brain activity signals of the regions of interest set for each of the regions (1) to (6) above. Functional connectivity is a pattern of statistical dependency between different regions in the nervous system. In the following explanation, as shown in Figure 2, the functional connectivity between the parietal or temporal l-PTA on the left surface or the parietal or temporal r-PTA on the right surface and the frontal l-FA on the left surface or the frontal r-FA on the right surface will be referred to as "first functional connectivity C1". In other words, the first functional connectivity C1 may be any of the following: functional connectivity between the vertex or temporal l-PTA on the left surface and the frontal l-FA on the left surface; functional connectivity between the vertex or temporal l-PTA on the left surface and the frontal r-FA on the right surface; functional connectivity between the vertex or temporal r-PTA on the right surface and the frontal l-FA on the left surface; or functional connectivity between the vertex or temporal r-PTA on the right surface and the frontal r-FA on the right surface.

[0021] Furthermore, the functional connectivity between the left surface vertex or temporal l-PTA and the medial vertex m-PA is denoted as "second functional connectivity C2L". The functional connectivity between the right surface vertex or temporal r-PTA and the medial vertex m-PA is denoted as "second functional connectivity C2R". The second functional connectivity C2L and the second functional connectivity C2R are collectively referred to as "second functional connectivity C2". The functional connectivity between the medial vertex m-PA and the medial frontal m-PFA is denoted as "third functional connectivity C3".

[0022] When determining whether or not there is functional connectivity between regions of interest i and j, the attention state estimation unit 10 first uses the time series data R of the BOLD signals in these regions of interest i and j. i (t) and R j (t) is generated. Next, the generated time series data R i (t) and R j Based on (t), the correlation coefficient Z(i,j) between brain activity in region i and region j is calculated according to equations (1) and (2) below, and it is determined whether the calculated correlation coefficient Z(i,j) is greater than or equal to a threshold.

number

number

[0023] For example, when determining the presence or absence of a second functional connectivity C2R between the right-side vertebral or temporal r-PTA and the medial vertebral m-PA, it is determined whether functional connectivity exists for each pair of regions of interest R11, R12, R13 within the right-side vertebral or temporal r-PTA and R21, R22, R23 within the medial vertebral m-PA (R11, R21), (R11, R22), (R11, R23), (R12, R21), (R12, R22), (R12, R23), (R13, R21), (R13, R22), and (R13, R23). If functional connectivity exists for one or more pairs of regions of interest, it is determined that a second functional connectivity C2R exists. On the other hand, if no pair of domains of interest has functional connectivity, it is determined that there is no second functional connectivity C2R. The number of pairs of domains of interest that have functional connectivity is obtained as the number of second functional connectivity C2Rs. The presence or absence of the first functional connectivity C1, the second functional connectivity C2L, and the third functional connectivity C3 is determined in the same manner.

[0024] The inventors of this disclosure presented subjects with videos of three types of events: events with multiple moving attention objects, events with only one moving attention object, and events with a stationary attention object, and analyzed the presence or absence of functional connectivity in the areas (1) to (6) described above. As a result, they experimentally confirmed that the functional connectivity of these areas differs depending on the attentional state of the subject. Therefore, the route guidance device 1 of the first embodiment estimates the driver's attentional state based on the presence or absence of the first functional connectivity C1, the second functional connectivity C2, and the third functional connectivity C3. In this way, the accuracy of estimating the driver's attentional state can be improved by making judgments based on the functional connectivity of specific brain regions. As a result, it is possible to evaluate not only the driver's level of arousal and concentration, but also the appropriateness of the driver's perception and judgment regarding risks in the external environment.

[0025] Next, the attention state estimation unit 10 determines whether or not there is a first functional connectivity C1 based on the determination result of S202 in Figure 6 (S203 in Figure 6). If it determines that there is no first functional connectivity C1 (S203 "No" in Figure 6), it determines whether or not there is a third functional connectivity C3 (S204 in Figure 6). If it determines that there is a third functional connectivity C3 (S204 "Yes" in Figure 6), it estimates that the driver's attention state is a fourth attention state (or a state in which the driver is not paying attention to the outside) where the driver is not paying attention to the external environment (S205 in Figure 6), and then terminates the attention state estimation process. The fourth attention state is a state in which a default mode network occurs, such as when the driver is daydreaming. For example, it is a state that occurs when vehicle C is parked or stopped. On the other hand, if the attention state estimation unit 10 determines that there is no third functional connectivity C3 (S204 "No" in Figure 6), it returns to S201 above.

[0026] On the other hand, if the attention state estimation unit 10 determines that there is a first functional connectivity C1 (S203 "Yes" in Figure 6), it determines whether there is a second functional connectivity C2 (S206 in Figure 6). If it determines that there is no second functional connectivity C2 (S206 "No" in Figure 6), it estimates that the driver's attention state is a third attention state, which is more attentive than the fourth attention state (S207 in Figure 6), and then terminates the attention state estimation process. The third attention state is a state in which a network is generated, such as when the driver is engaging in cognitive activity with a simple attention object. For example, it is a state that occurs when the driver is paying attention to a stationary attention object. On the other hand, if the attention state estimation unit 10 determines that there is a second functional connectivity C2 (S206 "Yes" in Figure 6), it determines whether the number of second functional connectivity C2s is greater than 1 (S208 in Figure 6). If it determines that the number of second functional connectivity C2s is 1 (S208 "No" in Figure 6), it estimates that the driver's attention state is a second attention state, which is more attentive than the third attention state (S209 in Figure 6), and then terminates the attention state estimation process. The second attention state is a state in which a network is generated in a specific area of ​​the brain when the driver is engaging in cognitive activities with respect to complex attention objects that require referencing information from experience or decision-making. For example, it is a state that occurs when the driver is paying attention to a single moving attention object. On the other hand, if the attention state estimation unit 10 determines that the number of second functional connectivity C2 is greater than 1 (S208 "Yes" in Figure 6), it estimates that the driver's attention state is a first attention state, which is more attentive than the second attention state (S210 in Figure 6), and then terminates the attention state estimation process. The first attention state is a state in which a network is generated in a specific area of ​​the brain when the driver is engaging in cognitive activities with respect to complex attention objects that require referencing information from experience or decision-making. For example, it is a state in which the driver is paying attention to multiple moving attention objects.

[0027] When the attention state estimation process is started, the required attention state estimation unit 11 performs a required attention state estimation process (S101 in Figure 5) to estimate the required attention state, which is the attention state required of the driver in relation to the external environment (the attention state the driver should be in), based on external environment information and vehicle information. As a result, the attention state estimation process and the required attention state estimation process are executed simultaneously, and the acquisition of brain activity heart data by the attention state estimation process (S201 in Figure 6) and the acquisition of vehicle information and external environment information by the required attention state estimation process (S301, S302 in Figure 7) are performed simultaneously. In the requested attention state estimation process, the requested attention state estimation unit 11 acquires vehicle information (for example, vehicle speed and shift position information of vehicle C) from the vehicle state detection unit 3 (S301 in Figure 7). Figure 7 is a flowchart showing the contents of the requested attention state estimation process. Next, the requested attention state estimation unit 11 acquires external environment information (for example, an image of the environment outside the vehicle) from the external environment detection unit 2 (S302 in Figure 7). Subsequently, based on the acquired external environment information, it detects potential attention targets that may be the driver's focus (for example, moving objects such as pedestrians, other vehicles, and bicycles around vehicle C, and stationary objects such as traffic lights, traffic signs, and road markings) (S303 in Figure 7).

[0028] Next, the required attention state estimation unit 11 determines whether the vehicle speed of vehicle C is "0" (i.e., whether vehicle C is stopped) based on the acquired vehicle information (S304 in Figure 7). If it determines that the vehicle speed is "0" (S304 "Yes" in Figure 7), it determines whether vehicle C is parked based on the vehicle information (shift position information) acquired in S301 (S305 in Figure 7). If it determines that vehicle C is parked (S305 "Yes" in Figure 7), it estimates that the required attention state is the fourth attention state, where the driver's attention to the external environment is not required (S306 in Figure 7), and then terminates the required attention state estimation process. On the other hand, if the attention-requiring state estimation unit 11 determines that vehicle C is not parked (S305 "No" in Figure 7), it estimates that the attention-requiring state is the third attention state, which requires more attention than the fourth attention state (S307 in Figure 7), and then terminates the attention-requiring state estimation process.

[0029] On the other hand, if the requested attention state estimation unit 11 determines that the vehicle speed of vehicle C is not "0" (S304 "No" in Figure 7), it determines whether the candidate for attention is a moving object (S308 in Figure 7). If it determines that it is not a moving object (S308 "No" in Figure 7), it estimates that the requested attention state is the third attention state (S307 in Figure 7) and then terminates the requested attention state estimation process. On the other hand, if the attention target candidate is determined to be a moving object (S308 "Yes" in Figure 7), the attention target candidate is determined to be greater than "1" (S309 in Figure 7). If the attention target candidate is determined to be "1" (S309 "No" in Figure 7), the attention target candidate is determined to be a second attention state requiring more attention than a third attention state (S310 in Figure 7), and then the attention target candidate estimation process is terminated. On the other hand, if the attention target candidate is determined to be greater than "1" (S309 "Yes" in Figure 7), the attention target candidate is determined to be a first attention state requiring more attention than a second attention state (S310 in Figure 7), and then the attention target candidate estimation process is terminated.

[0030] When the attention state estimation process and the required attention state estimation process are completed, the margin determination unit 12 determines whether the attention level of the driver's attention state (first attention state to fourth attention state) obtained in S101 is higher than the attention level of the required attention state (first attention state to fourth attention state) obtained in S101 (S102 in Figure 5). If the attention level of the driver's attention state is higher than the attention level of the required attention state (S102 "Yes" in Figure 5), the unit refers to the table shown in Figure 8 and determines the driver's margin based on the attention level of the driver's attention state and the attention level of the required attention state (S103 in Figure 5), and then terminates the margin determination process. Figure 8 is a table showing the relationship between the driver's attention state, the required attention state and the margin level. In the table in Figure 8, when the driver's attention state is the first attention state, the margin is determined to be "Level 4" when the required attention state is the second attention state, "Level 2" when it is the third attention state, and "Level 1" when it is the fourth attention state. Furthermore, when the driver's attention state is the second attention state, the margin is determined to be "Level 5" when the required attention state is the third attention state, and "Level 3" when it is the fourth attention state. Furthermore, when the driver's attention state is the third attention state, the margin is determined to be "Level 6" when the required attention state is the fourth attention state. On the other hand, if the margin determination unit 12 determines that the level of attention in the driver's attention state is less than or equal to the level of attention in the required attention state (S102 "No" in Figure 5), it terminates the margin determination process.

[0031] When the margin determination process is completed, the voice guidance control determination unit 13 executes a speech control process to control voice guidance based on the margin determined in the margin determination process. In the speech control process, it is determined whether the driver of vehicle C has a low margin (S401 in Figure 9). For example, if the margin is "Level 4" or lower, it is determined that the driver has a low margin. Figure 9 is a flowchart showing the contents of the speech control process. If the voice guidance control determination unit 13 determines that the driver has a low margin (S401 "Yes" in Figure 9), it determines whether voice guidance is on (S402 in Figure 9). If it determines that voice guidance is on (S402 "Yes" in Figure 9), it refers to the table shown in Figure 3, selects the speech pattern of the voice guidance based on the driver of vehicle C, and outputs a command (a command to control voice guidance) to the navigation device 6 to output the voice in the selected pattern (S403 in Figure 9). For example, as shown in Figure 3, when the driver's margin of safety is low, the voice guidance is changed to one that can suppress the decrease in the driver's margin of safety. Figure 3 illustrates a case where, when the driver's margin of safety is below a predetermined level (e.g., level 4), a command is output to the navigation device 6 to speed up the timing of the voice guidance compared to when it is above the predetermined level. For example, it is set to 30 to 60 seconds earlier than the normal timing. As a result, the navigation device 6 controls the timing of the voice guidance to the earlier timing set in S403. At that time, the navigation device 6 provides both voice guidance and display guidance, and outputs information on the route that vehicle C should take on both the display and speaker of the HMI 5.

[0032] On the other hand, if the voice guidance control determination unit 13 determines that voice guidance is off (S402 "No" in Figure 9), it suggests to the driver to switch voice guidance on (S404 in Figure 9). That is, if the driver's level of composure is below a predetermined level (for example, level 4), and the unit determines that voice guidance is off, it suggests to the driver to switch voice guidance on. The suggestion to the driver is made, for example, via the display of the HMI 5. Next, the voice guidance control determination unit 13 determines whether the driver has performed the operation to switch voice guidance on (S405 in Figure 9). If it determines that the driver has performed the operation to switch voice guidance on (S405 "Yes" in Figure 9), it refers to the table shown in Figure 3 and, based on the driver's composure level of vehicle C, selects the manner of voice guidance utterance and outputs a command to the navigation device 6 to output the voice in the selected manner (S403 in Figure 9). As a result, the navigation device 6 controls the timing of voice guidance utterance to the earlier timing set in S403. On the other hand, if the voice guidance control determination unit 13 determines that the driver has not performed the operation to switch voice guidance on (S405 "No" in Figure 9), it terminates the speech control process.

[0033] On the other hand, if the voice guidance control determination unit 13 determines that the driver's level of safety is high (for example, level 5 or higher) (S401 "No" in Figure 9), it determines whether voice guidance is on (S406 in Figure 9). If it determines that voice guidance is on (S406 "Yes" in Figure 9), it outputs a command to the navigation device 6 to output voice in the normal manner (for example, at the normal speaking timing). As a result, the navigation device 6 controls the speaking timing of the voice guidance to the normal speaking timing. On the other hand, if the voice guidance control determination unit 13 determines that voice guidance is off (S406 "No" in Figure 9), it terminates the speaking control process.

[0034] (Effects of the first embodiment) (1) In the first embodiment, voice guidance is provided regarding the route that vehicle C should take. The state of the driver operating vehicle C is also acquired, and the driver's level of safety during driving is determined based on the acquired state of the driver. Then, the voice guidance is controlled based on the determined level of safety of the driver. This makes it possible to provide route guidance in a manner that corresponds to the driver's level of safety. Therefore, for example, if the driver's level of safety is low, the voice guidance can be controlled to minimize the decrease in safety, thereby suppressing the decrease in safety associated with voice guidance.

[0035] (2) In the first embodiment, route information is output not only by voice guidance but also by display guidance. Voice guidance can be switched on and off. Display guidance is always on. Furthermore, if the driver's level of composure is below a predetermined level and it is determined that voice guidance is off, the system suggests to the driver to switch voice guidance on. This means that, for example, if the driver's level of composure is low and the driver does not have time to look at the display guidance, the system suggests to the driver to switch voice guidance on. Therefore, route guidance can be provided by voice guidance while minimizing the decrease in composure.

[0036] (3) In the first embodiment, when the driver's level of composure is below a predetermined level (for example, level 4), the timing of the voice guidance is set earlier compared to when it is above the predetermined level. This allows the timing of the voice guidance to be set earlier, for example, when the driver's composure is low and it takes time to understand the voice guidance. Therefore, the reduction in composure due to voice guidance can be minimized, and the content of the voice guidance can be conveyed to the driver quickly, allowing the driver to perform appropriate driving operations even if it takes time to understand the content.

[0037] (Modification of the first embodiment) (1) In the first embodiment, the voice guidance control determination unit 13 was shown to speed up the timing of voice guidance when the driver's margin level is below a predetermined level (e.g., level 4), but other configurations can also be adopted. For example, as shown in Figure 10, when the driver's margin level is below a predetermined level (e.g., level 3), the voice guidance speaking speed may be slowed down compared to when it is above the predetermined level. This allows the voice guidance speaking speed to be slowed down when, for example, the driver's margin is low and the voice guidance is difficult to hear. Therefore, the decrease in margin due to voice guidance can be minimized, and the content of the voice guidance can be made easier to hear, allowing the driver to perform driving operations more appropriately. In this case, in S401 of Figure 9, it is determined that the driver's margin is low when the margin level is below level 3. Also, in S403 of Figure 9, a command to slow down the voice guidance speaking speed (a command to use a slow speaking speed) is output to the navigation device 6 compared to the normal speaking speed. For example, if the normal speaking speed is 300 characters / minute, a slower speaking speed of 200 characters / minute can be used. Also, in S407 of Figure 9, a command is sent to the navigation device 6 to output the voice guidance audio at the normal speaking speed.

[0038] (2) Furthermore, for example, as shown in Figure 11, if the driver's level of composure is below a predetermined level (e.g., level 2), the system may be configured to use messages with fewer characters per sentence as the voice guidance messages (i.e., guidance texts) compared to when the level is higher than the predetermined level. This allows the voice guidance messages to be shorter when, for example, the driver's composure is low and the voice guidance is difficult to hear. Therefore, the reduction in composure due to voice guidance can be minimized, and the content of the voice guidance can be made easier to hear, allowing the driver to perform driving operations more appropriately. In this case, in S401 of Figure 9, it is determined that the driver's composure is low when the composure level is below level 2. Also, in S403 of Figure 9, a command is output to the navigation device 6 to use messages with fewer characters per sentence as the voice guidance messages compared to normal messages. For example, if the normal message is "Turn left into the left-turn-only lane 500m ahead," then a message with fewer characters per sentence could be created by dividing the normal message into several short sentences, such as "Enter the left-turn-only lane. Turn left 500m ahead." Also, in S407 of Figure 9, a command is output to the navigation device 6 to use the normal message as the message spoken in the voice guidance.

[0039] (3) Furthermore, as shown in Figure 12, for example, if the driver's level of safety is below a predetermined level (e.g., level 1), the system may be configured to minimize the content spoken in the voice guidance (e.g., only guidance on the direction of travel). This makes it possible to minimize the content spoken in the voice guidance when, for example, the driver's level of safety is low and the voice guidance is difficult to hear. Therefore, the decrease in safety due to voice guidance can be minimized, the frequency of voice guidance can be reduced, guidance on the direction of travel can be made easier to hear, and the driver can perform driving operations more appropriately. In this case, in S401 of Figure 9, it is determined that the driver's level of safety is low when the level of safety is below level 1. Also, in S403 of Figure 9, a command is output to the navigation device 6 to limit the content spoken in the voice guidance to only guidance on the direction of travel, unlike the content spoken under normal circumstances. Examples of guidance on the direction of travel include guidance on turning right or left at intersections, and guidance on lanes and side roads. Furthermore, in S408 of Figure 9, a command is output to the navigation device 6 that uses the content normally spoken in the voice guidance. Examples of content normally spoken include directions of travel, information such as "Continue straight for **km," refresh guidance, headlight activation guidance, and level crossing guidance.

[0040] (Second Embodiment) Next, a route guidance device 1 according to the second embodiment of this disclosure will be described. Figure 13 is a diagram showing the schematic configuration of the route guidance device 1 according to the second embodiment. Figure 14 is a diagram showing the functions realized by the controller 8 of the second embodiment. Figure 15 is a flowchart showing the contents of the speech control processing of the second embodiment. In Figures 13, 14, and 15, the same reference numerals are used for parts corresponding to Figures 1, 4, and 9, and redundant explanations are omitted. The route guidance device 1 according to the second embodiment differs from the route guidance device 1 according to the first embodiment in that, as shown in Figure 13, it includes a personal database 14, as shown in Figure 14, it has a learning unit 15 as a function implemented by the controller 8, and as shown in Figure 15, it has S501 before S401 and S502 after S403, S405, S406 and S407.

[0041] The individual DB14 stores information for each driver that shows the correlation between driver status information (e.g., brain activity, heart rate, etc.) acquired at regular intervals during route guidance by the navigation device 6 and the driver's margin of safety. For example, brain activity and the driver's margin of safety are stored in association. The storage of driver status information and the driver's margin of safety in the individual DB14 is performed before the determination in S401 (S501 in Figure 15) and after S403 and S405-S407 (S502 in Figure 15). The learning unit 15 refers to the individual DB 14 and learns the criteria for determining the driver's margin of safety. Examples of criteria include the threshold of brain activity used to determine the driver's margin of safety (for example, the numerical value "1" used in S208 in Figure 6). As shown in Figure 14, the criteria are output to the determination unit 9. The determination unit 9 selects the criteria for the current driver of vehicle C from among these criteria and uses the selected criteria to determine the driver's margin of safety. Figure 14 illustrates a case where the determination unit 9 determines the driver's margin of safety using the criteria for each driver based on the driver's state (driver state information). Examples of learning methods include supervised learning, unsupervised learning, and reinforcement learning. Examples of learning algorithms include support vector machines (SVM), decision tree learning, neural networks (NN), genetic programming (GP), Bayesian networks, and large-scale language models (LLM).

[0042] (Effects of the second embodiment) Here, the relationship between driver state information (brain activity) and the driver's margin of safety may differ from driver to driver. For example, even with the same brain activity, one driver may have a high level of margin of safety, while another driver may have a low level. Therefore, if the same criteria for determining margin of safety are used for all drivers, the accuracy of the margin of safety assessment may decrease. In contrast, in the second embodiment, the driver's margin is determined based on the driver's condition and using judgment criteria specific to each driver. This improves the accuracy of determining the driver's margin.

[0043] (Modified version of the second embodiment) (1) In the second embodiment, the determination unit 9 calculates the driver attention state and the required attention state based on driver state information, and determines the driver's margin of safety while driving based on the calculated driver attention state and the required attention state. However, other configurations can also be adopted. For example, as shown in Figure 16, the determination unit 9 may be configured to directly determine the driver's margin of safety from the driver state information. In this case, the learning unit 15 learns thresholds of driver state information that classify the level of the driver's margin of safety (e.g., thresholds for brain activity and heart rate) as criteria for determining the margin of safety for each driver, and stores the learned thresholds in the personal DB 14. The determination unit 9 then selects the threshold for the current driver of vehicle C from among the thresholds stored in the personal DB 14, and determines the driver's margin of safety by comparing the selected threshold with the driver state information obtained from the driver state detection unit 4. Figure 16 is a diagram showing the functions realized by the modified controller 8.

[0044] (2) In the second embodiment, an example was shown in which the learning unit 15 learns the criteria for determining the margin level for each driver, but other configurations can also be adopted. For example, as shown in Figure 17, the learning unit 15 may be configured to learn control rules that are referenced when selecting voice guidance control based on the margin level for each driver. In this case, the personal DB 14 stores information used for learning the control rules for each driver. For example, the required attention state estimated by the required attention state estimation unit 11, the margin level determined by the margin level determination unit 12 when that required attention state is estimated, the control content of the voice guidance selected by the voice guidance control determination unit 13 when that margin level is determined (the manner of speaking), the driver attention state estimated by the attention state estimation unit 19 when voice guidance (voice output) is performed in that manner, and the margin level determined by the margin level determination unit 12 are stored in association with each other.

[0045] The learning unit 15 then refers to the individual DB 14 and learns the optimal control rule for each driver (for example, a table showing the relationship between the driver's level of safety and the control content of the voice guidance, as shown in Figure 3, etc.) based on the driver's level of safety, the control content of the voice guidance corresponding to that level of safety, and the degree of impact on the level of safety when the voice guidance is given in that manner. For example, the system may be configured to learn which of the following should be used as the control of the voice guidance corresponding to the level of safety: "speech timing" (see Figure 3), "speech speed" (see Figure 10), "phrase" (see Figure 11), or "speech content" (see Figure 12). As an example, if the level of safety is level 5, the normal manner is used; if it is level 4, the "speech timing" is made earlier; if it is level 3, the "speech speed" is made slower; if it is level 2, the "phrase" is made into short sentences; and if it is level 1, the "speech content" is made to the minimum. Furthermore, for example, a threshold for switching the control of voice guidance speech can be learned (for example, "normal" and "fast" in Figure 3). As for the degree of influence on the margin, for example, the amount of decrease in the margin determined during voice guidance relative to the margin determined before voice guidance is given can be adopted. As for the optimal control rule, for example, the control rule with the smallest degree of influence on the margin can be adopted. The control rules are stored in the voice guidance control DB7. As a result, in S403 in Figure 9, the voice guidance control determination unit 13 refers to the control rules stored in the voice guidance control DB7 and selects the manner of voice guidance speech based on the margin of the driver of vehicle C.

[0046] Here, the cognitive load required to understand the content of the voice guidance may differ from person to person and from person to person (e.g., level of composure), even if the timing, speed, wording, and content of the voice guidance are the same. Therefore, if the voice guidance is controlled and the voice guidance is changed to suppress a decrease in the driver's composure, the degree to which the change in voice guidance affects the composure may also differ from person to person and from person to person. For this reason, in this modified example, a configuration was adopted in which control rules are learned for each driver. As a result, for example, by learning a control rule that minimizes the degree to which it affects the driver's composure, the impact of the decrease in composure caused by voice guidance can be minimized, and the driver can perform driving operations more appropriately. [Explanation of Symbols]

[0047] 1…Route guidance device, 2…External environment detection unit, 3…Vehicle state detection unit, 4…Driver state detection unit, 5…HMI, 6…Navigation device, 7…Voice guidance control DB, 8…Controller, 8a…Processor, 8b…Storage device, 9…Decision unit, 10…Attention state estimation unit, 11…Required attention state estimation unit, 12…Margin determination unit, 13…Voice guidance control decision unit, 14…Personal DB, 15…Learning unit

Claims

1. A route guidance method that provides voice guidance about the route a vehicle should take, The system acquires the status of the driver operating the vehicle. Based on the driver's condition obtained, the driver's margin of safety during driving is determined. A route guidance method that controls the voice guidance based on the driver's margin of safety determined by the driver.

2. The output of the aforementioned information is also provided through display guidance in addition to the voice guidance. The aforementioned voice guidance can be switched between on and off. The aforementioned display is always on. Furthermore, the route guidance method according to claim 1, which further involves proposing to the driver to switch on the voice guidance when it is determined that the driver's level of safety is below a predetermined level and the voice guidance is off.

3. The route guidance method according to claim 1, wherein if the driver's margin level is below a predetermined level, the timing of the voice guidance is set earlier than when it is above the predetermined level.

4. The route guidance method according to claim 1, wherein if the driver's margin level is below a predetermined level, the speaking speed of the voice guidance is slower than when it is above the predetermined level.

5. The route guidance method according to claim 1, wherein when the driver's margin level is below a predetermined level, the message spoken in the voice guidance uses a message with fewer characters per sentence compared to when it is above the predetermined level.

6. The route guidance method according to claim 1, wherein if the driver's level of safety is below a predetermined level, the content spoken by the voice guidance is limited to guidance on the direction of travel.

7. Based on the driver's state, the driver's attention state is estimated. External environment information, which is information about the external environment surrounding the vehicle, and vehicle information, which is information about the vehicle's status, are detected. Based on the detected external environment information and vehicle information, the required attention state, which is the state of attention required of the driver in relation to the external environment, is estimated. The route guidance method according to claim 1, which determines the driver's margin of safety based on the driver's attention level and the required attention level.

8. The route guidance method according to claim 1, which determines the driver's margin of safety based on the driver's condition and using the judgment criteria for each driver.

9. A route guidance device that provides voice guidance about the route a vehicle should take, A driving state detection unit that detects the state of the driver operating the vehicle, A determination unit determines the driver's margin of safety during operation based on the driver's state detected by the aforementioned driving state detection unit, A route guidance device comprising: a control unit that controls the voice guidance based on the driver's margin of safety determined by the determination unit;

Images