Audio guide presentation device and audio guide presentation method

Abstract

To provide a voice guide presentation apparatus.SOLUTION: An apparatus includes: a voice output unit for outputting a voice guide or an audible question to a driver of a vehicle; a voice input unit to which answer voice from the driver is input after outputting of the audible question; and a state estimation unit for computing an amount of time from outputting of the audible question to inputting of answer voice, and estimating that the driver is in a driving concentrated state where the driver is concentrated in driving of the vehicle when the time from the outputting of the audible question to inputting of the answer voice exceeds a given value, and estimating that the driver is not in a driving concentrated state where the driver is concentrated in driving of the vehicle when the time from the outputting of the audible question to inputting of the answer voice does not exceed the given value. In a case where the state estimation unit estimates that the driver is in a driving concentrated state, the voice output unit inhibits output of voice guide to the driver.SELECTED DRAWING: Figure 13

Description

[Technical field]

[0001] The present invention relates to a voice guidance providing device and a voice guidance providing method capable of providing dialogue or voice information to a driver of a vehicle at an appropriate timing. [Background technology]

[0002] It has been known for some time that there are situations in which a driver wants to concentrate on operations related to driving behavior while using a voice user interface, and techniques for detecting such situations are known. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2005-147625 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned conventional technology was not able to automatically detect a state of concentration on driving in which the driver feels like concentrating on driving, temporarily suspend the voice output of the response sentence of the voice user interface, or automatically detect when the state of concentration on driving has been unlocked.

[0005] The present invention has been made in consideration of the problems inherent in the conventional technology. An object of the present invention is to provide a voice guide presenting device and the like that can automatically detect a state of concentration on driving in which the driver wants to concentrate on driving, temporarily suspend the response voice output of the voice UI, and automatically detect that the state of concentration on driving has been unlocked. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems, an audio guidance presentation device according to one embodiment of the present invention includes an audio output unit that outputs audio guidance or a voice follow-up question in response to a question from the driver of a vehicle, and an audio input unit that inputs voice from the driver after outputting the voice follow-up question, and calculates the time from when the voice question is output to when the voice response is input, and if the calculated time exceeds a predetermined value, it presumes that the driver is not in a state of concentrating on driving and temporarily suspends audio output, and if the calculated time does not exceed the predetermined value, it presumes that the driver is not in a state of concentrating on driving and continues audio output.

[0007] In order to solve the above-mentioned problems, a voice guidance presentation method according to another aspect of the present invention includes a voice output step of outputting a voice guide or a voice follow-up question in response to a question from the driver of a vehicle, and a voice input step of inputting voice from the driver after outputting the voice follow-up question, and calculates the time from when the voice question is output to when the answer voice is input, and if the calculated time exceeds a predetermined value, it is presumed that the driver is in a state of concentrating on driving and the voice output is temporarily suspended, and if the calculated time does not exceed the predetermined value, it is presumed that the driver is not in a state of concentrating on driving and the voice output is continued. Effect of the Invention

[0008] According to the present invention, it is possible to automatically detect a state of concentration on driving in which the driver feels like concentrating on driving, temporarily suspend the response voice output of the voice UI, and automatically detect when the state of concentration on driving has been unlocked. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic diagram for explaining an outline of the operation of a voice guide presenting device according to a plurality of embodiments. [Diagram 2] FIG. 2 is a block diagram showing an example of the configuration of a voice guide providing device according to a plurality of embodiments. [Diagram 3] FIG. 3 is a flowchart illustrating an example of the operation of the voice guide providing device based on the machine learning model. [Figure 4] FIG. 4 is a flowchart showing an example of the operation of the audio guide providing device based on the exercise behavior model. [Diagram 5] FIG. 5 is a flowchart illustrating an example of the operation of the audio guide providing device based on the machine learning model and the exercise behavior model. [Figure 6] FIG. 6 is a diagram showing example parameters as input signals that can be input to the machine learning model and the motor behavior model. [Figure 7] FIG. 7 is a schematic diagram showing an example of a method for inputting the parameters of FIG. [Figure 8] FIG. 8 is a diagram showing an example of a scene related to an SCD. [Figure 9] FIG. 9 is a schematic diagram showing an example of an SCD and a non-SCD scene. [Figure 10] FIG. 10 is a schematic diagram showing an example of the configuration of a voice guide providing device. [Figure 11] FIG. 11 is a schematic diagram showing an example of a configuration in which a machine learning model and an exercise behavior model of a voice guide providing device are trained. [Figure 12] FIG. 12 is a schematic diagram showing an example of a method for processing parameters as input signals that can be input to the machine learning model and the motor behavior model. [Figure 13] FIG. 13 is a schematic diagram showing an example of a specific configuration of a voice guide providing device. [Figure 14] FIG. 14 is a schematic diagram showing an example of the driving concentration determination unit 132 of FIG. [Figure 15] FIG. 15 is a schematic diagram showing an example of the model unit 140 of FIG. [Figure 16] FIG. 16 is a schematic diagram showing an example of the audio information unit 110 of FIG. [Figure 17] FIG. 17 is a schematic diagram showing an example of the information acquisition unit 120 in FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] An example of a voice guide presentation device and a voice guide presentation method according to the present embodiment will be described in detail below with reference to the drawings. Note that the embodiments described below are comprehensive or specific examples. The numerical values, shapes, materials, components, installation positions and connection forms of the components, steps, and order of steps shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components that are not described in an independent claim showing the highest concept will be described as optional components. Furthermore, the dimensional ratios in the drawings are exaggerated for the convenience of explanation and may differ from the actual ratios.

[0011] Furthermore, the following embodiments and their modified examples may include similar components, and the similar components will be given the same reference numerals and redundant explanations will be omitted.

[0012] (Outline of audio guide presentation device and audio guide presentation method) The voice guidance presentation device of the embodiment stops voice questions when the driver is in a Scene of Concentration on Driving (SCD) and continues voice output after the driver is no longer in SCD.

[0013] For example, as shown in Fig. 1, even if the driver starts a dialogue in the dialogue-enabled scene S1, no questions are asked to the driver in the guide-disallowed scene S2, which is an SCD. Then, in the dialogue-enabled scene S3, the dialogue is resumed and a response from the driver is awaited. The resumed dialogue can be resumed from the voice output in the immediately preceding dialogue-enabled scene S1.

[0014] Fig. 2 is a schematic diagram showing a minimum configuration of a voice guidance providing device 100 according to an embodiment. In the upper part of Fig. 2, an ASR (automatic speech recognition device) 10 can recognize that a driver's utterance has been made by referring to a database 11 of speech models. An NLU (natural language understanding device) 20 understands the content of the utterance by referring to a database 21 of speech models. The understood content of the utterance is output by an SDM (voice dialogue management device) 30 as a command signal such as a voice guide or a voice question.

[0015] 2, various sensing information from a CAN (Controller Area Network), which is an in-vehicle network, and image information from a drive recorder are collected by a data collector 40 and input to a model 51, such as a machine learning model. Based on the input information, a guide permission device 52 determines whether to permit or prohibit audio output, and SDM 30 prohibits or releases audio output.

[0016] Fig. 3 is a diagram illustrating an example of operation when a machine learning model is used in the above-described configuration, using a flowchart. In step S301, information about driving is acquired by a drive recorder or the like described in the lower part of Fig. 2. In addition, in step S302, information about driving including various sensor information is acquired by a CAN described in the lower part of Fig. 2. In addition, in step S303, a voice signal described in the upper part of Fig. 2, particularly a voice signal of the driver, is acquired.

[0017] In step S304, the information acquired in steps S301, S302, and S303 is input to a machine learning model including information already learned, and output information from the machine learning model is referenced. The output information from the machine learning model may be information indicating whether or not there is SCD, or information indicating the likelihood of SCD.

[0018] In step S305, it is determined whether or not it is possible to present audio to the driver based on the output information from the machine learning model. If it is possible to present audio to the driver (step S305: YES), the process proceeds to step S307. If it is not possible to present audio to the driver (step S305: NO), the process proceeds to step S306.

[0019] In step S306, the SDM 30 in the upper part of Fig. 2 does not output voice even if a guidance presentation signal is input. In step S307, the SDM 30 in the upper part of Fig. 2 outputs voice when a guidance presentation signal is input.

[0020] Fig. 4 is a diagram illustrating an example of operation when the exercise behavior model is used in the above-mentioned configuration, using a flowchart. In step S401, information about driving is acquired by a drive recorder or the like described in the lower part of Fig. 2. In addition, in step S402, information about driving including various sensor information is acquired by a CAN described in the lower part of Fig. 2. In addition, in step S403, a voice signal described in the upper part of Fig. 2, particularly a voice signal of the driver, is acquired.

[0021] In step S404, the information acquired in steps S401, S402, and S403 is input to an exercise behavior model including information already learned, and output information from the exercise behavior model is referred to. The output information from the exercise behavior model may include information indicating whether or not the subject is SCD.

[0022] In step S405, it is determined whether or not it is possible to present audio to the driver based on the output information from the motor behavior model. If it is possible to present audio to the driver (step S405: YES), the process proceeds to step S307. If it is not possible to present audio to the driver (step S405: NO), the process proceeds to step S406.

[0023] In step S406, the SDM 30 in the upper part of Fig. 2 does not output voice even if a guidance presentation signal is input. In step S407, the SDM 30 in the upper part of Fig. 2 outputs voice when a guidance presentation signal is input.

[0024] 5 is a flowchart showing an example of an operation for re-learning a case other than SCD in a machine learning model and a motor behavior model that have been learned once. In step S501, information about driving is acquired by a drive recorder or the like described in the lower part of FIG. 2. In addition, in step S502, information about driving including various sensor information is acquired by a CAN described in the lower part of FIG. 2. In addition, in step S503, a voice signal described in the upper part of FIG. 2, particularly a voice signal of the driver, is acquired.

[0025] In step S504, the information acquired in steps S501, S502, and S503 is input to the machine learning model and the motor behavior model including the information already learned. In this case, if there is a voice output from the driver in step S503, the machine learning model and the motor behavior model are trained to determine that a signal group model of the same pattern is not SCD.

[0026] FIG. 6 illustrates information that can be input to the machine learning model and the motor behavior model. In the above description, CAN signal information was used as input signal 1, image signal information was used as input signal 2, and the driver's voice information in the voice signal information was used as input signal 3. However, since the microphone that inputs the driver's voice information is always waiting for input, it is possible to use other people's voice signal information as input signal 3 to determine SCD. In addition, it is also possible to input environmental sound from the microphone as signal information as input signal 4. Furthermore, information that can be input to the machine learning model and the motor behavior model and used to determine SCD includes the information described in input signals 5 to 10.

[0027] FIG. 7 is a schematic diagram showing an example of a sampling method of the input signal shown in FIG. 6. The sampling of the input signal can be set at any time interval determined by the voice guide presentation device 100. In FIG. 7, 10 Hz is selected as the sampling period. In FIG. 7, the vehicle speed, steering angle, acceleration, and brake depression angle are illustrated as examples of the input signal. Sampling is started at an appropriate timing of scene 1, and the next sampling period is started in the middle of the sampling period, so that the sampling periods are overlapped in scene 1. For example, the overlapping period is half the sampling period in the example of FIG. 7, but the voice guide presentation device 100 can set the overlapping period to any interval. The input information may be a sampled instantaneous value, a differential value at the moment of sampling, a second differential value at the moment of sampling, or a combination thereof. The input information may be expressed in the form of a matrix.

[0028] FIG. 8 specifically illustrates scenes related to SCD. Scene A is a scene where the positional relationship with other vehicles changes or may change. In particular, when the vehicle is following behind another vehicle, it is determined to be SCD. In addition, when the vehicle brakes, including when the vehicle is following, it is determined to be SCD. Scene B may be determined to be SCD, for example, when the road width narrows or when the legal speed limit is exceeded. In addition, it may be determined to be SCD when the vehicle gradually moves to the right or left side of the lane. In addition, scene C may be determined to be SCD when there is a signboard, a store that is being searched for using a navigation system, a large stuffed animal, etc. Furthermore, as scene D, it may be determined to be SCD when a speed limit sign, a no entry sign, a stop sign, a sign related to animals such as watch out for animals, etc. are detected, or when such information is displayed on the vehicle's display. Furthermore, as scene E, it may be determined to be SCD when there is a change in the driver's driving operation, when turning left or right, etc. Furthermore, as scene F, when the distance between the vehicle in front of the vehicle increases, when the vehicle in front of or behind the vehicle moves to the right or left side of the lane, or when the vehicle in front of or behind the vehicle crosses the lane, it may be determined to be SCD.

[0029] FIG. 9 is a schematic diagram showing cases of SCD and cases of not SCD (Not SCD) based on the driver's conversation response time. If the driver's conversation response time to question Q1 is 0.5 seconds, it is not determined to be SCD. In addition, it is learned that there is no SCD when the speed, steering angle, acceleration, and brake states are the same pattern. In addition, it is not determined to be SCD when the driver's conversation response time to question Q2 is 0.7 seconds. In addition, it is learned that there is no SCD when the speed, steering angle, acceleration, and brake states are the same pattern. However, in response to question Q3, if the driver's conversation response time is 5 seconds, it is determined to be SCD. In addition, it is learned that there is SCD when the speed, steering angle, acceleration, and brake states are the same pattern.

[0030] 10 is a block diagram showing a schematic overview of an example of the configuration of the voice guide providing device 100 of this embodiment. Details will be described with reference to FIGS.

[0031] Fig. 11 is a block diagram showing an outline of a configuration example in which learning is performed using a machine learning model and an exercise behavior model in the voice guide providing device 100 of this embodiment. Details will be described with reference to Figs. 13 to 17.

[0032] FIG. 12 is a diagram showing an example of processing an input signal when the input signal shown in FIG. 6 is processed and input to a machine learning model and a motor behavior model. The frame summation shows a method of adding the sum of input information in a sampling period to input data. The standard deviation is a method of processing input information to detect the variation of an operation. The maximum-minimum difference is a method of processing input information to detect the maximum amount of change in an operation. The kurtosis is a method of processing input information to detect whether or not an operation accompanied by a momentary large change has occurred. The kurtosis of a frequency is a method of processing input information to detect whether or not a periodic operation has occurred. The gradient summation is a method of processing input information to detect the total amount of an operation change. The gradient maximum-minimum difference is a method of processing input information to detect a fluctuation in an operation change (a large change that occurs suddenly). All or at least a part of these processed information can be input to the machine learning model and the motor behavior model.

[0033] (Details of the audio guide device) 13 is a block diagram showing an example of a specific example of the configuration of the voice guide presenting device 100. The voice guide presenting device 100 includes a voice information unit 110, an information acquisition unit 120 as a situation information acquisition unit, a control unit 130, a model unit 140, and a storage unit 150.

[0034] 16, the audio information unit 110 includes an audio input unit 111, an audio output unit 112, and an audio control unit 113. The audio control unit 113 can determine whether or not audio is to be output.

[0035] 17, the information acquisition unit 120 includes a host vehicle information acquisition unit 121, another vehicle information acquisition unit 122, a lane information acquisition unit 123, a target information acquisition unit 124, a map information acquisition unit 125, and a following information acquisition unit 126. The following information acquisition unit 126 can be configured to be capable of acquiring information relating to the input signal 8 to the input signal 10 from the input information shown in FIG.

[0036] The control unit 130 can be realized by using a microcomputer equipped with a CPU (Central Processing Unit) and the like. A computer program (audio guide presentation program) for making the microcomputer function as the control unit 130 is installed in the microcomputer and executed. As a result, the microcomputer functions as a plurality of information processing units equipped in the control unit 130. Note that in this specification, an example in which the control unit 130 is realized by software is shown, but it is of course possible to configure the control unit 130 by preparing dedicated hardware for executing each information processing. The dedicated hardware includes devices such as application specific integrated circuits (ASICs) and conventional circuit components arranged to execute the functions described in the embodiment. In addition, the plurality of information processing units included in the control unit 130 may be configured by individual hardware. Furthermore, the control unit 130 may also be used as an electronic control device used to control a moving object that is a target of moving object management.

[0037] The control unit 130 includes an information conversion unit 131 and a driving concentration determination unit 132. The information conversion unit 131 directly converts the input information shown in FIG. 6 into a first-order differential, a second-order differential, or a signal processed as shown in FIG.

[0038] The driving concentration determination unit 132 includes a state estimation unit 132a and a determination unit 132b. The state estimation unit 132a estimates whether or not the vehicle is in an SCD state, and the determination unit 132b determines whether or not the vehicle is in an SCD state.

[0039] The model unit 140 includes an exercise behavior model unit 141 and a machine learning model field 142. The exercise behavior model unit 141 and the machine learning model field 142 have been described above, so details thereof will be omitted to avoid duplication of description.

[0040] The storage unit 150 is a computer-readable storage medium. For example, the storage unit 150 may be a Read Only Memory (ROM) or an Erasable Programmable ROM (EPROM). The storage unit 150 may be an Electrically Erasable Programmable ROM (EEPROM), a Random Access Memory (RAM), a hard disk, or the like. The storage unit 150 may be called a register, a cache, a main memory (primary storage device), or the like. The storage unit 150 can store a program (program code), software module, or the like that is executable to perform the autonomous movement according to one embodiment of the present disclosure.

[0041] (Features and Effects of the Embodiments) The features and effects of the voice guide presentation device 100 and the like according to this embodiment will be described below.

[0042] The voice guidance presentation device 100 according to the first aspect of the present disclosure preferably includes a voice output unit 112 that outputs a voice guide or a voice question to the driver of the vehicle, and a voice input unit 111 that inputs a voice response from the driver after outputting the voice question. Furthermore, the voice guidance presentation device 100 preferably further includes a state estimation unit 132a that executes the following process. The state estimation unit 132a preferably calculates the time from outputting the voice question to inputting the voice response. When the time from outputting the question to inputting the voice response exceeds a predetermined value, the state estimation unit 132a preferably estimates that the driver is in a driving concentration state where the driver is concentrating on driving the vehicle. Furthermore, when the time from outputting the question to inputting the voice response does not exceed a predetermined value, the state estimation unit 132a preferably estimates that the driver is not in a driving concentration state where the driver is concentrating on driving the vehicle. When the state estimation unit 132a estimates that the driver is in a driving concentration state, the voice output unit 112 preferably prohibits output of a voice guide to the driver.

[0043] According to the above configuration, it is possible to automatically detect a state of concentration on driving in which the driver feels like concentrating on driving, temporarily suspend the response voice output of the voice UI, and automatically detect when the state of concentration on driving has been unlocked.

[0044] It is preferable that the audio output unit 112 of the audio guidance providing device 100 according to the second aspect of the present disclosure releases the prohibition on audio guidance output after a predetermined time has elapsed.

[0045] According to the above configuration, it is possible to automatically detect a state of driver concentration in which the driver wants to concentrate on driving, temporarily suspend the response voice output of the voice UI, and output the response voice output after the elapse of a time when it is estimated that the state of driver concentration has been released.

[0046] It is preferable that the audio output unit 112 of the audio guidance providing device 100 according to the third aspect of the present disclosure permits output of audio guidance to the driver based on the judgment of the state estimation unit 132a. For example, it is preferable to permit output of audio guidance to the driver when the state estimation unit 132a estimates that the driver is not in a driving concentration state after the state estimation unit 132a estimates that the driver is not in a driving concentration state after prohibiting output of audio guidance to the driver.

[0047] According to the above configuration, it is possible to automatically detect a state of concentration on driving in which the driver feels like concentrating on driving, temporarily suspend the response voice output of the voice UI, and automatically infer that the state of concentration on driving has been released, thereby resuming the response voice output.

[0048] The voice guide presenting device 100 according to the fourth aspect of the present disclosure preferably further includes an information acquiring unit 120 as a situation information acquiring unit that acquires situation information related to the vehicle situation. The state estimating unit 132a preferably inputs the situation information acquired by the information acquiring unit 120 to a trained learning model that has undergone machine learning to estimate whether or not the driver is in a state of concentrating on driving. Then, the state estimating unit 132a preferably acquires an estimation result of whether or not the driver is in a state of concentrating on driving from the learning model.

[0049] According to the above configuration, it is possible to automatically detect from a learning model whether or not the driver is in a state of concentration on driving in which he or she wants to concentrate on driving, temporarily suspend the response voice output of the voice UI, and automatically detect when the state of concentration on driving has been unlocked.

[0050] The voice guide presenting device 100 according to the fifth aspect of the present disclosure preferably further includes an information acquisition unit 120 as a situation information acquisition unit that acquires situation information related to the vehicle situation. The state estimation unit 132a preferably includes a neural network that includes situation information acquired by the information acquisition unit 120 as input data to the input layer and an estimation result of whether or not the driver is in a state of concentrating on driving as output data from the output layer. The state estimation unit 132a preferably includes a machine learning unit that trains the neural network using the actual values ​​of the input data and the output data as teacher data. The state estimation unit 132a preferably includes a load level estimation unit that inputs new input data to the neural network trained by the machine learning unit and acquires an estimation result of whether or not the driver is in a state of concentrating on driving based on new output data corresponding to the input data.

[0051] According to the above configuration, it is possible to automatically detect from a neural network whether or not the driver is in a state of concentration on driving in which the driver wants to concentrate on driving, temporarily suspend the response voice output of the voice UI, and automatically detect when the state of concentration on driving has been unlocked.

[0052] In the voice guide providing device 100 according to the sixth aspect of the present disclosure, the state of concentrating on driving is preferably a state in which it is difficult to speak or listen.

[0053] According to the above configuration, it is possible to automatically detect a state of concentration on driving in which the driver has difficulty speaking or listening, temporarily suspend the response voice output of the voice UI, and automatically detect when the state of concentration on driving has been unlocked.

[0054] The voice guide presenting device 100 according to the seventh aspect of the present disclosure preferably further includes an other vehicle information acquisition unit 122 that acquires other vehicle information related to a positional relationship between the vehicle and other vehicles around the vehicle. The state estimation unit 132a preferably estimates whether the driver is in a driving concentration state based on the other vehicle information acquired by the other vehicle information acquisition unit 122.

[0055] According to the above configuration, it is possible to automatically detect a state of concentration on driving in which the driver wants to concentrate on driving from information about other vehicles, temporarily suspend the response voice output of the voice UI, and automatically detect from information about other vehicles that the state of concentration on driving has been unlocked.

[0056] The other vehicle information in the voice guidance providing device 100 according to the eighth aspect of the present disclosure is preferably at least one of the relative position and the relative speed between the vehicle and the other vehicle.

[0057] According to the above configuration, it is possible to automatically detect the driving concentration state in which the driver wants to concentrate on driving from the other vehicle information, temporarily suspend the response voice output of the voice UI, and automatically detect from the other vehicle information that the driving concentration state has been unlocked. In addition, the other vehicle information is at least one of the relative position and the relative speed between the vehicle and the other vehicle.

[0058] The voice guide presenting device 100 according to the ninth aspect of the present disclosure preferably further includes a lane information acquiring unit 123 that acquires lane information regarding a positional relationship between a vehicle and a lane the vehicle is traveling in. The state estimating unit 132a preferably estimates whether the driver is in a state of concentrating on driving based on the lane information acquired by the lane information acquiring unit 123.

[0059] According to the above configuration, it is possible to automatically detect a state of concentration on driving, in which the driver feels like concentrating on driving, from lane information, temporarily suspend the response voice output of the voice UI, and automatically detect from the lane information that the state of concentration on driving has been unlocked.

[0060] The lane information in the voice guidance presentation device 100 according to the tenth aspect of the present disclosure is preferably at least one of the position of the vehicle in the width direction of the lane and the speed of change of the position of the vehicle in the width direction of the lane.

[0061] According to the above configuration, it is possible to automatically detect the driving concentration state in which the driver wants to concentrate on driving from the lane information, temporarily suspend the response voice output of the voice UI, and automatically detect from the lane information that the driving concentration state has been released. In addition, the lane information is at least one of the position of the vehicle in the width direction of the lane and the speed of change of the position of the vehicle in the width direction of the lane.

[0062] The voice guide presenting device 100 according to the eleventh aspect of the present disclosure preferably further includes a target information acquisition unit 124 that recognizes target information related to targets existing around the vehicle. The state estimation unit 132a preferably estimates whether the driver is in a state of concentrating on driving based on the target information acquired by the target information acquisition unit 124.

[0063] According to the above configuration, it is possible to automatically detect a state of concentration on driving in which the driver wants to concentrate on driving from target information, temporarily suspend the response voice output of the voice UI, and automatically detect from the target information that the state of concentration on driving has been unlocked.

[0064] The target information of the voice guidance providing device 100 according to the twelfth embodiment of the present disclosure is preferably at least one of a traffic light, a sign, and a road marking.

[0065] According to the above configuration, it is possible to automatically detect the driving concentration state in which the driver wants to concentrate on driving from the target information, temporarily suspend the response voice output of the voice UI, and automatically detect the unlocking of the driving concentration state from the target information. The target information includes at least one of a traffic light, a sign, and a road marking.

[0066] The voice guide presenting device 100 according to the thirteenth aspect of the present disclosure preferably further includes a behavior information acquisition unit that acquires behavior information related to the behavior of the vehicle. The state estimation unit 132a preferably estimates whether the driver is in a state of concentrating on driving based on the behavior information acquired by the behavior information acquisition unit.

[0067] According to the above configuration, it is possible to automatically detect a state of concentration on driving, in which the driver feels like concentrating on driving, from behavior information, temporarily suspend the response voice output of the voice UI, and automatically detect from the behavior information that the state of concentration on driving has been unlocked.

[0068] The behavior information in the voice guidance providing device 100 according to the fourteenth aspect of the present disclosure is preferably at least one of the acceleration, deceleration, and yaw rate of the vehicle.

[0069] According to the above configuration, it is possible to automatically detect the driver's driving concentration state, in which the driver wants to concentrate on driving, from the behavior information, temporarily suspend the response voice output of the voice UI, and automatically detect the driver's driving concentration state being released from the behavior information. The behavior information is at least one of the acceleration, deceleration, and yaw rate of the vehicle.

[0070] The voice guidance presenting method according to the fifteenth aspect of the present disclosure preferably includes a voice output step of outputting a voice guidance or a voice question to a driver of a vehicle, and a voice input step of inputting a voice response from the driver after outputting the voice question. The voice guidance presenting method preferably includes a state estimation step. In the state estimation step, it is preferable to calculate a time from outputting the voice question to inputting the voice response. In the state estimation step, it is preferable to estimate that the driver is in a driving concentration state where he / she is concentrating on driving the vehicle when the time from outputting the question to inputting the voice response exceeds a predetermined value. In the state estimation step, it is preferable to estimate that the driver is not in a driving concentration state where he / she is concentrating on driving the vehicle when the time from outputting the question to inputting the voice response does not exceed a predetermined value. In the state estimation step, it is preferable to prohibit output of a voice guidance to the driver when it is estimated that the driver is in a driving concentration state.

[0071] According to the above configuration, it is possible to automatically detect a state of concentration on driving in which the driver feels like concentrating on driving, temporarily suspend the response voice output of the voice UI, and automatically detect when the state of concentration on driving has been unlocked.

[0072] Although the present disclosure has been described in detail above, it is clear to those skilled in the art that the present disclosure is not limited to the embodiments described in the present disclosure. The present disclosure can be implemented in modified and altered forms without departing from the spirit and scope of the present disclosure as defined by the claims. Therefore, the description of the present disclosure is intended to be illustrative and does not have any limiting meaning on the present disclosure. [Explanation of symbols]

[0073] 111 Audio input section 112 Audio output section 120 Information Acquisition Department 121 Vehicle information acquisition unit 122 Other vehicle information acquisition unit 123 Lane Information Acquisition Unit 124 Target information acquisition unit 132a State Estimation Unit

Claims

1. a voice output unit that outputs a voice guide or a voice question to a driver of the vehicle; a voice input unit that inputs a voice response from the driver after outputting the voice question; a state estimation unit that calculates a time from outputting the voice question to inputting the voice of the answer, and, if the time from outputting the question to inputting the voice of the answer exceeds a predetermined value, estimates that the driver is in a state of concentration on driving the vehicle, and, if the time from outputting the question to inputting the voice of the answer does not exceed a predetermined value, estimates that the driver is not in a state of concentration on driving the vehicle, The voice output unit prohibits the output of the voice guidance to the driver after the question when the state estimation unit estimates that the driver is in the state of concentrating on driving.

2. The audio guide presenting device according to claim 1 , wherein the audio output unit releases the inhibition of the output of the audio guide after a predetermined time has elapsed.

3. The voice output unit prohibits the output of the voice guide to the driver when the state estimation unit estimates that the driver is in the driving concentration state, and then allows the output of the voice guide to the driver when the state estimation unit estimates that the driver is not in the driving concentration state.

4. A situation information acquisition unit that acquires situation information related to a situation of the vehicle, The voice guidance presentation device of claim 3, wherein the state estimation unit inputs the situation information acquired by the situation information acquisition unit into a trained learning model that has undergone machine learning to estimate whether the driver is in the state of concentrating on driving, and acquires an estimation result of whether the driver is in the state of concentrating on driving from the learning model.

5. A situation information acquisition unit that acquires situation information related to a situation of the vehicle, The state estimation unit is a neural network including the situation information acquired by the situation information acquisition unit as input data to an input layer, and including an estimation result of whether or not the driver is in the driving concentration state as output data from an output layer; a machine learning unit that trains the neural network using actual values ​​of the input data and the output data as training data; The voice guidance presentation device of claim 3, further comprising: a load level estimation unit that inputs new input data into the neural network trained by the machine learning unit, and obtains an estimation result of whether the driver is in the state of concentrating on driving based on the new output data corresponding to the input data.

6. The voice guide presenting device according to claim 1 , wherein the driving concentration state is a state in which it is difficult to speak or listen.

7. An other vehicle information acquisition unit that acquires other vehicle information relating to a positional relationship between the vehicle and other vehicles around the vehicle, The voice guidance presentation device according to any one of claims 3 to 6, wherein the state estimation unit estimates whether the driver is in the driving concentration state based on the other vehicle information acquired by the other vehicle information acquisition unit.

8. The voice guidance providing device according to claim 7 , wherein the other vehicle information is at least one of a relative position and a relative speed between the vehicle and the other vehicle.

9. A lane information acquisition unit that acquires lane information regarding a positional relationship between the vehicle and a lane in which the vehicle is traveling, The voice guidance presentation device according to any one of claims 3 to 8, wherein the state estimation unit estimates whether the driver is in the driving concentration state based on the lane information acquired by the lane information acquisition unit.

10. The voice guidance providing device according to claim 9 , wherein the lane information is at least one of a position of the vehicle in a width direction of the lane and a speed of change of the position of the vehicle in the width direction of the lane.

11. A target information acquisition unit that recognizes target information related to targets existing around the vehicle, The voice guidance presentation device according to claim 3 , wherein the state estimation unit estimates whether the driver is in the driving concentration state based on the target information acquired by the target information acquisition unit.

12. The voice guidance presenting device according to claim 11 , wherein the target information is at least one of a traffic light, a sign, and a road marking.

13. A behavior information acquisition unit that acquires behavior information related to the behavior of the vehicle, The voice guidance presentation device according to claim 3 , wherein the state estimation unit estimates whether the driver is in the driving concentration state based on the behavior information acquired by the behavior information acquisition unit.

14. The voice guidance presenting device according to claim 13 , wherein the behavior information is at least one of an acceleration, a deceleration, and a yaw rate of the vehicle.

15. a voice output step of outputting a voice guide or a voice question to a driver of the vehicle; a voice input step of inputting a voice response from the driver after outputting the voice question; a state estimating step of calculating a time from outputting the voice question to inputting the voice of the answer, and estimating that the driver is in a state of concentration on driving the vehicle when the time from outputting the question to inputting the voice of the answer exceeds a predetermined value, and estimating that the driver is not in a state of concentration on driving the vehicle when the time from outputting the question to inputting the voice of the answer does not exceed a predetermined value, A voice guidance presenting method, which prohibits output of the voice guidance to the driver after the question when it is estimated in the state estimation step that the driver is in the driving concentration state.

Images