Driving assistance device and driving assistance method

Abstract

To provide an operation support technique which makes an operator properly confirm an operation support function and then executes it.SOLUTION: An operation support device 10 comprises: an acquisition part 22 which acquires input result of an operator; an output control part 28 which outputs confirmation voice requesting acceptance of the operator according to the input of the operator to perform an operation support function; and a determination part 30 which determines execution of the operation support function according to an input of the operator responding to the confirmation voice. The output control part 28 outputs a text including a keyword as a confirmation voice. The determination part 30 does not accept approval of the operator regarding the confirmation voice before outputting the keyword.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a driving assistance technology that assists a driver in driving a vehicle. [Background technology]

[0002] Patent Document 1 discloses a driving assistance device that recognizes the content of voice information spoken by a driver, outputs information regarding the content of the processing to a meter before executing processing according to the results of the voice information recognition, and executes the processing when an approval signal approving the execution of the processing is input from an approval switch located on the steering wheel. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2020-69920 Summary of the Invention [Problem to be solved by the invention]

[0004] In the technology described in Patent Document 1, the driving assistance device outputs the details of the process to the meter to prompt the driver to confirm it, but the process is executed even if the driver presses the approval switch without looking at the meter. Therefore, it is desirable for the driver to be able to confirm the process without looking at the meter display.

[0005] An object of the present invention is to provide a driving assistance technology that allows the driver to appropriately confirm and execute a driving assistance function. [Means for solving the problem]

[0006] In order to solve the above problem, a driving assistance device according to one aspect of the present invention includes an acquisition unit that acquires an input result of a driver, and a driver input unit that executes a driving assistance function. audio A confirmation voice prompt requests driver consent in response to input A generator that generates the textThe vehicle includes an output control unit that outputs a confirmation voice, and a decision unit that decides to execute a driving assistance function in response to the driver's input in response to the confirmation voice. The generation unit reads out a text template corresponding to the driver's voice input, embeds keywords in the template, and generates text for the confirmation voice. The output control unit outputs the text in which the keyword is set as a confirmation voice, Information indicating that the keyword has been output is sent to the determination unit. The determination unit determining whether the driver's input in response to the confirmation voice has occurred after the keyword has been output based on information indicating that the keyword has been output; Accepts driver's confirmation voice before outputting the keyword First, the system accepts driver approval via a confirmation voice after outputting the keyword.

[0007] Another aspect of the present invention is a driving assistance method, in which each step is executed by a driving assistance device of a vehicle, and a driver's instruction for executing a driving assistance function is provided. audio obtaining input from the driver; audio Depending on the input a step of reading a text template, embedding a keyword in the template, and generating a confirmation voice in text to request consent from the driver; Confirmation voice By the output control section Output, Send information indicating that the keyword has been output to the decision unit. a step of acquiring a driver's input in response to the confirmation voice; and a step of executing a driving assistance function in accordance with the driver's input. By the decision unit and determining, in the determining step, The determination unit determines whether the driver's input in response to the confirmation voice is made after the keyword is output, based on the information indicating that the keyword has been output; Accepts driver's confirmation voice before outputting the keyword Accept driver approval via voice confirmation after outputting the keyword. . [Effects of the Invention]

[0008] According to the present invention, it is possible to provide a vehicle control device that allows the driver to understand the running state of the vehicle when starting tracking control. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram for explaining an overview of a driving assistance system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating a functional configuration of the driving assistance system. [Figure 3] 10 is a flowchart of a process for starting execution of a driving assistance function according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] 1 is a diagram for explaining an overview of a driving assistance system according to an embodiment. The driving assistance system according to the embodiment executes a driving assistance function that assists a driver 40 in driving. The driving assistance function includes an automatic driving function that causes a vehicle to drive autonomously, and may be, for example, cruise control, collision avoidance control, or following control. Cruise control is a control that causes a vehicle to automatically drive at a set speed.

[0011] In Fig. 1, a driving assistance system is performing cruise control while a driver 40 has his hands on a steering wheel 42. The steering wheel 42 is provided with input switches 14. The input switches 14 include a switch for moving a cursor and a switch for inputting a decision.

[0012] The driver 40 can input by voice whether or not to execute the driving assistance function and the settings of the function. For example, in Fig. 1, the driver 40 utters "Set the cruise control to 100 km / h," which is a voice command to instruct the cruise control to change the vehicle speed.

[0013] In response, the driving assistance system recognizes the driver's utterance and outputs a confirmation voice saying, "The set vehicle speed will be changed to 100 km / h. Please press the switch if you agree." This confirmation voice requests the driver's 40 consent to the driving assistance system executing the driving assistance function in accordance with the driver's 40 voice command. At this time, the display area 44a of the instrument panel 44 displays "Do you want to set the cruise control to 100 km / h?", "YES," and "NO." The cursor is positioned on "YES." If the driver 40 presses the enter switch in this state, consent will be given.

[0014] The driver 40 operates the input switch 14 to input whether or not he / she agrees to the confirmation voice. If the driver 40 agrees to the confirmation voice, the driving assistance system executes the driving assistance function. In other words, even if the driver 40 commands the execution of a driving assistance function with a voice command, the driving assistance function will not be executed unless the driver 40 responds with an approval or disapproval to the confirmation voice. This allows the driving assistance system to execute an appropriate confirmation even if it erroneously recognizes a voice command, thereby reducing the possibility of executing an erroneously recognized driving assistance function.

[0015] Since it would be cumbersome for the driver 40 to have to listen to the confirmation voice to the end before giving consent, the driving assistance system accepts input from the input switch 14 even while the confirmation voice is being output. However, if the driver 40 gives consent immediately after the confirmation voice starts, it means that he or she has not understood the content of the confirmation voice at all, and the driving assistance system's output of the confirmation voice would be pointless.

[0016] Therefore, the driving assistance system of the embodiment places keywords that the driver 40 wants to recognize in the text of the confirmation voice, and does not accept consent before outputting the keywords, but accepts consent after outputting the keywords. This allows the driving assistance system to have the driver 40 recognize important parts of the content shown in the confirmation voice and start the driving assistance function.

[0017] 2 is a diagram showing the functional configuration of the driving assistance system 1. In terms of hardware, each function of the driving assistance system 1 can be configured using circuit blocks, memory, and other LSIs, and in terms of software, it is realized by system software, application programs, etc. loaded into memory. Therefore, it will be understood by those skilled in the art that each function of the driving assistance system 1 can be realized in various forms using only hardware, only software, or a combination thereof, and is not limited to any one of them.

[0018] The driving assistance system 1 includes a driving assistance device 10, a microphone 12, an input switch 14, an on-vehicle sensor 16, a notification unit 18, and a driving device 20. The driving assistance device 10 executes driving assistance functions, particularly driving assistance functions related to autonomous driving.

[0019] The microphone 12 functions as a sound detection unit that detects sounds within the vehicle cabin. This allows the voice of the driver 40 to be detected. A plurality of microphones 12 may be provided within the vehicle cabin, and the voice of the driver 40 may be recognized based on the detection results of the plurality of microphones 12. For example, the voice of the driver 40 may be recognized based on the difference in volume between the plurality of microphones 12.

[0020] 1, the input switch 14 is provided on the steering wheel 42 and functions as an operation input unit. Instead of the input switch 14, a touch pad or a microphone may function as the operation input unit.

[0021] The on-board sensors 16 are object detection sensors and driving condition detection sensors, and are used to execute the driving assistance function. The object detection sensors include an on-board camera, millimeter-wave radar, optical radar, and sonic wave sensors, and detect objects located around the vehicle. The driving condition detection sensors include a vehicle speed sensor, a steering angle sensor, an acceleration sensor, and a brake pressure sensor, and detect the driving condition of the vehicle. The detection results of the microphone 12, the input switch 14, and the on-board sensors 16 are sent to the driving assistance device 10.

[0022] The notification unit 18 includes at least a speaker and may include a display in addition to the speaker. The notification unit 18 notifies the driver 40 of information, for example, by outputting a confirmation voice from the speaker and displaying a selection screen corresponding to the confirmation voice on the display area 44a of the instrument panel 44 shown in FIG.

[0023] The traveling device 20 has a driving means for applying a driving force to the wheels to rotate the wheels and move the vehicle forward, a steering means for steering the wheels, and a braking means for applying a braking force to the wheels. The driving means may be an engine, a motor, or a combination thereof. The traveling device 20 may be driven by the driver's operation or by automatic driving control.

[0024] The driving assistance device 10 includes an acquisition unit 22, a recognition processing unit 24, a generation unit 26, an output control unit 28, a determination unit 30, and a driving control unit 32. The acquisition unit 22 acquires input results from the driver 40 via the microphone 12 and the input switch 14, and acquires detection results from the in-vehicle sensor 16.

[0025] The recognition processing unit 24 recognizes the content of the speech of the driver 40 based on the detection result of the microphone 12. The recognition processing unit 24 may recognize the content from the speech of the driver 40 using a neural network method, for example, a deep learning method. If the speech of the driver 40 is a voice command to execute a driving assistance function, the recognition processing unit 24 sends the type and content of the voice command to the generation unit 26. In particular, if the voice command is content to execute a driving assistance function of automatic driving control, it is determined that a confirmation voice should be output. The confirmation voice is output in response to a voice command from the driver, and does not have to be output in response to a voice command from a passenger other than the driver.

[0026] The generator 26 generates a confirmation voice in text form in response to the driver's 40 voice command. The generation unit 26 reads out a text template corresponding to the voice command, embeds the keyword in the template, and generates text for a confirmation voice. The text for the confirmation voice requests the driver 40 to give consent for the execution of the driving assistance function. The generation unit 26 also generates text to be displayed in the display area 44a. The generation unit 26 sends the generated text to the output control unit 28 together with information indicating the position of the keyword.

[0027] An example of a confirmation voice text template is shown below. (1) A template for voice commands to change the cruise control speed "Change the set speed to A kilometers. If you agree, please input the switch." (2) A template for voice commands to change the following distance for tracking control "Change the distance to B. If you agree, press the switch." (3) A template for a voice command to turn on (or off) the cruise control function. "C function will be turned on. Please input the switch if you wish."

[0028] The keyword set by the generation unit 26 is "A kilo" in the above (1), an indicator indicating the "B" stage in the above (2), and "C function on" in the above (3). The generation unit 26 applies the recognition result of the recognition processing unit 24 to the template keyword. The keyword is not limited to one word, and may be composed of multiple words as shown in the above (3).

[0029] As shown in (1) to (3) above, the keyword set by the generation unit 26 is placed in the front part of the text that constitutes the confirmation voice. The front part of the text refers to the part that is preceding in the order of the words that constitute the text. The keyword may also be placed at the beginning of the text that constitutes the confirmation voice. By placing the keyword in the front part of the text in this way, the timing at which the driver 40 accepts consent can be accelerated, and the driving assistance device 10 can notify the driver 40 of the important parts of the confirmation voice. The keyword may be set to start being output at least one second after the confirmation voice starts being output. The keyword may also be set to start being output as the first or second word of the confirmation voice.

[0030] In the above (1), since the instruction is to change the vehicle speed, the vehicle speed is set as the keyword, and in the above (2), since the instruction is to change the inter-vehicle distance, an indicator of the inter-vehicle distance is set as the keyword. Furthermore, in the above (3), since the instruction is to turn the driving control function on / off, whether the function is on / off is set as the keyword. In this way, the keyword set by the generation unit 26 indicates changes in driving control when the driving assistance function is executed in response to an instruction from the driver 40. The changes in driving control indicate the control parts that are different before and after the execution of the driving assistance function in response to the voice command. This makes it possible to communicate the changes in driving control to the driver 40.

[0031] The output control unit 28 receives the text of the confirmation voice from the generation unit 26 and outputs the text as voice from the notification unit 18. The output control unit 28 also receives the text of the selection display corresponding to the confirmation voice from the generation unit 26 and outputs it as an image from the display area 44a. As shown in FIG. 1, the cursor is initially positioned on "YES," which indicates approval, so the driver 40 can approve by simply pressing the decision switch. If the driver 40 rejects the request, he or she presses the movement switch to move the cursor to "NO" and then presses the decision switch.

[0032] The output control unit 28 outputs a confirmation voice in response to a voice input from the driver 40 to execute the driving assistance function. The confirmation voice requests consent from the driver 40 and is set with a keyword.

[0033] The driver 40 responds to the confirmation voice by operating the input switch 14. This allows the driver 40 to confirm whether the driving assistance device 10 has erroneously recognized the voice input of the driver 40. The output control unit 28 determines that a keyword has been output based on the information indicating the position of the keyword, and sends information indicating that the keyword has been output to the determination unit 30.

[0034] The decision unit 30 decides to execute the driving assistance function according to the input of the driver 40 in response to the confirmation voice. The decision unit 30 does not accept approval from the driver 40 in response to the confirmation voice before outputting a keyword. This makes it possible to avoid accepting approval from the driver 40 before the driver 40 understands the keyword. The decision unit 30 determines whether a keyword has been output based on the output result of the output control unit 28.

[0035] The decision unit 30 receives an input from the driver 40 in response to the confirmation voice after outputting the keyword, and decides to execute the driving assistance function in accordance with the received input. This allows the driver 40 to acknowledge the confirmation voice even while the confirmation voice is being output. "Before outputting the keyword" may be before the output of the keyword is completed, and "after outputting the keyword" may be after the output of the keyword is completed.

[0036] The input of denial in response to the confirmation voice may be accepted after the output of the keyword, as with the input of approval, but may also be accepted regardless of the output of the keyword. In other words, the timing at which the input of approval and the input of denial in response to the confirmation voice can be accepted may differ, and the input of denial may be accepted before the input of approval.

[0037] The driving control unit 32 executes the driving assistance function upon receiving an instruction to execute the driving assistance function from the decision unit 30. In particular, the driving control unit 32 executes a driving control function related to autonomous driving. The driving control unit 32 generates command values ​​for controlling the driving device 20 based on the detection results of the on-board sensors 16 and sends them to the driving device 20. The driving device 20 drives in accordance with the received command values. This allows the driving assistance device 10 to drive the vehicle autonomously.

[0038] 3 is a flowchart of a process for starting execution of a driving assistance function according to an embodiment. The acquisition unit 22 acquires sounds from within the vehicle cabin using the microphone 12 (S10). The recognition processing unit 24 extracts the voice of the driver 40 from the sounds from within the vehicle cabin and recognizes the voice of the driver 40 (S12). The recognition processing unit 24 determines whether the recognized voice of the driver 40 is a predetermined voice command (S14).

[0039] If the voice of the driver 40 is not a predetermined voice command (N in S14), this process ends. If the voice of the driver 40 is a predetermined voice command (Y in S14), the generation unit 26 generates display text and text for a confirmation voice, and the output control unit 28 displays a selection image requesting input from the driver 40 in the display area 44a (S16). The output control unit 28 also starts outputting the confirmation voice generated by the recognition processing unit 24 (S18). The generation unit 26 sets a keyword in the text of this confirmation voice.

[0040] The acquisition unit 22 acquires the input result of the input switch 14, and the decision unit 30 determines whether the driver 40 has made an input in response to the confirmation voice (S20). If there has been no input by the driver 40 (N in S20), it is determined whether a predetermined time has elapsed since the start or end of output of the confirmation voice (S28).

[0041] If a predetermined time has not elapsed since the start or end of the confirmation voice output (N in S28), the decision unit 30 determines whether there is an input from the driver (S20). If a predetermined time has elapsed since the start or end of the confirmation voice output (Y in S28), the voice command is not permitted and the process ends (S30).

[0042] If there is an input from the driver 40 (Y in S20), the decision unit 30 determines whether or not the keyword has been output based on the output result of the output control unit 28 (S22). If the keyword has not yet been output (N in S22), the input is not accepted, and the decision unit 30 determines whether or not there is a next driver input (S20). In other words, if the driver presses the input switch 14 immediately after the confirmation voice is output, the input is not accepted. This makes it possible to avoid the driving assistance device 10 accepting the driver 40's consent before notifying the driver 40 of the keyword.

[0043] If the keyword has already been output (Y in S22), the decision unit 30 accepts input from the driver 40 (S24), and if approval is input, decides to execute the driving assistance function corresponding to the voice command. The driving control unit 32 executes the driving assistance function in accordance with the decision of the decision unit 30 (S26). As a result, if the driver repeatedly taps the input switch 14 immediately after the confirmation voice is output, input will not be accepted until at least after the keyword is output, and input will be accepted after the keyword is output. Note that denial input may be accepted before the keyword is output.

[0044] The present disclosure has been described above based on examples. The present disclosure is not limited to the above examples, and various modifications such as design changes may be made based on the knowledge of those skilled in the art.

[0045] In the embodiment, the driving assistance device 10 is provided with the recognition processing unit 24, but the present invention is not limited to this, and an external server device may be provided with the recognition processing unit 24. That is, the external server device may receive the driver's utterance, perform recognition processing, and send the recognition result back to the driving assistance device 10.

[0046] In addition, in the embodiment, the driver 40 presses the input switch 14 to input whether or not he / she approves the confirmation voice, but the present invention is not limited to this. For example, the driver 40 may input whether or not he / she approves the confirmation voice by speaking. [Explanation of symbols]

[0047] 1 Driving assistance system, 10 Driving assistance device, 12 Microphone, 14 Input switch, 16 In-vehicle sensor, 18 Notification unit, 20 Driving device, 22 Acquisition unit, 24 Recognition processing unit, 26 Generation unit, 28 Output control unit, 30 Decision unit, 32 Driving control unit, 40 Driver, 42 Steering wheel, 44 Instrument panel, 44a Display area.

Claims

1. an acquisition unit that acquires the driver's input results; a generation unit that generates a confirmation voice in text form requesting consent from the driver in response to a voice input from the driver to execute the driving assistance function; an output control unit that outputs the generated confirmation voice; a decision unit that decides to execute a driving assistance function in response to a driver's input in response to the confirmation voice, the generation unit reads a text template corresponding to the driver's voice input, embeds a keyword in the template, and generates text for the confirmation voice; the output control unit outputs the text in which the keyword is set as a confirmation voice, and sends information indicating that the keyword has been output to the determination unit; The decision unit determines whether the driver's input in response to the confirmation voice occurs after the keyword is output based on information indicating that the keyword has been output, and does not accept the driver's approval of the confirmation voice before the keyword is output, but accepts the driver's approval of the confirmation voice after the keyword is output.

2. the keyword is placed in front of the text constituting the confirmation voice; The driving assistance device according to claim 1 , wherein the determination unit receives an input from the driver in response to the confirmation voice after outputting the keyword, and determines whether to execute a driving assistance function in accordance with the received input.

3. 3. The driving assistance device according to claim 1, wherein the keyword indicates a change in control when the driving assistance function is executed in response to an input from the driver.

4. The driving assistance function is a driving control function related to automated driving, 3. The driving assistance device according to claim 1, wherein the output control unit outputs the confirmation voice in response to a voice input by the driver for executing a driving control function related to automatic driving.

5. A driving assistance method in which each step is executed by a driving assistance device of a vehicle, obtaining a driver's voice input for executing a driving assistance function; a step of reading a text template in response to a voice input from the driver, embedding a keyword in the template, and generating a confirmation voice in text form to request consent from the driver; outputting the generated confirmation voice and sending information indicating that the keyword has been output to a determination unit; obtaining a driver's input in response to the confirmation voice; determining by the determination unit whether to execute a driving assistance function in response to an input from a driver; A driving assistance method characterized in that, in the determining step, the determining unit determines whether the driver's input in response to the confirmation voice occurs after the keyword is output based on information indicating that the keyword has been output, and does not accept the driver's approval of the confirmation voice before the keyword is output, but accepts the driver's approval of the confirmation voice after the keyword is output.

Images