Method for controlling an at least partially autonomous vehicle via speech input

Abstract

The invention relates to a method with which a driver (100) can control an at least partially autonomous vehicle via speech input. The method comprises the following steps: capturing the speech input of the driver (100) via a microphone (303); identifying a potential driving instruction from the speech input of the driver (100) via a speech recognition unit (140); comparing the potential driving instruction with a list of possible driving instructions, said list of possible driving instructions being a function of possible driving instructions of an at least semi-autonomous vehicle control unit (190); and, if the potential driving instruction is contained in the list of possible driving instructions: outputting an action instruction to the driver (100) via a loudspeaker (103), said action instruction being a function of the driving instruction; checking, via the verification unit (150), a reaction of the driver (100) to the action instruction being output; and, if the reaction of the driver (100) is assessed as correct by the verification unit (150), using the potential driving instruction as an actuator instruction for controlling the actuators (200) of the vehicle via the at least partially autonomous vehicle control unit (190).

Description

[0001] Method for controlling at least a partially autonomous vehicle using voice input

[0002] Field of invention

[0003] The invention relates to a method for controlling an at least partially autonomous vehicle by a driver, in particular by means of voice input. The invention further relates to a device, a vehicle, a use, and a non-volatile, computer-readable storage medium.

[0004] background

[0005] Current technology includes voice input systems in vehicles that allow for the control of a variety of non-safety-critical vehicle functions. However, it would be a significant relief for the driver if a partially autonomous vehicle could also be controlled via voice input for safety-critical functions. This requires measures to verify the voice input.

[0006] Summary

[0007] The object of the invention is to provide a method by which a driver's voice input can be verified. This object is achieved by the subject matter of the independent patent claims.

[0008] Further developments of the invention are described in the dependent claims and the following description.

[0009] One aspect concerns a method for controlling an at least partially autonomous vehicle by a driver using speech input, comprising the steps of: capturing, by means of a microphone, the driver's speech input; determining, by means of a speech recognition unit, a potential driving instruction from the driver's speech input; comparing the potential driving instruction with a list of possible driving instructions, wherein the list of possible driving instructions is a function of possible driving instructions of an at least partially autonomous vehicle control unit;

[0010] IDNR: 14041 / V: 06-3.13 / B: If the potential driving instruction is included in the list of possible driving instructions: issue, by means of a loudspeaker, an instruction to the driver, the instruction being a function of the driving instruction; check, by means of the verification unit, the driver's reaction to the output of the instruction; if, by means of the verification unit, the driver's reaction is judged to be correct, use the potential driving instruction as an actuator instruction to control the vehicle's actuators by means of the at least semi-autonomous vehicle control unit.

[0011] A vehicle can be a road-bound vehicle, such as a passenger car, a truck, or a special-purpose vehicle. The vehicle can also be, for example, a boat. The vehicle is at least a partially autonomous vehicle, i.e., a vehicle of at least Level 1 according to SAE standard J3016. The language input is natural language input, meaning that in at least some embodiments, it may be permissible for not just one term to be understood as an instruction to the vehicle's actuators—or to the vehicle control unit, which controls the actuators at least partially autonomously—but rather for a multitude of terms to be permissible. For example, instead of just "slow down," terms such as "slower," "not so fast," etc., can also be considered actuator instructions, i.e., potential driving instructions.Even complex driving maneuvers, such as overtaking a vehicle in front or stopping the vehicle in the next parking space, can be considered a possible actuator instruction to the at least partially autonomous vehicle control unit in this context.

[0012] The driver's voice input is captured using a microphone. The microphone can be, for example, a dynamic microphone, condenser microphone, electret condenser microphone, piezoelectric microphone, and / or another type of microphone. The microphone can be configured as a multiple microphone array, such as a microphone field or array. The microphone may incorporate measures to improve voice quality, particularly through the use of multiple microphones. A microphone is considered particularly suitable for this process if it is capable of clearly capturing the driver's voice and also sound from a loudspeaker, as described below. The speech recognition unit can use the signal from the microphone to determine whether the voice input is a potential driving instruction directed to the vehicle's actuators.The speech recognition unit can not only be capable of recognizing words from the spoken input, but it can also include a list or database containing permissible actuator instructions and their respective meanings. Using the example above, the speech recognition unit recognizes, for instance, the words "slow down," "slower," and "not so fast" (a) as permissible, i.e., as potential driving instructions, and (b) assigns the single meaning "slow down" to all of these words. Other examples of actuator instructions might be: "left," "right," "overtake," "accelerate," etc. If the words are not recognized as actuator instructions, the spoken input can, for example, be ignored. In any case, words that are not recognized as actuator instructions or as potential driving instructions are not forwarded to the vehicle's actuators (or to the vehicle control unit).The vehicle's actuators can include, for example, the drivetrain, brakes, and steering.

[0013] If the voice input is recognized as a potential driving instruction, the potential driving instruction is compared with a list of possible driving instructions. This list is a function of the possible driving instructions of an at least partially autonomous vehicle control unit. In other words, the list of possible driving instructions contains those driving instructions that the vehicle control unit can implement. Comparing the potential driving instruction with the list of possible driving instructions can be particularly easy because the speech recognition unit can output a standardized "meaning" (in the example above, "slow down"). This can be easily compared with similarly standardized terms from the list of possible driving instructions. The list of possible driving instructions may depend on the autonomy level of the vehicle control unit.For example, a potential driving instruction "slow down" can be implemented by most vehicle control units and is therefore included in their list of possible driving instructions. Conversely, a potential driving instruction "overtake" can only be implemented by vehicle control units with a higher level of autonomy and is therefore only included in their list of possible driving instructions, but not in the lists of other vehicle control units. If a potential driving instruction is not included in the list of possible driving instructions, this driving instruction, or the corresponding voice input, is ignored. An error message may be displayed.

[0014] In some implementations, the list of possible driving instructions can be attributed. These attributes can indicate that the list of possible driving instructions depends not only on the autonomy level of the vehicle control unit but also on the current situation of the vehicle. An example of a dependency on the current situation of the vehicle is that the potential driving instruction "overtake" can only be implemented (or may only be implemented, according to legal regulations) on a highway by vehicle control units of a certain autonomy level, but not on other roads. If a potential driving instruction is not included in the attributed list of possible driving instructions and / or cannot be implemented in the current situation of the vehicle, this driving instruction, or the corresponding voice input, is ignored. An error message may be displayed.

[0015] If voice input is included in the list of possible driving instructions, an instruction can be given to the driver. The instruction could be, for example, "Confirm the deceleration by pressing button A" or "Confirm the deceleration by nodding your head." The output can be via a loudspeaker. The loudspeaker can be, for example, a dynamic, electrostatic, piezoelectric loudspeaker, and / or another type of transducer. The loudspeaker can consist of one or more speakers, including separate speakers, each reproducing a specific frequency range. For example, a piezoelectric loudspeaker can be configured to reproduce a frequency range above 22 kHz, and a dynamic loudspeaker can be configured to reproduce a frequency range below 22 kHz.The loudspeaker can preferably be positioned near the driver's head or ears, e.g. in a headrest of the vehicle.

[0016] The driver can react to the initial instruction. A reaction to the examples above could be, for instance, pressing button A or nodding the head. The verification unit can assess this reaction as correct. To do this, the verification unit can query one or more sensors. For example, pressing button A can be detected using a haptic sensor – such as a push button. Or, a head nod can be detected using an interior camera designed to capture (at least) this predefined movement of the driver, together with an image recognition unit designed to evaluate the movement captured by the interior camera, so that the verification unit recognizes the driver's head nod as a "head nod".If a different driver response, or no response at all, is observed, it will not be considered correct. A time window can be defined within which the driver's response will be considered correct. This time window could, for example, encompass a few seconds after the initial instruction is issued. If the correct driver response is not observed within this time window, it will also not be considered correct. To ensure that a particular response does not become the norm, the initial instruction can be varied from case to case.

[0017] If the driver's reaction is judged to be correct by the verification unit, the actuator instruction can be used to control the vehicle's actuators or forwarded to the at least semi-autonomous vehicle control unit.

[0018] This provides a method for verifying driver voice input. This allows verification—and in at least some cases, confirmation—that the driver's voice input was indeed intended as an actuator command. Furthermore, the capabilities of the at least partially autonomous vehicle control unit can be considered during the voice input verification process. This can contribute to a particularly consistent user experience for the vehicle's human-machine interface. Additionally, this can reduce the workload on the vehicle control unit, as it is relieved of requests or driving instructions that it cannot execute anyway.

[0019] By checking the driver's voice input, potential misunderstandings—such as instructions misinterpreted by the speech recognition unit—can be significantly reduced or even eliminated. Furthermore, errors caused by incorrect language settings (e.g., Japanese instead of German) can be minimized. Additionally, the risk of the driver misunderstanding instructions, for example, due to hearing impairment (e.g., after a loud concert) and / or excessive ambient noise (e.g., in a construction zone), can be reduced.

[0020] In some embodiments, the method includes the following additional step: After issuing the instruction, the driver is authenticated using a driver authentication unit. The driver authentication unit can, for example, include iris recognition. This can be implemented, for instance, using an interior camera – possibly in conjunction with a database of authorized drivers. Alternatively or additionally, driver authentication can be achieved using voice recognition. In one embodiment, if authentication is not successful, a further instruction can be issued to the driver, which can be confirmed, for example, by haptic input.

[0021] In some embodiments, the method includes the following additional step: After issuing the instruction, the driver's fitness to drive is checked using a fitness-to-drive detection unit. Fitness to drive can be assessed, for example, using microphone signals and / or image data from the interior camera. If driver fatigue is detected, the issuance and execution of the driving instruction can be prevented. Instead of the driving instruction, the driver can then be offered, for example, a route to a safe stopping point (e.g., the next rest area).

[0022] In some embodiments, the method includes the following additional step: After issuing the instruction, the driver's attention is checked using a proximity sensor. The proximity sensor can be implemented, for example, using a capacitive sensor and / or an IR sensor (IR: infrared). The proximity sensor can be located, for example, in the headrest and / or in an area of ​​the headliner where the driver's head is typically positioned. If the driver's head moves further than a predefined distance from the headrest, it can be assumed that the driver is looking down or to the side and is therefore not paying attention. Optionally, confirmation, for example, via haptic input, can be requested in this situation.

[0023] In some embodiments, the method includes the following further steps: outputting a control signal via the loudspeaker;

[0024] Capture the control signal using the microphone;

[0025] If the control signal is judged to be correct by the verification unit, the actuator instruction is used to control the vehicle's actuators.

[0026] The control signal can be, for example, an inaudible audio signal with a characteristic modulation. This audio signal could be, for example, modulated near-ultrasound, e.g., in the range between 25 and 30 kHz. The modulation could be, for example, a predefined signal sequence. If this audio signal with the characteristic modulation is detected, the verification unit can assess the control signal as correct. If this audio signal is not detected, the actuator instruction can be ignored. The audio signal might not be detected, for example, if the ratio of the sound pressure level of the audio signal to the ambient noise falls below a predefined value. In this case, the ambient noise might be so loud that the speech recognition unit malfunctions, making it impossible to control the vehicle via voice input.

[0027] In some embodiments, haptic input from the driver is the driver's reaction to the output of the instruction. Haptic input can be determined by a haptic sensor. The haptic sensor can be, for example, a button, switch, rotary knob, pressure sensor, proximity sensor, and / or a capacitive touch sensor on a screen or input field. The haptic sensor can be located, for example, on the steering wheel or in the area of ​​a center console. This can be a particularly intuitive input method because many vehicles use haptic sensors on the steering wheel or in the area of ​​the center console for numerous types of input.

[0028] In some embodiments, a predefined movement of the driver is the driver's reaction to the issuance of the instruction. The predefined movement can, for example, include a gesture such as waving or nodding.

[0029] In some embodiments, a movement of the steering wheel by the driver is the driver's reaction to the output of the instruction. The movement of the steering wheel can be interpreted as an objection by the driver to the actuator instruction, for example, if the driver wants to steer manually and does not want to transfer the steering to the at least partially autonomous vehicle control unit. The driver's reaction is therefore considered correct if the driver does not move the steering wheel.

[0030] In some embodiments, the driver's movement of a pedal is their response to the issued instruction. The pedal could be, for example, a brake or accelerator pedal, or possibly a clutch pedal. The movement of the pedal can also be interpreted as the driver's objection to the actuator instruction, for instance, if the driver intends to brake or accelerate themselves. The driver's response is therefore considered correct if the driver does not move the pedal.

[0031] One aspect concerns a device for controlling a vehicle by a driver using voice input. The device features:

[0032] An interface to a microphone designed to capture the driver's voice input;

[0033] A speech recognition unit designed to detect whether the spoken input is a potential driving instruction;

[0034] An interface to a list of possible driving instructions, wherein the list of possible driving instructions is a function of possible driving instructions of an at least partially autonomous vehicle control unit;

[0035] An interface to a loudspeaker that is set up to issue an instruction to the driver when the voice input is included in the list of possible driving instructions;

[0036] A verification unit, equipped to check the driver's response to the issuance of the instruction by means of a sensor; and

[0037] An interface to at least a semi-autonomous vehicle control unit, set up to control the vehicle's actuators based on the potential driving instruction as an actuator instruction, if the verification unit judges the driver's reaction to be correct.

[0038] The device can therefore advantageously build upon existing interfaces already present in many vehicles. This can significantly simplify and / or reduce the cost of controlling a vehicle via voice input.

[0039] In some embodiments, the loudspeaker is still configured to output a control signal, the microphone is still configured to receive the control signal from the loudspeaker, and the vehicle control unit is still configured to control the vehicle's actuators based on the actuator command. However, this requires that the verification unit evaluates the control signal from the microphone as correct. In cases where the control signal uses low-frequency ultrasound, the loudspeaker can, for example, be implemented as an additional piezoelectric transducer.

[0040] In some embodiments, the sensor is configured as at least one of the following devices: a haptic sensor configured to detect haptic input from the driver, wherein the haptic sensor is configured as a button, switch, rotary control, pressure sensor, proximity sensor, and / or as a capacitive touch sensor of a screen or input field; and / or an interior camera configured to detect a predefined movement of the driver, and an image recognition unit configured to evaluate the movement of the driver detected by the interior camera; and / or a steering wheel motion sensor configured to detect movement of a steering wheel by the driver; and / or a pedal motion sensor configured to detect movement of a pedal by the driver.

[0041] In some embodiments, the device further comprises at least one of the following: an interface to a driver authentication unit; an interface to a fitness-to-drive detection unit; and / or an interface to a proximity sensor.

[0042] One aspect concerns a vehicle with a device as described above and / or below.

[0043] One aspect concerns the use of a device as described above and / or below for the safe control of a vehicle by a driver using voice input.

[0044] One aspect concerns a non-volatile, computer-readable storage medium containing a program which, when executed on a processor of a device as described above and / or below, instructs the device to perform the steps as described above and / or below.

[0045] For further clarification, the invention is described with reference to embodiments illustrated in the figures. These embodiments are to be understood as examples only, and not as limitations.

[0046] Brief description of the characters

[0047] This shows:

[0048] Fig. 1 schematically shows a device for controlling a vehicle according to one embodiment;

[0049] Fig. 2 schematically shows a system for controlling a vehicle according to a further embodiment;

[0050] Fig. 3 shows a flowchart of a method according to one embodiment.

[0051] Detailed description of embodiments

[0052] Fig. 1 schematically shows a device for controlling a vehicle according to one embodiment. A driver 100 is shown using a microphone 303 to control the at least partially autonomous vehicle by means of voice input. The microphone 303 can be configured as a single microphone; more typically, the microphone 303 is configured as a plurality of microphones, e.g., as a microphone field or microphone array. The microphone 303 can include measures for improving speech quality, particularly by utilizing the plurality of microphones, e.g., implemented by means of a microcontroller. The microphone 303 can be a microphone that is "already present" in the vehicle (e.g., for implementing a hands-free system), so that only an interface to the microphone 303 needs to be provided, which supplies the microphone signal Ma, possibly optimized. The voice input (i.e.,The microphone signal (Ma) is sent to a speech recognition unit 140, which can recognize whether the speech input is an actuator instruction. For this purpose, the speech recognition unit 140 can not only recognize words from the speech input, but it can also include a list or database (not shown) in which permissible actuator instructions and their respective meanings are stored. The actuator instruction Ft is sent to a verification unit 150. The verification unit 150 has an interface Lf to the at least partially autonomous vehicle control unit 190. The vehicle control unit 190 includes a list of possible driving instructions that can be implemented by the vehicle control unit and / or by means of the actuators 200. The list of possible driving instructions can be compatible with the terms output by the speech recognition unit 140.

[0053] The verification unit 150 can optionally have additional interfaces. For example, Fig. 1 shows an interface to a driver authentication unit 170. The driver authentication unit 170 can, for example, include iris recognition. This can be implemented, for instance, using the interior camera 120. The driver authentication unit 170 can also include a database of authorized drivers (not shown), for example, to compare the result of the iris recognition with a driver profile. Fig. 1 also shows an interface to a fitness-to-drive assessment unit 160. Fitness to drive can be assessed, for example, using signals from the microphone 303 and / or image data from the interior camera 120. Fig. 1 further shows an interface to a proximity sensor 165. The proximity sensor 165 can, for example, be implemented using a capacitive sensor and / or an IR sensor (IR: infrared).be located in the headrest and / or in an area of ​​the headliner where the driver's head is usually located.

[0054] The verification unit 150 also has an interface Sa to a loudspeaker 103, which can be used to issue an initial instruction to the driver 100. This initial instruction is only issued if the voice input is recognized as an actuator instruction. The initial instruction could be, for example, "Confirm the actuator instruction by pressing button A" or "Confirm the actuator instruction by nodding your head." Pressing button A can be detected, for example, by a haptic input device, such as a button located on a steering wheel 110. Nodding your head can be detected by an interior camera 120, which is configured to capture the driver's movements, together with an image recognition unit (not shown) that is configured to evaluate the driver's movements captured by the interior camera and provides a signal K.In this way, the driver's head nod can be recognized as a "head nod" by the verification unit 150. The verification unit 150 can also check whether signals P are present from the pedals 112 or whether signals L are present from the steering wheel 110. The signals P can be determined by a pedal sensor 113. The signals L from the steering wheel 110 can be determined by a steering wheel sensor 114. The movement of the steering wheel 110 and / or one of the pedals 112 can be interpreted as an objection by the driver to the actuator instruction, for example, if the driver wants to steer, brake, and / or accelerate themselves and does not want to delegate these actions to at least a partially autonomous vehicle control unit.

[0055] The verification unit 150 checks the driver's 100 response to the output of the instruction using one of the aforementioned sensors. If the verification unit 150 assesses the driver's 100 response as correct, the actuator instruction is transmitted via interface Af to the at least semi-autonomous vehicle control unit 190, which controls the vehicle's actuators 200 via interface Ak.

[0056] Fig. 2 schematically shows a system for controlling a vehicle according to a further embodiment. Identical reference numerals denote identical or similar elements as in Fig. 1. In Fig. 2, a person 100 – hereinafter referred to as the “driver” 100 – is shown sitting in a driver's seat 101. The steering wheel 110 has easily accessible haptic input instruments 114 for the driver 100, which offer various input options, e.g., input buttons or input fields. The signals B from the haptic input instruments 114 and signals L from the steering wheel 110 are transmitted to a user control unit 180. The system includes an interior camera 120. This can be used to detect driver reactions to commands Fa, which may also relate to driving instructions assigned by the verification unit 150.The action prompts Fa are transmitted via a voice output 130 to one or more loudspeakers 103, which are advantageously located near the driver's head, e.g., in a headrest. The system has a microphone array comprising several microphones 303, 304, which are positioned in the vehicle to optimally capture both the driver's voice input 100 and the audio signals from the loudspeaker 103.

[0057] A sound evaluation unit 302 can be provided for processing the signals from the microphones 303 and 304. The sound evaluation unit 302 can separate the driver's voice input 100 from the audio signals of the loudspeaker 103 and can combine the microphone signals from 303 and 304, e.g., by filtering and / or delaying them, so that they amplify the voice input and the loudspeaker signals while attenuating other sound sources inside and outside the vehicle. The combined microphone signal Ma is then output to an audio output unit 130, a speech recognition unit 140, and optionally to a fitness-to-drive detection unit 160 and a driver authentication unit 170. The audio output unit 130 generates the audio signal Sa. It can also receive action prompts Fa from the verification unit 150 and, if necessary, forward them to the user control unit 180. In Fig. 2, both signal paths are labeled "Fa".Furthermore, it can generate an audio signal such that, for example, based on a characteristic modulation from the microphone signal Ma that is imperceptible to humans, it can be determined that the output acoustic signal Sa is the speech reproduction of a digital output signal Fa from the secure verification unit 150, e.g., a call to action. Additionally, the audio output unit 130 can calculate the ratio SN of a disturbing ambient noise to the desired signal Sa by comparing the microphone signal Ma with the input signal Fa.If the signal-to-noise ratio SN is too low or it cannot be confirmed that the emitted acoustic signal Sa is the speech reproduction of a digital output signal Fa of the secure verification unit 150, the forwarding of an action request Fa to the user control 180 by the audio output unit 130 can be suppressed or the output of Fa to the user control 180 can be replaced by an error message, since the action request Fa and the possibly contained driver instruction are not transmitted due to a technical malfunction (e.g. of the loudspeaker 301 or the audio output 130) or cannot be understood by the driver 100 due to excessive ambient noise.

[0058] Furthermore, the system features a speech recognition unit 140, which recognizes the driver's instructions (Ft) in the combined microphone signal (Ma). These instructions are then forwarded to the verification unit 150. Based on the recognized voice instruction from the driver (Ft), and if a clear match is possible with a list (Lf) of possible driving instructions provided by the autonomous vehicle control unit 190, the verification unit 150 derives a specific action request (Fa) corresponding to a particular driving instruction from the list (Lf). This action request (Fa) is then sent to the audio output unit 130. If this action request for the driving instruction is confirmed by the audio output unit through the evaluation of the combined microphone signals (Ma), it is forwarded to the user control unit 180.This then sends an actuator instruction Af, corresponding to the relevant driving instruction Fa, to the at least semi-autonomous vehicle control unit 190. The latter can then generate the necessary control signals Ak to execute the actuator instruction Af. Using vehicle sensors 192 (e.g., ultrasonic sensors, radar sensors, LiDAR and external camera sensors) to detect the vehicle's surroundings, the at least semi-autonomous vehicle control unit 190 controls the actuators 200, e.g., the vehicle's drive system ("power train"), brakes and steering, via control signals Ak.

[0059] Fig. 2 further shows an optional driver authentication unit 170, which can be implemented, for example, based on iris recognition with an interior camera, and / or voice recognition, and a list of drivers or driver profiles. If authentication fails, e.g., of the driver's voice via the signal Na, a haptic operating alternative can be suggested. Fig. 2 also shows an optional fitness-to-drive detection unit 160, which assesses fitness to drive based on the microphone signals Ma and interior camera image data Kd, and, for example, if fatigue is detected, prevents the issuance and execution of a driving instruction from the driver and, instead of the driver instruction, suggests a route to a safe stopping point, e.g., to the next rest area, e.g., via the signal Sh. Fig. 2 also shows an optional function of the user control 180.If the driving instruction data (Fa) is correctly recognized in the audio signal, the system can check whether the driver's head is near the headrest. If not, it can be assumed that the driver is distracted, e.g., looking at the ground. In this case, the voice input can be ignored and an error message can optionally be issued.

[0060] Fig. 3 shows a flowchart 400 for a method according to one embodiment. In step 402, a voice input from a driver 100 is captured by means of a microphone 303 (see, for example, Fig. 1). In step 404, a potential driving instruction is determined from the driver's voice input by means of a speech recognition unit 140. In step 406, the potential driving instruction is compared with a list of possible driving instructions. The list of possible driving instructions is a function of possible driving instructions of an at least partially autonomous vehicle control unit 190. If the potential driving instruction is included in the list of possible driving instructions, an action instruction is output to the driver 100 in step 410 by means of a loudspeaker 103. The action instruction is a function of the driving instruction.In step 412, the verification unit 150 is used to check the driver's 100 response to the issuing of the instruction.

[0061] In an optional step 414, driver 100 is authenticated using a driver authentication unit 170. If the authentication is unsuccessful, the voice input is ignored in step 416, and the procedure for that voice input is terminated. In an optional step 418, driver 100's fitness to drive is checked using a driving ability detection unit 160. If the fitness-to-drive assessment is incorrect, the voice input is ignored in step 420, and the procedure for that voice input is terminated. In an optional step 422, driver 100's attention is checked using a proximity sensor 165. If the attention check is incorrect, the voice input is ignored in step 424, and the procedure for that voice input is terminated.

[0062] In step 426, the driver's reaction 100 is determined. If the driver's reaction 100 is not judged to be correct, the speech input is ignored in step 428, and the procedure for this speech input is terminated. If the driver's reaction 100 is judged to be correct, in step 430, the potential driving instruction is used as an actuator instruction to control the vehicle's actuators 200 by means of the at least partially autonomous vehicle control unit.

Claims

1. Method for controlling at least a semi-autonomous vehicle by a driver (100) by means of speech input, comprising the steps of: capturing, by means of a microphone (303), the speech input of the driver (100); determining, by means of a speech recognition unit (140), a potential driving instruction from the speech input of the driver (100); comparing the potential driving instruction with a list of possible driving instructions, wherein the list of possible driving instructions is a function of possible driving instructions of an at least semi-autonomous vehicle control unit (190); if the potential driving instruction is included in the list of possible driving instructions: outputting, by means of a loudspeaker (103), an instruction to the driver (100), wherein the instruction is a function of the driving instruction; verifying, by means of the verification unit (150), a response of the driver (100) to the output of the instruction;and if, by means of the verification unit (150), the driver's response (100) is judged to be correct, the potential driving instruction is used as an actuator instruction to control the actuators (200) of the vehicle by means of the at least semi-autonomous vehicle control unit (190).

2. The method of claim 1, wherein the list of possible driving instructions is attributed, the attribute containing a restriction of the possible driving instructions.

3. Method according to one of the preceding claims, with the further Step: after issuing the instruction, authenticate the driver (100) using a driver authentication unit (170).

4. Method according to one of the preceding claims, comprising the further step: after issuing the instruction, checking the driver's fitness to drive (100) by means of a fitness-to-drive detection unit (160).

5. Method according to one of the preceding claims, comprising the further step: after issuing the instruction, checking the driver's attention (100) by means of a proximity sensor (165).

6. Method according to one of the preceding claims, comprising the further steps of: outputting a control signal by means of the loudspeaker (103); detecting the control signal by means of the microphone (303); and if, by means of the verification unit (150), the control signal is judged to be correct, using the potential driving instruction as an actuator instruction to control the actuators (200) of the vehicle by means of the at least semi-autonomous vehicle control unit (190).

7. Method according to one of the preceding claims, wherein a haptic input of the driver (100) is the driver's (100) reaction to the output of the instruction.

8. Method according to one of the preceding claims, wherein a predefined movement of the driver (100) is the driver's (100) reaction to the issuance of the instruction.

9. Method according to one of the preceding claims, wherein a movement of a steering wheel (110) and / or a movement of a pedal (112) by the driver (100) is the driver's (100) reaction to the issuance of the instruction.

10. Device for controlling an at least partially autonomous vehicle by a driver (100) by means of speech input, the device comprising: an interface to a microphone configured to capture the speech input of the driver (100); a speech recognition unit (140) configured to recognize whether the speech input is a potential driving instruction; an interface to a list of possible driving instructions, wherein the list of possible driving instructions is a function of possible driving instructions of an at least partially autonomous vehicle control unit (190); an interface to a loudspeaker (103) configured to output an instruction to the driver (100) when the speech input is included in the list of possible driving instructions; a verification unit (150) configured to check a response by the driver (100) to the output of the instruction by means of a sensor;and an interface to at least a semi-autonomous vehicle control unit (190), designed to control the actuators (200) of the vehicle on the basis of the potential driving instruction as; Actuator instruction when the verification unit (150) judges the driver's (100) response to be correct.

11. Device according to claim 10, wherein the sensor is configured as at least one of the following devices: as a haptic sensor (114) configured to detect haptic input from the driver (100), wherein the haptic sensor (114) is configured as a button, switch, rotary control, pressure sensor, proximity sensor, and / or as a capacitive touch sensor of a screen or input field; and / or as an interior camera function configured to detect a predefined movement of the driver (100), and an image recognition unit configured to evaluate the movement of the driver (100) detected by the interior camera; and / or as a steering wheel motion sensor (111) configured to detect movement of a steering wheel (110) by the driver (100); and / or as a pedal motion sensor (113) configured to detect movement of a pedal (112) by the driver (100).

12. Device according to claim 10 or 11, further comprising at least one of: an interface to a driver authentication unit (170); an interface to a fitness-to-drive detection unit (160); and / or an interface to a proximity sensor (165).

13. Vehicle with a device according to claim 10 or 11.

14. Use of a device according to claim 10 or 11 for safely controlling a vehicle by a driver (100) by means of speech input.

15. Non-volatile, computer-readable storage medium with a program stored therein which, when executed on a processor of a device according to any one of claims 10-12, instructs the device to perform the steps according to any one of claims 1-9.

Images