Method for controlling a motor vehicle with an automated driving function using acoustic commands, and motor vehicle

Abstract

The invention relates to a method for controlling a motor vehicle (10) with an automated driving function using acoustic commands. The automated driving function comprises at least one control program (58) for controlling a steering unit (14), a drive unit (12) and / or a deceleration unit (92) of the motor vehicle (10) which controls the motor vehicle (10) in the automated driving mode. According to the invention, the control program (58) of the automated driving function is adapted on the basis of an acoustic command (94) from a passenger (32) of the motor vehicle (10). The invention also relates to a motor vehicle (10) having a drive unit (12), a steering unit (14) and / or a deceleration unit (92), and having a control unit (40) for carrying out a method of this kind.

Description

[0001] Description

[0002] Method for controlling a motor vehicle with an automated driving function by acoustic commands and motor vehicle

[0003] The invention relates to a method for controlling a motor vehicle with an automated driving function by means of acoustic signals and to a motor vehicle for carrying out such a method according to the preamble of the independent patent claims.

[0004] With automated driving functions, especially with partially or fully autonomous driving modes in motor vehicles, the individual experience of assisted driving presents a challenge, as every passenger in a motor vehicle has different preferences regarding the feeling of being driven. In human-driven vehicles, such preferences can be easily implemented through appropriate steering commands and commands via the accelerator and / or brake pedal. Different people also react differently to the respective driving style of the person driving. Autonomous or partially autonomous vehicles usually have a driving program that moves the vehicle from its starting point to its destination. Alternatively, several different driving programs can be applied for autonomous ferry operation.

[0005] US 2019 / 0227 553 A1 discloses a method for controlling an autonomous vehicle using voice commands. The vehicle has a voice output system that responds to the occupants' voice commands, for example, a query about the arrival time, with a voice output.

[0006] US 2020 / 0225676 A1 describes a method for controlling a vehicle. Data on human driving styles is collected, and machine learning is used to generate multiple driving profiles based on the collected data, reflecting how human drivers react to different traffic scenarios. From these driving profiles, an optimal driving profile is selected based on feedback from a passenger.

[0007] US 2020 / 0 126 544 A1 discloses a computer system in which a spoken command in natural language is translated into an intermediate command in constructed language using a first deep neural network. This intermediate command is translated using a second deep neural network based on the receipt of vehicle information and controls a function of the vehicle. The computer system can translate the constructed intermediate command into a spoken natural language response using a third deep neural network and operate a vehicle based on the vehicle command.

[0008] The use of speech to communicate with robots is known in principle. A method for controlling a robot in a crowded space, such as a hall or pedestrian zone, is already known from the state of the art. This method incorporates the subjective sensory level of the human into the robot's behavior, and the robot maintains a greater or lesser distance when moving past people depending on the human's perception.

[0009] Furthermore, methods are known from the prior art in which the driving behavior of human drivers is observed and simulated within the framework of neural learning with artificial intelligence in order to increase the comfort of "being driven" and to adapt the driving experience in an autonomous vehicle to a familiar driving experience in a vehicle steered by a human driver and thus to make it as acceptable as possible for the average passenger.

[0010] A disadvantage of the solutions known from the state of the art, however, is that the action performed by the robot is not evaluated appropriately by human feedback, and the robot does not take this feedback into account when executing the next movement. Furthermore, the robot's behavior is generally limited to very specific situations and voice interactions, making intuitive control impossible for the customer. Furthermore, no known solution from the state of the art offers the ability to adapt the driving behavior of an autonomous or semi-autonomous vehicle to the customer's individual preferences through a learning process.

[0011] The invention is based on the object of improving the individually perceived driving comfort in an autonomous motor vehicle or a semi-autonomous operation of a motor vehicle and of at least partially overcoming the disadvantages known from the prior art. This object is achieved by a method for controlling a motor vehicle with an automated driving function using acoustic commands. The automated driving function comprises at least one control program for controlling a steering unit, a drive unit, and / or a deceleration unit of the motor vehicle. The control program controls the motor vehicle in the automated driving function. The control program of the automated driving function is adapted based on an acoustic command from a passenger of the motor vehicle.

[0012] In this context, a motor vehicle is understood to mean any vehicle such as passenger cars, trucks, buses, agricultural machinery, construction machinery, and the like, which is powered by a drive engine and can transport a passenger. An automated driving function is understood to be a function for the autonomous or semi-autonomous driving of a motor vehicle, in which the control program controls the motor vehicle, accelerates it, or brakes it without any corresponding control action from a passenger of the motor vehicle. An acoustic command is understood to mean, in particular, a natural voice input from a passenger who comments on the autonomous or semi-autonomous driving style of the motor vehicle and thus adapts it to their personal preferences. This allows the passenger to adapt the driving style of the vehicle to their personal wishes through voice input.Responding to acoustic commands, especially natural speech, offers a particularly simple way to control a motor vehicle and to adapt a control program to carry out and adapt an automated journey to the individual wishes of the passenger or passengers.

[0013] The additional features listed in the dependent claims provide advantageous improvements and further developments of the method for controlling the motor vehicle listed in independent claim 1.

[0014] In a preferred embodiment of the method, the acoustic commands are issued in the form of a passenger's natural speech. Natural speech is a particularly simple and convenient way to individually adapt a control program for an automated driving function. Natural speech input offers the significant advantage that the passenger does not have to memorize or look up specific commands, but can adapt the driving style of the vehicle through their natural speech input.

[0015] Alternatively or additionally, it is advantageous for the acoustic commands to be delivered in the form of human screams or cheers, sobs, whistles, or clapping. In addition to natural speech, other acoustic commands can also indicate satisfaction or a request for change in the current control program for automated driving. For example, if a child or baby begins to cry after a sharp acceleration, the vehicle can accelerate more gently the next time.

[0016] In a preferred embodiment of the invention, the acoustic commands are linked to corresponding road sections of the road traveled by the motor vehicle. For example, in a first step, the comments of a human passenger while driving along certain road sections can be recorded and classified accordingly depending on the motor vehicle's operating parameters, such as speed, distance from the edge of the road, and distance from other road users. In this way, the passenger's driving preferences are stored, allowing the motor vehicle to apply them when repeatedly driving along the same road section.For example, if the passenger wants to reduce the speed by means of an acoustic command on a certain type of road, such as a local road with cobblestones, which may be typical in residential areas, because children often play there, the motor vehicle can automatically reduce the speed the next time it drives through the same road or a road of a similar type.

[0017] In a further preferred embodiment of the invention, it is provided that the acoustic commands are linked to the corresponding behavior of other road users. For example, one can adapt one's own speed to the speed of the car in front. Furthermore, the automated motor vehicle can react to the passenger's requests for a greater distance from commercial vehicles or cyclists in order to reduce the potential occurrence of dangerous situations and thus improve the perceived driving experience for the passenger. It is particularly preferred if the other road users are classified and divided into groups, with the acoustic commands being assigned to at least one group of other road users. For example, two-wheeled vehicles can be classified as a group and, for example, more distance can generally be left for this group when overtaking.Alternatively, for example, if there are many vehicles parked on the side of the road, the distance when driving past can be increased in order to increase the safety of other road users when getting in and out and to avoid colliding with suddenly opened vehicle doors.

[0018] In an advantageous embodiment of the invention, acoustic and / or visual feedback is provided by an output unit of the motor vehicle in response to the passenger's adjustment request expressed by the acoustic command. This allows the passenger to be explained why a certain driving style has been selected and / or why a certain adjustment cannot be implemented, or what consequences this change would have, for example, on fuel consumption or a potential violation of the law, such as excessive speed.

[0019] A further aspect of the invention relates to a motor vehicle with a drive unit, a steering unit, and / or a deceleration unit, as well as a control unit for implementing a method for controlling the motor vehicle described in the preceding sections. Such a motor vehicle can increase the acceptance of autonomous passenger transport because the driving style can be adapted to the individual preferences of the passengers. This can increase the perceived driving comfort and reduce stress for the passenger.

[0020] In a preferred embodiment of the motor vehicle, the control unit comprises an interaction module, a translation module, and an automation module. This allows an acoustic command, in particular a human-made adaptation request expressed in natural language, to be easily accepted and translated into a machine-readable command in the form of program code for the control program for automated driving of the motor vehicle. A further improvement of the invention provides that the control unit additionally comprises a testing module that examines the parameters of the acoustic command for feasibility and, if necessary, rejects their adaptation.

[0021] In a particularly preferred embodiment of the invention, the translation module establishes a connection between a passenger's natural language, at least one driving situation, and the specific parameters, settings, or objectives of a control program for automated driving, and adapts these connections based on the passenger's formulated adaptation requests. The interaction module establishes a connection between a passenger's natural language, the driving situations of the autonomously or semi-autonomously driving motor vehicle, and the specific parameters, settings, or objectives of the control program. Furthermore, the translation module can develop an understanding of corresponding scenarios and traffic situations through a learning process within the framework of artificial intelligence.The parameters of the control program for controlling the automated driving function are adjusted by the passenger's acoustic commands, provided that these requests can be implemented safely within the framework of the legislation and / or driving physics.

[0022] Further preferred embodiments of the invention emerge from the remaining features mentioned in the subclaims.

[0023] The various embodiments of the invention mentioned in this application can be advantageously combined with one another, unless otherwise stated in the individual case.

[0024] The invention is explained below in exemplary embodiments with reference to the accompanying drawings. They show:

[0025] Figure 1 shows a motor vehicle with an automated driving function for carrying out a method according to the invention, Figure 2 shows a schematic representation of a preferred embodiment of a method according to the invention for controlling a motor vehicle with an automated driving function by acoustic commands,

[0026] Figure 3 is a schematic representation of a further preferred embodiment of a method according to the invention for controlling a motor vehicle with an automated driving function by acoustic commands,

[0027] Figure 4 is a further schematic representation of another preferred

[0028] Design of a method according to the invention for controlling a motor vehicle with an automated driving function by acoustic commands, and

[0029] Figure 5 shows a schematic road layout to explain a method according to the invention for controlling a motor vehicle with an automated driving function by acoustic commands.

[0030] Figure 1 shows a preferred embodiment of a motor vehicle 10 according to the invention, which adapts the driving style to the wishes of the passenger 32 or passengers 32 through acoustic commands 94. The motor vehicle 10 comprises a drive unit 12, a steering unit 14, and a deceleration unit 92, which control the motor vehicle 10 via a control unit 40. The control unit 40 comprises an interaction module 42, which receives the acoustic commands 94 of the passenger 32 via a microphone 34 located in the vehicle interior 18 and translates them into program text or commands for the control unit 40. Furthermore, the interaction module 42 comprises an artificial intelligence, which translates the commands of the control unit 40 into human language and responds to the acoustic commands of the passenger 32.The outputs can be verbalized and output via the output unit, in particular via a loudspeaker 36 and / or a visual output unit 38. Alternatively or additionally, the output of the artificial intelligence can be visualized by the visual output unit 38 and passed on to the passenger 32. The control unit 40 further comprises a translation module 44, which establishes the connection between natural language 96, the driving situations, and the specific parameters, settings, or objectives of a program for assisted or autonomous driving and can adapt these based on the textually formulated adaptation requests of the passenger 32. The control unit 40 further comprises a testing module 46, which examines parameters 99 of the acoustic commands of the passenger 32, for example a desired speed, for feasibility and, if necessary, rejects their adaptation.Ideally, such a rejection is made with an explanation of the reasons for the rejection by the interaction module 42. The control unit 40 further comprises an automation module 48, which plans the driving task and executes it by intervening in the drive unit 12, steering unit 14, and deceleration unit 92. For this purpose, the motor vehicle 10 has wheels 16 and seats 30 on which the passengers 32 sit. The motor vehicle 10 also comprises a camera unit 68, a distance measuring unit 70, a radar sensor system 72, and a lidar sensor system 74, which transmits data to the computing unit 52 for controlling an automated journey. In a less preferred embodiment, the radar sensor system 72 and / or the lidar sensor system 74 can be omitted, with the automated journey being controlled exclusively via the images recorded by the camera unit 68.The control unit 40 or the elements 42, 44, 46, 48 of the control unit 40 can each have a memory unit 50, a learning unit 54, and a computing unit 52. A computer program code 56 is stored in the memory unit 50, which includes a control program 58 configured to execute a method according to the invention for controlling a motor vehicle 10 using acoustic commands 94.

[0031] Figure 2 shows a preferred embodiment of a control unit 40 according to the invention for driving adaptation through acoustic commands 94. The control unit 40 comprises four modules, namely an interaction module 42, a translation module 44, a testing module 46, and an automation module 48. The interaction module 42 is configured to record the acoustic commands 94, in particular natural speech 96, of the passenger 32 via a microphone 34, translate them into machine-readable program text, and forward them to the automation unit 48. Furthermore, the interaction module 42 is configured to translate the output of the artificial intelligence into natural speech 96 and output it via a loudspeaker 36.The interaction module 42 forwards the adjustment requests of the passenger 32, translated into program text, to the translation module 44, which establishes the connection between natural language 96, the driving situations, and the specific parameters 99, settings, or objectives of a program for assisted or autonomous driving, and can adapt these based on the textually formulated adjustment requests of the passenger 32. Not only the acoustic commands 94 in the form of natural language 96 of the passenger 32 serve as input, but also external influences, such as other road users 76, such as buses 82 or passenger cars 84, whose driving behavior is continuously analyzed by the camera unit 68, distance measuring unit 70, radar sensors 72, and lidar sensors 74. The parameters 99 desired to be changed by the passenger 32 are then examined for feasibility in a verification module 46 and, if necessary,their adjustment is rejected, especially if the requested changes are inconsistent with applicable legislation. For example, a request for a higher speed is rejected if the desired speed is above a threshold, particularly above a speed limit applicable in that section of the route. Ideally, the reasons for the rejection are explained to the passenger 32 via the interaction module 42. If the parameters 99 can be changed, the automation module 48 implements this change via a control program 58, taking into account external influences such as other road users 76, and changes the driving style of the motor vehicle 10.

[0032] Figure 3 shows a further preferred embodiment of a control unit 40 for personalized control via acoustic commands 94. The control unit 40 comprises three modules, namely an interaction module 42, a testing module 46, and an automation module 48. The interaction module 42 receives the acoustic commands 94, in particular natural speech 96, of the passenger 32 via a microphone 34 and translates these acoustic commands 94 into machine-readable program text. Furthermore, the interaction module 42 translates the output of the artificial intelligence into natural speech 96 and outputs it to the passenger 32 via a loudspeaker 36.In this exemplary embodiment, the translation module 44 is integrated into the interaction module 42, whereby the interaction module 42 establishes the connection between natural language 96, the driving situations, and the specific parameters 99, settings, or objectives of a program for assisted or autonomous driving, and can adapt these based on the textually formulated adaptation requests of the passenger 32. Alternatively or additionally, the interaction module 42 can communicate directly with the automation module 48, using not only the acoustic commands 94, in the form of natural language 96, of the passenger as input, but also external influences, such as other road users 76, such as buses 82 or passenger cars 84, whose driving behavior is continuously analyzed by the camera unit 68, distance measuring unit 70, radar sensors 72, and lidar sensors 74.The parameters 99 desired to be changed by the passenger 32 are examined for feasibility in a test module 46, and if necessary, their adjustment is rejected. Ideally, the reasons for rejection are explained to the passenger 32 via the interaction module 42. If the parameters 99 can be changed, the automation module 48 implements this change via a control program 58, taking into account external influences such as other road users 76, and changes the driving style of the motor vehicle 10.

[0033] Figure 4 shows a further preferred embodiment of a control unit 40 according to the invention for driving adaptation through acoustic commands 94. The control unit 40 has two modules, namely an adaptation unit 49 and an automation module 48. The adaptation unit 49 receives the acoustic commands 94, in particular natural speech 96, of the passenger 32 via a microphone 34 and translates them into machine-readable program text. Furthermore, the adaptation unit translates the outputs of the artificial intelligence into natural speech 96 and outputs them via a loudspeaker 36. The adaptation unit 49 establishes the connection between natural speech 96, the driving situations, and the specific parameters 99, settings, or objectives of a program for assisted or autonomous driving, adapting them based on the textually formulated adaptation requests of the passenger 32 and communicating directly with the automation module 48.Not only do the passenger's acoustic commands 94, in the form of natural speech 96, serve as input, but external influences, such as other road users 76, such as buses 82 or passenger cars 84, are also continuously analyzed by the camera unit 68, distance measurement unit 70, radar sensors 72, and lidar sensors 74. The test module 46 can be omitted in this version; however, the test module 46 is preferably integrated into the adaptation unit 49. The automation module 48 executes the desired change to the parameters 99 via a control program 58, taking into account external influences such as other road users 76, and changes the driving style of the motor vehicle 10.

[0034] Figure 5 shows a road course 98 during the implementation of a method according to the invention for adapting the driving style by means of acoustic commands 94. In this case, the driving style of the motor vehicle 10 can be adapted when encountering other road users 76. If the motor vehicle 10 overtakes a cyclist 86, for example, the lateral distance when overtaking can be increased for greater safety for the cyclist 86. When crossing a road 20 with children 90 at the side, the passenger 32 of the motor vehicle 10 can request slower driving and the motor vehicle 10 will then reduce its speed and do so automatically the next time it crosses the same road 20 or another road section which has similar driving situation characteristics. In the same way, a greater distance can be kept from the edge of the road for pedestrians 88 if the passenger of the motor vehicle 10 so desires.Other road users 76 are preferably divided into groups, such as commercial vehicles 82, which could include, for example, buses 82 or trucks 80, two-wheeled vehicles, etc., whereby for these different groups, driving characteristics of the motor vehicle 10, such as distance to the front or distance and speed when overtaking, can be adjusted to the personal preferences of the passenger 32 of the motor vehicle 10. In addition, the motor vehicle 10 recognizes place-name signs 24, as well as route-related restrictions 60 such as speed limits 62 or overtaking bans 64 and can thus, for example, accelerate faster or slower to the new maximum speed when leaving built-up areas depending on the personal preferences of the passenger 32. Likewise, when driving towards a town entrance, braking may occur too early or abruptly.The motor vehicle 10 can also always drive, for example, 5 km / h below the speed limit if the passenger 32 so wishes, and overtaking maneuvers in no-overtaking zones can be automatically prevented by means of appropriate acoustic commands. It is also possible to adapt the driving around curves 22, so that certain curves 22 can be taken faster or slower, or even cut, depending on the preference of the passenger 32. The motor vehicle 10 can also differentiate between a local road 28 and a country road 26, and, for example, continue to drive in the center of a local road 28 and further along the edge of road 20 on a country road 26. On roads 20 with poor road markings 66, driving further along the right edge of the road could also be set. The acoustic commands 94 of the passenger 32 can always be given in natural language 96, which makes the application as simple as possible.

[0035] List of reference symbols

[0036] Motor vehicle drive unit steering unit

[0037] wheel

[0038] Vehicle interior

[0039] Road Curve Town Sign Country Road Local Road

[0040] seat

[0041] Passenger microphone

[0042] Speaker visual output unit

[0043] Control unit Interaction module Translation module Testing module Automation module

[0044] Adaptation module

[0045] Storage unit Computing unit Learning unit

[0046] Computer program code control program route-related limitation speed limit overtaking ban

[0047] Road marking camera unit

[0048] Distance measuring unit Radar sensors Lidar sensors of other road users Commercial vehicle

[0049] Truck Bus

[0050] Passenger cars Cyclists

[0051] pedestrian

[0052] child

[0053] Delay unit acoustic commands natural language road course parameters

Claims

Patent claims 1. Method for controlling a motor vehicle (10) with an automated driving function by means of acoustic commands, wherein the automated driving function comprises at least one control program (58) for controlling a steering unit (14), a drive unit (12) and / or a deceleration unit (92) of the motor vehicle (10), which control program controls the motor vehicle (10) in the automated driving function, characterized in that an adaptation of the control program (58) of the automated driving function takes place on the basis of an acoustic command (94) of a passenger (32) of the motor vehicle (10).

2. A method for controlling a motor vehicle (10) according to claim 1, wherein the acoustic commands (94) are issued in the form of natural speech (96) of a passenger (32).

3. A method for controlling a motor vehicle (10) according to claim 1 or 2, wherein the acoustic commands (94) are given in the form of human screams or cheers, sobs, whistles or clapping.

4. A method for controlling a motor vehicle (10) according to one of claims 1 to 3, wherein the acoustic commands (94) are linked to corresponding road courses (98) of the road (20) traveled by the motor vehicle (10).

5. Method for controlling a motor vehicle (10) according to one of claims 1 to 4, wherein the acoustic commands (94) are linked to a corresponding behavior of other road users (76).

6. A method for controlling a motor vehicle (10) according to claim 5, wherein the other road users (76) are classified and divided into groups, wherein the acoustic commands (94) are assigned to at least one group of other road users (76).

7. Method for controlling a motor vehicle (10) according to one of claims 1 to 6, wherein an acoustic and / or visual feedback is provided on the signal generated by the acoustic The adjustment request of the passenger (32) expressed in the command (94) is output by an output unit (36, 38) of the motor vehicle (10).

8. Motor vehicle (10) with a drive unit (12), a steering unit (14) and / or a deceleration unit (92) and a control unit (40) for carrying out a method according to one of claims 1 to 7.

9. Motor vehicle (10) according to claim 8, wherein the control unit (40) comprises an interaction module (42), a translation module (44) and an automation module (48).

10. Motor vehicle (10) according to claim 9, wherein the translation module (44) establishes a connection between natural language (96), at least one driving situation and the specific parameters (99), settings or objectives of a control program (58) for automated driving and adapts these on the basis of the formulated adaptation requests of the passenger.

Images