Voice control automatic driving system

Abstract

A voice-controlled automated driving system that determines a trajectory of a vehicle based on voice commands includes one or more controllers that execute instructions to receive an electrical signal representative of a voice command, where the voice command indicates a driving operation. The controller converts the electrical signal into a text-based command, classifies the text-based command as one of a plurality of pre-compiled driving operations based on one or more machine learning algorithms, classifies the pre-compiled driving operations as a route-based operation or a behavior-based operation, and in response to determining that the pre-compiled driving operation is a route-based driving operation, identifying one or more related lane segments whose map data is affected by the pre-compiled driving operation, and determining an updated route plan by updating a current route plan currently followed by the vehicle to include the one or more related lane segments.

Description

Technical Field

[0001] The present disclosure relates to a voice-controlled automated driving system that determines a vehicle trajectory based on voice commands generated by a vehicle occupant. Background Art

[0002] Voice commands can be used to operate various features and systems that are part of a vehicle. For example, an occupant of a vehicle can instruct the vehicle's audio system to play a particular song or other type of audio file in response to receiving a voice command. As another example, an occupant can make a phone call within the vehicle based on a voice command without having to manually access his or her smartphone. An occupant can also utilize voice commands to provide a destination address to the vehicle's navigation system. Therefore, it should be understood that voice commands provide occupants with hands-free access to various features and systems within a vehicle. However, there remains a need to expand voice commands to other features and systems that are part of a vehicle to enhance the occupant's overall driving experience and increase customer satisfaction.

[0003] Therefore, while current vehicle systems achieve their intended purposes, there remains a need in the art for an improved method to provide voice activation functionality to various vehicle systems. Summary of the invention

[0004] According to several aspects, a voice-controlled autonomous driving system for determining a vehicle trajectory based on a voice command is disclosed. The voice-controlled autonomous driving system includes one or more controllers that execute instructions to receive an electrical signal representing a voice command generated by a vehicle occupant, wherein the voice command indicates a driving operation that changes the motion of the vehicle. The one or more controllers convert the electrical signal into a text-based command based on one or more speech recognition algorithms. The one or more controllers classify the text-based command as one of a plurality of pre-compiled driving operations based on one or more machine learning algorithms, wherein the pre-compiled driving operation represents a potential driving operation that the vehicle may perform. The one or more controllers classify the pre-compiled driving operation as a route-based driving operation or a behavior-based driving operation. In response to classifying the pre-compiled driving operation as a route-based driving operation, the one or more controllers compare the pre-compiled driving operation with map data to identify one or more relevant road segments of the map data affected by the pre-compiled driving operation. The one or more controllers determine an updated route plan by updating a current route plan currently followed by the vehicle to include one or more relevant road segments, wherein a trajectory of the vehicle is determined based on the updated route plan.

[0005] In another aspect, in response to classifying the precompiled driving maneuver as a behavior-based driving maneuver, the one or more controllers compare the precompiled driving maneuver to a plurality of behavior planner parameters to determine one or more updated behavior planner parameters.

[0006] In yet another aspect, one or more controllers execute instructions to determine an updated target state for the vehicle based on the one or more updated behavior planner parameters.

[0007] In one aspect, the updated target state indicates a driving maneuver, one or more updated behavior planner parameters, and the behavior planner parameters followed to execute the pre-compiled driving maneuver, and a trajectory of the vehicle is determined based on the updated target state.

[0008] In another aspect, a confidence score indicative of a confidence of an algorithm that classifies the text-based command is associated with the pre-compiled driving maneuver.

[0009] In yet another aspect, a voice-controlled automated driving system includes a human-machine interface (HMI) in electronic communication with one or more controllers.

[0010] In one aspect, one or more controllers execute instructions to compare a confidence score associated with a precompiled driving maneuver to a confidence threshold, and in response to determining that the confidence score associated with the precompiled driving maneuver is less than the confidence threshold, the one or more controllers instruct the HMI to generate a message indicating that the voice-controlled autonomous driving system is unable to execute the voice command.

[0011] In another aspect, in response to determining that the confidence score associated with the precompiled driving maneuver is greater than or equal to a confidence threshold, the one or more controllers compare the precompiled driving maneuver to a traffic rule set associated with a specific location of the vehicle. In response to determining that the precompiled driving maneuver is permitted based on the traffic rule set, the one or more controllers compare the precompiled driving maneuver to a plurality of potential proactive voice commands stored in a memory to determine whether there is a conflict between the precompiled driving maneuver and one of the potential proactive voice commands.

[0012] In yet another aspect, in response to determining that the precompiled driving maneuver conflicts with one of the potential proactive voice commands, the one or more controllers instruct the HMI to generate a message requesting the occupant to select the precompiled driving maneuver or the proactive voice command that conflicts with the precompiled driving maneuver.

[0013] In one aspect, in response to determining that there is no conflict between the precompiled driving maneuver and one of the potential proactive voice commands, the precompiled driving maneuver is classified as a route-based driving maneuver or a behavior-based driving maneuver.

[0014] In another aspect, the voice-controlled automated driving system further includes a microphone in electronic communication with the one or more controllers, wherein the microphone converts voice commands into electrical signals representing the voice commands.

[0015] In yet another aspect, route-based driving maneuvers represent maneuvers that change the current route plan that the vehicle is currently following.

[0016] In one aspect, behavior-based driving maneuvers represent maneuvers that control the behavior of the vehicle but do not change the current route plan that the vehicle is currently following.

[0017] In another aspect, a method for determining a vehicle trajectory based on a voice command by a voice-controlled autonomous driving system is disclosed. The method includes receiving, by one or more controllers, an electrical signal representing a voice command generated by a vehicle occupant, wherein the voice command indicates a driving operation that changes the motion of the vehicle. The method also includes converting, by one or more controllers, the electrical signal into a text-based command based on one or more speech recognition algorithms. The method also includes classifying, by one or more controllers, the text-based command as one of a plurality of pre-compiled driving operations based on one or more machine learning algorithms, wherein the pre-compiled driving operation represents a potential driving operation that the vehicle may perform. The method also includes classifying the pre-compiled driving operation as a route-based driving operation or a behavior-based driving operation. In response to classifying the pre-compiled driving operation as a route-based driving operation, the method includes comparing the pre-compiled driving operation with map data to identify one or more relevant lane segments of the map data affected by the pre-compiled driving operation. Finally, the method includes determining an updated route plan by updating a current route plan currently followed by the vehicle to include one or more relevant lane segments, wherein the trajectory of the vehicle is determined based on the updated route plan.

[0018] In one aspect, a voice-controlled autonomous driving system for determining a vehicle trajectory based on a voice command is disclosed. The voice-controlled autonomous driving system includes one or more controllers that execute instructions to receive an electrical signal representing a voice command generated by a vehicle occupant, wherein the voice command indicates a driving operation that changes the motion of the vehicle. The one or more controllers convert the electrical signal into a text-based command based on one or more speech recognition algorithms. The one or more controllers classify the text-based command as one of a plurality of pre-compiled driving operations based on one or more machine learning algorithms, wherein the pre-compiled driving operation represents a potential driving operation that the vehicle may perform. The one or more controllers classify the pre-compiled driving operation as a route-based driving operation or a behavior-based driving operation. In response to classifying the pre-compiled driving operation as a route-based driving operation, the one or more controllers compare the pre-compiled driving operation with map data to identify one or more relevant road segments of the map data affected by the pre-compiled driving operation. The one or more controllers determine an updated route plan by updating a current route plan currently followed by the vehicle to include one or more relevant road segments, wherein the trajectory of the vehicle is determined based on the updated route plan. In response to classifying the precompiled driving maneuver as a behavior-based driving maneuver, the one or more controllers compare the precompiled driving maneuver to a plurality of behavior planner parameters to determine one or more updated behavior planner parameters. The one or more controllers determine an updated target state for the vehicle based on the one or more updated behavior planner parameters.

[0019] In another aspect, the updated target state indicates a driving maneuver, one or more updated behavior planner parameters, and the behavior planner parameters followed to execute the pre-compiled driving maneuver, and a trajectory of the vehicle is determined based on the updated target state.

[0020] In yet another aspect, a confidence score indicative of a confidence of an algorithm that classifies a text-based command is associated with the pre-compiled driving maneuver.

[0021] In one aspect, the voice-controlled automated driving system also includes an HMI in electronic communication with the one or more controllers.

[0022] In another aspect, one or more controllers compare a confidence score associated with the precompiled driving maneuver to a confidence threshold and, in response to determining that the confidence score associated with the precompiled driving maneuver is less than the confidence threshold, instruct the HMI to generate a message indicating that the voice-controlled autonomous driving system is unable to execute the voice command.

[0023] In another aspect, the voice-controlled automated driving system further includes a microphone in electronic communication with the one or more controllers, wherein the microphone converts voice commands into electrical signals representing the voice commands.

[0024] Further areas of applicability will become apparent from the description provided herein.It should be understood that the description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings described herein are for illustration purposes only and are not intended to limit the scope of the present disclosure in any way.

[0026] Figure 1 is a schematic diagram of a vehicle including a disclosed voice-controlled automated driving system according to an exemplary embodiment, the system including one or more controllers in electronic communication with a plurality of perception sensors and microphones;

[0027] Figure 2 According to an exemplary embodiment Figure 1 a diagram of the software architecture of the one or more controllers shown; and

[0028] Figure 3 is a process flow diagram illustrating a method for determining a vehicle trajectory based on voice commands generated by a vehicle occupant via a disclosed voice-controlled automated driving system, according to an exemplary embodiment. DETAILED DESCRIPTION

[0029] The following description is merely exemplary in nature and is not intended to limit the present disclosure, application, or uses.

[0030] refer to Figure 1 , a schematic diagram of an exemplary voice-controlled autonomous driving system 10 for a vehicle 12 is shown. It should be understood that the vehicle 12 can be any type of vehicle, such as, but not limited to, a car, a truck, a sport utility vehicle, a van, or an RV. It should be understood that the voice-controlled autonomous driving system 10 provides at least Level 2 autonomous driving control, such as an advanced driver assistance system (ADAS). The voice-controlled autonomous driving system 10 includes one or more controllers 20 in electronic communication with a plurality of perception sensors 22, a microphone 24, and a human-machine interface (HMI) 26. The plurality of perception sensors 22 are configured to collect perception data indicative of the environment surrounding the vehicle 12. In the example shown in FIG. Figure 1 In the non-limiting embodiment shown, the plurality of perception sensors 22 include one or more cameras 30, an inertial measurement unit (IMU) 32, a global positioning system (GPS) 34, a radar 36, and a laser radar (LiDAR) 38, however, it should be understood that additional sensors may also be used. The HMI 26 represents a user interface through which an occupant 40 of the vehicle 12 receives messages from and provides user input to the voice-controlled automated driving system 10. An example of the HMI 26 is a touch screen that receives user input and displays images to the occupant 40 on the display 18.

[0031] Microphone 24 represents a device that converts sound waves into electrical signals, where the electrical signals are received by one or more controllers 20. Specifically, microphone 24 converts voice commands generated by an occupant 40 of vehicle 12 into electrical signals representing the voice commands. As explained below, one or more controllers 20 transform the electrical signals representing the voice commands into text-based commands 42 ( Figure 2 ). One or more controllers 20 determine a trajectory 90 for the vehicle 12 based on the text-based command 42, wherein the trajectory 90 specifies the movement of the vehicle 12 within a predefined time range. The voice command generated by the occupant 40 indicates a driving maneuver that changes the movement of the vehicle 12 while the vehicle 12 is traveling. In other words, the voice command generated by the occupant 40 indicates a driving maneuver that changes the current trajectory of the vehicle 12. Some examples of voice commands include, but are not limited to: It should also be understood that the occupant 40 is an authorized operator of the vehicle 12. The occupant 40 can be authorized by any type of command to exit from the next exit along the highway, change lanes, overtake another vehicle, and change the destination location of the route plan.

[0032] Figure 2 Shows Figure 1 The software architecture of one or more controllers shown in FIG. The one or more controllers 20 include a speech recognition module 50, a command classification module 52, a rule verification module 54, a memory module 56, a map association module 58, a feedback message generation module 60, a route planning module 62, a planner adaptation module 64, a behavior planner module 66, a user update routing module 68, and a motion control module 70. Figure 1 and Figure 2 , a speech recognition module 50 of one or more controllers 20 is in electronic communication with the microphone 24. The speech recognition module 50 receives an electrical signal from the microphone 24 that represents a speech command generated by the occupant 40 and converts the electrical signal into a text-based command 42 based on one or more speech recognition algorithms. Some examples of speech recognition algorithms for converting electrical signals into text-based commands 42 include, but are not limited to, for example, perceptual linear prediction (PLP) features, Viterbi algorithms, deep neural networks, and discrimination training. The command classification module 52 then receives the text-based command 42.

[0033] The text-based command 42 is classified by the command classification module 52 of the one or more controllers 20 as a pre-compiled driving operation, which is part of a plurality of pre-compiled driving operations. The pre-compiled driving operations represent a list of potential driving operations that the vehicle 12 may perform. Some examples of potential driving operations include, but are not limited to, commands to leave from the next exit, change lanes (left lane change or right lane change), and pass another vehicle (through the left lane or the right lane). For example, if the occupant 40 creates a voice command instructing the vehicle 12 to fly over traffic, this is not a driving operation that the vehicle 12 may perform and therefore will not be included in the list of pre-compiled driving operations.

[0034] The command classification module 52 classifies the text-based command 42 into one of a plurality of pre-compiled driving maneuvers based on one or more machine learning algorithms. In one embodiment, the command classification module 52 classifies the text-based command 42 based on a neural network trained to classify the text-based command 42. The command classification module 52 also assigns a confidence score associated with the pre-compiled driving maneuver, wherein the confidence score indicates the confidence of the algorithm that classifies the text-based command 42. The confidence score is represented by a probability from 0 to 1.

[0035] The command classification module 52 compares the confidence score associated with the precompiled driving action to a confidence threshold. In one embodiment, the confidence threshold is 0.6. However, it should be understood that the confidence threshold is based on the level of accuracy targeted by the voice-controlled automated driving system 10. For example, in another embodiment with enhanced accuracy, the confidence threshold is 0.9. In response to determining that the confidence score associated with the precompiled driving action is less than the confidence threshold, the command classification module 52 sends an error message 44 to the feedback message generation module 60, indicating that the text-based command 42 cannot be accurately classified as one of the precompiled driving actions. The feedback message generation module 60 then instructs the HMI 26 to generate a message to the occupant 40 indicating that the voice command is not executable.

[0036] In response to determining that the confidence score associated with the precompiled driving maneuver is greater than or equal to the confidence threshold, the command classification module 52 sends the precompiled driving maneuver to the rule verification module 54. The rule verification module 54 stores a set of traffic rules associated with the specific location where the vehicle 12 is currently located. The traffic rule set indicates regulatory rules, such as a speed limit associated with the road where the vehicle 12 is currently traveling, and whether passing is allowed within the current driving lane. The rule verification module 54 compares the precompiled driving maneuver with the traffic rules, which are associated with the specific location of the vehicle 12. In response to determining that the precompiled driving maneuver is not permitted based on the traffic rules associated with the specific location where the vehicle 12 is located, the rule verification module 54 generates a message 46 indicating that the precompiled driving maneuver is not permitted, wherein the message 46 is received by the feedback message generation module 60. The feedback message generation module 60 then instructs the HMI 26 to generate a message to the occupant 40 indicating that the voice command is not permitted. For example, if the precompiled driving maneuver is to increase speed and the vehicle 12 has reached the speed limit, the feedback message generation module 60 instructs the display 18 of the HMI 26 to read "Speed ​​Limit Reached".

[0037] In response to determining that the precompiled driving maneuver is permitted based on the traffic rule set, the rule verification module 54 sends the precompiled driving maneuver to the memory module 56. The memory module 56 compares the precompiled driving maneuver with a plurality of potential active voice commands stored in the memory of the one or more controllers 20 to determine whether the precompiled driving maneuver conflicts with one of the potential active voice commands. The potential active voice commands are stored in the memory of the one or more controllers 20 and represent previously generated precompiled driving maneuvers that are currently being executed by the voice-controlled automated driving system 10. For example, if the precompiled driving maneuver is to overtake surrounding vehicles and one of the potential active voice commands instructs the vehicle 12 to stay in the current lane, the memory module 56 determines that there is a conflict. In addition to the previously generated precompiled driving commands, the potential active voice commands also include redundant driving maneuvers that have been generated by the occupant 40.

[0038] In response to determining that the precompiled driving maneuver conflicts with one of the potential active voice commands, the memory module 56 sends the message 48 to the feedback message generation module 60. The feedback message generation module 60 instructs the HMI 26 to generate a message requesting the occupant 40 to select the precompiled driving maneuver or the active voice command that conflicts with the precompiled driving maneuver. The HMI 26 receives user input from the occupant 40 indicating the precompiled driving maneuver or the active voice command. The user update routing module 68 receives the user input from the HMI 26 and sends the user input to the memory module 56. The memory module 56 then replaces the conflicting precompiled driving maneuver and the active voice command with the response provided by the occupant 40.

[0039] In response to determining that the precompiled driving operation does not conflict with one of the potential active voice commands, the memory module 56 classifies the precompiled driving operation as a route-based driving operation or a behavior-based driving operation. A route-based driving operation represents an operation that changes the current route plan currently being followed by the vehicle 12. For example, exiting from the next exit along a highway or changing the destination location are both route-based operations. Behavior-based driving operations represent operations that control the behavior of the vehicle 12 but do not change the current route plan currently being followed by the vehicle 12. Some examples of behavior-based driving operations include changing from the current lane of travel to an adjacent lane and parking along the current route plan. For example, a precompiled driving operation may be changing lanes to overtake a slower vehicle.

[0040] In response to classifying the pre-compiled driving operation as a route-based driving operation, the memory module 56 sends the pre-compiled driving operation to the map association module 58. The map association module 58 stores map data indicating the current location of the vehicle 12. The map data includes lane-level data, which includes information related to static scene elements located in the environment surrounding the vehicle 12, such as lane marking locations and types, traffic sign types, number of lanes, and lane operation types. The map association module 58 compares the pre-compiled driving operation with the map data to identify one or more relevant lane segments whose map data is affected by the pre-compiled driving operation. For example, if the pre-compiled driving operation is a left turn, the lane segment associated with the left turn operation is one or more relevant lane segments. The map association module 58 sends the one or more relevant lane segments to the route planning module 62, wherein the route planning module 62 determines an updated route plan as described below.

[0041] In one embodiment, the map association module 58 is unable to identify one or more relevant road segments. For example, the occupant 40 may request to travel along a road that does not exist in real life. In the event that the map association module 58 is unable to identify one or more relevant road segments, the map association module 58 sends a message 80 indicating that the relevant road segments were not identified to the feedback message generation module 60. The feedback message generation module 60 then instructs the HMI 26 to generate a message to the occupant 40 indicating that the voice-controlled automated driving system 10 is unable to identify the route that the vehicle 12 is to follow.

[0042] In one embodiment, in the event that the map association module 58 cannot identify one or more relevant road segments, the map association module 58 checks a recommendation system (not shown) to find an alternative route to replace the updated route plan. If an alternative route is identified, the map association module 58 sends a message 92 to the feedback message generation module 60. The feedback message generation module 60 then instructs the HMI 26 to generate a message to the occupant 40 requesting the occupant 40 to confirm the alternative route. The user update routing module 68 receives user input from the HMI 26 and sends the user input to the memory module 56.

[0043] In response to receiving user feedback accepting the alternate route, the HMI 26 sends the user input to the user update routing module 68. The user update routing module 68 then sends the alternate route to the memory module 56. The memory module 56 compares the alternate route to a plurality of potential active voice commands stored in the memory of the one or more controllers 20 to determine whether the alternate route conflicts with one of the potential active voice commands. The alternate route represents a route that may be used to replace the original pre-compiled driving maneuver. For example, if the pre-compiled driving maneuver selects to leave from the next exit on the right, but there is no exit on the right, the alternate route may be the next exit on the left.

[0044] Determining an updated route plan will now be described. The route planning module 62 determines the updated route plan by updating the current route plan currently followed by the vehicle 12 to include one or more relevant lane segments. In one embodiment, if the occupant 40 has indicated that the voice command is urgent, the route planning module 62 sends the updated route plan directly to the motion control module 70. The motion control module 70 determines the trajectory 90 of the vehicle 12 based on the updated route plan. If the occupant 40 indicates that the voice command is not urgent, the route planning module 62 sends a message 82 indicating that the voice command is not urgent to the feedback message generation module 60. The feedback message generation module 60 instructs the HMI 26 to generate a message requesting the occupant to first confirm the updated route plan. The user update routing module 68 receives user input from the HMI 26.

[0045] If the occupant 40 agrees with the updated route plan, the occupant 40 provides user input to the HMI 26 indicating that the occupant 40 has confirmed the updated route plan, and the user update routing module 68 receives the indication. The user update routing module 68 then sends the updated route plan to the motion control module 70. If the occupant 40 enters user input into the HMI 26 rejecting the updated route plan, the user update routing module 68 sends a message to the memory module 56 to remove the pre-compiled driving maneuvers from the memory, the voice-controlled automated driving system 10 terminates the current program, and the vehicle 12 continues to travel along the current route plan.

[0046] In response to classifying the pre-compiled driving operation as a behavior-based driving operation, the memory module 56 sends the pre-compiled driving operation to the planner adaptation module 64. The planner adaptation module 64 stores a plurality of behavior planner parameters that define the behavior of the vehicle 12 during autonomous driving. Some examples of behavior planner parameters include, but are not limited to, the acceleration of the vehicle 12 after stopping, the deceleration of the vehicle 12 when approaching a stop, the maximum acceleration level, the target lane, the target speed, and the target driving intensity level. In one embodiment, there are three driving intensity levels: a conservative level, a medium level, and an aggressive level intensity. However, it should be understood that fewer or more driving intensity levels may also be used.

[0047] In one embodiment, if the planner adaptation module 64 is unable to find and adapt the behavior planner parameters, the planner adaptation module 64 transmits a message 84 to the feedback message generation module 60 indicating that the behavior planner parameters were not found. In response to receiving the message 84, the feedback message generation module 60 instructs the HMI 26 to generate a message notifying the occupant 40 that the pre-compiled driving maneuver could not be performed.

[0048] In one embodiment, in the absence of finding the behavior planner parameters, the planner adaptation module 64 can check the recommendation system (not shown) to find an alternative driving operation to replace the pre-compiled driving operation. When the alternative driving operation is identified, the planner adaptation module 64 sends a message 94 to the feedback message generation module 60, requesting the occupant 40 to confirm the alternative driving operation. In response to receiving the message 94, the feedback message generation module 60 instructs the HMI 26 to generate a message requesting the occupant 40 to confirm the alternative driving operation. In response to receiving the user input accepting the alternative driving operation, the user update routing module 68 sends the alternative driving operation to the memory module 56. The alternative driving operation represents a driving operation that may be used to replace the original pre-compiled driving operation. For example, if the original pre-compiled driving operation is to stop the vehicle 12 on the left side of the road, the alternative voice command is changed to stop on the right side of the road.

[0049] Determining an updated target state for the vehicle 12 will now be described. The planner adaptation module 64 compares the precompiled driving maneuver to a plurality of behavior planner parameters to determine one or more updated behavior planner parameters. The updated behavior planner parameters represent the behavior planner parameters that are updated to allow the vehicle 12 to perform the precompiled driving maneuver. For example, if the precompiled driving maneuver is to change to the right lane to overtake a slower vehicle, the planner adaptation module 64 will determine that the one or more updated behavior planner parameters are the target driving lane. Specifically, the target driving lane is changed from the current driving lane to the right driving lane.

[0050] The planner adaptation module 64 sends one or more updated behavior planner parameters to the behavior planner module 66. The behavior planner module 66 determines the updated target state of the vehicle 12 based on the one or more updated behavior planner parameters. The updated target state indicates the driving operation, one or more updated behavior planner parameters, and the behavior planner parameters followed by the pre-compiled driving operation. For example, if the pre-compiled driving operation is to change to the right lane to overtake a slower vehicle, and one or more updated behavior planner parameters are changing the target driving lane from the current driving lane to the right driving lane, the updated target state will include the driving operation (changing to the right driving lane), the target driving lane, the target driving intensity level and the target speed. In one embodiment, if the occupant 40 indicates that the voice command is urgent, the behavior planning module 66 sends the updated target state directly to the motion control module 70. The motion control module 70 determines the trajectory 90 of the vehicle 12 based on the updated target state. If the occupant 40 does not indicate that the voice command is urgent, the behavior planner module 66 sends a message 86 to the feedback message generation module 60 , which instructs the HMI 26 to generate a message requesting the occupant to first confirm the updated target state before determining a trajectory 90 for the vehicle 12 .

[0051] If the occupant 40 agrees with the updated target state, the occupant 40 provides user input to the HMI 26 indicating that the occupant 40 has confirmed the updated target state, and the motion control module 70 receives the indication. The motion control module 70 then determines the trajectory 90 of the vehicle 12 based on the updated target state. If the occupant 40 enters user input to the HMI 26 rejecting the updated target state, the behavior planner module 66 transmits a message to the memory module 56 to remove the pre-compiled driving maneuver from the memory, the voice-controlled automated driving system 10 terminates the current process, and the vehicle 12 continues to travel along the current route plan.

[0052] Figure 3 is a process flow diagram illustrating an exemplary method 300 for determining a trajectory 90 of a vehicle 12 based on a voice command generated by an occupant 40 via the disclosed voice controlled automated driving system 10. Figure 1-Figure 3 , the method 300 begins at block 302. In block 302, the speech recognition module 50 of the one or more controllers 20 receives an electrical signal from the microphone 24 representing a speech command generated by an occupant 40 of the vehicle 12, wherein the speech command indicates a driving maneuver that changes the motion of the vehicle 12. The method 300 may then proceed to block 304.

[0053] In block 304 , the speech recognition module 50 of the one or more controllers 20 converts the electrical signal into a text-based command 42 based on one or more speech recognition algorithms. The method 300 may then proceed to block 306 .

[0054] In block 306, the command classification module 52 of the one or more controllers 20 classifies the text-based command 42 into one of a plurality of pre-compiled driving maneuvers based on one or more machine learning algorithms. The pre-compiled driving maneuvers represent potential driving maneuvers that the vehicle 12 may perform. As described above, the confidence score indicates the confidence of the algorithm that classified the text-based command 42. The method 300 may then proceed to decision block 308.

[0055] In decision block 308, the command classification module 52 of the one or more controllers 20 compares the confidence score associated with the pre-compiled driving maneuver to a confidence threshold. In response to determining that the confidence score associated with the pre-compiled driving maneuver is less than the confidence threshold, the method 300 proceeds to block 310. In block 310, the command classification module 52 sends an error message 44 to the feedback message generation module 60, indicating that the text-based command 42 cannot be accurately classified as one of the pre-compiled driving maneuvers. The feedback message generation module 60 then instructs the HMI 26 to generate a message to the occupant 40 indicating that the voice command is not operable. The method 300 may then terminate, or in the alternative, return to block 302, where the occupant 40 generates another voice command.

[0056] In response to determining that the confidence score associated with the precompiled driving maneuver is greater than or equal to the confidence threshold, the method 300 proceeds to decision block 312. In decision block 312, the rule validation module 54 compares the precompiled driving maneuver to a traffic rule set associated with the specific location of the vehicle 12. In response to determining that the precompiled driving maneuver is not permitted based on the traffic rule set associated with the specific location where the vehicle 12 is located, the method 300 proceeds to block 314. In block 314, the feedback message generation module 60 instructs the HMI 26 to generate a message to the occupant 40 indicating that the voice command is not permitted. The method 300 may then terminate.

[0057] Then, in response to determining that the pre-compiled driving maneuver is permitted based on the traffic rule, the method 300 may proceed to decision block 316. In decision block 316, the memory module 56 of the one or more controllers 20 compares the pre-compiled driving maneuver with the potential proactive voice commands stored in memory to determine whether there is a conflict or redundancy between the pre-compiled driving maneuver and one of the potential proactive voice commands.

[0058] Then, in response to determining that there is a conflict between the pre-compiled driving operation and one of the potential active voice commands, the method 300 can proceed to box 318. In box 318, the memory module 56 sends a message 48 indicating the inconsistency to the feedback message generation module 60. Then, the feedback message generation module 60 instructs the HMI 26 to generate a message requesting the occupant 40 to select the pre-compiled driving operation or the active voice command that conflicts with the pre-compiled driving operation. If the pre-compiled driving operation is redundant with respect to the potential active voice command, the pre-compiled driving operation is ignored. Then, the method 300 can proceed to decision box 320. Then, in response to determining that there is no conflict between the pre-compiled driving operation and one of the potential active voice commands, the method 300 can proceed to decision box 320.

[0059] In decision block 320 , the memory module 56 classifies the pre-compiled driving maneuver as a route-based driving maneuver or a behavior-based driving maneuver. In response to classifying the pre-compiled driving maneuver as a route-based driving maneuver, the method 300 proceeds to decision block 322 .

[0060] In decision block 322, the map association module 58 of the one or more controllers 20 compares the pre-compiled driving maneuvers to the map data to identify one or more relevant road segments of the map data that are affected by the pre-compiled driving maneuvers. If the map association module 58 fails to identify one or more relevant road segments, the method 300 proceeds to decision block 324.

[0061] In decision block 324, the map association module 58 checks a recommendation system (not shown) to find an alternative route to replace the updated route plan. If no alternative route is found, the method 300 proceeds to block 326. In block 326, the HMI 26 generates a message to the occupant 40 indicating that the voice-controlled automated driving system 10 cannot identify an alternative route for the vehicle 12 to follow. The method 300 may then terminate.

[0062] Referring to block 324, if an alternate route is found, the method 300 proceeds to decision block 328. In decision block 328, the HMI 26 generates a message requesting the occupant 40 to confirm the alternate route. If the occupant 40 rejects the alternate route, the method 300 may terminate. In response to receiving user feedback accepting the alternate route, the user update routing module 68 transmits the alternate route to the memory module 56, and the method returns to block 316.

[0063] Referring to block 322, if the map association module 58 of the one or more controllers 20 identifies one or more relevant road segments whose map data is affected by the pre-compiled driving operation, the method 300 proceeds to block 330. In block 330, the route planning module 62 determines an updated route plan by updating the current route plan currently followed by the vehicle 12 to include the one or more relevant road segments, wherein the trajectory 90 of the vehicle 12 is determined based on the updated route plan. The method 300 can then proceed to decision block 332.

[0064] In decision box 332, if the occupant 40 does not indicate that the voice command is urgent, the method 300 proceeds to decision box 334. In decision box 334, the HMI 26 generates a message requesting the occupant 40 to first confirm the updated route plan. In the event that the occupant 40 confirms the route plan, the method 300 proceeds to box 336. If the occupant 40 rejects the route plan, the method 300 can terminate.

[0065] Referring to decision block 332, if the occupant 40 indicates that the voice command is urgent, the method 300 proceeds to block 336. In block 336, the voice-controlled automated driving system 10 executes trajectory 90 and the method 300 terminates.

[0066] Referring to decision block 320 , in response to classifying the pre-compiled driving maneuver as a behavior-based driving maneuver, the method 300 proceeds to decision block 338 . In decision block 338 , if the planner adaptation module 64 cannot locate the behavior planner parameters, the method 300 proceeds to block 340 .

[0067] In decision block 340, the planner adaptation module 64 checks a recommendation system (not shown) to find an alternative driving maneuver to replace the pre-compiled driving maneuver. If no alternative driving maneuver is found, the method 300 proceeds to block 342. In block 342, the planner adaptation module 64 transmits a message 84 to the feedback message generation module 60 indicating that no behavior planner parameters were found, and the HMI 26 generates a message notifying the occupant 40 that the pre-compiled driving maneuver cannot be performed. The method 300 may then terminate.

[0068] Referring to block 340, if an alternative driving maneuver is identified, the method 300 proceeds to decision block 344. In decision block 344, the HMI 26 generates a message requesting the occupant 40 to confirm the alternative driving maneuver. If the occupant 40 rejects the alternative driving maneuver, the method 300 may terminate. In response to receiving user feedback accepting the alternative driving maneuver, the user update routing module 68 sends the alternative driving maneuver to the memory module 56, and the method returns to block 316.

[0069] Referring to decision box 338, if the planner adaptation module 64 finds behavior planner parameters, the method 300 proceeds to box 346. In box 346, the planner adaptation module 64 compares the pre-compiled driving maneuver to the plurality of behavior planner parameters to determine one or more updated behavior planner parameters, wherein the updated behavior planner parameters represent behavior planner parameters that are updated to allow the vehicle 12 to perform the pre-compiled driving maneuver. The method 300 can then proceed to box 348.

[0070] In block 348, the behavior planner module 66 determines an updated target state for the vehicle 12 based on the one or more updated behavior planner parameters, wherein the updated target state indicates the driving maneuver, the one or more updated behavior planner parameters, and the behavior planner parameters to follow in executing the precompiled driving maneuver, wherein the trajectory 90 of the vehicle 12 is determined based on the updated target state. The method 300 may then proceed to decision block 350.

[0071] In decision block 350, if the occupant 40 indicates that the voice command is not urgent, the method 300 proceeds to decision block 352. In decision block 352, the HMI 26 generates a message requesting the occupant 40 to first confirm the updated target state. In the event that the occupant 40 confirms the updated target state, the method 300 proceeds to block 336. If the occupant 40 rejects the updated target state, the method 300 may terminate.

[0072] Referring to decision block 350, if the occupant 40 indicates that the voice command is urgent, the method 300 proceeds to block 354. In block 354, the voice-controlled automated driving system 10 executes trajectory 90 and the method 300 terminates.

[0073] Referring generally to the accompanying drawings, the disclosed voice-controlled autonomous driving system provides a number of technical effects and benefits. Specifically, the disclosed voice-controlled autonomous driving system determines a trajectory of a vehicle based on voice commands generated by a vehicle occupant. The voice-controlled autonomous system verifies the voice command to determine that the driving operation indicated by the voice command is permitted based on current traffic regulations and does not conflict with other commands currently being executed. It should also be understood that the voice-controlled autonomous driving system communicates various system limitations and messages to the occupants of the vehicle through the HMI. Finally, in an embodiment, the occupant may be provided with an opportunity to first confirm or reject the driving operation before the vehicle performs the driving operation.

[0074] A controller may refer to an electronic circuit, a combinational logic circuit, a field programmable gate array (FPGA), a processor (shared, dedicated or grouped) that executes code, or a combination of some or all of the above (e.g., a system on a chip), or a portion of the above components. In addition, the controller may be microprocessor-based, such as a computer having at least one processor, memory (RAM and / or ROM), and associated input and output buses. The processor may run under the control of an operating system residing in the memory. The operating system may manage computer resources so that a computer program code embodied as one or more computer software applications (e.g., applications residing in the memory) may have instructions executed by the processor. In an alternative embodiment, the processor may execute the application directly, in which case the operating system may be omitted.

[0075] The description of the present disclosure is merely exemplary in nature, and variations that do not depart from the gist of the present disclosure are intended to fall within the scope of the present disclosure. These variations should not be regarded as departing from the spirit and scope of the present disclosure.

Claims

1. A voice-controlled autonomous driving system for determining a vehicle trajectory based on a voice command, the voice-controlled autonomous driving system comprising: One or more controllers that execute instructions to: receiving an electrical signal representing the voice command generated by an occupant of the vehicle, wherein the voice command indicates a driving maneuver that changes the motion of the vehicle; converting the electrical signals into text-based commands based on one or more speech recognition algorithms; classifying the text-based command as one of a plurality of pre-compiled driving maneuvers based on one or more machine learning algorithms, wherein the pre-compiled driving maneuvers represent potential driving maneuvers that the vehicle may perform; classifying the precompiled driving maneuver as a route-based driving maneuver or a behavior-based driving maneuver; In response to classifying the precompiled driving maneuver as the route-based driving maneuver, comparing the precompiled driving maneuver with map data to identify one or more associated roadway segments of the map data that are affected by the precompiled driving maneuver; and An updated route plan is determined by updating a current route plan currently followed by the vehicle to include the one or more relevant road segments, wherein the trajectory of the vehicle is determined based on the updated route plan.

2. The voice-controlled automatic driving system according to claim 1, wherein: The one or more controllers execute instructions to: In response to classifying the precompiled driving maneuver as a behavior-based driving maneuver, the precompiled driving maneuver is compared to a plurality of behavior planner parameters to determine one or more updated behavior planner parameters.

3. The voice-controlled automatic driving system according to claim 2, wherein: The one or more controllers execute instructions to: An updated target state for the vehicle is determined based on the one or more updated behavior planner parameters.

4. The voice-controlled automatic driving system according to claim 3, wherein: The updated target state indicates a driving maneuver, the one or more updated behavior planner parameters, and the behavior planner parameters followed by performing the precompiled driving maneuver, and determining the trajectory of the vehicle based on the updated target state.

5. The voice-controlled automatic driving system according to claim 1, wherein: A confidence score is associated with the pre-compiled driving maneuver, the confidence score indicating a confidence of an algorithm that classifies the text-based command.

6. The voice-controlled automated driving system of claim 5 , further comprising a human-machine interface (HMI) in electronic communication with the one or more controllers.

7. The voice-controlled automatic driving system according to claim 6, wherein: The one or more controllers execute instructions to: comparing the confidence score associated with the precompiled driving maneuver to a confidence threshold; as well as In response to determining that the confidence score associated with the pre-compiled driving maneuver is less than the confidence threshold, the HMI is instructed to generate a message indicating that the voice-controlled automated driving system is unable to execute the voice command.

8. The voice-controlled automatic driving system according to claim 7, wherein: The one or more controllers execute instructions to: In response to determining that the confidence score associated with the precompiled driving maneuver is greater than or equal to the confidence threshold, comparing the precompiled driving maneuver to a traffic rule set associated with the particular location of the vehicle; as well as In response to determining that the precompiled driving maneuver is permissible based on the traffic rule set, the precompiled driving maneuver is compared to a plurality of potential proactive voice commands stored in a memory to determine whether the precompiled driving maneuver conflicts with one of the potential proactive voice commands.

9. The voice-controlled automatic driving system according to claim 8, wherein: The one or more controllers execute instructions to: In response to determining that the precompiled driving maneuver conflicts with one of the potential proactive voice commands, the HMI is instructed to generate a message requesting the occupant to select the precompiled driving maneuver or the proactive voice command that conflicts with the precompiled driving maneuver.

10. The voice-controlled automatic driving system according to claim 8, wherein: The one or more controllers execute instructions to: In response to determining that the pre-compiled driving maneuver does not conflict with one of the potential proactive voice commands, the pre-compiled driving maneuver is classified as the route-based driving maneuver or the behavior-based driving maneuver.

Images