Intelligent virtual assistant selection

Abstract

The present disclosure provides "intelligent virtual assistant selection". A query is received from a user device. A domain and / or task corresponding to the query is determined. A collaborative selector is used to identify a set of similar queries relative to the query. A plurality of virtual assistants (VAs) are ranked based on an average value of customer feedback received from executing the similar query, the customer feedback including a rating of a response to the similar query. One of the plurality of VAs is selected as a one having a highest average value of the customer feedback for use in responding to the query. The query is replied using a selected response generated by the one of the plurality of VAs.

Description

Technical Field

[0001] Aspects of the present disclosure generally relate to methods for intelligent virtual assistant selection. Background Art

[0002] Speech-to-text and voice assistant applications can provide drivers or passengers with the ability to interact with a computing system to obtain information, perform actions, or receive responses to queries. However, it can be difficult to manage multiple available speech-to-text or voice assistant services. Summary of the invention

[0003] In one or more illustrative examples, a system for intelligent virtual assistant selection includes an intelligent virtual assistant selection (IVAS) service executed by one or more hardware devices. The IVAS service is configured to receive a query from a user device; determine a domain and / or task corresponding to the query; use a collaborative selector to identify a set of similar queries relative to the query; select one of a plurality of virtual assistants (VAs) for responding to the query based on the similar queries; and reply to the query using a selected response generated by the one of the plurality of VAs.

[0004] In one or more illustrative examples, a method for intelligent virtual assistant selection through an IVAS service includes: receiving a query from a user device; determining a domain and / or task corresponding to the query; using a collaborative selector to identify a set of similar queries relative to the query; ranking multiple VAs based on an average of customer feedback received from executing the similar queries, the customer feedback including ratings of responses to the similar queries; selecting one of the multiple VAs as the one with the highest average of the customer feedback to respond to the query; and replying to the query using a selected response generated by the one of the multiple VAs.

[0005] In one or more illustrative examples, a non-transitory computer-readable medium includes instructions that, when executed by one or more hardware devices of an IVAS service, cause the IVAS service to perform operations including: receiving a query from a user device; determining a domain and / or task corresponding to the query; using a collaborative selector to identify a set of similar queries relative to the query; ranking multiple VAs based on an average of customer feedback received from executing the similar queries, the customer feedback including ratings of responses to the similar queries; selecting one of the multiple VAs as the one with the highest average of the customer feedback for responding to the query; and replying to the query using a selected response generated by the one of the multiple VAs. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 An exemplary system for implementing intelligent virtual assistant selection is shown;

[0007] Figure 2 An example of user preferences selected using an intelligent virtual assistant is shown;

[0008] Figure 3 An exemplary data flow for intelligent virtual assistant selection is shown;

[0009] Figure 4 An exemplary data log of a user query is shown;

[0010] Figure 5 An example of the operation of an intelligent virtual assistant selection service in explicit mode is shown;

[0011] Figure 6 An example of the operation of an intelligent virtual assistant selection service in implicit mode is shown;

[0012] Figure 7 An exemplary process for training an intelligent virtual assistant to select a service to operate in implicit mode is shown;

[0013] Figure 8 An exemplary process for operating an intelligent virtual assistant selection service in implicit mode is shown; and

[0014] Fig. 9 An example of a computing device for implementing aspects of an intelligent virtual assistant selection service is shown. DETAILED DESCRIPTION

[0015] Embodiments of the present disclosure are described herein. However, it should be understood that the disclosed embodiments are merely examples, and other embodiments may take on various forms and alternative forms. The drawings are not necessarily drawn to scale; some features may be enlarged or minimized to show the details of a particular component. Therefore, the specific structural details and functional details disclosed herein should not be interpreted as limiting, but only as a representative basis for teaching those skilled in the art to adopt the embodiments in different ways. As will be understood by those of ordinary skill in the art, the various features shown and described with reference to any one of the drawings may be combined with the features shown in one or more other drawings to produce embodiments that are not explicitly shown or described. The combination of features shown provides representative embodiments for typical applications. However, various combinations and modifications of features consistent with the teachings of the present disclosure may be desired for specific applications.

[0016] There are multiple voice-enabled virtual assistants (VAs) available today. A user can send a query to the VA, which can respond with an answer or by performing the requested action. Some VAs are specialized for different tasks or in different domains. However, it may not be clear to the user which VA to select for a given query. When a user makes a request to a particular assistant and receives an unhelpful response, the user may try again with a different VA. This can lead to an unpleasant user experience.

[0017] Many users have preferences for which VA to use for which task or domain. For example, some users prefer one VA for weather reports but another VA for navigation tasks. Similarly, some users prefer one VA to handle IoT / smart home requests and a different VA to handle vehicle control requests. However, because VAs and their capabilities are always evolving, it is difficult for users to keep track of these changes to fully utilize these virtual assistants.

[0018] Various aspects of the present disclosure relate to methods for automatically selecting the best VA to handle a user's query without the user having to know the capabilities of each VA. This can reduce poor responses from VAs and bring a more seamless experience to the user. The method can automatically select a VA to handle a task based on factors such as: user preferences, insights gained from the user's interaction patterns and user feedback, and collaborative filtering of aggregated user behavior data. Additional aspects of the present disclosure are discussed in detail herein.

[0019] Figure 1 An exemplary system 100 implementing intelligent virtual assistant selection (IVAS) is shown; the system 100 includes a vehicle 102 having a telematics control unit (TCU) 104 and a human-machine interface (HMI) 106. The TCU 104 can allow the vehicle 102 to communicate with remote devices (such as multiple VAs 110 and IVAS services 112) via a communication network 108. The IVAS service 112 may include a preference engine 114, an interaction data recorder 116, a VA selector 118, a feedback engine 120, a machine learning (ML) model 122, and a collaboration selector 124. It should be noted that the system 100 is merely an example, and a system 100 with more, fewer, or different elements may be used. For example, although a vehicle 102 with a TCU 104 is shown, the disclosed method may be applicable to other environments where the VA 110 may be used, such as a smart phone or smart speaker device.

[0020] Vehicle 102 may include various types of automobiles, crossover utility vehicles (CUVs), sport utility vehicles (SUVs), trucks, recreational vehicles (RVs), boats, jeeps, airplanes, or other mobile machines for transporting people or goods. In many cases, vehicle 102 may be powered by an internal combustion engine. As another possibility, vehicle 102 may be a battery electric vehicle (BEV) powered by one or more electric motors. As another possibility, vehicle 102 may be a hybrid electric vehicle powered by both an internal combustion engine and one or more electric motors, such as a series hybrid electric vehicle, a parallel hybrid electric vehicle, or a parallel / series hybrid electric vehicle. Since the type and configuration of vehicle 102 may vary, the capabilities of vehicle 102 may vary accordingly. As some other possibilities, vehicle 102 may have different capabilities in terms of passenger capacity, traction capacity and capacity, and storage capacity. Some vehicles 102 may be operator-controlled, while other vehicles 102 may be autonomously or semi-autonomously controlled.

[0021] The vehicle 102 may include a TCU 104 configured to communicate via a communication network 108. The TCU 104 may be configured to provide telematics services to the vehicle 102. As some non-limiting possibilities, these services may include navigation, turn-by-turn directions, vehicle health reports, local business searches, accident reports, and hands-free calling. The TCU 104 may accordingly be configured to communicate with the communication network 108 using a transceiver.

[0022] The TCU 104 may include various types of computing devices that support the execution of the functions of the TCU 104 described herein. In an example, the TCU 104 may include one or more processors configured to execute computer instructions, and a storage medium on which computer executable instructions and / or data may be maintained. Computer-readable storage media (also referred to as processor-readable media or storage devices) include any non-temporary (e.g., tangible) media that participate in providing data (e.g., instructions) that can be read by a computer (e.g., by a processor). Generally speaking, the processor receives instructions and / or data, such as from a storage device, etc., to a memory and uses the data to execute the instructions, thereby performing one or more processes, including one or more of the processes described herein. Computer executable instructions may be compiled or interpreted according to a computer program created using a variety of programming languages ​​and / or technologies, which include, but are not limited to, Java, C, C++, C#, Fortran, Pascal, Visual Basic, Python, JavaScript, Perl, etc., individually or in combination.

[0023] The vehicle 102 may also include an HMI 106 located within the cabin of the vehicle 102. The HMI 106 may be configured to receive voice input from an occupant of the vehicle 102. The HMI 106 may include one or more input devices, such as a microphone or a touch screen, and one or more output devices, such as a display or a speaker.

[0024] The HMI 106 may use an input device to collect audio from the cabin or interior of the vehicle 102. For example, one or more microphones may receive audio including voice commands or other audio data from within the cabin. The TCU 104 may perform an action in response to the voice command. In one example, the HMI 106 may forward the command to another device for processing.

[0025] The HMI 106 may provide output to the cabin or interior of the vehicle 102 using an output device. For example, one or more displays may be used to display information or entertainment content to the driver or passenger. The display may include one or more of a built-in display, an instrument cluster display, a second row display screen, a third row display screen, or any other display at any other location in the vehicle 102. For example, for entertainment purposes, a video or other content may be displayed on the display. In addition, notifications, prompts, the status of the vehicle 102, the status of the connected device, etc. may be displayed to the user. In another example, one or more speakers may include a sound system or other speakers for playing music, notification sounds, phone call audio, responses from a voice assistant service, etc. For example, the HMI 106 may provide audio to the speakers, such as music, audio accompanying a video, an audio response to a user request, etc.

[0026] The communication network 108 may provide communication services, such as packet-switched network services (e.g., Internet access, Voice over Internet Protocol (VoIP) communication services), to devices connected to the communication network 108. An example of a communication network 108 is a cellular telephone network. For example, the TCU 104 may access a cellular network via a connection to one or more cellular towers. To facilitate communications over the communication network 108, the TCU 104 may be associated with a unique device identifier (e.g., a mobile device number (MDN), an Internet Protocol (IP) address, etc.) to identify communications of the TCU 104 over the communication network 108 as being associated with the vehicle 102.

[0027] VA 110 may include various digital assistants that use various technologies to understand voice input and provide relevant results or perform requested actions. VA 110 may perform speech recognition to convert received audio input from an audio signal into text. VA 110 may also perform other analysis on the input, such as semantic analysis to understand the user's emotions. VA 110 may also perform language processing on the processed input to understand what task is being asked of VA 110. VA 110 may perform the requested task and utilize speech synthesis to return a result or an indication of whether to perform the requested function. The input provided to VA 110 may be referred to as a prompt or intent. As some non-limiting examples, VA 110 may include AMAZON ALEXA, GOOGLE ASSISTANT, APPLE SIRI, FORD SYNC, and MICROSOFT CORTANA.

[0028] The IVAS service 112 may be a computing device configured to communicate with the vehicle 102 and the VA 110 via the communication network 108. The IVAS service 112 may be configured to assist the user in selecting and personalizing the use of the various VAs 110. Such selection and personalization may be accomplished in explicit and implicit methods.

[0029] In the explicit approach, the IVAS service 112 selects the VA 110 to be used based on preferences explicitly configured and set by the user. This information may be part of the user's personal profile. For example, the HMI 106 may be used to allow the user to select a mapping of available VAs 110 to various domains (or to specific tasks in other examples).

[0030] A domain may refer to a specific knowledge, subject, or feature that the VA 110 can handle. For example, navigation may be a domain, weather may be a domain, music may be a domain, etc. However, tasks may be individual elements within a domain. In the example, moving to the next song, requesting a specific song to be played, and changing the volume may be tasks within the music domain. Receiving directions to a destination, asking for alternate routes, adding a fuel stop, etc. may be tasks within the navigation domain.

[0031] The preference engine 114 may be configured to allow a user to set preferences for different domains using the VA 110. The preferences may be stored as a lookup table with domain to VA mappings.

[0032] As one possibility, the preference engine 114 may interact with the HMI 106 to provide a list of domains (such as navigation, music, weather, etc.), where for each category, the user can explicitly select which VA 110 is to be used. For example, the user may select to use a first VA 110 for navigation, a second VA 110 for music, and a third VA for weather. As another possibility, the HMI 106 may additionally or alternatively provide a list of tasks, such as by category of domain. In some examples, the user may be able to set a VA 110 for a domain and also override the selection of a specific task within the domain.

[0033] Figure 2 An example of such user preferences 200 is shown. As shown, an exemplary mapping of user preferences 200 to six VAs 110 (i.e., VA1, VA2, VA3, VA4, VA5, VA6) for a set of tasks is shown. For example, for the task of playing music from a memory stick, the user prefers to use VA4. Or for a navigation task, the user may prefer VA4 if available (e.g., if the vehicle 102 is equipped with navigation, then use the local vehicle navigation VA 110), but it may also be acceptable to use VA2 or VA3.

[0034] The user may set the user preferences 200 to have each domain and / or task handled by a specific VA 110, or may select multiple domains to be handled by a single VA 110. In one example, the user preferences 200 may be implemented as a hash map containing a key-value pair data table, where the key is defined to indicate the domain and the value indicates the VA 110 selected by the user.

[0035] Return to reference Figure 1 , the IVAS service 112 selects the VA 110 to use based on factors that are either an alternative or in addition to the user's explicit settings. For example, these factors may include learning the user's usage patterns, collaborative filtering, and user feedback. For example, when performing a navigation task, the IVAS service 112 may identify a second VA 110 that performs best in terms of navigation based on the user's previous interactions, interactions of other users, and / or user feedback ratings of the second VA 110.

[0036] Figure 3 An example data flow 300 for the operation of the IVAS service 112 is shown. Figure 3 And continue to refer to Figure 1To implement the explicit and / or implicit methods, the IVAS service 112 may implement one or more of a preference engine 114, an interaction data recorder 116, a VA selector 118, a feedback engine 120, an ML model 122, and a collaboration selector 124. It should be noted that while the IVAS service 112 is discussed with respect to the IVAS service 112, one or more aspects of the operation of the IVAS service 112 may be implemented on the vehicle 102 and / or using another device such as a user's mobile phone.

[0037] In the data flow 300, a user may provide a query 302 to the IVAS service 112. For example, the user may utilize the HMI 106 of the vehicle 102 to capture a spoken command in an audio signal, which may be provided to the IVAS service 112 by the TCU 104 of the vehicle 102 via the communication network 108. In some examples, speech-to-text processing may be performed by the vehicle 102, and a text version of the query 302 may be provided to the IVAS service 112.

[0038] In response to IVAS service 112 receiving query 302, IVAS service 112 can perform intent classification to identify the domain and / or task to which query 302 belongs. Various natural language processing (NLP) techniques can be used to perform the intent classification. In an example, a set of tasks can be defined. These tasks are sometimes referred to as intent. Each task can be triggered by a set of similar phrases that fall under a common name. A labeled training set of such phrases mapped to the corresponding tasks can be used to train a machine learning model. At runtime, in inference mode, the machine learning model can be used to bin the received input into its corresponding task and / or domain including the task. For example, query 302 "What is the weather like in Chicago?" can be identified as a "weather" domain request. Similarly, query 302 "Get the route to the nearest Starbucks" can be identified as a "navigation" domain request.

[0039] In the explicit method, based on the user preferences 200, the IVAS service 112 can send a query 302 to the VA 110 selected by the user and receive a response 304 from the VA 110 selected by the user. The IVAS service 112 can provide the response 304 from the VA 110 selected by the user back to the user. This can be referred to as a selected response 306. The selected response 306 can be returned from the IVAS service 112 to the TCU 104 via the communication network 108 and passed to the HMI 106 to be provided to the user.

[0040] Turning to the implicit method, the IVAS service 112 can learn the user's VA 110 usage pattern in the background in a shadow mode for an initial duration (e.g., 60 days). In the shadow mode, when the selected response 306 is provided from the VA 110, user feedback 308 is requested from the user. For example, the vehicle 102 can use the HMI 106 to ask the user for user feedback 308 after presenting the selected response 306. Then, similar to the way the query 302 is sent, the user feedback 308 can be sent by the vehicle 102 to the IVAS service 112.

[0041] The user feedback 308 may include a “rating score” on a scale (e.g., positive, neutral, or negative; a score along a scale such as zero to five, one to five, negative three to positive three, etc.). The user may provide the user feedback 308 to indicate whether the VA 110 successfully processed the query 302, and / or to provide the user's perception of the quality of the selected response 306 provided to the user.

[0042] User feedback 308 may be elicited by the feedback engine 120. The feedback engine 120 may be configured to catalog which VA 110 the user prefers for which domain of the query 302, e.g., the user prefers a first VA 110 for navigation queries 302 and a second VA 110 for shopping, etc. This may allow the engine 120 to build and maintain the user preferences 200.

[0043] The interaction data recorder 116 may be configured to record user interactions with the IVAS service 112. These interactions may include user feedback 308 and other contextual information.

[0044] Figure 4 An exemplary data log 400 of user interactions with the VA 110 is shown. As shown, the data log 400 includes various information, such as an identifier of the user providing feedback, a textual representation of the query 302, an inferred domain and / or task of the query 302, an indication of which VA 110 processed the query 302, a textual representation of a response 304 from the VA 110 to the query 302, and a user feedback 308 rating of the response 304 and / or overall interaction with the VA 110.

[0045] Return to Figure 3, using the data log 400, the ML model 122 can be configured to learn patterns from the user's interactions with the VA 110 and the rating scores of the user feedback 308 during the explicit mode. This can sometimes be referred to as a shadow mode of operation because the system 100 shadows the user's interactions. Using the information, the IVAS service 112 can train the ML model 122 to update the user preferences 200 to select a VA 110 for future interactions for a particular task and / or domain. Once trained, in the inference mode, the ML model 122 can receive a query 302 and can provide an ML recommendation 310 indicating which VA 110 (or a set of preferred VAs 110 in descending order of relevance) to use to respond to the query 302.

[0046] The IVAS service 112 can use any of a variety of machine learning techniques to train the ML model 122, such as a decision tree approach, to learn and update preferences for VA 110 selections for future interactions. The method is not limited to decision trees, and other ML techniques may also be used. For training, the ML model 122 may receive one or more of the following as input: (i) data log 400 records, which include messages such as the type or domain of the request being issued, an indication of the VA 110 that processes the request, and a response 304 from the VA 110; (ii) user feedback 308, which includes a rating score provided by a user for a corresponding response 304 from the VA 110; (iii) the frequency of similar requests made by other users; and / or (iv) ratings of user feedback 308 from other users on similar requests.

[0047] In response to the ML model 122 of the IVAS service 112 learning the user preference 200 with a confidence of at least a predefined confidence threshold, the IVAS service 112 can activate the implicit mode for the user. For example, the IVAS service 112 can show the learned user preference 200 to the user for confirmation. Once confirmed by the user, the IVAS service 112 can deploy the ML model 122 to switch to the implicit mode. Alternatively, the IVAS service 112 can automatically apply the ML model 122 in response to the ML model 122 reaching the accuracy level and / or confidence threshold. Once activated in implicit mode, the user does not need to track which VA 110 can handle what task. The user can simply make a query 302, and the best VA 110 for handling the task can be automatically provided in the ML suggestion 310 from the ML model 122.

[0048] Thus, in implicit mode, the VA selector 118 may be configured to select an appropriate VA 110 to process the requested query 302 based on the ML suggestions 310 from the ML model 122 .

[0049] In addition, the VA selector 118 may also be configured to utilize preferences learned from multiple users to further enhance the recommendations. For example, the IVAS service 112 may utilize the collaborative selector 124 to determine preferences for tasks and / or domains similar to those of the query 302 across users.

[0050] In the example, for a query 302 in the navigation domain, if the fourth VA 110 is the VA 110 most requested by most people for such tasks, and if the rating scores provided by those users are positive and high, the collaboration selector 124 can indicate that the fourth VA 110 can be automatically selected to handle the collaboration suggestion 312 for the query 302 from the user.

[0051] More formally, for a specific query 302Q requested by a user, the collaboration selector 124 may perform a collaboration operation including:

[0052]

[0053] in:

[0054] Q sim is based on collaborative filtering of similar queries from other users 302;

[0055] VA max is the maximum number of times a particular VA 110 is selected to handle a particular task or domain;

[0056] R avg is the average of the user feedback 308 ratings provided by other users for similar requests for the particular VA 110;

[0057] R threshold is the minimum value of user feedback 308 required to automatically select the VA 110; and

[0058] Q threshold is used to automatically select the Q of VA 110 sim The minimum quantity.

[0059] In addition to automatic selection, it should be noted that the VA selector 118 may also allow a user to override the selected VA 110. Moreover, the VA selector 118 may be configured to store the responses 304 from each VA 110 for a particular query 302Q and send the selected response 306 to the user. This also allows the user to cycle through or otherwise select different responses 304 from multiple VAs 110 in shadow mode (e.g., if the selected response 306 is not helpful) without having to perform a second query 302 that cycles to the VA 110.

[0060] Figure 5An example 500 of the operation of the IVAS service 112 in explicit mode is shown. As shown, the user preferences 200 indicate an explicit mapping of domains and / or tasks to the VAs 110. Using the mapping, the VA selector 118 selects the VA 110 listed in the user preferences 200. However, in this approach, if the VA 110 is not capable of handling a particular query 302 or type of query 302, the query 302 may be completed without a useful selected response 306.

[0061] Figure 6 An example 600 of the operation of the IVAS service 112 in implicit mode is shown. Here, the ML model 122 is used instead to select the selected response 306. In addition, because the VA selector 118 can ask multiple VAs 110 whether the selected VA 110 has a problem, if the first highest rated VA 110 cannot complete the particular query 302, the VA selector 118 can automatically move to the second highest rated VA 110.

[0062] Figure 7 An exemplary process 700 for training the IVAS service 112 to operate in an implicit mode is shown. In an example, the process 700 can be performed by components of the IVAS service 112 in the context of the system 100.

[0063] At operation 702, IVAS service 112 initializes operation in explicit mode. In explicit mode, IVAS service 112 can utilize preference engine 114 to receive user preference 200 from user and manage it. In an example, preference engine 114 can interact with HMI 106 to provide a list of domains (such as navigation, music, weather, etc.), wherein for each category, the user can explicitly select which VA 110 will be used. In explicit mode, based on user preference 200, IVAS service 112 can send any received query 302 to VA 110 selected by the user and receive response 304 from VA 110 selected by the user. IVAS service 112 can provide the response 304 from VA 110 selected by the user back to the user. IVAS service 112 can also use feedback engine 120 to receive user feedback 308 about the provided response 304.

[0064] At operation 704, the IVAS service 112 collects entries of the data log 400. The data log 400 may include various information, such as an identifier of the user providing feedback, a textual representation of the query 302, an inferred domain and / or task of the query 302, an indication of which VA 110 processed the query 302, a textual representation of a response 304 from the VA 110 to the query 302, and a user feedback 308 rating of the response 304 and / or overall interaction with the VA 110. Figure 4 An exemplary data log 400 is shown in FIG.

[0065] At operation 706, the IVAS service 112 trains the ML model 122 using the data log 400. Using the data log 400 as training data, the ML model 122 can learn patterns from the user's interactions with the VA 110 during the explicit mode and the rating scores of the user feedback 308. For example, the IVAS service 112 can train the ML model 122 to update the user preferences 200 to select the VA 110 for future interactions for a specific task and / or domain.

[0066] Once trained, in inference mode, the ML model 122 may receive a query 302 and may provide an ML recommendation 310 indicating which VA 110 (or a set of preferred VAs 110 in descending order of relevance) to use to respond to the query 302 .

[0067] At operation 708, the IVAS service 112 determines whether the IVAS service 112 is trained for use. In an example, the IVAS service 112 can split the data log 400 into a training portion and a test portion. Periodically, from time to time and / or when a new data log 400 entry is received, the IVAS service 112 can determine whether the accuracy of the ML model 122 is sufficient for implicit mode. In an example, the IVAS service 112 can use the training portion of the data to train the IVAS service 112, and can use the test portion with the indicated user preference 200 to confirm that the ML model 122 provides accurate results within at least a predefined accuracy level and / or confidence. If yes, control proceeds to operation 710. If not, control returns to operation 704 to wait for additional data log 400 entries and / or perform additional training cycles.

[0068] At operation 710, the IVAS service 112 operates in implicit mode. In implicit mode, a user can simply make a query 302 and the best VA 110 for handling the task can be provided automatically in the ML suggestions 310 from the ML model 122 and / or via collaboration suggestions 312 from the collaboration selector 124. Figure 8 Further aspects of the performance of the system 100 in the implicit mode are discussed in detail. After or during operation 710, it should be noted that continued training and / or refinement of the ML model 122 may be performed, for example, in accordance with operations 704 and 706.

[0069] Figure 8An example process 800 is shown for operation of the IVAS service 112 in implicit mode. In an example, as with the process 700, the process 800 can be performed by components of the IVAS service 112 in the context of the system 100.

[0070] At operation 802, the IVAS service 112 receives a query 302 from a user device. In an example, the user device may be a vehicle 102, and the user may utilize the HMI 106 of the vehicle 102 to capture a verbal command in an audio signal, which may be provided to the IVAS service 112 by the TCU 104 of the vehicle 102 via the communication network 108. In some examples, speech-to-text processing may be performed by the vehicle 102, and a text version of the query 302 may be provided to the IVAS service 112. In another example, the user device may be a mobile phone or a smart speaker, which may similarly send the query 302 to the IVAS service 112.

[0071] At operation 804, the IVAS service 112 determines the domain and / or task specified by the query 302. In the example, the IVAS service 112 performs intent classification to identify the domain and / or task to which the query 302 belongs. Various NLP techniques can be used to perform the intent classification. In the example, a set of tasks can be defined. These tasks are sometimes referred to as intents. Each task can be triggered by a set of similar phrases that fall under a common name. A labeled training set of such phrases mapped to the corresponding tasks can be used to train a machine learning model. At runtime, in inference mode, the machine learning model can be used to bin the received input into its corresponding task and / or domain including the task.

[0072] At operation 806, the IVAS service 112 identifies similar queries 302 that are related to the received query 302. In an example, the IVAS service 112 may access the data log 400 to retrieve queries 302 that are categorized into the same domain and / or task as the received query 302.

[0073] At operation 808, the IVAS service 112 ranks the plurality of VAs 110 that use similar queries 302. In an example, the IVAS service 112 may rank the plurality of VAs 110 based on an average of customer feedback received from executing similar queries 302. In some examples, the IVAS service 112 may exclude from consideration VAs that have not received at least a minimum number of user feedback. In some examples, the IVAS service 112 may exclude from consideration VAs that have not received at least a minimum average rating score from customer feedback. Figure 3 The detailed description of the collaborative operation discusses additional aspects of the ranking.

[0074] At operation 810, the IVAS service 112 selects a VA 110 from the plurality of VAs 110 based on the ranking. In an example, the IVAS service 112 may select one of the plurality of VAs 110 having the highest average of customer feedback for responding to the query 302.

[0075] At operation 812, the IVAS service 112 provides the selected response 306 from the VA 110 to reply to the query 302. In an example, the reply may be provided to the user device in response to receiving the query 302. Thus, the IVAS service 112 may allow the system 100 to personalize and select the best VA 110 for a particular task and / or domain based on the user preferences 200, collaborative filtering via the collaborative selector 124, and user feedback 308. After operation 812, the process 800 ends.

[0076] Variations of process 800 are possible. In an example, IVAS service 112 can select a second VA with a second highest average of customer feedback from among multiple VAs 110 for responding to query 302 in response to receiving selected response 306 is not the desired user feedback 308, identifying second selected response 306 as one of responses 304 from second selected VA.

[0077] In addition, the IVAS service 112 can continue to learn and adapt over time to an individual's usage and interaction patterns, as well as usage patterns from other users and associated user feedback 308 ratings. Thus, the IVAS service 112 can automatically select the best VA to handle the user's query 302 without the user having to know the capabilities of each VA. This can reduce poor responses 304 from VA 110 and result in a more seamless experience for the user.

[0078] Fig. 9 An example 900 of a computing device 902 for implementing aspects of an intelligent virtual assistant selection service is shown. Fig. 9 And refer to Figures 1 to 8 , the vehicle 102 or other user device, the TCU 104, the communication network 108, the VA 110, and the IVAS service 112 may be examples of such computing devices 902. As shown, the computing device 902 may include a processor 904 operatively connected to a storage device 906, a network device 908, an output device 910, and an input device 912. It should be noted that this is merely an example, and a computing device 902 having more, fewer, or different components may be used.

[0079] The processor 904 may include one or more integrated circuits that implement the functions of a central processing unit (CPU) and / or a graphics processing unit (GPU). In some examples, the processor 904 is a system on a chip (SoC) that integrates the functionality of the CPU and GPU. The SoC may optionally include other components (such as, for example, a storage device 906 and a network device 908) into a single integrated device. In other examples, the CPU and GPU are connected to each other via a peripheral connection device (such as a peripheral component interconnect (PCI) express) or another suitable peripheral data connection. In one example, the CPU is a commercially available central processing device that implements an instruction set, such as one of the microprocessor (MIPS) instruction set families of x86, ARM, Power, or non-interlocked pipeline stages.

[0080] Regardless of the details, during operation, the processor 904 executes stored program instructions retrieved from the storage device 906. The stored program instructions (such as those of the VA 110, the preference engine 114, the interaction data recorder 116, the VA selector 118, the feedback engine 120, and the collaboration selector 124) include software that controls the operation of the processor 904 to perform the operations described herein. The storage device 906 may include both non-volatile memory and volatile memory devices. Non-volatile memory includes solid-state memory, such as "NAND" (NAND) flash memory, magnetic and optical storage media, or any other suitable data storage device that retains data when the system is disabled or loses power. Volatile memory includes static and dynamic random access memory (RAM) that stores program instructions and data during operation of the system 100. As non-limiting examples, the data may include the ML model 122, the user preferences 200, and the data log 400.

[0081] The GPU may include hardware and software for displaying at least two-dimensional (2D) and optionally three-dimensional (3D) graphics to an output device 910. The output device 910 may include a graphics or visual display device, such as an electronic display screen, a projector, a printer, or any other suitable device for reproducing a graphical display. As another example, the output device 910 may include an audio device, such as a speaker or headphones. As yet another example, the output device 910 may include a tactile device, such as a mechanically elevable device, which in an example may be configured to display Braille or another physical output that may be touched to provide information to a user.

[0082] Input device 912 may include any of a variety of devices that enable computing device 902 to receive control input from a user. Examples of suitable input devices that receive human-computer interface input may include a keyboard, mouse, trackball, touch screen, voice input device, graphics tablet, etc.

[0083] The network devices 908 may each include any of a variety of devices that enable the devices discussed herein to send and / or receive data from an external device over a network. Examples of suitable network devices 908 include an Ethernet interface, a Wi-Fi transceiver, a Li-Fi transceiver, a cellular transceiver, or a Bluetooth or Bluetooth Low Energy (BLE) transceiver or other network adapter or peripheral interconnect device that receives data from another computer or external data storage device, which may be useful for receiving large amounts of data in an efficient manner.

[0084] Although exemplary embodiments are described above, these embodiments are not intended to describe all possible forms covered by the claims. The words used in the specification are descriptive words rather than restrictive words, and it should be understood that various changes can be made without departing from the spirit and scope of the present disclosure. As previously described, the features of various embodiments can be combined to form other embodiments of the present invention that may not be explicitly described or shown. Although various embodiments may have been described as providing advantages or being superior to other embodiments or prior art implementations in terms of one or more desired characteristics, it should be recognized by those of ordinary skill in the art that one or more features or characteristics can be compromised to achieve the desired overall system properties, depending on the specific application and implementation. These properties may include, but are not limited to, strength, durability, life cycle, marketability, appearance, packaging, size, serviceability, weight, manufacturability, ease of assembly, etc. For this reason, in terms of one or more characteristics, to the extent that any embodiment is described as less than other embodiments or prior art implementations are ideal, these embodiments are not outside the scope of the present disclosure and may be desired for specific applications.

[0085] According to the present invention, a system for intelligent virtual assistant selection is provided, which has: an intelligent virtual assistant selection (IVAS) service executed by one or more hardware devices, and the IVAS service is configured to: receive a query from a user device; determine a domain and / or task corresponding to the query; use a collaborative selector to identify a set of similar queries relative to the query; select one of multiple virtual assistants (VAs) to respond to the query based on the similar queries; and reply to the query using a selected response generated by the one of the multiple VAs.

[0086] According to one embodiment, the IVAS service is also configured to: rank the multiple virtual assistants (VAs) based on an average of customer feedback received from executing the similar queries; and select the one of the multiple VAs as the one with the highest average of the customer feedback for responding to the query.

[0087] According to one embodiment, the IVAS service is further configured to: initialize operation of the IVAS service in an explicit mode, in which user preferences are utilized to select the one of the multiple VAs for responding to the query; collect entries in a data log, the entries including the customer feedback on responses received from the multiple VAs; use the data log to train a machine learning (ML) model to infer updates to the user preferences; and update the user preferences to select the VA for future interactions for a specific task and / or domain.

[0088] According to an embodiment, the IVAS service is further configured to exclude from consideration any of the VAs that has not received at least a minimum amount of user feedback.

[0089] According to an embodiment, the IVAS service is further configured to exclude from consideration any of the VAs that have not received at least the minimum average rating score from the customer feedback.

[0090] According to an embodiment, the customer feedback includes ratings of responses to the similar queries.

[0091] According to an embodiment, the IVAS service is also configured to: in response to receiving user feedback that the selected response is not the expected response: select a second VA among the multiple VAs having the second highest average of the customer feedback to respond to the query; identify the second selected response as one of the responses from the second selected VA; and send the second selected response to the user device.

[0092] According to an embodiment, the IVAS service is further configured to: send the query to each of the plurality of VAs; receive a response from each of the plurality of VAs; and identify the selected response as the response from the selected one of the VAs.

[0093] According to an embodiment, the user device is a vehicle.

[0094] According to an embodiment, the user device comprises a mobile phone.

[0095] According to the present invention, a method for selecting an intelligent virtual assistant through an IVAS service includes: receiving a query from a user device; determining a domain and / or task corresponding to the query; using a collaborative selector to identify a set of similar queries relative to the query; ranking multiple VAs based on an average of customer feedback received from executing the similar queries, the customer feedback including ratings of responses to the similar queries; selecting one of the multiple VAs as the one with the highest average of the customer feedback to respond to the query; and replying to the query using a selected response generated by the one of the multiple VAs.

[0096] In one aspect of the invention, the method further includes: initializing operation of the IVAS service in an explicit mode, in which user preferences are utilized to select the one of the multiple VAs for responding to the query; collecting entries in a data log, the entries comprising the customer feedback on responses received from the multiple VAs; using the data log to train a machine learning (ML) model to infer updates to the user preferences; and updating the user preferences to select the VA for future interactions for a specific task and / or domain.

[0097] In one aspect of the invention, the method includes eliminating from consideration any of the VAs that has not received at least a minimum amount of user feedback.

[0098] In one aspect of the invention, the method includes eliminating from consideration any of the VAs that have not received at least a minimum average rating score from the customer feedback.

[0099] In one aspect of the present invention, the method includes, in response to receiving user feedback that the selected response is not an expected response: selecting a second VA among the multiple VAs having the second highest average of the customer feedback to respond to the query; identifying the second selected response as one of the responses from the second selected VA; and sending the second selected response to the user device.

[0100] In one aspect of the invention, the method includes: sending the query to each of the plurality of VAs; receiving a response from each of the plurality of VAs; and identifying the selected response as the response from the selected one of the VAs.

[0101] According to the present invention, a non-transitory computer-readable medium is provided, which has instructions, which when executed by one or more hardware devices of the IVAS service cause the IVAS service to perform operations including the following: receiving a query from a user device; determining a domain and / or task corresponding to the query; using a collaborative selector to identify a set of similar queries relative to the query; ranking multiple VAs based on an average of customer feedback received from executing the similar queries, the customer feedback including ratings of responses to the similar queries; selecting one of the multiple VAs as the one with the highest average of the customer feedback for responding to the query; and replying to the query using a selected response generated by the one of the multiple VAs.

[0102] According to an embodiment, the present invention is also characterized by instructions, which, when executed by the one or more hardware devices of the IVAS service, cause the IVAS service to perform operations including: initializing the operation of the IVAS service in an explicit mode, in which user preferences are utilized to select the one of the multiple VAs for responding to the query; collecting entries in a data log, the entries including the customer feedback on responses received from the multiple VAs; using the data log to train a machine learning (ML) model to infer updates to the user preferences; and updating the user preferences to select the VA for future interactions for a specific task and / or domain.

[0103] According to an embodiment, the present invention is also characterized by instructions, which, when executed by the one or more hardware devices of the IVAS service, cause the IVAS service to perform one or more of the following operations: excluding from consideration any VA among the VAs that has not received at least a minimum number of user feedback; or excluding from consideration any VA among the VAs that has not received at least a minimum average rating score from the customer feedback.

[0104] According to an embodiment, the present invention is also characterized by instructions, which when executed by the one or more hardware devices of the IVAS service cause the IVAS service to perform operations including: sending the query to each of the multiple VAs; receiving a response from each of the multiple VAs; and in response to receiving that the selected response is not the expected user feedback: selecting a second VA among the multiple VAs having the second highest average of the customer feedback to respond to the query; identifying a second selected response as one of the responses from the second selected VA; and sending the second selected response to the user device.

Claims

1. A system for intelligent virtual assistant selection, comprising: An Intelligent Virtual Assistant Selection (IVAS) service executed by one or more hardware devices, the IVAS service being configured to: receiving a query from a user device; determining a domain and / or task corresponding to the query; identifying a set of similar queries relative to the query using a collaborative selector; selecting one of a plurality of virtual assistants (VAs) for responding to the query based on the similar queries; as well as The query is replied to using a selected response generated by the one of the plurality of VAs.

2. The system according to claim 1, wherein the IVAS service is further configured to: ranking the plurality of virtual assistants (VAs) based on an average of customer feedback received from executing the similar queries; and The one of the plurality of VAs is selected as the one having the highest average of the customer feedback for responding to the query.

3. The system according to claim 2, wherein the IVAS service is further configured to: Initializing operation of the IVAS service in an explicit mode in which the one of the plurality of VAs is selected using a user preference to respond to the query; collecting entries in a data log, the entries comprising the customer feedback to responses received from the plurality of VAs; using the data logs to train a machine learning (ML) model to infer updates to the user preferences; as well as The user preferences are updated to select the VA for future interactions for a particular task and / or domain.

4. The system of claim 2, the IVAS service being further configured to exclude from consideration any of the VAs that have not received at least a minimum amount of user feedback.

5. The system of claim 2, wherein the IVAS service is further configured to exclude from consideration any of the VAs that have not received at least a minimum average rating score from the self-customer feedback.

6. The system of claim 2, wherein the customer feedback comprises ratings of responses to the similar queries.

7. The system according to claim 2, wherein the IVAS service is further configured to: In response to receiving user feedback that the selected response was not desired: and selecting a second VA among the plurality of VAs having a second highest average of the customer feedback for use in responding to the query; identifying a second selected response as one of the responses from the second selected VA; and The second selected response is sent to the user device.

8. The system according to claim 7, wherein the IVAS service is further configured to: sending the query to each of the plurality of VAs; receiving a response from each of the plurality of VAs; and The selected response is identified as the response from the selected one of the VAs.

9. The system of claim 1, wherein the user device is a vehicle.

10. The system of claim 1, wherein the user device comprises a mobile phone.

11. A method for intelligent virtual assistant selection via IVAS service, comprising: receiving a query from a user device; determining a domain and / or task corresponding to the query; identifying a set of similar queries relative to the query using a collaborative selector; ranking the plurality of VAs based on an average of customer feedback received from executing the similar queries, the customer feedback comprising ratings of responses to the similar queries; selecting one of the plurality of VAs as the one having the highest average of the customer feedback for use in responding to the query; as well as The query is replied to using a selected response generated by the one of the plurality of VAs.

12. The method according to claim 11, further comprising: Initializing operation of the IVAS service in an explicit mode in which the one of the plurality of VAs is selected using a user preference to respond to the query; collecting entries in a data log, the entries comprising the customer feedback to responses received from the plurality of VAs; using the data logs to train a machine learning (ML) model to infer updates to the user preferences; as well as The user preferences are updated to select the VA for future interactions for a particular task and / or domain.

13. The method of claim 11, further comprising eliminating from consideration any of the VAs that has not received at least a minimum amount of user feedback.

14. The method of claim 11, further comprising eliminating from consideration any of the VAs that have not received at least a minimum average rating score from the feedback from customers.

15. The method of claim 11, further comprising, in response to receiving user feedback that the selected response was not desired: and selecting a second VA among the plurality of VAs having a second highest average of the customer feedback for use in responding to the query; identifying a second selected response as one of the responses from the second selected VA; and The second selected response is sent to the user device.

Images