Techniques for providing explanations for text classification

Abstract

A chatbot system is configured to execute code to perform: determining, by the chatbot system, a classification result for a discourse and determining one or more anchors, each of the one or more anchors corresponding to one or more anchor words in the discourse. For each of the one or more anchors, generating one or more synthetic discourses and determining one or more classification results for the one or more synthetic discourses. Generating, by the chatbot system, a report including a representation of a particular anchor of the one or more anchors, in which the particular anchor corresponds to a highest confidence value. The one or more synthetic discourses can be used to generate a new training dataset for training a machine learning model. The training dataset can be refined according to a confidence threshold to filter out a dataset for training.

Description

[0001] Cross-references to related applications

[0002] This application claims the benefit and priority of U.S. Provisional Application No. 63 / 068,608, filed August 21, 2020, and U.S. Provisional Application No. 63 / 174,336, filed April 13, 2021, pursuant to 35U.SC119(E). The entire contents of the foregoing applications are incorporated herein by reference in their entirety for all purposes. Technical Field

[0003] This disclosure generally relates to chatbot systems, and more specifically to techniques for providing interpretations for text classification. Background Technology

[0004] To achieve immediate responses, many users around the world utilize instant messaging or chat platforms. Organizations frequently use these platforms to engage in real-time conversations with customers (or end users). However, hiring service personnel to communicate with customers or end users in real time can be very expensive for organizations. Therefore, chatbots (hereinafter also referred to as "bots" or "skills") have been developed to engage in conversations with end users, especially via the Internet. End users or customers can communicate with bots through messaging applications that they have already installed and are using. Intelligent bots (typically powered by artificial intelligence (AI)) can communicate more intelligently and context-sensitively in real-time conversations, thus allowing for more natural conversations between bots and end users to improve the conversational experience. Instead of end users learning a fixed set of keywords or commands from which the bot knows how to respond, intelligent bots are able to understand the end user's intent based on natural language utterances and respond accordingly.

[0005] However, chatbots are difficult to build because these automated solutions require domain-specific knowledge and the application of certain technologies that may only be within the capabilities of specialized developers. As part of building such a chatbot, developers can begin by understanding the needs of the enterprise and end-users. Developers can then analyze and make decisions related to, for example: selecting the dataset to be used for analysis; preparing the input dataset for analysis (e.g., cleaning, extracting, formatting, and / or transforming data before analysis, performing data feature engineering, etc.); identifying appropriate machine learning (ML) techniques or (multiple) ML models for performing the analysis; and improving the techniques or models to improve results / effects based on feedback. The task of identifying appropriate models may include developing multiple models (possibly in parallel), iteratively testing and experimenting with these models, and then identifying specific models (or multiple models) for use. Furthermore, supervised learning-based solutions typically involve a training phase, followed by an application (i.e., inference) phase, and an iterative cycle between the training and application phases. Developers can be responsible for carefully implementing and monitoring these phases to achieve the optimal solution.

[0006] Despite their widespread adoption, machine learning models, including neural network models, typically operate as black boxes, performing inference (output) by performing certain mathematical or logical operations on input data. The output of a machine learning model can include, for example, a prediction or classification, and / or a corresponding confidence level. However, the output of a machine learning model, or the output of a particular inference, may not provide insights into the model's behavior (such as why a particular input produces a particular output) to determine whether the model and / or a particular prediction are trustworthy and how to improve the training data and the model. Furthermore, training sets developed for machine learning models are often manually generated and then fed into the model. The time required to train or retrain a machine learning model may be limited by the ability to generate these training sets. There is no comprehensive method to integrate the inference explanation system into the retraining system to improve the machine learning model. Summary of the Invention

[0007] The techniques disclosed herein generally relate to chatbots. More specifically and without limitation, the techniques disclosed herein relate to techniques for providing interpretation for text classification in, for example, chatbots. Chatbots can classify user utterances into different categories, such as a user's predefined intent. The classifier of a chatbot may include a trained ML model that generates an output (e.g., intent) based on an input (e.g., user utterance). Classification is typically a black box, lacking interpretability and explanatory power.

[0008] The techniques disclosed herein provide explanations for classification, enabling chatbot owners to determine whether a classifier and / or a particular classification is trustworthy and how to improve training utterances and classifiers based on these explanations. Various embodiments are described herein, including methods, systems, non-transitory computer-readable storage media storing programs, code, or instructions executable by one or more processors.

[0009] According to various embodiments, a computer-implemented method is provided, the method comprising: receiving, by a chatbot system, a request to interpret an inference result of an intent classifier of the chatbot system on a utterance; obtaining the inference result of the utterance from the intent classifier; selecting one or more anchors based on the utterance, each of the one or more anchors including one or more anchor words in the utterance; for each of the one or more anchors, changing one or more words in the utterance that do not belong to the anchor to generate a variant of the utterance; obtaining an inference result of the variant of the utterance from the intent classifier; determining a corresponding confidence level associated with each of the one or more anchors based on the inference result of the utterance and the inference result of the variant of the utterance; and generating a report including the anchor with the highest confidence level among the one or more anchors.

[0010] In some embodiments, determining the confidence level includes determining a match or mismatch between the inference result for the utterance and each of the inference results for the variants of the utterance.

[0011] In some embodiments, selecting the one or more anchor points includes performing one or more rounds of selection on the one or more anchor points using a cluster search technique.

[0012] In some embodiments, the report includes a JavaScript Object Notation (JSON) file.

[0013] In some embodiments, the report includes anchors with confidence levels greater than a threshold.

[0014] In some embodiments, the computer-implemented method further includes storing the variants of the utterance and the inference results for the variants of the utterance in a database or a cloud-based storage structure.

[0015] According to various embodiments, a computer-implemented method is provided, the method comprising: receiving, by a chatbot system, a plurality of training results, the plurality of training results including a plurality of utterances and a plurality of corresponding confidence values; comparing the plurality of training results with a threshold; based on the comparison, obtaining one or more utterances whose confidence values ​​are less than or equal to the threshold; determining one or more training categories corresponding to the obtained one or more utterances; and training a machine learning model using the one or more utterances based on the one or more training categories.

[0016] In some embodiments, the computer-implemented method further includes determining each of the plurality of corresponding confidence values ​​based on a match or mismatch between a utterance in the plurality of utterances and a corresponding predicted training category of the utterance.

[0017] In some embodiments, the computer-implemented method further includes generating one or more synthetic utterances based on the plurality of utterances from the plurality of training results.

[0018] In some embodiments, the computer-implemented method further includes generating one or more synthetic utterances based on the obtained one or more utterances.

[0019] In some embodiments, determining the one or more training categories includes receiving input indicating a specific utterance in the one or more utterances and a specific corresponding training category in the one or more training categories.

[0020] In some embodiments, the computer-implemented method further includes receiving input indicating the threshold. The input can be received from a user of an interactive interface by inputting the threshold into an interactive interface element.

[0021] In some embodiments, the one or more training categories are one or more correct intentions, each of the one or more correct intentions corresponding to a utterance in the one or more utterances used to retrain the machine learning model.

[0022] In some embodiments, comparing the plurality of training results with a threshold includes calculating at least one or more differences representing the numerical difference between at least one or more confidence values ​​and the threshold, and obtaining the one or more utterances includes selecting utterances whose differences indicate that the confidence value is less than or equal to the threshold.

[0023] In some embodiments, the computer-implemented method further includes grouping the one or more utterances into training batches and inputting the training batches into the machine learning model.

[0024] In some embodiments, the machine learning model is constructed as an intent classifier, which includes multiple model parameters learned by using an objective function, and training the machine learning model includes minimizing or maximizing the objective function, which measures the difference between the predicted intent and the correct intent.

[0025] According to various embodiments, a system is provided that includes one or more data processors and a non-transitory computer-readable storage medium containing instructions that, when executed on the one or more data processors, cause the one or more data processors to perform part or all of the methods disclosed herein.

[0026] According to various embodiments, a computer program product is provided, which is tangibly embodied in a non-transitory machine-readable storage medium and includes instructions configured to cause one or more data processors to perform some or all of the methods disclosed herein.

[0027] In some embodiments, multiple utterances used to train a machine learning model are compared to a confidence threshold. Multiple utterances with confidence values ​​equal to or lower than the confidence threshold are determined. These multiple utterances are presented in a comprehensive user interface for selecting correspondences between the intended intents of specific utterances for retraining the machine learning model. The multiple utterances are modified by a utterance generation engine to form one or more modified utterances for retraining the machine learning model. The one or more modified utterances may also be presented in the comprehensive user interface to select the same or different intended intents for retraining. These utterances may be displayed as part of a comprehensive retraining interface for interpreting intent classification and generating a retraining dataset to improve the machine learning model.

[0028] The technologies and infrastructure disclosed in this paper can allow robot designers to understand why utterances are classified into specific categories by showing the relative importance of words in utterances. This can allow robot designers to improve robot classifiers for future utterance classification.

[0029] The techniques described above and below can be implemented in various ways and in various contexts. Several example implementations and contexts are provided with reference to the following figures, as described in more detail below. However, the following implementations and contexts are only a few of many implementations and contexts. Attached Figure Description

[0030] Figure 1 This is a simplified block diagram of a distributed environment incorporating exemplary embodiments.

[0031] Figure 2Distributed systems according to various embodiments are described, which implement robotic systems for communicating with end users using one or more messaging applications.

[0032] Figure 3 An integrated system according to various embodiments is described, the integrated system including a robotic system and a robot analysis system for monitoring, analyzing, visualizing and improving the performance of the robotic system.

[0033] Figure 4 This is a simplified flowchart illustrating examples of processes for developing robots according to various embodiments.

[0034] Figure 5 This is a simplified flowchart illustrating examples of processes for interpreting inference results according to various embodiments.

[0035] Figure 6 Examples of systems for providing interpretation for text classification, according to various embodiments, are illustrated.

[0036] Figure 7 Examples of systems for providing interpretation for text classification, according to various embodiments, are illustrated.

[0037] Figure 8A Examples of simulated CPU usage during computation interpretation are illustrated according to various embodiments. Figure 8B Examples of analog memory usage during computation interpretation are illustrated according to various embodiments.

[0038] Figure 9A Examples of simulated CPU usage during computation are illustrated according to various embodiments. Figure 9B Examples of analog memory usage during computation interpretation are illustrated according to various embodiments.

[0039] Figure 10A The illustration shows an example of simulated CPU usage during computation interpretation, according to certain embodiments. Figure 10B Examples of analog memory usage during computation interpretation are illustrated according to various embodiments.

[0040] Figure 11 This is a simplified flowchart illustrating examples of processes for retraining a model using human utterance, according to various embodiments.

[0041] Figure 12 The illustration shows example interfaces for retraining models using artificial utterances, according to various embodiments.

[0042] Figure 13 The illustration shows example interfaces for retraining models using artificial utterances, according to various embodiments.

[0043] Figure 14 The illustration shows example interfaces for interpreting inference results according to various embodiments.

[0044] Figure 15 The illustration shows example interfaces for interpreting inference results according to various embodiments.

[0045] Figure 16 The illustration shows example interfaces for interpreting inference results according to various embodiments.

[0046] Figure 17 The illustration shows example interfaces for interpreting inference results according to various embodiments.

[0047] Figure 18 Examples of outputs for interpreting the classification of dialogues according to various embodiments are illustrated.

[0048] Figure 19 A simplified block diagram is depicted as an example of a distributed system for implementing various embodiments.

[0049] Figure 20 This is a simplified block diagram of examples of cloud-based system environments for implementing various embodiments.

[0050] Figure 21 Examples of computer systems used to implement various embodiments are illustrated. Detailed Implementation

[0051] introduction

[0052] This disclosure generally relates to chatbots. More specifically and without limitation, the techniques disclosed herein relate to providing explanations for text classification in chatbots. Chatbots can classify user utterances into different categories, such as certain predefined intentions of the user. Classifiers are typically black boxes, lacking interpretability and explanatory power. Users may know the classification results but may not understand why a particular input was classified into a corresponding category. The techniques disclosed herein can provide explanations for the classifications, enabling improvements to the training of utterances and classifiers based on these explanations, and allowing users to obtain confidence in the classification results. Various embodiments are described herein, including methods, systems, non-transitory computer-readable storage media storing programs, code, or instructions executable by one or more processors.

[0053] Enterprises may wish to create chatbot systems for a variety of purposes. A chatbot system can include one or more user intent classification engines for identifying end-user intent based on user utterances, and one or more dialogue engines for intelligently and context-wise composing messages in response to user utterances based on the determined end-user intent. However, building a chatbot system (including both the user intent classification engine that can determine the end-user's intent based on user utterances and the dialogue engine that intelligently and context-wise generates responses) is a challenging task, partly due to the subtlety and uncertainty of natural language, as well as the dimensionality of the input space (e.g., possible user utterances) and the size of the output space (number of intents).

[0054] A chatbot's classifier may include a trained machine learning (ML) model that infers end-user intent from end-user utterances. ML models typically operate as black boxes, performing certain mathematical or logical operations on inputs (e.g., user utterances) to generate outputs (e.g., intents) based on those inputs. ML models often lack interpretability and explainability, meaning that chatbot owners may lack specific knowledge of how the chatbot was constructed and may not understand why it generates certain outputs given certain inputs. More specifically, the output of an ML model may include, for example, a classification (or another prediction) and / or a corresponding confidence level of that prediction (i.e., how closely the chatbot determines the predicted output intent matches the utterance input), but may not provide insights into the ML model that would allow users to understand its behavior (such as why a particular input produces a particular output) to determine whether the ML model and / or a particular prediction are trustworthy and how to improve the training data and the ML model.

[0055] Furthermore, there is no way to automatically and comprehensively generate and use training data to correct identified behavioral problems in an ML model. For example, given a report, a client might understand how the ML model makes some incorrect predictions, but still perceive an incorrect model. The client might try to correct these problems by seeking new training datasets or attempting to create their own. The client has no way of knowing that any training dataset they find or create will correct the specific problems appearing in the report.

[0056] Therefore, different approaches are needed to address these issues. In various embodiments, a computer-implemented method is provided, the method comprising: receiving a request to interpret the inference results of an intent classifier of a chatbot system on a utterance; obtaining the inference results on the utterance from the intent classifier; selecting one or more anchors, wherein each of the one or more anchors may include one or more anchor words in the utterance; for each of the one or more anchors, changing one or more words in the utterance that do not belong to the anchor to generate a variant of the utterance; obtaining the inference results for the variant of the utterance from the intent classifier; determining a confidence level associated with each of the one or more anchors based on the inference results on the utterance and the inference results for the variant of the utterance; and generating a report including the anchor with the highest confidence level among the one or more anchors. In some embodiments, selecting the one or more anchors may include one or more rounds of selection using a cluster search technique. The variant of the utterance, the inference results for the variant of the utterance, and the report may be in a database.

[0057] The techniques disclosed in this paper provide explanations for classification, enabling chatbot owners to determine whether a classifier and / or a particular classification is trustworthy (in order to take action based on the classification or prediction) and how to improve training utterances and classifiers based on these explanations. For example, an ML model can be retrained using new utterances used when generating reports. The new utterances are similar to the original utterances used to request reports. Therefore, if the intent to be predicted is known given the new utterances as input, the ML model can be retrained using the new utterances to improve future predictions. Because the new utterances have already been generated, and because they are similar to the original utterances used to request reports, the ML model can be retrained in a more efficient and practical way.

[0058] As used herein, "chatbot," "bot," "skill," or "skilled bot" can refer to a computer program designed to simulate conversations with human end-users, especially via the Internet. Individual skills can be designed to interact with end-users and perform specific types of tasks, such as ordering food, making reservations, changing contact information, providing technical support, and offering customer service. Each skill can assist the end-user in completing the task through a combination of visual, audio, or text messages and UI elements such as buttons, tables, lists, etc.

[0059] As used herein, the term "intent" can refer to the category of actions or tasks that an end user expects a skill to perform. The term "entity" can refer to variables that identify information from user input that enables a skill to complete a task. The term "part" can refer to various functions that a skill can use to respond to an end user, such as outputting text, returning information from a backend, and executing custom logic. The term "dialogue flow" can refer to the definition of skill-user interaction and can describe how a skill responds and behaves based on user input. The term "channel" can refer to a platform-specific configuration that allows a skill to access a messaging platform or client messaging application. A single skill can be configured with multiple channels, allowing the skill to run simultaneously on different services or platforms that an end user might prefer to use.

[0060] As used herein, a “utterance” or “message” can refer to one or more sentences exchanged during a session, where a session can refer to a communication session that may include one or more utterances or messages. A session may include one or more stages or states. A session flow can be an abstraction of multiple sessions that include the same stages or states and the same transitions from stage (or state) to stage (or state). Each session can be a specific instance of a corresponding session flow. The state (or stage) of a session (or session flow) can be associated with the state of a state machine maintained by the robotic system for engaging in conversations with other robotic systems or people. In some cases, a state can correspond to the intent or goal of an end user. As used herein, an end user can refer to the end user of the robotic system, such as a person or another entity engaging in conversations with the robotic system through a messaging application or platform. For example, an end user can be a customer or client of a business that owns the robotic system. As used herein, a user of a robotic system can refer to the owner, operator, administrator, or developer of the robotic system.

[0061] In the following description, specific details are set forth for purposes of explanation in order to provide a thorough understanding of the examples of this disclosure. However, it will be apparent, however, that various examples may be practiced without these specific details. The following description is merely illustrative and is not intended to limit the scope, applicability, or configuration of this disclosure. Rather, the subsequent description of the examples will provide those skilled in the art with enabling descriptions for implementing the examples. It should be understood that various changes may be made to the function and arrangement of elements without departing from the spirit and scope of this disclosure as set forth in the appended claims. The drawings and descriptions are not intended to be restrictive. Circuits, systems, networks, processes, and other components may be shown as components in block diagram form to avoid obscuring the examples with unnecessary details. In other instances, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary details to avoid obscuring the examples. The teachings disclosed herein can also be applied to various types of applications such as mobile applications, non-mobile applications, desktop applications, web applications, enterprise applications, etc. Furthermore, the teachings of this disclosure are not limited to a particular operating environment (e.g., operating system, device, platform, etc.), but rather can be applied to multiple different operating environments.

[0062] Furthermore, it should be noted that individual examples can be described as processes depicted as flowcharts, flow diagrams, data flow diagrams, structure diagrams, or block diagrams. While flowcharts can describe operations as sequential processes, many operations can be performed in parallel or simultaneously. Additionally, the order of operations can be rearranged. When the operations of a process are completed, the process terminates, but may have additional steps not included in the diagram. A process can correspond to a method, function, process, subroutine, subprogram, etc. When a process corresponds to a function, the termination of the process can correspond to the function returning to the calling function or the main function.

[0063] The terms “example” or “exemplary” are used herein to mean “serving as an example, instance, or illustration.” Any embodiment or design described herein as “exemplary” or “example” is not necessarily to be construed as being more preferred or advantageous than other embodiments or designs.

[0064] The terms "machine-readable storage medium" or "computer-readable storage medium" include, but are not limited to, portable or non-portable storage devices, optical storage devices, and various other media capable of storing, containing, or carrying instructions and / or data. Machine-readable or computer-readable storage media may include non-transitory media, in which data may be stored and which do not include carrier waves and / or transient electronic signals propagated wirelessly or via a wired connection. Examples of non-transitory media may include, but are not limited to, magnetic disks or magnetic tapes, optical storage media such as optical discs (CDs) or digital versatile discs (DVDs), flash memory, memory, or memory devices. Computer program products may include code and / or machine-executable instructions that may represent processes, functions, subroutines, programs, routines, subroutines, modules, software packages, classes, or any combination of instructions, data structures, or program statements. Code segments may be coupled to another code segment or hardware circuitry by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, independent variables, parameters, data, etc., can be transmitted, forwarded, or transmitted through any suitable means (including memory sharing, message passing, token passing, network transmission, etc.).

[0065] Furthermore, examples can be implemented using hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, program code or code segments (e.g., computer program products) used to perform the necessary tasks can be stored on a machine-readable medium. Multiple processors can perform the necessary tasks. Some of the systems depicted in the figures can be provided in various configurations. In some examples, the system can be configured as a distributed system, where one or more components of the system are distributed across one or more networks in a cloud computing system. Where a component is described as being "configured" to perform certain operations, such configuration can be accomplished, for example, by designing electronic circuits or other hardware to perform the operation, by programming or controlling electronic circuits (e.g., microprocessors or other suitable electronic circuits) to perform the operation, or any combination thereof.

[0066] Robots and Analysis Systems

[0067] A bot (also known as a skill, chatbot, conversational bot, or talkative bot) is a computer program capable of engaging in conversation with an end user. Bots typically respond to natural language messages (e.g., questions or comments) via messaging applications that use natural language messaging. Businesses can use one or more bot systems to communicate with end users through messaging applications. The messaging application (which may be referred to as a channel) can be a preferred messaging application that the end user has already installed and is familiar with. Therefore, the end user does not need to download and install a new application to chat with the bot system. Messaging applications can include, for example, over-the-top (OTT) messaging channels (such as Facebook Messenger, Facebook WhatsApp, WeChat, Line, Kik, Telegram, Talk, Skype, Slack, or SMS), virtual personal assistants (such as Amazon Dot, Echo, or Show, Google Home, Apple HomePod, etc.), native or hybrid extended mobile and web application extensions / responsive mobile or web applications with chat functionality, or voice-based input (such as devices or applications with Siri, Cortana, Google Voice, or other voice input for interaction).

[0068] In some examples, a robot system may be associated with a Uniform Resource Identifier (URI). A URI can identify a robot system using a string of characters. A URI can be used as a webhook for one or more messaging application systems. A URI may include, for example, a Uniform Resource Locator (URL) or a Uniform Resource Name (URN). The robot system may be designed to receive messages from a messaging application system (e.g., a Hypertext Transfer Protocol (HTTP) POST request message). An HTTP POST request message may involve a URI from a messaging application system. In some embodiments, a message may be different from an HTTP POST request message. For example, the robot system may receive messages from a Short Message Service (SMS). While the discussion herein may refer to communications received by the robot system as messages, it should be understood that a message can be an HTTP POST request message, an SMS message, or any other type of communication between two systems.

[0069] End users can interact with robot systems through conversational interactions (sometimes called conversational user interface (UI)), much like human interactions. In some cases, the interaction may include the end user saying "Hello" to the robot and the robot responding with "Hi" and asking how the robot can help. In other cases, the interaction may be transactional with, for example, a banking robot, such as transferring money from one account to another; informational with, for example, an HR robot, such as checking holiday balances; or with, for example, a retail robot, such as discussing returning purchased goods or seeking technical support.

[0070] In some embodiments, a robot system can intelligently handle end-user interactions without interaction with the robot system's administrator or developer. For example, an end-user can send one or more messages to the robot system to achieve a desired goal. Messages may include some form of content, such as text, emojis, audio, images, video, or other methods of conveying the message. In some embodiments, the robot system can convert the content into a standardized form (e.g., using appropriate parameters for a Representational State Transition (REST) ​​call to an enterprise service) and generate a natural language response. The robot system may also prompt the end-user for additional input parameters or request additional information. In some embodiments, the robot system may also initiate communication with the end-user rather than passively responding to end-user utterances. Various techniques are described herein for identifying explicit calls to a robot system and determining the input of the called robot system. In some embodiments, explicit call analysis is performed by a master robot based on the detected call name in the utterance. In response to the detection of the call name, the utterance can be refined for input to the skill robot associated with the call name.

[0071] A conversation with a chatbot can follow a specific conversation flow that includes multiple states. This flow can be defined based on input to determine what will happen next. In some embodiments, the chatbot system can be implemented using a state machine that includes user-defined states (e.g., end-user intentions) and actions to be taken within or between states. The conversation can take different paths based on end-user input, which can affect the chatbot's decisions regarding the flow. For example, in each state, based on end-user input or utterances, the chatbot can determine the end-user's intention in order to determine the next appropriate action to take. As used herein and in the context of utterances, the term "intention" refers to the intention of the user providing the utterance. For example, a user might intend to involve the chatbot in a conversation about ordering pizza, so the user's intention can be expressed through the utterance "Order pizza." User intentions can relate to specific tasks that the user wants the chatbot to perform on their behalf. Therefore, utterances can be expressed as questions, commands, requests, etc., that reflect the user's intention. Intentions can include goals that the end-user wants to accomplish.

[0072] In the context of chatbot configuration, the term "intent" as used herein refers to configuration information used to map a user's utterances to a specific task / action or a specific type of task / action that the chatbot can perform. To distinguish the intent of the utterances (i.e., the user's intent) from the intent of the chatbot, the latter is sometimes referred to as "bot intent." A bot intent can include one or more utterances associated with the intent. For example, the intent to order pizza can have various permutations of utterances expressing the expectation of placing an order for pizza. These associated utterances can be used to train the chatbot's intent classifier so that the intent classifier can subsequently determine whether an input utterance from the user matches the pizza-ordering intent. A bot intent can be associated with one or more conversational flows used to initiate a conversation with the user in a certain state. For example, the first message for the pizza-ordering intent could be the question "What kind of pizza would you like?". In addition to associated utterances, bot intents can further include named entities related to the intent. For example, the pizza-ordering intent can include variables or parameters for performing the task of ordering pizza, such as topping 1, topping 2, pizza type, pizza size, number of pizzas, etc. The values ​​of the entities are typically obtained through conversation with the user.

[0073] Figure 1This is a simplified block diagram of a distributed environment 100 incorporating an exemplary embodiment. The distributed environment 100 includes a Digital Assistant Builder Platform (DABP) 102 that enables enterprises to create and deploy digital assistants for their end users. DABP 102 can be used to create one or more digital assistants (DAs). DABP 102 can be used by multiple enterprises to create digital assistants for their end users. For example, as... Figure 1 As shown, a user 104 representing a specific business can use DABP 102 to create and deploy a digital assistant 106 for the end users of that specific business. For example, a restaurant (e.g., a pizzeria) can use DABP 102 to create and deploy a digital assistant that enables the restaurant's customers to order food (e.g., order pizza).

[0074] For the purposes of this disclosure, "digital assistant" refers to an entity that assists its end user in performing various tasks through natural language conversation. Digital assistants can be implemented using software alone (e.g., a digital assistant is a digital entity implemented using a program, code, or instructions executable by one or more processors), hardware, or a combination of hardware and software. Digital assistants can be embodied or implemented in a variety of physical systems or devices such as computers, mobile phones, watches, appliances, and vehicles. Digital assistants are sometimes also referred to as chatbot systems.

[0075] For example, such as Figure 1 As shown, end user 108 can use digital assistant 106 to perform various tasks through natural language-based conversations with digital assistant 106. As part of the conversation, the user can provide one or more user inputs 110 and receive a response 112 from digital assistant 106. Through these conversations, the user can request one or more tasks to be performed by digital assistant, and in response, digital assistant can perform the requested tasks and respond to the user with an appropriate response.

[0076] User input 110 can be in natural language and can be referred to as speech. User speech can be in text form (e.g., when a user types something as input to the digital assistant 106) or in audio or speech form (e.g., when a user speaks something as input to the digital assistant 106). Speech is typically in the language spoken by the end user 108. When user input is in speech form, the user input can be converted into text speech in the specific language, and then the text speech is processed by the digital assistant 106. Various speech-to-text processing techniques can be used to convert speech or audio input into text speech, which is then processed by the digital assistant 106.

[0077] Text utterances, generated by user input or by converting speech input into text form, can be text fragments, sentences, multiple sentences, etc. Digital Assistant 106 is configured to apply Natural Language Understanding (NLU) technology to the text utterances to understand the meaning of the user input. As part of the NLU processing for the utterances, Digital Assistant 106 is configured to perform processing for understanding the meaning of the utterances, which involves identifying one or more intentions and one or more entities corresponding to the utterances. After understanding the meaning of the utterances, Digital Assistant 106 can perform one or more actions or operations in response to the understood meaning or intention.

[0078] For example, a user input could request to order a pizza, such as, "I want to order a pizza." The digital assistant 106 is configured to understand the meaning of the utterance and take an appropriate action, which could involve responding to the end user by asking them to input the type of pizza they wish to order, its size, any toppings, etc. The response provided by the digital assistant 106 could also be in natural language form, which could involve natural language generation (NLG) processing performed by the digital assistant 106. Once the digital assistant 106 has obtained the necessary information from the user, it can then place the pizza order. The digital assistant 106 can end the conversation with the end user by outputting a message indicating that the pizza has been ordered.

[0079] In some embodiments, utterances received as input by the digital assistant 106 undergo a series of processing steps or a pipeline of processing steps. These steps may include, for example, performing syntactic analysis on the utterances, understanding the meaning of the utterances, refining and reorganizing the utterances to develop a more comprehensible structure, determining the actions to be performed in response to the utterances, performing the actions, generating a response to be output to the end user in response to the user's utterances, and outputting the response to the end user, etc.

[0080] The NLU processing performed by the digital assistant (such as digital assistant 106) may include various NLP-related processes such as sentence parsing (e.g., tokenization, classification by inflectional form, identifying part-of-speech tags, identifying named entities in sentences, generating dependency trees to represent sentence structure, segmenting sentences into clauses, analyzing individual clauses, parsing pronouns, executing chunks, etc.). In some embodiments, NLU processing, or portions thereof, may be performed by digital assistant 106 itself. In some other embodiments, digital assistant 106 may use other resources to perform portions of NLU processing. For example, the syntax and structure of a sentence may be identified by processing the sentence using a parser, a part-of-speech tagger, and / or a named entity recognizer. In one implementation, for the English language, a parser, a part-of-speech tagger, and a named entity recognizer provided by the Stanford Natural Language Processing (NLP) Group are used to analyze sentence structure and syntax. These are provided as part of the Stanford CoreNLP toolkit.

[0081] While the various examples provided in this disclosure illustrate utterances in the English language, this is merely illustrative. In some embodiments, the digital assistant 106 is also capable of handling utterances in languages ​​other than English. In some embodiments, the digital assistant 106 provides subsystems (e.g., components implementing NLU functionality) configured to handle different languages. These subsystems can be implemented as pluggable units that can be invoked from the NLU core server using service calls. This makes NLU processing flexible and scalable for each language, including allowing different processing sequences. Language packs can be provided for each corresponding language, wherein the language packs can register a list of subsystems that can be served from the NLU core server, and, if needed, can also utilize provided general subsystems.

[0082] Digital assistants (such as digital assistant 106) can be made available to their end users through various channels, including but not limited to certain applications (also known as apps), social media platforms, various messaging services and applications, and other applications or channels. A single digital assistant can be configured with several channels, allowing it to run on different services and be accessed through different services simultaneously.

[0083] Digital assistants contain or are associated with one or more skills. In some embodiments, these skills are individual chatbots designed to interact with end users and perform specific types of tasks, such as tracking inventory, submitting time cards, creating expense reports, ordering food, checking bank accounts, making reservations, purchasing widgets, etc. For example, for... Figure 1In the depicted embodiments, the digital assistant 106 includes skill robots 116-1, 116-2, 116-3, etc. As described above, in this disclosure, the terms "skill" and "skills" are used synonymously with the terms "skillbot" and "skill bots," respectively.

[0084] Each skill bot associated with the digital assistant assists the end user in completing tasks through a conversation with the end user, wherein the conversation may include a combination of text or audio input provided by the end user and responses provided by the skill bot. These responses may take the form of text or audio messages to the end user and / or simple user interface elements (e.g., a selection list) presented to the end user for selection.

[0085] There are various methods to add skills or skill bots to a digital assistant. In some instances, a business can develop a skill bot and then add it to a digital assistant using DABP 102. In other instances, a skill bot can be developed and created using DABP 102 and then added to a digital assistant created using DABP 102. In yet another instance, DABP 102 provides an online digital store (called the "Skill Store") that offers multiple skills covering a wide range of tasks. Skills offered through the Skill Store can showcase various cloud services. DABP 102 users can access the Skill Store, select desired skills, and add the selected skills to a digital assistant created using DABP 102. Skills from the Skill Store can be added to the digital assistant as is or in a modified form. For example, a DABP 102 user can select and clone a specific skill bot offered by the Skill Store, customize or modify the selected skill bot, and then add the modified skill bot to a digital assistant created using DABP 102.

[0086] In some embodiments, the digital assistant created and deployed using DABP 102 is implemented using a master robot / secondary (or sub) robot paradigm or architecture. According to this paradigm, the digital assistant is implemented as a master robot that interacts with one or more secondary robots that are skill-based robots. For example, in Figure 1 In the depicted embodiments, the digital assistant 106 includes a master robot 114 and skill robots 116-1, 116-2, 116-3, etc., which are secondary robots of the master robot 114. In some embodiments, the digital assistant 106 itself acts as the master robot.

[0087] Digital assistants implemented using a master-slave robot architecture enable end users to interact with multiple skills through a unified user interface. When an end user engages with the digital assistant, user input is received by the master robot, which then processes the input to identify the user request. Based on the processing, the master robot determines whether the user request can be handled by the master robot itself. If it determines that the user request may not be handled by the master robot, the master robot selects an appropriate skill robot to handle the user request and routes the session to the selected skill robot. This allows the end user to interact with and use several skill robots configured to perform specific tasks through a common, single interface. For example, for a digital assistant developed for an enterprise, the master robot of the digital assistant can interface with skill robots with specific functions, such as a Customer Relationship Management (CRM) robot for performing functions related to Customer Relationship Management, an Enterprise Resource Planning (ERP) robot for performing functions related to Enterprise Resource Planning, and a Human Capital Management (HCM) robot for performing functions related to Human Capital Management. In this way, the end user or customer of the digital assistant only needs to know how to access the digital assistant.

[0088] In a master robot / secondary robot infrastructure, the master robot is configured to know a list of skill robots. The master robot has access to metadata identifying the various available skill robots, and for each skill robot, the capabilities include the tasks that can be performed by the skill robot. Upon receiving a user request in the form of a utterance, the master robot is configured to identify or predict from among the multiple available skill robots which specific skill robot can best serve or process the user request. The master robot then routes the utterance (or a portion of the utterance) to that specific skill robot for further processing. Thus, control flows from the master robot to the skill robot. The master robot can support multiple input and output channels.

[0089] Although Figure 1 The embodiments shown illustrate a digital assistant 106 including a main robot 114 and skill robots 116-1, 116-2, and 116-3, but this is not intended to be limiting. A digital assistant may include various other components (e.g., other systems and subsystems) that provide the functionality of the digital assistant. These systems and subsystems may be implemented solely in software (e.g., code, instructions stored on a computer-readable medium and executable by one or more processors), solely in hardware, or through a combination of software and hardware.

[0090] The DABP 102 provides the infrastructure, along with various services and features, that enable the use of the DABP 102 to create digital assistants that include one or more skill bots associated with the digital assistant. For example, skill bots can be created by cloning an existing skill bot, cloning an existing skill bot and then modifying it, or they can be created from scratch using the tools and services provided by the DABP 102. In some embodiments, the DABP 102 provides a skill store or skill catalog that offers multiple skill bots for performing various tasks. Users of the DABP 102 can clone skill bots from the skill store and create new skill bots.

[0091] DABP 102 also enables users (e.g., skill robot designers) to create skill robots from scratch. In some embodiments, creating a skill robot at a high level involves the following operations:

[0092] (1) Configure settings for the new skill robot;

[0093] (2) Configure one or more intentions for the skill robot;

[0094] (3) Configure entities for one or more intents;

[0095] (4) Training skills robots;

[0096] (5) Create a dialogue flow for the skill bot;

[0097] (6) Add custom parts to the skill robot; and

[0098] (7) Test and deploy skill robots.

[0099] (1) Configuring New Skill Robots — Skill robot designers can specify one or more invocation names for the skill robot being created. These invocation names can be used in utterances to explicitly identify and invoke the skill robot in the digital assistant. Skill robot designers can also specify example utterances for the skill robot. These example utterances represent the utterances of the skill robot. When user input is received, the digital assistant's intent analysis engine compares the user input with these example utterances to determine whether to invoke a specific skill robot.

[0100] (2) Configuring One or More Intents for a Skill Robot—A skill robot designer can configure one or more intents (also called robot intents) for a skill robot being created. These intents identify the tasks the skill robot can perform for the end user of the digital assistant. Each intent is given a name. For example, for a skill robot configured to help users perform various banking transactions, the intent can be specified by the skill robot designer for the skill robot, such as “Check Balance,” “Transfer,” “Deposit Inquiry,” etc. For each intent, the skill robot designer specifies a set of example statements that represent and illustrate the meaning of the intent and are typically associated with the task performed by the intent. For example, for the “Check Balance” intent, example statements could include “What is my savings account balance?”, “How much money is in my current account?”, “How much money is in my account?”, etc. A typical arrangement of user requests and statements can also be specified as example statements for the intent.

[0101] (3) Configuring Entities for One or More Intents of a Skilled Robot — In some instances, additional context may be required for the skilled robot to respond appropriately to user requests. For example, there may be situations where two or more user input utterances are parsed into the same intent in the skilled robot. For example, in the example above, the utterances “What is my savings account balance?” and “How much money is in my current account?” are both parsed into the same “Check Balance” intent, but these utterances are different requests for different things. To clarify such requests, one or more entities are added to the intent. Using the banking skill example, an entity called “Account Type” (which defines values ​​called “Current Account” and “Savings”) enables the skilled robot to perform syntactic analysis on user requests and respond appropriately. One or more entities can be specified for certain intents configured for the skilled robot. Thus, entities are used to add context to the intent itself. Entities help to more fully describe the intent and enable the skilled robot to fulfill the user request. In some embodiments, there are two types of entities: (a) built-in entities provided by DABP 102; and (2) custom entities that can be specified by the skilled robot designer. Built-in entities are general entities that can be used with various robots. Examples of built-in entities include, but are not limited to, entities related to time, date, address, number, email address, duration, cycle time, currency, phone number, URL, etc. Custom entities are used for more customized applications. For example, for banking skills, the "account type" entity could be defined by a skill bot designer to enable various banking transactions by examining keywords entered by the user (such as "current account," "savings," "credit card," etc.).

[0102] (4) Training the Skill Robot—The skill robot is configured to receive user input, perform syntactic analysis, or otherwise process the received user input and identify or select an intent associated with the received user input. To enable this, the skill robot can be trained. In some embodiments, the skill robot is trained based on intents configured for the skill robot and example utterances associated with those intents (collectively, training data), such that the skill robot can parse user input into one of its configured intents. In some embodiments, the skill robot is represented by a model trained using the training data that allows the skill robot to discern what the end user says (or, in some cases, is attempting to say). DABP 102 provides a variety of different training techniques that can be used by skill robot designers to train skill robots, including various machine learning-based training techniques, rule-based training techniques, and / or combinations thereof. In some embodiments, a portion of the training data (e.g., 80%) is used to train the skill robot model and another portion (e.g., the remaining 20%) is used to test or validate the model. Once trained, the skill robot can then be used to process and respond to user utterances. In some cases, the user's utterances may be questions that require only a single response and no further conversation. To handle this situation, a Q&A (Question and Answer) intent can be configured for the skill bot. This allows the skill bot to output a response to the user's request without having to update the dialog box definition. Q&A intents are created in a similar manner to regular intents. However, the dialogue flow for Q&A intents differs from that for regular intents.

[0103] (5) Creating a Dialogue Flow for the Skilled Robot—The dialogue flow specified for the skilled robot describes how it reacts as it parses different intentions in response to received user input. The dialogue flow defines the actions or behaviors the skilled robot will take (e.g., how it responds to user utterances, prompts for user input, returns data, etc.). The dialogue flow is like the process followed by the skilled robot. Figure 1 Skill bot designers specify the dialogue flow using languages ​​such as standard language. In some embodiments, a version of YAML known as OBotML may be used to specify the dialogue flow for the skill bot. The dialogue flow definition for the skill bot acts as a model of the session itself, a model that allows the skill bot designer to carefully orchestrate the interactions between the skill bot and the end users it serves.

[0104] In some embodiments, the dialogue flow definition comprises three parts:

[0105] (a) Contextual part;

[0106] (b) Default transformation section; and

[0107] (c) State section

[0108] The context section—skill bot designers can define variables used in the session flow in the context section. Other variables that can be named in the context section include, but are not limited to: variables for error handling, variables for built-in or custom entities, user variables that enable the skill bot to recognize and persist user preferences, etc.

[0109] The Default Transition section—transitions for the skill bot can be defined either in the dialog flow state section or in the Default Transition section. Transitions defined in the Default Transition section act as fallbacks and are triggered when no applicable transition is defined within the state or when the conditions required to trigger a state transition cannot be met. The Default Transition section can be used to define routes that allow the skill bot to handle unexpected user actions appropriately.

[0110] The state component—the dialogue flow and its associated operations—is defined as a sequence of temporary states that manage the logic within the dialogue flow. Each state node within the dialogue flow definition is named a component, which provides the functionality required at that point in the dialogue. Therefore, states are built around components. States contain component-specific properties and define transitions to other states that are triggered after a component's execution.

[0111] Special cases can be handled using the state section. For example, sometimes it might be desirable to provide the end user with the option to temporarily engage with a first skill within the digital assistant to perform actions within a second skill. In one example, if the end user is busy conversing with the shopping skill (e.g., the user has made some purchase choices), the end user might want to jump to the banking skill (e.g., the end user might want to ensure he / she has enough money for the purchase) and then return to the shopping skill to complete the order. To address this, the action in the first skill can be configured to initiate an interaction with a second, different skill within the same digital assistant and then return to the original stream.

[0112] (6) Adding Custom Parts to the Skill Robot—As described above, the states specified in the skill robot's dialogue flow name parts that provide the required functionality for those states. Parts enable the skill robot to perform functions. In some embodiments, DABP 102 provides a set of pre-configured parts for performing a wide range of functions. The skill robot designer can select one or more of these pre-configured parts and associate them with states in the skill robot's dialogue flow. The skill robot designer can also use the tools provided by DABP 102 to create custom parts or new parts and associate custom parts with one or more states in the skill robot's dialogue flow.

[0113] (7) Testing and Deploying Skilled Robots — DABP 102 provides several features that enable skilled robot designers to test the skilled robots they are developing. The skilled robots can then be deployed and included in digital assistants.

[0114] While the above description outlines how to create skill robots, similar techniques can be used to create digital assistants (or master robots). At the master robot or digital assistant level, built-in system intents can be configured for the digital assistant. These built-in system intents are used to identify general tasks that the digital assistant itself (i.e., the master robot) can handle without invoking the skill robots associated with the digital assistant. Examples of system intents defined for the master robot include: (1) Exit, which applies when the end user signals that they want to exit the current session or context within the digital assistant; (2) Help, which applies when the end user requests help or direction; and (3) Unresolved Intent, which applies to user input that doesn't quite match the Exit and Help intents. The digital assistant also stores information about one or more skill robots associated with it.

[0115] At the master robot or digital assistant level, when an end user inputs a phrase or utterance into the digital assistant, the digital assistant is configured to perform processing to determine how to route the conversation. The digital assistant uses a routing model to determine this, which can be rule-based, AI-based, or a combination thereof. The digital assistant uses the routing model to determine whether the conversation corresponding to the user's input should be routed to a specific skill for processing, processed by the digital assistant or master robot itself according to built-in system intent, or processed into a different state within the current conversation flow.

[0116] In some embodiments, as part of this process, the digital assistant determines whether the user input identifies a skill bot using its invocation name. The invocation name present in the user input can be considered an explicit invocation of a skill bot corresponding to that invocation name. In this scenario, the digital assistant can route the user input to the explicitly invoked skill bot for further processing. In some embodiments, if no specific invocation is made, the digital assistant evaluates the received user input and calculates a confidence score for the system intent and skill bot associated with the digital assistant. The score calculated for the skill bot or system intent indicates how likely the user input represents a task that the skill bot is configured to perform or represents a system intent. Any system intent or skill bot whose associated calculated confidence score exceeds a threshold (e.g., a confidence threshold routing parameter) is selected as a candidate for further evaluation. The digital assistant then selects a specific system intent or skill bot from the identified candidates for further processing of the user input. In some embodiments, after one or more skill bots are identified as candidates, intents associated with those candidate skills are evaluated (according to the intent model for each skill) and a confidence score is applied to each intent. Any intent with a confidence score exceeding a threshold is generally considered part of the candidate stream. If a specific skill robot is selected, the user input is routed to that skill robot for further processing. If a system intent is selected, one or more actions are performed based on the selected system intent.

[0117] As described above, a robot (also known as a bot, chatbot, conversational bot, skill bot, or talkative bot) is a computer program capable of performing conversations with an end user. Bots typically respond to natural language messages (e.g., questions or comments) via messaging applications that use natural language messaging. Businesses can use one or more bot systems to communicate with end users through messaging applications. The messaging application (which may be referred to as a channel) can be a preferred messaging application that the end user has already installed and is familiar with. Therefore, the end user does not need to learn a programming language and download and install a new application to chat with the bot system. Messaging applications can include, for example, over-the-top (OTT) messaging channels (such as Facebook Messenger, Facebook WhatsApp, WeChat, Line, Kik, Telegram, Talk, Skype, Slack, or SMS), virtual personal assistants (such as Amazon Dot, Echo, or Show, Google Home, Apple HomePod, etc.), native or hybrid extended mobile and web applications / responsive mobile or web applications with chat functionality, or voice-based input (such as devices or applications with voice input using Siri, Cortana, Google Voice, or other voice inputs for interaction).

[0118] In some examples, a robot system may be associated with a Uniform Resource Identifier (URI). A URI can identify a robot system using a string of characters. A URI can be used as a webhook for one or more messaging application systems. A URI may include, for example, a Uniform Resource Locator (URL) or a Uniform Resource Name (URN). The robot system may be designed to receive messages from a messaging application system (e.g., a Hypertext Transfer Protocol (HTTP) POST request message). An HTTP POST request message may relate to a URI from a messaging application system. In some embodiments, a message may be different from an HTTP POST request message. For example, the robot system may receive messages from a Short Message Service (SMS). While the discussion herein may refer to the communication received by the robot system as a message, those skilled in the art will recognize that a message can be an HTTP POST request message, an SMS message, or any other type of communication between two systems.

[0119] End users can interact with robotic systems through conversational interactions (sometimes called conversational user interface (UI)), much like human-to-human interactions. In some cases, the interaction may include the end user saying "Hello" to the robot and the robot responding with "Hi" and asking how the robot can help. In other cases, the interaction may be transactional, such as with a banking robot, like transferring money from one account to another. It may also be informational, such as with a human resources (HR) robot, like checking vacation balances. Finally, it may be with a retail robot, such as discussing returning purchased goods or seeking technical support.

[0120] In some embodiments, a bot system can intelligently handle end-user interactions without interaction with the bot system's administrator or developers. For example, an end-user can send one or more messages to the bot system to achieve a desired goal. Messages may include some form of content, such as text, emojis, audio, images, video, or other methods of conveying a message. In some embodiments, the bot system can convert the content into a standardized form (e.g., using appropriate parameters for a Representational State Transition (REST) ​​call to an enterprise service) and generate a natural language response. The bot system can also prompt the end-user for additional input parameters or request other additional information. In some embodiments, the bot system can also initiate communication with the end-user rather than passively responding to their utterances.

[0121] A conversation with the robot can follow a specific conversation flow that includes multiple states. This flow can define what will happen next based on input. In some embodiments, the robot system can be implemented using a state machine that includes user-defined states (e.g., end-user intentions) and actions to be taken in or between states. The conversation can take different paths based on end-user input, which can affect the robot's decisions regarding the flow. For example, in each state, based on end-user input, the robot can determine the end-user's intention in order to determine the appropriate next action to take.

[0122] Intents can include goals that an end user wants to achieve. Intents map end-user input to actions that the backend system can perform for the end user. Therefore, based on phrases spoken by the end user in natural language, the bot can map end-user utterances to specific use cases or tasks, such as ordering pizza, checking account balance, transferring money, making a purchase, receiving a reward, etc. Human conversations are typically non-linear in nature. End users can often be divided into different states during a conversation. For example, if an end user wants to transfer funds from account A to a recipient, the end user can begin a conversation with the bot system by, for example, asking the bot to pay for the recipient's dinner. The bot can respond with, for example, "Which account to use?" The end user might choose a checking account but then realize they are unsure of the account balance. Therefore, the end user can switch contexts to request the balance and recent transactions, etc. In other words, the end user might trigger changes in flow and state, for example, from transferring money to checking the balance and then to recent transactions. At some point, the end user might decide to return to the original intent—to pay the recipient. Therefore, one task of the bot system is to dynamically determine the end-user intent from natural language utterances.

[0123] Robots can use natural language processing (NLP) engines and / or machine learning models (e.g., intent classifiers) to map end-user utterances to specific intents. For example, a machine learning-based NLP engine can learn to understand and classify natural language conversations from end-users and extract necessary information from the conversations to take precise actions, such as executing transactions or retrieving data from backend recording systems.

[0124] Figure 2 A distributed system 200 according to certain embodiments is depicted, which implements a robotic system for communicating with end users using a messaging application. System 200 may include a robotic system 220, one or more messaging application systems 215, and one or more end-user devices (such as one or more mobile devices 210). In some examples, the messaging application may be installed on an electronic device (e.g., a desktop computer, a laptop computer, mobile device 210, etc.). While the discussion herein will refer to mobile devices and messaging applications, those skilled in the art will recognize that any electronic device and any messaging platform or messaging application can be used, such as... Messenger Instant messaging software Mobile text and voice messaging services Messenger Messenger, Skype Messenger, Short Message Service (SMS), or any other messaging application that provides a platform for end-user communication. In other examples, it could be communicated via a browser (such as Google) installed on a mobile device 210. Browser The messaging application can be run using a browser and an Internet Explorer browser. In some embodiments, two or more messaging applications may be installed on the end-user device for communication through two or more messaging platforms (such as two or more messaging application systems 215).

[0125] Messaging applications can be facilitated through messaging platforms such as messaging application system 215. Mobile device 210 can connect to messaging application system 215 via a first network (e.g., the Internet). Messaging application system 215 can be a messaging platform provided by a third party such as Facebook, Tencent, Google, or Microsoft. Messaging application system 215 can manage content sent and received across multiple mobile devices or other end-user devices via messaging applications.

[0126] Robot system 220 (e.g., implemented on one or more servers) can also communicatively connect to messaging application system 215 to send and receive messages. Communication between messaging application system 215 and robot system 220 can be via a second network (e.g., the Internet). The first and second networks can be the same network, or they can be similar or completely different networks. Messaging application system 215 can use the Internet to route content (e.g., messages or information from messages) from mobile device 210 to robot system 220. In some embodiments, the destination of the content (e.g., an identifier of robot system 220) can be included in the content as a nominal recipient. In some embodiments, robot system 220 can also be configured to communicate with two or more messaging application systems 215.

[0127] As discussed above, the content exchanged between end users or between an end user and the robot system can include, for example, text, emojis, audio, media (e.g., images, videos, links), or any other method of conveying a message. The robot system 220 receives content from, for example... Examples of messages received by Messenger may include:

[0128]

[0129] Robot system 220 can receive content from messaging application system 215 using connector 230, which acts as an interface between messaging application system 215 and robot system 220. In some embodiments, connector 230 can normalize content from messaging application system 215, enabling robot system 220 to analyze content across different messaging application systems. Content normalization processing may include formatting content from each type of messaging application into a common format for processing. In some embodiments, for each messaging application (e.g., ...), ... Messenger Instant messaging software Mobile text and voice messaging services Messenger Messenger and Skype Messenger (Short Message Service), robot system 220 may include one or more connectors. In some embodiments, connector 230 may route content to message inbound queue 240. Message inbound queue 240 may include buffers (e.g., first-in-first-out (FIFO) buffers) that store content in the order it is received. In some embodiments, each connector 230 may be associated with one or more message inbound queues.

[0130] When message processor 250 becomes available, message input queue 240 can send content to message processor 250. In some embodiments, message processor 250 can pull content from message input queue 240. As described in detail below, message processor 250 can perform syntactic analysis on messages and determine the intent of the syntactically analyzed messages. In some embodiments, message processor 250 may include natural language processor 252 and intent determination subsystem 254. Natural language processor 252 can perform syntactic analysis on messages and perform some kind of semantic analysis, such as identifying subject, predicate (e.g., action), and / or object. Intent determination subsystem 254 can determine the end-user intent based on the syntactically analyzed message. As described above, intent may include the purpose of the message. For example, the purpose of the message may be ordering pizza, ordering a computer, transferring money, asking questions about delivery, etc. In some embodiments, parameters associated with the intent (which may be referred to as entities) that may be more specifically defined or clarified as the action to be taken can also be extracted from the message by natural language processor 252 and / or intent determination subsystem 254.

[0131] After message processor 250 determines the end-user intent based on the content, the determined intent (and parameters associated with the intent) can be sent to action engine 260. Action engine 260 can be used to determine the action to be performed based on the intent (and parameters associated with the intent) as described above and the current state (or context) of the state machine. For example, action engine 260 can send some outbound content as a response to message output queue 270 and / or can send commands to or retrieve information from some enterprise service (e.g., enterprise service 225). Message output queue 270 can send the outbound content to connector 230. Connector 230 can then send the outbound content to a messaging application system indicated by action engine 260, which may be the same as or different from messaging application system 215. Messaging application system 215 can then forward the outbound content to a messaging application on mobile device 210.

[0132] Robot system 220 can communicate with one or more enterprise services (e.g., enterprise service 225), one or more storage systems for storing and / or analyzing messages received by robot system 220, or content systems for providing content to robot system 220. Enterprise service 225 can communicate with one or more connectors 230, motion engine 260, or any combination thereof. Enterprise service 225 can communicate with connector 230 in a manner similar to messaging application system 215. Enterprise service 225 can send content to connector 230 to associate with one or more end users. Enterprise service 225 can also send content to connector 230 to cause robot system 220 to perform operations associated with end users. Motion engine 260 can communicate with enterprise service 225 to obtain information from enterprise service 225 and / or instruct enterprise service 225 to take actions recognized by motion engine 260.

[0133] In some embodiments, the robot system 220 may include one or more timers. The timers may cause the motion engine 260 to send content to the end user after a certain period of time using the connector 230 and the messaging application system 215. In some embodiments, the timers may send content to the robot system 220 in a manner similar to that of an end user or enterprise service 225. For example, the timers may send messages to the robot system 220 for analysis, as would be done with messages from the end user.

[0134] In one particular embodiment, an end user can send a message to the robot system 220 via a messaging application system 215 using a mobile device 210. The message may include a greeting such as “Hello” or “Hi”. The robot system can determine that a new session with the end user has begun and initiate a state machine. In some embodiments, the robot system can identify one or more characteristics of the end user. For example, the robot system can identify the end user's name using a profile associated with the end user via the messaging application system. Using said one or more characteristics, the robot system can respond to the end user via the messaging application. The response may include a message addressed to the end user that responds to a message received from the end user. For example, the response may include a greeting with the end user's name, such as “Hi, Tom, what can I do for you?”. Depending on the business associated with the robot system, the robot system may continue to achieve the business's goals. For example, if the robot system is associated with a pizza delivery business, the robot system may send a message to the end user asking if the end user wants to order pizza. The session between the robot system and the end user can continue from there, back and forth, until the robot system completes the session or the end user stops responding to the robot system.

[0135] In some embodiments, the robot system can initiate a session with the end user. The session initiated by the robot system can be in response to a previous session with the end user. For example, the end user might have ordered a pizza in a previous session. The robot system can then initiate a new session once the pizza is ready. In some embodiments, the robot system can determine that the pizza is ready when it receives an instruction from the enterprise associated with it (e.g., an employee sends a message to the robot system that the pizza is ready). The session may include a message sent to the end user indicating that the pizza is ready.

[0136] In some embodiments, the robot system can send a message to the end user that a previously received message has been sent via a different messaging application than the messaging application itself. For example, the robot system might determine to use Short Message Service (SMS) instead of... Messenger is used to send messages. In this implementation, the robot system can integrate multiple messaging applications.

[0137] In some embodiments, the robotic system may determine the start of a session based on a timer. For example, the robotic system may determine to schedule a one-week timer for the end user after a pizza has been ordered. The expiration of the one-week timer may cause the robotic system to start a new session with the end user for ordering another pizza. The timer may be configured by the enterprise and implemented by the robotic system.

[0138] As described above, in some embodiments, the motion engine 260 may send commands to or retrieve information from enterprise services 225. For example, when the robot system 220 (more specifically, message processor 250) determines an intent to check a balance, the robot system 220 may determine which of several accounts (e.g., current or savings accounts) to check. If the end user enters “What is the balance in my savings account?”, the robot system 220 may retrieve “Savings” and send a command to the bank server to check the balance and then send the received balance information to the end user via a message. If the end user initially only says “What is the balance in my account?”, the robot system 220 may send a message prompting the end user to further specify a particular account, or it may retrieve information on all of the end user's accounts and send the account information to the end user for selection.

[0139] In some embodiments, the robot system can maintain information between sessions. This information can be used later, eliminating the need for the robot system to ask questions every time a new session begins between the end user and the robot system. For example, the robot system can store information about the end user's previous pizza orders. In a new session, the robot system can send a message to the end user asking if they want the same order as before.

[0140] In some embodiments, the robot system 220 may store information associated with the end user in a cache. After an outbound message is sent from connector 230 to the messaging application system, the cache may be written to a database to save the information. In other embodiments, the cache may write data at different times (e.g., after a specific event, after each event, after a certain period of time, or any other metric used to determine when to write to the database).

[0141] When deceleration is detected, robot system 220 can allow scaling of each component. For example, if robot system 220 detects that the number of messages arriving at connector 230 exceeds a threshold, one or more additional connectors can be added to connector 230. Additionally, the number of message input queues, message processors, motion engine instances, and message output queues can increase depending on the location of the deceleration. In this implementation, additional components can be added without adding other additional components. For example, connectors can be added without adding additional instances of the motion engine. In some implementations, one or more components or a portion of components of robot system 220 can run on a virtual machine. By running on a virtual machine, additional virtual machines can be started as desired.

[0142] As described above, building bot systems (such as user intent classification engines that can determine end-user intent based on end-user utterances) is a challenging task, partly due to the subtlety and ambiguity of natural language, as well as the dimensionality of the input space (e.g., possible end-user utterances) and the size of the output space (number of intents). Thus, improving the performance and user experience of a bot system may require monitoring, debugging, and modification of new bot systems. In many cases, without the use of analytics or optimization tools, it can be difficult to identify more specifically the root causes of a bot system's performance falling short of expectations and to determine how to improve it.

[0143] In some cases, robot owners, developers, or administrators may want to monitor the robot's operational status and understand how it is being used and where end users abandon it in order to improve the robot. For example, robot owners, developers, or administrators may want to know which robot sessions are successful and which are unsuccessful in order to identify and diagnose elements that are causing performance issues with the robot system.

[0144] According to some embodiments, the analysis system can be integrated with the robot system. The analysis system can monitor events occurring during sessions between the end user and the robot system, aggregate and analyze the collected events, and graphically provide information about the sessions at different levels of generalization (e.g., all sessions, different categories of sessions, and individual sessions) on a graphical user interface. For example, the graphical user interface can display options for filtering or selecting certain types of sessions or individual sessions and graphically display the selected information, for example, by visualizing the path of the session. The analysis system can also provide suggestions, options, or other information to improve the robot system.

[0145] Figure 3An integrated system 300 according to certain embodiments is depicted, comprising a robot system (such as robot system 220) and a robot analysis system for monitoring, analyzing, visualizing, and improving the performance of the robot system. As illustrated, the robot system may include a connector 330 and multiple robot engines 310 such as a dialogue engine 312, an intent modeler 314, an entity parser 316, and custom components 318. The robot system may also include a database 340, a management application programming interface (API) 350, a user interface 354, and a UI server 352. The robot analysis system may include a collector 355, an enrichment engine 360, a database 370, and a REST server 380. The robot analysis system may also include a user interface 392 and a UI server 390. The collector 355 of the robot analysis system may collect events 305 occurring at the robot system. Feedback 394 from the robot analysis system may be provided to the robot system through the user interface 392 and the user interface 354.

[0146] Connector 330 can act as an interface between the robot system and one or more end users through one or more channels (such as channels 320 and 322). Each channel can be a messaging application, such as a messaging channel (e.g., Facebook Messenger, Facebook WhatsApp, WeChat, Line, Kik, Telegram, Talk, Skype, Slack, or SMS), a virtual personal assistant (e.g., Amazon Dot, Echo, or Show, Google Home, Apple HomePod, etc.), a native or hybrid extended mobile and web application extension / responsive mobile or web application with chat functionality, or voice-based input (e.g., a device or application with voice input using Siri, Cortana, Google Voice, or other voice inputs for interaction). In some embodiments, connector 130 can normalize content from different channels, allowing the robot system to analyze content across different messaging application systems. Content normalization processing can include formatting content from each type of messaging application into a common format for processing. In some embodiments, the robot system can include one or more connectors for each channel.

[0147] Intent modeler 314 can be used to determine end-user intents associated with end-user utterances. In some embodiments, intent modeler 314, used to determine end-user intents based on one or more messages received by the robot system from the end-user, can use a natural language processor to label lemmas (verbs, nouns, adjectives), look up lemmas / stems (runs / running / ran->run) and label entities (Texas->LOCATION). In some embodiments, intent modeler 314 can normalize messages. For example, “Mary ran to Texas” can become “PERSON ran to LOCATION”. The intent modeler can also include logic for detecting words with the same meaning in end-user messages. For example, if the training dataset includes “Mary ran to Texas” and “Bob walked to Detroit”, both of which map to the same intent and run / walk appear in the same set of intents, then intent modeler 314 can learn that, for intent parsing purposes, run = walk. In an illustrative example, "Mary ran to Texas" can be transformed into "PERSON ran to LOCATION," and "Bob walked to Detroit" can be transformed into "PERSON walked to LOCATION." In the illustrated example, both sentences can be associated with the same intent because, for intent resolution purposes, "noun ran to noun" is the same as "noun walked to noun." In another example, "I want to order a large cheese pizza" and "I want to order a small pepperoni pizza" can both be normalized to "I want to order Bots_PizzaSize Bots_Toppings(robot_pizzasize_robot_toppings) pizza."

[0148] After normalization, the probability that the occurrence of a word indicates a certain intention can be determined. In some examples, basic probabilistic algorithms can be used to combine probabilities, as if the probabilities were independent. For example, if "order" implies a 20% probability of ordering pizza and "pizza" implies a 10% probability of ordering pizza, the total probability would be 1 - (1 - 0.2)(1 - 0.1) = 28%. Some probabilities can be based on the presence of a word or on certain linguistic elements (such as negation or the presence of personal pronouns).

[0149] Another rule level could be template rules, which are combinations of words. In some examples, every sentence in the training dataset can automatically become a rule once normalized. In such examples, the training dataset could include a very small number of short sentences. Template rules can return a probability of 1. New rules can be generated from the existing rules through an inductive process. For example, the following sentences could belong to tracking spending: “How much did I spend on gasoline last month?” and “How much did I spend on food in May?” Sentences can be used to inductively summarize the rule for “How much did I spend?” because this is a shared part between them. In other examples, the training dataset could include the phrase “How much did I spend?” to achieve the same result.

[0150] The example described above allows the definition of intent to exclude duplicates, such as variations in named entities (e.g., "remit money to Sue" and "remit money to Bob"). However, it can be trained using similar sentences that differ in one or two words. Similar sentences allow the model to learn which words are likely to have the same intent parsing meaning and which words are likely to be common spelling errors.

[0151] If a particular word or set of words (such as a verb) is important to an intent, the probability can be manipulated by making more examples use that word (and its synonyms) for that intent and fewer examples use that word for other intents.

[0152] Examples can also be provided to prevent the model from making incorrect assertions. For example, specific subphrases or words that appear only for a particular intent may lead to incorrect assertions. Similarly, it is possible to prevent the model from synthesizing broad rules that use similar sentences belonging to different intents for training.

[0153] Entity parser 316 can identify entities (e.g., objects) associated with end-user intents. For example, in addition to end-user intents (such as “order pizza”) identified by intent modeler 314, entity parser 316 can also parse entities associated with intents, such as pizza type and toppings.

[0154] The dialogue engine 312 can be used to handle conversations between an end user and a robotic system. For example, the dialogue engine 312 can respond to end user utterances based on end user intentions identified by the intention modeler 314 and entities associated with those intentions identified by the entity parser 316. In some embodiments, the dialogue engine 312 can use a state machine that includes user-defined states (e.g., end user intentions) and actions to be taken in or between states to handle conversations with the end user.

[0155] Custom component 318 may include custom modules for a specific robot system. For example, a financial robot may include custom components that can be used for, for example, checking balances, transferring funds, or paying bills.

[0156] Database 340 can be used to store data from the robot system, such as classification model data and session logs. The management API 350 can be used by the robot system's administrators or developers to manage the system, such as retraining the classification model, editing intents, or otherwise modifying the robot system. Administrators or developers can use the user interface 354 and the UI server 352 to manage the robot system.

[0157] Various events can be generated during the operation of a robot system. Events can be generated based on one or more instructions included in the robot system. For example, an event can be generated when the robot system has entered a specific state, defined by the robot system's administrator or developer. Upon event generation, the event can be collected, stored, and analyzed by a robot analysis system. When an event is captured, additional information associated with the event can also be collected, indicating the current context in which the event was generated.

[0158] For example, session events can be generated by the dialogue engine 312. Session events can include messages (referred to as msg_received) received by the robot system from the end-user device. msg_received can include one or more of the following parameters or variables: the content of the message, the time the message was received by the robot system, the language of the received message, device characteristics (e.g., version or name), operating system characteristics (e.g., version or name), geographic location characteristics (e.g., Internet Protocol address, latitude, longitude, etc.), identification information (e.g., user ID, session ID, robot system ID, tenant ID, etc.), timestamps (e.g., timestamps created by the device, timestamps sent by the device, timestamps obtained by the collector), channels, etc.

[0159] Session events may also include messages (referred to as msg_sent) sent by the robot system to the end-user device. msg_sent may include one or more of the following: the content of the message (e.g., the text or HTML of the message), the time the message was sent by the robot system, the language of the message, the creator of the message (e.g., the robot system or the end-user device), the device nature, the operating system nature, the browser nature (e.g., version or name), the application nature (e.g., version or name), the geographic location nature (e.g., Internet Protocol address, latitude, longitude, etc.), identification information (e.g., user ID, session ID, robot system ID, tenant ID, etc.), the channel (e.g., Facebook or Webhook), etc.

[0160] The dialogue engine 312 can also generate dialogue state execution events. As described above, the dialogue engine 312 can use a state machine to determine the conversation flow with the end user. The state machine can include a set of states and transition rules between states. The dialogue engine 312 can execute the state machine for each end user session, and dialogue state execution events can be generated for each state processed by the dialogue engine 312 sentence by sentence to handle the end user's utterance. Attributes of dialogue state execution events can include, for example, state name, component name, next action, entity match, intent match, variable, user query, response, execution time, communication language, device nature, operating system nature, geographic location nature, identification information, timestamp, channel, etc. The state name can be the name of the currently executing state or an "error state". The component name can be the name of the robot component executing for the current state. The next action can be the next action to be executed. The entity match can be the entity resolved in the current message. The intent match can be the intent resolved with a score value. The variable can be the variable value of the current state. The query can be a message sent by the end user. The response can be a message sent to the end user. The execution time can be a timestamp of the completed state execution. The language of communication can be the language of the messages exchanged. Device characteristics and / or operating system characteristics can be associated with the end user interacting with the robot system. Browser characteristics and / or application characteristics can be associated with the end user interacting with the robot system. Geographic location characteristics can be the location of the end user interacting with the robot system.

[0161] Intent parsing events may occur due to the execution of intent modeler 314. Intent modeler 314 can identify end-user intents from a set of intents based on end-user utterances using a trained or otherwise defined classification model. The results of intent classification can be captured as intent parsing event attributes, which may include, for example, the final intent classification result (e.g., the identified intents) and a confidence score associated with each corresponding intent in the set of intents.

[0162] Entity parser 316 can generate entity parser events. An entity is an object associated with an end-user intent. When a robot system is created, entity definition rules can be determined. For example, in addition to parsing end-user intents such as "order pizza," the robot system can also use entity parser 316 to parse associated entities such as pizza size and toppings. Entity parser events can be captured during entity parsing. Examples of attributes associated with entity parser events may include entity name, applied rules, search terms, parsed state, query statement, entity type, execution time, communication language, device nature, operating system nature, browser nature, application nature, geolocation nature, identification information, timestamp, channel, etc. Entity name can be the name of the currently parsed entity. Applied rules can be, for example, preceding, following, or aggregated. Search terms can be from, to, destination, origin, etc. Parsed state can be the dialogue state for entity parsing. Query statement can be a message containing entity values. Entity type can be system or obtained. Execution time can be the timestamp of entity parsing. Communication language can be the language of the conversation message. Device properties and / or operating system properties can be associated with the end user interacting with the robot system. Browser properties and / or application properties can be associated with the end user interacting with the robot system. Geographic location properties can be the location of the end user interacting with the robot system.

[0163] Custom components can also generate events such as predefined events or custom events. Predefined events can be properties captured when a custom component is executed. Examples of predefined event attributes can include: component name, event name, payload, execution time, communication language, device property, operating system property, browser property, application property, geolocation property, identification information, timestamp, channel, etc. The component name can be the name of the currently executed custom component. The event name can be invoked, invocation_failed, replied, replied_failed, etc. The payload can be the reason for the failure (in the case of failure), stack trace, etc. The execution time can be a timestamp indicating when the event occurred. The communication language can be the language of the message being communicated. Device properties and / or operating system properties can be associated with the end user interacting with the robot system. Browser properties and / or application properties can be associated with the end user interacting with the robot system. The geolocation property can be the location of the end user interacting with the robot system.

[0164] Custom components can also issue custom events during their execution. Examples of custom event attributes can include component name, event name, custom payload, execution time, communication language, device nature, operating system nature, browser nature, application nature, geolocation nature, identification information, timestamp, channel, etc. The component name can be the name of the currently executing custom component. The event name can be a user-defined event name (e.g., Balance_Retrieved). For example, the payload could be {"amount":"100 USD100","account":"checking"}. The execution time can be a timestamp indicating when the event occurred. The communication language can be the language of the message being communicated. Device nature and / or operating system nature can be associated with the end user interacting with the robot system. Browser nature and / or application nature can be associated with the end user interacting with the robot system. The geolocation nature can be the location of the end user interacting with the robot system.

[0165] Error events and timeout events can also be generated by the robot system during execution. An error event can be generated when an error occurs. A timeout event can be generated when an end-user session has been inactive for a period of time; these timeout events can be configured at the channel level.

[0166] When the robot system engages in a dialogue with the end user and generates corresponding events, the robot analytics system can collect these events and additional information. For example, collector 355 can collect these events and additional information and send the collected information to a queue. In some embodiments, collector 355 can be configurable and programmed to collect the various events and / or event attributes described above as desired. For example, collector 355 can be configured to capture dialogue state attributes, intent resolution attributes, entity resolution attributes, and error and timeout attributes. In some embodiments, collector 355 can also be configured to collect information about events 395 generated by systems other than the robot system.

[0167] The enrichment engine 360 ​​can perform validation and enrichment on the collected events and other information and write the collected events and other information to the database 370. For example, based on the collected IP addresses, the enrichment engine 360 ​​can determine the location of the end user associated with the IP address. As another example, the enrichment engine 360 ​​can extract certain features from the collected information, such as determining the web browser or channel used by the end user. The REST server 380 can analyze the enriched events and other information and generate various reports based on a certain aggregate metric 372. The reports can be displayed to the owner, administrator, or developer of the robot system on the user interface 392 through the UI server 390. The owner, administrator, or developer of the robot system can provide feedback 394 to the robot system for improvement.

[0168] Figure 4 This is a simplified flowchart 400, illustrating an example of a process for developing skills according to certain embodiments. The process may include creating an intent at 410, training the skill at 420, creating an entity at 430, integrating custom components at 440, creating a dialogue flow at 450, testing the skill at 460, routing to a channel at 470, and reviewing insights reports for improving the skill at 480.

[0169] At point 410, the intent of a skill can be created. An intent describes the various actions a skill can help its end user perform. For example, if a skill enables a user to perform various banking transactions, the intent of the skill could include, for instance, "check balance" or "transfer funds." Intents not only describe what a skill can do, but can also be a component of the skill's intelligence. Intents enable a skill to recognize user input because each intent can have a set of typical user statements (i.e., utterances) associated with it. While these utterances can share the same meaning, they can be different (e.g., "What is my savings account balance?" and "How much money do I have in my current account?").

[0170] At point 420, when a skill performs syntactic analysis on user input, it can be trained to infer user intent. Specifically, the skill can be trained using intents and their utterances (collectively referred to as training data) so that it can parse user input into intents within intents. The trained skill can not only recognize sample phrases belonging to each intent, but also similar phrases corresponding to each intent.

[0171] At 430, an entity for the skill can be created. In some embodiments, a skill may require a different context to fulfill a user request. While some user requests may be parsed into the same intent (e.g., “What is my savings account balance?” and “How much money is in my current account?” will both be parsed into the “Check Balance” intent), they request different things. To clarify the request, one or more entities can be added to the intent. Using the banking skill example, the entity “Account Type” (which defines values ​​such as “current account” and “savings”) enables the skill to perform syntactic analysis on the user request and respond appropriately.

[0172] At point 440, custom widgets can be integrated into skills. Before integrating widgets, a skill can recognize user input but may not be able to respond to it. Widgets enable skills to perform their functions. Widgets can perform functions (such as outputting text based on intent parsed from the end user's message) or perform skill-specific tasks (such as checking account balance).

[0173] At position 450, a dialogue flow can be created. The dialogue flow describes how the skill reacts when different intentions are parsed. The dialogue flow defines the actions or behaviors that the skill bot will take (such as how the skill bot responds to user utterances, how the skill bot prompts the end user for input, how the skill bot returns data, etc.). In some embodiments, the dialogue flow can be a flowchart described in a simple markup language such as YAML (e.g., OBotML).

[0174] At point 460, skills can be tested after development and deployment in, for example, a digital assistant as described above. Skills can be tested by engaging in one or more conversations with the skill using one or more dialogue streams.

[0175] Optionally, at 470, skills can be routed to one or more channels used for user messaging and other capabilities. For example, if the skill is not added to the digital assistant, it can be added to one or more user messaging channels. Users can chat with the skill through one or more of these user messaging channels (such as various messaging platforms, proprietary messaging applications, and web pages). Skills will run on any of these channels, whether they are plain text or can support scrolling cards, UI elements, images, and other non-text content. In addition to channels connected to the user interface, skills can also be routed to other channels (such as channels that link skills to customer support systems or channels that route notifications from external applications that prompt the skill to initiate a session).

[0176] At point 480, developers can review insight reports to improve skills. For example, developers can review skill-related insight reports to determine whether customers are using the skill as intended. Insight reports can include, for example, high-level usage metrics and session trends, individual views of intents, their execution paths, and session transcripts. Insight reports can provide different perspectives on how well the skill supports its customers and where the skill prevents customers from completing tasks. These insight reports not only allow developers to quickly identify problems but also suggest ways to improve the interpretation of user input and intents in the skill.

[0177] In some embodiments, different versions of a skill may be generated. In some embodiments, a skill may be a composite skill that includes two or more related skills to perform a more complex function as described above. For example, composite skill A may include skill B and skill C, wherein, for example, the output from skill B may be used as input to skill C.

[0178] As mentioned above, robots can use machine learning models (e.g., intent classifiers) to map end-user utterances to specific intents. Despite their widespread adoption, machine learning models, including neural network models, typically operate as black boxes, performing reasoning (output) by performing certain mathematical or logical operations on input data. The output of a machine learning model can include, for example, a prediction or classification, and / or a corresponding confidence level. However, the output of a model or specific inference may not provide insights into the model, allowing users to understand its behavior and the reasons behind the reasoning. Therefore, users may find it difficult to determine whether a model and / or specific inference is trustworthy, enabling them to make decisions or take actions based on the reasoning or to choose whether to deploy the model. For example, when using machine learning for medical diagnosis or terrorism detection, one might not blindly trust predictions and take action because the consequences could be catastrophic. A holistic evaluation of the model is also necessary before deployment. Currently, models are typically evaluated using accuracy metrics or other statistics based on available validation datasets. However, real-world data often varies significantly, and furthermore, evaluation metrics may not represent the product's objectives.

[0179] Regarding classifiers for robotic systems, robot owners or developers may want to understand, for example, why a classifier infers a specific user intent from a particular user utterance, and different user intents from different user utterances, before deploying a classifier in a robotic system. Gaining insights into the classifier can also help robot developers reduce the amount of training data (e.g., training utterances) used to train the classifier, or more intelligently generate more useful training data to improve the classifier.

[0180] According to some embodiments, the robotic system may include an intent classifier and an inference interpretation engine. The inference interpretation engine can provide insights into the inferences made by the intent classifier about individual user utterances. For example, the inference interpretation engine can identify certain words or combinations of words that cause the intent classifier to make a certain inference from the utterance with a certain confidence level. The inference interpretation engine may include, for example, a local agent model or a scope rule model.

[0181] The surrogate model is trained to approximate the predictions of the underlying black-box model. The purpose of an interpretable surrogate model is to approximate the predictions of the underlying model as accurately as possible using an interpretable surrogate model (e.g., a prediction function). The surrogate model can be obtained by: (1) selecting a dataset, which may be the same dataset used to train the underlying black-box model or a subset of the same dataset, or a new dataset from the same distribution; (2) generating predictions using the underlying black-box model for the selected dataset; (3) selecting an interpretable model type (e.g., a linear model, a decision tree, etc.); (4) training the interpretable model using the selected dataset and the predictions made by the underlying black-box model to generate the surrogate model; and (5) determining how well the surrogate model replicates the predictions of the underlying black-box model, and retraining the surrogate model if necessary. The trained surrogate model can then be interpreted.

[0182] Locally Interpretable Model-Independent Explanation (LIME) does not train a global surrogate model, but focuses on training a local surrogate model to explain individual predictions. In LIME, the goal is to understand why a machine learning model makes a certain prediction. LIME generates a new dataset that includes variants of the input samples (e.g., permutation samples) and corresponding predictions made by the underlying machine learning model. This new dataset is then used to train an interpretable model. The interpretable model can be, for example, a minimum absolute shrinkage and selection operator (Lasso) or a decision tree. The trained interpretable model can be a good approximation (locally faithful) of the underlying machine learning model's predictions of variants of the input samples, but does not need to be a good global approximation of all input samples. The local surrogate model can be obtained by: (1) selecting samples of interest that may require explanation of the predictions of the underlying machine learning model; (2) perturbing the samples of interest and generating predictions for the perturbed samples using the underlying machine learning model; (3) weighting the perturbed samples according to their proximity to the samples of interest; and (4) training a weighted interpretable model using the perturbed samples and the corresponding predictions made by the underlying machine learning model. The predictions of the underlying machine learning model for samples of interest can then be explained by interpreting the trained interpretable model.

[0183] An example of a scoped rule model is the anchoring technique. Anchoring techniques can explain individual predictions of a black-box machine learning model (e.g., a classifier) ​​by finding decision rules that sufficiently “anchor” the predictions. The rule anchors the predictions when changes in other feature values ​​do not affect the prediction. One example anchoring technique can leverage reinforcement learning techniques combined with graph search algorithms to minimize the number of model calls (and therefore runtime) while still being able to recover from local optima. Anchoring techniques utilize perturbation-based strategies to generate local explanations for the predictions of the black-box machine learning model. In anchoring techniques, instead of using surrogate models, the resulting explanations are expressed using easily understood IF-THEN rules. These rules are scoped and reusable. For example, anchoring might include the concept of coverage, which specifies other (potentially unseen) instances to which the rule might apply.

[0184] Finding anchors may involve solving an exploration or multi-armed slot machine (MAB) problem, which originates from the discipline of reinforcement learning. To find an anchor, neighbors or perturbations can be created for the instance to be explained in one or more rounds, and these neighbors or perturbations can be evaluated. In each round, perturbations that match the currently observed rule (e.g., the anchor) can be created and evaluated by calling the model. MAB can be used to efficiently explore and exploit different policies (called arms, similar to a coin-operated machine) using sequential selection. In a given setting, each candidate rule is considered an arm that can be pulled. Each time a candidate rule is pulled, its corresponding neighbors are evaluated, and thus more information about the utility of the candidate rule (e.g., precision, indicating how well the rule describes the instance to be explained) can be obtained. More samples can be used when no candidate rule has a utility exceeding a certain threshold of statistical confidence. In each subsequent round, the best candidate from the previous round can be expanded using additional feature predicates not yet included in it.

[0185] The above process can be combined into a beam search process, which is a graph search algorithm and a variant of the breadth-first search algorithm. The beam search process carries the B best candidate rules from each round to the next round (where B is the beam width). These B best candidate rules are then used to create new rules. The beam search process can include at most feature count (x) rounds, since each feature can be included in a rule at most once. In each round i, the beam search process generates candidates with i predicates and selects the B best candidate rules from them. Therefore, by setting B higher, the algorithm is more likely to avoid local optima, but may use a large number of model calls, potentially increasing the computational load.

[0186] Techniques for interpreting inference results

[0187] In some embodiments, the client-facing service for hosting chatbots implements techniques for interpreting inference results, as described herein. The described techniques provide utility to clients (e.g., entities hosting / owning chatbots) enabling them to review chatbot operations and their outcomes using a comprehensive interpretation. This opens the "black box" of ML classification for clients to review and improve the chatbot system. In some embodiments, these techniques are implemented as part of a client-facing system and interface through which clients can review various aspects of the chatbots hosted by the client. For example, clients can review a list of chatbots hosted by the client, along with various reports and suggestions related to the functionality of these chatbots, through the interface.

[0188] The chatbot list in the interface can further include information related to various interactions associated with different chatbots over a period of time. For example, the client can view various interactions, including utterances received by the chatbots over a period of time. These interactions, including utterances, may be associated with specific inference outcomes predicted by the chatbots or the absence of such inference outcomes. Inference outcomes are the result of the chatbots' interpretation of specific utterances. For example, inference outcomes may include, but are not limited to, the chatbots' classification results of utterances (e.g., predictions of intents / services corresponding to utterances), confidence thresholds determined by the classification (e.g., the confidence level threshold required for a specific intent / service to be predictively associated with the utterance), etc. Clients gain significant value and utility from understanding how inference outcomes are generated for specific utterances.

[0189] Figure 5 This is a simplified flowchart 500, illustrating an example of a process for interpreting inference results according to certain embodiments. Figure 5 The described process can be implemented as software (e.g., code, instructions, programs), hardware, or a combination thereof, executed by one or more processing units (e.g., processors, cores) of a corresponding system. The software can be stored on a non-transitory storage medium (e.g., a memory device). Figure 5 The methods presented and described below are intended to be illustrative rather than restrictive. Although Figure 5 The individual processing steps, occurring in a specific sequence or order, are described, but this is not intended to be limiting. In some alternative embodiments, the steps may be performed in a different order, or some steps may be performed in parallel. In some embodiments, such as in Figures 1 to 4 In the embodiments depicted, Figure 5The processing described herein can be performed by a preprocessing subsystem to interpret the inference results according to certain embodiments. The machine learning model can include, for example, a neural network model or any other machine learning model, and can be used as a classifier, for example, to classify user intent based on user utterances.

[0190] At point 510, a request to interpret the reasoning results of the dialogue is received. Specifically, a request is received from the entity to obtain a report related to the reasoning results generated by the syntactic analysis of the chatbot's dialogue. For example, a client hosting the chatbot can generate and send a request that is received as part of process 500 to generate a report for the client to view, which details the reasoning results output by the chatbot at some point prior to the input of the dialogue.

[0191] As described herein, the inference result includes a result or prediction that comprises some data relating to how the chatbot application syntactically analyzes and interprets utterances. In some embodiments, the request to interpret the inference result is received from an entity, such as a client hosting the chatbot application, which obtains the inference result for the utterance. For example, the chatbot may receive utterances from a client and responsively take some action based on the client's utterances. The request may be submitted, for example, via a command line or a client-accessible user interface. The process of inducing the action generates inference data that helps explain how the chatbot application induces an action given the client's utterances. The inference result may include, for example, a classification result of the utterances based on the chatbot application's processing of them, which is useful for the client to understand how the chatbot application works.

[0192] In some embodiments, a client-hosted chatbot application includes a classifier that categorizes user intent based on user utterances. For example, the classifier may be a subsystem of the chatbot system, including a machine learning model trained to take utterances as input and output a predicted classification of the utterances. In some embodiments, the classifier is a trained neural network machine learning model configured to map inputs included in the utterances to individual nodes of the neural network model. The individual nodes of the neural network model can then be processed to produce an output corresponding to the utterance classification. The classification may, for example, be the chatbot predicting the intent / service that the utterance will correspond to. In some embodiments, the classification result of the above inference is generated by the classifier.

[0193] In some embodiments, the system stores data related to the inference results and classifications made in real time by the chatbot system in computer memory. For example, when the chatbot receives a utterance, the system may store the received utterance. After the chatbot outputs prediction data (e.g., data related to the prediction intent), the system may store the prediction data as part of the inference result. The client may review this information stored in the system to determine the received utterances and the inference results generated by the chatbot. When the client determines that the chatbot is malfunctioning, or is otherwise dissatisfied with the chatbot's functionality, the client may request a report explaining the inference results on the utterances. In some embodiments, the system for storing data related to utterances and inference results is the same as the system in execution process 500.

[0194] At point 520, the classification result of the utterance is obtained from the classifier. Specifically, when the utterance is input into the chatbot, the classification result can be replicated using a classifier (such as the classifier of the chatbot system in point 510 or a similar classifier). For example, once the utterance specified by the client in the report request and the chatbot system are known, the utterance can be input into the chatbot system or a similar system to obtain a classification result as output (e.g., the intent predicted to correspond to the utterance).

[0195] In some embodiments, a classification result is obtained by inputting utterances into a classifier, such as into a machine learning model included in the classifier. Obtaining the classification result allows the system to determine that the classification result is part of the inference result for which an explanation is requested in step 510. The classification result can then be obtained as output to perform further steps of process 500. In some embodiments, the classification result is saved as part of previous operations of the chatbot application using the classifier. For example, when a client sends utterances to the chatbot application, the chatbot application's classifier can generate a classification result and store it on a local or external system (such as a client database). The classification result can then be retrieved from the database at 520.

[0196] In some embodiments, the classification results include classification data corresponding to predicted intents / services, which correspond to utterances as predicted by a chatbot. For example, the classification data may include utterance-related data, one or more predicted intents / services for the utterance, and one or more confidence values ​​corresponding to the degree of confidence that the one or more predicted intents / services correspond to the utterance. In some embodiments, the confidence values ​​correspond to the degree of correspondence between nodes of a trained neural network machine learning model, as used by a classifier when performing syntactic analysis on the utterance.

[0197] In some embodiments, the classifier is a fax classifier that simulates the operation of a chatbot system. For example, the system may store the classifier as a standalone classifier without a corresponding chatbot model. The system can use the classifier to obtain classification results for the utterances used, without requiring the entire chatbot system to perform syntactic analysis on the utterances. The system may store, for example, a trained machine learning model corresponding to the classifier used by the chatbot system, since the classifier may be the only requirement to perform step 520. In some embodiments, the system hosting the fax classifier is the same as the system performing process 500.

[0198] Once the classification result is obtained, at position 530, a new set of anchors, including anchor words in the discourse, is selected, and a synthetic discourse is generated by replacing non-anchor words in the discourse with similar words. Specifically, a subset of the words constituting the discourse is grouped into a set of anchors, representing a group of words that are relatively important to the generated inference result. Words not in this set of anchors are replaced with similar words to form the synthetic discourse. For example, one or more words in the discourse can be designated as anchor words, representing words predicted to be relatively more important to the classification than other words in the discourse. Then, words not designated as anchors are replaced with similar words to form a new synthetic discourse that is semantically different from the original discourse but contextually similar.

[0199] The result of replacing non-anchor words is the generation of synthetic discourse that maintains a certain degree of similarity to the original discourse due to the replacement of similar words. In some embodiments, selecting one or more anchor words from the discourse involves identifying one or more words in the discourse that correspond to a higher degree of uniqueness (e.g., the word or phrase has few known synonyms). In some embodiments, anchor words are the words in the discourse that are least likely to be replaced to form similar discourse. For example, the discourse “I want to order a pizza” contains the words “I”, “want”, “to”, “order”, “a”, and “pizza”. In this phrase, the words “I” and “pizza” have fewer alternative similar words than the others, and are therefore potential candidates for anchor words. For example, the phrase “I desire to purchase one pizza” is similar to “I want to order a pizza” because the former may correspond to an intention similar to the latter. In contrast, the phrase “John wants to order a treehouse” is not a similar phrase, although the words “want(s)”, “to”, “order”, and “a” are used as anchor words between the two phrases.

[0200] A new set of anchors can be selected in any manner necessary to allow the generation of synthetic discourse as described herein. In some embodiments, selecting a set of anchors involves using a clustering algorithm to select potential candidates for anchor words in the discourse. For example, the process can be based on some heuristic (such as the uniqueness metric described above) to determine one or more best candidates for anchor words. One or more additional words can then be selected to form this set of anchors. All words not selected for this set of anchors will be non-anchor words that are replaced with non-anchor words in the synthetic discourse. In some embodiments, a set of anchors is selected only if the set of anchors meets a heuristic anchor selection threshold (e.g., the average uniqueness score of each word in the group must reach or exceed the uniqueness threshold for the group to be selected as a set of anchors).

[0201] Step 530 allows the generation of new synthetic utterances that differ from the original utterances but can be used to test inferences about the similarities and differences between the classified utterances. By using anchors as constants in these new utterances, the new utterances reveal at the word level how specific inferences of the utterances are determined. For example, these new synthetic variants of the utterances can also be classified by a classifier to determine one or more predictive intentions of the new utterances.

[0202] At 540, the classification results of the synthesized utterances and the confidence levels of the selected set of anchors are obtained from the classifier. Specifically, each synthesized utterance generated in 530 is input into a classifier model to obtain a classification prediction result for the synthesized utterance. For example, for each synthesized utterance generated in 530, the synthesized utterance is input into an ML model operating as part of the classifier used in 520 to obtain a classification result for the synthesized utterance. If the ML model is trained to output a predicted intent / skill given an input utterance, then the ML model will output one or more predicted intents / skills given one or more synthesized utterances input into the ML model. Based on the predicted intent / skill of the synthesized utterances, the confidence values ​​of the selected set of anchors in 530 are determined. The confidence values ​​can correspond to any metric related to the generated synthesized utterances and the predicted intent / skill of these synthesized utterances. In some embodiments, the confidence values ​​of a set of anchors are the percentage of generated synthesized utterances whose classification results are the same as those of the original utterances. For example, if 100 synthetic utterances are generated for a set of anchors, and 87 of these 100 synthetic utterances are identified as corresponding to predicted intents that match the predicted intent classification results obtained in 520, then the confidence value of this set of anchors could be 87% or 0.87.

[0203] In some embodiments, each predicted classification may correspond to a confidence metric associated with the confidence level of a particular predicted intent / skill corresponding to an input utterance. For example, in some embodiments where the ML model is a trained neural network ML model, the confidence metric may be based on a numerically derived correspondence metric between nodes of the trained neural network when processing the input utterance. In these embodiments, a higher confidence metric would correspond to a relatively higher degree of correspondence between nodes of the trained neural network when outputting a predicted intent / skill. In some embodiments, the confidence values ​​of an associated set of anchors may be derived in part from the confidence metric of each synthesized utterance generated for that associated set of anchors.

[0204] The classification results of synthesized utterances are useful for identifying aspects of chatbots used to predict intentions / skills based on utterances. For example, both the original and synthesized utterances may be associated with the same predictive intention / skill. Therefore, it is reasonable to infer that the anchor words shared between the original and synthesized utterances are the primary factor in generating the inference results of the original utterances. Conversely, when the original and synthesized utterances do not share the same predictive intention / skill, it is reasonable to infer that the anchor words shared between the original and synthesized utterances are not the primary factor in the inference results, or that the ML model is dysfunctional in processing certain words or phrases in the utterances. For these and similar reasons, the classification results of the generated synthesized utterances are highly valuable in determining the inference results of the original utterances.

[0205] At 550, a set of anchors associated with the highest confidence level is identified. Specifically, confidence values ​​of a set of anchors corresponding to the highest confidence level can be selected to be included in a report to be sent responsively in response to the request received in 510. For example, process 500 may include generating multiple sets of synthetic utterances based on the selected multiple sets of anchors. When reviewing reports on inference results, the multiple sets of anchors with the highest confidence values ​​are most likely to be the most valuable. Therefore, at 550, the classification results and confidence levels generated and obtained in 540 can be compared with the confidence levels of other previously generated sets of anchors. The set of anchors with the highest confidence levels among the compared sets of anchors and their corresponding synthetic utterances can be identified and saved for later inclusion in the report.

[0206] In some embodiments, multiple sets of anchors may be included in the report based on a threshold confidence level, rather than based on a single highest confidence value selected within each set of anchors. For example, a confidence threshold can be used as a reporting cutoff value for whether or not multiple sets of anchors will be included in the report. If the confidence level of each set of anchors exceeds the confidence threshold, then that set of anchors and its corresponding synthetic utterance will be included in the report.

[0207] The operation at positions 530-550 can be performed iteratively in multiple rounds, for example, in the bundle search process described above, where different combinations of words can be used as anchors in each round. In one example, in the first round, each word in the utterance can be used as an anchor word to generate variants of the utterance as a synthetic utterance, which can be classified by a classifier, and after the first round, an anchor with a confidence level can be selected. In some embodiments, in the second round, any combination of the anchor word selected in the first round with another word in the utterance can be used as an anchor word to generate more variants of the utterance as a synthetic utterance; the synthetic utterance can be classified by a classifier; the classification result can be compared with the classification result of the original utterance, and after the second round, an anchor with the highest confidence level can be selected.

[0208] In some embodiments, in the second round, new combinations of anchor words not selected in the first round can be selected and used to generate synthetic utterances, which are then categorized and compared as described above. The iterations of operations 530-550 can stop when all important anchors have been selected and synthetic utterances have been generated for these anchors. For example, a uniqueness threshold can be used as part of a bundle search. Once all groups of anchors satisfying the uniqueness threshold have been selected at 530, the iteration can stop at 550 and process 500 can proceed to 560.

[0209] At point 560, based on the results of multiple rounds at point 550, a report can be generated. The report may include one or more sets of anchors and their corresponding confidence levels. In some embodiments, the report may include the classification results of the synthesized utterance. Based on the report, a user can understand which word or combination of words caused the classifier to infer a specific intent from the original utterance. For example, the report might show the original utterance “I want to order a pizza” and the synthesized utterance, such as “I desire to purchase one pizza”. The report might indicate that, for example, both the original and synthesized utterances are predicted to correspond to similar intents, such as the intent called “ORDER FOOD,” with similar confidence levels. Therefore, the user can infer from the report that the shared anchor words “I” and “Pizza” are the main factors in determining the same intent and are the reason for the relatively high confidence values ​​of this particular set of anchors.

[0210] Table 1 below shows the explanation section of a sample report in the JSON file. The report includes the classifier's inference results for the original utterance "money available". The report also includes: for each anchor, a variant of the original utterance generated using said anchor; the corresponding classification result of the classifier; and the corresponding precision determined based on the number of matches and non-matches between the classification results of the variants of the original utterance and the classification results of the original utterance. For example, when the anchor only includes the word "available", variants of the original utterance might include "#available", "amenities available", "bicycles available", "+available", "2017available", etc. The classifier can classify these utterances as associated with the intent "balances", which matches the classification result of the original utterance. However, the classifier can classify the utterances "commercially available", "only available", "universally available", and "Currently available" as associated with the intent "unresolved intent", which does not match the classification result of the original utterance. The report shows that the accuracy associated with the anchor word "available" is approximately 47.5%. When the anchor word contains both the words "money" and "available," the accuracy is approximately 100%.

[0211] Table 1: Explanation of the Sample Report (JSON Format)

[0212]

[0213]

[0214]

[0215]

[0216]

[0217]

[0218]

[0219] Figure 6An example of a system 600 for interpreting inference results according to certain embodiments is illustrated. For example, system 600 may be a system for causing implementation process 500 or a similar process as described herein. System 600 includes a user interface 605, a management API 610, an intent server 615, a gateway 620, a workflow engine 625 (e.g., a server), and a database or cloud storage device 630. The intent server 615 may include a classifier that infers user intent based on user utterances. The workflow engine 625 may (e.g., using Python code) execute application 635 to perform the above-mentioned, for example, regarding... Figure 5 The process is described in 500 categories.

[0220] Clients can use asynchronous calls to submit interpretation requests with categorized queries (e.g., utterances) to gateway 620 via user interface 605 and management API 610. For example, a client can use its client device to interact with user interface 605 to view aspects of the chatbot or otherwise interact with services associated with the chatbot. In some embodiments, clients can use system 600 to view utterances processed by the chatbot at a previous time and the inference results generated for those utterances. For example, a client might use a specific chatbot as part of its own service.

[0221] The client may notice that a specific utterance has been processed and an unexpected inference result has been generated for that utterance (e.g., the utterance "I would like to order a pizza" is predicted to correspond to a request to process a customer refund). The client can then request an explanation of the utterance-based inference result, as described in 510. The client can enter this request at user interface 605, which then routes the request to management API 610. Management API 610 may be an API that operates as part of a chatbot interpretation system separate from the client device used by the client.

[0222] In some embodiments, after receiving an asynchronous call, the management API 610 forwards the call to the intent server 615 for routing to the gateway 620. In other instances, after receiving an asynchronous call, the management API 610 forwards the call directly to the gateway 620. In instances where the intent server 615 receives the call, the intent server 615 forwards the call to the gateway 620. The gateway 620 provides a working identifier for the interpretation request, and the working identifier is returned to the user via the intent server 615, the management API 610, and the user interface 605. The gateway 620 initiates a workflow for processing the interpretation request by forwarding the interpretation request and the associated working identifier to the workflow engine 625. The working identifier may be, for example, an identifier corresponding to a report generation process such as process 500.

[0223] Workflow engine 625 executes application 635 to perform classification interpretation, for example, using the cluster search technique discussed above. Application 635 may generate new utterances (e.g., approximately 500 to 2000 new utterances on average) using various anchors in one or more rounds, said new utterances being variations of the original utterances (i.e., the classification query within the interpretation request) and feed these new synthesized utterances to intent server 615. In some instances, application 635 uses caching to ensure that only unique utterances are submitted to intent server 615. For example, some of the synthesized utterances generated by application 635 may be known utterances previously received in an organized manner from users of the chatbot system. Intent server 615 exposes an API to obtain a list of intents and new utterances from workflow engine 625. Workflow engine 625 generates classification results for the new utterances based on the obtained list of intents. According to procedures such as process 500, application 635 can determine the confidence level of anchors based on the classification results of the previous (or more) rounds, select the anchor with the highest confidence level, determine new anchors based on the selected anchors, generate new queries based on the new anchors, and feed the new queries to intent server 615. The results, including anchors, new synthesized utterances, classification results of synthesized utterances, confidence levels associated with anchors, or any combination thereof, can be sent by workflow engine 825 and / or intent server 815 to a database or cloud storage device 830 (e.g., object storage) for storage, along with work identifiers.

[0224] Figure 7An example of a system 700 for obtaining an interpretation of inference results according to certain embodiments is illustrated. Like system 600, system 700 may include a user interface 705, a management API 710, an intent server 715, a gateway 720, and a database or cloud storage device 730. Users can submit interpretation result requests with a working identifier to gateway 720 via asynchronous calls, for example, through user interface 705 and management API 710. In some instances, after receiving an asynchronous call, management API 710 forwards the call to intent server 715 for routing to gateway 720. In other instances, after receiving an asynchronous call, management API 710 forwards the call directly to gateway 720. In instances where intent server 715 receives the call, intent server 715 forwards the call to gateway 720. Gateway 720 obtains results based on the working identifier, including anchors, synthesized utterances, classification results of synthesized utterances, confidence levels associated with anchors, or any combination thereof. The results are returned to the user via the intent server 715, the management API 710, and the user interface 705. Multiple interpretation result requests can be executed at once using multiple job identifiers.

[0225] Figure 8A The illustration shows an example of simulated CPU usage during interpreted computation, according to certain embodiments. In the illustrated example, the utterance to be categorized and interpreted is "another agent question." The runtime is approximately 234 seconds. Specifically, Figure 8A The figure illustrates the level of central processing unit (CPU) resource usage for performing the interpretation of inference results compared to the time required to complete the request.

[0226] Figure 8B The illustration shows an example of analog memory usage during computation interpretation according to certain embodiments. Specifically, Figure 8B The figure illustrates the level of memory resource usage for interpreting the inference results compared to the time required to complete the request.

[0227] Figure 9A The illustration depicts an example of simulated CPU usage during computation interpretation according to certain embodiments. In the illustrated example, the utterance to be categorized and interpreted is "I would like to have a large cheese pizza with an all-glass of Coke." The runtime is approximately 933 seconds. Specifically, Figure 9A The figure illustrates the level of central processing unit (CPU) resource usage for performing the interpretation of inference results compared to the time required to complete the request.

[0228] Figure 9B The illustration shows an example of analog memory usage during computation interpretation according to certain embodiments. Specifically, Figure 9B The figure illustrates the level of memory resource usage for interpreting the inference results compared to the time required to complete the request.

[0229] Figure 10A The illustration shows an example of simulated CPU usage during interpreting computation, according to certain embodiments. In the illustrated example, the utterance to be categorized and interpreted is "question". The execution time is approximately 101 seconds. Specifically, Figure 10A The figure illustrates the level of central processing unit (CPU) resource usage for performing the interpretation of inference results compared to the time required to complete the request.

[0230] Figure 10B The illustration shows an example of analog memory usage during computation interpretation according to certain embodiments. Specifically, Figure 10B The figure illustrates the level of memory resource usage for interpreting the inference results compared to the time required to complete the request.

[0231] Techniques for generating training datasets

[0232] As mentioned above, generating reports that include indications of the synthesized utterances, predicted intentions / skills of the synthesized utterances, and corresponding confidence values ​​of the synthesized utterances is useful for determining how a chatbot system processes received utterances. These reports can be used to identify certain deficiencies in the chatbot system. For example, the chatbot system might incorrectly process certain anchor words in the utterance and predict incorrect intentions / skills as associated with that utterance. While eliminating the “black box” identified by ML is very useful for identifying deficiencies in a chatbot system, the chatbot system must subsequently be retrained to prevent erroneous predictions from recurring.

[0233] Training and retraining chatbots is a difficult and resource-intensive process. For example, the first step is to identify the deficient areas of the chatbot's ML model. Then, a training dataset must be generated to effectively reduce these deficiencies. The training dataset will include multiple sets of training utterances and corresponding "foundational" intents / skills. The ML model will then be trained using the training utterances and the foundational intents / skills to refine the model and better predict the intents / skills of the utterances. Obtaining a training dataset is challenging, and manually selecting and constructing one is inefficient and resource-intensive.

[0234] Figure 11 This is a simplified flowchart illustrating an example of a process for retraining a model using synthetic utterances, according to certain embodiments. Figure 11The described process can be implemented as software (e.g., code, instructions, programs), hardware, or a combination thereof, executed by one or more processing units (e.g., processors, cores) of a corresponding system. The software can be stored on a non-transitory storage medium (e.g., a memory device). Figure 11 The methods presented and described below are intended to be illustrative rather than restrictive. Although Figure 11 The individual processing steps, occurring in a specific sequence or order, are described, but this is not intended to be limiting. In some alternative embodiments, the steps may be performed in a different order, or some steps may be performed in parallel. In some embodiments, such as in Figures 1 to 5 In the embodiments depicted, Figure 11 The processing described herein can be performed by a preprocessing subsystem to retrain the model using synthesized utterances according to certain embodiments. The machine learning model can include, for example, a neural network model or any other machine learning model, and can be used as a classifier, for example, to classify user intent based on user utterances.

[0235] More specifically, process 1100 describes the procedure for obtaining a training dataset and retraining the ML model to predict intent / skill based on input utterances. As mentioned above, refer to... Figure 5 In process 500, multiple synthetic utterances are generated, corresponding to a set of anchor points for the original utterance and an ML model operating as part of the chatbot system. The synthetic utterances generated as part of process 500 can be used to identify deficiencies in the ML model, specifically those in the original utterance and similar utterances. In this case, it is beneficial to retrain the ML model using the original utterance and / or similar utterances along with corresponding basic facts to improve model functionality. It should be understood that the synthetic utterances generated in steps 530-550 are the best candidates for retraining the machine learning model.

[0236] Process 1100 may optionally begin at 550 of process 500, wherein a set of anchors with the highest confidence level is identified. In some embodiments, process 1100 begins when process 550 has completed the last iteration of the iterations of boxes 530-550 and a set of anchors with the highest confidence level has been identified. This set of anchors and the corresponding synthetic utterances are then used as part of process 1100 to generate a training dataset for retraining the ML model. In some embodiments, process 1100 is combined with... Figure 5 The 560 parallel operations. Specifically, the synthesized discourse can be used in process 1100 while generating a report that includes this set of anchors.

[0237] Process 1100 continues to 1110, wherein one or more utterances are received, each utterance corresponding to a confidence level among one or more confidence levels. In some embodiments where process 1100 begins at step 550 of process 500, the one or more utterances are synthetic utterances generated as part of a report generation process. For example, in response to receiving a request to explain the results of inference, the system performing process 500 may also automatically generate a retraining dataset for a specific ML model associated with the utterance specified in the request.

[0238] In other embodiments, one or more utterances are utterances stored in computer memory at some previous time. For example, one or more utterances may be utterances corresponding to predictive intentions / skills predicted by an ML model that will be retrained according to process 1100. The utterances may have been previously stored for future training of the ML model, for example, as part of a previous iteration of process 500.

[0239] Each utterance corresponds to a confidence level. As described herein, a confidence level represents the strength of a set of anchors associated with a particular utterance, and the proportion of synthesized utterances based on this set of anchors that are classified as having the same intent / skill as the particular utterance. For example, each synthesized utterance may correspond to a determined confidence value for a set of anchors used to generate the synthesized utterance. In some embodiments, each synthesized utterance corresponds independently to a confidence value. For example, the degree of correspondence between nodes in a trained neural network ML model can be used to determine the confidence level of a synthesized utterance when predicting a particular intent / skill given a particular utterance as input. In some embodiments where process 1100 begins at 550 of process 500, one or more synthesized utterances have already corresponded to a confidence level as determined in step 540 of process 500.

[0240] At 1120, one or more confidence values ​​received at 1110 are compared with a confidence threshold. Specifically, each of the one or more confidence values ​​is compared with the confidence threshold to determine a subset of confidence values ​​that do not reach or exceed the threshold. In various embodiments, the confidence threshold is a value representing a relatively permissible level of correspondence to the predicted intent / skill for a particular utterance. For example, a training dataset is typically most useful for training an ML model when the ML model can no longer predict the intent / skill associated with utterances in the training dataset with a high level of confidence. The result of training will be that the ML model will predict the intent / skill corresponding to similar utterances with a higher level of confidence after training. Therefore, it is most useful to select utterances that correspond to confidence levels that fall at or below the desired confidence threshold and include them in the training dataset. Eutterances that correspond to confidence levels above the threshold may be useful for training the ML model, but will not affect model training as efficiently as utterances that correspond to confidence levels below the threshold.

[0241] In some embodiments, the confidence level threshold can be determined based on some input from the entity associated with the chatbot system and the ML model. Specifically, the entity that owns or operates the chatbot system, and therefore the ML model, can specify a confidence threshold to optimize the generation of new training datasets. For example, a client hosting the chatbot system can decide to set the confidence threshold to a value that generates a sufficient training dataset but keeps it small enough to retrain the model over a certain amount of time. In some embodiments, a confidence threshold for use can be suggested to the client based on a metric associated with the utterances received in 1110.

[0242] At 1130, a subset of training utterances is determined, where each utterance in the subset corresponds to a confidence value less than or equal to a confidence threshold. Any utterance among one or more received utterances corresponding to a confidence level less than or equal to the confidence threshold is included in the subset of training utterances and ultimately used to retrain the model. For example, the confidence threshold can be set to a value of 0.80 (or 80% confidence). The confidence value corresponding to the first utterance in one or more utterances can be 0.90 (or the utterance corresponds to a 90% confidence level for a specific intent / skill). The confidence value corresponding to the second utterance can be 0.67 (or 67%). The first utterance is determined not to belong to the subset of training utterances because its corresponding confidence level is greater than the confidence threshold. The second utterance is determined to belong to the subset of training utterances because its corresponding confidence level is less than or equal to the confidence threshold. In some embodiments, the threshold can further specify a particular criterion for obtaining utterances, for example, the subset of training utterances only includes utterances corresponding to confidence values ​​less than or equal to the confidence threshold. Those skilled in the art will understand that any criterion related to the threshold can be used. For example, in alternative embodiments, to fine-tune the ML model to specifically predict a particular intent / skill of a utterance, only utterances with a probability greater than or equal to the threshold may be selected. For the client, understanding which utterances correspond to predictions with lower confidence levels is often important for understanding how to retrain the ML model. In some cases, the client may also request an explanation of the utterance classifications with higher confidence values ​​to understand why the model predicts intent / skills with higher confidence levels.

[0243] At 1140, one or more training categories corresponding to a subset of training utterances are determined. Specifically, the one or more training categories are basic facts associated with a subset of training utterances that will be used to train the ML model. For example, the client may have access to a list of subsets of training utterances. The client can then select a basic fact intention / skill for each utterance in the subset of training utterances to form a complete training dataset. In some embodiments, one or more training categories can be represented by an interactive list of possible intentions / skills that the utterances can correspond to as correct basic fact predictions for the ML model. Basic fact intentions can be selected from an interactive list of one or more known intentions associated with the ML model. Once selected, the basic fact intentions are used to create a new training dataset. This document describes the interactive list of basic fact intentions associated with a subset of training utterances, including those below. Figure 12 middle.

[0244] At 1150, a subset of training utterances is used to train the model based on one or more determined training categories. New training datasets can be fed into one or more ML models to train them. For example, the determined subset of training utterances is used to retrain the trained ML model based on the training categories determined in 1140. One result is that utterances previously classified as incorrectly predicted intentions / skills by reports (e.g., reports generated in 560) will be used to train the ML model to more efficiently predict correct intentions / skills for similar utterances. Another result is that utterances previously classified as correctly predicted intentions / skills but with confidence levels below an acceptable confidence level (i.e., a confidence threshold) are used to further strengthen the model's correct predictive behavior.

[0245] exist Figure 11 In various embodiments not depicted herein, one or more new synthetic utterances similar to the one or more utterances received in 1110 and based on a confidence threshold can be generated. For example, to create a more comprehensive training dataset for retraining the model, one or more new synthetic utterances can be generated based on word or phrase substitutions, similar to step 530 of process 500. The new synthetic utterances can be based on anchor substitutions or another technique described above in 530. For example, one or more new synthetic utterances can be generated in response to determining that a subset of the training utterances is at or below a confidence threshold. These artificial utterances can be included in the retraining dataset and can correspond to training categories determined by the user. For example, if the training dataset must contain a certain number of utterances, one or more new synthetic utterances can be generated and used to supplement the training dataset. In other embodiments, the confidence threshold can be automatically increased or decreased to meet the required number of utterances in the training dataset.

[0246] exist Figure 11In various embodiments not depicted, a subset of training utterances undergoes a context-based filtering process before being used to retrain the ML model. The context-based filtering process refines the utterances to fit the environment-specific context. For example, a customer service chatbot might belong to a context where profanity or offensive language patterns are prohibited. Before retraining, one or more utterances and / or synthesized utterances may be filtered to remove all profanity or offensive language from the utterances. For example, profanity words may be replaced with non-profanity-related similar words before training the model to reduce profanity bias in the newly trained ML model. In another example, the machine learning model might belong to a context where including personally identifiable information in any response to a query is prohibited. A subset of training utterances may be filtered to replace any proper nouns or personally identifiable information in the utterances with generic information / words. The filtering process may include any filtering to achieve the goals of an environment-specific training process.

[0247] Boxes 1110-1150 of process 1100 can be executed iteratively, wherein a subset of training utterances is generated iteratively based on the results of previous retraining. For example, after retraining the model 1150, the utterances in the training dataset can be modified to generate a test dataset with utterances similar to the subset of training utterances. The test dataset can be fed into a newly trained ML model to predict the classification of each utterance in the test dataset. Step 1110 of process 1100 can then be repeated using the utterances in the test dataset and their corresponding confidence values ​​to further retrain the ML model. In various embodiments, operations 1110-1150 are repeated iteratively until the confidence values ​​of the generated utterances are not less than or equal to a threshold.

[0248] Figure 12 The illustration depicts an example interface for retraining a model using synthetic utterances, according to certain embodiments. Specifically, Figure 12 An example interface 1200 is depicted for generating training datasets for retraining machine learning models. Figure 12 The interface depicted can be used by entities such as clients or users to generate training datasets for improving the functionality of ML models. Interface 1200 includes a threshold field 1210. Threshold field 1210 can be a field displaying a confidence threshold, as used in process 1100. Threshold field 1210 can be an interactive portion of interface 1200, which a client can manipulate to input a confidence threshold. For example, as referenced above... Figure 11 As described, a confidence threshold can be set to determine the confidence level above the confidence threshold (e.g., as...). Figure 12 The subset of training discourses that does not correspond to the described "confidence score".

[0249] like Figure 12The described, received utterances (one or more) are displayed in a comprehensive list on interface 1200. For example, as... Figure 12 As depicted, interface 1200 includes a utterance list 1220. In some embodiments, the utterance list 1220 is a list of one or more received utterances (such as those received in step 1110 of process 1100). Regarding Figure 12 It is worth noting that the utterances obtained and displayed in utterance list 1220 are variants of similar utterances and may be the result of the process for synthesizing utterances described above with respect to process 500 and specific box 530. Interface 1200 includes a predicted intent list 1230. The predicted intent list 1230 is displayed side-by-side with utterance list 1220 so as part of the synthesis interface to show that the displayed utterances are predicted as corresponding intents. For example, the utterances depicted in utterance list 1220 are displayed side-by-side with the predicted intents previously predicted in the classification result step (e.g., 540 of process 500) in intent list 1230.

[0250] Interface 1200 includes a confidence score list 1240. The confidence score list 1240 can be displayed side-by-side with the utterance list 1220 and the predicted intent list 1230 to show the same predicted intent and its relationship to the predicted intent as part of a comprehensive interface. Figure 1 The interface 1200 displays the corresponding confidence level associated with the input utterance. Interface 1200 includes an interactive intent selector 1250. The interactive intent selector 1250 is an interactive element of interface 1200 that allows a client using the interface to select a base fact intent corresponding to a given utterance to include in the training dataset, for example, as part of process 1100 1140. The utterance selected in the interactive intent selector 1250 can be used as the base fact intent for the corresponding utterance, causing the utterance to be included in a subset of the training utterances, provided that the utterance is associated with a confidence score less than or equal to the confidence score. Therefore, a client using interface 1200 can generate a comprehensive training dataset with utterances without needing to manually create new training utterances from scratch.

[0251] Figure 13 The illustration depicts an example interface for retraining a model using synthetic utterances, according to certain embodiments. Specifically, Figure 13 Depicting Figure 12 The comprehensive interface 1200 includes additional interactive fields. The threshold input field 1300 is an interactive threshold field into which a specified confidence threshold can be entered. The threshold input field 1300 can correspond to the threshold field 1210 after the user has already interacted with it. Figure 13 Users of the interface described herein can utilize the interface to perform the methods described herein, including Figure 11The process described herein. For example, a user can first specify a confidence threshold in the threshold input field 1300, which will be used as part of process 1100 and specific boxes 1120 and 1130. The input confidence threshold will be used to determine a subset of utterances to be included in the training dataset for training the ML model.

[0252] In various embodiments, Figure 13 Not described herein, buttons or other interactive elements may be coupled to displayed utterances, and when interacted with, these interactive elements will cause the generation of synthetic utterances similar to the displayed utterances. Clients can select interactive buttons or elements of the interface to begin generating a training dataset based on client-provided input. In various embodiments, only utterances manually selected by the client using checkboxes next to the displayed utterances are used for retraining data, rather than only utterances associated with confidence values ​​that reach or fall below a confidence threshold. Examples of generating synthetic utterances according to certain embodiments are shown below in JavaScript Object Notation (JSON) script. For example, the generation of synthetic utterances described in the script below can be used in synthetic utterance generation processes such as 530 and / or process 1100.

[0253] Table 2 provides an explanation of the examples of generated synthetic utterances (in JSON format).

[0254]

[0255]

[0256]

[0257] Specifically, the steps for generating similar synthetic utterances based on specific utterances are shown. These steps correspond to creating a set of synthetic utterances (not manually written utterances or utterances retrieved from a utterance repository) for retraining the model. As part of the retraining data generation, synthetic utterances are generated and added to the data object "utterances_with_similar_predictions". These utterances are added using the "b'Add" command.

[0258] A combined interface for requesting reports and generating training datasets.

[0259] Figure 14 The illustration depicts an example interface, according to certain embodiments, for interpreting inference results as part of the training dataset generation process. Specifically, Figure 14 An interface 1400, according to certain embodiments, is depicted for displaying a comprehensive analysis of one or more inference results in the context of a given utterance. For example, Figure 14It includes an interactive interface 1400 that displays the inference results associated with predictions made by a machine learning model given a particular input utterance. Interface 1400 depicts a list of various inference results displayed according to certain criteria. For example... Figure 14 The depicted image shows the inference results, which include all utterances that classify the input utterances as matching the intent "unresolvedIntent". The corresponding utterances, along with the corresponding inference results, are displayed on interface 1400.

[0260] The inference results include the intent most closely predicted to correspond to the utterance, as well as other intents that may already correspond to the utterance. For example, the first row displays the utterance “add another song to the Cita Romnticaplaylist” and various aspects of the utterance identified in the inference results. In addition to the “unresolvedIntent” intents predicted to correspond to the utterance, a “win margin” is also displayed. The win margin is a score corresponding to the predicted probability that the resulting intent corresponds to the utterance. In some embodiments, the win margin is a calculated difference between a first confidence value indicating that the primary intent / skill is predicted to correspond to the utterance and a second confidence value indicating that the secondary intent / skill is predicted to correspond to the utterance.

[0261] The inference report also includes an intent score chart, showing the breakdown of potential predictable intents and the relative confidence level of the correspondence between utterances and predicted intents. The inference report includes an interactive intent selection field. This interactive intent selection field can be presented near a suggestion button, which automatically populates the field with the suggested corresponding intent when interacted with. The automatically populated field can be used to generate utterance and ground fact training category pairs to be included in the training dataset for retraining the model. For example, a client can select a checkbox next to the utterance “add another song to the CitaRomntica playlist.” The client can then press a button, such as the “AddExample” button on interface 1400, causing training data points to be generated, which include the utterance and the corresponding ground fact intent selected by the client. These training data points can then be included in the training dataset to train the ML model of the chatbot system in the future.

[0262] Interface 1400 also includes buttons for analyzing the inference results queue. For example, a client can press the "Add to Analyze Queue" button to request an explanation report (such as...) related to the inference results for a given utterance. Figure 5 (as described in 500), instead of pressing the "Add Example" button to generate training data points.

[0263] Figure 15 The illustration depicts an example interface for interpreting inference results according to certain embodiments. Specifically, Figure 15 An interface similar to interface 1400 is depicted, used to introduce the analysis queue after an inference result and corresponding utterance have been selected. Notification 1500 can be included on the interface to inform the user that the selected inference result has been added to the requested inference result queue. For example, by selecting an inference result using a checkbox in interface 1400 and clicking the "Analyze Queue" button, a process such as process 500 can be initiated for the corresponding utterance and ML model associated with the selected inference result and utterance. Notification 1500 can then remind the user that a specific inference result is being analyzed and an interpretation report is being generated.

[0264] Figure 16 The illustration depicts an example interface for interpreting inference results according to certain embodiments. Specifically, Figure 16 An interface similar to interface 1400 / 1500 is depicted after the discourse has been analyzed by a discourse analysis tool (such as the discourse analysis tool using process 500). Notification 1600 may be included on the interface to inform the user that the selected inference result / discourse has been analyzed and the report is ready for viewing.

[0265] Figure 17 The illustration depicts an example interface for interpreting inference results according to certain embodiments. Specifically, Figure 17 This describes an interface similar to interfaces 1400 / 1500 / 1600 when the explanation report is pending and has not yet been generated. For example, when the client or user of the interface is waiting for the explanation report to be generated, the user can see... Figure 17 The display interface depicted is intended to inform users that the analysis is not yet complete.

[0266] The following is an example of a request to interpret the classification results of submitted dialogue, presented in the form of a JavaScript Object Notation (JSON) script, according to certain embodiments.

[0267] Table 3 provides an explanation of the examples of generated synthetic utterances (in JSON format).

[0268]

[0269]

[0270]

[0271]

[0272] In the example shown, a request can be submitted by entering a query (e.g., the phrase "money available") followed by the command "getexplain". As shown, a JavaScript Object Notation (JSON) file can be generated for the query and explanation request.

[0273] Figure 18 The illustration shows an example of the output of an interpretation of utterance classification according to certain embodiments. In the illustrated example, the utterance being analyzed is “another agent question” as shown in the table above. The output may include a link 1810 to a report, a list 1820 of anchors selected as part of process 530 of 500, and a corresponding confidence level 1830 for each set of anchors in the anchor list 1820. For example, the confidence level associated with the anchor set “question” might be approximately 33.8%, corresponding to a confidence metric of 0.33, which is a prediction of the correct intent of the input utterance. The confidence level associated with the anchor “question another” using the anchor words “question” and “another” is approximately 100%. This indicates a completely accurate correct answer predicted given the anchor words (e.g., the ML model has been extensively trained on these anchor words previously).

[0274] Figure 19 A simplified diagram of a distributed system 1900 for implementing some embodiments is depicted. In the illustrated example, the distributed system 1900 includes one or more client computing devices 1902, 1904, 1906, and 1908 coupled to a server 1912 via one or more communication networks 1910. The client computing devices 1902, 1904, 1906, and 1908 can be configured to execute one or more applications.

[0275] In various examples, server 1912 may be adapted to run one or more services or software applications implementing one or more embodiments described in this disclosure. In some examples, server 1912 may also provide other services or software applications that may include non-virtual and virtual environments. In some examples, these services may be provided as web-based services or cloud services (such as under a Software as a Service (SaaS) model) to users of client computing devices 1902, 1904, 1906, and / or 1908. Users operating client computing devices 1902, 1904, 1906, and / or 1908 may then use one or more client applications to interact with server 1912 to utilize the services provided by these components.

[0276] exist Figure 19 In the depicted configuration, server 1912 may include one or more components 1918, 1920, and 1922 that implement the functions performed by server 1912. These components may include software components that can be executed by one or more processors, hardware components, or a combination thereof. It should be understood that various different system configurations, different from distributed system 1900, are possible. Therefore, Figure 19 The example shown is an example of a distributed system for implementing the example system and is not intended to be restrictive.

[0277] Users can use client computing devices 1902, 1904, 1906, and / or 1908 to execute one or more applications, which can generate one or more storage requests that can then be serviced according to the content taught in this disclosure. The client device can provide an interface that enables users of the client device to interact with it. The client device can also output information to the user via this interface. Although Figure 19 It describes only four client computing devices, but can support any number of client computing devices.

[0278] Client devices can include various types of computing systems, such as portable handheld devices, general-purpose computers like personal computers and laptops, workstation computers, wearable devices, gaming systems, thin clients, various messaging devices, sensors, or other sensing devices. These computing devices can run various types and versions of software applications and operating systems (e.g., Microsoft). Apple Or UNIX-like operating systems, such as Google Chrome TM OS (such as Linux or Linux-like operating systems), including various mobile operating systems (e.g., Microsoft Windows) Windows Android TM , Palm Portable handheld devices can include cellular phones, smartphones (e.g., ), tablet computers (e.g., Wearable devices include Google's personal digital assistants (PDAs), personal digital assistants (PDAs), and more. Head-mounted displays and other devices. Gaming systems can include various handheld gaming devices, internet-enabled gaming devices (e.g., with or without internet access). Microsoft gesture input devices Game console, Sony System, by Various game systems and others are provided. Client devices can run a variety of different applications, such as various Internet-related applications, communication applications (e.g., email applications, short message service (SMS) applications), and can use various communication protocols.

[0279] The (multiple) communication networks 1910 can be any type of network familiar to those skilled in the art that supports data communication using any of the various available protocols, including but not limited to TCP / IP (Transmission Control Protocol / Internet Protocol), SNA (System Network Architecture), and IPX (Internet Packet Switching). Etc. By way of example only, the (multiple) communication networks 1910 can be a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., according to the IEEE 802.11 protocol suite), etc. (and / or any other wireless protocol operating on a network) and / or any combination of these networks and / or other networks.

[0280] Server 1912 may consist of: one or more general-purpose computers, dedicated server computers (including PC (personal computer) servers by way of example), Server 1912 may be a server, mid-range server, mainframe computer, rack server, server cluster, or any other suitable arrangement and / or combination thereof. Server 1912 may include one or more virtual machines running a virtual operating system or other computing architectures involving virtualization, such as logical storage devices that can be virtualized to maintain one or more flexible pools of virtual storage devices for the server. In various examples, server 1912 may be adapted to run one or more services or software applications that provide the functionality described in the foregoing disclosure.

[0281] The computing system in server 1912 can run one or more operating systems, including any of the operating systems discussed above and any commercially available server operating system. Server 1912 can also run any of a variety of additional server applications and / or middleware applications, including HTTP (Hypertext Transfer Protocol) servers, FTP (File Transfer Protocol) servers, CGI (Common Gateway Interface) servers, etc. Servers, database servers, etc. Exemplary database servers include, but are not limited to, those that can be accessed from... Those database servers acquired through purchases by companies such as IBM.

[0282] In some implementations, server 1912 may include one or more applications to analyze and merge data feeds and / or event updates received from users of client computing devices 1902, 1904, 1906, and 1908. As an example, data feeds and / or event updates may include, but are not limited to, those provided in the original text. feed, The server 1912 may receive real-time updates from one or more third-party information sources and continuous data streams. These real-time updates may include real-time events related to sensor data applications, financial reporting machines, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, vehicle traffic monitoring, and the like. The server 1912 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of client computing devices 1902, 1904, 1906, and 1908.

[0283] The distributed system 1900 may also include one or more data stores 1914, 1916. In some examples, these data stores may be used to store data and other information. For example, one or more of the data stores 1914, 1916 may be used to store information such as information related to the storage virtual machine, information mapping application IDs to selected storage virtual machines, and other information used by the server 1912 when performing authentication functions. Data stores 1914, 1916 may reside in various locations. For example, the data store used by the server 1912 may be local to the server 1912 or may be located remotely to the server 1912 and communicate with the server 1912 via a network-based or dedicated connection. Data stores 1914, 1916 may be of different types. In some examples, the data store used by the server 1912 may be a database, such as a relational database, such as one built by Oracle. Databases from other vendors. One or more of these databases can be adapted to respond to SQL-formatted commands to store, update, and retrieve data from the database.

[0284] In some examples, one or more of the data stores 1914 and 1916 can also be used by the application to store application data. The data store used by the application can be of different types, such as key-value stores, object stores, or general-purpose stores supported by the file system.

[0285] In some examples, the functionality described in this disclosure may be provided as a service via a cloud environment. Figure 20 This is a simplified block diagram of a cloud-based system environment system 2000 for implementing some embodiments. In the cloud-based system environment system 2000, according to some examples, various services can be provided as cloud services. Figure 20 In the depicted example, cloud infrastructure system 2002 can provide one or more cloud services that can be requested by a user using one or more client computing devices 2004, 2006, and 2008. Cloud infrastructure system 2002 may include one or more computers and / or servers, which may include those computers and / or servers described above with respect to server 1912. The computers in cloud infrastructure system 2002 may be organized as general-purpose computers, dedicated server computers, server groups, server clusters, or any other suitable arrangement and / or combination.

[0286] Multiple networks 2010 can facilitate data communication and exchange between client computing devices 2004, 2006, and 2008 and cloud infrastructure system 2002. Multiple networks 2010 may include one or more networks. The networks may be of the same or different types. Multiple networks 2010 may support one or more communication protocols (including wired and / or wireless protocols) to facilitate communication.

[0287] Figure 20 The example depicted is merely one example of a cloud infrastructure system and is not intended to be limiting. It should be understood that in some other examples, the cloud infrastructure system 2002 can have more than... Figure 20 The depicted components may have more or fewer components, may combine two or more components, or may have different component configurations or arrangements. For example, although... Figure 20 Three client computing devices are depicted, but in alternative examples, any number of client computing devices can be supported.

[0288] The term cloud service is generally used to refer to services that become available to users on demand through a service provider's systems (e.g., cloud infrastructure systems 2002) and via communication networks such as the Internet. Typically, in a public cloud environment, the servers and systems that make up the cloud service provider's systems differ from the customer's own on-premises servers and systems. The cloud service provider's systems are managed by the cloud service provider. Therefore, customers can utilize cloud services provided by the cloud service provider without having to purchase separate licenses, support, or hardware and software resources for the services. For example, the cloud service provider's systems can host applications, and users can subscribe to and use the applications on demand via the Internet without having to purchase the infrastructure resources to run the applications. Cloud services are designed to provide easy, scalable access to applications, resources, and services. Several providers offer cloud services. For example, Oracle, based in Redwood Shores, California. It provides several cloud services, such as middleware services, database services, Java cloud services, and other services.

[0289] In some examples, cloud infrastructure system 2002 may use different models (such as Software as a Service (SaaS) model, Platform as a Service (PaaS) model, Infrastructure as a Service (IaaS) model, and other models (including hybrid service models) to provide one or more cloud services. Cloud infrastructure system 2002 may include a set of applications, middleware, databases, and other resources to enable the provisioning of various cloud services.

[0290] The SaaS model enables applications or software to be delivered as a service to customers via communication networks such as the Internet, without the customer having to purchase the underlying application's hardware or software. For example, the SaaS model can be used to provide customers with access to on-demand applications hosted by a cloud infrastructure system (System 2002). Examples of the SaaS services offered include, but are not limited to, various services for human resources / capital management, customer relationship management (CRM), enterprise resource planning (ERP), supply chain management (SCM), enterprise performance management (EPM), analytics services, social applications, and others.

[0291] The IaaS model is typically used to provide customers with infrastructure resources (such as servers, storage, hardware, and networking resources) as cloud services to offer elastic computing and storage capabilities. (Produced by Oracle) It provides a variety of IaaS services.

[0292] The PaaS model is typically used to provide a platform and environment resources as a service that enable customers to develop, run, and manage applications and services without requiring them to purchase, build, or maintain such resources. (Provided by Oracle) Examples of PaaS services provided include, but are not limited to, Oracle Java Cloud Service (JCS), Oracle Database Cloud Service (DBCS), Data Management Cloud Service, various application development solutions services, and other services.

[0293] Cloud services are typically delivered in an on-demand, self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. For example, a customer can subscribe to one or more services provided by a cloud infrastructure system 2002 via a subscription order. The cloud infrastructure system 2002 then performs processing to provide the services requested in the customer's subscription order. For instance, a user can request the cloud infrastructure system to register an application and provide services to the application according to its specified requirements, as described above. The cloud infrastructure system 2002 can be configured to provide one or more cloud services.

[0294] Cloud infrastructure system 2002 can provide cloud services through different deployment models. In a public cloud model, cloud infrastructure system 2002 can be owned by a third-party cloud service provider, and cloud services are provided to any general public customer, which can be an individual or a business. In some other examples, under a private cloud model, cloud infrastructure system 2002 can be operated within an organization (e.g., within a business organization) and services can be provided to customers within the organization. For example, customers can be various departments within the enterprise, such as human resources or payroll departments, or even individuals within the enterprise. In some other examples, under a community cloud model, cloud infrastructure system 2002 and the services provided can be shared by several organizations in the relevant community. Various other models, such as hybrids of the models mentioned above, can also be used.

[0295] Client computing devices 2004, 2006, and 2008 can be of different types and can be capable of operating one or more client applications. Users can use the client devices to interact with the cloud infrastructure system 2002, such as requesting services provided by the cloud infrastructure system 2002. For example, a user can use the client device to request authentication-related services as described in this disclosure.

[0296] In some examples, the processing performed by the cloud infrastructure system 2002 to provide services may involve big data analytics. This analytics may involve using, analyzing, and manipulating large datasets to detect and visualize various trends, behaviors, relationships, etc., within the data. This analytics may be performed by one or more processors, potentially processing the data in parallel, performing simulations, etc. For example, big data analytics may be performed by the cloud infrastructure system 2002 to determine which storage virtual machine to select for a particular application based on application-based claims and authentication requirements. The data used for this analytics may include structured data (e.g., data stored in a database or structured according to a structured model) and / or unstructured data (e.g., data blocks (binary large objects)).

[0297] like Figure 20 As illustrated in the examples, cloud infrastructure system 2002 may include infrastructure resources 2030 used to facilitate the provision of various cloud services offered by cloud infrastructure system 2002. Infrastructure resources 2030 may include, for example, processing resources, storage or memory resources, networking resources, etc. In some examples, a storage virtual machine that can be used to service storage requested from an application may be part of cloud infrastructure system 2002. In other examples, the storage virtual machine may be part of a different system.

[0298] In some examples, to facilitate efficient provisioning of these resources to support various cloud services offered by Cloud Infrastructure System 2002 to different customers, resources can be bundled into resource groups or resource modules (also known as “pods”). Each resource module or pod may include a pre-integrated and optimized combination of one or more types of resources. In some examples, different pods may be pre-provisioned for different types of cloud services. For example, a first pod may be provisioned for a database service, a second pod may be provisioned for a Java service (the second pod may include a different combination of resources than the pods in the first pod), and so on. For some services, resources allocated for provisioning the service may be shared between services.

[0299] The cloud infrastructure system 2002 itself can internally use services 2032 shared by different components of the cloud infrastructure system 2002 and that facilitate the provision of services by the cloud infrastructure system 2002. These internal shared services may include, but are not limited to, security and identity services, integration services, enterprise repository services, enterprise manager services, virus scanning and whitelisting services, high availability, backup and recovery services, services for enabling cloud support, email services, notification services, file transfer services, etc.

[0300] Cloud infrastructure systems 2002 can include multiple subsystems. These subsystems can be implemented in software or hardware, or a combination thereof. Figure 20 The depicted subsystem may include a user interface subsystem 2012 that enables users or customers of the cloud infrastructure system 2002 to interact with the cloud infrastructure system 2002. The user interface subsystem 2012 may include various interfaces, such as a web interface 2014, an online store interface 2016 (where advertising and customers can purchase cloud services provided by the cloud infrastructure system 2002), and other interfaces 2018. For example, a customer may use a client device to request (service request 2034) one or more services provided by the cloud infrastructure system 2002 using one or more of the interfaces 2014, 2016, and 2018. For example, a customer may access an online store, browse cloud services provided by the cloud infrastructure system 2002, and place a subscription order for one or more services provided by the cloud infrastructure system 2002 that the customer wishes to subscribe to. A service request may include information identifying the customer and the one or more services the customer wishes to subscribe to. For example, a customer may place a subscription order for services provided by the cloud infrastructure system 2002. As part of the order, the customer may provide information identifying the application for which the service is offered, and one or more credentials for the application.

[0301] In some examples (such as) Figure 20 In the illustrated example, cloud infrastructure system 2002 may include an order management subsystem (OMS) 2020 configured to process new orders. As part of this process, OMS 2020 may be configured to: create accounts for customers (if not already created); receive billing and / or billing information from customers to be used to bill customers for requested services; verify customer information; place orders for customers after verification; and schedule various workflows to prepare orders for supply.

[0302] Once correctly verified, OMS 2020 can invoke the Order Provisioning Subsystem (OPS) 2024, which is configured as order provisioning resources (including processing resources, storage resources, and networking resources). Provisioning may include allocating resources for an order and configuring resources to facilitate the service requested by the customer's order. The manner in which resources are provisioned for an order and the type of resources provided may depend on the type of cloud service the customer has already subscribed to. For example, according to a workflow, OPS 2024 may be configured to determine the specific cloud service being requested and identify the number of groups that may have been pre-configured for that specific cloud service. The number of groups allocated to an order may depend on the size / volume / tier / scope of the requested service. For example, the number of groups to be allocated may be determined based on the number of users the service is to support, the duration of the requested service, etc. The allocated groups can then be customized for a specific requesting customer to provide the requested service.

[0303] In some examples, the setup phase processing described above can be performed by the cloud infrastructure system 2002 as part of the provisioning process. The cloud infrastructure system 2002 can generate application IDs and select storage virtual machines for the application from storage virtual machines provided by the cloud infrastructure system 2002 itself or from storage virtual machines provided by other systems besides the cloud infrastructure system 2002.

[0304] The cloud infrastructure system 2002 can send a response or notification 2044 to the requesting client to indicate when the requested service is now ready for use. In some instances, information (e.g., a link) enabling the client to begin using and taking advantage of the benefits of the requested service can be sent. In some examples, for the client requesting the service, the response may include an application ID generated by the cloud infrastructure system 2002 and information identifying the virtual machine selected by the cloud infrastructure system 2002 corresponding to the application ID.

[0305] The cloud infrastructure system 2002 can provide services to multiple customers. For each customer, the cloud infrastructure system 2002 is responsible for managing information related to one or more subscription orders received from the customer, maintaining customer data related to the orders, and providing the requested services to the customer. The cloud infrastructure system 2002 can also collect usage statistics about customers' use of subscribed services. For example, it can collect statistics such as storage usage, data transfer volume, number of users, system uptime, and system downtime. This usage information can be used to issue bills to customers. Billing can be done, for example, on a monthly basis.

[0306] Cloud infrastructure system 2002 can provide services to multiple customers in parallel. Cloud infrastructure system 2002 can store information about these customers (potentially including proprietary information). In some examples, cloud infrastructure system 2002 includes an Identity Management Subsystem (IMS) 2028 configured to manage customer information and provide separation of the managed information so that information related to one customer cannot be accessed by another customer. IMS 2028 can be configured to provide various security-related services such as identity services, which may include, for example, information access management, authentication and authorization services, services for managing customer identities and roles, and related capabilities.

[0307] Figure 21 An example of a computer system 2100 for implementing some embodiments is illustrated. In some examples, the computer system 2100 can be used to implement any of the application systems, access management systems, data center systems, and various server and computer systems described above. Figure 21As shown, the computer system 2100 includes various subsystems, including a processing subsystem 2104 that communicates with multiple other subsystems via a bus subsystem 2102. These other subsystems may include a processing acceleration unit 2106, an I / O subsystem 2108, a storage subsystem 2118, and a communication subsystem 2124. The storage subsystem 2118 may include non-transitory computer-readable storage media, including computer-readable storage media 2122 and system memory 2110.

[0308] Bus subsystem 2102 provides a mechanism for allowing the various components and subsystems of computer system 2100 to communicate with each other as intended. Although bus subsystem 2102 is schematically shown as a single bus, alternative examples of bus subsystems may utilize multiple buses. Bus subsystem 2102 may be any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a local bus using any of a variety of bus architectures, etc. For example, such architectures may include an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus (which may be implemented as a mezzanine bus manufactured to the IEEE P1386.1 standard), etc.

[0309] Processing subsystem 2104 controls the operation of computer system 2100 and may include one or more processors, application-specific integrated circuits (ASICs), or field-programmable gate arrays (FPGAs). Processors may include single-core or multi-core processors. The processing resources of computer system 2100 may be organized into one or more processing units 2132, 2134, etc. A processing unit may include one or more processors, one or more cores from the same or different processors, a combination of cores and processors, or other combinations of cores and processors. In some examples, processing subsystem 2104 may include one or more dedicated coprocessors such as graphics processors or digital signal processors (DSPs). In some examples, some or all of the processing units of processing subsystem 2104 may be implemented using custom circuitry such as ASICs or FPGAs.

[0310] In some examples, the processing units in processing subsystem 2104 can execute instructions stored in system memory 2110 or on computer-readable storage medium 2122. In various examples, the processing units can execute various program or code instructions and can maintain multiple concurrently executing programs or processes. At any given time, some or all of the program code to be executed can reside in system memory 2110 and / or on computer-readable storage medium 2122 (potentially including residing on one or more storage devices). With suitable programming, processing subsystem 2104 can provide the various functions described above. In an instance where computer system 2100 is executing one or more virtual machines, one or more processing units can be assigned to each virtual machine.

[0311] In some examples, a processing acceleration unit 2106 may optionally be provided for performing custom processing or for offloading some of the processing performed by the processing subsystem 2104, thereby accelerating the overall processing performed by the computer system 2100.

[0312] I / O subsystem 2108 may include devices and mechanisms for inputting information to and / or outputting information from or via computer system 2100. Generally, the term input device is intended to include all possible types of devices and mechanisms for inputting information to computer system 2100. User interface input devices may include, for example, keyboards, pointing devices such as mice or trackballs, touchpads or touchscreens integrated into displays, scroll wheels, click wheels, dial pads, buttons, switches, keypads, audio input devices with voice command recognition systems, microphones, and other types of input devices. User interface input devices may also include motion sensing and / or gesture recognition devices, such as Microsoft's [unclear - possibly a specific technology or feature], which enables users to control and interact with input devices. Motion sensor, Microsoft The 360 ​​game controller provides an interface for receiving input using gestures and spoken commands. The user interface input device may also include eye gesture recognition devices, such as detecting eye movements from the user (e.g., "blinking" when taking a photo and / or making menu selections) and translating the eye gestures into the input device (e.g., Google). Google input Blink detector. Additionally, user interface input devices may include those enabling users to interact with a voice recognition system (e.g., ...) via voice commands. A voice recognition sensing device for interaction with navigators.

[0313] Other examples of user interface input devices include, but are not limited to, 3D mice, joysticks or pointing sticks, game controllers and graphics tablets, and audio / video devices such as speakers, digital cameras, digital camcorders, portable media players, webcams, image scanners, fingerprint scanners, barcode readers, 3D scanners, 3D printers, laser rangefinders, and eye-tracking devices. Additionally, user interface input devices may include, for example, medical imaging input devices such as computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), and medical ultrasound examination equipment. User interface input devices may also include, for example, audio input devices such as MIDI keyboards, digital musical instruments, etc.

[0314] Generally, the term "output device" is intended to encompass all possible types of devices and mechanisms for outputting information from computer system 2100 to a user or other computer. User interface output devices may include display subsystems, indicator lights, or non-visual displays such as audio output devices. Display subsystems may be cathode ray tube (CRT), flat panel devices (such as those using liquid crystal displays (LCDs) or plasma displays), projection devices, touchscreens, etc. For example, user interface output devices may include, but are not limited to, various display devices that visually convey text, graphics, and audio / video information, such as monitors, printers, speakers, headsets, car navigation systems, plotters, voice output devices, and modems.

[0315] Storage subsystem 2118 provides a repository or data store for storing information and data used by computer system 2100. Storage subsystem 2118 provides a tangible, non-transitory, computer-readable storage medium for storing basic programming and data constructs that provide some example functionality. Storage subsystem 2118 may store software (e.g., programs, code modules, instructions) that provides the functionality described above when executed by processing subsystem 2104. The software may be executed by one or more processing units of processing subsystem 2104. Storage subsystem 2118 may also provide certification according to the teachings of this disclosure.

[0316] The storage subsystem 2118 may include one or more non-transitory memory devices, which may include volatile memory devices and non-volatile memory devices. For example... Figure 21As shown, the storage subsystem 2118 includes system memory 2110 and computer-readable storage medium 2122. System memory 2110 may include multiple memories, including volatile main random access memory (RAM) for storing instructions and data during program execution and non-volatile read-only memory (ROM) or flash memory where fixed instructions are stored. In some embodiments, a basic input / output system (BIOS) containing basic routines that help transfer information between elements within computer system 2100 during startup may typically be stored in ROM. RAM typically contains data and / or program modules currently operated and executed by processing subsystem 2104. In some embodiments, system memory 2110 may include various types of memory such as static random access memory (SRAM), dynamic random access memory (DRAM), etc.

[0317] By using examples rather than restrictions, such as Figure 21 As depicted, system memory 2110 can load an executing application 2112 (which may include various applications such as web browsers, middleware applications, relational database management systems (RDBMS), etc.), program data 2114, and operating system 2116. By way of example, operating system 2116 may include various versions of Microsoft... Apple and / or Linux operating system, and various commercially available... Or a UNIX-like operating system (including but not limited to various GNU / Linux operating systems, Google...) OS, etc.) and / or such as iOS, Phone OS OS OS operating system and other mobile operating systems.

[0318] Computer-readable storage medium 2122 can store programming and data structures that provide some example functionality. Computer-readable storage medium 2122 can provide storage for computer-readable instructions, data structures, program modules, and other data for computer system 2100. Software (programs, code modules, instructions) that provides the functionality described above, when executed by processing subsystem 2104, can be stored in storage subsystem 2118. By way of example, computer-readable storage medium 2122 may include, for example, hard disk drives, disk drives, optical disc drives (such as CD-ROMs, DVDs, etc.). Non-volatile memory such as disks or other optical media. Computer-readable storage medium 2122 may include, but is not limited to, [other types of storage media]. Drives, flash memory cards, Universal Serial Bus (USB) flash memory drives, Secure Digital (SD) cards, DVD discs, digital videotapes, etc. Computer-readable storage media 2122 may also include SSDs based on flash memory, such as flash memory-based solid-state drives (SSDs), enterprise-class flash memory drives, SSDs based on non-volatile memory such as solid-state ROM, SSDs based on volatile memory such as solid-state RAM, dynamic RAM, static RAM, DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs using a combination of DRAM and flash memory-based SSDs.

[0319] In some examples, storage subsystem 2118 may also include a computer-readable storage medium reader 2120 that can be further connected to computer-readable storage medium 2122. Reader 2120 may receive data from and be configured to read data from storage devices such as disks, flash memory drives, etc.

[0320] In some examples, computer system 2100 may support virtualization technologies, including but not limited to virtualization of processing and memory resources. For example, computer system 2100 may provide support for executing one or more virtual machines. In some examples, computer system 2100 may execute programs such as hypervisors that facilitate the configuration and management of virtual machines. Each virtual machine may be allocated memory, computing (e.g., processor, cores), I / O, and networking resources. Each virtual machine typically runs independently of other virtual machines. Virtual machines typically run their own operating systems, which may be the same as or different from the operating systems executed by other virtual machines executed by computer system 2100. Therefore, multiple operating systems may potentially be run simultaneously by computer system 2100.

[0321] The communication subsystem 2124 provides an interface to other computer systems and networks. The communication subsystem 2124 serves as an interface for receiving data from other systems and transmitting data from computer system 2100 to other systems. For example, the communication subsystem 2124 enables computer system 2100 to establish a communication channel to one or more client devices via the Internet for receiving information from and / or sending information to client devices. For example, when computer system 2100 is used to implement... Figure 1 When the robot system 120 is described, the communication subsystem can be used to communicate with the application system and the system that executes the storage virtual machine selected for the application.

[0322] The communication subsystem 2124 may support both wired and / or wireless communication protocols. In some examples, the communication subsystem 2124 may include radio frequency (RF) transceiver components (e.g., using cellular phone technologies, advanced data network technologies such as 3G, 4G, EDGE (Global Evolution Enhanced Data Rate) or 5G, WiFi (IEEE 802.XX family of standards or other mobile communication technologies or any combination thereof), GPS receiver components, and / or other components for accessing wireless voice and / or data networks. In some examples, in addition to or instead of a wireless interface, the communication subsystem 2124 may provide wired network connectivity (e.g., Ethernet).

[0323] The communication subsystem 2124 can receive and transmit data in various forms. In some examples, among other forms, the communication subsystem 2124 can also receive input communications in the form of structured and / or unstructured data feeds 2126, event streams 2128, event updates 2130, etc. For example, the communication subsystem 2124 can be configured to receive (or send) data feeds 2126 in real time from users of social media networks and / or other communication services, such as... feed, Updates, web feeds (such as rich site summary (RSS) feeds) and / or real-time updates from one or more third-party information sources.

[0324] In some examples, the communication subsystem 2124 may be configured to receive data in the form of a continuous data stream, which may include an event stream 2128 and / or event updates 2130 of real-time events (which may be inherently continuous or unbounded and may not have an explicit end). Examples of applications that generate continuous data may include, for example, sensor data applications, financial reporting machines, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, vehicle traffic monitoring, etc.

[0325] The communication subsystem 2124 can also be configured to transmit data from computer system 2100 to other computer systems or networks. Data can be transmitted in various forms, such as structured and / or unstructured data feeds 2126, event streams 2128, event updates 2130, etc., to one or more databases that can communicate with one or more streaming data source computers coupled to computer system 2100.

[0326] Computer system 2100 can be one of various types, including handheld portable devices (e.g., Cellular phone Computing tablets, PDAs), wearable devices (e.g., Google...) Head-mounted displays, personal computers, workstations, mainframes, self-service kiosks, server racks, or any other data processing systems. Due to the constantly evolving nature of computers and networks, [the following is relevant:] Figure 21 The description of the computer system 2100 is intended only as a concrete example. It has a higher... Figure 21 Many other configurations with more or fewer components are possible for the system depicted. Based on this disclosure and the teachings provided herein, those skilled in the art should understand other ways and / or methods for implementing the various examples.

[0327] While specific examples have been described, various modifications, alterations, alternative constructions, and equivalents are possible. The examples are not limited to operations in a particular data processing environment but are free to operate in multiple data processing environments. Furthermore, although certain examples have been described using specific series of transactions and steps, it will be apparent to those skilled in the art that this is not intended to be restrictive. While some flowcharts describe operations as sequential processes, many operations can be performed in parallel or simultaneously. Additionally, the order of operations can be rearranged. Processes may have additional steps not included in the diagrams. Various features and aspects of the examples described above can be used individually or in combination.

[0328] Furthermore, while certain examples have been described using specific combinations of hardware and software, it should be recognized that other combinations of hardware and software are also possible. Some examples may be implemented using only hardware, only software, or a combination thereof. The various processes described herein can be implemented on the same or different processors in any combination.

[0329] When a device, system, component, or module is described as being configured to perform certain operations or functions, this configuration can be accomplished, for example, by designing electronic circuitry to perform operations, by programming programmable electronic circuitry (such as a microprocessor) to perform operations (such as by executing computer instructions or code), or by a processor or core programmed to execute code or instructions stored on a non-transitory memory medium, or any combination thereof. Processes can communicate using a variety of techniques, including but not limited to conventional techniques for inter-process communication, and different pairs of processes may use different techniques, or the same pair of processes may use different techniques at different times.

[0330] Specific details are set forth in this disclosure to provide a thorough understanding of the examples. However, the examples can be practiced without these specific details. For example, well-known circuits, processes, algorithms, structures, and techniques have been shown without unnecessary details to avoid obscuring the examples. This description is merely illustrative and is not intended to limit the scope, applicability, or configuration of other examples. Rather, the foregoing description of the examples will provide those skilled in the art with enabling descriptions for implementing the various examples. Various changes can be made to the function and arrangement of the elements.

[0331] Therefore, this specification and drawings should be viewed in an illustrative rather than restrictive sense. However, it will be apparent that additions, omissions, deletions, and other modifications and changes may be made without departing from the broader spirit and scope set forth in the claims. Thus, while specific examples have been described, they are not intended to be restrictive. Various modifications and equivalents are within the scope of the following claims.

[0332] In the foregoing description, various aspects of this disclosure have been described with reference to specific examples; however, those skilled in the art will recognize that this disclosure is not limited thereto. Various features and aspects of the disclosure described above may be used individually or in combination. Furthermore, the examples may be utilized in any number of environments and applications other than those described herein without departing from the broader spirit and scope of the specification. Therefore, the specification and drawings should be considered illustrative rather than restrictive.

[0333] In the foregoing description, the methods have been described in a specific order for illustrative purposes. It should be understood that, in alternative examples, the methods may be performed in a different order than described. It should also be understood that the methods described above may be executed by hardware components or may be embodied in a sequence of machine-executable instructions that can be used to cause a machine (such as a general-purpose or special-purpose processor or logic circuit programmed with the instructions) to execute the methods. These machine-executable instructions may be stored on one or more machine-readable media, such as CD-ROMs or other types of optical discs, floppy disks, ROMs, RAMs, EPROMs, EEPROMs, magnetic or optical cards, flash memory, or other types of machine-readable media suitable for storing electronic instructions. Alternatively, the methods may be executed by a combination of hardware and software.

[0334] When a component is described as being configured to perform certain operations, such configuration can be accomplished, for example, by designing electronic circuits or other hardware to perform operations, by programming programmable electronic circuits (e.g., microprocessors or other suitable electronic circuits) to perform operations, or any combination thereof.

[0335] Although illustrative examples of this application have been described in detail herein, it should be understood that the inventive concepts may be embodied and employed in other ways, and the appended claims are intended to be construed as including such variations, except where limited by the prior art.

Claims

1. A computer-implemented method, comprising: The chatbot system determines the classification results of the utterances; The chatbot system determines one or more anchors based on the utterance, each of the one or more anchors corresponding to one or more anchor words in the utterance, wherein determining the one or more anchors includes using a cluster search technique to select the one or more anchors in one or more rounds, wherein each of the selected one or more anchors satisfies a uniqueness threshold, such that the selected one or more anchors correspond to fewer alternative similar words compared to other candidate anchor words that were not selected as anchors; For each of the one or more anchor points: The chatbot system generates one or more synthetic utterances corresponding to the anchor points based on the anchor points and the utterances, and each of the one or more synthetic utterances includes the one or more anchor point words of the anchor points; The chatbot system determines one or more classification results for the one or more synthesized utterances, and determines the confidence value of the anchor point based on the one or more classification results; and The chatbot system generates a report that includes a representation of a specific anchor among the one or more anchors, where the specific anchor corresponds to the highest confidence value.

2. The computer-implemented method of claim 1, further comprising: The chatbot system determines one or more specific synthesized utterances corresponding to the specific anchor point based on the specific anchor point; The chatbot system generates a training dataset for training a machine learning model based on one or more specific synthesized utterances, the training dataset including a subset of the one or more specific synthesized utterances; as well as The machine learning model is trained by the chatbot system using the training dataset.

3. The computer-implemented method as described in claim 2, wherein, The training dataset is generated by selecting the subset of the one or more specific synthesized utterances by comparing the confidence values ​​of the subset with a confidence threshold.

4. The computer-implemented method as described in claim 1, wherein, Determining the confidence value of each of the one or more anchor points includes determining one or more matches between the classification result of the utterance and the one or more classification results of the one or more synthesized utterances.

5. The computer-implemented method as described in claim 1, wherein, Determining the confidence value for each of the one or more anchor points includes: The chatbot system determines one or more confidence values ​​for the one or more synthesized utterances based on the one or more classification results of the one or more synthesized utterances; and The chatbot system generates the confidence value of the anchor point based on the confidence values ​​of the one or more synthesized utterances.

6. The computer-implemented method as described in claim 1, wherein, The report includes multiple anchor points, each of which corresponds to a confidence value greater than a confidence threshold.

7. The computer-implemented method of claim 1, further comprising receiving a request from the chatbot system to generate a report for the utterance, wherein, The determination of the one or more anchor points is performed in response to receiving the request to generate the report.

8. The computer-implemented method as described in claim 1, wherein, Generating the one or more synthetic utterances includes generating synthetic utterances, each synthetic utterance including the one or more anchor words of the anchor and one or more replacement words, each of the replacement words of the synthetic utterance being a variant of a non-anchor word in the utterance.

9. The computer-implemented method as claimed in claim 1, wherein: The classification result of the utterance is based on a machine learning model, which is configured to output a classification result based on the input utterance; and The one or more classification results are generated based on the machine learning model.

10. A chatbot system, comprising: One or more processors; as well as A non-transitory computer-readable storage medium storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including the following: Determine the classification results of the discourse; Based on the discourse, one or more anchors are determined, each of the one or more anchors corresponding to one or more anchor words in the discourse, wherein determining the one or more anchors includes using a cluster search technique to select the one or more anchors in one or more rounds, wherein each of the selected one or more anchors satisfies a uniqueness threshold such that the selected one or more anchors correspond to fewer alternative similar words compared to other candidate anchor words that were not selected as anchors; For each of the one or more anchor points: Based on the anchor point and the discourse, one or more synthetic discourses corresponding to the anchor point are generated, and each of the one or more synthetic discourses includes the one or more anchor point words of the anchor point; Determine one or more classification results for the one or more synthesized utterances, and determine the confidence value of the anchor point based on the one or more classification results; and A report is generated that includes a representation of a specific anchor point among the one or more anchor points, wherein the specific anchor point corresponds to the highest confidence value.

11. The chatbot system of claim 10, wherein the one or more processors perform further operations including: Based on the specific anchor point, determine one or more specific synthesized utterances corresponding to the specific anchor point; A training dataset for training a machine learning model is generated based on the one or more specific synthesized utterances; the training dataset includes a subset of the one or more specific synthesized utterances; and The machine learning model is trained using the training dataset.

12. The chatbot system as described in claim 11, wherein, The training dataset is generated by selecting the subset of the one or more specific synthesized utterances by comparing the confidence values ​​of the subset with a confidence threshold.

13. The chatbot system as described in claim 10, wherein, Determining the confidence value of each of the one or more anchor points includes determining one or more matches between the classification result of the utterance and the one or more classification results of the one or more synthesized utterances.

14. The chatbot system of claim 10, wherein the one or more processors perform further operations including: Based on the classification results of the one or more synthesized utterances, determine one or more confidence values ​​for the one or more synthesized utterances; and The confidence value of the anchor is generated based on the confidence values ​​of the one or more synthesized utterances.

15. The chatbot system as described in claim 10, wherein, Generating the one or more synthetic utterances includes generating synthetic utterances, each synthetic utterance including the one or more anchor words of the anchor and one or more replacement words, each of the replacement words of the synthetic utterance being a variant of a non-anchor word in the utterance.

16. A computer-implemented method, comprising: One or more synthetic utterances are generated based on the utterance, and each of the one or more synthetic utterances includes one or more words shared with the utterance; Determine one or more confidence values, wherein each confidence value is associated with each corresponding synthetic utterance in the one or more synthetic utterances, the determination comprising processing the one or more synthetic utterances by a machine learning model to obtain an intent prediction for each corresponding synthetic utterance and a confidence value for the predicted intent, the confidence value indicating that a particular predicted intent of the corresponding synthetic utterance corresponds to a confidence level of the corresponding synthetic utterance; Compare the one or more confidence values ​​with a confidence threshold; Based on the comparison, a subset of the one or more synthesized utterances is determined, the determination including selecting a synthesized utterance for the subset of the one or more synthesized utterances if the synthesized utterance corresponds to a confidence value less than or equal to the confidence threshold; Receive one or more training categories, each of the one or more training categories corresponding at least to a synthetic utterance in the subset of the one or more synthetic utterances; and A training dataset for tuning the machine learning model is generated based on the subset of the one or more synthesized utterances and one or more training categories.

17. The computer-implemented method of claim 16, further comprising adjusting the machine learning model using the training dataset.

18. The computer-implemented method of claim 17, wherein, The machine learning model is constructed as an intent classifier, which includes multiple model parameters learned by using an objective function, and training the machine learning model includes minimizing or maximizing the objective function, which measures the difference between the predicted intent and the correct intent.

19. The computer-implemented method of claim 16, wherein, The confidence threshold is received from a user of the interactive interface, wherein the user inputs the confidence threshold into an interactive interface element.

Images