Computer implementation method, computer program, and computer system for improving intent determination in messaging dialog management systems (improvement of intent determination in messaging dialog management systems)

Abstract

To provide a computer-implemented method for enhancing intent determination in a messaging dialog manager system, a computer program product, and a computer system.SOLUTION: A method includes one or more processors configured to receive first agent entry data corresponding to a first agent communicating in a messaging dialog interface, determine that the first agent entry data expects a response that is within a first response domain, and determine that a first user entry entered in the messaging dialog interface is not within the first response domain. Further, the method includes identifying a second agent configured with a second response domain including the first user entry, and transmitting the first user entry to the second agent to facilitate a seamless transition of an established communicational flow between the first agent and a first user.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention generally relates to the field of data processing, and more specifically, to messaging dialog management C in the stem figure to improve decision-making.

Background Art

[0002] Conversational agents have become a popular online method for product and service providers to interact with their customers and users. A conversational agent is in the form of a software agent and can be referred to as a chatbot, chatterbot, Artificial Conversational Entities, smart assistant, chat robot, or other chat schemes. A conversational agent is a system that mimics a conversation with a user either by voice or text to provide information and services to the user and also to solicit and receive information from the user. The purpose is for the conversational agent to accurately understand the user query and respond accordingly. For example, a conversational agent can receive a user query, determine the intent and purpose of the user query, return a relevant response, or execute a task or action.

[0003] However, conversational agents are still burdened with a number of limitations that prevent their widespread use. One important function is to identify the intent behind a message entry (e.g., utterance, text), i.e., whether the utterance is an inquiry about information or providing information. Identifying the intent behind a message entry can significantly improve the agent's decision-making when experiencing an unexpected response from the user.

[0004] Conversational agents are typically designed for a limited number of use cases, often a single use case. For example, a user may use a weather conversational agent for weather forecasts I asked him related suppliers versus By accessing a conversational agent, users can book travel or entertainment tickets, purchase products from retailers, and record information such as receipts using an interactive agent in an accounting system. profit Interactive agents enable users to interact with product and service providers in a simple and intuitive way. [Overview of the Initiative] [Problems that the invention aims to solve]

[0005] During a chatbot session with a user, the user may provide ambiguous utterances that the chatbot may not be able to distinguish between a question and a statement. Misunderstandings can be serious when machines and humans interact, especially in natural language. result This could happen. To avoid ambiguous statements that could cause unexpected effects, a certain expression (e.g., via an agent or entity) requests information. Is that so? Providing information Is that so? It is important to identify the intentions behind specific interactions. [Means for solving the problem]

[0006] This disclosure addresses the identified shortcomings described above. Embodiments of the present invention relate to messaging dialog management. C In the stem figure Discloses computer implementation methods, computer program products, and computer systems for improving decision-making. Messaging dialog management. C In the stem figureA computer implementation method for improving decision-making involves receiving first agent entry data corresponding to a first agent communicating in a messaging dialog interface, determining that the first agent entry data anticipates a response within a first response domain, determining that a first user entry entered into the messaging dialog interface is not within the first response domain, identifying a second agent consisting of a second response domain containing the first user entry, and establishing a relationship between the first agent and the first user communicating in the messaging dialog interface. Ta To facilitate a seamless transition of the communication flow, the system may include one or more processors configured to transmit a first user entry to a second agent.

[0007] In an embodiment, the computer implementation method may further include the steps of identifying agent text data corresponding to natural language (NL) text in first agent entry data, and identifying user text data corresponding to NL utterances in first user entry.

[0008] In an embodiment, the computer implementation method may further include the steps of: determining a first agent data entry topic based on agent text data, wherein the first response domain corresponds to the first agent data entry topic; determining a first user entry topic based on user text data; and comparing the first agent data entry topic with the first user entry topic to determine whether a similarity threshold is met.

[0009] In one embodiment, the computer implementation method may further include the step of determining that a first user entry is not in a first response domain, at least based on the determination that the similarity threshold does not exceed a predetermined value.

[0010] In an embodiment, the computer implementation method may further include the steps of processing agent text data with a first trained machine learning model to generate model output data corresponding to the response expectation classification, and determining the NL text as first agent entry data that is expected to respond, if the response expectation classification satisfies the conditions.

[0011] In an embodiment, the first user entry may include a document entry, and the computer implementation method may further include the steps of extracting document entry data from the document entry, and processing the document entry data to determine natural language text data, wherein the step of determining that the first user entry is not in the first response domain is at least based on the natural language text data.

[0012] In this embodiment, the step of transmitting the first entry to the second agent may correspond to a subject-official dialogue flow between the first user and the second agent in the messaging dialog interface.

[0013] In an embodiment, the step of determining that a first agent entry is expected to respond may further include: attaching part-of-speech (POS) tags to one or more words in the NL text to generate tagged NL text data; encoding the agent text data into sentence embeddings having 768 or fewer dimensions; processing the tagged NL text data and sentence embeddings with a second trained machine learning model to generate model output data corresponding to the response expectation classification; and determining that the NL text is the first agent entry data that is expected to respond if the response expectation classification satisfies the conditions. [Brief explanation of the drawing]

[0014] [Figure 1]A block diagram of a distributed data processing environment for improving intent determination in a messaging dialog system according to an embodiment of the present invention is shown.

[0015] [Figure 2] A block diagram of a system for improving intent determination in a messaging dialog system according to an embodiment of the present invention is shown.

[0016] [Figure 3] A model for improving intent determination in a messaging dialog management system according to an embodiment of the present invention is shown.

[0017] [Figure 4] An operational stage of a computer-implemented method for improving intent determination in a messaging dialog system according to an embodiment of the present invention is shown.

[0018] [Figure 5] A block diagram of components of a server computer within the distributed data processing environment of FIG. 1 according to an embodiment of the present invention is shown.

Best Mode for Carrying Out the Invention

[0019] Embodiments of the present invention describe a computer-implemented method, system, and computer program product for improving intent determination in a messaging dialog management C stem. As described herein, a data-driven machine learning approach is provided for identifying interrogative, imperative, and declarative sentences by compiling an English-labeled dataset and training multiple models on syntactic features and figure embedding sentences. Natural language interaction is such that natural language utterances are structured Based on transformation ​It is highly likely that this information has not been recorded, and that the content may change significantly, making it a difficult task for machines to learn. Ambiguous or unclear statements, speakers randomly changing the subject or moving away from incomplete conversations, and other common user behaviors increase the challenge for machines conversing with people.

[0020] Focusing on natural language comprehension, the goal is to understand natural language utterances ranging from their syntactic structure and word roles within a sentence to their semantic meaning, context, and other tasks. The embodiments described herein are dialogues. Specific grammatical sentence structures in an environment , In particular Interrogative sentences and life ordinance (for all other than these) The focus is on the task of identification. Identifying sentence types is useful for conversational agents in multiple scenarios. For example, identifying statements that are awaiting a response (i.e., question or command statements) allows the chatbot to determine whether a deviation has occurred if the statement is not addressed or not responded to appropriately.

[0021] Also, multi-agent chatbots environment In this context, identifying the type of utterance can contribute to the dialog manager's entity determination process, which handles responses, conversational exchanges, context, and other dialogue artifacts. Finally, classifying sentences or data entries allows interactive agents (embodied or virtual) to determine whether they are information providers and whether they are information recipients. This determination updates a corresponding knowledge base, configured to facilitate seamless transitions between different topics within an ongoing communication session. for It can be provided to the dialog manager.

[0022] The embodiments described herein are machine learning models in response Answer Natural language utterances that anticipate (For example, "Who are the top 5 banks?" and "Get the names of the top 5 banks") or not expecting a response. Natural Language Utterance(For example, "The following banks are in the top 5") knowledge separate Train This provides a computer implementation method configured as follows. Unlike performing intent recognition, which also considers the domain of the request, this classification does not need to be domain-dependent. For example, a dataset of natural language utterances in a particular language (e.g., English) may be compiled from existing benchmarks in the literature, and the dataset may be labeled based on two classes (i.e., expect a response, do not expect a response). Furthermore, a machine learning model (e.g., shallow) can be used. 、 Deep learning involves various features, including text embeddings and syntactic features. Based on It can be trained. As a result, the identification of specific natural language tasks and their importance in a messaging dialogue system is defined and contributes to the embodiments described herein. Furthermore, the embodiments described herein include the collection of labeled sentences and a machine learning model trained on classification tasks.

[0023] The embodiments described herein may apply to both user utterances or responses generated by a conversational agent, and a deviation from the conversation occurs when a message entered into a messaging dialog system is responded to. of Identification may be based at least on whether it is expected or not. In a multi-turn, goal-oriented interactive agent system, the flow of a dialogue can be visualized as branches traversing leaves from the root in a dialogue tree. In a multi-turn conversation, the flow may involve hopping between many branches as the user deviates from one topic to another. Understanding when an utterance expects a response helps determine whether the conversation should continue along a branch or deviate instead.

[0024] The embodiments described herein provide machine learning models that capture the syntactic structure and overall meaning of a sentence. The characteristics of the sentence response Whether or not to expectIdentify the following. For example, if an agent asks a question, there is a high probability that the conversation will proceed in multiple turns. Embodiments described herein are task Represents and unknown Generalizes well to the data do The focus is on generating the most suitable features for this purpose. Furthermore, embodiments in this specification focus on selecting models that incorporate multiple methods in which the intent to request information or a response may be constituted. Simply employing an "is question" classifier would miss utterances such as "Please specify the column you would like to plot."

[0025] In an embodiment, an appropriate dataset is created to train a machine learning model in order to learn a classification task. The dataset mainly consists of two types of sentences in a conversation, namely, responses. of What to expect (Class 1) It distinguishes between utterances that do not elicit a response (Class 0). Class 0 utterances are those that search for information. oh Instead, demand We have not done so These may be opinions, declarations, or comments from either a user or a chatbot. Class 1 utterances are those that Are you asking a question, or Change the state of the system or the world RequestTherefore, a response is expected. For example, in the context of a conversation between a user and a loan agent, the user might ask, "Can I check the status of my loan application?" (Class 1), and the agent might respond, "Yes, that's the information you requested" (Class 0). However, identifying a Class 1 utterance is not always as simple as checking whether the sentence contains a question mark, as other grammatical structures may be used. Examples may include: Who is the top borrower? (i.e., a WH-word question, e.g., who, what, when, where, why, how), Do I have any pending applications?, Are the pending applications for Manager 1 or Manager 2? (i.e., a yes-no or multiple-choice question), I don't have any pending requests, do I? (i.e., a disjunctive or tag question), Show me the application I submitted (i.e., an imperative verb), I want to know / need to know the status of my request (i.e., an imperative), I'd like my credit score, please (i.e., an imperative), What is my annual income? (i.e., other).

[0026] Furthermore, the embodiments described herein may also take into account the characteristics of an interactive environment (for example, the user may not always provide grammatically correct sentences). Noisy data may include questions without question marks, incomplete sentences or phrases, or any number of other grammatical errors or misspellings.

[0027] Embodiments described herein may include encoding a sentence into sentence embeddings using a Siamese Bidirectional Encoder Representations from Transformers (BERT) network having a pre-trained BERT model (bert-base-nli-mean-tokens). For example, two different sets of embeddings (e.g., base and Compression may be used to generate classifications. For example, from a pre-trained model, the principal component analysis (PCA) method may be used. BERT Default 768-dimensional vector mosquitoThese may be used to compress to dimensions of 100, 200, and 400.

[0028] In addition to creating text embeddings, the embodiments described herein are models 、 Since it may be possible to learn the patterns behind the tags that trigger responses, part-of-speech (POS) tags are configured to capture the syntactic structure of a sentence. For example, POS tagging in the Natural Language Toolkit (NLTK) may be used to create count and position features by comparing the frequency of occurrence of tags in a sentence with the position of their first occurrence. To calculate the position, the distance of each word from the center of the sentence, measured by its index, is used to determine the length of the sentence. in Divide and others Location values ​​close to -1 are closer to the beginning of a sentence, and values ​​close to 1 are closer to the end. Location values ​​may be shifted by +1 to remove negative values ​​when used as input to some feature selection algorithms that require non-negative inputs.

[0029] The embodiments described herein may include a plurality of feature selection algorithms for performing automatic feature selection. For example, Pearson correlation filtering, chi-squared filtering, and Kullback-Leibler divergence may be used to perform automatic feature selection.

[0030] The embodiments described herein may include one or more models selected for a particular application based on certain application parameters. For example, application parameters may include, among other things, the complexity / simplicity of the problem, computational constraints on training / testing (e.g., spatial and temporal), and problem characteristics (e.g., feature independence / dependence, amount of data noise, randomness, stationarity). Each model may have a set of hyperparameters (e.g., feature mapping capabilities, number of hidden neurons) that can be tuned to optimize training.

[0031] The embodiments described herein may include various types of machine learning models, and the techniques for training the machine learning models may include messaging dialog management. C In the stem figure These techniques are used to improve decision-making. For example, supervised learning techniques may be applied to shallow models (e.g., SVM, k-nearest neighbors (kNN), random forest, decision tree, naive Bayes) that serve as a baseline for comparison with deep learning models. Furthermore, embodiments described herein may include performing grid searches to fine-tune the hyperparameters of SVM models, kNN models, or multilayer perceptron (MLP) models.

[0032] Embodiments described herein emulate a bidirectional-long short-term memory (LSTM) model to perform natural language processing (NLP) tasks. of It may include deep learning models that provide the latest results. For example, in an interactive environment, both the first and second halves of a sentence may be analyzed to provide context and help determine the meaning or intent of the sentence. Thus, a bidirectional LSTM is different from a unidirectional LSTM. In comparison, Front and rear Department Both minutes ite This can provide improved results when analyzing text.

[0033] Embodiments described herein train machine learning models using arrays of datasets from interactive natural language literature. With respect to ground truth, the data are 45% (i.e., label: expects response)–55% (label: does not expect response). of To create data partitions, annotations can be added using a semi-automatic method. For example, semi-automatic annotations can be used to create data partitions. So This may include identifying labels for data points on the dataset, and example sentences from the dataset that do not necessitate a response in a multi-turn conversation (e.g., "a very, very, very slow, purposeless film about a young man who is shipwrecked and adrift") would be labeled as such. TheOther sentences similar to the example sentence have the same characteristics and structure. share It would be similarly labeled if it were to do so. Alternatively, in a multi-turn conversation, another example sentence from the dataset that evokes the need for a response (e.g., "What was the last year this team was in the USL A League?") would be labeled as such.

[0034] Embodiments described herein may include training a machine learning model using a combination of pre-trained BERT sentence embeddings and POS tags as input features for the machine learning model. For example, the input sentence embeddings may include 768, 400, 200, or 100 dimensions, scaled by approximately two times. Furthermore, the sentence embeddings are paired with a 768-dimensional sentence vector. death Furthermore, by applying a t-distribution-type stochastic neighbor embedding method (t-SNE) with a perplexity of 50, it can be visualized in a two-dimensional space. In addition, 10-fold cross-validation is performed. but This can be performed on sentence vectors to report training / validation performance.

[0035] Embodiments described herein may include incorporating more natural language structure from sentences by attaching sentence POS tags (e.g., count, location) to the feature set. By including POS tags, the model achieves improved performance because the POS tags assist the model in learning natural language sentence structure. Furthermore, by aggregating all shallow model results and feature sets, the SVM-based classifier model When trained with POS tag features and BERT text embeddings , resulting in minimal bias, unknown Generalize sufficiently to the data.

[0036] Embodiments described herein may include bidirectional LSTM models for text embeddings of dimensions 768, 400, 200, or 100. Text embeddings and POS tags may be included as feature sets for the bidirectional LSTM model to produce satisfactory results demonstrating that the bidirectional LSTM model performs best with the addition of syntactic linguistic features to the text embeddings.

[0037] Embodiments of the present invention provide an efficient and convenient method for managing messaging dialogs. C In the stem figure We recognize that custom solutions are desired to improve decision-making. Implementations of embodiments of the present invention may take various forms, and details of exemplary implementations will be described later with reference to the figures.

[0038] Figure 1 shows a messaging dialog management according to an embodiment of the present invention. C In the stem figure A block diagram of a distributed data processing environment for improving decision-making is shown. Figure 1 is provided only as an illustration of one embodiment of the present invention and does not imply any limitation to an environment in which different embodiments may be implemented. In the embodiment shown, the distributed data processing environment 100 includes user devices 120, a server 125, and a database 124 interconnected via a network 110. The network 110 can be, for example, a local area network (LAN), a wide area network (WAN), or a combination of the two, and operates as a computing network that may include wired, wireless, or fiber optic connections. Generally, the network 110 can be any combination of connections and protocols that support communication between the user devices 120, the server 125, and the database 124. The distributed data processing environment 100 may include additional servers, computers, or other devices not shown.

[0039] As used herein, the term “distributed” describes a computer system comprising multiple physically separate devices operating together as a single computer system. Many modifications to the described environment can be made by those skilled in the art without departing from the scope of the invention as described in the claims.

[0040] User device 120 manages messaging dialogs. C In the stem figure Computer programs to improve decision-making Small Even without Part It may be able to operate in such a way. In an embodiment, the user device 120 may be configured to send and / or receive data from one or more of the database 124 and the server 125 via the network 110. The user device 120 may include a user interface 122 configured to facilitate interaction between the user and the user device 120. For example, the user interface 122 may include a display as a mechanism for displaying data to the user, which may be, for example, a touchscreen, a light-emitting diode (LED) screen, or a liquid crystal display (LCD) screen. The user interface 122 may also include a keypad or text entry device configured to receive alphanumeric entries from the user. The user interface 122 may also include other peripheral components that further facilitate user interaction or data entry by the user associated with the user device 120.

[0041] In some embodiments, the user device 120 may be a management server, web server, or any other electronic device or computing system capable of receiving and transmitting data. In some embodiments, the user device 120 may be a laptop computer, tablet computer, netbook computer, personal computer (PC), desktop computer, smartphone, or any programmable electronic device capable of communicating with the database 124 and server 125 via the network 110. The user device 120 may include components as described in more detail in Figure 5.

[0042] Database 124 acts as a repository for data from and flowing to network 110. Examples of data include data corresponding to communications entering and receiving via user interface 122. A database is a systematic collection of data. Database 124 is stored in a database server, hard disk drive, or flash memory, etc. 、 User device 120 a It can be implemented in any type of storage device capable of storing data and configuration files that can be accessed and used. In one embodiment, the database 124 is accessed by a user device 120 to store data corresponding to messaging communication with the user interface 122 via the user interface 122. In another embodiment, the database 124 is accessed by a network 110. can Subject to the condition that it may reside elsewhere within the distributed network environment 100.

[0043] Server 125 can be a standalone computing device, management server, web server, or any other electronic device or computing system capable of receiving, transmitting, and processing data, and communicating with user devices 120 or database 124 or both via network 110. In another embodiment, Server 125 represents a server computing system that utilizes multiple computers as a server system, such as in a cloud computing environment. In yet another embodiment, Server 125 is within a distributed data processing environment 100. in This represents a computing system that utilizes clustered computers and components (e.g., database server computers, application server computers, etc.) that, when accessed, operate as a single, seamless pool of resources. Server 125 may include components as described in more detail in Figure 5.

[0044] Figure 2 shows the messaging dialog system according to an embodiment of the present invention. figure A block diagram of System 200 for improving decision-making is shown.

[0045] In one embodiment, the system 200 is For example, the user device 120 in Figure 1, etc. User interface 210 on the display of a computing device of The system may include one or more processors configured to generate. Furthermore, the system 200 may include one or more processors configured to generate a messaging dialog interface via a user interface 210, the messaging dialog interface may be configured to facilitate communication between a user and an agent (e.g., an interactive agent, a chatbot).

[0046] In this embodiment, the system 200 facilitates the interaction or communication flow between the user and the agent by including agent entry data 212 in the messaging dialog interface.1-N and user entry data 214 1-N The system may include one or more processors configured to receive, where N can represent any number of instances of agent entry data and user entry data. For example, one or more processors may be configured to receive first agent entry data 2121 and first user entry data 2141 in a messaging dialog interface (e.g., user interface 210). The entry data may include data corresponding to events (e.g., natural language phrases, non-natural language document entries). Additional agent and user entries may be exchanged within the messaging dialog interface to establish a user-agent interaction or communication flow similar to a normal dialog. The interaction or communication flow may be interrupted if the agent is unable to respond to a user entry, and the agent may respond with an error message or with an agent entry that is inconsistent with what the user expected.

[0047] System 200 uses a natural language processing (NLP) engine (not shown) to execute within the user interface 210. So The NLP engine can be configured to interpret the entry data received in the messaging dialog. At any point in time This is the core component that interprets utterances and converts them into structured input that the system can process. The NLP engine may include evolutionary machine learning algorithms that identify the intent in user and agent utterances, and further, the user's intent in a list of available actions supported by the chatbot present in the system. figureTo match them. For example, the NLP engine may use either a finite-state automated model or a deep learning model to generate system-generated responses to user and agent utterances. The NLP engine may include an intent classifier and an entity extractor, the intent classifier may be configured to interpret the natural language of the utterance, and the entity extractor may be configured to extract key information or keywords from the utterance.

[0048] In an embodiment, the system 200 may include one or more processors configured to identify agent text data corresponding to natural language (NL) text in a first agent entry data 2121. Furthermore, one or more processors may be configured to identify user text data corresponding to natural language (NL) utterances in a first user entry 2141. For example, an NLP engine may be configured to process agent text data to identify NL text in the first agent entry and to process user text data to identify NL text in the first user entry.

[0049] In one embodiment, the system 200 receives agent entry data 212 1-N It may include an expected response component 220 configured to determine whether it expects a response within the response domain. For example, the expected response component 220 may perform feature extraction on the NL text entry data and the extracted features and The Process NL text entry data, The It may include one or more machine learning models configured to determine whether NL text entry data is expected to respond or not.

[0050] The above describes an implementation of a machine learning model, but this disclosure is not limited thereto. In at least some embodiments, a machine learning model may implement a trained component or a trained model configured to perform the operations described above. The trained component may include, but is not limited to, one or more machine learning models, one or more classifiers, one or more neural networks, one or more probability graphs, one or more decision trees, and others. In other embodiments, the trained component is 、 Natural language input complicated or not complicated The system may include a rule-based engine for determining whether the input is natural language, one or more statistics-based algorithms, one or more mapping functions, or other types of functions / algorithms. In some embodiments, the trained component may be configured to perform binary classification, where the natural language input can be classified into one of two classes / categories. In some embodiments, the trained component may be multiclass classification Alternatively, the system may be configured to perform multi-class classification, where the natural language input can be classified into one of three or more classes / categories. In some embodiments, the trained component may be configured to perform multi-label classification, where the natural language input can be associated with more than one class / category.

[0051] Various machine learning techniques can be used to train and manipulate pre-trained components to perform the various processes described herein. Models can be trained and manipulated according to various machine learning techniques. Such techniques may include, for example, neural networks (e.g., deep neural networks or recurrent neural networks or both), inference engines, and pre-trained classifiers. Examples of pre-trained classifiers include support vector machines (SVMs), neural networks, decision trees, AdaBoost (short for "Adaptive Boosting") combined with decision trees, and random forests. Focusing on SVMs as an example, SVMs analyze data and The This is a supervised learning model that uses relevant learning algorithms to recognize patterns in data, and these learning algorithms are generally used for classification and regression analysis. Each item is marked as belonging to one of two categories. Examples of a series of training exercises Given Then, S VM training algorithms new examples on the other hand Category or other Category Build a model to assign to, This model Let this be a non-probabilistic binary linear classifier. More complex SVM models are: 2 It can be built using a training set that identifies more than one category. SVM determines which category is most similar to the input data. The SVM model can map examples of distinct categories so that they are separated by a clear gap. New examples are then mapped into the same space, on either side of the gap. New example Based on where it is located to belong category but It is predicted. The classifier will use the data to determine which is the most important. near It may issue a "score" indicating which category it matches. data How much Soon It can provide an indication of whether it matches the category.

[0052] In order to apply machine learning techniques, the machine learning processes themselves need to be trained. Training machine learning components is an example of training. ofIt requires establishing "ground truth." In machine learning, the term "ground truth" refers to the accuracy of classification of a training set for supervised learning techniques. This includes backpropagation, statistical learning, supervised learning, semi-supervised learning, probabilistic learning, or other known techniques. Various technologies but , Mo It may be used to train Dell.

[0053] In an embodiment, the expected response component 220 may further include one or more processors configured to process agent text data with a first trained machine learning model to generate model output data corresponding to the expected response classification, and to determine the NL text as first agent entry data that expects a response if the expected response classification satisfies the conditions. The expected response classification may correspond to a first class (Class 1) indicating that the entry data expects a response, or a second class (Class 0) indicating that the entry data does not expect a response. Class 0 entry data, including NL utterances, are searching for information. oh did not Or, request Perform This may be an opinion, declaration, or comment from a non-existent user or agent. Class 1 NL utterances are: quality Asking a question or Change the state of the system or the world request We are doing Therefore, a response is expected. The conditions may include binary classification or scores corresponding to binary classification.

[0054] In one embodiment, the first trained The machine learning model may include a shallow model, as described above herein, which is configured to generate model output data in response to receiving and processing NL text data. 、 Various features (for example) 、 The model is trained on text embeddings and syntactic features. Model output data may include binary classification indicating whether the NL text data is expected to respond or not.

[0055] In one embodiment, the first trainedThe machine learning model may include one or more shallow models or deep learning models, as described herein, and the shallow and deep learning models are configured to generate model output data in response to receiving and processing NL text data. 、 The model is trained on various features (e.g., text embeddings, syntactic features). Model output data may include binary classification indicating whether the NL text data is expected to respond or not. This determination improves the dialogue experience in situations where deviation or deambiguation occurs.

[0056] In some embodiments, the expected response component 220 may further include one or more processors configured to attach part-of-speech (POS) tags to one or more words in the NL text to generate tagged NL text data. Furthermore, the expected response component 220 may include one or more processors configured to encode the agent text data into sentence embeddings having 768 or fewer dimensions.

[0057] In one embodiment, the expected response component 220 processes tagged NL text data and sentence embeddings using a second trained machine learning model. hand The system may include one or more processors configured to generate model output data corresponding to the response expectation classification. Furthermore, one or more processors may be configured to determine the NL text as the first agent entry data that expects a response, if the response expectation classification satisfies certain conditions.

[0058] In an embodiment, the first user entry may include a document entry, and the system 200 may further include one or more processors configured to extract document entry data from the document entry, process the document entry data, and determine the NL text data, and the determination that the first user entry is not in the first response domain is based at least on the natural language text data.

[0059] In an embodiment, one or more processors may be configured to compare a first agent data entry topic with a first user entry topic to determine whether a similarity threshold is met. For example, if the first user entry topic is a promise topic and the first agent data entry topic is a general help query topic, the topics are not similar and therefore the similarity threshold is not met. Na stomach 。 As another example, if the first user entry topic is a reserved topic and the first agent data entry topic is also a reserved topic, the similarity threshold is satisfied because the topics are the same. ru .

[0060] In embodiments, the system 200 may include a response domain component 230 configured to determine whether a response is within a response domain. For example, the response domain component 230 may include one or more processors configured to determine a first agent data entry topic based on agent text data, where the first response domain corresponds to the first agent data entry topic. Furthermore, the response domain component 230 may include one or more processors configured to determine a first user entry topic based on user text data. Furthermore, the response domain component 230 may include one or more processors configured to compare the first agent data entry topic with the first user entry topic and determine whether a similarity threshold is met. For example, if the first agent data entry topic is determined to be a reservation topic based on agent text data including a question about a reservation, and also based on user text data including a statement about a reservation, the first user entry topic too If it is determined to be a reserved topic, the similarity threshold may be met because the first agent entry topic and the first user entry topic are the same. In response If the first agent entry topic is determined to be different from the first user entry topic, the similarity threshold is not met.

[0061] In an embodiment, the response domain component 230 may include one or more processors configured to determine that a first user entry is not in the first response domain, at least based on the determination that the similarity threshold does not exceed a predetermined value.

[0062] In this embodiment, the system 200 receives user entry data 214 1-N The system may include an agent identification component 240 configured to identify and communicate with an agent having a corresponding response domain that includes the first user entry. For example, in response to determining that a first user entry entered into the messaging dialog interface is not in a first response domain, the agent identification component 240 may be configured to identify a second agent having a second response domain that includes the first user entry.

[0063] In the embodiment, in response to identifying a second agent, the agent identification component 240 establishes a communication between the first agent and the first user in the messaging dialog interface. Ta To facilitate a seamless transition of the communication flow, it may be configured to transmit the first user entry to the second agent. Furthermore, the first User Transmitting entries to a second agent involves the first user and the second agent in the messaging dialog interface. 1 It may accommodate off-topic dialogue flows between the agent and the user.

[0064] In one embodiment, regression data corresponding to the determination that a deviation has occurred may be communicated to an agent server configured to facilitate agent operations within a messaging dialog interface. The regression data is configured to be communicated within the messaging dialog interface so that user entries unrelated to agent entries do not interfere with the progress of an ongoing communication session. 、 This can improve the ease of communication between the agent and the user. Rather, irrelevant user entries can be identified as deviations and responded to appropriately. Response Agents composed of response domains are identified and continue the deviant conversation.

[0065] In the embodiment, one or more processors process a string of communication (e.g., NL text) and generate structured text containing keywords that are extracted and further processed, thereby enabling the user interface 210 inside execution So Messaging dialog to Agent entry data 212 1-N Alternatively, it may include an NLP engine configured to interpret any other user entries.

[0066] Figure 3 shows a messaging dialog management according to an embodiment of the present invention. C In the stem figure This presents Model 300 for improving decision-making.

[0067] In one embodiment, messaging dialog management C In the stem figureModel 300 for improving decision-making may be configured to receive natural language entry data 310 from one or more agents and / or users. Furthermore, Model 300 may include one or more processors configured to encode the agent text data received as part of the natural language entry data 310 into a text embedding 320 having 768 or fewer dimensions. In other words, Model 300 may be configured to convert the natural language entry data 310 into a 768-dimensional BERT text embedding 320, as described herein. Furthermore, Model 300 may be configured to attach part-of-speech (POS) tags 330 to one or more words in the NL text to generate tagged NL text data. Furthermore, Model 300 may, tagged NL Te The model may include a trained model 340 configured to process the quist data and text embeddings 320 to generate output data corresponding to the response expectation classification 350. Furthermore, the model 300 may include one or more processors configured to determine the NL text as first agent entry data that expects a response if the response expectation classification satisfies certain conditions.

[0068] In an embodiment, Model 300 may include a response domain 360 configured to determine whether a user entry is within the response domain of an agent entry. For example, the response domain 360 may include one or more processors configured to determine a first agent data entry topic based on agent text data, where the first response domain corresponds to the first agent data entry topic. Furthermore, the response domain 360 may include one or more processors configured to determine a first user entry topic based on user text data and to compare the first agent data entry topic with the first user entry topic to determine if a similarity threshold is met. If the comparison between the first agent data entry topic and the first user entry topic exceeds the similarity threshold, the user entry is within the response domain of the agent entry. On the other hand, if the comparison between the first agent data entry topic and the first user entry topic does not exceed the similarity threshold, the user entry is not within the response domain of the agent entry. If the user entry is not within the response domain of the agent entry, one or more processors may be configured to determine that a deviation has occurred. If a user entry is within the response domain of an agent entry, one or more processors may be configured to determine that no deviation has occurred.

[0069] In one embodiment, the computer implementation method may further include a step in which the step of determining that a first user entry is not in a first response domain is at least based on the determination that the similarity threshold does not exceed a predetermined value.

[0070] Figure 4 shows the messaging dialog system according to an embodiment of the present invention. figureThe operational stages of a computer implementation method 400 for improving decision-making are shown. It should be understood that Figure 4 provides only an illustration of one implementation and does not imply any limitation to the environment in which different embodiments may be implemented. Many modifications to the environment shown are possible.

[0071] In a messaging dialog system figure A computer implementation method 400 for improving decision-making may include one or more processors configured to receive first agent entry data corresponding to a first agent communicating in a messaging dialog interface 402.

[0072] In an embodiment, the computer implementation method 400 may further include one or more processors configured to identify agent text data corresponding to natural language (NL) text in the first agent entry data.

[0073] In an embodiment, the computer implementation method 400 may further include one or more processors configured to process agent text data with a first trained machine learning model to generate model output data corresponding to response expectation classification.

[0074] Furthermore, one or more processors may be configured to determine the NL text as the first agent entry data that is expected to respond, if the response expectation classification satisfies the conditions.

[0075] Furthermore, the computer implementation method 400 may further include one or more processors configured to identify user text data corresponding to natural language (NL) utterances in a first user entry.

[0076] In an embodiment, the computer implementation method 400 may further include one or more processors configured to determine a first agent data entry topic based on agent text data, wherein the first response domain corresponds to the first agent data entry topic.

[0077] Furthermore, the computer implementation method 400 may further include one or more processors configured to determine a first user entry topic based on user text data.

[0078] Furthermore, the computer implementation method 400 may further include one or more processors configured to compare a first agent data entry topic with a first user entry topic and determine whether a similarity threshold is met.

[0079] The computer implementation method 400 may also be configured to determine 404 that the first agent entry data expects a response that is in the first response domain.

[0080] In an embodiment, the determination that a first agent entry anticipates a response 404 may further include one or more processors configured to attach part-of-speech (POS) tags to one or more words in the NL text to generate tagged NL text data.

[0081] Furthermore, one or more processors may be configured to encode agent text data into text embeddings having 768 or fewer dimensions.

[0082] Furthermore, one or more processors may be configured to process tagged NL text data and sentence embeddings using a second trained machine learning model to generate model output data corresponding to response expectation classification.

[0083] Furthermore, one or more processors may be configured to determine the NL text as the first agent entry data that is expected to respond, if the response expectation classification satisfies the conditions.

[0084] The computer implementation method 400 may also be configured to determine 406 that a first user entry entered into a messaging dialog interface is not in a first response domain.

[0085] In an embodiment, the computer implementation method 400 may include one or more processors configured to determine that a first user entry is not in a first response domain, at least based on the determination that a similarity threshold does not exceed a predetermined value.

[0086] Computer implementation method 400 identifies a second agent comprising a second response domain containing a first user entry 408 Ta It may be structured in this way.

[0087] In one embodiment, the first user entry may include a document entry, and the computer implementation method 400 may further include one or more processors configured to extract document entry data from the document entry.

[0088] Furthermore, one or more processors may be configured to process document entry data and determine natural language text data, and the step of determining that a first user entry is not in a first response domain is based at least on natural language text data.

[0089] In one embodiment, the first User The step of transmitting the entry to the second agent may correspond to a subject-off-topic dialogue flow between the first user and the second agent in the messaging dialog interface.

[0090] The computer implementation method 400 establishes a communication between a first agent and a first user in a messaging dialog interface. Ta To facilitate a seamless transition of the communication flow, the system may be configured to transmit the first user entry to the second agent.

[0091] Figure 5 shows a block diagram of the components of a server computer in the distributed data processing environment of Figure 1 according to an embodiment of the present invention.

[0092] The computing device 500 includes a cache 516, memory 506, persistent storage 508, a communication unit 510, and a communication fabric 502 that provides communication between the input / output (I / O) interface 512. The communication fabric 502 is connected to the processors in the system (e.g., microprocessors, communication and network processors, etc.). and, It can be implemented in any architecture designed to pass data or control information, or both, between system memory and peripheral devices and any other hardware components. For example, the communication fabric 502 can be implemented with one or more buses or crossbar switches.

[0093] Memory 506 and persistent storage 508 are computer-readable storage media. In this embodiment, memory 506 includes random access memory (RAM). Generally, memory 506 can include any volatile or non-volatile computer-readable storage media. Cache 516 stores recently accessed data from memory 506 and a Accessed data Nearby data This is a high-speed memory that enhances the performance of the computer processor 504 by holding data.

[0094] The program may be stored in persistent storage 508 and memory 506 for execution or access, or both, by one or more of the respective computer processors 504 via the cache 516. In some embodiments, persistent storage 508 includes a local hard disk drive. Alternatively, or in addition to the local hard disk drive, persistent storage 508 may include a solid-state hard drive, a semiconductor storage device, read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, or other computer-readable storage medium capable of storing program instructions or digital information.

[0095] The medium used by persistent storage 508 may be removable. For example, a removable hard drive may be used for persistent storage 508. Other examples include optical and magnetic disks, thumb drives, and smart cards inserted into the drive for transfer to another computer-readable storage medium, which is also part of persistent storage 508.

[0096] In these examples, the communication unit 510 provides communication with other data processing systems or devices. In these examples, the communication unit 510 includes one or more network interface cards. The communication unit 510 may provide communication through the use of either or both physical and wireless communication links. The program may be downloaded to persistent storage 508 through the communication unit 510 as described herein.

[0097] The I / O interface 512 enables data input and output with other devices that may be connected to the user device 120. For example, the I / O interface 512 may provide connection to an external device 518 such as an image sensor, keyboard, keypad, touchscreen, or several other suitable input devices or a combination thereof. The external device 518 may also include portable computer-readable storage media such as a thumb drive, portable optical or magnetic disk, and memory card. Software and data 514 used to carry out embodiments of the present invention can be stored on such portable computer-readable storage media and loaded onto persistent storage 508 via the I / O interface 512. The I / O interface 512 also connects to a display 520.

[0098] The display 520 provides a mechanism for displaying data to the user, and may be, for example, a computer monitor.

[0099] The software and data 514 described herein are identified based on the applications implemented in particular embodiments of the present invention. However, it should be understood that any particular programming terms used herein are provided solely for convenience, and therefore the present invention should not be limited to the specific uses identified or suggested by such terms or both.

[0100] The programs described herein are identified based on the applications implemented in specific embodiments of the present invention. However, it should be understood that any particular programming terms used herein are for convenience only, and therefore the present invention should not be limited to any specific use identified or suggested by such terms or both.

[0101] The present invention may be a computer system, a computer implementation method, a computer program product, or a combination thereof. The computer program product may include a computer-readable storage medium (or a plurality of mediums) having computer-readable program instructions therein, in order to cause a processor to execute an aspect of the present invention.

[0102] A computer-readable storage medium can be any tangible device capable of holding and storing instructions for use by an instruction-executing device. A computer-readable storage medium may, for example, be an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash® memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital multipurpose disks (DVDs), memory sticks, floppy disks, mechanically encoded devices such as punch cards or raised structures in grooves on which instructions are recorded, and any suitable combination thereof. Computer-readable storage media should not be interpreted as being transient signals in themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through optical fiber cables), or electrical signals transmitted through wires, as used herein.

[0103] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to each computing / processing device, or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing / processing device receives computer-readable program instructions from the network and transfers the computer-readable program instructions for storage in a computer-readable storage medium within each computing / processing device.

[0104] The computer-readable program instructions for performing the operation of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including, for example, object-oriented programming languages ​​such as Smalltalk® or C++, or conventional procedural programming languages ​​such as the C programming language or similar programming languages. The computer-readable program instructions may run entirely on the user's computer, partially on the user's computer as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), and the connection may be to an external computer (for example, via the Internet using an Internet Service Provider). In some embodiments, for example, an electronic circuit including a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA) may execute computer-readable program instructions by personalizing the electronic circuit using state information of computer-readable program instructions in order to perform aspects of the present invention.

[0105] Aspects of the present invention are described herein with reference to flowcharts or block diagrams, or both, of computer implementation methods, apparatus (systems), and computer program products according to embodiments of the present invention. It will be understood that each block in a flowchart or block diagram, or both, and combinations of blocks in a flowchart and / or block diagram, can be implemented by computer-readable program instructions.

[0106] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a dedicated computer, or another programmable data processing device to generate a machine, such that instructions executed via the processor of a computer or other programmable data processing device create means for implementing functions / operations specified in one or more blocks of a flowchart or block diagram, or both. These computer-readable program instructions, which can instruct a computer, a programmable data processing device, or other device, or a combination thereof, to function in a particular manner, may also be stored on a computer-readable storage medium, and as a result, the computer-readable storage medium having the stored instructions comprises a product containing instructions that implements modes of functions / operations specified in one or more blocks of a flowchart or block diagram, or both.

[0107] Furthermore, computer-readable program instructions may be loaded into a computer, another programmable data processing device, or another device to execute a series of operational steps on the computer, another programmable device, or another device, thereby generating a computer implementation process in which the instructions executed on the computer, another programmable device, or another device implement the functions / operations specified in one or more blocks of a flowchart or block diagram, or both.

[0108] The flowcharts and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions containing one or more executable instructions for implementing a particular logical function. In some alternative implementations, the functions described in a block may be performed in an order different from the order shown in the drawings. For example, depending on the functions involved, two consecutively shown blocks may actually be executed substantially simultaneously, or in some cases, the blocks may be executed in reverse order. It should also be noted that each block in a block diagram or flowchart, or both, and combinations of blocks in a block diagram or flowchart, or both, can be implemented by a dedicated hardware-based system that performs a particular function or operation, or a combination of dedicated hardware and computer instructions.

[0109] The descriptions of various embodiments of the present invention are presented for illustrative purposes only and are not intended to be exhaustive or limit the scope to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the invention. The terminology used herein has been selected to best describe the principles of the embodiments, their practical applications, or the technical improvements to the technology available on the market, or to enable those skilled in the art to understand the embodiments disclosed herein.

Claims

1. A computer implementation method for improving intent determination in a messaging dialog management system, wherein the computer implementation method is The process involves one or more processors receiving first agent entry data corresponding to a first agent communicating in a messaging dialog interface, The steps include: processing the first agent entry data with one or more processors and determining whether the first agent entry data anticipates a response, thereby determining that the first agent entry data anticipates the response within the first response domain of the first agent; The steps include: determining by one or more processors that the first user entry data corresponding to the first agent entry data input to the messaging dialog interface is not within the first response domain of the first agent; The steps include: identifying a second agent having a second response domain containing the first user entry data using one or more of the aforementioned processors; The one or more processors transmit the first user entry data to the second agent in order to facilitate a seamless transition of the established communication flow between the first agent and the first user. A computer implementation method comprising the above.

2. The computer implementation method according to claim 1, wherein the first agent entry data is classified into a first class and a second class, the first class corresponding to expecting a response to the first agent entry data and the second class corresponding to not expecting a response to the first agent entry data.

3. The computer implementation method according to claim 1, wherein processing the first agent entry data includes extracting features from the first agent entry data, calculating a score based on the extracted features indicating the degree to which the first agent entry data belongs to each of at least two categories, determining which of the at least two categories the first agent entry data belongs to based on the score, and determining the classification of the first agent entry data as to whether or not the response is expected based on the determined category.

4. The steps include: identifying agent text data corresponding to natural language (NL) text in the first agent entry data using one or more of the aforementioned processors; The steps include: identifying user text data corresponding to natural language (NL) utterances in the first user entry data using one or more of the aforementioned processors; The computer implementation method according to claim 1, further comprising:

5. The steps include: determining a first agent entry data topic based on the agent text data using one or more processors, wherein the first response domain corresponds to the first agent entry data topic; The steps include determining a first user entry data topic based on the user text data using one or more of the aforementioned processors, The steps include: comparing the first agent entry data topic and the first user entry data topic using one or more processors to determine whether a similarity threshold is met; The computer implementation method according to claim 4, further comprising:

6. The computer implementation method according to claim 5, wherein the step of determining that the first user entry data is not within the first response domain of the first agent is at least based on determining that the similarity threshold does not exceed a predetermined value.

7. The first step involves processing the agent text data with a first trained machine learning model to generate model output data corresponding to the response expectation classification. The step of determining the NL text as the first agent entry data that anticipates the response, if the response expectation classification satisfies the conditions, by one or more of the aforementioned processors. A computer implementation method according to any one of claims 4 to 6, further comprising:

8. The first user entry data is a document entry, and the computer implementation method is The steps include: extracting document entry data from the document entry using one or more of the aforementioned processors; The step of processing the document entry data with one or more processors to determine natural language text data, wherein the step of determining that the first user entry data is not in the first response domain is based at least on the natural language text data. A computer implementation method according to any one of claims 1 to 4, further comprising the above.

9. The computer implementation method according to any one of claims 1 to 6, wherein the step of transmitting the first user entry data to the second agent corresponds to a subject-official dialogue flow between the first user and the first agent in the messaging dialog interface.

10. The step in which the first agent entry data determines that the response is expected is: The steps include: using one or more processors to attach part-of-speech (POS) tags to one or more words in the NL text to generate tagged NL text data; The steps include encoding the agent text data into a text embedding having 768 or fewer dimensions using one or more of the aforementioned processors, The process involves a second trained machine learning model processing the tagged NL text data and the text embeddings to generate model output data corresponding to the response expectation classification. The step of determining the NL text as the first agent entry data that anticipates the response, if the response expectation classification satisfies the conditions, by one or more of the aforementioned processors. A computer implementation method according to any one of claims 4 to 6, further comprising the above.

11. A computer program for improving intent determination in a messaging dialog management system, wherein the processor A procedure for receiving first agent entry data corresponding to a first agent communicating in a messaging dialog interface, A procedure for determining whether the first agent entry data anticipates a response by processing the first agent entry data and determining whether the first agent entry data anticipates a response within the first response domain of the first agent, A procedure for determining that the first user entry data for the first agent entry data received in the messaging dialog interface is not within the first response domain of the first agent, A procedure for identifying a second agent having a second response domain containing the first user entry data, To facilitate a seamless transition of the established communication flow between the first agent and the first user, a procedure for transmitting the first user entry data to the second agent is provided. A computer program designed to execute something.

12. The processor, A procedure for identifying agent text data corresponding to natural language (NL) text in the first agent entry data, A procedure for identifying user text data corresponding to natural language (NL) utterances in the first user entry data, and The computer program according to claim 11, which further executes the following.

13. The aforementioned processor, A procedure for determining a first agent entry data topic based on the agent text data, wherein the first response domain corresponds to the first agent entry data topic, and the procedure is as follows: A procedure for determining a first user entry data topic based on the user text data, A procedure for comparing the first agent entry data topic and the first user entry data topic to determine whether the similarity threshold is met. The computer program according to claim 12, which further executes the following.

14. The computer program according to claim 13, wherein the step of determining that the first user entry data is not in the first response domain of the first agent is at least based on determining that the similarity threshold does not exceed a predetermined value.

15. The aforementioned processor, A procedure for processing the agent text data using a first trained machine learning model to generate model output data corresponding to the response expectation classification, If the response expectation classification satisfies the conditions, the procedure involves determining the NL text as the first agent entry data that expects the response. A computer program according to any one of claims 12 to 14, which further performs the following:

16. The first user entry data is a document entry, and the computer program sends the processor, The procedure for extracting document entry data from the aforementioned document entry, A procedure for processing document entry data to determine natural language text data, wherein the procedure determines that the first user entry data is not in the first response domain, is based at least on the natural language text data. A computer program according to any one of claims 11 to 14, which further performs the following:

17. The computer program according to any one of claims 11 to 14, wherein the procedure for transmitting the first user entry data to the second agent corresponds to an off-topic dialogue flow between the first user and the first agent in the messaging dialog interface.

18. The procedure for determining that the first agent entry data anticipates the response is: A procedure for generating tagged NL text data by attaching part-of-speech (POS) tags to one or more words in the NL text, A procedure for encoding the agent text data into a text embedding having 768 or fewer dimensions, A second trained machine learning model processes the tagged NL text data and the text embeddings to generate model output data corresponding to the response expectation classification. If the response expectation classification satisfies the conditions, the procedure involves determining the NL text as the first agent entry data that expects the response. A computer program according to any one of claims 12 to 14, further comprising:

19. A computer system for improving intent determination in a messaging dialog management system, wherein the computer system is One or more computer processors, One or more computer-readable storage media, For execution by at least one of the one or more computer processors, program instructions stored together on the one or more computer-readable storage media and The program instructions stored therein are A program instruction for receiving first agent entry data corresponding to a first agent communicating in a messaging dialog interface, A program instruction that processes the first agent entry data and determines whether the first agent entry data expects a response, thereby determining that the first agent entry data expects the response within the first response domain of the first agent, A program instruction that determines, in the messaging dialog interface, that the first user entry data corresponding to the first agent entry data is not within the first response domain of the first agent, A program instruction that identifies a second agent having a second response domain containing the first user entry data, In order to facilitate a seamless transition of the established communication flow between the first agent and the first user, a program instruction is provided to transmit the first user entry data to the second agent. A computer system having the following features.

20. A program instruction that identifies agent text data corresponding to natural language (NL) text in the first agent entry data, A program instruction that identifies user text data corresponding to natural language (NL) utterances in the first user entry data, and The computer system according to claim 19, further comprising:

21. A program instruction for determining a first agent entry data topic based on the agent text data, wherein the first response domain comprises a program instruction corresponding to the first agent entry data topic. A program instruction that determines a first user entry data topic based on the user text data, A program instruction to compare the first agent entry data topic and the first user entry data topic and determine whether the similarity threshold is met. The computer system according to claim 20, further comprising:

22. The computer system according to claim 21, wherein the program instruction for determining that the first user entry data is not within the first response domain of the first agent is based at least on a program instruction for determining that the similarity threshold does not exceed a predetermined value.

23. A program instruction that processes the agent text data using a first trained machine learning model to generate model output data corresponding to the response expectation classification, If the response expectation classification satisfies the conditions, a program instruction determines the NL text as the first agent entry data that expects the response. A computer system according to any one of claims 20 to 22, further comprising:

24. The first user entry data is a document entry, and the computer system, A program instruction for extracting document entry data from the aforementioned document entry, A program instruction that processes the document entry data and determines natural language text data, wherein the program instruction determines that the first user entry data is not in the first response domain, is based at least on the natural language text data, and A computer system according to any one of claims 19 to 22, further comprising the above.

25. The computer system according to any one of claims 19 to 22, wherein the program instruction for transmitting the first user entry data to the second agent corresponds to a subject-official dialogue flow between the first user and the first agent in the messaging dialog interface.

26. The program instruction that determines that the first agent entry data anticipates the response is: A program instruction that attaches part-of-speech (POS) tags to one or more words in the NL text to generate tagged NL text data, A program instruction for encoding the agent text data into a text embedding having 768 or fewer dimensions, A second trained machine learning model processes the tagged NL text data and the text embeddings to generate model output data corresponding to the response expectation classification, and a program instruction for this process. If the response expectation classification satisfies the conditions, a program instruction determines the NL text as the first agent entry data that expects the response. A computer system according to any one of claims 20 to 22, further comprising:

Images