Dynamic deployment of small language models

Abstract

Methods and systems are presented for reducing computer power consumption and speeding up system response times by dynamically generating and deploying one or more small language models (SLMs) to facilitate automated interactions with a user. A context is derived for a chat session based on a user-submitted utterance and other contextual information associated with the chat session. An SLM is generated specifically for the chat session based on the context. The SLM can be generated by extracting one or more portions of an internal structure of a large language model (LLM), or by merging two or more pre-generated SLMs. The SLM is deployed to generate content for the chat session. When it is detected that the context has changed, the SLM can be updated by incorporating additional parameters from the LLM to continue facilitating automated interactions with the user during the chat session.

Description

Technical Field

[0001] This application generally relates to machine learning models, and more specifically, to providing a computer framework for dynamically deploying small language models according to various embodiments of this disclosure. Background Technology

[0002] Large Language Models (LLMs) have been organized to facilitate conversational-based automated interactions with users. Typical LLMs (such as GPT-4, BERT, LLaMA, etc.) are powerful and flexible because they are capable of learning and generating content (e.g., responses to user queries) in natural language formats across a wide range of topics (also known as “domains”). However, the internal structure of typical LLMs is highly complex. For example, LLMs often contain more than 500 billion parameters, requiring the incorporation of highly complex computer software architectures. Due to their highly complex internal structure, LLMs typically consume significant amounts of computing power, and the generation, training, deployment, and / or utilization of LLMs require substantial time, which can significantly hinder the performance of systems that leverage LLMs, such as chat systems that provide automated interactions with users. Therefore, the applicant recognizes the need for a more computationally and power-efficient solution for facilitating conversational-based automated interactions with users. Attached Figure Description

[0003] Figure 1 This is a block diagram illustrating an electronic trading system according to an embodiment of the present disclosure; Figure 2 This is a block diagram illustrating a chat module according to an embodiment of the present disclosure; Figure 3 Various small language models generated based on large language models according to embodiments of the present disclosure are shown; Figure 4 An exemplary dialogue in an online chat session according to an embodiment of this disclosure is shown; Figure 5 Another exemplary dialogue in an online chat session according to an embodiment of this disclosure is shown; Figure 6 This is a flowchart illustrating a process for dynamically generating and modifying a small language model for a chat session according to embodiments of the present disclosure; and Figure 7 This is a block diagram of a system for implementing a device according to embodiments of the present disclosure.

[0004] The embodiments and advantages of this disclosure will be best understood by referring to the following detailed description. It should be understood that the same reference numerals are used to identify the same elements shown in one or more of the figures, wherein the illustrations in the figures are for illustrative purposes and not for limiting the embodiments of this disclosure. DETAILED DESCRIPTION

[0005] The present disclosure describes a computer framework for dynamically generating, deploying, and / or utilizing small language models (SLMs) to facilitate automated interactions with users, in accordance with various embodiments. As described above, large language models (LLMs) can be used to provide automated interactions with users. For example, an organization can deploy an LLM to connect to or integrate within a chat system for conducting conversations. Such an LLM can be configured and trained to provide responses to user queries across multiple domains related to the organization. Thus, a user can be able to submit a query associated with any of these domains via a chat interface provided by the organization, and the LLM can be utilized by the chat system to generate an automated response for the user.

[0006] To enable the LLM to learn topics and provide intelligent responses to users across different domains, the internal structure of the LLM can be correspondingly large and complex. For example, a typical LLM can include over 500 billion parameters that can be used to learn, digest, and generate content associated with different domains. Each parameter associated with the LLM can be responsible for performing a particular task based on analyzing one or more input values. Some parameters can be associated with tasks that are available across multiple domains, while some parameters can be associated with tasks that are specific to a respective domain. For example, one parameter can be associated with classifying an intent of a customer based on a user-provided utterance (the parameter is available across different domains). Another parameter can be associated with generating instructions for disputing a transaction (the instructions are only available for the “dispute” domain).

[0007] The internal structure of a model (or simply “structure”) includes computer software data structures (e.g., data variables, objects, etc.) and software logic that performs various computer processes associated with the data structures. For example, when a model is implemented as an artificial neural network, the internal structure can include computer nodes in layers within the artificial neural network, connections between the nodes, and computer logic that processes data within each node.

[0008] Due to the complexity of the internal structure of an LLM, generating, deploying, and / or utilizing an LLM typically requires a large amount of computer processing power and time. For example, an LLM can take several seconds to generate a response to a user query. Such a long response time can be deemed unacceptable by many users and can thus cause users to leave the chat system. As a result, users can terminate a chat session with the chat system due to the long response time and can instead directly contact a human agent of the organization, which is an undesirable outcome for many organizations. Moreover, the longer response time requires the LLM to use more processing power and can delay or increase processing of other operations.

[0009] Accordingly, in accordance with various embodiments of the present disclosure, a computer framework is provided for dynamically generating, deploying, and utilizing different SLMs to facilitate automated interactions with users to improve the performance of a chat system. An SLM is a lightweight artificial intelligence model. Like an LLM, an SLM is also trained on a large amount of data to facilitate useful automated conversations with users. However, an SLM is typically much simpler than an LLM (e.g., has fewer parameters than an LLM). For example, while an LLM typically has over 500 billion parameters, a typical SLM can have approximately 1 million parameters. This reduction in scale makes SLMs more flexible and efficient than LLMs. For example, it takes much less computer processing power and time to generate, train, deploy, and / or utilize an SLM than an LLM. While an LLM can take seconds to generate a response to a user query, an SLM can take only milliseconds to generate a response. The time and / or processing cycles to generate an LLM can also be several times the time required to generate and / or deploy an SLM.

[0010] In some embodiments, instead of or in addition to using an LLM, a chat system can utilize one or more SLMs to facilitate automated interactions with users. The chat system can still utilize an LLM to perform certain functions, but can deploy one or more SLMs (rather than a single LLM) to interact with users. Due to the reduced complexity of the internal structure of an SLM (e.g., reduced number of parameters), an SLM can not be as powerful and flexible as an LLM. For example, an SLM can not be able to generate content (e.g., responses to user queries) associated with all domains related to an organization. Accordingly, in some embodiments, a chat system can generate, train, deploy, and / or utilize different SLMs to facilitate conversations with different users (or different contexts associated with a user).

[0011] In some embodiments, when the chat system receives a first utterance from a user via a chat interface during a chat session between the user and an organization, the chat system can determine a context of the chat session. The chat system can then access or otherwise generate an SLM based on the context to facilitate an automated interaction with the user. In some embodiments, the chat system can determine the context of the chat session based on different information associated with the chat session. For example, the chat system can analyze words in the first utterance submitted by the user. The first utterance can include a query submitted by the user, such as “How do I dispute my last transaction.” The chat system can analyze the words in the first utterance and can determine a particular domain associated with the first utterance based on the words. In some embodiments, the chat system can identify a keyword (e.g., the word “dispute”) within the first utterance and match the keyword to a particular domain (e.g., match the keyword “dispute” to a “dispute” domain). In some embodiments, the chat system can use a machine learning model (e.g., an LLM, etc.) to predict an intent (or domain) associated with the first utterance based on the words in the first utterance. In some embodiments, the chat system can further predict the intent based on a history of the user (e.g., previous interactions with the chat system, previous purchases or returns, etc.).

[0012] In some embodiments, the chat system can also generate an account context for the user account based on analyzing the account information of the account. For example, the chat system can determine a status of the account (e.g., the account is active, inactive, suspended, locked, etc.). The chat system can also determine statistics associated with transactions made through the account, such as a frequency of transactions, an average amount of transactions, transaction trends, merchant and / or item categories purchased, etc. The chat system can determine the context of the chat session based on the domain and the account context of the user.

[0013] In some embodiments, the context determined by the chat system can also include a passive context. The passive context is not derived from the substantive content of the first utterance or the account of the user, but rather from peripheral factors associated with the submission of the first utterance, such as a tone used in the first utterance, a location of the user when the first utterance was submitted, other people that can have been proximate to the user (e.g., within a threshold distance of the user) when the user submitted the first utterance, etc.

[0014] In some embodiments, the chat system can determine a configuration of an SLM for facilitating automated interactions with the user during the chat session based on the context. The configuration can specify a number of parameters to include in the SLM and the types of parameters (what the parameters are configured and trained to do). As described herein, the chat system can be associated with an LLM that is configured and trained to perform automated interactions with users across all domains related to the organization. Accordingly, the parameters of the LLM can include different subsets of parameters related to different domains of the organization, different subsets of parameters that can be used to interact with users having different account contexts (e.g., different account statuses, different transaction histories, etc.), different subsets of parameters that can be used to interact with users having different passive contexts, etc. In some embodiments, the chat system can select a subset of parameters for the SLM from the parameters of the LLM based on the context derived for the chat session.

[0015] The chat system can then determine whether one or more existing SLMs have been generated and trained for the context derived for the chat session. In some embodiments, the chat system can have generated a set of SLMs for different contexts prior to receiving the first utterance from the user. For example, the chat system can generate different SLMs for different domains related to the organization, different SLMs for different account contexts, different SLMs for different passive contexts, etc. To generate an SLM for a particular context (e.g., a particular domain, a particular account context, a particular passive context, etc.), the chat system can determine a subset of parameters in the LLM that are associated with the particular context, which can include parameters associated with tasks available across multiple different contexts (including the particular context) and parameters associated with tasks available only for the particular context (e.g., parameters configured and trained for a particular domain, parameters configured and trained to interact with users having a particular account context, parameters configured and trained to interact with users having a particular passive context, etc.). The chat system can then access one or more portions of the computer structure of the LLM associated with the subset of parameters and generate the SLM by copying one or more portions of the data structure of the LLM (e.g., regenerating the portions of the data structure of the LLM for the SLM). Since each SLM includes an exact structure (including data structures, computer logic, weights, etc.) corresponding to one or more portions of the structure of the LLM, the SLM can inherit the “knowledge” acquired by the LLM for the particular context (e.g., a particular domain, a particular account context, a particular passive context, etc.) based on the configuration and training it has undergone.

[0016] In some embodiments, the chat system can also retrain the SLM using training data that is different from the training data used to train the LLM and specific to the particular context associated with the SLM to further improve the performance of the SLM. Such retraining can modify one or more parameters incorporated into the SLM to further improve the performance of the SLM. For example, when a parameter of the LLM used to determine a customer intent based on an utterance (e.g., classifying the utterance into one of a plurality of domains, such as a “dispute” domain, an “account management” domain, a “reward” domain, etc.) is incorporated into an SLM generated for the “dispute” domain, the parameter can be modified by retraining the SLM using training data specific to the “dispute” domain to determine a dispute reason (e.g., billing error, product damage, delayed delivery, etc.) based on one or more utterances provided by the customer. In some embodiments, the retraining can also cause the parameter to include predicted attributes. For example, in addition to training the parameter to determine a dispute reason, the parameter can be further trained to predict one or more additional domains (e.g., a “refund” domain, etc.) associated with a subsequent utterance provided by the customer, a predicted resolution time, or other attributes associated with the chat session. In other words, the retraining builds on the knowledge base inherited by the parameter from the LLM and further customizes the parameter to the specific needs of the SLM. As described herein, the SLM can undergo additional training, such as using subsequent utterances provided by the customer as feedback, until the SLM matures (e.g., when the accuracy performance of the SLM has reached a threshold, etc.).

[0017] As the contexts derived for different chat sessions can be specific to those chat sessions (based on a combination of domains, account contexts, passive contexts, etc.) and the number of possible contexts for different chat sessions can be large (e.g., different permutations of different variables can exceed a threshold), it can be inefficient for the chat system to generate an SLM for every possible context that can be associated with a chat session. Accordingly, the chat system can generate SLMs that serve as building blocks for various contexts (also referred to as “building block SLMs”). For example, the chat system can generate a set of building block SLMs corresponding to different domains of the organization, where each building block SLM can correspond to one or more different domains relevant to the organization. The chat system can also generate another set of building block SLMs corresponding to different account contexts (e.g., different account statuses, different transaction histories, etc.). The chat system can also generate another set of building block SLMs corresponding to different passive contexts.

[0018] Accordingly, after determining the configuration of the SLM for the chat session, the chat system can access a plurality of building block SLMs related to the configuration (e.g., SLMs that include a portion of the parameters specified in the configuration, etc.). For example, the chat system can access a first building block SLM corresponding to the particular domain determined for the chat session. The chat system can also access a second building block SLM corresponding to the account context determined for the user (e.g., account status, transaction history, etc.). The chat system can also access a third building block SLM corresponding to the passive context derived for the user.

[0019] The chat system can then generate the SLM for the chat session by merging the plurality of existing building block SLMs (e.g., the first building block SLM, the second building block SLM, and the third building block SLM). For example, the chat system can combine the structures associated with the different parameters included in the building block SLMs to generate the merged SLM. As a result, the merged SLM includes parameters (and corresponding structures) related to the particular domain determined to be relevant to the chat session, the account context determined for the user, and the passive context derived for the user. In other words, the merged SLM is a custom model specifically generated for the chat session. The chat system can deploy the merged SLM to facilitate automated interactions with the user during the chat session.

[0020] In cases where the chat system has not generated the base building block SLM when the chat system receives the first utterance from the user, the chat system can dynamically generate the SLM for the chat session by accessing one or more portions of the structure of the LLM and the subset of parameters related to the context of the chat session using the techniques described herein. Accordingly, the chat system can access the respective portions of the structure of the LLM and the parameters associated with the particular domain determined for the chat session, the account context associated with the chat session, and the passive context associated with the chat session, rather than merging different existing building block SLMs. The chat system can then generate the SLM by copying the portions of the structure of the LLM and the parameters corresponding to the context of the chat session.

[0021] It is contemplated that a user can engage in multiple topics with the chat system during the same chat session. For example, the user can first start the chat session by inquiring about a recent transaction, then request to dispute the transaction, and then request to add a funding source to the account. Accordingly, the SLM generated based on the initial context (e.g., the context determined based on the initial utterance related to inquiring about the transaction) can not be sufficient to facilitate interactions with the user during the entire chat session. In this regard, the chat system can apply different measures to ensure that a sufficient SLM is deployed to facilitate automated interactions with the user during the chat session, where the SLM is capable of providing relevant and intelligent content to the user during the chat session.

[0022] In some embodiments, the chat system can determine the configuration of the SLM to deploy within a chat session not only based on the initial utterance provided by the user, but also based on predicted utterances that the user can provide during the same chat session. For example, after determining the context of the chat session, the chat system can enrich the context using a prediction model. The prediction model, which can be a machine learning model, can be used by the chat system to predict subsequent utterances (e.g., a second utterance, a third utterance, etc.) that the user will submit within the chat session based on the first utterance, the user’s past history or account information, and / or the context. The chat system can provide the first utterance, the past history or account information, and the context as input values to the prediction model and obtain an output indicating one or more utterances that the user is predicted to submit to the chat system. In one example, if the user has a history of disputing prior transactions, upon receiving an initial utterance asking about a recent transaction that was conducted, the prediction model can predict that the user can submit a subsequent utterance related to disputing the recent transaction. Accordingly, the chat system can update the context to include not only a “transaction history” domain, but also a “dispute” domain for generating the SLM. The chat system can then generate a merged SLM model by combining an existing building block SLM corresponding to the “transaction history” domain, an existing building block SLM corresponding to the “dispute” domain, and possibly other building block SLMs based on the enriched context.

[0023] Instead of, or in addition to, generating the SLM for a different chat session using the enriched context, the chat system can continuously monitor and modify the SLM during the same chat session to ensure that the SLM is able to facilitate automated interactions with the user. In some embodiments, after generating the SLM and deploying the SLM to facilitate automated interactions with the user during the chat session, the chat system can continue to monitor the interactions between the user and the deployed SLM. The chat system can detect whether the context of the chat session has changed based on these interactions. For example, when the user requests to add a funding source to an account during the chat session, the chat system can detect that the domain associated with this utterance is different from the domain associated with the context that was determined for the chat session (e.g., the “dispute” domain). Based on the detected change in context, the chat system can modify the SLM that has been deployed for the chat session. For example, the chat system can access (or otherwise generate) an SLM corresponding to the new context (e.g., a “funding source modification” domain) and can incorporate the structure of this SLM into the deployed SLM.

[0024] In another example, if the chat system determines that the domain associated with the context determined for the chat session is no longer applicable (e.g., the dispute has been resolved, etc.), the chat system can modify the deployed SLM by removing the parameters and / or portions of the structure of the SLM that correspond to that domain. In other words, the chat system can dynamically add and / or remove parameters and structure corresponding to new contexts to the SLM while the SLM is deployed to facilitate automated interactions with the user during the chat session. The chat system can then use the modified SLM to continue facilitating automated interactions with the user during the chat session.

[0025] The chat system can continue to monitor interactions between the SLM and the user and modify the SLM as needed according to updated contexts of the chat session. By dynamically deploying and modifying various SLMs during a chat session, the chat system can use these SLMs to facilitate interactions with the user in an efficient manner without incurring the computational costs of generating, deploying, and utilizing LLMs.

[0026] In some embodiments, after the chat session terminates, the chat system can use interactions between the SLM and the user during the chat session to retrain the deployed SLM and / or the underlying building block SLMs used to generate the deployed SLM. Continuously retraining various SLMs using actual interactions with the user can further improve the performance of the SLMs, which in turn will improve the performance of SLMs generated for subsequent chat sessions.

[0027] In some embodiments, the chat system can determine whether to store the SLM that has been deployed for the chat session for future use. For example, the chat system can assign an expiration time to the SLM. If the user initiates another chat session with the chat system before the expiration time of the SLM, the chat system can deploy the same SLM to facilitate automated interactions with the user during the new chat session. Since the SLM is generated based on the context associated with the previous session, the SLM has knowledge of the user’s interaction history, which can be useful in facilitating interactions with the user in the new chat session.

[0028] Figure 1An electronic transaction system 100 within which a chat system can be implemented according to one embodiment of the present disclosure is shown. The electronic transaction system 100 includes a service provider server 130 associated with a service provider and user devices 110, 170, 180, and 190 that can be communicatively coupled to one another via a network 160. In one embodiment, the network 160 can be implemented as a single network or a combination of multiple networks. For example, in various embodiments, the network 160 can include the Internet and / or one or more intranets, fixed telephone networks, wireless networks, and / or other appropriate types of communication networks. In another example, the network 160 can include a wireless telecommunication network (e.g., a cellular telephone network) suitable for communicating with other communication networks, such as the Internet.

[0029] In one embodiment, the user device 110 can be utilized by a user 140 to interact with the service provider server 130 and / or other user devices similar to the user device 110 (e.g., user devices 170, 180, and 190, etc.) via the network 160. For example, the user 140 can use the user device 110 to log into a user account of the service provider to access account services or conduct electronic transactions with the service provider server 130 (e.g., account transfers or payments, purchase goods and / or services, sell goods and / or services, receive payment for sales, access or receive content or data, etc.). Further, the user 140 represented herein can be a natural person, a group of people, a community, and / or a business entity. Examples of business entities include merchant sites, resource information sites, utility sites, real estate management sites, social networking sites, etc. that provide various items for purchase and process payment for purchases.

[0030] In various embodiments, the user device 110 can be implemented using any appropriate combination of hardware and / or software configured for wired and / or wireless communication via the network 160. In various implementations, the user device 110 can include at least one of a wireless cellular telephone, a wearable computing device, a PC, a laptop computer, etc.

[0031] In one embodiment, the user device 110 includes a user interface (UI) application 112 (e.g., a web browser) that the user 140 can utilize to conduct electronic transactions (e.g., access data, sales, shopping, purchases, bidding, etc.) with the service provider server 130 via the network 160. In one implementation, the user interface application 112 includes a software program, such as a graphical user interface (GUI), that is executable by a processor configured to interface and communicate with the service provider server 130 via the network 160. In another implementation, the user interface application 112 includes a browser module that provides a web interface for browsing information available on the network 160. For example, the user interface application 112 can be implemented in part as a web browser for viewing information available on the network 160.

[0032] The user device 110 can also include a chat client 170 for facilitating an online chat session with another chat client (e.g., a chat client of another device, a chat system of a service provider, etc.). The chat client 170 can be a software application executing on the user device 110 for providing a chat client interface to the user 140 and for exchanging (e.g., sending and receiving) messages with other chat clients of a chat system. For example, during an online chat session with another entity (e.g., a chat system of a service provider), the chat client 170 can present a chat interface that enables the user 140 to input data (e.g., textual data such as utterances, audio data, multimedia data, etc.) to send to the other entity (via the other chat client or chat system, etc.). The chat interface can also present messages received from the other entity via the other chat client or chat system. In some embodiments, the messages can be presented in chronological order on the chat client interface according to a chat stream of the online chat session. The chat client 170 can be an embedded application embedded within another application (e.g., the UI application 112). Alternatively, the chat client 170 can be a standalone chat client program (e.g., a mobile application such as WhatsApp®, Facebook® Messenger, iMessages®, etc.) that is separate from any other software application executing on the user device 110.

[0033] In various embodiments, user device 110 can include other applications 116 that can be desirable in one or more embodiments of the present disclosure to provide additional features available to user 140. For example, applications 116 can include security applications for implementing client-side security features, programmatic client applications for interfacing with appropriate application programming interfaces (APIs) via network 160, and / or various other types of well-known programs and / or software applications. In still other examples, other applications 116 can interface with user interface application 112 to improve efficiency and convenience.

[0034] In one embodiment, user device 110 can include at least one identifier 114, which can be implemented, for example, as an operating system registry entry, a cookie associated with user interface application 112, an identifier associated with hardware of user device 110 (e.g., a media access control (MAC) address), or various other appropriate identifiers. Identifier 114 can include one or more attributes related to user 140 of user device 110, such as personal information related to the user (e.g., one or more user names, passwords, photographic images, biometric IDs, addresses, phone numbers, social security numbers, etc.) and banking information and / or sources of funds (e.g., one or more banking institutions, credit card issuers, user account numbers, security data and information, etc.). In various implementations, identifier 114 can be embedded in messages sent to other chat clients during an online chat session, and identifier 114 can be used by service provider server 130 to associate the user with a particular user account maintained by service provider server 130.

[0035] In various implementations, user 140 can be able to enter data and information into an input component (e.g., a keyboard) of user device 110 to provide user information with a transaction request, such as a login request, a funds transfer request, a request to add an additional source of funds (e.g., a new credit card), a request for data or content, or other types of requests. The user information can include user identification information.

[0036] Each of user devices 170, 180, and 190 can have similar hardware and software components as user device 110. For example, each of user devices 170, 180, and 190 can include a respective chat client. As such, users of user devices 170, 180, and 190 can be able to engage in online chat sessions with other chat clients (or chat systems) using the respective chat clients.

[0037] In one embodiment, the service provider server 130 can be maintained by an online service provider that can provide services (e.g., perform electronic transactions such as electronic payment transactions, data access transactions, data processing transactions, etc.) for its users (e.g., the user 140 and users of the user devices 170, 180, and 190, etc.). Thus, the service provider server 130 can include a service application 138 that can be adapted to interact with user devices (e.g., the user devices 110, 170, 180, and 190, etc.) via the network 160 to facilitate searching, selection, purchase, payment, and / or other services of items provided by the service provider server 130. In one example, the service provider server 130 can be provided by the PayPal® company located in San Jose, California, USA and / or one or more service entities or respective intermediaries that can provide a plurality of point-of-sale devices in various locations to facilitate transaction routing between merchants and the service entity, for example.

[0038] In some embodiments, the service application 138 can include a payment processing application (not shown) for processing purchases and / or payments of electronic transactions between users and merchants or between any two entities. In one implementation, the payment processing application helps resolve electronic transactions by verifying, delivering, and settling. Thus, the payment processing application settles debts between users and merchants, where accounts can be directly and / or automatically debited and / or credited with monetary funds in a manner accepted by the banking industry.

[0039] The service provider server 130 can also include a web server 134 configured to provide web content to users in response to HTTP requests. Thus, the web server 134 can include pre-generated web content ready to be provided to users. For example, the web server 134 can store a login page and be configured to provide the login page to a user to log into a user account of the user associated with the service provider server 130 to access various services, data, or content provided by the service provider server 130. The web server 134 can also include other web pages associated with different services provided by the service provider server 130. As a result, a user (e.g., the user 140) can access a user account associated with the user and access various services provided by the service provider server 130 by generating an HTTP request to the service provider server 130.

[0040] In one embodiment, the service provider server 130 can be configured to maintain one or more user accounts (e.g., buyer accounts, seller accounts, etc.) in an account database 136, each of which can include account information associated with one or more users (e.g., the user 140 associated with the user device 110). For example, the account information can include private financial information of the user and the merchant, such as one or more account numbers, passwords, credit card information, bank information, digital wallets used, transaction history, or other types of financial information. In certain embodiments, the account information also includes user purchase profile information, such as account funding options and payment options associated with the user, payment information, receipts, and other information collected in response to completed funding and / or payment transactions.

[0041] In one implementation, a user can have an identity attribute stored at the service provider server 130, and the user can have credentials for authenticating or verifying the identity with the service provider server 130. The user attribute can include personal information, bank information, and / or a source of funds. In various aspects, the user attribute can be passed to the service provider server 130 as part of a login, search, selection, purchase, and / or payment request, and the user attribute can be utilized by the service provider server 130 to associate the user with one or more particular user accounts maintained by the service provider server 130.

[0042] The service provider server 130 can also include a chat module 132 that implements the functionality of the chat system as disclosed herein. In some embodiments, the chat module 132 can be configured to provide automated interactions with a user via a chat client of a user device. For example, the user 140 can initiate a chat request to the chat module 132 via the chat client 170. The chat module 132 can establish a chat session with the chat client 170 based on the request. In some embodiments, the chat module 132 can assign a session identifier for the chat session and establish a communication channel for the chat session with the chat client 170 such that any messages sent between the chat client 170 and the chat module 132 via the communication channel are incorporated into the chat session.

[0043] In establishing the chat session between the chat module 132 and the chat client 170, the chat module 132 can monitor chat utterances provided by the user 140. The chat module 132 can determine a context of the chat session based on analyzing the chat utterances and other information associated with the user 140. The chat module 132 can then generate a customized SLM for facilitating automated interactions with the user during the chat session.

[0044] Figure 2A block diagram of the chat module 132 is shown in accordance with embodiments of the present disclosure. The online chat module 132 includes a chat manager 202, a prediction module 204, a context module 206, an SLM generation module 208, and a training module 210. The chat manager 202 can detect that a chat client (e.g., the chat client 170) has initiated a chat request with the chat module 132 and can establish a chat session 220 between the chat module 132 and the chat client 170. After establishing the chat session 220, the chat manager 202 can begin monitoring utterances exchanged between the user 140 and the chat module 132 during the chat session 220. For example, the chat manager 202 can detect utterances 222 submitted by the user 140 via the chat client 170 during the chat session 220. The utterances 222 can indicate an intent of the user 140 for the chat session 220. For example, the utterances 222 can be a request from the user 140, such as a query for a past transaction (e.g., “I want to know the status of the payment I made last week”), a dispute request (e.g., “I want to dispute the payment I made two weeks ago”), a request to add a new funding source to an account (e.g., “Can I link this bank account to my user account”), or any other type of request for a service or content provided by the service provider.

[0045] In some embodiments, the chat manager 202 can determine a context of the chat session 220 using the context module 206. The context module 206 can determine the context of the chat session 220 based on different information associated with the chat session 220. For example, the context module 206 can analyze words in the utterances 222 submitted by the user 140. As described herein, the utterances 222 can indicate an intent of the user 140. Thus, by analyzing the words in the utterances 222, the context module 206 can determine a particular domain associated with the chat session 220 from different domains related to the service provider. For example, the service provider associated with the service provider server 130 can be related to a plurality of different domains (or topics). Each domain can be associated with a different type of service provided by the service provider. Example domains of the service provider can include a “payment freeze” domain for resolving a payment freeze situation of a user, a “rewards” domain related to information and services associated with one or more reward programs offered by the service provider, a “password reset” domain associated with resolving a credential issue of a user, a “transaction dispute” domain associated with disputing any transaction made with the service provider, a “funding source modification” domain associated with modifying one or more funding sources linked to an account of a user, a “transaction query” domain for querying information associated with any transaction made with the service provider, a “security” domain associated with various security issues related to an account of a user of the service provider, and other domains related to services provided by the service provider.

[0046] utterance, the chat system can identify keywords within the first utterance (e.g., the words “dispute,” “credit card,” “detailed information,” etc.) and match the keywords to a particular domain (e.g., match the keyword “dispute” to the “dispute” domain, match the keyword “credit card” to the “funding source modification” domain, match the keyword “detailed information” to the “transaction query” domain, etc.). In some embodiments, the context module 206 can use a machine learning model to predict an intent (or domain) associated with the utterance 222 based on the words in the utterance 222.

[0047] In some embodiments, the context module 206 can also analyze account information for the account of the user 140 to determine an account context for the chat session 220. For example, the context module 206 can access account information for the account of the user 140 stored in the account database 136. The context module 206 can determine a status of the account (e.g., whether the account is active, inactive, suspended, locked, etc.). The context module 206 can also determine statistics associated with transactions made through the account, such as a frequency of transactions, an average amount of transactions, transaction trends, merchants and / or categories of items purchased, etc. The chat system can determine a context for the chat session 220 based on the domain and the account context for the user 140. The context or intent can also be determined based on past interactions with the chat system and / or entities associated with the chat system and any other data available to the chat system (e.g., user interactions or information from chat boards, social networks, etc.).

[0048] In some embodiments, the context determined by the context module 206 can also include passive context. Passive context is not derived from the substantive content of the utterance 222 or the account of the user 140, but rather from peripheral factors associated with the submission of the utterance 222. For example, the context module 206 can analyze the words in the utterance 222 to determine the tone associated with the utterance 222. The context module 206 can also determine the location of the user device 110 at the time the utterance 222 was submitted (based on communication with a location component of the user device 110, such as a GPS component, etc.), as well as other people that can have been proximate to the user 140 at the time the user 140 submitted the utterance 222 (e.g., other devices associated with users of the service provider that are within a threshold distance from the user device 110, etc.). The context module 206 can then incorporate the passive context into the context determined for the chat session 220.

[0049] In some embodiments, the chat manager 202 can use the prediction module 204 to enrich the context determined for the chat session 220. As the user 140 can submit utterances related to different domains during the same chat session 220, by predicting future utterances that the user 140 will submit during the chat session 220, the chat manager 202 can incorporate additional domains related to future utterances that the user 140 can submit in the chat session 220 into the context and can generate an SLM that is capable of facilitating automated interactions with the user 140 throughout the chat session 220. For example, the chat manager 202 can provide the utterance, the user history, and / or the context determined for the chat session 220 to the prediction module 204. The prediction module 204 can provide the utterance and / or the context to a machine learning model that can generate an output based on the utterance and / or the context. The prediction module 204 can determine a topic (or domain) associated with future utterances that the user 140 can submit during the chat session 220. The chat manager 202 can then enrich the context by incorporating the additional domain into the context determined for the chat session 220.

[0050] In some embodiments, the chat manager 202 can determine a configuration of the SLM for facilitating automated interactions with the user 140 during the chat session 220 based on the context (or the enriched context). The configuration can specify the number of parameters and the parameter types to be included in the SLM (what the parameters are configured and trained to do). In some embodiments, the chat module 132 can be associated with an LLM that is configured and trained to perform automated interactions with users across all domains related to the service provider. Thus, the parameters of the LLM can include different subsets of parameters related to different domains of the organization, different subsets of parameters that can be used to interact with users having different backgrounds (e.g., different account statuses, different transaction histories, etc.), different subsets of parameters that can be used to interact with users having different passive contexts, etc.

[0051] Figure 3 An LLM 302 that can be associated with the chat module 132 in accordance with various embodiments of the present disclosure is shown. As shown, the LLM 302 can include a parameter set 350 that includes parameters a-u. While the LLM 302 is shown as including only twenty-one parameters for purposes of illustration, it has been contemplated that the LLM 302 can include a much greater number of parameters (e.g., trillions of parameters). Figure 3

[0052] In some embodiments, the parameter set 350 of the LLM 302 can include parameters associated with different contexts. For example, the parameter set 350 of the LLM 302 can include different subsets of parameters related to different domains of the service provider, different subsets of parameters that can be used to interact with users having different backgrounds (e.g., different account statuses, different transaction histories, etc.), different subsets of parameters that can be used to interact with users having different passive contexts, etc. In some embodiments, the chat manager 202 can select a subset of the parameters 350 for the SLM from the parameter set 350 of the LLM 302 based on the context derived for the chat session 220. For example, the chat manager 202 can determine from the parameters a-u that parameters a and c are associated with a domain determined for the chat session 220, parameters i and k are related to an account configuration similar to the account of the user 140, and parameters r and u are related to a passive context determined for the chat session 220. Accordingly, the chat manager 202 can determine a configuration that includes parameters a, c, i, k, r, and u for the SLM.

[0053] In some embodiments, the chat manager 202 can use the SLM generation module 208 to generate the SLM for the chat session 220 based on the parameters a, c, i, k, r, and u from the LLM 302. For example, the SLM generation module 208 can access one or more portions of the internal structure of the LLM 302 associated with the parameters a, c, i, k, r, and u (e.g., nodes, connections, data objects, computer software logic, weights, etc.) and generate a structure of the SLM based on the one or more portions of the internal structure (e.g., replicate the one or more portions of the internal structure and incorporate the replicated structure into the SLM, etc.).

[0054] ​In some embodiments, to improve efficiency of generating a custom SLM specifically for a different chat session, the chat module 132 can generate different building block SLMs for different contexts before receiving an utterance from a user. For example, the chat manager 202 can use the SLM generation module 208 to generate various building block SLMs, such as SLMs 312, 314, 316, 318, and 320 for different contexts. In this example, the SLM 312 is generated based on parameters b, d, and g and one or more portions of the internal structure of the LLM 302 associated with the parameters b, d, and g. The SLM 314 is generated based on parameters e, g, and j and one or more portions of the internal structure of the LLM 302 associated with the parameters e, g, and j. The SLM 316 is generated based on parameters f, h, and k and one or more portions of the internal structure of the LLM 302 associated with the parameters f, h, and k. The SLM 318 is generated based on parameters l, m, and p and one or more portions of the internal structure of the LLM 302 associated with the parameters l, m, and p. The SLM 320 is generated based on parameters q, r, and t and one or more portions of the internal structure of the LLM 302 associated with the parameters q, r, and t. The parameters associated with the different SLMs can be completely non-overlapping or partially overlapping. Since each of the SLMs 312, 314, 316, 318, and 320 includes respective portions of the structure of the LLM 302, each of the SLMs 312, 314, 316, 318, and 320 inherits the knowledge acquired and retained by the LLM 302 for the respective context.

[0055] Furthermore, some of the SLMs 312, 314, 316, 318, and 320 can be associated with different domains of the service provider, some of the SLMs 312, 314, 316, 318, and 320 can be associated with different account configurations, and some of the SLMs 312, 314, 316, 318, and 320 can be associated with different passive contexts. Accordingly, the SLM generation module 208 can select two or more building block SLMs from the SLMs 312, 314, 316, 318, and 320 and generate a custom SLM for the chat session 220 based on the selected building block SLMs.

[0056] Referring back to Figure 2 , the SLM generation module 208 can access building block SLMs that have been pre-generated in the SLM database 212. The SLM generation module 208 can then select building block SLMs, such as SLMs 232 and 234 (which can correspond to the SLMs 312 and 314, respectively), for the chat session 220 based on the context determined for the chat session 220. Figure 3For example, SLM 232 can be associated with the domain determined for chat session 220, and SLM 234 can be associated with the account context of user 140. SLM generation module 208 can then merge SLMs 232 and 234 to generate SLM 240. For example, SLM generation module 208 can combine the parameters and structure of SLMs 232 and 234 to generate SLM 240. Since SLM 240 includes the parameters and structure of SLMs 232 and 234, SLM 240 inherits the knowledge acquired by SLMs 232 and 234. In some embodiments, chat manager 202 can also select training data related to the context determined for chat session 220 from training database 214, and train SLM 240 using the selected training data.

[0057] Chat manager 202 can then deploy SLM 240 to facilitate automated interactions with user 140 during chat session 220. For example, chat manager 202 can provide utterance 222 and / or the context as input to SLM 240. Based on utterance 222, the context, and the knowledge acquired indirectly from LLM 302, SLM 240 can generate response 224, which can be sent to user client 170 via the communication channel during chat session 220. In some embodiments, SLM 240 can be configured to provide a series of interactions (e.g., multiple responses to user utterances) during chat session 220. For example, when user 140 asks about transactions conducted in the past week, SLM 240 can prompt user 140 to provide an identification of a particular transaction, and can provide any details about the transaction. When user 140 requests to dispute a transaction, SLM 240 can prompt user 140 to provide additional information, such as the reason for the dispute, the amount of the dispute, etc.

[0058] In some embodiments, the chat manager 202 can also continuously monitor and modify the SLM 240 during the chat session 220 to ensure that the SLM is sufficient to facilitate automated interactions with the user 140 during the chat session 220. In some embodiments, after generating the SLM 240 and deploying the SLM 240 to facilitate automated interactions with the user 140 during the chat session 220, the chat manager 202 can continue to monitor interactions between the user 140 and the deployed SLM 240. The chat manager 202 can detect whether the context of the chat session 220 has changed based on these interactions. For example, when the user submits another utterance 226 during the chat session 220, the chat manager 202 can determine an updated context based on the utterance 226. In some embodiments, the chat manager 202 can determine the updated context based on the new utterance 226, the original context determined for the chat session 220, and all previous utterances included in the chat session 220. For example, if the user 140 requests to add a credit card to an account after requesting to dispute a transaction, the chat manager 202 can determine that a new domain (e.g., a “funding source modification” domain) is now associated with the chat session 220. In another example, if it is determined that the task requested by the user 140 (e.g., dispute a transaction, etc.) has been completed, the chat manager 202 can determine that the domain (e.g., a “dispute” domain) can no longer be applicable to the chat session 220.

[0059] Based on the detected change in context, the chat manager 202 can modify the SLM 240 that has been deployed for the chat session using the SLM generation module 208. For example, the chat system can access (or otherwise generate) an SLM corresponding to the new context (e.g., a “funding source modification” domain) (e.g., one of SLMs 312, 314, 316, 318, and 320), and can incorporate the structure of that SLM into the deployed SLM 240. In other words, the chat module 132 can dynamically add (and / or remove) parameters and structure corresponding to the new context to the SLM 240 while the SLM 240 is deployed to facilitate automated interactions with the user 140 during the chat session 220. The chat manager 202 can then use the modified SLM 240 to continue to facilitate automated interactions with the user 140 during the chat session 220.

[0060] Chat manager 202 can continue to monitor interactions between SLM 240 and user 140 during chat session 220 and modify SLM 240 as needed according to any updated context determined for chat session 220. By dynamically deploying and modifying various SLMs during different chat sessions with various users, chat module 132 can use these SLMs to facilitate interactions with users in an efficient manner without incurring the computational costs of generating, deploying, and utilizing LLMs.

[0061] In some embodiments, after chat session 220 terminates, training module 210 can use the interactions (e.g., utterances 222, 224, 226, etc.) between SLM 240 and user 140 deployed during chat session 220 to retrain the underlying building block SLMs (e.g., SLMs 232 and 234). For example, if the generated responses resulted in more questions or requests, indicating that the responses were inaccurate, the underlying building block SLMs can be retrained such that future similar utterances by the user or users similar to the situation will result in different responses that are more accurate. Training module 210 can also store the interactions in training database 214 for use in training SLMs generated for future chat sessions. Continuously retraining the underlying building block SLMs using actual interactions with users can further improve the performance of the building block SLMs, which in turn will improve the performance of the merged SLMs generated based on the underlying building block SLMs for different chat sessions. In some embodiments, chat manager 202 can determine whether to store SLM 240 that has been deployed for chat session 220 for future use. For example, chat manager 202 can assign an expiration time for SLM 240. If user 140 initiates another chat session with chat module 132 before the expiration time of SLM 240, chat manager 202 can deploy the same SLM 240 to facilitate automated interactions with user 140 during the new chat session. Since SLM 240 is generated based on the context associated with previous session 220, SLM 240 has knowledge of the interaction history with user 140, which can be useful in facilitating interactions with user 140 in the new chat session.

[0062] Figure 4 An example chat interface 402 provided by chat client 170 is shown. As Figure 4As shown, the chat interface 402 includes a chat presentation portion 412 for displaying messages and / or content during a chat session (e.g., chat session 220). The chat session 220 can include messages exchanged between the chat client 170 and the chat module 132. The chat interface 402 also includes an input portion 414 that enables the user 140 to input a message (e.g., an utterance that can include textual data, audio data, multimedia data, etc.) for sending to the chat module 132 and a "send" button 416 for submitting a message entered in the input portion 414.

[0063] In this example, the user 140 can send the message 432 (e.g., an utterance) "Hello, I want to dispute a recent transaction" by entering the message 432 in the input portion 414 and selecting the "send" button 416. The user 140 can also speak the message, which can then be converted to text or not. When the user 140 selects the "send" button 416, the message 432 is sent by the chat client 170 to the chat module 132. The chat manager 202 can obtain the message 432 sent by the chat client 170 as the chat manager 202 monitors activity within the chat session 220.

[0064] The chat manager 202 can analyze the utterance 432 and other information (e.g., account configuration of the account of the user 140) to generate a context for the chat session 220. The SLM generation module 208 can then generate an SLM (e.g., SLM 240) for the chat session 220 based on the context. The SLM can provide automated interactions with the user 140 during the chat session 220. For example, the SLM 240 generates a response 438 and provides the response to the user 140 via the chat client 170 during the chat session 220. In this example, the response 438 includes several transactions 442, 444, and 446 recently made by the user 140 and enables the user 140 to select one of the transactions for dispute. The response 438 can include audio, or alternatively be presented via audio. The format of the response can depend on the response, the user, the device, and other factors.

[0065] Figure 5 Another example utterance 532 provided by the user 140 during the chat session 220 is shown. As Figure 5As shown, the utterance 532 is "Hello, I would like to transfer $$$ to my other account." In some embodiments, the chat manager 202 can update the context determined for the chat session 220 based on new utterances submitted by the user 140 during the chat session 220. In this example, after requesting a dispute for a transaction, the user 140 requests a funds transfer to an account. In this example, the chat manager 202 can determine, based on the utterance 532, that a new domain (e.g., a "funds transfer" domain) can now be associated with the chat session 220. The chat manager 202 can modify the SLM 240 based on the new context using the SLM generation module 208. For example, the SLM generation module 208 can add additional parameters and structures associated with the "funds transfer" domain to the SLM 240. The chat manager 202 can then use the modified SLM 240 to facilitate interactions with the user 140 during the chat session 220. Based on the new structures and parameters added to the SLM 240, the SLM 240 is now able to generate relevant responses for the user 140 in the "funds transfer" domain. In this example, the modified SLM 240 generates responses 538 and 540 for the chat session 220 based on the utterance 532. Specifically, the responses 538 and 540 prompt the user 140 to provide an account to which the funds will be transferred.

[0066] Figure 6 A process 600 for facilitating automated interactions with a user according to various embodiments of the present disclosure is shown. At least some of the steps in the process 600 can be performed by the chat module 132. The process 600 begins with receiving a first utterance from a device of a user via a chat session (at step 605). For example, after establishing a chat session 220 with the chat client 170 of the user device 110, the chat manager 202 can monitor any utterances provided by the user 140 during the chat session 220. In one example, the chat manager can detect the utterance 222 submitted by the user 140 during the chat session 220.

[0067] The process 600 then predicts (at step 610) one or more utterances that the user will submit and generates (at step 615) a context for the chat session. For example, the chat manager 202 can use the context module 206 to generate a context for the chat session 220 based on the utterance 222 and other information associated with the chat session 220 and / or the user 140. In some embodiments, the context module 206 can analyze the words in the utterance and determine a domain related to the service provider based on the words. In some embodiments, the context module 206 can also access additional information, such as account information associated with the user 140 (e.g., account status, transaction history, etc.), environmental information associated with the user 140 at the time the utterance 222 is submitted (e.g., tone of the utterance 222, location of the user 140, one or more other users within a threshold distance of the user 140, etc.). The context module 206 can generate a context based on the domain associated with the utterance 222 and the additional information.

[0068] Since the user 140 can submit utterances related to different domains of the service provider during the same chat session 220, the context module 206 of some embodiments can use the prediction module 204 to predict one or more additional utterances that the user 140 will submit during the chat session 220. In some embodiments, the context module 206 and / or the chat manager 202 can provide the utterance 222 and additional information (e.g., context) to the prediction module 204 for use in predicting utterances that the user 140 can submit during the chat session 220. In some embodiments, the prediction module 204 can use the utterance 222 and additional information to predict additional utterances that the user 140 can submit during the chat session 220. In some embodiments, the prediction module 204 can provide the utterance 222 and additional information to a machine learning model configured to predict future utterances of the user. The prediction module 204 can obtain an output from the machine learning model. Based on the output, the prediction module 204 can determine one or more additional domains that can be relevant to the chat session 220. In some embodiments, the context module 206 can incorporate the additional domains determined by the prediction module 204 into the context determined for the chat session 220.

[0069] The chat manager 202 can then generate an SLM for facilitating automated interactions with the user 140 during the chat session 220 based on the context using the SLM generation module 208. By incorporating the additional domains into the context, the chat manager can generate an SLM that is more likely to be able to facilitate automated interactions with the user 140 throughout the chat session 220.

[0070] The process 600 then determines (at step 620) whether one or more SLMs related to the context are available. If no SLMs related to the context are available, the process 600 uses (at step 625) the subset of parameters from the LLM to generate an SLM for the chat session. On the other hand, if one or more SLMs related to the context are available, the process 600 selects (at step 630) one or more pre-generated SLMs based on the context and merges (at step 635) the one or more pre-generated SLMs. As described herein, the chat manager 202 can use the SLM generation module 208 to generate an SLM specifically for the chat session 220 based on the context determined for the chat session 220. In some embodiments, to generate an SLM for the chat session 220, the SLM generation module 208 can determine a configuration of the SLM. The configuration can include a number of parameters and parameter types to include in the SLM. The parameter types can correspond to a subset of parameters of an existing LLM (e.g., the LLM 302) associated with the chat module 132. The SLM generation module 208 can then access the selected parameters and portions of the internal structure of the LLM 302 associated with the selected parameters and replicate them for the SLM. As a result, the SLM can inherit the knowledge that the LLM has already acquired with respect to the context determined for the chat session 220.

[0071] In some embodiments, to improve the performance of generating SLMs for different chat sessions, the SLM generation module 208 can pre-generate SLMs for different contexts. However, since the number of different possible contexts (e.g., different combinations of domains, different account contexts, different passive contexts, etc.) can be very large, the SLM generation module 208 can generate building block SLMs associated with different specific sub-contexts. For example, the SLM generation module 208 can generate a set of building block SLMs for different domains related to service providers, a set of building block SLMs for different account contexts, and a set of building block SLMs for different passive contexts. Each building block SLM can be generated using a different subset of parameters of the LLM 302 and a corresponding portion of the internal structure of the LLM 302.

[0072] To generate an SLM for the chat session 220, the SLM generation module 208 can select different building block SLMs that are relevant to the context determined for the chat session 220. For example, the SLM generation module 208 can select a first set of building block SLMs that are associated with a domain determined to be relevant to the chat session 220. The SLM generation module 208 can also select a second set of building block SLMs that are associated with an account context determined for the chat session 220, and a third set of building block SLMs that are associated with a passive context determined for the chat session 220. The SLM generation module 208 can then merge the selected building block SLMs (e.g., SLMs 232, 234, etc.). For example, the SLM generation module 208 can combine the parameters and corresponding structures of the building block SLMs into a single SLM 240. The single SLM 240 becomes a customized SLM that is specifically generated for the chat session 220.

[0073] The process 600 deploys the SLM to interact with the user during the chat session (at step 640). For example, the chat manager 202 can deploy the generated SLM 240 to facilitate automated interactions with the user 140 during the chat session 220. The SLM can analyze utterances submitted by the user 140 (e.g., utterance 222) during the chat session 220, and can generate content (e.g., response 224) based on the utterance 222. The chat manager 202 can provide the response 224 to the user 140 during the chat session 220.

[0074] The process 600 then determines whether the context of the chat session has changed (at step 645). If it is determined that the context has changed, the process 600 modifies the SLM (at step 650). For example, the chat manager 202 can continue to monitor utterances exchanged between the user 140 and the chat module 132 during the chat session 220. The chat manager 202 can also use the context module 206 to analyze the utterances being exchanged during the chat session 220. If it is detected that the context of the chat session 220 has changed due to a new utterance (e.g., utterance 226 submitted by the user 140), the chat manager 202 can use the SLM generation module 208 to modify the SLM 240 based on updated context determined by the context module 206. For example, when the updated context determines an additional domain related to a service provider, the SLM generation module 208 can incorporate additional parameters and structures (e.g., from one or more existing building block SLMs, etc.) into the SLM 240. The chat manager 202 can then deploy the modified SLM 240 to facilitate automated interactions with the user 140 during the chat session 220.

[0075] Figure 7is a block diagram of a computer system 700 suitable for implementing one or more embodiments of the present disclosure, including the service provider server 130 and the user devices 110, 170, 180, and 190. In various implementations, each of the user devices 110, 170, 180, and 190 can include a mobile cellular telephone, a personal computer (PC), a laptop computer, a wearable computing device, or the like suitable for wireless communication, and the service provider server 130 can include a network computing device, such as a server. Thus, it should be understood that the devices 110, 170, 180, 190, and 130 can be implemented as the computer system 700 in the following manner.

[0076] The computer system 700 includes a bus 712 or other communication mechanism for communicating information data, signals, and information between various components of the computer system 700. These components include an input / output (I / O) component 704 that processes a user action, such as selecting keys from a keypad / keypad, selecting one or more buttons or links, and the like, and sends appropriate signals to the bus 712. The I / O component 704 can also include output components, such as a display 702 and a cursor control 708 (e.g., a keyboard, keypad, mouse, etc.). The display 702 can be configured to present a login page for logging into a user account, a checkout page for purchasing an item from a merchant, or a chat interface for facilitating an online chat session. An optional audio input / output component 706 can also be included to allow a user to input information using voice commands that are converted into audio signals. The audio I / O component 706 can allow the user to hear audio. A transceiver or network interface 720 transmits and receives signals between the computer system 700 and other devices (e.g., another user device, a merchant server, or a service provider server) via a network 722. In one embodiment, the transmission is a wireless transmission, although other transmission media and methods can also be suitable. A processor 714, which can be a microcontroller, a digital signal processor (DSP), or other processing component, processes these various signals, such as for display on the computer system 700 or transmission to other devices via a communication link 724. The processor 714 can also control transmission of information (e.g., cookies or IP address) to other devices.

[0077] The components of the computer system 700 also include a system memory component 710 (e.g., RAM), a static storage component 716 (e.g., ROM), and / or a disk drive 718 (e.g., a solid state drive, a hard disk drive). The computer system 700 performs specific operations by the processor 714 and other components executing one or more sequences of instructions contained in the system memory component 710. For example, the processor 714 can perform the automated online chat functions described herein according to the process 600.

[0078] The logic can be encoded in a computer readable medium, which can refer to any medium that participates in providing instructions to the processor 714 for execution. Such a medium can take many forms, including but not limited to, nonvolatile media, volatile media, and transmission media. In various implementations, nonvolatile media includes optical or magnetic disks, volatile media includes dynamic memory, such as the system memory component 710, and transmission media includes coaxial cables, copper wire, and fiber optics, including the wires that comprise the bus 712. In one embodiment, the logic is encoded in non-transitory computer readable medium. In one example, transmission media can take the form of acoustic or light waves, such as those generated during radio, optical, and infrared data communications.

[0079] Some common forms of computer readable media include, for example, floppy disk, flexible disk, hard disk, magnetic tape, any other magnetic medium, CD-ROM, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, RAM, PROM, EPROM, FLASH-EPROM, any other memory chip or cartridge, or any other medium from which a computer is adapted to read.

[0080] In various embodiments of the present disclosure, execution of instructions sequences practicing the present disclosure can be performed by the computer system 700. In various other embodiments of the present disclosure, multiple computer systems 700 coupled by a communication link 724 to a network, such as a LAN, WLAN, PTSN, and / or various other wired or wireless networks, including telecommunications, mobile and cellular telephone networks, can coordinate performance of the instructions sequences to practice the present disclosure.

[0081] Where applicable, various embodiments provided by the present disclosure can employ hardware, software, or combinations thereof. Moreover, where applicable, the various hardware components and / or software components set forth herein can be combined into a composite component comprising software, hardware, and / or both, without departing from the spirit of the present disclosure. Where applicable, various hardware components and / or software components set forth herein can be separated into sub-components, including software, hardware, and / or both, without departing from the scope of the present disclosure. In addition, where applicable, the various hardware components and / or software components set forth herein can be implemented as a "virtual component" which employs a physical component, without departing from the scope of the present disclosure. In addition, where applicable, the various components set forth herein can be implemented as a hardware component comprising a physical machine, without departing from the scope of the present disclosure.

[0082] Software according to this disclosure, such as program code and / or data, can be stored on one or more computer readable media. It is also contemplated that software identified herein can be implemented using one or more general purpose or special purpose computers and / or computer systems, which can be networked and / or otherwise

[0083] Various features and steps described herein can be implemented as a system including one or more memories storing various information described herein and one or more processors coupled to the one or more memories and a network, where the one or more processors are operable to perform steps as described herein; as a non-transitory machine readable medium including a plurality of machine readable instructions adapted to cause one or more processors to perform a method including steps described herein when executed by the one or more processors; and as a method performed by one or more devices, such as hardware processors, user devices, servers, and other devices described herein.

Claims

1. A system comprising: Non-transitory memory; as well as One or more hardware processors, coupled to the non-transitory memory, are configured to read instructions from the non-transitory memory to cause the system to perform operations including: During a chat session between the system and a device associated with the user, a first utterance is received from the device, wherein the first utterance comprises multiple words; The context of the chat session is derived by analyzing the multiple words in the first utterance, wherein the context indicates a specific knowledge domain from multiple knowledge domains associated with the service provider; Based on the context derived for the chat session, two or more small language models are selected from a plurality of small language models corresponding to different contexts; A model is generated for the chat session by merging the two or more smaller language models; and The model is then used to generate a response to the first utterance.

2. The system according to claim 1, wherein, The operation also includes: Based on the first utterance, predict one or more utterances that the user will submit within the chat session; and The context is enriched based on one or more predicted utterances, wherein the two or more small language models are further selected based on the enriched context.

3. The system according to claim 1, wherein, The context includes account context, and wherein deriving the context for the chat session includes: Access account information associated with the user's account, wherein the account information includes account status and transaction history; and The account context is derived based on the account information.

4. The system according to claim 1, wherein, The operation also includes: The multiple smaller language models are generated for the different contexts using parameters and internal structures associated with the larger language model.

5. The system according to claim 4, wherein, The operation also includes: Selecting different subsets of the parameters for the different contexts; and The multiple smaller language models are implemented using different subsets of the parameters from the larger language model.

6. The system according to claim 1, wherein, The operation also includes: After the model generates a response to the first utterance, a second utterance is received from the device during the chat session; The updated context for the chat session is derived based on the second utterance; Modify the model based on the updated context; and The modified model generates a second response to the second utterance.

7. The system according to claim 6, wherein, Modifying the model includes at least one of the following: adding one or more additional parameters to the model, or removing one or more parameters from the model.

8. A method comprising: During a chat session established between the chat client and the computer system, a first message is received from the chat client; The computer system derives the context of the chat session based on the first utterance; Based on the context derived for the chat session, two or more small language models are selected from a plurality of small language models corresponding to different contexts; The computer system generates a small language model for the chat session by merging the two or more small language models. The small language model is used to generate a response to the first utterance; as well as The response is sent to the chat client.

9. The method according to claim 8, further comprising: Receive a second utterance from the chat client during the chat session; as well as The small language model is trained using the second utterance.

10. The method according to claim 8, wherein, The two or more mini-language models include at least a first mini-language model and a second mini-language model, and wherein generating the mini-language models for the chat session includes: The first small language model is determined to include a first set of parameters; and Add the first set of parameters to the second small language model.

11. The method of claim 10, further comprising: Train at least one of the first small language model or the second small language model.

12. The method according to claim 10, wherein, Generating the small language model also includes: Add the first internal structure of the first small language model to the second internal structure of the second small language model.

13. The method of claim 8, further comprising: Analyze the words included in the first utterance; and Based on the analysis of the words, a specific domain associated with the chat session is determined from multiple domains related to the service provider, wherein the context is further derived based on the specific domain.

14. The method of claim 8, further comprising: Access information associated with the account corresponding to the chat client; as well as The account context is determined based on the analysis of the information, wherein the account context indicates at least one of the account status of the account or the transaction history of the account, and wherein the context is further derived based on the account context.

15. A non-transitory machine-readable medium storing machine-readable instructions thereon, the machine-readable instructions being executable to cause a machine to perform operations including: Receive the first utterance from the device within a chat session established with the user's associated device; The context of the chat session is derived by analyzing the words included in the first utterance, wherein, The context indicates a specific domain from among multiple domains associated with the service provider; The configuration of a small language model is determined based on the context, wherein the configuration indicates a subset of parameters associated with a large language model configured to facilitate automated chat session interactions with users of the service provider; The small language model is generated using one or more parts of the structure corresponding to the subset of parameters of the large language model; as well as The small language model is then used to generate a response to the first utterance.

16. The non-transitory machine-readable medium according to claim 15, wherein, The operation also includes sending the response to the device within the chat session.

17. The non-transitory machine-readable medium according to claim 15, wherein, The operation also includes: Based on the first utterance, predict one or more utterances that the user will submit within the chat session; and The context is enriched based on the predicted one or more utterances, wherein the configuration of the small language model is further determined based on the enriched context.

18. The non-transitory machine-readable medium according to claim 15, wherein, The context includes account context, and wherein deriving the context for the chat session includes: Access account information associated with the user's account, wherein the account information includes account status and transaction history; and The account context is derived based on the account information.

19. The non-transitory machine-readable medium according to claim 15, wherein, The operation also includes: Receive a second message from the device within the chat session; The updated context for the chat session is derived based on the second utterance; Modify the small language model based on the updated context; and The modified small language model generates a second response to the second utterance.

20. The non-transitory machine-readable medium according to claim 19, wherein, Modifying the small language model includes at least one of the following: adding one or more additional parameters to the small language model, or removing one or more parameters from the small language model.

Images