Reverse Text Normalization for Contact Center Communications

Abstract

A method for inverse text normalization of contact center communications according to one embodiment includes performing named entity recognition on text from the contact center communications to identify one or more entities within the text; normalizing each of the one or more identified entities within the text using a weighted finite state transducer; and, in response to determining that at least one entity identified within the text could not be normalized using the weighted finite state transducer, normalizing the at least one entity identified within the text using a large-scale language model.

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS AND PRIORITY CLAIMS) This application claims priority to U.S. Patent Application No. 18 / 088,261, filed December 23, 2022, entitled "INVERSE TEXT NORMALIZATION OF CONTACT CENTER COMMUNICATIONS." [Background technology]

[0002] In contact center analytics, speech recognition is used to transcribe conversations between agents and customers, for example, to detect important events, detect emotions, summarize the content of the conversation, and / or for various other purposes. In some situations, calls may be automatically transcribed at contact centers so that the transcription can be displayed to agents or supervisors during or after the call. The output from a speech recognition system typically includes a sequence of words spelled exactly in the text as they were spoken. For example, dates, numbers, and addresses are presented in text format instead of the typical numeric format in which humans expect to read that information. Thus, automatic transcripts are often more difficult to read than human-generated transcripts, because human-generated transcripts are normalized in that they include information such as numeric entities (e.g., dates, numbers, etc.) in a human-readable format. Summary of the Invention

[0003] One embodiment is directed to unique systems, components, and methods for inverse text normalization of contact center communications. Other embodiments are directed to apparatus, systems, devices, hardware, methods, and combinations thereof for inverse text normalization of contact center communications.

[0004] According to one embodiment, a method for inverse text normalization of contact center communications may include performing named entity recognition on text from the contact center communications to identify one or more entities within the text; normalizing each of the one or more identified entities within the text using a weighted finite state transducer; and, in response to determining that at least one entity identified within the text could not be normalized using the weighted finite state transducer, normalizing the at least one entity identified within the text using a large-scale language model.

[0005] In some embodiments, the contact center communications may include a speech-to-text transcript of at least a portion of a contact center conversation with a user.

[0006] In some embodiments, normalizing each of the one or more identified entities in the text using a weighted finite state transducer may include normalizing a plurality of identified entities in the text using a weighted finite state transducer.

[0007] In some embodiments, normalizing the plurality of identified entities in the text using a weighted finite state transducer may include using a dedicated graph for each entity type of the plurality of identified entities.

[0008] In some embodiments, normalizing at least one entity identified in the text using a large-scale language model may include: receiving, for each entity of the at least one entity, entity text associated with the respective entity identified in the text; comparing the entity text to a set of training examples of the large-scale language model; selecting a training example from the set of training examples that is most similar to the entity text; and applying the large-scale language model to the entity text using a large-scale language model prompt that includes the selected training example.

[0009] In some embodiments, selecting a training example from the set of training examples that is most similar to the entity text may include selecting a training example from the set of training examples that share the most common words.

[0010] In some embodiments, selecting training examples from the set of training examples that are most similar to the entity text may include selecting at least five training examples.

[0011] In some embodiments, normalizing at least one entity identified in the text using the large-scale language model may include calculating an average of the probabilities from the word-probability pairs output by the large-scale language model, determining whether the average exceeds a predefined confidence threshold, and accepting the normalized output of the large-scale language model in response to determining that the average exceeds the predefined confidence threshold.

[0012] In some embodiments, the method may further include rejecting the normalized output of the large-scale language model in response to determining that the average does not exceed a predefined confidence threshold.

[0013] In some embodiments, performing named entity recognition on text from contact center communications to identify one or more entities within the text may include identifying at least one of a date, a phone number, a currency, a social security number, a time, or a zip code.

[0014] According to another embodiment, a computing system for inverse text normalization of contact center communications may include at least one processor and at least one memory having stored thereon a plurality of instructions that, upon execution by the at least one processor, cause the computing system to perform named entity recognition on text from the contact center communications to identify one or more entities within the text, normalize each of the one or more entities identified within the text using a weighted finite state transducer, and, in response to a determination that at least one entity identified within the text could not be normalized using the weighted finite state transducer, normalize the at least one entity identified within the text using a large-scale language model.

[0015] In some embodiments, the contact center communications may include a speech-to-text transcript of at least a portion of a contact center conversation with a user.

[0016] In some embodiments, normalizing each of the one or more identified entities in the text using a weighted finite state transducer may include normalizing a plurality of identified entities in the text using a weighted finite state transducer.

[0017] In some embodiments, normalizing a plurality of identified entities in text using a weighted finite state transducer may include using a dedicated graph for each entity type of the plurality of identified entities, where the plurality of identified entities may represent a plurality of different entity types.

[0018] In some embodiments, normalizing at least one entity identified in the text using a large-scale language model may include: receiving, for each entity of the at least one entity, entity text associated with the respective entity identified in the text; comparing the entity text to a set of training examples of the large-scale language model; selecting a training example from the set of training examples that is most similar to the entity text; and applying the large-scale language model to the entity text using a large-scale language model prompt that includes the selected training example.

[0019] In some embodiments, selecting a training example from the set of training examples that is most similar to the entity text may include selecting a training example from the set of training examples that share the most common words.

[0020] In some embodiments, normalizing at least one entity identified in the text using the large-scale language model may include calculating an average of the probabilities from the word-probability pairs output by the large-scale language model, determining whether the average exceeds a predefined confidence threshold, accepting the normalized output of the large-scale language model in response to a determination that the average exceeds the predefined confidence threshold, and rejecting the normalized output of the large-scale language model in response to a determination that the average does not exceed the predefined confidence threshold.

[0021] According to yet another embodiment, one or more non-transitory machine-readable storage media may include a plurality of instructions stored thereon that, upon execution by a processor, cause a computing system to: perform named entity recognition on text from contact center communications to identify a plurality of entities in the text; normalize each of the plurality of entities identified in the text using a weighted finite-state transducer and using a dedicated graph for each entity type of the plurality of entities identified in the text; determine that at least one entity identified in the text could not be normalized using the weighted finite-state transducer; and normalize the at least one entity identified in the text using a large-scale language model.

[0022] In some embodiments, normalizing at least one entity identified in the text using a large-scale language model may include: receiving, for each entity of the at least one entity, entity text associated with the respective entity identified in the text; comparing the entity text to a set of training examples of the large-scale language model; selecting a training example from the set of training examples that is most similar to the entity text; and applying the large-scale language model to the entity text using a large-scale language model prompt that includes the selected training example.

[0023] In some embodiments, normalizing at least one entity identified in the text using the large-scale language model may include calculating an average of the probabilities from the word-probability pairs output by the large-scale language model, determining whether the average exceeds a predefined confidence threshold, accepting the normalized output of the large-scale language model in response to a determination that the average exceeds the predefined confidence threshold, and rejecting the normalized output of the large-scale language model in response to a determination that the average does not exceed the predefined confidence threshold.

[0024] This Summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used as an aid in limiting the scope of the claimed subject matter. Further embodiments, forms, features, and aspects of the present application will become apparent from the description and figures provided herewith. [Brief explanation of the drawings]

[0025] The concepts described herein are by way of example and not limitation in the accompanying drawings. For simplicity and clarity of illustration, elements illustrated in the figures have not necessarily been drawn to scale. Where considered appropriate, reference labels have been repeated among the figures to indicate corresponding or analogous elements. [Figure 1] FIG. 1 illustrates a simplified block diagram of at least one embodiment of a contact center system. [Figure 2] FIG. 1 is a simplified block diagram of at least one embodiment of a computing device. [Figure 3] FIG. 1 is a simplified flow diagram of at least one embodiment of a method for inverse text normalization of contact center communications. [Figure 4] FIG. 1 is a simplified flow diagram of at least one embodiment of a method for normalizing text using a large-scale language model. [Figure 5] 1 is an example of unnormalized text and corresponding normalized text from a contact center communication. [Figure 6] 1 shows various examples of weighted finite state transducer (WFST) graphs. [Figure 7] 1 shows various examples of weighted finite state transducer (WFST) graphs. [Figure 8] 1 shows various examples of weighted finite state transducer (WFST) graphs. [Figure 9] Here are various examples of unrecoverable errors made by the normalization system. [Figure 10] 10 shows various examples of recoverable errors made by the normalization system. [Figure 11] 1 shows various examples of input text and normalization errors. [Figure 12] 1 is a table illustrating various named entity types and corresponding descriptions. [Figure 13] 1 is an example of text used to build prompts for a large-scale language model. [Figure 14] 10 is a table providing results from an experimental test comparing most similar selection of training examples for large-scale language model prompts with random selection. [Figure 15] 1 is a graph based on experimental test data plotting recall against precision using large-scale language model probability as the confidence threshold. DETAILED DESCRIPTION OF THE INVENTION

[0026] While the concepts of the present disclosure are susceptible to various modifications and alternative forms, specific embodiments have been shown by way of example in the drawings and are herein described in detail. It should be understood, however, that there is no intention to limit the concepts of the present disclosure to the particular forms disclosed, but on the contrary, the intention is to cover all modifications, equivalents, and alternatives consistent with the scope of this disclosure and the appended claims.

[0027] References herein to "one embodiment," "an embodiment," "an illustrative embodiment," and the like indicate that the described embodiment may include a particular feature, structure, or characteristic, but that all embodiments may or may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases do not necessarily refer to the same embodiment. It should be further understood that references to "preferred" components or features may indicate the desirability of a particular component or feature with respect to an embodiment, but that the present disclosure is not so limited with respect to other embodiments that may omit such component or feature. Furthermore, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to implement such feature, structure, or characteristic in connection with other embodiments, whether or not explicitly described. Furthermore, particular features, structures, or characteristics may be combined in any suitable combinations and / or subcombinations in various embodiments.

[0028] Additionally, items in a listing of the form "at least one of A, B, and C" can mean (A), (B), (C), (A and B), (B and C), (A and C), or (A, B, and C). Similarly, items in a listing of "at least one of A, B, or C" can mean (A), (B), (C), (A and B), (B and C), (A and C), or (A, B, and C). Further, with respect to the claims, use of words and phrases such as "a," "an," "at least one," and / or "at least a portion" should not be construed as limiting to only one of such elements unless specifically stated to the contrary, and use of phrases such as "at least a portion" and / or "a portion" should be construed to encompass both embodiments including only a portion of such elements and embodiments including the entirety of such elements unless specifically stated to the contrary.

[0029] The disclosed embodiments may, in some cases, be implemented in hardware, firmware, software, or a combination thereof. The disclosed embodiments may also be implemented as instructions stored on or executed by one or more transitory or non-transitory machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. A machine-readable storage medium may be embodied as any storage device, mechanism, or other physical structure for storing or transmitting information in a form readable by a machine (e.g., volatile or non-volatile memory, media disk, or other media device).

[0030] In the drawings, some structural or method features may be shown in a specific arrangement and / or ordering. However, it should be understood that such specific arrangement and / or ordering may not be required. Rather, in some embodiments, such features may be arranged in a different manner and / or order than that shown in the illustrative drawings, unless indicated to the contrary. Additionally, the inclusion of a structural or method feature in a particular figure does not imply that such feature is required in all embodiments, and in some embodiments, it may not be included or may be combined with other features.

[0031] It should be understood that contact center systems utilize speech recognition for a variety of situations, and that output from speech recognition systems typically includes a sequence of words spelled in text as they were spoken, rather than in a more human-friendly format. It should be understood that the process of converting text to words (e.g., "19" to "nineteen"), as may be performed, for example, to verbalize a word (e.g., "19") in a text-to-speech (TTS) system, is referred to as "normalization." The inverse process of normalizing speech-recognized text (e.g., "nineteen" to "19"), as may be performed, for example, to display numbers in a format more readable to a human reader, is referred to as "inverse-text normalization." However, as the context will be clear from this disclosure, inverse-text normalization may be referred to herein as "normalization" for brevity and simplicity of explanation.

[0032] 5 is a table that provides an example of unnormalized text (e.g., as output from a speech recognition module) in the first column and the corresponding normalized text in the second column. When comparing the unnormalized text to the normalized text, it should be understood that the majority of the text remains the same; however, text containing numbers (as indicated by bold / underlined text) has been normalized (e.g., through inverse text normalization) to present the text in the expected human-readable numeric format. More specifically, "one nine three six nine oh oh four oh five" has been recognized as an Israeli phone number and normalized to be "1-936-900-405," "three eight two two two forty nine hundred" has likewise been recognized as an international phone number and normalized to be "3-822-240-900," and "8,1,8,3,3" has been recognized as a PIN number and normalized to be "81833."

[0033] Many different approaches can be used to perform inverse text normalization, each with its own set of benefits and drawbacks. For example, in various embodiments, the normalization system may rely on graph-based methods or deep neural networks. Graph-based methods may rely on weighted finite-state transducers (WFSTs) to map a sequence of input symbols to a sequence of output symbols, with each output path having an associated weight / cost. For example, FIG. 6 shows a graph 600 (or subgraph) that transforms the word sequence "twenty three" into either the path "20 3" or the path "23." As shown, the "20 3" path includes three subpaths with corresponding weights: the "20" subpath 602, which has an associated weight / cost of 1.7; the [space] subpath 604, which has an associated weight / cost of 0; and the "3" subpath 606, which has an associated weight / cost of 1.7 and a total cost of 3.4. However, the "23" path includes a single "23" subpath 608, which has an associated weight / cost of 1.7. The total weight / cost of the "23" route is less than the weight / cost of the "203" route, so the system selects that route.

[0034] It should be understood that for text normalization applications, the input to the graph is typically a sequence of words of a known entity. For example, the sequence of words may be an address that needs to be verbalized in a text-to-speech system such as a satellite navigation system. However, in inverse text normalization, the input to the graph is often complete sentences as output from a speech recognition system. Within these complete sentences, there may or may not be substrings that require normalization, and the normalization graph does not know whether there are strings that need to be normalized, and if so, where those strings are located within the complete text.

[0035] A normalization graph may be constructed from subcomponents called "subgraphs," which may be instantiated in some structure relative to each other (e.g., concatenated, instantiated in parallel, etc.). One of the more important subgraphs is the "cardinality" subgraph, which may be instantiated inside normalization graphs for other entities because digits and numbers constitute the majority of the content that needs to be normalized from text output by a speech recognition system. The cardinality subgraph converts textual digits in word form into numeric form. As an example, FIG. 7 shows a currency normalization graph 700 that normalizes dollar amounts, or more specifically, normalizes text that follows the format "X dollars and Y cents," where X can be any number and Y can be any number between 1 and 99. As shown, currency normalization graph 700 includes cardinality subgraph 702, the word "dollars" 704, the word "and" 706, cardinality subgraph 708 limited to numbers between 1 and 99, and the word "cents" 710. However, as noted above, in some embodiments, the text to be normalized may be contained within a larger section of text. Thus, in some embodiments, currency normalization graph 700 of FIG. 7 may be modified to include “character loop” subgraphs 802 at the beginning and end of the graph, as shown by currency normalization graph 800 of FIG. 8 . It should be understood that “character loop” subgraph 802 simply accepts and outputs the same character for all characters in a language. In other embodiments, subgraphs 700, 800 may be further modified by incorporating “currency subgraphs” to account for multiple types of currencies (e.g., dollars and euros). In another embodiment, a global normalization graph may be constructed such that subgraphs similar to those described above are instantiated in parallel with the character loop subgraphs at the beginning and end. It should be understood that graph-based normalization is characterized by very high precision, but often at the cost of relatively low recall, since the graphs are constructed manually and generally do not include all of the variability of human speech.

[0036] As mentioned above, in other embodiments, the normalization technique may rely on a deep neural network. In such embodiments, a sequence of words may be input to the neural network, and the neural network may output a sequence of words with normalized sections. However, to train a neural network for normalization, a significant amount of annotated training material is used, and human annotators convert every spoken word sequence into its corresponding normalized form. Additionally, other drawbacks of using a neural network for normalization include the fact that neural networks typically attempt to produce normalized outputs even when the normalized output is incorrect, and biased training material can lead to normalization errors. For example, a neural network may output "The price is $30" when the input was "The price is 30 euros" if the training data contains a significant number of prices in dollars and too few prices in euros.

[0037] There are two different types of errors produced by inverse text normalization systems: unrecoverable errors and recoverable errors. Unrecoverable errors are errors that, if they occur, change the semantic meaning of the text. Recoverable errors are errors where the output may not be optimal, but a human reader can determine the correct information by reading the text. The example above of the normalization of "the price is 30 euros" as "The price is $30" is an example of an unrecoverable error because a reader of the normalized text cannot determine that the original audio actually referred to the price in euros. Figure 9 shows various other examples of unrecoverable errors, and Figure 10 shows various examples of recoverable errors.

[0038] As described in more detail below, the inverse text normalization techniques described herein may include performing named entity recognition on text from contact center communications to identify entities referenced in the text, normalizing each of those entities using a weighted finite-state transducer, and, for entities identified in the text that could not be normalized using a weighted finite-state transducer, normalizing them using a large-scale language model.

[0039] 1, there is shown a simplified block diagram of at least one embodiment of a communications infrastructure and / or content center system that may be used in conjunction with one or more of the embodiments described herein. The contact center system 100 may be embodied as any system that provides contact center services (e.g., call center services, chat center services, SMS center services, etc.) to end users and otherwise enables the functionality described herein. The exemplary contact center system 100 includes a customer device 102, a network 104, a switch / media gateway 106, a call controller 108, an interactive media response (IMR) server 110, a routing server 112, a storage device 114, a statistics server 116, agent devices 118A, 118B, 118C, a media server 120, a knowledge management server 122, a knowledge system 124, a chat server 126, a web server 128, an interaction (iXn) server 130, a universal contact server 132, a reporting server 134, a media services server 136, and an analytics module 138.The exemplary embodiment of FIG. 1 includes one customer device 102, one network 104, one switch / media gateway 106, one call controller 108, one IMR server 110, one routing server 112, one storage device 114, one statistics server 116, one media server 120, one knowledge management server 122, one knowledge system 124, one chat server 126, one iXn server 130, one universal contact server 132, one reporting server 134, one media services server 136, and one analytics module 138. Although only one module 138 is shown, in other embodiments, contact center system 100 may include multiple customer devices 102, networks 104, switch / media gateways 106, call controllers 108, IMR servers 110, routing servers 112, storage devices 114, statistics servers 116, media servers 120, knowledge management servers 122, knowledge systems 124, chat servers 126, iXn servers 130, universal contact servers 132, reporting servers 134, media services servers 136, and / or analytics modules 138. Furthermore, in some embodiments, one or more of the components described herein may be excluded from system 100, and one or more of the components described as being independent may form part of another component, and / or one or more of the components described as forming part of another component may be independent.

[0040] It should be understood that, as used herein, the term "contact center system" refers to the system illustrated in Figure 1 and / or its components, while the term "contact center" refers more generally to contact center systems, the customer service providers that operate those systems, and / or the organizations or companies associated therewith. Thus, unless specifically limited otherwise, the term "contact center" generally refers to a contact center system (e.g., contact center system 100), associated customer service providers (e.g., particular customer service providers / agents that provide customer service via contact center system 100), and the organizations or companies on behalf of which that customer service is provided.

[0041] By way of background, customer service providers may offer many types of services through contact centers. Such contact centers may be staffed with employees or customer service agents (or simply “agents”) who serve as an interface between a company, enterprise, government agency, or organization (hereinafter interchangeably referred to as an “organization” or “enterprise”) and people, such as users, individuals, or customers (hereinafter interchangeably referred to as “individuals,” “customers,” or “contact center clients”). For example, contact center agents may assist customers in making purchasing decisions, placing orders, or resolving issues related to products or services they have already received. Within a contact center, such interactions between contact center agents and external entities or customers may occur via various communication channels, such as, for example, voice (e.g., telephone calls or voice over IP, i.e., VoIP calls), video (e.g., video conferencing), text (e.g., email and text chat), screen sharing, co-browsing, and / or other communication channels.

[0042] Operationally, contact centers generally strive to provide quality service to customers while minimizing costs. For example, one way contact centers operate is to handle all customer interactions with live agents. While this approach may be entirely successful from a service quality perspective, it would likely be prohibitively expensive due to the high cost of agent labor. For this reason, most contact centers utilize some level of automated processes, such as interactive voice response (IVR) systems, interactive media response (IMR) systems, Internet robots (i.e., "bots"), automated chat modules (i.e., "chatbots"), and / or other automated processes, in place of live agents. In many cases, this has proven to be a successful strategy, as automated processes can be highly efficient at handling certain types of interactions and effective in reducing the need for live agents. Such automation allows contact centers to target the use of human agents to more difficult customer interactions, while the automated processes handle more repetitive or routine tasks. Furthermore, automated processes can be structured in a way that optimizes efficiency and promotes repeatability. Although human agents, i.e., live agents, may forget to ask certain questions or pursue certain details thoroughly, such errors are typically avoided through the use of automated processes. While customer service providers increasingly rely on automated processes to interact with customers, the use of such technologies by customers remains far less developed. Thus, on the contact center side of the interaction, IVR systems, IMR systems, and / or bots are used to automate parts of the interaction, while actions on the customer side remain manually performed by the customer.

[0043] It should be understood that the contact center system 100 can be used by customer service providers to provide various types of services to customers. For example, the contact center system 100 can be used to engage in and manage interactions in which automated processes (or bots) or human agents communicate with customers. As should be understood, the contact center system 100 can be an in-house facility of a business or enterprise for performing sales and customer service functions for products and services available through the enterprise. In another embodiment, the contact center system 100 can be operated by a third-party service provider contracted to provide services for another organization. Furthermore, the contact center system 100 can be deployed on equipment dedicated to the enterprise or third-party service provider and / or in a remote computing environment, such as, for example, a private or public cloud environment with infrastructure to support multiple contact centers for multiple enterprises. The contact center system 100 can include software applications or programs that can run on-site, remotely, or some combination thereof. Furthermore, it should be understood that the various components of the contact center system 100 can be distributed across various geographic locations, but need not necessarily be contained in a single location or computing environment.

[0044] Furthermore, unless specifically limited otherwise, it should be understood that any of the computing elements of the present invention may be implemented within a cloud-based or cloud computing environment. As used herein and further described below with reference to computing device 200, "cloud computing" or simply "cloud" is defined as a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services), which can be rapidly provisioned via virtualization, released with minimal management effort or service provider interaction, and then scaled accordingly. Cloud computing can be comprised of a variety of characteristics (e.g., on-demand self-service, wide area network access, resource pooling, rapid scalability, measurable services, etc.), service models (e.g., Software as a Service ("SaaS"), Platform as a Service ("PaaS"), Infrastructure as a Service ("IaaS"), and deployment models (e.g., private cloud, community cloud, public cloud, etc.)). Cloud execution models, often referred to as "serverless architectures," generally involve a service provider dynamically managing the allocation and provisioning of remote servers to achieve desired functionality.

[0045] It should be understood that any of the computer-implemented components, modules, or servers described in connection with Figure 1 may be implemented via one or more types of computing devices, such as, for example, computing device 200 of Figure 2. As will be appreciated, contact center system 100 generally manages resources (e.g., employees, computers, telecommunications equipment, etc.) to enable the delivery of services via telephone, email, chat, or other communication mechanisms. Such services may vary depending on the type of contact center and may include, for example, customer service, help desk functions, emergency response, telemarketing, order taking, and / or other features.

[0046] A customer desiring to receive service from the contact center system 100 may initiate inbound communications (e.g., telephone calls, emails, chats, etc.) to the contact center system 100 via a customer device 102. While FIG. 1 shows one such customer device, namely, the customer device 102, it should be understood that any number of customer devices 102 may be present. The customer device 102 may be, for example, a communication device such as a telephone, smartphone, computer, tablet, or laptop. In accordance with the functionality described herein, a customer may generally use the customer device 102 to initiate, manage, and conduct communications with the contact center system 100, such as telephone calls, emails, chats, text messages, web browsing sessions, and other multimedia transactions.

[0047] Incoming and outgoing communications from and to the customer device 102 may traverse a network 104, the nature of which typically depends on the type of customer device being used and the mode of communication. By way of example, the network 104 may include a telephone, mobile telephone, and / or data service communication network. The network 104 may be a private or public switched telephone network (PSTN), a local area network (LAN), a private wide area network (WAN), and / or a public WAN such as the Internet. Additionally, the network 104 may include a wireless carrier network, including a code division multiple access (CDMA) network, a global system for mobile communications (GSM) network, or any wireless network / technology conventional in the art, including, but not limited to, 3G, 4G, LTE, 5G, etc.

[0048] The switch / media gateway 106 may be coupled to the network 104 for receiving and transmitting telephone calls between customers and the contact center system 100. The switch / media gateway 106 may include a telephone or communication switch configured to act as a central switch for agent-level routing within the center. The switch may be a hardware switching system or may be implemented via software. For example, the switch 106 may include an automatic call distributor, a private branch exchange (PBX), an IP-based software switch, and / or any other switch having dedicated hardware and software configured to receive interactions from customers, from the Internet, and / or from the telephone network, and route those interactions to, for example, one of the agent devices 118. Thus, in general, the switch / media gateway 106 establishes a connection between the customer device 102 and the agent device 118, thereby establishing a voice connection between the customer and the agent.

[0049] As further shown, the switch / media gateway 106 may be coupled to a call controller 108, which functions, for example, as an adapter or interface between the switch and other routing, monitoring, and communication processing components of the contact center system 100. The call controller 108 may be configured to process PSTN calls, VoIP calls, and / or other types of calls. For example, the call controller 108 may include computer-telephone integration (CTI) software for interfacing with the switch / media gateway and other components. The call controller 108 may include a session initiation protocol (SIP) server for processing SIP calls. The call controller 108 may also extract data about incoming interactions, such as the customer's phone number, IP address, or email address, and then communicate them to other contact center components when processing the interaction.

[0050] The interactive media response (IMR) server 110 can be configured to enable self-help or virtual assistant functions. Specifically, the IMR server 110 can be similar to an interactive voice response (IVR) server, except that the IMR server 110 is not limited to voice and can also cover various media channels. In one example illustrating voice, the IMR server 110 can be configured with IMR scripts to query customers about their needs. For example, a bank contact center may instruct customers via an IMR script to "press 1" if they want to retrieve their account balance. Through ongoing interaction with the IMR server 110, customers can receive service without needing to speak with an agent. The IMR server 110 can also be configured to determine why a customer is contacting the contact center so that the communication can be routed to the appropriate resource. IMR configuration can be implemented through the use of self-service and / or assisted-service tools, including web-based tools for developing IVR and routing applications that run within the contact center environment.

[0051] The routing server 112 may function to route incoming interactions. For example, when it is determined that an incoming communication should be handled by a human agent, functionality within the routing server 112 may select the most appropriate agent and route the communication to that agent. This agent selection may be based on which available agent is best suited to handle the communication. More specifically, the selection of the appropriate agent may be based on a routing strategy or algorithm implemented by the routing server 112. In doing so, the routing server 112 may query data related to the incoming interaction, such as data related to the particular customer, available agents, and type of interaction, which may be stored in a particular database as described herein. Once an agent is selected, the routing server 112 may interact with the call controller 108 to route (i.e., connect) the incoming interaction to a corresponding agent device 118. As part of this connection, information about the customer may be provided to the selected agent via their agent device 118. This information is intended to enhance the service the agent can provide to the customer.

[0052] It should be appreciated that the contact center system 100 may include one or more mass storage devices (generally represented by the storage device 114) for storing data in one or more databases related to the contact center's functions. For example, the storage device 114 may store customer data maintained in a customer database. Such customer data may include, for example, customer profiles, contact information, service level agreements (SLAs), and interaction history (e.g., details of previous interactions with particular customers, including the nature of the previous interactions, disposition data, wait times, handling times, and actions taken by the contact center to resolve the customer's issues). As another example, the storage device 114 may store agent data in an agent database. The agent data maintained by the contact center system 100 may include, for example, agent availability, as well as agent profiles, schedules, skills, handling times, and / or other related data. As another example, the storage device 114 may store interaction data in an interaction database. The interaction data may include, for example, data related to numerous past interactions between customers and the contact center. More generally, unless otherwise specifically specified, it should be understood that storage device 114 may include databases and / or be configured to store data related to any of the types of information described herein, with those databases and / or data being accessible to other modules or servers of contact center system 100 in a manner that facilitates the functions described herein. For example, a server or module of contact center system 100 may query such databases to retrieve data stored therein or transmit data to a database for storage. Storage device 114 may take the form of, for example, any conventional storage medium and may be housed locally or operated remotely.By way of example, the database may be a Cassandra database, a NoSQL database, or a SQL database, and may be managed by a database management system such as Oracle, IBM DB2, Microsoft SQL Server, Microsoft Access, PostgreSQL, or the like.

[0053] Statistics server 116 may be configured to record and aggregate data related to the performance and operational aspects of contact center system 100. Such information may be compiled by statistics server 116 and made available to other servers and modules, such as reporting server 134, which may then use the data to generate reports used to manage operational aspects of the contact center and take automated actions in accordance with the functionality described herein. Such data may relate to the status of contact center resources, such as average wait times, abandon rates, agent occupancy, and others as may be required by the functionality described herein.

[0054] The agent devices 118 of the contact center system 100 may be communication devices configured to interact with the various components and modules of the contact center system 100 in a manner that facilitates the functionality described herein. For example, the agent devices 118 may include telephones adapted for regular telephone calls or VoIP calls. The agent devices 118 may further include computing devices configured to communicate with servers of the contact center system 100, perform business-related data processing, and interface with customers via voice, chat, email, and other multimedia communication mechanisms in accordance with the functionality described herein. While FIG. 1 shows three such agent devices 118, namely, agent devices 118A, 118B, and 118C, it should be understood that any number of agent devices 118 may be present in a particular embodiment.

[0055] The multimedia / social media server 120 may be configured to facilitate media (non-voice) interactions with the customer device 102 and / or server 128. Such media interactions may relate to, for example, email, voicemail, chat, video, text messaging, web, social media, co-browsing, etc. The multimedia / social media server 120 may take the form of any IP router conventional in the art having dedicated hardware and software for receiving, processing, and forwarding multimedia events and communications.

[0056] The knowledge management server 122 may be configured to facilitate interactions between customers and the knowledge system 124. Generally, the knowledge system 124 may be a computer system capable of receiving questions or inquiries and providing answers in response. The knowledge system 124 may be included as part of the contact center system 100 or may be operated remotely by a third party. The knowledge system 124 may include an artificial intelligence computer system capable of answering questions posed in natural language by extracting information from sources such as encyclopedias, dictionaries, newswire articles, literary works, or other documents submitted to the knowledge system 124 as reference material. As an example, the knowledge system 124 may be embodied as an IBM Watson or similar system.

[0057] The chat server 126 may be configured to conduct, direct, and manage electronic chat communications with customers. Generally, the chat server 126 is configured to conduct and maintain chat conversations and generate chat transcripts. Such chat communications may be conducted by the chat server 126 in a manner such that customers communicate with automated chatbots, human agents, or both. In an exemplary embodiment, the chat server 126 may function as a chat orchestration server that allocates chat conversations to chatbots and available human agents. In such cases, the processing logic of the chat server 126 may be rules driven to leverage intelligent workload distribution among available chat resources. The chat server 126 may also implement, manage, and facilitate user interfaces (UIs) associated with the chat feature, including UIs generated on either the customer device 102 or the agent device 118. The chat server 126 may be configured to transfer chats between automated and human sources within a single chat session with a particular customer, such as transferring a chat session from a chatbot to a human agent or from a human agent to a chatbot. The chat server 126 may also be coupled to the knowledge management server 122 and the knowledge system 124 to receive suggestions and answers to inquiries posed by the customer during the chat, for example, to provide links to related articles.

[0058] A web server 128 may be included to provide site hosts for various social interaction sites to which customers subscribe, such as Facebook, Twitter, and Instagram. While depicted as part of the contact center system 100, it should be understood that the web server 128 may be provided by a third party and / or maintained remotely. The web server 128 may also serve web pages to businesses or organizations supported by the contact center system 100. For example, customers may view web pages to receive information about a particular business's products and services. Within such businesses' web pages, mechanisms may be provided for initiating interactions with the contact center system 100, for example, via web chat, voice, or email. One example of such a mechanism is a widget that may be deployed on a web page or website hosted on the web server 128. As used herein, a widget refers to a user interface component that performs a specific function. In some embodiments, a widget may include a graphical user interface control that may be overlaid on a web page displayed to customers via the Internet. A widget may include buttons or other controls that display information, such as in a window or text box, or allow a customer to access a particular function, such as sharing or opening a file or initiating a communication. In some implementations, a widget includes a user interface component with a portable portion of code that can be installed and executed within a separate web page without being compiled. Some widgets may include corresponding or additional user interfaces and may be configured to access various local resources (e.g., calendar or contact information on the customer device) or remote resources over a network (e.g., instant messaging, email, or social networking updates).

[0059] The interaction (iXn) server 130 may be configured to manage contact center deferrable activities and their routing to human agents for completion. As used herein, deferrable activities include back-office work that can be performed offline, such as responding to emails, participating in training, and other activities that do not require real-time communication with customers. As an example, the interaction (iXn) server 130 may be configured to interact with the routing server 112 to select an appropriate agent to handle each deferrable activity. Once assigned to a particular agent, the deferrable activity is pushed to that agent, such that the deferrable activity is displayed on the selected agent's agent device 118. The deferrable activity may be displayed in a work bin as a task for the selected agent to complete. The work bin functionality may be implemented via any conventional data structure, such as a linked list, an array, and / or other suitable data structure. Each agent device 118 may include a work bin. As an example, the work bin may be maintained in a buffer memory of the corresponding agent device 118.

[0060] A universal contact server (UCS) 132 may be configured to retrieve information stored in a customer database and / or send information to a customer database for storage in the customer database. For example, UCS 132 may be utilized as part of a chat function to facilitate maintaining a history of how chats with particular customers were handled, which may then be used as a reference for how future chat communications should be handled. More generally, UCS 132 may be configured to facilitate maintaining a history of customer preferences, such as preferred media channels and best times to contact. To do this, UCS 132 may be configured to identify data related to each customer's interaction history, such as data regarding comments from agents, customer communication history, etc. Each of these data types may then be stored in customer database 222 or other modules and retrieved as needed by the functions described herein.

[0061] The reporting server 134 may be configured to generate reports from data compiled and aggregated by the statistics server 116 or other sources. Such reports may include near real-time or historical reports and may relate to the status of contact center resources and performance characteristics, such as average wait times, abandon rates, and / or agent occupancy. Reports may be generated automatically or upon specific request from a requestor (e.g., an agent, a manager, a contact center application, etc.). The reports may then be used to manage contact center operations in accordance with the functionality described herein.

[0062] Media services server 136 may be configured to provide audio and / or video services to support contact center functions, such as IVR or IMR system prompts (e.g., playing audio files), music on hold, voicemail / single-party recording, multi-party recording (e.g., of audio and / or video calls), screen recording, speech recognition, dual tone multi-frequency (DTMF) recognition, fax, audio and video transcoding, secure real-time transport protocol (SRTP), audio conferencing, video conferencing, coaching (e.g., support for a coach to eavesdrop on an interaction between a customer and an agent and for the coach to provide comments to an agent without the customer hearing the comments), call analysis, keyword spotting, and / or other related functions, according to the functionality described herein.

[0063] The analytics module 138 may be configured to provide systems and methods for performing analytics on data received from multiple different data sources, as may be required by the functionality described herein. According to illustrative embodiments, the analytics module 138 may also generate, update, train, and modify predictors or models based on collected data, such as, for example, customer data, agent data, and interaction data. The models may include customer or agent behavior models. The behavior models may be used to predict, for example, customer or agent behavior in various situations, thereby enabling embodiments of the present invention to adjust interactions based on such predictions or allocate resources in preparation for predicted characteristics of future interactions, thereby improving overall contact center performance and customer experience. While the analytics module is described as being part of the contact center, it will be understood that such behavior models may also be implemented in customer systems (or, as used herein, the “customer side” of the interaction) and used to the benefit of the customer.

[0064] According to an example embodiment, the analytics module 138 may have access to data stored in the storage device 114, including a customer database and an agent database. The analytics module 138 may also have access to an interaction database that stores data related to interactions and interaction content (e.g., transcripts of interactions and events detected therein), interaction metadata (e.g., customer identifier, agent identifier, interaction medium, interaction length, interaction start and end times, department, tagged categories), and application settings (e.g., interaction paths through the contact center). Additionally, the analytics module 138 may be configured to retrieve data stored in the storage device 114 for use in developing and training algorithms and models, for example, by applying machine learning techniques.

[0065] One or more of the included models may be configured to predict customer or agent behavior and / or aspects related to contact center operation and performance. Furthermore, one or more of the models may be used for natural language processing, including, for example, intent recognition. The models may be developed based on known first-principles equations describing the system, data resulting in empirical models, or a combination of known first-principles equations and data. When developing models for use in the present embodiments, first-principles equations are often not available or easily derived, so building empirical models based on collected and stored data may generally be preferred. To adequately capture the relationships between manipulated / disturbance variables and controlled variables of a complex system, in some embodiments, it may be preferred that the model be nonlinear. This is because nonlinear models may exhibit curvilinear relationships between manipulated / disturbance variables and controlled variables, rather than linear relationships, as is common in complex systems such as those discussed herein. Given the aforementioned requirements, machine learning or neural network-based approaches may be preferred for implementing the models. For example, neural networks may be developed based on empirical data using advanced regression algorithms.

[0066] The analysis module 138 may further include an optimizer. As will be appreciated, an optimizer may be used to minimize a "cost function" under a set of constraints, where the cost function is a mathematical expression of a desired objective or system behavior. Because the model may be nonlinear, the optimizer may be a nonlinear programming optimizer. However, it is contemplated that the techniques described herein may be implemented using a variety of different types of optimization approaches, individually or in combination, including, but not limited to, linear programming, quadratic programming, mixed-integer nonlinear programming, stochastic programming, global nonlinear programming, genetic algorithms, particle / swarm techniques, etc.

[0067] According to some embodiments, the model and optimizer may be used together in an optimization system. For example, the analytics module 138 may utilize the optimization system as part of an optimization process in which aspects of contact center performance and operation are optimized or at least enhanced. This may include, for example, features related to customer experience, agent experience, interaction routing, natural language processing, intent recognition, or other functionality related to automated processes.

[0068] The various components, modules, and / or servers in FIG. 1 (as well as other figures contained herein) may each include one or more processors that execute computer program instructions and interact with other system components to perform the various functions described herein. Such computer program instructions may be stored in memory implemented using standard memory devices such as, for example, random-access memory (RAM), or may be stored on other non-transitory computer-readable media such as, for example, a CD-ROM, a flash drive, or the like. While each of the server functions is described as being provided by a particular server, those skilled in the art should understand that the functions of various servers may be combined or integrated into a single server, or the functions of a particular server may be distributed across one or more other servers without departing from the scope of the present invention. Furthermore, the terms “interaction” and “communication” are used interchangeably and generally refer to any real-time and non-real-time interaction using any communication channel, including, but not limited to, telephone calls (PSTN or VoIP calls), email, Vmail, video, chat, screen sharing, text messages, social media messages, WebRTC calls, etc. Access to and control of components of the contact center system 100 may be affected through user interfaces (UIs) that may be generated on the customer device 102 and / or the agent device 118.

[0069] As previously mentioned, in some embodiments, contact center system 100 may operate as a hybrid system in which some or all components are hosted remotely, such as in a cloud-based or cloud computing environment. It should be understood that each of the devices of contact center system 100 may be embodied as part of, include, or form part of one or more computing devices similar to computing device 200 described below with reference to FIG. 2.

[0070] 2, a simplified block diagram of at least one embodiment of a computing device 200 is shown. The illustrative computing device 200 depicts at least one embodiment of each of the computing devices, systems, services, controllers, switches, gateways, engines, modules, and / or computing components (e.g., which may be collectively referred to interchangeably as computing devices, servers, or modules for ease of description) described herein. For example, various computing devices may be processes or threads executing on one or more processors of one or more computing devices 200, which processors may execute computer program instructions and interact with other system modules to perform various functions described herein. Unless specifically limited otherwise, functionality described in the context of multiple computing devices may be integrated into a single computing device, or various functionality described in the context of a single computing device may be distributed across several computing devices. Additionally, with respect to the computing systems described herein, such as the contact center system 100 of FIG. 1, the various servers and computer devices of the system may be located on a local computing device 200 (e.g., on-site in the same physical location as the contact center agents), a remote computing device 200 (e.g., off-site or in a cloud-based or cloud computing environment, e.g., in a remote data center connected via a network), or some combination thereof.In some embodiments, functionality provided by a server located on an off-site computing device may be accessed and provided via a virtual private network (VPN) as if such server were on-site, or functionality may be provided using software as a service (SaaS) accessed over the internet using various protocols, e.g., by exchanging data via extensible markup language (XML), JSON, and / or functionality may be otherwise accessed / utilized.

[0071] In some embodiments, computing device 200 may be embodied as a server, a desktop computer, a laptop computer, a tablet computer, a notebook, a netbook, an Ultrabook™, a mobile phone, a mobile computing device, a smartphone, a wearable computing device, a personal digital assistant, an Internet of Things (IoT) device, a processing system, a wireless access point, a router, a gateway, and / or any other computing, processing, and / or communication device capable of performing the functions described herein.

[0072] The computing device 200 includes a processing device 202 that executes algorithms and / or processes data according to operational logic 208, an input / output device 204 that enables communication between the computing device 200 and one or more external devices 210, and a memory 206 that stores data received from the external device 210 via the input / output device 204, for example.

[0073] The input / output devices 204 enable the computing device 200 to communicate with external devices 210. For example, the input / output devices 204 may include a transceiver, a network adapter, a network card, an interface, one or more communication ports (e.g., a USB port, a serial port, a parallel port, an analog port, a digital port, VGA, DVI, HDMI, FireWire, CAT5, or any other type of communication port or interface), and / or other communication circuitry. The communication circuitry of the computing device 200 may be configured to conduct such communication using any one or more communication technologies (e.g., wireless or wired communication) and associated protocols (e.g., Ethernet, Bluetooth, Wi-Fi, WiMAX, etc.) depending on the particular computing device 200. The input / output devices 204 may include hardware, software, and / or firmware suitable for implementing the techniques described herein.

[0074] External device 210 may be any type of device that allows data to be input or output from computing device 200. For example, in various embodiments, external device 210 may be embodied as one or more of the devices / systems described herein and / or portions thereof. Furthermore, in some embodiments, external device 210 may be embodied as another computing device, a switch, a diagnostic tool, a controller, a printer, a display, an alarm, a peripheral device (e.g., a keyboard, a mouse, a touchscreen display, etc.), and / or any other computing, processing, and / or communication device capable of performing the functions described herein. Furthermore, it should be understood that in some embodiments, external device 210 may be integrated into computing device 200.

[0075] Processing device 202 may be embodied as any type of processor capable of performing the functions described herein. In particular, processing device 202 may be embodied as one or more single-core or multi-core processors, microcontrollers, or other processors or processing / control circuitry. For example, in some embodiments, processing device 202 may include or be embodied as an arithmetic logic unit (ALU), a central processing unit (CPU), a digital signal processor (DSP), a graphics processing unit (GPU), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), and / or another suitable processor. Processing device 202 may be of a programmable type, a state machine implemented by dedicated hardware, or a combination thereof. A processing device 202 having multiple processing units may utilize distributed processing, pipelined processing, and / or parallel processing in various embodiments. Furthermore, processing device 202 may be dedicated to performing only the operations described herein, or may be utilized in one or more additional applications. In an illustrative embodiment, processing device 202 is programmable, executing algorithms and / or processing data according to operating logic 208 defined by programming instructions (e.g., software or firmware) stored in memory 206. Additionally or alternatively, operating logic 208 of processing device 202 may be defined at least in part by hardwired logic or other hardware. Furthermore, processing device 202 may include one or more components of any type suitable for processing signals received from input / output device 204 or from other components or devices and providing a desired output signal.Such components may include digital circuits, analog circuits, or a combination thereof.

[0076] Memory 206 may be of one or more types of non-transitory computer-readable media, such as solid-state memory, electromagnetic memory, optical memory, or combinations thereof. Furthermore, memory 206 may be volatile and / or non-volatile, and in some embodiments, some or all of memory 206 may be of a portable type, such as a disk, tape, memory stick, cartridge, and / or other suitable portable memory. In operation, memory 206 may store various data and software used during operation of computing device 200, such as an operating system, applications, programs, libraries, and drivers. It should be understood that memory 206 may store data manipulated by operational logic 208 of processing device 202, such as, for example, data representing signals received from and / or transmitted to input / output devices 204, in addition to or instead of storing programming instructions defining operational logic 208. As shown in FIG. 2, memory 206 may be included with and / or coupled to processing device 202, depending on the particular embodiment. For example, in some embodiments, processing device 202, memory 206, and / or other components of computing device 200 may form part of a system-on-a-chip (SoC) and be integrated into a single integrated circuit chip.

[0077] In some embodiments, various components of computing device 200 (e.g., processing device 202 and memory 206) may be communicatively coupled via an input / output subsystem, which may be embodied as circuits and / or components for facilitating input / output operations with processing device 202, memory 206, and other components of computing device 200. For example, the input / output subsystem may be embodied as or otherwise include a memory controller hub, an input / output control hub, firmware devices, communication links (i.e., point-to-point links, bus links, wires, cables, light guides, printed circuit board traces, etc.), and / or other components and subsystems for facilitating input / output operations.

[0078] Computing device 200 may, in other embodiments, include other or additional components, such as those commonly found in a typical computing device (e.g., various input / output devices and / or other components). It should further be understood that one or more of the components of computing device 200 described herein may be distributed across multiple computing devices. In other words, the techniques described herein may be employed by a computing system including one or more computing devices. Additionally, while only a single processing device 202, I / O device 204, and memory 206 are illustratively shown in FIG. 2 , it should be understood that a particular computing device 200 may, in other embodiments, include multiple processing devices 202, I / O devices 204, and / or memory 206. Furthermore, in some embodiments, two or more external devices 210 may communicate with computing device 200.

[0079] Computing device 200 may be one of multiple devices connected by a network or to other systems / resources via a network. The network may be embodied as any one or more types of communications network capable of facilitating communication between various devices communicatively connected via the network. Thus, the network may include one or more networks, routers, switches, access points, hubs, computers, client devices, endpoints, nodes, and / or other intervening network devices. For example, the network may be embodied as or otherwise include one or more cellular networks, telephone networks, local or wide area networks, publicly available global networks (e.g., the Internet), ad hoc networks, short-range communications links, or combinations thereof. In some embodiments, the network may include circuit-switched voice or data networks, packet-switched voice or data networks, and / or any other network capable of carrying voice and / or data. In particular, in some embodiments, the network may include an Internet Protocol (IP)-based network and / or an Asynchronous Transfer Mode (ATM)-based network. In some embodiments, the network may handle voice traffic (e.g., via a Voice over IP (VOIP) network), web traffic, and / or other network traffic, depending on the particular embodiment and / or the devices in the system that communicate with each other.In various embodiments, the networks may include analog or digital wired and wireless networks (e.g., IEEE 802.11 networks, Public Switched Telephone Networks (PSTN), Integrated Services Digital Networks (ISDN), and Digital Subscriber Lines (xDSL)), third generation (3G) mobile networks, fourth generation (4G) mobile networks, fifth generation (5G) mobile networks, wired Ethernet networks, private networks (e.g., intranets, etc.), radio, television, cable, satellite, and / or any other distribution or tunneling mechanism for carrying data, or any suitable combination of such networks. It should be understood that various devices / systems may communicate with each other over different networks depending on the source and / or destination device / system.

[0080] It should be understood that computing device 200 may communicate with other computing devices 200 via any type of gateway or tunneling protocol, such as Secure Sockets Layer or Transport Layer Security. Network interfaces may include built-in network adapters, such as network interface cards, suitable for interfacing a computing device to any type of network capable of performing the operations described herein. Furthermore, the network environment may be a virtual network environment in which various network components are virtualized. For example, the various machines may be virtual machines implemented as software-based computers running on a physical machine. The virtual machines may share the same operating system, or in other embodiments, different operating systems may run on each virtual machine instance. For example, a "hypervisor" type of virtualization is used in which multiple virtual machines run on the same host physical machine, each functioning as if it had its own dedicated box. Other types of virtualization may be employed in other embodiments, such as networks (e.g., via software-defined networking) or functions (e.g., via network function virtualization).

[0081] Accordingly, one or more of the computing devices 200 described herein may be embodied as or form a part of one or more cloud-based systems. In a cloud-based embodiment, the cloud-based system may be embodied, for example, as a server-agnostic computing solution that executes instructions on demand, executes instructions only when prompted by specific activities / triggers, and does not consume computing resources when not in use. That is, the system may be embodied as a virtual computing environment residing “on” a computing system (e.g., a distributed network of devices) in which various virtual functions (e.g., Lambda functions, Azure Functions, Google Cloud Functions, and / or other suitable virtual functions) may be executed corresponding to the functionality of the system described herein. For example, when an event occurs (e.g., data is transferred to the system for processing), the virtual computing environment may be communicated (e.g., via a request to the virtual computing environment's API), which may then route the request to the correct virtual function (e.g., a specific server-agnostic computing resource) based on a set of rules. Thus, when a request for transmission of data is made by a user (e.g., via an appropriate user interface to the system), an appropriate virtual function may be executed to perform the action before deleting the instance of the virtual function.

[0082] 3, in use, a computing system (e.g., contact center system 100 and / or computing device 200) may perform a method 300 for inverse text normalization of contact center communications. It should be understood that certain blocks of method 300 are illustrated by way of example, and that such blocks may be combined or divided, added or removed, and / or reordered in whole or in part, depending on the particular embodiment, unless stated to the contrary.

[0083] The exemplary method 300 begins at block 302, where a computing system receives text from a contact center communication. For example, in some embodiments, the computing system may receive a speech-to-text transcript (completely or partially) of a contact center conversation with a user (e.g., between an agent and the user, between a chatbot and the user, etc.). It should be understood that the text may be received by the computing system in real time, near real time, or at some point after the conversation has occurred.

[0084] At block 304, the computing system performs named entity recognition on the text to identify entities within the text. It should be understood that named entity recognition involves analyzing unstructured text and classifying named entities mentioned in the text according to predefined categories, such as numbers, names, currency, places, and other types of entities. It should be further understood that the computing system may utilize any named entity recognition technology or service suitable for performing the functions described herein. In some embodiments, the output of the named entity recognition includes subsections of the text and corresponding entity types / classes for each identified entity. In other words, the named entity recognition system identifies boundaries within the broader text for named entities.

[0085] For example, the table of FIG. 12 illustrates various named entities that may be identified in at least one embodiment. More specifically, the named entities in such an embodiment include CARD_NUMBER (e.g., credit card number and CVV), CURRENCY (e.g., amount), DATE (e.g., "June 3rd"), and the like. 日"), DATE_REL (e.g., dates for which today's date is desired, such as "tomorrow" and "next Tuesday"), GEO (e.g., addresses and place names), ORG (e.g., organization names), PERSON (e.g., person's names), PHONE_NUMBER (e.g., phone numbers), PIIOTHER (e.g., personally identifiable information without a dedicated tag, such as account numbers, license plate numbers, etc.), PRODUCT (e.g., product names), SPELL_OUT (e.g., spelling of any word), SSN (e.g., Social Security numbers and other national ID numbers), TIME (e.g., time of day), URL (e.g., web URL), USERINFO (e.g., user information such as email, password, and username), ZIP_CODE (e.g., zip code), NORMALIZE (e.g., any sequence of words that needs to be normalized but is not included in the list above), and / or other named entities. It should be understood that in other embodiments, the predefined named entities may include additional and / or alternative entities. For example, in another embodiment, the list of named entities may focus on numeric entities (eg, card numbers, dates, phone numbers, social security numbers, currency, time, zip codes, etc.).

[0086] At block 306, the computing system normalizes, or more specifically, attempts to normalize, each of the identified entities (i.e., from the named entity recognition) using a weighted finite-state transducer. In doing so, at block 308, the computing system uses dedicated graphs for each different entity type / class. In an illustrative embodiment, dedicated graphs are constructed and used by the computing system for multiple named entity types / classes (e.g., all or a subset of the types / classes) predefined by the named entity recognition system. For example, in an embodiment using named entities described with reference to FIG. 12, the computing system may utilize a separate dedicated graph for each of the CARD_NUMBER, CURRENCY, DATE, DATE_REL, GEO, ORG, PERSON, PHONE_NUMBER, PIIOTHER, PRODUCT, SPELL_OUT, SSN, TIME, URL, USERINFO, ZIP_CODE, and NORMALIZE entity types / classes. In such an embodiment, it should be understood that the NORMALIZE entity type / class is a "catch-all" that includes any identified entity graph that does not fall into one of the other classes of FIG.

[0087] It should be appreciated that such a specialized graph is smaller and more accurate than the global normalization graph. It should further be appreciated that the techniques described herein reduce the number of errors associated with normalizing text by normalizing only entities identified using a weighted finite-state transducer instead of normalizing the text as a whole. A common error of normalization systems that normalize all of the text is the inability to detect instances that do not need to be normalized. By way of example, FIG. 11 shows various examples of input text and text normalized using the global normalization graph. It should be appreciated that the output will contain errors, as text should never be normalized and human readers would prefer to read the original text. For example, it may be desirable to normalize "one second, (please wait)" as "1 2 nd " is particularly significant because the term "second" is used as an original measure of time, not an ordinal number. Similarly, the error associated with normalizing "is seventeen twenty-five and eight three-four south" to "is 17:25 and 834 south" reflects another very bad normalization, because addresses are interpreted as times by the global normalization graph.

[0088] In contrast, by first performing named entity recognition, as described herein, a computing system can detect portions of text associated with entities to be normalized and overcome the challenge of determining the type / class of such entities. For example, if a computing system determines that "ninety pounds" in a speech recognition output represents currency, it can normalize it to "90 pounds" using a specialized currency graph, while if "90 pounds (£)" in a speed recognition output represents weight, it can normalize it to "90 pounds (lb)" using a specialized weight graph. Similarly, a computing system can normalize the standalone string "twelve twenty-five" to "1225" if the identified entity is an address, or to "12.25 dollars" if the identified entity is currency (or perhaps not be normalized at all to avoid an unrecoverable error between $12.25 and $1225). Furthermore, because the text boundaries of the identified entities are known from named entity recognition, the risk of errors similar to those in Figure 11 is substantially reduced, even when processed by a weighted finite-state transducer associated with a "catch-all" entity type / class (e.g., NORMALIZE from Figure 12).

[0089] At block 310, the computing system determines whether each entity identified in the text was normalized using a weighted finite-state transducer. If not, method 300 may end. However, if the computing system determines at block 312 that at least one entity was not normalized using a weighted finite-state transducer (e.g., there was one or more rejected cases), method 300 proceeds to block 314, where the computing system normalizes the rejected cases (i.e., entities that could not be normalized) using a large-scale language model. In some embodiments, to do so, the computing system may perform method 400 of FIG. 4, described below, for each rejected case.

[0090] It should be appreciated that the final normalized text may be transmitted and / or displayed to a user, agent, administrator, customer, and / or other human reader, depending on the particular context. For example, in some embodiments, a chatbot may output the final normalized text to a user / customer.

[0091] Although blocks 302-314 are described relatively serially, it should be understood that various blocks of method 300 may be performed in parallel in some embodiments.

[0092] 4, in use, a computing system (e.g., contact center system 100 and / or computing device 200) may perform method 400 for normalizing text using a large-scale language model. It should be understood that certain blocks of method 400 are illustrated by way of example, and that such blocks may be combined or divided, added or removed, and / or reordered in whole or in part, depending on the particular embodiment, unless stated to the contrary. In some embodiments, method 400 may be performed for each identified entity that could not be normalized using a weighted finite-state transducer as described above.

[0093] The exemplary method 400 begins at block 402, in which a computing system receives text associated with an identified entity. At block 404, the computing system compares the text to training examples, and at block 406, the computing system selects training examples that are most similar to the text to be normalized. For example, in some embodiments, the computing system may select training examples that share the most common words with the text to be normalized. Additionally, it should be understood that the number of training examples to be selected may vary depending on the particular embodiment. For example, in some embodiments, the computing system selects at least five training examples (e.g., between 5 and 10 examples).

[0094] In block 408, the computing system applies the large-scale language model to the text using a prompt including the selected training examples. It should be understood that the computing system may use any large-scale language model suitable for performing the functions described herein (e.g., GPT-2, GPT-3, T0pp, etc.). When using a large-scale language model for inverse text normalization, the computing system constructs a prompt and instructs the model to continue the text. FIG. 13 shows an example of text used to construct a prompt to normalize the date "zero nine two thousand and six." As shown, the large-scale language model prompt 1300 includes a task description 1302 and a set of examples 1304 that perform the same task.

[0095] It should be understood that large-scale language models work by predicting the next word in a sequence along with the associated probability of that word. Thus, the output from a large-scale language model is a normalized version of the text (w i p i ), the occurrence of a newline character can signify the end of the normalized string.

[0096] At block 410, the computing system calculates an average of probabilities from the word-probability pairs output by the large-scale language model for the normalized text. At block 412, the computing system determines whether the average exceeds a predefined confidence threshold. If at block 414, the computing system determines that the average exceeds the predefined confidence threshold, method 400 proceeds to block 416, where the computing system accepts the normalization from the large-scale language model. However, if at block 414, the computing system determines that the average does not exceed the predefined confidence threshold, method 400 proceeds to block 418, where the computing system rejects the normalization from the large-scale language model. In various embodiments, the computing system may utilize different confidence thresholds, depending, for example, on the desired precision and recall of a particular system. For example, in one embodiment, the computing system may utilize a predefined confidence threshold of 0.90, while in another embodiment, the computing system may utilize a predefined confidence threshold of 0.95.

[0097] It should be understood that by selecting similar examples for large-scale language model prompts (rather than randomly selecting examples), a computing system can improve the rate of true positive results and reduce the rate of false positive results. For example, FIG. 14 is a table providing results from an experimental test comparing the most similar selection with random selection of training examples for large-scale language model prompts. As shown, selecting similar examples more than doubled the true positive rate and halved the false positive rate. As seen in FIG. 14, a true positive indicates that the model output the correct string; a false positive indicates that the model output the wrong string; a true negative indicates that a string that should not have been normalized was not normalized; and a false negative indicates that a string that should have been normalized was not normalized.

[0098] It should be further appreciated that by utilizing a confidence threshold, a computing system can further reduce the number of false positives for a large-scale language model. Figure 15 is a graph based on experimental test data plotting recall against precision using large-scale language model probability as the confidence threshold. Each point represents a different confidence threshold, and the confidence level produces a different precision / recall working point. As can be seen from Figure 15, by using a large-scale language model in the manner described herein, recall can be increased by approximately 20% of the examples rejected from normalization by the weighted finite-state transducer without increasing false positives.

[0099] Although blocks 402-418 are described relatively serially, it should be understood that various blocks of method 400 may be performed in parallel in some embodiments.

Claims

1. 1. A method for inverse text normalization of contact center communications, comprising: performing named entity recognition on text from the contact center communications to identify one or more entities within the text; normalizing each of the identified one or more entities in the text using a weighted finite-state transducer; and in response to determining that at least one entity identified in the text could not be normalized using the weighted finite state transducer, normalizing the at least one entity identified in the text using a large-scale language model.

2. The method of claim 1 , wherein the contact center communications include speech-to-text transcripts of at least a portion of contact center conversations with users.

3. 2. The method of claim 1 , wherein normalizing each of the one or more identified entities in the text using a weighted finite-state transducer comprises normalizing a plurality of identified entities in the text using a weighted finite-state transducer.

4. 4. The method of claim 3, wherein normalizing the plurality of identified entities in the text using weighted finite state transducers comprises using a dedicated graph for each entity type of the plurality of identified entities.

5. normalizing the at least one entity identified in the text using the large-scale language model; receiving, for each entity of the at least one entity, entity text associated with the respective entity identified within the text; comparing the entity text with a set of training examples of the large-scale language model; selecting a training example from the set of training examples that is most similar to the entity text; applying the large scale language model to the entity text using a large scale language model prompt that includes the selected training examples.

6. 6. The method of claim 5, wherein selecting the training examples from the set of training examples that are most similar to the entity text comprises selecting training examples from the set of training examples that share the most common words.

7. The method of claim 6 , wherein selecting the training examples from the set of training examples that are most similar to the entity text comprises selecting at least five training examples.

8. normalizing the at least one entity identified in the text using the large-scale language model; calculating an average probability from the word-probability pairs output by the large-scale language model; determining whether the average exceeds a predefined confidence threshold; and accepting a normalized output of the large-scale language model in response to determining that the average exceeds the predefined confidence threshold.

9. 9. The method of claim 8, further comprising: rejecting the normalized output of the large-scale language model in response to determining that the average does not exceed the predefined confidence threshold.

10. 10. The method of claim 1, wherein performing named entity recognition on the text from the contact center communication to identify the one or more entities within the text comprises identifying at least one of a date, a phone number, a currency, a social security number, a time, or a zip code.

11. 1. A computing system for inverse text normalization of contact center communications, comprising: at least one processor; and at least one memory containing a plurality of instructions stored therein, the instructions, upon execution by the at least one processor, causing the computing system to: performing named entity recognition on text from the contact center communications to identify one or more entities within the text; normalizing each of the identified one or more entities in the text using a weighted finite-state transducer; and in response to determining that at least one entity identified in the text could not be normalized using the weighted finite state transducer, normalizing the at least one entity identified in the text using a large-scale language model.

12. The computing system of claim 11 , wherein the contact center communications include speech-to-text transcripts of at least a portion of contact center conversations with users.

13. 12. The computing system of claim 11, wherein normalizing each of the identified one or more entities in the text using a weighted finite state transducer comprises normalizing multiple identified entities in the text using a weighted finite state transducer.

14. normalizing the plurality of identified entities in the text using a weighted finite state transducer includes using a dedicated graph for each entity type of the plurality of identified entities; The computing system of claim 13 , wherein the plurality of identified entities represents a plurality of different entity types.

15. normalizing the at least one entity identified in the text using the large-scale language model; receiving, for each entity of the at least one entity, entity text associated with the respective entity identified within the text; comparing the entity text with a set of training examples of the large-scale language model; selecting a training example from the set of training examples that is most similar to the entity text; and applying the large-scale language model to the entity text using large-scale language model prompts that include the selected training examples.

16. 16. The computing system of claim 15, wherein selecting the training examples from the set of training examples that are most similar to the entity text comprises selecting training examples from the set of training examples that share the most common words.

17. normalizing the at least one entity identified in the text using the large-scale language model; calculating an average probability from the word-probability pairs output by the large-scale language model; determining whether the average exceeds a predefined confidence threshold; In response to determining that the average exceeds the predefined confidence threshold, accepting a normalized output of the large-scale language model; and rejecting the normalized output of the large-scale language model in response to determining that the average does not exceed the predefined confidence threshold.

18. One or more non-transitory machine-readable storage media including a plurality of instructions stored thereon, the instructions, upon execution by a processor, causing a computing system to: performing named entity recognition on text from the contact center communication to identify a plurality of entities within the text; normalizing each entity of the plurality of entities identified in the text using a weighted finite state transducer and using a dedicated graph for each entity type of the plurality of entities identified in the text; determining that at least one entity identified within the text could not be normalized using the weighted finite state transducer; and normalizing the at least one entity identified in the text using a large-scale language model.

19. normalizing the at least one entity identified in the text using the large-scale language model; receiving, for each entity of the at least one entity, entity text associated with the respective entity identified within the text; comparing the entity text with a set of training examples of the large-scale language model; selecting a training example from the set of training examples that is most similar to the entity text; and applying the large scale language model to the entity text using large scale language model prompts that include the selected training examples.

20. normalizing the at least one entity identified in the text using the large-scale language model; calculating an average probability from the word-probability pairs output by the large-scale language model; determining whether the average exceeds a predefined confidence threshold; In response to determining that the average exceeds the predefined confidence threshold, accepting a normalized output of the large-scale language model; and rejecting the normalized output of the large-scale language model in response to determining that the average does not exceed the predefined confidence threshold.

Images