Natural language processing with contact center data

Abstract

The invention relates to natural language processing of contact center transcript data. In particular, but not exclusively, the present disclosure relates to natural language processing techniques for automatically tagging contact center transcript data. More specifically, but not exclusively, the disclosure relates to forming a pairing model from tagged contact center transcript data.

Description

[0001] Related Applications

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 376,631, filed on September 22, 2022, and U.S. Provisional Application No. 63 / 391,582, filed on July 22, 2022. The entire contents of each of the above provisional applications are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to natural language processing of contact center transcript data. In particular, but not exclusively, the present disclosure relates to natural language processing techniques for automatically labeling contact center transcript data. More particularly, but not exclusively, the present disclosure relates to forming a pairing model from labeled contact center transcript data. Background Art

[0004] In many contact center systems, an agent may perform or present one or more actions or offers to a contact as part of an interaction (e.g., a call between the agent and the contact). It can be difficult for an agent to select the most relevant or appropriate action or offer; this problem is exacerbated when the number of possible actions or offers available for the agent to suggest is large (e.g., one hundred or more). This difficulty may result in suboptimal suggested actions or offers, which may increase interaction time, result in suboptimal allocation of actions / offers to other agents in the contact center, and result in inefficient use of contact center resources because the contact is not presented with the action or offer that is most appropriate for its situation; further, the contact's selection of an offer / action may limit access to offers / actions for future contacts in the contact center system.

[0005] Understanding historical call data can be helpful in order to determine a pairing model for a contact center system, or in order to determine which offers or actions an agent should perform or present to a contact. However, there are some challenges. For example, there are some shortcomings in processing transcript data to find actions or products discussed in interactions between agents and contacts. Such transcript data is inherently noisy due to factors such as misrecognition of words by speech transcription engines (e.g., due to the diversity of human accents involved in interactions within a call center). Second, action data is typically only labeled for transcripts associated with known outcomes (i.e., actions or offers that were made and accepted). Existing techniques do not consider actions or offers that were made within an interaction but not accepted.

[0006] Therefore, there is a need for improved techniques for forming pairing models from natural language transcript data. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] In order to facilitate a more complete understanding of the present disclosure, reference is now made to the accompanying drawings, in which like elements are represented by like reference numerals. These drawings should not be construed as limiting the present disclosure, but are for illustration purposes only.

[0008] Figure 1A and Figure 1B A block diagram of a contact distribution center according to an embodiment of the present disclosure is shown;

[0009] FIG. 2A to FIG. 2C A block diagram of a contact distribution center system according to an embodiment of the present disclosure is shown;

[0010] Figure 3 A pairing module according to one aspect of the present disclosure is shown;

[0011] Figure 4 shows a plurality of transcripts from which a pairing model may be formed according to an embodiment of the present disclosure;

[0012] Figure 5 shows the action probabilities determined using the first method and the second method according to an embodiment of the present disclosure;

[0013] Figure 6 A transcript marking pipeline according to one aspect of the present disclosure is shown;

[0014] Figure 7 A training process for training a transcript labeling pipeline according to one aspect of the present disclosure is shown;

[0015] Figure 8 A method for decision-making behavior pairing in a contact center system according to one aspect of the present disclosure is shown;

[0016] Fig. 9A and 9B A method of determining a label vector from a related transcript according to aspects of the present disclosure is shown;

[0017] Fig.10 A method for training a transcript labeling pipeline according to one aspect of the present disclosure is shown;

[0018] Fig.11 A method for automatic vectorization of plain text transcript data according to one aspect of the present disclosure is shown; and

[0019] Fig.12 Table showing the results of applying various classification methods on the noisy transcript dataset. DETAILED DESCRIPTION

[0020] A typical contact distribution system algorithmically distributes contacts that arrive at a contact distribution center to agents that are available to handle those contacts. At times, a contact distribution center may be in an "L1 state" and have agents available and waiting to be assigned to contacts. At other times, a contact distribution center may be in an "L2 state" and have contacts waiting in one or more queues for agents to become available for assignment. At other times, a contact distribution system may be in an "L3" state and have multiple agents available and multiple contacts waiting to be assigned. An example of a contact distribution system is a contact center system that receives contacts (e.g., phone calls, Internet chat sessions, emails, etc.) to be assigned to agents.

[0021] In some traditional contact distribution centers, contacts (e.g., callers) are assigned to agents that are sorted based on arrival time, and agents are sorted to receive contacts based on the time at which those agents become available. This strategy may be referred to as a "first-in, first-out," "FIFO," or "round-robin" strategy. For example, in an L2 environment, when an agent becomes available, the contact at the head of the queue is selected to be assigned to that agent. In other traditional contact distribution centers, a performance-based routing (PBR) strategy may be implemented to prioritize higher performing agents for contact distribution. For example, under PBR, the highest performing agent among the available agents receives the next available contact.

[0022] The present disclosure relates to optimization strategies, such as "behavior pairing" or "BP" strategies, for assigning contacts to agents that improve upon traditional assignment methods. The goal of BP is to balance the utilization of agents while improving the performance of the overall contact distribution center, which may be beyond what FIFO or PBR methods can achieve in practice. This is a remarkable achievement because BP acts on the same contacts and the same agents as FIFO or PBR methods, roughly balancing the utilization of agents provided by FIFO, while improving the overall performance of the contact distribution center, which exceeds the performance provided by FIFO or PBR in practice. BP improves performance by assigning agents and contact pairs in a way that takes into account the assignment of potential subsequent agents and contact pairs, so that when the benefits of all assignments are aggregated, they may exceed the benefits of FIFO and PBR strategies.

[0023] Various BP strategies may be used, such as a diagonal model BP strategy or a network flow (or "off-diagonal") BP strategy. These contact allocation strategies and others are described in detail for contact center contexts in, for example, U.S. Patent Nos. 9,300,802; 9,781,269; 9,787,841 and 9,930,180; all of which are incorporated herein by reference. BP strategies may be applied to L1 environments (agent surplus, one contact; select among multiple available / free agents), L2 environments (contact surplus, one available / free agent; select among multiple contacts in a queue), and L3 environments (multiple agents and multiple contacts; select among paired arrangements).

[0024] The various BP strategies discussed above can be considered two-dimensional (2-D), where one dimension is related to agents and a second dimension is related to contacts (e.g., callers), and the various BP strategies consider information about agents and contacts to pair them. As explained in detail below, embodiments of the present disclosure are directed to decision BP strategies that consider higher dimensional assignments. For a three-dimensional (3-D) example, a BP strategy may assign an agent to a contact and both a set of actions that the agent can take during the contact assignment or a set of offers that the agent can make.

[0025] These decision BP strategies may also consider historical outcome data, e.g., agent-contact-action or agent-contact-offer pairings, to build BP models and apply BP strategies to "pair" contacts with agents and specific actions and / or specific sets of offers (throughout the specification, the noun and verb "pairing" and other forms such as "behavior pairing" may be used to describe triples and higher dimensional groupings).

[0026] In use, various BP strategies may cause the contact distribution system to change operation, thereby causing the contact distribution system to operate in a new manner. As will be described in more detail below, assigning contacts to agents according to a pairing strategy may cause technical changes in the operation of the contact distribution system (e.g., sending instructions to one or more switches within the contact distribution system to connect contacts to agents), which may cause the contact distribution system to operate in a new and improved manner.

[0027] Figure 1A A block diagram of a contact distribution center 100A according to an embodiment of the present disclosure is shown.

[0028] The description herein describes a network element, computer and / or component of a system and method for pairing strategy and natural language processing in a contact distribution system, which system and method may include one or more modules. As used herein, the term "module" may be understood to refer to computing software, firmware, hardware and / or various combinations thereof. However, a module should not be interpreted as software that is not implemented on hardware, firmware or recorded on a non-transitory processor-readable and recordable storage medium (i.e., the module itself is not software). It should be noted that modules are exemplary. Modules may be combined, integrated, separated and / or replicated to support various applications. In addition, instead of or in addition to the functions performed at a specific module, the functions performed at a specific module described herein may be performed at one or more other modules and / or by one or more other devices. In addition, modules may be implemented across multiple devices and / or other components that are local or remote to each other. In addition, a module may be moved from one device and added to another device, and / or may be included in two devices.

[0029] As shown in FIG. 1 , the contact distribution center 100A may include a central switch 110. The central switch 110 may receive incoming contacts (e.g., telephone calls, Internet chat sessions, emails, etc.) or support outgoing connections to contacts via a dialer, telecommunications network, or other module (not shown). The central switch 110 may include routing hardware and software to help route contacts among one or more sub-centers, or to one or more private branch exchanges (“PBXs”) or automatic call distribution (ACD) routing components or other queuing or switching components within the contact distribution center 100A. If there is only one sub-center, or if there is only one PBX or ACD routing component in the contact distribution center 100, the central switch 110 may not be necessary.

[0030] If more than one sub-center is part of the contact distribution center 100A, each sub-center may include at least one switch (e.g., switches 120A and 120B). Switches 120A and 120B may be communicatively coupled to the central switch 110. Each switch of each sub-center may be communicatively coupled to multiple (or "pools") of agents. Each switch may support a certain number of agents (or "seats") logged in at the same time. At any given time, a logged-in agent may be available and waiting to be connected to a contact, or a logged-in agent may be unavailable for a variety of reasons, such as connecting to another contact, performing certain post-call functions (such as logging in information about the call), or resting. In the example of FIG. 1 , the central switch 110 routes the contact to one of the two sub-centers via switches 120A and 120B, respectively. Each of switches 120A and 120B is shown with two agents. Agents 130A and 130B may be logged in to switch 120A, while agents 130C and 130D may be logged in to switch 120B.

[0031] Contact distribution center 100A may also be communicatively coupled to integration services from, for example, third-party vendors. In the example of FIG. 1 , behavior pairing module 140 may be communicatively coupled to one or more switches in a switch system of contact distribution center 100A, such as central switch 110, switch 120A, and switch 120B. In some embodiments, a switch of contact distribution center 100A may be communicatively coupled to multiple behavior pairing modules. In some embodiments, behavior pairing module 140 may be embedded within a component of contact distribution center 100A (e.g., embedded within a switch or otherwise integrated with a switch).

[0032] The behavior pairing module 140 may receive information about agents (e.g., agents 130A and 130B) logged into the switch from a switch (e.g., switch 120A) and information about incoming contacts via another switch (e.g., central switch 110), or in some embodiments, information received from a network (e.g., the Internet or a telecommunications network) (not shown). The behavior pairing module 140 may process this information to determine which agents should be paired (e.g., matched, assigned, distributed, routed) with which contacts, as well as other dimensions (e.g., offers, actions, channels, non-monetary rewards, monetary rewards or compensation, physical resources, agents for physical resources, etc.). Once it is determined which agents should be paired with which contacts, the behavior pairing module 140 may send one or more instructions to the relevant switches to connect the agents to the contacts, thereby causing a state change in the contact distribution center 100A. Thus, the pairing module 140 causes a direct change in the way the contact distribution center 100A operates.

[0033] For example, in the L1 state, multiple agents may be available and waiting to be connected to a contact, and the contact arrives at the contact distribution center 100A via the network or central switch 110. As described above, without the behavioral matching module 140, the switch would typically automatically distribute new contacts to the available agent who has been waiting the longest for an agent under the FIFO strategy, or the available agent determined to have the highest performance under the PBR strategy. In one example of the behavioral matching module 140, scores (e.g., percentiles or percentile ranges / bandwidths) can be assigned to contacts and agents based on a matching model or other artificial intelligence data model so that the contact can be matched, paired, or otherwise connected to a preferred agent. High-dimensional analysis of BP decisions will be explained in more detail below.

[0034] In the L2 state, multiple contacts are available and waiting to be connected to an agent, and an agent becomes available. These contacts can be queued in a switch (such as a PBX or ACD device). If there is no behavioral pairing module 140, the switch will typically connect the newly available agent to the contact that has been waiting in the queue the longest, such as in a FIFO strategy or a PBR strategy when the agent selects unavailable. In some contact distribution centers, priority queuing can also be combined, as described above. In an example of a behavioral pairing module 140 in this L2 scenario, as in the above-mentioned L1 state, percentiles (or percentile ranges / bandwidths, etc.) can be assigned to contacts and agents based on, for example, a model (such as an artificial intelligence model) so that the agents that become available are matched, paired, or otherwise connected to the preferred contacts. The high-dimensional analysis of BP decisions will be explained in more detail below.

[0035] Figure 1B A block diagram of a contact distribution system 100B according to an embodiment of the present disclosure is shown.

[0036] The contact distribution system 100B may be included in a contact distribution center (eg, contact distribution center 100A), or incorporated into a component or module of a contact distribution center (eg, behavior matching module 140), to help allocate agents for grouping among various contacts and other dimensions.

[0037] The contact distribution system 100B may include a contact distribution module 150 configured to pair (eg, match, distribute) incoming contacts to available agents. (High-dimensional analysis of BP decisions will be explained in more detail below.) Figure 1B In the example of , m contacts 160A-160m are received within a given time period, and n agents 170A-170n are available during the given time period. Each of the m contacts may be assigned to one of the n agents for service or other type of contact handling. Figure 1BIn the example of , m and n can be arbitrarily large finite integers greater than or equal to one. In a real-world contact distribution center (such as a contact center), there may be tens, hundreds, etc. of agents logged into the contact center to interact with contacts during a shift, and the contact center may receive tens, hundreds, thousands, etc. of contacts (e.g., phone calls, Internet chat sessions, emails, etc.) during a shift.

[0038] In some embodiments, the contact allocation strategy module 180 may be communicatively coupled to the contact allocation system 200 and / or configured to operate in the contact allocation system 200. The contact allocation strategy module 180 may implement one or more contact allocation strategies (or "pairing strategies") for assigning individual contacts to individual agents (e.g., pairing contacts with contact center agents). The contact allocation strategy module 180 may design and implement a variety of different contact allocation strategies. In some embodiments, a FIFO strategy may be implemented, where, for example, the agent with the longest waiting time receives the next available contact (in an L1 environment) or the contact with the longest waiting time is assigned to the next available agent (in an L2 environment). In other embodiments, a PBR strategy may be implemented to prioritize higher performing agents for contact allocation. For example, under PBR, the agent with the best performance among the available agents receives the next available contact. In other embodiments, a BP strategy may be used to optimally assign contacts to agents using information about contacts or agents or both. Various BP strategies may be used, such as a diagonal model BP strategy or a network flow ("off-diagonal") BP strategy. See U.S. Patent Nos. 9,300,802; 9,781,269; 9,787,841 and 9,930,180.

[0039] In some embodiments, when a contact is assigned to a particular agent, the contact assignment strategy module 180 may implement a decision BP strategy that considers the next best action for the contact. For a contact-agent pair, the decision BP strategy may also assign an action or set of actions available to the agent to complete the contact. In the context of a contact center system, an action or set of actions may include an offer or set of offers that the agent can present to the customer. For example, in a contact center system, a decision BP strategy may pair a contact with an agent and an action or set of offers (e.g., an offer or set of offers) that the agent can make to the customer based on the expected outcome of a contact-agent interaction using that particular action or set of actions. By influencing the choices or options among the actions available to the agent, the decision BP strategy goes beyond pairing a contact with an agent by optimizing the outcome of a single interaction between an agent and a contact.

[0040] For example, if agent 170A likes sports, then agent 170A may be more suitable for selling sports packages. Therefore, sports packages may be included in agent 170A's offers for some or all contact types. On the other hand, agent 170B may like movies and may be more suitable for selling premium movie packages; therefore, premium movie packages may be included in agent 170B's offers for some or all contact types. In addition, based on artificial intelligence processes (such as machine learning), decision BP models can automatically segment customers according to various variables and data types. For example, the decision BP model may suggest providing a package containing sports to a first type of customer ("customer type 1") who may be suitable for this particular type of contact. The decision BP model may suggest providing a premium movie package to a second type of customer ("customer type 2") who may be suitable for the contact type. The decision BP strategy may preferably pair customer type 1 with agent 170A and an offer set with a sports package, and pair customer type 2 with agent 170B and an offer set with a premium movie package. This results in a change in the state of the contact distribution system 100B by forming a connection between the contact and the agent based on the pairing determined by the decision BP policy. Thus, the change in pairing ultimately causes the contact distribution system 100B to operate in a different manner.

[0041] As with the previously disclosed BP strategies, the decision BP strategy optimizes the overall performance of the contact distribution system, rather than each individual instant contact-agent pairing. For example, in some embodiments, the decision BP system will not always offer sports activities to customer type 1, nor will it always give agent 170A an offer option based on a sports package. This may occur when the marketing department of the company running the contact center system may have a budget for a limited, restricted number of deals (e.g., a limited number of discounted sports packages), other restrictions on the frequency of certain offers, a limit on the total amount of discounts that can be made in a given time period (e.g., any discount or discount package), etc. Similarly, customer type 2 may sometimes be offered sports-based deals, and agent 170B may sometimes be given an offer option based on a sports package.

[0042] To optimize the overall performance of the contact allocation system, the decision BP strategy may take into account all types of customers waiting in line, agents available to customers, and any other dimensions used for pairing, such as the number and type of offers remaining, agent compensation or other non-monetary incentives, next best action, etc. In some embodiments, a probability distribution may be assigned based on the likelihood of an incoming contact or customer type to accept a given offer level based on the agent paired with the contact or customer.

[0043] For example, for contact type 1, if the discount offered is 0%, then contacts of contact type 1 have a 0% chance of accepting an offer from a regular agent, a 0% chance of accepting an offer from agent 170A specifically, and a 0% chance of accepting an offer from agent 170B specifically. For a 20% discount offer, contacts of contact type 1 may have a 30% chance of accepting an offer from a regular agent, a 60% chance of accepting an offer from agent 170A, and a 25% chance of accepting an offer from agent 170B. In a situation where a regular agent, agent 170A, and agent 170B are all assigned to a queue and are available, agent 170A may perform much better than either the regular agent or agent 170B.

[0044] In some embodiments, output measurements may be attached to each contact before and after interacting with an agent. For example, a revenue figure may be attached to each caller before and after the call. The decision BP system may measure changes in revenue and the impact of a contact based on an offer or set of offers presented by an agent. For example, contact type 1 is more likely to renew its existing plan regardless of the discount offered or regardless of the capabilities of a single agent. Contact type 2 may be preferentially assigned to a lower performing agent who has a higher discount cap in the offer set. In contrast, if contact type 2 is paired with a higher performing agent or an agent authorized to offer a larger discount, she may be more likely to upgrade her plan.

[0045] In some embodiments, the decision BP strategy can sequentially pair one or more dimensions in any order. For example, the decision BP strategy can first pair agents with contacts, then pair offer sets with agent-contact pairs, then pair rewards with agent-contact-offer set pairs, and so on.

[0046] In other embodiments, the decision BP strategy can perform "fully coupled" simultaneous multi-dimensional pairing. For example, the decision BP strategy can consider all dimensions simultaneously to select the best 4-D agent-contact-offer-reward pairing.

[0047] The same contact may arrive at the contact distribution system multiple times (e.g., the same caller calls the call center multiple times). In these "multi-touch" scenarios, in some embodiments, the contact distribution system may always assign the same items to one or more dimensions to promote consistency. For example, if a contact is paired with a particular action or set of actions when it first arrives, the contact will be paired with the same action or set of actions each subsequent time the contact arrives (e.g., for a given issue, within a given time period, etc.).

[0048] In some embodiments, the historical allocation module 190 may be communicatively coupled to the contact allocation system 100B via other modules (such as the contact allocation module 150 and / or the contact allocation policy module 180) and / or configured to operate in the contact allocation system 100B via other modules (such as the contact allocation module 150 and / or the contact allocation policy module 180). The historical allocation module 190 may be responsible for various functions, such as monitoring, storing, retrieving, and / or outputting information about contact agent allocations and high-dimensional allocations that have been made. For example, the historical allocation module 190 may monitor the contact allocation module 150 to collect information about contact allocations in a given period. Each record of the historical contact allocation may include information such as an agent identifier, a contact or contact type identifier, an action or action set identifier (e.g., an offer or offer set identifier), result information, or a pairing model identifier (i.e., an identifier indicating whether the contact allocation was made using a BP strategy, a decision BP strategy, or some other pairing model (such as a FIFO or PBR pairing model)).

[0049] In some embodiments and for certain contexts, additional information may be stored. For example, in a call center context, the history distribution module 190 may also store information about the call start time, call end time, the phone number dialed, and the caller's phone number.

[0050] In some embodiments, the historical assignment module 190 can generate a pairing model, a decision BP model, or a similar computer processor-generated model based on a set of historical assignments over a period of time (e.g., the past week, the past month, the past year, etc.), which the contact assignment strategy module 180 can use to make contact assignment recommendations or instructions to the contact assignment module 150.

[0051] In some embodiments, the historical allocation module 190 can analyze historical results data to create or determine new or different offer sets, which are then incorporated into the decision BP model, rather than relying on a predetermined set of actions to generate the decision BP model. This approach may be preferred when the number of specific action sets that can be performed is limited. For example, the marketing department may have limited the contact center system to five hundred discounted sports packages and five hundred discounted movie packages per month, and the company may want to optimize total revenue regardless of how many sports and movie packages are sold, with or without discounts. In this case, the decision BP model may be similar to the previously disclosed BP diagonal model, except that in addition to the "contact or contact percentile" (CP) dimension and the "agent percentile" (AP) dimension, there may be a third "revenue or offer set percentile" dimension. In addition, all three dimensions can be normalized or processed with mean regress (e.g., Bayesian mean regression (BMR) or hierarchical BMR).

[0052] In some embodiments, the history allocation module 190 can generate a decision BP model that optimizes the contact-agent-offer set pairing based on a single channel or multi-channel interactions. The history allocation module 190 can handle different channels in different ways. For example, the decision BP model can preferably pair contacts with different agents or action sets based on whether the contact calls a call center, initiates a chat session, sends an email or text message, enters a retail store, etc.

[0053] In some embodiments, the contact allocation strategy module 180 can proactively create contacts or other actions (e.g., recommend outgoing contact interactions, next best actions, etc.) based on information about the contact or customer, available agents, and available offer sets. For example, the contact allocation system 100B can determine that a customer's contract is about to expire, that the customer's usage is declining, etc. The contact allocation system 100B can also determine that the customer is unlikely to renew the contract at the customer's current rate (e.g., based on information from the historical allocation module 190). The contact allocation system 100B can determine that the next best action is to call the customer (contact selection, channel selection, and time selection), connect with a specific agent (agent selection), and have the agent choose to offer a downgrade at a specific discount or discount range (offer set selection). If the customer does not reach an agreement during the call, the contact allocation system 100B can further determine that the customer is more likely to accept the downgrade discount offer if the agent follows up with information about the discount and how to confirm via text message (multi-channel selection and optimization).

[0054] In some embodiments, similar to how Kappa ( ) parameter to adjust / bias the agent percentile or percentile range (see U.S. Patent No. 9,781,269) and how to use Rho ( ) parameters to adjust / bias the contacts or contact percentiles or percentile ranges (see U.S. Patent No. 9,787,841), the contact allocation strategy module 180 may use Iota ( ) parameters are applied to a third dimension (or higher), such as an action or action set percentile or percentile range in a decision BP strategy. Using the Iota parameter, the contact allocation strategy module 180 can, for example, adjust the action or action set percentile or percentile range (or other dimension) to bias the contact agent action pairing towards higher performance actions and unbalance the action set availability. The Iota parameter can be applied in an L1 or L2 environment and can be used in conjunction with the Kappa or Rho parameter, or can be applied together with the Kappa and Rho parameters in an L3 environment. For example, if the contact allocation strategy module 180 determines that the expected wait time for a contact has exceeded 100 seconds (high congestion), it can apply the Iota parameter so that the agent is more likely to provide more available relevant actions, which may be accepted or taken more quickly to reduce congestion and expected wait time. This can speed up contact-agent interactions, thereby reducing average handling time (AHT), and reducing overall memory and network bandwidth limitations of the contact center system. When congestion levels are low, expected wait times may be low, and Iota parameters may be adjusted to make only less generous offers.

[0055] In some embodiments, the contact allocation strategy module 180 can optimize performance by optimizing multiple goals or other objectives simultaneously. When goals compete with each other (e.g., discount amount and retention rate), the contact allocation strategy module 180 can balance the trade-offs between the two goals. For example, the contact allocation strategy module 180 can balance between increasing (or maintaining) revenue and maintaining or minimizing the reduction in retention rate, or can balance between reducing (or maintaining) AHT and increasing (or maintaining) customer satisfaction, etc.

[0056] In some embodiments, the contact allocation strategy module 180 may implement a decision BP strategy that considers agent compensation instead of offers or offer sets. This framework is similar to the above description, except that instead of influencing customers through offers or offer sets, the decision BP strategy influences the agent's performance through the compensation that the agent may receive. In other words, instead of performing a three-dimensional pairing of contact-agent-offer set, the decision BP strategy performs a three-dimensional pairing of contact-agent-reward. In some embodiments, the decision BP strategy may perform a four-way pairing of contact-agent-offer-reward.

[0057] Decision BP strategies can provide variable agent compensation based on contact-agent pairings, thereby improving transparency and fairness. For example, some contact distribution (e.g., contact center) systems may see a mix of more challenging and less challenging contacts and employ a mix of higher performing and lower performing agents. Under a FIFO strategy or a PBR strategy, agents of any ability are equally likely to be paired with more challenging contacts or less challenging contacts. Under a FIFO strategy, the overall performance of the contact center system may be low, but average agent compensation may be transparent and fair. Under a PBR strategy, agent utilization may be skewed, and compensation may also be skewed toward higher performing agents. Under the previously disclosed BP strategies, more challenging contact types may be preferentially paired with higher performing agents, while less challenging contact types may be preferentially paired with lower performing agents. For example, if high performing agents receive more difficult calls on average, this "call type skew" may result in lower conversion rates and lower compensation for high performing agents.

[0058] Therefore, adjusting the compensation of the agent up or down for a given contact (or contact) type may improve the fairness of compensation under the previously disclosed BP strategy. When an agent is paired with a contact or contacts, the decision BP strategy may inform the agent of the expected value of the contact to the contact center and / or how much commission or other non-monetary reward the agent will receive for handling the contact or achieving a specific result. The decision BP strategy may influence the behavior of the agent by providing variable compensation. For example, if the decision BP strategy determines that the agent should handle the contact quickly (lower AHT, lower income), the decision BP strategy may choose a lower compensation. Therefore, the agent may not be motivated to spend a lot of time to earn a relatively lower commission. On the contrary, if the decision BP strategy determines that the agent should devote more energy to higher value calls (higher AHT, higher income), it may choose a higher compensation. Such a decision BP strategy can maximize the agent's reward while improving the overall performance of the contact distribution system 100B.

[0059] The historical allocation module 190 can model a decision BP model based on historical information about contact type, agent, and compensation amount so that contact-agent-reward selection can be performed simultaneously. The amount of increase or decrease in compensation may be different and depends on each combination of a single agent and contact type, with the goal of improving the overall performance of the contact allocation system 100B.

[0060] Similar to applying Iota parameters to the offer set or next best action percentile or percentile range dimensions, and as described above, the contact distribution strategy module 180 can apply Iota parameters to other dimensions, such as deviating agent compensation to a greater or lesser degree, or to a generally higher or generally lower value. In some embodiments, the number and type of Iota parameters applied to agent compensation or other non-monetary incentives can be based at least in part on factors in the contact distribution system 100B (e.g., the expected wait time for callers held in the call center).

[0061] In some embodiments employing a BP strategy similar to the diagonal model, the variable compensation can be viewed as temporarily influencing the available agent percentile (AP) of the available agents to be higher or lower in order to bring the available contact-agent pairings closer to the optimal diagonal. Similarly, adjusting the offer value to be higher or lower can be viewed as influencing the available contact percentile (CP) of waiting contacts to be higher or lower in order to bring the available contact-agent pairings closer to the optimal diagonal.

[0062] In some embodiments, the benchmarking module 195 can be communicatively coupled to the contact distribution system 100B and / or configured to run in the contact distribution system 100B via other modules (such as the contact distribution module 150 and / or the historical distribution module 190). The benchmarking module 195 can benchmark the relative performance of two or more pairing strategies (e.g., FIFO, PBR, BP, decision BP, etc.) using historical distribution information (e.g., which can be received from the historical distribution module 190). In some embodiments, the benchmarking module 195 can perform other functions, such as establishing a benchmarking plan to cycle through various pairing strategies, tracking groups (e.g., historical distribution base and measurement groups), etc. For example, benchmarking for contact center contexts is described in detail in U.S. Patent No. 9,712,676 (which is hereby incorporated by reference herein).

[0063] In some embodiments, the benchmarking module 195 may output or otherwise report or use relative performance measures. The relative performance measures may be used to evaluate the quality of a contact allocation strategy to determine, for example, whether a different contact allocation strategy (or a different pairing model) should be used, or to measure the overall performance (or performance gain) achieved within the contact allocation system 100B while the contact allocation system 100B is optimized or otherwise configured to use one contact allocation strategy instead of another.

[0064] In some embodiments, the benchmarking module 195 may benchmark the decision BP strategy against one or more alternative pairing strategies, such as FIFO combined with offer set availability. For example, in a contact center system, agents may have a matrix of nine offers - three tiers of service levels, each with three discount levels. During "off" calls, the agent who has been waiting the longest may be connected to the caller who has been waiting the longest, and the agent may offer any of the nine offers. A high-performing agent may be more likely to sell a higher tier of service at a higher price, while a low-performing agent may not try as hard and offer the biggest discount right away. During "on" calls, the decision BP strategy may pair a contact with an agent, but restrict the agent to a subset of the nine available offers. For example, for some contact types, if the contact allocation strategy module 180 determines based on the decision BP model that the overall performance of the contact center system can be optimized by selectively restricting the offer set in a given way for a given contact-agent pairing, then a higher-performing agent may be entitled to make any of the nine offers, while a lower-performing agent may only be able to offer smaller discounts at certain tiers. Furthermore, if a provider (eg, a vendor) that provides a contact allocation system with a decision BP strategy uses a benchmarking and revenue sharing business model, the provider may contribute a portion of the revenue gain from benchmarking to the agent compensation pool.

[0065] In some embodiments, the contact distribution system 100B may provide dashboards, visualizations, or other analytics and interfaces to improve the overall performance of the system. For each agent, the analytics provided may vary depending on the relative abilities or behavioral characteristics of the agent. For example, competitive or higher performing agents may benefit from ranking widgets or other "gamification" elements (e.g., badges or achievements for unlocking points and scoreboards, notifications when agents surpass each other in the rankings, etc.). On the other hand, less competitive or lower performing agents may benefit from periodic messages of encouragement, recommendations for training / educational courses, etc.

[0066] Figure 2A A block diagram of an example contact distribution system 200A is shown in accordance with an embodiment of the present disclosure.

[0067] like Figure 2AAs shown, the contact distribution system 200A may include one or more agent endpoints 211A, 211B and one or more contact endpoints 212A, 212B. The agent endpoints 211A, 211B may include an agent terminal and / or an agent computing device (e.g., a laptop computer, a mobile phone). The contact endpoints 212A, 212B may include a contact terminal and / or a contact computing device (e.g., a laptop computer, a mobile phone). Depending on the capabilities of the endpoint devices, the agent endpoints 211A, 211B and / or the contact endpoints 212A, 212B may be connected to a contact center as a service (CCaaS) 230 via the Internet or a public switched telephone network (PSTN).

[0068] Figure 2B An example contact distribution system 200B with an example configuration of a CCaaS 230 is shown. For example, the CCaaS 230 may include multiple data centers 240A, 240B. The data centers 240A, 240B may be physically separated and may even be located in different countries and / or continents. The data centers 240A, 240B may communicate with each other. For example, one data center is a backup for another data center; thus, in some embodiments, only one data center 240A or 240B receives agent endpoints 211A, 211B and contact endpoints 212A, 212B at a time.

[0069] Each data center 240A, 240B includes a web demilitarized zone device 231A and 231B, respectively, which is configured to receive the agent endpoints 211A, 211B and the contact endpoints 212A, 212B, which are communicatively connected to the CCaaS 230 via the Internet. The web demilitarized zone (DMZ) devices 231A and 231B can be operated outside the firewall to connect with the agent endpoints 211A, 211B and the contact endpoints 212A, 212B, while the remaining components of the data center 240A, 240B can be inside the firewall (except for the phone DMZ devices 232A, 232B, which can also be outside the firewall). Similarly, each data center 240A, 240B includes a phone DMZ device 232A and 232B, respectively, which is configured to receive the agent endpoints 211A, 211B and the contact endpoints 212A, 212B, which are communicatively connected to the CCaaS 230 via the PSTN. The telephony DMZ devices 232A and 232B may operate outside a firewall to connect the agent endpoints 211A, 211B and the contact endpoints 212A, 212B, while the remaining components of the data center 240A, 240B (excluding the Web DMZ devices 231A, 231B) may be inside the firewall.

[0070] In addition, each data center 240A, 240B can include one or more nodes 233A, 233B and 233C, 233D, respectively. All nodes 233A, 233B and 233C, 233D can communicate with web DMZ devices 231A and 231B, respectively, and with phone DMZ devices 232A and 232B, respectively. In some embodiments, only one node in each data center 240A, 240B can communicate with web DMZ devices 231A, 231B and phone DMZ devices 232A, 232B at a time.

[0071] Each node 233A, 233B, 233C, 233D may have one or more pairing modules 234A, 234B, 234C, 234D, respectively. Figure 1A Similar to the pairing module 140 of the contact distribution center 100A of FIG. 1 , the pairing modules 234A, 234B, 234C, 234D can pair contacts with agents. For example, the pairing modules can alternate between enabling pairing via a behavioral pairing (BP) module and enabling pairing using a first-in, first-out (FIFO) module. In other embodiments, a pairing module (e.g., a BP module) can be configured to emulate other pairing strategies.

[0072] When pairing an agent endpoint with a contact endpoint, the pairing module (e.g., pairing module 234A, 234B, 234C, 234D) causes a change in the technical state of the contact distribution system 200B. The pairing strategy or pairing model used by the pairing module can improve the technical operation of the contact distribution system 200B by utilizing the physical resources of the contact distribution system 200B in a more efficient manner.

[0073] Now go to Figure 2C The disclosed CCaaS communication system (eg, FIG. 2A and / or 2B) can support multi-tenancy, so that multiple contact centers (or contact center operations or businesses) can operate in a shared environment. That is, each tenant can have a separate, non-overlapping agent pool. Figure 2C The CCaaS 230 shown in FIG. includes two tenants 250A and 250B. Return to Figure 2B For example, multiple tenants may be supported by node 233A supporting tenant 250A and node 233B supporting 250B. In another embodiment, data center 240A supports tenant 250A and data center 240B supports tenant 250B. In another example, multiple tenants may be supported by shared machines or shared virtual machines; for example, node 233A may support both tenants 250A and 250B, and nodes 233B, 233C, and 233D may also support similarly.

[0074] In other embodiments, the system may be configured for a single tenant within a dedicated environment, such as a private machine or private virtual machine.

[0075] Figure 3 A pairing model 300 is shown according to one aspect of the present disclosure.

[0076] The pairing model 300 includes a plurality of elements or values, which may be arranged according to an agent dimension 302 (or an agent axis, an agent percentile dimension / axis), a contact type dimension 304 (or a contact type axis, a contact dimension / axis, or a contact percentile dimension / axis), and an action probability dimension 306 (or an action probability axis, an action dimension / axis, or an action percentile dimension / axis). Figure 3 It also shows the first seat (e.g. Figure 1B 1 , a first position 308 of the agent dimension associated with the agent 170A in FIG. 1 , a second position 310 of the contact type dimension associated with the first contact type (e.g., “Customer Type 1”), and a third position 312 of the action probability dimension associated with the action probability value. The third position 312 of the action probability dimension is displayed as part of a plurality of possible action probability values ​​314.

[0077] The number of elements on the agent dimension 302 of the pairing model 300 can correspond to the number of agents associated with the contact center. For example, the size n of the agent dimension 302 can correspond to the n agents available during a given period (such as Figure 1B In some embodiments, the agent dimension 302 of the pairing model 300 can be associated with a set of unique agent identifiers.

[0078] The number of elements on the contact type dimension 304 of the pairing model 300 may correspond to the number of contact types associated with the contact center. For example, the size l of the agent dimension 302 corresponds to l contact types associated with the contact center. As described above, the incoming contacts may be assigned contact types based on an artificial intelligence process so that the incoming contacts are paired according to the assigned contact types. In some embodiments, the contact type dimension 304 of the pairing model 300 may be associated with a set of unique contact types, or may be associated with a set of unique contact identifiers.

[0079] The number of elements on the action probability dimension 306 can correspond in size to the number of actions that an agent can take within the contact center. Thus, the action probability dimension 306 can also be alternatively referred to as an action dimension or axis, a next action dimension or axis, or an offer dimension or axis. In some embodiments, the action probability dimension 306 of the pairing model 300 can be associated with a set of unique action identifiers.

[0080] A particular agent-contact type-action triple results in a particular action probability value within the pairing model 300. Figure 3 In the example shown, an action probability value is displayed at a third position 312 within a plurality of possible action probability values ​​314. All of the action probability values ​​within the plurality of possible action probability values ​​314 are associated with a first agent (associated with the first position 308 of the agent dimension) and a first contact type (associated with the second position 310 of the contact type dimension). In this example, the action probability value displayed at the third position 312 corresponds to the most likely value within the plurality of possible action probability values ​​314. Therefore, if the agent associated with the first position 308 is paired with a contact of the contact type associated with the second position 310, the action probability value displayed at the third position 312 can be understood as the most likely action to be taken. Therefore, the pairing model 300 can be used to pair decision behaviors by identifying an action or set of actions that an agent can use to complete a contact. By influencing the selection or options among the available actions for an agent, the decision BP strategy goes beyond pairing a contact with an agent by optimizing the outcome of a single interaction between the agent and the contact.

[0081] Traditional methods of learning actions or offers and determining which offers or actions should be suggested focus only on actions that have been accepted (i.e., with positive outcomes). Learning which actions or offers should be taken only from those actions or proposals that have been accepted may result in a biased or inaccurate representation of the likelihood that an action or offer will be accepted. Therefore, traditional techniques for determining actions or offers are insufficient because they only use data corresponding to offers that have been accepted. The present disclosure newly proposes a system and method for determining actions and offers based on offers that were rejected by a contact during a contact-agent interaction. According to one aspect of the present disclosure, elements within a pairing model associated with agents, contact types, and action probabilities are learned from call transcript data using a natural language processing pipeline.

[0082] The natural language processing method of the present disclosure is particularly effective because it can form elements of a pairing model from exposed or hidden actions within a transcript. As will be described in more detail below, exposed actions are those actions that are known because they have been recorded with the outcome of a particular interaction (e.g., an action was provided to a contact by an agent and the contact subsequently accepted the action, causing the action to be recorded as a result of the interaction). In contrast, hidden or latent actions are those actions that occurred within the interaction but are unknown because they have not yet been recorded (e.g., because the actions are not associated with a positive outcome (such as an accepted action)). These hidden or latent actions are inaccessible to traditional pairing strategies. Forming a pairing model from exposed and hidden actions results in a more accurate determination of the probability of taking a given action, resulting in an improved pairing model and improved operations at the contact center.

[0083] Figure 4 An example transcript of a portion of a set of transcripts from which a pairing model may be formed according to an embodiment of the present disclosure is shown.

[0084] Plurality of transcripts 402 includes a first subset of transcripts 404, a second subset of transcripts 406, and a third subset of transcripts 408. Figure 4 Also shown is an example transcript 410 that includes a first substring 412 and an associated related action-result pair 414 that includes a related action 416 and a result 418. The example transcript 410 also includes a second substring 420 associated with a hidden related action 422.

[0085] The first subset 404 of transcripts includes transcripts that are known to be related to related actions because they have associated related action-result pairs. Here, related actions should be understood as actions in the set of all possible actions that are of particular interest to a given task. The second subset 406 of transcripts includes transcripts that are related to related actions, but this relationship is unknown (i.e., because each transcript in the second subset 406 of transcripts does not have an associated related action-result pair). The third subset 408 of transcripts includes transcripts that are related to unrelated actions. The transcripts within the third subset of transcripts may still have associated action-result pairs, but the actions in the action-result pairs are not related actions.

[0086] A transcript within the first subset of transcripts 404 is illustrated by an example transcript 410 including an associated related action-result pair 414. The associated related action-result pair 414 includes a related action 416 to which the example transcript 410 is known to be related and a result 418 of the related action 416. For example, the related action may correspond to a product (e.g., "Samsung A20") that an agent offers to a contact during a call, while the result may correspond to the contact accepting the product offer. Since the result of the call is that the contact accepts the product offer, it is known that the product was offered (i.e., the related action). Thus, for a call transcript with a known result, at least one of the actions associated with the call transcript is also known.

[0087] Although it may be known that the call transcript is associated with an action, the specific text string within the plain text representation of the call transcript that is associated with the action may not be known. Figure 4 In the example shown, the first substring 412 of the example transcript 410 is associated with the related action 416 in the related action-result pair 414, but the first substring 412 may not be directly related to the text of the related action 416, so the exact form of the first substring 412 may not be known. This may be due to several factors.

[0088] First, the plain text transcript is generated with the aid of an automatic speech-to-text conversion method known in the art. While this method is able to transcribe most calls with high accuracy, the generated transcript is still noisy. This noise can occur for a number of reasons, including artifacts or noise in the original call audio (e.g., unstable / noisy connection due to low microphone quality or limited cellular connection), the accent of the agent or contact causing the automatic speech-to-text conversion algorithm to misrecognize words, and general transcription errors introduced by the automatic speech-to-text conversion algorithm (e.g., due to the use of technical or obscure words outside the algorithm's corpus). These noise sources may cause the speech-to-text conversion method to mistranscribe certain words or phrases that are directly related to the action, making the direct association between the action and the related substring difficult. In fact, because the action or offer is often technical or obscure words, the transcript is particularly noisy for the intended use of determining the correlation between the action and the related substring. For example, the plain text representation of the action corresponding to the product "Samsung A20" may appear in the form of "Samsung a twenty", "Samsung a twenty", "Sam champion twin tea", etc. Therefore, unless all possible permutations of the term's transcripts are known in advance, a simple string matching approach that attempts to associate substrings with actions is insufficient to identify the cross-referenced product "Samsung A20".

[0089] Secondly, due to language differences, it may not be possible to know the direct association between a substring (such as the first substring 412) and an action (such as the related action 416). These language differences may be exacerbated by the speech-to-text conversion process, particularly when the call is maintained in one language (e.g., French) but references certain words in another language (e.g., English). As a result, the call may include discussions of various actions that are mistranslated within the call transcript, making the direct association between the related substring and the action difficult. For example, the text-to-speech process may know that the call being transcribed is in French, and therefore will attempt to identify French words within the audio. This may result in various nouns or proper nouns referring to specific actions (e.g., product names or offers) being incorrectly translated into French (e.g., the proper noun "Samsung" is transcribed as the French word "cinq cent").

[0090] In addition, the call transcript may contain further actions that are not relevant to the result but are still important for recognition. Figure 5As described in more detail, identifying these actions allows for improved pairings to be formed, which in turn results in improved functionality and operation of the contact center system. Since these actions are not associated with a result, they are unknown and may be referred to as hidden or latent actions. This is illustrated in the example transcript 410 that includes a second substring 420 associated with a hidden related action 422. Since the result of this action or the exact form of the second substring 420 is not known, the hidden related action 422 is not known.

[0091] While the plurality of transcripts 402 includes a first subset 404 of tagged transcripts (i.e., each transcript within the first subset 404 of transcripts has a known relevant action and an associated result), a second subset 406 of transcripts includes transcripts associated with hidden relevant actions. Identifying the second subset 406 of transcripts and using them to form a pairing model will help provide an improved pairing model.

[0092] Therefore, identifying exposed and hidden actions from the first subset of transcripts 404 and the second subset of transcripts 406 helps improve the formation of the pairing model, thereby improving the performance of the contact center system. Figure 5 are described in the table shown.

[0093] Figure 5 The determination of action probabilities using two different methods is illustrated.

[0094] Figure 5 A first table 502 associated with a first method for determining action probabilities and a second table 504 associated with a second method for determining action probabilities are shown. Both tables show the probability of an action being accepted for a particular agent and contact pairing. The action recommended or taken for a particular pairing is selected from a set of three possible actions. For example, the tables may be associated with a unique agent identifier and a contact type identifier, or the tables may be associated with a unique agent identifier and a contact identifier.

[0095] The first table 502 shows the action probabilities determined based only on so-called exposed actions, i.e., actions that were identified from the transcript data because they have known outcomes (such as Figure 44. The second column of the first table 502 shows the number of times different actions were provided by the agent within the transcript data and were accepted or taken by the contact. Thus, the second column of the first table 502 lists the number of times the action and the associated result appeared within the transcript data. In one example of determining the probability of an action being accepted or taken, each value within the second column of the first table 502 is divided by the sum of the column. Using this approach, action 3 is determined to be the most likely action. The present disclosure contemplates that there may be other methods to determine the probability of an action being accepted, however, no conventional technique is capable of examining the probability of an action based on an exposed or hidden action.

[0096] The second table 504 shows the action probabilities determined according to the exposed and hidden actions described in the present disclosure. As described above, hidden actions are actions that appear in the transcript data but are not associated with the results. Therefore, their presence in the transcript data cannot be identified only from existing metadata (e.g., from tag information attached to the transcript). Hidden actions may correspond to actions that have been provided but not accepted or taken. The second column of the second table 504 shows the number of times each action in the transcript data is provided but not taken. In order to determine the probability of an action being accepted or taken, the ratio of the number of times the action is provided and accepted to the total number of times the action is provided (i.e., the number of times the action is provided and rejected plus the number of times the action is provided and accepted) is calculated. Using this method, although action 3 has been accepted 10 times, it has been provided 30 times. Therefore, the acceptance rate of action 3 is about 33%. In contrast, action 1 was accepted 5 times out of 10 times, so the acceptance rate is 50%.

[0097] Therefore, using hidden or latent actions as well as exposed actions to determine the probability of taking an action can determine a more accurate acceptance rate or action probability. This in turn can lead to an improved and more efficient pairing model that can improve the operation of the contact center system by avoiding inefficient pairings, thereby reducing AHT, memory and bandwidth limitations, and inefficient resource allocation of agents, contacts, and offers / actions. Therefore, improving the pairing model used by the contact allocation system can directly improve the actions and behavior of the contact allocation system. For the above Figure 3 Improved identification of the probability of taking an action can lead to better pairings between agents and contacts, as well as improved identification of the most appropriate action to offer in an interaction. This may help reduce the time spent on a single interaction, thereby improving AHT and / or may help provide efficient resource allocation of agents, contacts, and offers / actions; all of which enable the contact center system to handle more interactions more efficiently.

[0098] Figure 6A transcript labeling pipeline 600 is shown for determining labels that can be used to form a pairing model. The pairing model can be used for decision action pairing in a contact center system as described above.

[0099] The transcript marking pipeline 600 includes a feature extraction process 602, a coarse-grained classifier 604, and a fine-grained classifier 606. In some embodiments, the transcript marking pipeline 600 also includes a model formation step 608.

[0100] In general, the transcript labeling pipeline 600 receives as input a plurality of transcripts 610 and provides as output one or more related transcripts 612 and one or more label vectors 614 corresponding to the one or more related transcripts 612. In some embodiments, the model formation step 608 receives the one or more related transcripts 612 and the one or more label vectors 614 and generates a pairing model 616.

[0101] The plurality of transcripts 610 may be obtained from historical transcripts recorded at the contact center system (e.g., recorded and stored by the historical assignment module 190). The plurality of transcripts 610 may indicate the agents, contacts, and / or contact types involved in each interaction. The plurality of transcripts 610 may be generated by processing the audio recordings of the interactions using speech-to-text processing algorithms known in the art.

[0102] The transcript labeling pipeline 600 employs a two-step classification process in order to identify the one or more relevant transcripts 612 and the one or more label vectors 614. A coarse-grained classifier 604 is used to filter the one or more relevant transcripts 612 and the one or more irrelevant transcripts 618 from the plurality of transcripts 610, without regard to which specific actions are present in the one or more relevant transcripts 612. A fine-grained classifier 606 assigns a label vector to each transcript within the one or more relevant transcripts 612 to form one or more label vectors 614. The label vectors indicate which relevant actions are associated with the corresponding transcript.

[0103] Using a coarse-grained classifier to filter relevant transcripts avoids training a fine-grained classifier on all possible actions, which can degrade prediction performance. By focusing the fine-grained classifier on a smaller subset of (relevant) labels, the prediction performance of the fine-grained classifier can be improved. Furthermore, training a fine-grained classifier on a smaller subset of possible actions results in a more efficient model, which can be trained faster and more efficiently.

[0104] The feature extraction process 602 extracts natural language features used by the coarse-grained classifier 604 and the fine-grained classifier 606. Figure 7Described in more detail, feature extraction process 602 receives a transcript (i.e., a plain text representation of a call) and outputs a natural language feature vector representing the transcript. In some embodiments, multiple transcripts 610 are preprocessed before using feature extraction process 602. Preprocessing includes performing one or more preprocessing operations, which include text truncation, data enhancement, data cleaning, and spell checking. Feature extraction process 602 extracts features of different representation levels from the plain text representation. For example, feature extraction process 602 can extract character-level features, word-level features, different context-level features that capture the context of words in the transcript, and transcript-level features. The different representation level features are then combined into a single natural language feature vector or vectorized representation. Feature extraction process 602 is configurable so that the contribution of different representation levels to the natural language feature vector can be adjusted.

[0105] The present disclosure points out that traditional methods for natural language processing applied to transcript data make extensive use of transformer-based models. However, training a transformer-based model from scratch requires both a lot of computation and a lot of resources. To solve this problem, traditional methods can use transfer learning methods (e.g., fine-tuning a pre-trained transformer model), which allows a base model (such as a pre-trained transformer) to be adapted to the specific problem at hand without training the transformer from scratch. Although the use of transfer learning and base models can help reduce the computational and resource burden required to generate trained models, they still have some disadvantages. For example, transformer models may have difficulty recognizing rare and / or unknown words. In addition, transformer models tend to perform well when the class size is quite small (e.g., binary), but transformer models tend to perform poorly when the class size is large (e.g., 100 or more classes). This may limit the effectiveness of transformer models in many cases.

[0106] In contrast, the feature extraction process 602 of the present disclosure includes a set of feature extractors that help address the above-mentioned limitations of pure transformer-based methods. The character-level and word-level feature extractors used by the feature extraction process 602 are independent of vocabulary and language (i.e., unlike pre-trained transformer models, they are not limited to a single vocabulary), and provide outputs with improved discriminative power by penalizing high-frequency words within the transcript. In addition, the feature extraction process 602 is better adapted to the complexity of the speech-to-text system that generates the transcript data. For example, the speech-to-text system tends to repeat erroneous transcripts across transcripts, resulting in consistency with misspelled words. The feature extraction process 602 can be adapted to this consistency because it does not rely on a model trained on documents that do not exhibit such erroneous transcripts. Therefore, the feature extraction process 602 of the present disclosure can provide better improvements than transformer-based methods when processing noisy transcript data.

[0107] A coarse-grained classifier 604 (which may alternatively be referred to as a relevance classifier, a binary classifier, a filtering classifier, or a filter) determines whether a transcript is related to a relevant action or an irrelevant action. More specifically, the coarse-grained classifier 604 includes a binary classifier that receives a natural language feature vector associated with the transcript and assigns a score to the feature vector indicating whether the transcript is related to one or more relevant actions. Because there is a direct association between a transcript and a natural language feature vector extracted from the transcript, references to transcripts associated with the operation of the transcript marking pipeline 600 hereinafter will be understood by those skilled in the art to be references to natural language feature vectors associated with the transcript. In one embodiment, the score generated by the coarse-grained classifier 604 is the probability that the transcript contains one or more substrings associated with one or more relevant actions. Alternatively, the score is a binary value that takes the value "1" if the transcript is considered relevant (i.e., it contains one or more substrings associated with one or more relevant actions) and takes the value "0" otherwise.

[0108] In one embodiment, the coarse-grained classifier 604 includes a trained binary extreme gradient boosting (XGBoost) model. Advantageously, the use of the XGBoost model can help provide a more powerful classifier in terms of label size (i.e., the number of related actions) than the classification layer of the transformer-based model. However, the skilled person will recognize that other binary classification models can be used, such as support vector machines (SVMs), decision trees, random forests, artificial neural networks, etc.

[0109] If the coarse-grained classifier 604 identifies the transcript as an irrelevant transcript, e.g., because the score generated by the coarse-grained classifier 604 is below a predetermined threshold, the transcript is not passed to the fine-grained classifier 606 and forms part of one or more irrelevant transcripts 618. In one embodiment, the transcript is assigned a null label so that it is not associated with any relevant label. Conversely, if the coarse-grained classifier 604 identifies the transcript as a relevant transcript, e.g., because the score generated by the coarse-grained classifier 604 is equal to or above a predetermined threshold, the transcript is passed to the fine-grained classifier 606.

[0110] The fine-grained classifier 606 (which may alternatively be referred to as an action classifier, a multi-label classifier, or an assignment classifier) ​​determines a label vector for the relevant transcript, wherein the label vector identifies which relevant actions are associated with the relevant transcript (i.e., which relevant actions are mentioned in the plain text transcript of the relevant transcript). More specifically, the fine-grained classifier 606 includes a multi-class classifier that receives a natural language feature vector associated with the relevant transcript and assigns a score to the relevant transcript for each of a plurality of relevant actions. All assigned scores form a label vector for the relevant transcript. Thus, even if the relevant action is not associated with the result (i.e., it is a hidden action), the fine-grained classifier 606 identifies the relevant action within the transcript. In some embodiments, the label vector may also indicate whether the action is known or hidden. For example, the label vector may include an action-result pair for each relevant action, wherein a value (1, 0) indicates that the relevant action is present in the transcript but is hidden, and a value (1, 1) indicates that the relevant action is present in the transcript but is known (i.e., it is associated with the result).

[0111] In one embodiment, the fine-grained classifier 606 includes a trained multi-class XGBoost model. Those skilled in the art will appreciate that other multi-class classification models may be used, such as support vector machines, decision trees, random forests, artificial neural networks, and the like.

[0112] In one embodiment, the label vector generated by the fine-grained classifier 606 includes multiple probabilities associated with multiple related actions. For example, for three related actions {"Samsung A20", "Nokia 3210", "Microsoft Surface Duo"}, the label vector [0.9, 0.1, 0.25] of the transcript will indicate that the transcript mentions Samsung A20 with a probability of 0.9, the transcript mentions Nokia 3210 with a probability of 0.1, and the transcript mentions Microsoft Surface Duo with a probability of 0.25.

[0113] In one embodiment, a thresholding process is used to convert the probabilities in the label vector into binary values. For example, if the transcripts are reviewed by a human and manually labeled for each contact-agent interaction, then determining the "true value" action corresponding to each transcript would be a time-consuming and resource-intensive operation. Therefore, the present disclosure contemplates a thresholding process (described below in conjunction with Figure 7 ), the thresholding process assigns a value of "1" to all probabilities that exceed or reach the threshold and a value of "0" to all other probabilities. Manually labeled data can be used in the disclosed transcript labeling pipeline 600; however, because the use of such manually labeled data may result in low accuracy of the final labeled transcript, the present disclosure contemplates separating any manually labeled data into a first set for use in the thresholding process (e.g., as described below in process 720), and separating the remainder into a second set for use in training a model (e.g., as discussed below with respect to semi-supervised learning).

[0114] The transcript labeling process (involving the feature extraction process 602, the coarse-grained classifier 604, and the fine-grained classifier 606) outputs one or more relevant transcripts 612 identified by the coarse-grained classifier 604 and one or more label vectors 614 identified by the fine-grained classifier. The one or more relevant transcripts 612 and the one or more label vectors 614 can then be used within the decision-making process of the contact center system.

[0115] In one embodiment, one or more related transcripts 612 and one or more label vectors 614 are used within a decision-making process of a contact center system by forming a pairing model 616 using the model forming step 608. The pairing model 616 can then be used within a decision-making process of a contact center system as described above.

[0116] The model formation step 608 receives one or more relevant transcripts 612 and one or more label vectors 614 and outputs a pairing model 616. In one embodiment, the model formation step 608 generates a new pairing model based on the provided transcripts and labels. In an alternative embodiment, the model formation step 608 uses the provided transcripts and labels to update elements within an existing pairing model.

[0117] Elements within the pairing model 616 may be determined by combining agent information (e.g., agent identity), contact or contact type information (e.g., contact type identity), and action information (e.g., probability of taking an associated action) from the transcript. Each transcript within the one or more associated transcripts 612 is associated with an agent and a contact. This association may accompany the transcript in the form of metadata. The metadata may include information such as a unique identifier for an agent involved in the call associated with the transcript, a unique contact identifier for the contact involved in the call associated with the transcript, and a contact type indicating a classification or grouping of contact associations. The agent and contact information contained in the metadata is used in conjunction with the one or more label vectors 614 to determine elements within the pairing model 616.

[0118] Specifically, for a pairing model that includes a first dimension corresponding to agents, a second dimension corresponding to contact types, and a third dimension corresponding to the probability of taking an action, an element within the pairing model may represent the probability that a particular contact-agent pair will result in a particular action being taken (as described above with respect to Figure 3 Deterministically, the probability is determined based on both exposed actions (i.e., actions with known outcomes) and hidden actions. The probability that an interaction between the first agent and the first contact type results in the first action being taken is calculated by identifying all relevant transcripts involving the first agent and the first contact type and determining the number of times the first action was taken and the total number of times the action was offered (including the number of times the first action was taken and the number of times the first action was rejected). The probability is then the ratio of the number of times the first action was taken to the total number of times the action was offered. As described above with respect to Figure 4 As described, an improved pairing model is provided by utilizing the exposed and hidden actions present in the transcript data.

[0119] Figure 7 It is shown that the training Figure 6 The training process 700 of the transcript labeling pipeline is shown.

[0120] The training process 700 generates a transcript marking pipeline 702 from a plurality of transcripts 704. The plurality of transcripts 704 includes a first subset 704-A corresponding to relevant transcripts (i.e., transcripts known to be relevant to relevant actions because they have associated relevant action-result pairs) and a second subset 704-B corresponding to all other transcripts. The training process 700 includes a feature extraction process 706, a binary classifier training process 708 that generates a coarse-grained classifier 710, and a multi-class classifier training process 712 that generates a fine-grained classifier 714. The feature extraction process includes a character-level feature extraction model 706-A, a word-level feature extraction model 706-B, a first context-level feature extraction model 706-C, a second context-level feature extraction model 706-D, and a third context-level feature extraction model 706-E. The transcript marking pipeline 702 is formed by the feature extraction process 706, the coarse-grained classifier 710, and the fine-grained classifier 714. In one embodiment, the training process 700 includes a pre-processing step 716 and / or a threshold selection process 718. The threshold selection process 718 produces a threshold operation 720 that can be included in the transcript marking pipeline 702.

[0121] In general, the training process 700 generates a transcript labeling pipeline 702 from a plurality of transcripts 704 by extracting a plurality of natural language feature vectors using a feature extraction process 706. The plurality of natural language feature vectors are then used in a binary classifier training process 708 to generate a coarse-grained classifier 710 and a fine-grained classifier 714 in a multi-class classifier training process 712. The feature extraction process 706, the coarse-grained classifier 710, and the fine-grained classifier 714 are then used to form the transcript labeling pipeline 702.

[0122] The plurality of transcripts 704 may be obtained from historical transcripts recorded at the contact center system over a period of time (e.g., recorded and stored by the historical assignment module 190). The plurality of transcripts 704 may indicate the agents, contacts, and / or contact types involved in each interaction. The plurality of transcripts 704 may be generated by processing the audio recordings of the interactions by a speech-to-text processing algorithm known in the art.

[0123] The feature extraction process 706 is used to extract multiple natural language feature vectors from multiple transcripts 704. Specifically, the feature extraction process 706 generates a vectorized representation of the call transcript. In one embodiment, the multiple transcripts 704 are preprocessed using a preprocessing step 716 before the feature extraction process 706. The preprocessing step 716 applies one or more preprocessing actions, which include text truncation, data enhancement, data cleaning, and spell checking. The preprocessing action can be language-specific, so that the preprocessing operation performed depends on the language of the transcript. For example, if the transcript is in French, a French spelling check operation is performed, and if the transcript is in English, an English spelling check operation is performed.

[0124] The feature extraction process 706 applies a multi-level feature extractor including multiple feature extraction models to the plain text representation of the transcript to generate corresponding multiple intermediate natural language feature vectors. The multi-level feature extractor is particularly effective in identifying and combining different language features within the plain text transcript, resulting in an improved natural language representation of the plain text transcript. Each of the feature extraction models extracts natural language features of different representation levels from the transcript. Therefore, the feature extraction process 706 extracts character-level features from the plain text representation using a character-level feature extraction model 706-A, extracts word-level features from the plain text representation using a word-level feature extraction model 706-B, and extracts context-based features from the plain text representation using a first context-level feature extraction model 706-C, a second context-level feature extraction model 706D, and a third context-level feature extraction model 706-E. The feature extraction process 706 can also extract transcript-based or document-based features from the plain text representation using a transcript-level feature extraction model (not shown). For example, the transcript-level representation can be determined using an average value (e.g., mean) of context embeddings. Additionally, character- and word-level TF-IDF scores give transcript-level scores (even across multiple transcripts).

[0125] Beneficially, the multi-level feature extractor produces a hierarchy of representations of a given transcript. This allows different levels of features to be extracted from the text, thereby improving the performance of classifiers that use these features. Furthermore, by linking different feature extractors of the multi-level feature extractor to different representation levels (e.g., characters, words, context, etc.), the configuration of the multi-level feature extractor is conceptually simpler, allowing the development of heuristics to determine which configurations to use in various implementation settings.

[0126] In one embodiment, the character-level feature extraction model 706-A includes a term frequency-inverse document frequency (TF-IDF) model of character n-grams. Advantageously, the use of character n-grams helps capture accents and spelling errors, thereby improving the quality of features extracted by the feature extraction process 706. In an example embodiment, unigrams, bigrams, and trigrams are extracted, and the TF-IDF model is limited to 70,000 features.

[0127] In one embodiment, the word-level feature extraction model 706-B includes a TF-IDF model of word n-grams. In one example embodiment, unigrams, bigrams, and trigrams are extracted, and the TF-IDF model is limited to 70,000 features.

[0128] In one embodiment, the first contextual feature extraction model 706-C and the second contextual feature extraction model 706-D include trained embedding models such as word2vec vectorizers and transformer-based models such as BERT or CamemBERT (CBERT) based on RoBERTa. In an example embodiment, word2vec is trained using the skipgram model with 200,000 training transcripts. The embedding dimension is set to 300 and an Adam optimizer with negative sampling loss and a learning rate of 0.003 is used. In another example embodiment, a 10-kb optimization with a learning rate of 0.004 is used. and The CBERT model is fine-tuned on 200,000 training transcripts using the Adam optimizer. Beneficially, by training the context-level feature extraction model on specific speech-to-text data, custom representations of words specific to the speech-to-text output data can be generated. Using such custom representations may help improve the performance of the feature extraction process 706 when processing noisy transcript data.

[0129] In one embodiment, the third context-level feature extraction model 706-E includes a model for generating a fuzzy embedding (i.e., a probability class distribution for a given transcript). The fuzzy embedding can take the form of a vector having a dimension equal to the number of possible actions (or related actions), such that each value of the vector corresponds to a probability associated with an action (or related action). Here, references to classes can be understood as references to possible actions (or related actions). Typically, the fuzzy embedding corresponds to the normalized frequency of words in the transcript relative to their class distribution in the training data set. For example, consider the transcript text, "arepair of your phone may be appropriate". In this example, there are two classes corresponding to two possible actions. For each word in the transcript text, a 2-dimensional embedding (i.e., an embedding equal to the number of possible actions) is created. The 2-dimensional embedding corresponds to a class or action distribution. For example, the embedding of "a" in the transcript can be (0.5, 0.5), which corresponds to the normalized total number of times the word "a" appears in the transcript associated with each possible action. The embedding vector can be understood as indicating that the word "a" appears the same number of times in the transcript of each possible action. Similarly, the 2-dimensional embedding of the word "repair" may be (0.25, 0.75), which indicates that the word "repair" is three times more likely to appear in the transcript associated with the second possible action than in the transcript associated with the first possible action. After generating embeddings for all words in the transcript, the average of the embeddings is calculated. This average corresponds to the fuzzy embedding of the transcript. Therefore, the fuzzy embedding context-level feature extraction model can be viewed as a supervised feature extraction method because it relies on the action-based distribution of words within the training data.

[0130] Multiple feature extraction models in the multi-level feature extractor are applied to the plain text representation of the transcript to generate multiple intermediate language vectors. The feature vectors produced by each feature extraction model are weighted. Let t represent a transcript with n words, i.e., . A vectorized representation of the transcript is then created by:

[0131]

[0132] in, , Corresponds to each feature extraction model in the multi-level feature extractor. Weight The contribution of the corresponding feature extraction model to the vectorized representation is determined. Thus, by modifying the weights, the vectorized representation of the transcript can be fine-tuned for different contexts. Advantageously, this allows the feature extraction process 706 to be implemented efficiently across different contexts (e.g., language, domain, task setting, etc.). In an example embodiment, each weight is a binary value such that . Alternatively, each weight is real-valued such that .

[0133] Wherein the character level feature extraction model 706-A includes character n-grams TF-IDF model, word-level feature extraction model 706-B includes word n-grams The first context-level feature extraction model 706-C includes a trained word2vec vectorizer, , the second context-level feature extraction model 706-D includes a fine-tuned CBERT model, , and the third context-level feature extraction model 706-E includes a fuzzy embedding model, In the embodiment of the present invention, the vectorized representation of the transcript t is created by the following formula:

[0134]

[0135] here, In an embodiment, the average of the embeddings produced by the first context-level feature extraction model 706-C, the second context-level feature extraction model 706-D, and the third context-level feature extraction model 706-D, or a combination thereof, is used to generate transcript-level features, which can be concatenated with other features as part of the above-mentioned vectorized representation.

[0136] After extracting vectorized representations of the plurality of transcripts 704 , they are output from the feature extraction process 706 as a plurality of natural language feature vectors for use in generating a coarse-grained classifier 710 and a fine-grained classifier 714 .

[0137] The coarse-grained classifier 710 is generated by training the binary classifier using multiple natural language feature vectors and indicator vectors in the binary classifier training process 708. In one embodiment, the binary classifier is a binary extreme gradient boost (XGBoost) model. The XGBoost model is trained on multiple natural language feature vectors and indicator vectors using a logarithmic loss function, a learning rate of 0.1, and 200 estimators. The technician will understand that other binary classification models such as support vector machines (SVMs), decision trees, random forests, artificial neural networks, etc. can be used. The indicator vector identifies the first subset 704-A (known to be related to the relevant action) from the multiple transcripts 704. Therefore, the indicator vector can take the form of a binary vector that indicates the true or false value of each transcript in the multiple transcripts (wherein the true or false value indicates whether the corresponding transcript is known to be related to the relevant action). Therefore, the indicator vector becomes the target vector for the binary classifier to be trained.

[0138] In many cases, the number of "labeled" transcripts (i.e., transcripts in the first subset 704-A that are known to be associated with the relevant action) is much less than the number of unlabeled transcripts. In some cases, the proportion of such unlabeled transcripts is about 90% of all transcripts. To address this problem, in some embodiments, a semi-supervised training method is used to train a binary classifier. Semi-supervised learning is a learning method that uses a large amount of unlabeled data together with labeled data to train the final model. The insight behind the semi-supervised learning technique is that relevant information can be learned from both labeled data and unlabeled data. In one embodiment, semi-supervised training is performed using self-training. In self-training, a binary classifier is first trained on the natural language feature vectors associated with the transcripts in the first subset 704-A. After observing the predicted probabilities generated by the binary classifier, the indicator vector is updated to include the labeled natural language feature vector with the highest confidence built into the multiple natural language feature vectors (i.e., the vector with the highest predicted probability). The binary classifier is retrained, and the procedure is repeated until the labeling is stable or stabilized within a predetermined threshold.

[0139] Once trained, the coarse-grained classifier 710 assigns a relevance label of "1" to natural language feature vectors associated with transcripts that are predicted to be relevant (i.e., they are predicted to be relevant to the relevant action) and assigns a relevance label of "0" to natural language feature vectors associated with transcripts that are predicted to be irrelevant.

[0140] A fine-grained classifier 714 is generated in a multi-class classifier training process 712 by training a multi-class classifier using the first subset 704-A and the action vectors associated with the first subset 704A. In one embodiment, the multi-class classifier is a multi-class XGBoost model. The XGBoost model is trained on the first subset 704-A and the action vectors using a logarithmic loss function, a learning rate of 0.1, and 200 estimators. The skilled person will appreciate that other multi-class classification models may be used, such as support vector machines, decision trees, random forests, artificial neural networks, etc. For each transcript within the first subset 704-A, the action vector includes relevant actions known to be associated with the corresponding transcript within the first subset 704-A. Thus, the action vector includes relevant actions for each relevant transcript, such as regarding Figure 5 The associated action 516 of the associated action-result pair 514 is described.

[0141] For example, consider an action vector L associated with transcript t. The action vector L includes k elements corresponding to k possible related actions with a number of related actions K. At position A value of 1 at K indicates that the i-th relevant action of K is associated with transcript t (e.g., the i-th relevant action mentioned or provided in transcript t). A multi-class classifier is trained to predict the label vector x associated with transcript t. ,,in, The i-th value in indicates whether t is predicted to be associated with the i-th related action of K. In one embodiment, The elements are such as Alternatively, The elements of are real-valued and indicate the score or probability of each associated action.

[0142] exist In embodiments where the elements of are real-valued scores or probabilities, a threshold operation may be performed to The elements of are converted to binary values. In such an embodiment, the training process 700 includes a threshold selection process 718 that produces a threshold operation 720. The threshold operation can then be included as part of the transcript marking pipeline 702. The threshold selection process 718 uses a fine-grained classifier 714 on a reserved set 722 of manually annotated transcripts. Each transcript within the reserved set 722 is assigned a label indicating one or more relevant actions associated with the transcript. In an example embodiment, the reserved set 722 includes approximately 200 manually labeled transcripts. A search for possible thresholds is performed (e.g., using a grid search method), and for each iteration of the search, a test threshold is selected and a random subset of the reserved set 722 is selected as a training set (e.g., 50%, 75%, 90%, etc. of the reserved set 722). The fine-grained classifier 714 is applied to each natural language feature vector in the random subset, and the test threshold is applied to the resulting label vector to generate a binary value label vector. The accuracy of the test threshold is determined using the following formula:

[0143]

[0144] Where A is the set of relevant actions in the manually labeled vector for transcript t, and B is the set of relevant actions generated from the predictions produced by the fine-grained classifier 714 for t with the test threshold applied. The accuracy is recorded for each threshold in the search, and the threshold with the highest recorded accuracy is selected for the threshold operation 720. Thus, the threshold operation 720 converts the real-valued label vector produced by the fine-grained classifier 714 into a binary-valued label vector by assigning a value of 1 to all values ​​in the real-valued label vector that are greater than or equal to the selected threshold.

[0145] Then, the transcript marking pipeline 702 is formed according to the feature extraction process 706, the coarse-grained classifier 710, the fine-grained classifier 714, and the optional threshold operation 720. The transcript marking pipeline 702 in use is as shown in Figure 6 Operate in the manner described in the related operation.

[0146] Thus, the transcript labeling pipeline 702 can be used to label a set of transcripts for use in various machine learning processes within a contact center system. In one embodiment, the transcript labeling pipeline 702 is used to process a set of transcripts to identify exposed and hidden actions, and then use these actions to form a pairing model, as described above with respect to Figure 3 and Figure 6 As described above, the model can be used in a contact allocation system to efficiently allocate contacts and agents.

[0147] Figure 8A method 800 for decision-making behavior pairing in a contact center system according to one aspect of the present disclosure is shown. In one embodiment, the method 800 is performed by a pairing module, such as Figure 1A The behavior matching module 140 or Figure 1B The contact distribution strategy module 180.

[0148] The method 800 includes the steps of obtaining 802 a pairing model, determining 804 a plurality of possible contact-agent pairings, and selecting 806 a pairing model for assignment. The method 800 optionally includes the step of connecting 808 based on the selected pairing model.

[0149] Thus, method 800 enables a contact center system to operate in a new and improved manner. Identifying the most relevant contact-agent pairings, as well as the actions most likely to be accepted during interactions involving that pairing, results in an improved contact center system because contacts can be serviced more efficiently, e.g., by reducing AHT and / or more effectively allocating contact, agent, and offer / action resources, thereby enabling the contact center system to handle more interactions more efficiently.

[0150] At the acquisition 802 step of method 800, a pairing model including multiple elements is acquired. The pairing model is determined based on transcript data related to calls in a contact center system involving multiple agents and multiple contact types (e.g., Figure 3-6 An element of the plurality of elements indicates a probability that a contact-agent pairing will be associated with an action of the plurality of possible actions being taken. The probability is determined based on instances of offered and taken actions occurring within the transcript data and instances of offered and declined actions occurring within the transcript data.

[0151] At step 804 of determining in method 800, a plurality of possible contact-agent pairings are determined from the plurality of contacts waiting to be assigned and the at least one agent available for assignment. The plurality of possible contact-agent pairings are determined based on the agent dimension and the contact type dimension of the pairing model (e.g., based on Figure 3 The agent dimension 302 and the contact type dimension 304 shown in the pairing model 300 of FIG.

[0152] At the selection 806 step of method 800, a first contact-agent pairing is selected for allocation in the contact center system using the contact type dimension and the action probability dimension of the pairing model. The first contact-agent pairing is selected to increase the likelihood that the contact-agent pairing results in an offer and an action being taken. That is, the first contact-agent pairing is selected from a plurality of possible contact-agent pairings based on a plurality of possible action probabilities determined from the action probability dimension of the pairing model. In one embodiment, the action probability dimension is related to a plurality of potential offers to be provided by the plurality of agents.

[0153] In some embodiments, the first contact-agent pairing is selected based on the action probability dimension of the pairing model and resource availability. For example, in embodiments where the action corresponds to an offer that the agent can present to the contact, the contact center may have limited availability for each offer (e.g., only 100 offers are available for product A, and only 100 offers are available for product B). Consider two contact types CT1 and CT2, both of which accept offer A, but only CT2 accepts offer B. For this example, consider that during a fixed time period, 200 contacts arrive, of which 120 are of type CT1 and 80 are of type CT2, both of which arrive randomly at the contact center.

[0154] Under a conventional system that does not optimize resource allocation, the agent may have determined that all contacts accept Offer A. Therefore, the agent will not suggest Offer B until there are no more Offer A. Given this, it can be expected that only 140 of the 200 contacts can be served during a fixed time period and under a conventional system. That is, the agent will suggest Offer A to the first 100 contacts, of which 60 are CT1 and 40 are CT2. For the remaining 100 contacts, the agent will suggest Offer B, and only the remaining CT2 contacts will accept (e.g., the remaining 40 CT2 contacts).

[0155] In contrast, if, according to the presently described invention, the service can be increased so that 180 contacts are served by correctly deciding which products or offers should be provided to which contact types, that is, the invention will determine that the agent should suggest Offer B to all 80 CT2 contacts and Offer A to the first 100 CT1 contacts.

[0156] Thus, the present disclosure always provides better resource allocation than traditional systems. Specifically, method 800 causes the contact center system to operate in a new way. The pairings determined from the pairing model cause one or more switches in the contact center system to route contacts to agents. As described above, method 800 optimizes these connections to improve contact handling and reduce factors that affect contact center operations, such as AHT.

[0157] Fig. 9A 900A is shown for determining a label vector from a related transcript according to one aspect of the present invention. Thus, method 900A corresponds to Figure 6 A transcript marking pipeline 600 is shown.

[0158] The method 900A includes the steps of obtaining 902 a plurality of transcripts, filtering 904 the plurality of transcripts to identify a plurality of relevant transcripts, and determining 906 a plurality of label vectors. In one embodiment, the plurality of label vectors are used to form a pairing model to decide on behavioral pairings in a contact center system, such that the method 900A includes the step of forming 908 elements of the pairing model, and optionally includes the steps of determining 910 possible contact-agent pairings from the pairing model and selecting 912 a contact-agent pairing from the pairing model for assignment. In another embodiment, in addition to or in lieu of the step of forming 908, the method 900A includes the step of outputting 914 relevant transcripts and label vectors.

[0159] At the obtaining 902 step of method 900A, a plurality of transcripts (eg, Figure 6 The plurality of transcripts 610 in the contact center system are provided. Each transcript includes a plain text representation of a call held within the contact center system. The plain text representation of each transcript has at least one substring associated with at least one action of the predetermined plurality of actions. The predetermined plurality of actions includes a plurality of related actions and one or more unrelated actions. In one embodiment, the plurality of actions corresponds to a plurality of offers and the plurality of related actions corresponds to a plurality of related offers.

[0160] In the filtering 904 step of method 900A, a relevance classifier (or a coarse-grained classifier; for example, Figure 6 The coarse-grained classifier 604 in FIG. 6 is used to filter the multiple transcripts to identify multiple related transcripts. This operation is described above with respect to Figure 4-6 . Each of the plurality of related transcripts includes a substring associated with one of the plurality of related actions. In one embodiment, the relevance classifier includes a binary classifier trained on a transcript training set, such as a binary extreme gradient boosting classifier. It is known that each transcript in the transcript training set has at least one substring associated with at least one of the predetermined plurality of actions.

[0161] In the determination 906 step of method 900A, an action classifier (or a fine-grained classifier; for example, Figure 6 A fine-grained classifier 606 in the transcript classifier is used to determine multiple label vectors for multiple related transcripts. Label vectors within the multiple label vectors are associated with the related transcripts and indicate which of the multiple related actions are associated with the related transcripts. In one embodiment, the action classifier includes a multi-class classifier trained on the transcript training set, such as a multi-class extreme gradient boosting classifier. It is known that each transcript in the transcript training set has at least one substring associated with at least one of the multiple related actions.

[0162] In one embodiment, method 900A includes forming 908 a step (e.g., Figure 6 At formation 908, multiple elements of the pairing model are formed based on one or more agents associated with the multiple related transcripts, one or more contact types associated with the multiple related transcripts, and the multiple label vectors (e.g., Figure 6 616 in the pairing model). An element of the plurality of elements indicates a probability that a contact-agent pairing will be associated with the action being taken.

[0163] In an embodiment including the forming 908 step, the method 900A may include a determining 910 step and a selecting 912 step. In the determining 910 step, a plurality of possible contact-agent pairings are determined from a plurality of contacts waiting to be assigned and at least one agent available for assignment. The plurality of possible contact-agent pairings are determined by a first dimension of a pairing model and a second dimension of the pairing model. The first dimension may be an agent dimension associated with a plurality of agents of the contact center system (e.g., Figure 3 The second dimension may be a contact dimension associated with a plurality of contact types of the contact center system (e.g., Figure 3 The third dimension may be an action probability dimension associated with a predetermined plurality of actions (e.g., Figure 3 In one example embodiment, the plurality of actions corresponds to a plurality of offers to be provided by a plurality of agents of the contact center system.

[0164] At the selection step 912, a first contact-agent pairing is selected for allocation in the contact center system. The first contact-agent pairing is selected from a plurality of possible contact-agent pairings based on the third dimension of the pairing model. A switch of the contact center system may then be instructed to form a connection within the contact center system that affects the first contact-agent pairing.

[0165] In another embodiment, method 900A further includes, in addition to or instead of forming step 908, the step of outputting 914 the associated transcript and label vector. The associated transcript and label vector may be output for storage at a contact center system or for further analysis.

[0166] Fig. 9B Another method 900B for determining a label vector from related transcripts according to one aspect of the present disclosure is shown. Thus, method 900A corresponds to Figure 6 A transcript marking pipeline 600 is shown.

[0167] The method 900B includes the steps of obtaining 916 a plurality of transcripts, extracting 918 a plurality of feature vectors from the plurality of transcripts, obtaining 920 a relevance classifier, applying 922 the relevance classifier to the plurality of feature vectors, and applying 924 an action classifier to the plurality of feature vectors to determine a plurality of label vectors. In one embodiment, the plurality of label vectors are used to form a pairing model to determine a pairing of behaviors in a contact center system, such that the method 900B includes the step of forming 908, as described with respect to Fig. 9A Method 900A as described above, and optionally including steps of determining 910 and selecting 912, as described with respect to Fig. 9A In another embodiment, in addition to or in place of the step of forming 908, the method 900B further comprises: Fig. 9A The output 914 step of method 900A is described.

[0168] At the acquisition 916 step of method 900B, a plurality of transcripts (eg, Figure 6 The plurality of transcripts 610 in the plurality of transcripts includes a plain text representation of a call involving an agent of the contact center system and a contact having a contact type. The plain text representation has at least one substring associated with at least one action of the predetermined plurality of actions. The predetermined plurality of actions includes a plurality of related actions and one or more unrelated actions. In one embodiment, the plurality of actions corresponds to a plurality of offers and the plurality of related actions corresponds to a plurality of related offers.

[0169] At the extracting 918 step of method 900B, a feature extraction process (e.g., Figure 6 The feature extraction process 602 shown in FIG. 1 extracts multiple natural language feature vectors from multiple transcripts. The feature extraction process will be described below with respect to Fig.11 In one embodiment, the method 900B further comprises applying one or more pre-processing operations to the plurality of transcripts before the step of extracting 918. The one or more pre-processing operations include one or more of text truncation, data enhancement, data cleaning, and spell checking.

[0170] At the acquisition 920 step of method 900B, a relevance classifier (eg, Figure 6 The relevance classifier assigns a relevance score to the natural language feature vector. The relevance score indicates the probability that the transcript associated with the natural language feature vector is associated with one or more of the plurality of relevant actions.

[0171] At the applying 922 step of method 900B, a relevance classifier is applied to the plurality of natural language feature vectors (extracted at the extracting 918 step) to identify a plurality of relevant natural language feature vectors from the plurality of natural language feature vectors. The plurality of relevant natural language feature vectors are associated with a plurality of relevant transcripts associated with a plurality of relevant actions in the plurality of transcripts. In one embodiment, the relevance classifier comprises a binary classifier trained on a transcript training set, such as a binary extreme gradient boosting classifier. It is known that each transcript in the transcript training set has at least one substring associated with at least one action in a predetermined plurality of actions.

[0172] At the apply 924 step of method 900B, an action classifier (e.g., fine-grained classifier 606) is applied to the plurality of relevant natural language feature vectors to determine a plurality of label vectors. A label vector in the plurality of label vectors indicates which of the plurality of relevant actions are associated with the transcript associated with the label vector. In one embodiment, the action classifier comprises a multi-class classifier trained on a transcript training set, such as a multi-class extreme gradient boosting classifier. It is known that each transcript in the transcript training set has at least one substring associated with at least one of the plurality of relevant actions.

[0173] Fig.10 A method for training a transcript labeling pipeline (e.g., in Fig. 9A Method 900A and Fig. 9B Method 1000 of the method 900B). Therefore, method 1000 corresponds to Figure 7 The training process 700 shown in FIG.

[0174] The method 1000 includes the steps of obtaining 1002 a plurality of transcripts, obtaining 1004 an action vector, and deriving 1006 an indicator vector. The method 1000 also includes the steps of extracting 1008 a plurality of natural language feature vectors, generating 1010 a coarse-grained classifier, generating 1012 a fine-grained classifier, and forming 1016 a labeling pipeline. Optionally, the method 1000 includes the steps of identifying 1014 a threshold operation and outputting 1018 a transcript labeling pipeline.

[0175] At the acquisition 1002 step of method 1000, a plurality of transcripts (eg, Figure 7704). Each of the plurality of transcripts includes a plain text representation of a call held between an agent and a contact. The plain text representation has at least one substring associated with at least one of the predetermined plurality of actions; however, the association may not be marked. For example, as described elsewhere herein, the substring "cinq cent A20" may be associated with the action "Samsung A20", but the association may not be marked, making the association hidden or latent. The predetermined plurality of actions includes a plurality of related actions and one or more unrelated actions. In one embodiment, the plurality of actions corresponds to a plurality of offers and the plurality of related actions corresponds to a plurality of related offers.

[0176] At the acquisition 1004 step of method 1000, an action vector associated with a subset of the plurality of transcripts (e.g., the first subset 704-A) is acquired. For a transcript within the subset of the plurality of transcripts, the action vector includes an action from the plurality of related actions. The action is associated with a substring within the plain text representation of the transcript.

[0177] At the deriving 1006 step of method 1000, an indicator vector is derived from the action vector. The indicator vector identifies a subset of the plurality of transcripts within the plurality of transcripts.

[0178] At the extracting 1008 step of method 1000, a plurality of natural language feature vectors are extracted from the plurality of transcripts using a feature extraction process (eg, feature extraction process 706), as described below with respect to Fig.11 The plurality of natural language feature vectors include a subset of natural language feature vectors corresponding to a subset of the plurality of transcripts. Optionally, method 1000 further includes applying one or more pre-processing actions to the plurality of transcripts prior to step 1008 of extracting. The one or more pre-processing actions include one or more of text truncation, data enhancement, data cleaning, and spell checking.

[0179] At the generating 1010 step of method 1000, the algorithm is trained using a plurality of natural language feature vectors and indicator vectors (e.g., using Figure 7 The binary classifier training process 708) is used to generate a coarse-grained classifier (e.g., Figure 7 In one embodiment, the coarse-grained classifier 710 in FIG. 1 ). For a natural language feature vector associated with a transcript, the coarse-grained classifier estimates a probability that the transcript is associated with a plurality of related actions. In one embodiment, the coarse-grained classifier comprises a binary extreme gradient boosting classifier. In one embodiment, a semi-supervised training method is used to train the binary classifier. The semi-supervised training method may include repeatedly training the binary classifier on a plurality of natural language feature vectors and indicator vectors using a self-training method.

[0180] At the generating 1012 step of method 1000, by using a subset of natural language feature vectors and action vector training (e.g., using Figure 7 The multi-class classifier training process 712 in the multi-class classifier is used to generate a fine-grained classifier (e.g., Figure 7 The fine-grained classifier 714 in FIG. 1 ). For the natural language feature vector associated with the transcript, the fine-grained classifier determines a label vector having a plurality of label values ​​associated with a plurality of related actions. Each label value in the plurality of label values ​​indicates whether the transcript includes a substring associated with a related action in the plurality of related actions. In one embodiment, the multi-class classifier includes a multi-class gradient boosting classifier.

[0181] At optional identifying 1014 of method 1000, identifying (e.g., using Figure 7 Threshold selection process 718) threshold operation (e.g., Figure 7 The threshold operation 720 in the example above is used to convert the label vector into a binary value label vector. The threshold operation is included as part of the transcript labeling pipeline. In one embodiment, the threshold operation is based on a threshold identified using a grid search and a set of transcripts known to be associated with multiple related actions (e.g., Figure 7 In such an embodiment, each of the plurality of tag values ​​is a binary value.

[0182] At step 1016 of method 1000, a transcript marking pipeline (e.g., Figure 7 The transcript labeling pipeline 702 in ) consists of a feature extraction process (e.g., Figure 7 706), coarse-grained classifiers (e.g., Figure 7 ), a coarse-grained classifier 710 in , a fine-grained classifier (e.g., Figure 7 714) and an optional threshold operation (e.g., Figure 7 The transcript marking pipeline formed receives the transcript and passes the transcript to a feature extraction process to generate a vectorized representation. The vectorized representation is passed to a coarse-grained classifier to determine the relevance of the transcript. When the estimated probability determined by the coarse-grained classifier for the natural language feature vector exceeds or meets a predetermined threshold, the vectorized representation is passed to a fine-grained classifier. If the estimated probability does not exceed the predetermined threshold, the transcript marking pipeline can provide an empty label vector as an output.

[0183] At an optional output 1018 of method 1000, the transcript marking pipeline is output for use within a contact center system (e.g., as described in relation to Figure 6 and 8 -9).

[0184] Fig.11 A method 1100 for automatic vectorization of plain text transcript data according to one aspect of the present disclosure is shown. The method 1000 can be used as the extraction 916 step of the method 900B and / or Fig.10 Part of the extraction 1008 step.

[0185] The method 1100 comprises the steps of obtaining 1102 a transcript, obtaining 1104 a multi-stage feature extractor, applying 1106 the multi-stage feature extractor to the transcript, and creating 1108 a vectorized representation of the transcript. The method 1100 optionally comprises the step of outputting 1110 the vectorized representation.

[0186] At the obtain 1102 step of method 1100 , a transcript including a plain text representation of a call on hold between an agent and a contact is obtained.

[0187] At the acquisition step 1104 of method 1100, a multi-level feature extractor is acquired for extracting a plurality of natural language feature vectors from the input transcript. The multi-level feature extractor includes a plurality of feature extraction models (e.g., Figure 7 The feature extraction models 706-A, 706-B, 706C, and 706-D in the embodiment include: each feature extraction model generates a natural language feature vector in a plurality of natural language feature vectors. Each multi-level feature extractor extracts natural language features of different representation levels from the transcript. In one embodiment, the plurality of feature extraction models include two or more of the following: a character-level feature extraction model (e.g., Figure 7 Character-level feature extraction model 706-A in ; word-level feature extraction model (e.g., Figure 7 706-B in the word-level feature extraction model); and at least one context-level feature extraction model (e.g., Figure 7 The first context-level feature extraction model 706-C or the second context-level feature extraction model 706-D in . The character-level feature extraction model may include a term frequency-inverse document frequency model of character n-grams. The word-level feature extraction model may include a term frequency-inverse document frequency model of word n-grams. At least one context-level feature extraction model may include a trained embedding model, such as a trained word2vec model or a Transformer model, such as BERT or CBERT.

[0188] At the applying 1106 step of method 1100 , a multi-stage feature extractor is applied to the transcript, thereby generating a plurality of intermediate natural language vectors.

[0189] At the create 1108 step of method 1100, a vectorized representation of the transcript is created by weighting each of the plurality of intermediate natural language vectors according to a corresponding weighting factor in a plurality of weighting factors. Each of the plurality of weighting factors determines a contribution of the corresponding intermediate natural language vector to the vectorized representation. The plurality of weighting factors may be obtained prior to the step of creating the vectorized representation. In one embodiment, creating the vectorized representation includes concatenating the plurality of intermediate natural language vectors weighted according to the plurality of weighting factors. Alternatively, an average value of each of the plurality of intermediate vectors is concatenated to create the vectorized representation. The plurality of weighting factors may be binary values.

[0190] At the optional output 1110 step of method 1100, a vectorized representation of the transcript is output for use in a machine learning pipeline (e.g., as described in relation to Figure 6-7 and 9-10).

[0191] Example Results

[0192] The following results are provided by way of example only and are not intended to limit the scope of the present disclosure in any way. The above outlines the effectiveness of the disclosed techniques in processing natural language documents for transcript tagging and forming a pairing model for decision action pairing in a contact center system.

[0193] Fig.12 Table 1200 shows the results of various methods applied to two noisy transcript datasets.

[0194] Results table 1200 shows the transcript tagging results for two datasets (dataset A and dataset B) according to the method 1202 of the present disclosure and the baseline method 1204. The method 1202 of the present disclosure corresponds to a coarse-grained classifier and a fine-grained classifier based on XGBoost, with different feature extractors (a combination of a word-level feature extractor and two different context-level feature extractors). The baseline method 1204 corresponds to BiLSTM with Attention and Word2Vec, CBert, ByT5, and Electra.

[0195] Dataset A corresponds to a collection of 48,657 French transcripts with 6 possible actions (i.e., 6 outcomes, labels, or categories). These transcripts were obtained by applying a third-party speech-to-text system to 48,657 recorded contact center interactions. The 6 possible actions correspond to the 6 possible contacts that an agent might provide during the interaction. The dataset uses a 0.2 train-test split, 503 manually labeled transcripts, 251 transcripts for threshold determination, and 252 transcripts for testing.

[0196] Dataset B corresponds to a collection of 26,252 French transcripts with 96 possible actions (i.e., 96 outcomes, labels, or categories). These transcripts were obtained by applying a third-party speech-to-text system to contact center interactions on the 26,252 transcripts. The 96 possible actions correspond to the 96 possible products that an agent might offer during the interaction. The dataset uses a 0.2 train-test split, 268 manually labeled transcripts, 134 transcripts for threshold determination, and 134 transcripts for testing.

[0197] For both datasets, the above Figure 7 The transcript tagging pipeline disclosed in . Preprocessing operations were performed to convert all text to lowercase and remove redundant characters (i.e., punctuation marks). In both datasets, certain types of noise were observed, such as connected words (e.g., "todayisnice" as opposed to "today is nice") and mismapped words (e.g., "oui oui" instead of "Huawei"). The noise ratio for both datasets was about 90%.

[0198] The results in result table 1200 according to the method of the present disclosure 1202 utilized an XGBoost model (as described in more detail above). The TF-IDF word n-gram model utilized French stop words. An n-gram range of (1, 3) was used, and up to 70,000 features were extracted. As shown in result table 1200, the word-level feature extractor (TF-IDF word n-gram) and the context-level feature extractor (Word2Vec) provided the highest performance for single label prediction and multi-label prediction across both datasets. This approach outperformed the baseline BERT and ByT5 methods in all settings. This shows that the combined techniques of the present disclosure provide improved tagging performance compared to state-of-the-art neural network models. Therefore, the feature extraction techniques of the present disclosure help address issues associated with the sensitivity of neural network methods to noise.

[0199] At this point, it should be noted that the contact allocation and natural language processing according to the present disclosure as described above may involve the processing of input data and the generation of output data to some extent. The input data processing and output data generation can be implemented in hardware or software. For example, specific electronic components can be used in a behavior pairing module or similar or related circuits to implement the functions associated with the task allocation according to the present disclosure as described above. Alternatively, one or more processors operating according to instructions can implement the functions associated with the task allocation according to the present disclosure as described above. If this is the case, such instructions can be stored on one or more non-temporary processor-readable storage media (e.g., disks or other storage media), or transmitted to one or more processors via one or more signals contained in one or more carriers, which is within the scope of the present disclosure.

[0200] The scope of the present disclosure is not limited to the specific embodiments described herein. In fact, in addition to the embodiments and modifications described herein, it will be clear to those skilled in the art from the above description and the accompanying drawings that various other embodiments and modifications of the present disclosure are possible. Therefore, these other embodiments and modifications are intended to fall within the scope of the present disclosure. In addition, although the present disclosure has been described herein in the context of at least one specific embodiment for at least one specific purpose in at least one specific environment, a person of ordinary skill in the art will recognize that its usefulness is not limited thereto, and the present disclosure can be beneficially implemented for any number of purposes in any number of environments. Therefore, the claims described below should be interpreted in accordance with the full scope and spirit of the present disclosure as described herein.

Claims

1. A method for automatic vectorization of plain text transcript data, the method comprising: acquiring, by at least one computer processor, a transcript comprising a plain text representation of a call on hold between an agent and a contact; Obtaining, by the at least one computer processor, a multi-stage feature extractor for extracting a plurality of natural language feature vectors from an input transcript, the multi-stage feature extractor comprising a plurality of feature extraction models, each of the plurality of feature extraction models generating a natural language feature vector from the plurality of natural language feature vectors, wherein each multi-stage feature extractor extracts natural language features of different representation levels from the transcript; applying, by the at least one computer processor, a multi-stage feature extractor to the transcript to generate a plurality of intermediate natural language vectors; creating, by the at least one computer processor, a vectorized representation of the transcript by weighting each of the plurality of intermediate natural language vectors according to a corresponding weighting factor of a plurality of weighting factors, wherein each of the plurality of weighting factors determines a contribution of a corresponding intermediate natural language vector to the vectorized representation; and The vectorized representation of the transcript is output, by the at least one computer processor, for use in a machine learning pipeline.

2. The method according to claim 1, wherein: The plurality of feature extraction models include two or more of the following: Character-level feature extraction model; Word-level feature extraction model; at least one context-level feature extraction model, and Transcript-level feature extraction model.

3. The method according to claim 2, wherein: The character-level feature extraction model includes: a term frequency-inverse document frequency model of character n-gram grammar.

4. The method according to claim 2, wherein: The word-level feature extraction model includes: a character-level term frequency-inverse document frequency model of word n-grams.

5. The method according to claim 2, wherein: The at least one context-level feature extraction model comprises a trained embedding model.

6. The method according to claim 5, wherein: The trained embedding model is one of a trained word2vec model or a trained transformer model.

7. The method according to claim 2, wherein: The at least one context-level feature extraction model comprises a fuzzy embedding model.

8. The method according to claim 1, wherein: The plurality of weighting factors are binary valued.

9. The method according to claim 1, wherein: The step of creating the vectorized representation of the transcript includes concatenating the plurality of intermediate natural language vectors weighted according to the plurality of weighting factors.

10. The method according to claim 1, wherein: The step of creating the vectorized representation of the transcript includes concatenating a plurality of averages weighted according to the plurality of weighting factors, the plurality of averages corresponding to the averages determined for each of the plurality of intermediate natural language vectors.

11. The method according to claim 1, further comprising: The plurality of weighting factors are obtained, by the at least one computer processor, prior to the step of creating the vectorized representation of the transcript.

12. A non-transitory computer readable medium comprising instructions which, when executed by a device comprising at least one computer processor, cause the device to perform the steps of claim 1.

13. A system comprising at least one computer processor and a memory storing instructions, which when executed by the at least one computer processor cause the system to perform the steps of claim 1.

14. A method for forming a pairing model for decision-making behavior pairing in a contact center system, the method comprising: Retrieving, by at least one computer processor, a plurality of transcripts, each transcript of the plurality of transcripts comprising a plain text representation of a call held within the contact center system, the plain text representation having at least one substring associated with at least one action of a predetermined plurality of actions, wherein the predetermined plurality of actions comprises a plurality of related actions and one or more unrelated actions; filtering, by the at least one computer processor and using a relevance classifier, the plurality of transcripts to identify a plurality of related transcripts, wherein each related transcript of the plurality of related transcripts includes a substring associated with a related action of the plurality of related actions; determining, by the at least one computer processor and using an action classifier, a plurality of label vectors for the plurality of related transcripts, wherein the label vectors for the related transcripts indicate which of the plurality of related actions are associated with the related transcript; and A plurality of elements of a pairing model are formed, with the at least one computer processor, based on one or more agents associated with the plurality of related transcripts, one or more contact types associated with the plurality of related transcripts, and the plurality of label vectors, wherein an element of the plurality of elements indicates a probability that a contact-agent pairing will be associated with an action being taken.

15. A method for forming a pairing model for decision-making behavior pairing in a contact center system, the method comprising: Retrieving, by at least one computer processor, a plurality of transcripts, each transcript of the plurality of transcripts comprising a plain text representation of a call involving an agent of the contact center system and a contact having a contact type, the plain text representation having at least one substring associated with at least one action of a predetermined plurality of actions, wherein the predetermined plurality of actions comprises a plurality of related actions and one or more unrelated actions; extracting, by the at least one computer processor, a plurality of natural language feature vectors from the plurality of transcripts using a feature extraction process; Retrieving, by the at least one computer processor, a relevance classifier that assigns a relevance score to a natural language feature vector, wherein the relevance score indicates a probability that a transcript associated with the natural language feature vector is associated with one or more of the plurality of related actions; applying, by the at least one computer processor, the relevance classifier to the plurality of natural language feature vectors to identify a plurality of relevant natural language feature vectors from the plurality of natural language feature vectors, wherein the plurality of relevant natural language feature vectors are associated with a plurality of relevant transcripts from the plurality of transcripts that are related to the plurality of relevant actions; applying, by the at least one computer processor, an action classifier to the plurality of related natural language feature vectors to determine a plurality of label vectors, wherein a label vector in the plurality of label vectors indicates which of the plurality of related actions are associated with a transcript associated with the label vector; and A plurality of elements of a pairing model are formed, with the at least one computer processor, based on one or more agents associated with the plurality of related transcripts, one or more contact types associated with the plurality of related transcripts, and the plurality of label vectors, wherein an element of the plurality of elements indicates a probability that a contact-agent pairing will be associated with an action being taken.

16. The method according to claim 15, further comprising: The pairing model for decision-making behavior pairing is output in the contact center system by the at least one computer processor.

17. The method according to claim 15, further comprising, after forming the pairing model: A plurality of possible contact-agent pairings are determined, by the at least one computer processor, among a plurality of contacts awaiting assignment and at least one available assigned agent, wherein: The plurality of possible contact-agent pairings are determined from the first dimension of the pairing model and the second dimension of the pairing model; as well as Selecting, by the at least one computer processor, a first contact-agent pairing for assignment in the contact center system, wherein the first contact-agent pairing is selected from the plurality of possible contact-agent pairings based on a third dimension of the pairing model.

18. The method according to claim 17, wherein: The first dimension is an agent dimension associated with a plurality of agents of the contact center system. The second dimension is a contact dimension associated with a plurality of contact types of the contact center system, and The third dimension is an action dimension associated with the predetermined plurality of actions.

19. The method according to claim 15, wherein: The predetermined plurality of actions corresponds to a plurality of offers to be provided by a plurality of agents of the contact center system.

20. The method according to claim 15, wherein: The feature extraction process comprises the method according to claim 1.

21. A non-transitory computer readable medium comprising instructions which, when executed by a device comprising at least one computer processor, cause the device to perform the steps of claim 15.

22. A system comprising at least one computer processor and a memory storing instructions which, when executed by the at least one computer processor, cause the system to perform the steps of claim 15.

23. A method for decision-making behavior pairing in a contact center system, the method comprising: obtaining, by at least one computer processor communicatively coupled to the contact center system and configured to operate in the contact center system, a pairing model comprising a plurality of elements determined from transcript data associated with calls within the contact center system involving a plurality of agents and a plurality of contact types, wherein an element of the plurality of elements indicates a probability that a contact-agent pairing will be associated with an action of a plurality of possible actions being taken, the probability being determined based on instances of offer and take actions occurring within the transcript data and instances of offer and decline actions occurring within the transcript data; determining, by the at least one computer processor, a plurality of possible contact-agent pairings among a plurality of contacts awaiting assignment and at least one available assigned agent, wherein the plurality of possible contact-agent pairings are determined from an agent dimension and a contact type dimension of the pairing model; and selecting, by the at least one computer processor, a first contact-agent pairing for assignment in the contact center system using the contact type dimension and the action probability dimension of the pairing model, wherein the first contact-agent pairing is selected to increase the likelihood that the contact-agent pairing results in an offer and an action being taken.

24. A method for decision-making behavior pairing in a contact center system, the method comprising: obtaining, by at least one computer processor communicatively coupled to the contact center system and configured to operate in the contact center system, a pairing model comprising a plurality of elements determined from transcript data associated with calls within the contact center system involving a plurality of agents and a plurality of contact types, wherein an element of the plurality of elements indicates a probability that a contact-agent pairing will be associated with an action of a plurality of possible actions being taken, the probability being determined from instances of offer and take actions occurring within the transcript data and instances of offer and decline actions occurring within the transcript data; Determining, by the at least one computer processor, a plurality of possible contact-agent pairings among a plurality of contacts waiting to be assigned and at least one available assigned agent, wherein the plurality of possible contact-agent pairings are determined from an agent dimension and a contact type dimension of the pairing model; and A first contact-agent pairing is selected, by the at least one computer processor, for assignment in the contact center system, wherein the first contact-agent pairing is selected from the plurality of possible contact-agent pairings based on a plurality of possible action probabilities determined from an action probability dimension of the pairing model.

25. The method according to claim 24, wherein: The action probability dimension is associated with a plurality of potential offers provided by the plurality of agents.

26. The method according to claim 25, wherein: The first contact-agent pairing is selected based on the action probability dimension and resource availability of the pairing model.

27. A non-transitory computer readable medium comprising instructions which, when executed by a device comprising at least one computer processor, cause the device to perform the steps of claim 24.

28. A system comprising at least one computer processor and a memory storing instructions which, when executed by the at least one computer processor, cause the system to perform the steps of claim 24.

29. A method for marking a transcript in a contact center system, the method comprising: Retrieving, by at least one computer processor, a plurality of transcripts, each transcript of the plurality of transcripts comprising a plain text representation of a call held within the contact center system, the plain text representation having at least one substring associated with at least one action of a predetermined plurality of actions, wherein the predetermined plurality of actions comprises a plurality of related actions and one or more unrelated actions; filtering, by the at least one computer processor and using a relevance classifier, the plurality of transcripts to identify a plurality of related transcripts, wherein each transcript of the plurality of related transcripts includes a substring associated with one of the plurality of related actions; determining, by the at least one computer processor and using an action classifier, a plurality of label vectors for the plurality of related transcripts, wherein the label vectors for the related transcripts indicate which of the plurality of related actions are associated with the related transcripts; and The plurality of related transcripts and the plurality of label vectors are output, via the at least one computer processor, for use within a decision-making process of the contact center system.

30. A method for marking a transcript in a contact center system, the method comprising: Retrieving, by at least one computer processor, a plurality of transcripts, each of the plurality of transcripts comprising a plain text representation of a call held between an agent and a contact of the contact center system, the plain text representation having at least one substring associated with at least one action of a predetermined plurality of actions, wherein the predetermined plurality of actions comprises a plurality of related actions and one or more unrelated actions; extracting, by the at least one computer processor, a plurality of natural language feature vectors from the plurality of transcripts using a feature extraction process; Retrieving, by the at least one computer processor, a relevance classifier that assigns a relevance score to a natural language feature vector, wherein the relevance score indicates a probability that a transcript associated with the natural language feature vector is associated with one or more of the plurality of related actions; applying, by the at least one computer processor, the relevance classifier to the plurality of natural language feature vectors to identify relevant natural language feature vectors from the plurality of natural language feature vectors, wherein the relevant natural language feature vectors are associated with relevant transcripts from the plurality of transcripts that are related to the plurality of relevant actions; applying, by the at least one computer processor, an action classifier to the relevant natural language feature vector to determine a label vector for the relevant transcript, wherein the label vector comprises a plurality of label values, each label value in the plurality of label values ​​being associated with an action in the plurality of relevant actions, such that the label vector indicates which relevant actions in the plurality of relevant actions are associated with the relevant transcript; and The associated transcript and the label vector are output, via the at least one computer processor, for use within a decision-making process of the contact center system.

31. The method according to claim 30, wherein: The relevance classifier comprises a binary classifier trained on a training set of transcripts, each of the transcripts being known to have at least one substring associated with at least one action of the predetermined plurality of actions.

32. The method according to claim 31, wherein: The binary classifier includes a binary extreme gradient boosting classifier.

33. The method of claim 30, wherein: The action classifier comprises a multi-class classifier trained on a training set of transcripts, each of the transcripts being known to have at least one substring associated with at least one of the plurality of related actions.

34. The method of claim 33, wherein: The multi-class classifier includes a multi-class extreme gradient boosting classifier.

35. The method of claim 30, wherein: The plurality of actions correspond to a plurality of offers, and The plurality of related actions corresponds to a plurality of related offers.

36. The method according to claim 30 further comprises, before the step of extracting the plurality of feature vectors: One or more pre-processing operations are applied, by the at least one computer processor, to the plurality of transcripts.

37. The method of claim 36, wherein: The one or more pre-processing operations include one or more of the following: Text truncation; Data augmentation; Data cleaning; as well as Spell check.

38. The method of claim 30, wherein: The feature extraction process comprises the method according to claim 1.

39. A non-transitory computer readable medium comprising instructions which, when executed by a device comprising at least one computer processor, cause the device to perform the steps of claim 30.

40. A system comprising at least one computer processor and a memory storing instructions, which when executed by the at least one computer processor cause the system to perform the steps of claim 30.

41. A method for training a transcript labeling pipeline, the method comprising: Retrieving, by at least one computer processor, a plurality of transcripts, each of the plurality of transcripts comprising a plain text representation of a call held between an agent and a contact, the plain text representation having at least one substring associated with at least one action of a predetermined plurality of actions, wherein the predetermined plurality of actions comprises a plurality of related actions and one or more unrelated actions; obtaining, by the at least one computer processor, an action vector associated with a subset of the plurality of transcripts, the action vector comprising an action from a plurality of related actions for transcripts within the subset of the plurality of transcripts, wherein the action is associated with a substring within the plain text representation of the transcript; deriving, by the at least one computer processor, an indicator vector from the action vector, wherein the indicator vector identifies the subset of the plurality of transcripts within the plurality of transcripts; extracting, by the at least one computer processor, a plurality of natural language feature vectors from the plurality of transcripts using a feature extraction process, the plurality of natural language feature vectors comprising a subset of the natural language feature vectors corresponding to the subset of the plurality of transcripts; generating, by the at least one computer processor, a coarse-grained classifier by training a binary classifier using the plurality of natural language feature vectors and the indicator vector, wherein, for a natural language feature vector associated with a transcript, the coarse-grained classifier estimates a probability that the transcript is associated with the plurality of related actions; generating, by the at least one computer processor, a fine-grained classifier by training a multi-class classifier using a subset of the natural language feature vector and the action vector, wherein, for the natural language feature vector associated with the transcript, the fine-grained classifier determines a label vector having a plurality of label values ​​associated with the plurality of related actions, each label value in the plurality of label values ​​indicating whether the transcript includes a substring associated with a related action in the plurality of related actions; forming, by the at least one computer processor, the transcript marking pipeline from the feature extraction process, the coarse-grained classifier, and the fine-grained classifier, wherein when an estimated probability determined by the coarse-grained classifier for the natural language feature vector exceeds a predetermined threshold, passing the natural language feature vector to the fine-grained classifier; and The transcript marking pipeline is output, via the at least one computer processor, for use within a contact center system.

42. The method according to claim 41, wherein: The plurality of actions correspond to a plurality of offers, and The plurality of related actions corresponds to a plurality of related offers.

43. The method of claim 41, wherein: A semi-supervised training method is used to train the binary classifier.

44. The method of claim 43, wherein: The semi-supervised training method comprises: The binary classifier is repeatedly retrained on the plurality of natural language feature vectors and the indicator vector using a self-training method.

45. The method of claim 41, wherein: When the estimated probability does not exceed the predetermined threshold, a null label vector is provided as an output from the transcript labeling pipeline.

46. ​​The method of claim 41, wherein: The binary classifier includes a binary extreme gradient boosting classifier.

47. The method of claim 41, wherein: The multi-class classifier includes a multi-class extreme gradient boosting classifier.

48. The method of claim 41, wherein: Each label value of the plurality of label values ​​is a binary value.

49. The method according to claim 48 further comprises, after generating the fine-grained classifier: identifying, by the at least one computer processor, a threshold operation to convert the label vector into a binary valued label vector, wherein: The threshold operation is included as part of the transcript marking pipeline.

50. The method of claim 49, wherein: The threshold operation is based on a threshold identified using a retained set of transcripts known to be associated with the plurality of related actions.

51. The method of claim 50, wherein: A grid search is used to identify the threshold.

52. The method according to claim 41 further comprises, before the step of extracting the plurality of natural language feature vectors: One or more pre-processing operations are applied, by the at least one computer processor, to the plurality of transcripts.

53. The method of claim 52, wherein: The one or more pre-processing operations include one or more of the following: Text truncation; Data augmentation; Data cleaning; as well as Spell check.

54. The method of claim 41, wherein: The feature extraction process comprises the method according to claim 1.

55. A non-transitory computer readable medium comprising instructions which, when executed by a device comprising at least one computer processor, cause the device to perform the steps of claim 41.

56. A system comprising at least one computer processor and a memory storing instructions which, when executed by the at least one computer processor, cause the system to perform the steps of claim 41.

Images