Methods, systems, and articles of manufacture for verifying dispatch policies in contact center system
By using the combination method of training weights and verification weights in the contact center system, the weighted results collection is generated, which solves the problem of overfitting in in-sample verification, and realizes more accurate model performance evaluation and dispatch strategy optimization, improving the system's Overall performance and customer satisfaction.
Patent Information
- Application Number
- CN202411225059.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-02
- Publication Date
- 2024-11-22
AI Technical Summary
In contact center systems, existing model verification technologies, such as in-sample verification, are difficult to effectively detect overfitting, especially in the case of scarcity of data, resulting in inaccurate model performance evaluation and affecting the optimization of dispatching strategies.
By receiving the historical paired results set, the training weight set and the verification weight set are determined, the training weight and verification weight set are assigned to each record, the training weight set and the verification weight set are generated, the expected performance of the dispatch strategy is calculated, and the expected performance is selected based on the expected performance Optimal strategy, and then pair agents with contacts.
Effectively detect and prevent overfitting, improve the generalization ability of the model and the accuracy of performance evaluation, thereby optimizing the contact center's dispatch strategy, improving the overall performance and customer satisfaction of the system.
Smart Images

Figure CN119011731A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates generally to validating dispatch policies, and more particularly to techniques for validating dispatch policies in a contact center system. Background Art
[0002] A contact center system is a system used to receive or transmit a large number of contacts, such as voice calls, Internet text chats, emails, and video calls. Contact centers can include outbound contact centers, which transmit outgoing contacts, and / or inbound contact centers, which receive incoming contacts. Outbound contact centers are often used for applications such as selling products, collecting outstanding credit balances, or surveying customer sentiment. Inbound contact centers are also used to sell or cross-sell products, service customer support or technical support inquiries, retain customers, upgrade and downgrade services, or for other applications.
[0003] Various dispatch strategies may be used to pair agents with contacts in a call center. In some traditional contact centers, contacts are dispatched to agents sequentially based on the time the contacts arrive or when agents become available. This strategy may be referred to as a First-In-First-Out (FIFO) strategy. For example, if multiple agents are available to receive contacts, a FIFO strategy would dispatch incoming contacts to the agent who has been waiting the longest. Similarly, if multiple contacts are waiting to be dispatched, a FIFO strategy would dispatch the contact with the longest wait time to the first available agent. However, FIFO only seeks to balance the wait times of agents or contacts and may not align with other contact center metrics, such as revenue generation, improved customer satisfaction, or reduced costs.
[0004] To optimize these other contact center metrics, some contact centers employ alternative agent-contact assignment strategies. One such example is a Performance-Based-Routing (PBR) strategy. In a PBR strategy, when there are multiple agents available to receive a contact, the incoming contact will be assigned to the agent with the highest historical performance compared to the other available agents. Similarly, if all agents are occupied and many contacts are already waiting to be assigned, the PBR strategy will assign the contact that it determines to have the highest expected value (highest potential for making a sale) to the first agent to become available.
[0005] Another example is a behavioral pairing (BP) strategy. Relative to the FIFO strategy, the BP strategy generally seeks to balance the utilization of agents and the waiting time of contacts, while still providing improvements in contact center metrics. In the BP strategy, contacts and agents are paired so that the total expected value for all agent and contact pairs is optimized.
[0006] BP strategies are typically built using machine learning techniques and data from historical interactions between agents and contacts. Since the output from models built using machine learning techniques inevitably involves some randomness, it is necessary to verify the quality of such models. The process of finalizing the BP strategy may involve multiple iterations of "training" (creating) candidate models, verifying models, and discarding poorly verified models. Model training processes historical data to establish candidate BP models. Model verification processes the same or different historical data to estimate the performance of candidate BP models. Multiple candidate models can be trained and verified, allowing comparisons between the estimated performance of those candidate models and selecting a preferred model for use in the contact center.
[0007] There are different model validation techniques. In-sample validation is a validation technique that uses the same data for both model training and validation. In-sample validation may lead to incorrect validation results. For example, in-sample validation may not detect overfitting, which is a situation where a small sample of data unduly influences the model parameters.
[0008] Overfitting is particularly prevalent when there is limited data. Representative historical data in a contact center environment is often a scarce resource. Representative data is data that accurately represents the characteristics of agents and contact behavior. Changing marketing campaigns, product offerings, agent numbers and compensation, contact behavior, and various other factors can limit the amount of representative data. For the purposes of model training and validation, the ideal approach is to use recent representative data. However, there often arises a need to balance the trade-off between data recency and data usability. Therefore, there is a need to use representative data effectively.
[0009] One solution to detecting overfitting is to split the historical data into separate sets for training and validation, respectively. This is called out-of-sample validation. For example, U.S. Pat. No. 10,122,860 teaches that a single historical data set can be split into a first subset (i.e., a training set) and a reserved subset (i.e., a validation set) that does not overlap with the first subset, the training set is used for model training, and the reserved subset is used for model validation.
[0010] In this patent, we disclose a methodology for in-sample validation that can detect overfitted models. Summary of the invention
[0011] An embodiment of the present disclosure provides a method for verifying a dispatch strategy in a contact center system, comprising: receiving a set of historical pairing results; determining a training weight set and a verification weight set; assigning a training weight from the training weight set and a verification weight from the verification weight set to each record from the set of historical pairing results; determining a training weighted result set based on the set of historical pairing results and the training weight set; determining a verification weighted result set based on the set of historical pairing results and the verification weight set; determining one or more dispatch strategies based on the training weighted result set; calculating expected performance of one or more dispatch strategies based on the verification weighted result set; and selecting a dispatch strategy based on the expected performance.
[0012] Optionally, the above method further comprises: pairing the agent with the contact based on the selected dispatch strategy.
[0013] Optionally, in the above method, the training weights from the training weight set range from 0 to 1, and the validation weights from the validation weight set range from 0 to 1.
[0014] Optionally, in the above method, the sum of the training weight and the validation weight of each historical pairing result is 0 or 1.
[0015] Optionally, in the above method, the training weights and the validation weights have an inverse corresponding relationship.
[0016] Optionally, the above method further comprises: receiving a first record set from a contact record system; receiving a second achievement record set from a customer record system; and assigning the second achievement record set to the first record set based on the first record set and the second achievement record set.
[0017] Optionally, the above method further comprises: adjusting the selected pairing strategy based on real-time conditions in the contact center system.
[0018] An embodiment of the present disclosure also provides a system for verifying a dispatch strategy in a contact center system, comprising: at least one computer processor connected to a dispatch module, the dispatch module being configured to perform a verification operation in the contact center system, wherein the at least one computer processor is further configured to: receive a set of historical pairing results; determine a training weight set and a verification weight set; assign a training weight from the training weight set and a verification weight from the verification weight set to each record from the set of historical pairing results; determine a training weighted result set based on the set of historical pairing results and the training weight set; determine a verification weighted result set based on the set of historical pairing results and the verification weight set; determine one or more dispatch strategies based on the set of training weighted results; calculate expected performance of one or more dispatch strategies based on the set of verification weighted results; and select a dispatch strategy based on the expected performance.
[0019] Optionally, in the above system, the at least one computer processor is further configured to: pair an agent with a contact based on a selected dispatch strategy.
[0020] Optionally, in the above system, the training weights from the training weight set range from 0 to 1, and the validation weights from the validation weight set range from 0 to 1.
[0021] Optionally, in the above system, the sum of the training weight and the validation weight of each historical pairing result is 0 or 1.
[0022] Optionally, in the above system, the training weights and the validation weights have an inverse corresponding relationship.
[0023] Optionally, in the above system, at least one computer processor is further configured to: receive a first record set from a contact record system; receive a second achievement record set from a customer record system; and assign the second achievement record set to the first record set based on the first record set and the second achievement record set.
[0024] Optionally, in the above system, the at least one computer processor is further configured to: adjust the selected pairing strategy based on real-time conditions in the contact center system.
[0025] Embodiments of the present disclosure also provide an article for verifying a dispatch strategy in a contact center system, comprising: a non-transitory processor-readable medium; and instructions stored on the medium, wherein the instructions are configured to be readable from the medium by at least one computer processor, the at least one computer processor is connected to a dispatch module, the dispatch module is configured to perform a verification operation in the contact center system, and thereby operate the at least one computer processor to: receive a set of historical pairing results; determine a training weight set and a verification weight set; assign a training weight from the training weight set and a verification weight from the verification weight set to each record from the historical pairing results set; determine a training weighted results set based on the historical pairing results set and the training weight set; determine a verification weighted results set based on the historical pairing results set and the verification weight set; determine one or more dispatch strategies based on the training weighted results set; calculate expected performance of one or more dispatch strategies based on the verification weighted results set; and select a dispatch strategy based on the expected performance.
[0026] Optionally, in the above article, the at least one computer processor is further caused to operate to: pair an agent with a contact based on the selected dispatch strategy.
[0027] Optionally, in the above article, the training weights from the training weight set range from 0 to 1, and the validation weights from the validation weight set range from 0 to 1.
[0028] Optionally, in the above article, the sum of the training weight and the validation weight of each historical pairing result is 0 or 1.
[0029] Optionally, in the above article, the training weights and the validation weights have an inverse corresponding relationship.
[0030] Optionally, in the above article, at least one computer processor is further configured to: receive a first record set from a contact record system; receive a second achievement record set from a customer record system; and assign the second achievement record set to the first record set based on the first record set and the second achievement record set.
[0031] Optionally, in the above article, the at least one computer processor is further caused to operate to: adjust the selected pairing strategy based on real-time conditions in the contact center system. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 is a schematic diagram showing a contact center system according to an embodiment of the present disclosure.
[0033] Figure 2 is a schematic diagram showing a dispatch module according to an embodiment of the present disclosure.
[0034] Figure 3 is a flow chart showing a method for verifying a dispatch strategy in a contact center system according to an embodiment of the present disclosure.
[0035] Figure 4 is a schematic diagram showing an example of verifying a dispatch strategy in a contact center system according to an embodiment of the present disclosure.
[0036] Figure 5 , Figure 6 and Figure 7 is a schematic diagram showing a chart of training weights and validation weights according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0037] In order to illustrate the technical solution of the present disclosure in a coherent manner, the following will briefly describe the figures expected by the embodiments of the present disclosure. Obviously, the following figures only relate to some embodiments of the present disclosure. Based on these figures, those skilled in the art can obtain other embodiments without any creative efforts.
[0038] As used herein, the term "module" may be understood to refer to computing software, firmware, hardware, and / or various combinations thereof, and a "module" may be configured as a component of a network element, a computer, and / or a system. A module should not be interpreted as software that is not implemented in hardware or firmware, or software that is recorded on a processor-readable storage medium. These modules may be combined, integrated, separated, and / or copied to support various applications. Modules may be implemented on multiple devices and / or other components, which may be local or remote. In addition, these modules may be removed from one device and added to another device, and / or may be included in two devices.
[0039] Figure 1 1 is a schematic diagram showing a contact center system according to an embodiment of the present disclosure, and shows an example of a contact center system 100 . The contact center system 100 may include: an automatic contact distribution (“Automated Contact Distribution, ACD”) system 120 , a dispatch module 130 ; and an agent 140 .
[0040] The ACD system 120 may be configured to receive incoming contacts 110 via a telecommunications network, or to support outbound connections to contacts. The ACD system may include contact routing hardware and software that assists in routing contacts in one or more contact centers. The ACD system may also include a queuing or switching component within the contact center system 100.
[0041] The dispatch module 130 is connected to the ACD system 120 and is configured to pair contacts with agents based on a dispatch policy.
[0042] Figure 2 2 is a schematic diagram showing a dispatch module according to an embodiment of the present disclosure. The dispatch module 130 may further include a data storage module 210 , a weighting module 220 , a strategy generation module 230 and an optimal dispatch module 240 , wherein the strategy generation module 230 includes a training module 231 and a verification module 232 .
[0043] The data storage module 210 can be configured for functions such as monitoring, receiving, storing, retrieving, and outputting historical pairing information between contacts and agents. Each pairing record, for example, can include an agent identifier, a contact identifier, and a performance indicator (ie, historical pairing efforts).
[0044] The weighting module 220 can be configured to: determine a set of training weights and a set of validation weights, wherein the training weights and the validation weights may or may not be associated with each other; assign the training weights and the validation weights to historical pairing results; and determine a set of training weighted results and a set of validation weighted results, respectively.
[0045] The strategy generation module 230 may be configured to: determine one or more dispatch strategies based on the training weighted achievement set; calculate the expected performance of one or more dispatch strategies based on the verification weighted achievement set; and select a dispatch strategy based on the expected performance. To achieve the above purpose, the strategy generation module 230 may include a training module 231 and a verification module 232.
[0046] The training module 231 may be configured to determine one or more dispatching strategies based on the training weighted effort set. Then, the one or more dispatching strategies may be applied in the verification module 232.
[0047] The verification module 232 may be configured to calculate the expected performance of one or more dispatch strategies based on the verification weighted effort set. Based on the expected performance, the strategy generation module 230 may select an optimal dispatch strategy from the one or more dispatch strategies.
[0048] The optimal dispatch module 240 may be configured to receive the optimal dispatch strategy from the strategy generation module 230 and communicate the optimal contact-agent pairing to the ACD system 120 according to the optimal dispatch strategy.
[0049] Figure 3 The present invention is a flowchart showing a method for verifying a dispatch strategy in a contact center system according to an embodiment of the present invention. The method of the present invention may include: S301, receiving a historical pairing result set; S302, determining a training weight set and a verification weight set; S303, assigning a training weight from a training weight set and a verification weight from a verification weight set to each record from the historical pairing result set; S304, determining a training weighted result set based on the historical pairing result set and the training weight set; S305, determining a verification weighted result set based on the historical pairing result set and the verification weight set; S306, determining one or more dispatch strategies based on the training weighted result set; S307, calculating the expected performance of one or more dispatch strategies based on the verification weighted result set; and S308, selecting a dispatch strategy based on the expected performance.
[0050] Figure 4 FIG. 4 is a diagram showing an example of verifying a dispatch strategy in a contact center system according to an embodiment of the present disclosure. Figure 4 As shown, a historical pairing data set is received, the historical pairing data set including 20 contacts and 20 corresponding outcomes (“Outcome, O”), wherein contact 1 is paired with agent Bob and brings an outcome of $10, contact 2 is paired with agent Jerry and brings an outcome of $0, contact 3 is paired with agent Bob and brings an outcome of $20, ..., and contact 20 is paired with agent Bob and brings an outcome of $5.
[0051] Determine a training weight ("Training Weight, TW") set and a validation weight ("Validation Weight, VW") set, and assign the training weight set and the validation weight set to the historical pairing result set to generate a training weighted outcome ("Training Weighted Outcome, TWO") set and a validation weighted outcome ("Validation Weighted Outcome, VWO"). Figure 4 As shown, contact 1 is assigned a training weight of 0.999 and a validation weight of 0.001, contact 2 is assigned a training weight of 0.997 and a validation weight of 0.003, contact 3 is assigned a training weight of 0.995 and a validation weight of 0.005, ..., and contact 20 is assigned a training weight of 0.001 and a validation weight of 0.999.
[0052] After assigning and determining weights, a training weighted achievement set is determined based on the historical pairing achievement set and the training weight set. Contact 1 paired with agent Bob is determined to have a training weighted achievement of $9.99 (calculated from $10*0.999); Contact 2 paired with agent Jerry is determined to have a training weighted achievement of $0 (calculated from $0*0.997); Contact 3 paired with agent Bob is determined to have a training weighted achievement of $19.9 (calculated from $20*0.995); ..., and Contact 20 paired with agent Bob is determined to have a training weighted achievement of $0.005 (calculated from $5*0.001). The verification weighted achievement set is determined based on the historical pairing achievement set and the verification weight set. Contact 1, paired with agent Bob, is determined to have a verification weighted effort of $0.01 (calculated from $10*0.001); contact 2, paired with agent Jerry, is determined to have a verification weighted effort of $0 (calculated from $0*0.003); contact 3, paired with agent Bob, is determined to have a verification weighted effort of $0.1 (calculated from $20*0.005); ..., and contact 20, paired with agent Bob, is determined to have a verification weighted effort of $4.995 (calculated from $5*0.999).
[0053] The candidate BP model is generated by the training module 231 using the training weighted effort, and the expected performance of the candidate BP model is calculated by the verification module 232 using the verification weighted effort.
[0054] Figure 5 , Figure 6 and Figure 7 is a schematic diagram showing a chart of training weights and validation weights according to an embodiment of the present disclosure.
[0055] A popular approach for splitting data into training and test sets for out-of-sample validation is to split the data by recency. The most recent historical data is kept for validation, while the remaining data is used for training. This simulates real-world conditions where a model generated from past data is used in the future. Figure 5 It is shown how this idea can be applied in an embodiment of the present disclosure. The training and validation weights are assigned so that newer samples have higher validation weights and lower training weights. The weights are generated by applying a sigmoid function to the adjusted timestamps of the contacts so that both the training weights and the validation weights are constrained between 0 and 1.
[0056] Another popular approach to splitting the data into training and test sets is called k-fold validation. In out-of-sample k-fold validation, the data is split into "k" groups where a subset of the k groups are selected for training and the remainder are used for validation. Similar to temporal validation, samples that are adjacent in time are likely to belong to the same group. This has the added benefit of having the ability to run multiple validations on the same model (by varying which groups belong to the training and validation sets), thereby increasing the statistical accuracy of the validation results.
[0057] Figure 6 It is shown how the idea of k-folds can be used as an embodiment of the present disclosure. We choose k=10 folds: numbered 1-10, 1 being the least recent and 10 being the most recent. We randomly select folds 2, 3, and 6 to be used primarily for validation, and folds 1, 4, 5, 7, 8, 9, and 10 to be used primarily for model training. The training and validation weights can be generated by using a linear combination of multiple sigmoid functions.
[0058] For example, in Figure 6 The weights in the graph in can be generated by adding a sigmoid function centered on 2023 / 01 / 28 and subtracting a sigmoid function centered on 2023 / 02 / 23; adding a sigmoid function centered on 2023 / 04 / 01; and subtracting a sigmoid function centered on 2023 / 04 / 18.
[0059] Figure 7 The training weights generated using an embodiment of the present disclosure are shown, which combines a time-based method for calculating training and validation weights with a k-fold-based method for calculating training and validation weights. First, using time-based validation (as in Figure 5 ) and k-fold-based validation (as described in Figure 6 For each sample, we randomly choose whether to use the weights from time-based validation or the weights from k-fold-based validation. Figure 7It is evident from Fig. 2 that the properties from both time-based validation and k-fold-based validation are still present in the combined weights.
[0060] An experienced practitioner will validate multiple models and select the best model (based on the validation results). This best model will then be trained on the entire dataset (without training and validation weights) and will be used to make real-time pairing decisions.
[0061] The present disclosure is not limited in scope by the specific embodiments described herein. In fact, in addition to those described herein, various other embodiments of the present disclosure and modifications to the present disclosure will be apparent to those of ordinary skill in the art from the foregoing description and the accompanying drawings. Therefore, such other embodiments and modifications are intended to fall within the scope of the present disclosure. Further, although the present disclosure has been described herein in the context of at least one specific embodiment in at least one specific environment for at least one specific purpose, those of ordinary skill in the art will recognize that the practicality of the present disclosure is not limited thereto, and the present disclosure can be beneficially implemented in any number of environments for any number of purposes. Therefore, the set of claims set forth below should be parsed in view of the full breadth and spirit of the present disclosure as described herein.
Claims
1. A method for verifying a dispatch strategy in a contact center system, comprising: Receive a set of historical pairing results; Determine the training weight set and the verification weight set; Assigning a training weight from the training weight set and a validation weight from the validation weight set to each record from the historical pairing result set; Determine a training weighted achievement set based on the historical pairing achievement set and the training weight set; Determine a verification weighted achievement set based on the historical pairing achievement set and the verification weight set; Determining one or more dispatching strategies based on the set of training weighted efforts; calculating an expected performance of the one or more dispatch strategies based on the set of verification weighted efforts; as well as A dispatch strategy is selected based on the expected performance.
2. The method according to claim 1, further comprising: Pairs agents with contacts based on the selected dispatch strategy.
3. The method according to claim 1, wherein: The training weights from the training weight set range from 0 to 1, and the validation weights from the validation weight set range from 0 to 1.
4. The method according to claim 1, wherein: The sum of the training weight and validation weight of each historical pairing outcome is 0 or 1.
5. The method according to claim 1, wherein: The training weight and the verification weight have an inverse corresponding relationship.
6. The method according to claim 1, wherein: The training weight set and the validation weight set are determined based on a combination of two or more weight generation strategies.
7. The method according to claim 1, wherein: Receiving the historical pairing result set includes: receiving a first set of records from a contact record system; receiving a second set of achievement records from a customer system of record; and The second set of effort records is assigned to the first set of records based on the first set of records and the second set of effort records.
8. The method according to claim 1, further comprising: The selected pairing strategy is adjusted based on real-time conditions in the contact center system.
9. A system for verifying a dispatch strategy in a contact center system, comprising: at least one computer processor connected to a dispatch module, the dispatch module being configured to perform a verification operation in the contact center system, wherein the at least one computer processor is further configured to: Receive a set of historical pairing results; Determine the training weight set and the verification weight set; Assigning a training weight from the training weight set and a validation weight from the validation weight set to each record from the historical pairing result set; Determine a training weighted achievement set based on the historical pairing achievement set and the training weight set; Determine a verification weighted achievement set based on the historical pairing achievement set and the verification weight set; Determining one or more dispatching strategies based on the set of training weighted efforts; calculating an expected performance of the one or more dispatch strategies based on the set of verification weighted efforts; and A dispatch strategy is selected based on the expected performance.
10. The system according to claim 9, wherein: The at least one computer processor is further configured to: Pairs agents with contacts based on the selected dispatch strategy.
11. The system according to claim 9, wherein: The training weights from the training weight set range from 0 to 1, and the validation weights from the validation weight set range from 0 to 1.
12. The system according to claim 9, wherein: The sum of the training weight and validation weight of each historical pairing outcome is 0 or 1.
13. The system according to claim 9, wherein: The training weight and the verification weight have an inverse corresponding relationship.
14. The system according to claim 9, wherein: The at least one computer processor is further configured to: receiving a first set of records from a contact record system; receiving a second set of achievement records from a customer system of record; and The second set of effort records is assigned to the first set of records based on the first set of records and the second set of effort records.
15. The system of claim 9, wherein: The at least one computer processor is further configured to: The selected pairing strategy is adjusted based on real-time conditions in the contact center system.
16. An article of manufacture for validating a dispatching strategy in a contact center system, comprising: a non-transitory processor-readable medium, and instructions, stored on said medium, wherein the instructions are configured to be readable from the medium by at least one computer processor, the at least one computer processor being connected to a dispatch module, the dispatch module being configured to perform a verification operation in the contact center system, and thereby causing the at least one computer processor to operate to: Receive a set of historical pairing results; Determine the training weight set and the verification weight set; Assigning a training weight from the training weight set and a validation weight from the validation weight set to each record from the historical pairing result set; Determine a training weighted achievement set based on the historical pairing achievement set and the training weight set; Determine a verification weighted achievement set based on the historical pairing achievement set and the verification weight set; Determining one or more dispatching strategies based on the set of training weighted efforts; calculating an expected performance of the one or more dispatch strategies based on the set of verification weighted efforts; and A dispatch strategy is selected based on the expected performance.
17. The article of claim 16, wherein: The at least one computer processor is also caused to: Pairs agents with contacts based on the selected dispatch strategy.
18. The article of claim 16, wherein: The training weights from the training weight set range from 0 to 1, and the validation weights from the validation weight set range from 0 to 1.
19. The article of claim 16, wherein: The sum of the training weight and validation weight of each historical pairing outcome is 0 or 1.
20. The article of claim 16, wherein: The training weight and the verification weight have an inverse corresponding relationship.
21. The article of claim 16, wherein: The at least one computer processor is further configured to: receiving a first set of records from a contact record system; receiving a second set of achievement records from a customer system of record; and The second set of effort records is assigned to the first set of records based on the first set of records and the second set of effort records.
22. The article of claim 16, wherein: The at least one computer processor is also caused to: The selected pairing strategy is adjusted based on real-time conditions in the contact center system.
