Using machine learning to improve customer delivery net satisfaction score

Abstract

A system / method for obtaining insights from delivery KPI data uses machine learning to improve customer NSSoD. The system / method groups customers into a cluster based on the customer's delivery experience, and then associates NSSoDs from customers in the cluster with their delivery-related KPIs to identify high impact KPIs for the customers. The delivery-related KPI may be a predefined set of KPIs selected according to the need of a particular application. The system / method generates a customer-specific KPI score for a predefined group KPI based on KPI data for the customer for the last month and historical KPI data for the past 12 months of the entire cluster. The use of historical KPI data provides a larger group of data to perform analysis, thereby providing a more accurate insight. The above method allows a company to pay attention to a specific delivery KPI within the cluster that may increase NSSoD.

Description

Technical Field

[0001] The present disclosure relates to techniques for improving customer satisfaction through the use of machine learning, and more particularly to systems and methods for using machine learning to gain insights from key performance indicators (KPIs) to improve customer Net Satisfaction Scores for Delivery (NSSoD). Background Art

[0002] The Net Satisfaction Score of Delivery (NSSoD) is a metric often used by companies to measure customer satisfaction with the company's product or service delivery. This customer satisfaction metric can serve as a useful tool for companies to identify areas where their delivery processes can be improved to increase customer satisfaction. One way to calculate a company's NSSoD is through a survey in which customers rate their experience with a specific delivery from the company on a scale of 0 to 10, with 0 indicating dissatisfaction and 10 indicating satisfaction. The company's NSSoD can then be found by taking the average of these customer ratings.

[0003] However, the above approach lumps all customers together and fails to consider the preferences of different types of customers. Because customer experience forms the foundation of any company's success, companies must understand why a given customer is happy or unhappy with a given delivery experience. Specifically, companies need to understand how changes in their internal delivery-related KPIs (such as delivery time and delivery quantity) affect various customer perceptions. For example, business-to-business (B2B) customers may be more concerned with customer support than with on-time delivery.

[0004] Therefore, while progress has been made in the area of customer satisfaction, continuous improvement is still needed. Summary of the Invention

[0005] Embodiments of the present disclosure provide systems and methods for using machine learning (ML) to gain insights from delivery KPI data to improve customer NSSoD. The systems and methods first group customers into clusters based on their delivery experience, then correlate the NSSoD provided by customers in a cluster with their delivery-related KPIs, thereby identifying high-impact KPIs for a cluster that may impact customers within the cluster. Delivery-related KPIs can be predefined group KPIs that can be selected based on the needs of a specific application. This approach allows companies to focus on specific delivery KPIs within a cluster that may increase the NSSoD of a given customer within the cluster.

[0006] In one embodiment, the systems and methods herein generate a customer-specific KPI score that reflects the customer's delivery experience. The customer KPI score is calculated based on the customer's most recent month's KPI data and the historical KPI data of the entire cluster over the past 12 months (for a predefined group of KPIs). Using the historical KPI data of all customers within the cluster provides a larger set of data to perform analysis, thereby providing more accurate insights into the customer compared to the data of a single customer or a small number of customers. In some embodiments, the KPI score for each customer can be in the range of -1 to +1 (the KPI score formula is given below), which is used as an indicator of the overall delivery performance of the enterprise and its impact on the customer's perception of the enterprise. A negative KPI score indicates poor performance of the KPI, and a positive score indicates good performance.

[0007] In some embodiments, the disclosed systems and methods present each customer's score on a dashboard that can be displayed on any suitable display, such as a laptop, tablet, mobile device, etc. The dashboard allows the user to filter the scores based on various filters, such as at a cluster level, a segment level, a country level, etc. The scores can also be color-coded based on the score value, such that negative scores can be displayed in appropriate shades of red, while positive scores can be displayed in appropriate shades of green.

[0008] In some embodiments, the dashboard may also provide scores and analysis results in the form of a box tree chart that shows the importance of each KPI for an individual customer, a subset of customers, and the like. A box tree chart visualizes scores and results as a set of nested rectangles or "boxes" that represent certain categories within a selected dimension or parameter and are sorted in a hierarchy or "tree." A box tree chart represents the relationship of the part to the whole, allowing users to quickly compare quantities, patterns, and other information displayed in a visually limited space. In such an embodiment, the higher the importance of the KPI, the larger the area or size of the rectangle or "box" used to display the KPI. In some embodiments, the dashboard may also highlight the top five high-impact KPIs. In some embodiments, the dashboard also has an alert function that automatically notifies the user whenever a KPI score is negative, declining, or otherwise suggests that a customer is being negatively impacted.

[0009] In general, in one aspect, embodiments of the present disclosure relate to a system for improving customer satisfaction. The system includes, among other things, a processor, a display unit coupled to the processor, and a storage unit accessible by the processor. The storage unit stores an application thereon, which, when executed by the processor, causes the system to obtain performance data for a predefined group of indicators for a plurality of customers, the performance data comprising historical performance data for the predefined group of indicators and current performance data for the predefined group of indicators. When executed by the processor, the application further causes the system to perform an impact analysis using the historical performance data for the predefined group of indicators for the plurality of customers and the current performance data for the predefined group of indicators for a selected customer, the impact analysis determining whether each indicator in the predefined group of indicators has a negative impact, a positive impact, or a neutral impact on the selected customer. When executed by the processor, the application further causes the system to perform a corrective action related to the selected customer in response to determining that the indicator has a negative impact on the selected customer.

[0010] In general, in another aspect, embodiments of the present disclosure relate to a method for improving customer satisfaction. The method includes, among other things, obtaining performance data for a predefined set of indicators for a plurality of customers, the performance data including historical performance data for the predefined set of indicators and current performance data for the predefined set of indicators. The method also includes performing an impact analysis using the historical performance data for the predefined set of indicators for the plurality of customers and the current performance data for the predefined set of indicators for a selected customer, the impact analysis determining whether each indicator in the predefined set of indicators has a negative impact, a positive impact, or a neutral impact on the selected customer. The method also includes, in response to determining that an indicator has a negative impact on the selected customer, executing a corrective action associated with the selected customer.

[0011] In general, in yet another aspect, embodiments of the present disclosure relate to a computer-readable medium having computer-readable instructions stored thereon. The computer-readable instructions cause a processor to obtain performance data for a predefined set of indicators for a plurality of customers, the performance data comprising historical performance data for the predefined set of indicators and current performance data for the predefined set of indicators. The computer-readable instructions further cause the processor to perform an impact analysis using the historical performance data for the predefined set of indicators for the plurality of customers and the current performance data for the predefined set of indicators for a selected customer, the impact analysis determining whether each indicator in the predefined set of indicators has a negative impact, a positive impact, or a neutral impact on the selected customer. The computer-readable instructions further cause the processor to perform a corrective action associated with the selected customer in response to determining that the indicator has a negative impact on the selected customer.

[0012] According to any one or more of the foregoing embodiments, a plurality of customers constitute a customer cluster, the customer cluster being selected from a plurality of customer clusters, each customer cluster being defined by an application, the application causing the system to perform cluster analysis based on historical performance data of a predefined set of metrics.

[0013] According to any one or more of the foregoing embodiments, the application further enables the system to: obtain historical customer net satisfaction scores for a plurality of customers, each net satisfaction score having a value within a predefined range of values; and perform a correlation analysis using the historical performance data and customer net satisfaction scores of a predefined group of indicators, the correlation analysis providing an importance value for each indicator in the predefined group of indicators.

[0014] According to any one or more of the foregoing embodiments, the application further causes the system to generate a metric score for each metric in the predefined set of metrics based on the metric's importance value and the metric's impact on the selected customer.

[0015] According to any one or more of the foregoing embodiments, the application further causes the system to generate a customer score for the selected customer based on the sum of the indicator scores for the predefined group of indicators and the sum of the importance values for the predefined group of indicators, the customer score providing an indication of the likelihood that the selected customer will increase or decrease its net satisfaction score based on the current performance data for the predefined group of indicators.

[0016] According to any one or more of the foregoing embodiments, the application further causes the system to generate one or more interactive display screens based on the cluster analysis, correlation analysis, influence analysis, and customer scores, the one or more interactive display screens allowing a user to selectively display the results of the cluster analysis, correlation analysis, influence analysis, and / or customer scores.

[0017] According to any one or more of the aforementioned embodiments, one or more interactive displays display the impact analysis using boxes having different colors and / or different sizes. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 An exemplary system for monitoring customer satisfaction according to an embodiment of the present disclosure is shown;

[0019] Figure 2 An exemplary delivery KPI analysis and alert system and its application according to an embodiment of the present disclosure are shown;

[0020] Figure 3 An exemplary ML-based process that may be employed by a delivery KPI analysis and alerting application according to an embodiment of the present disclosure is shown;

[0021] Figure 4 An exemplary high-level flow chart illustrating a delivery KPI analysis and alerting application according to an embodiment of the present disclosure is shown;

[0022] Figure 5 shows exemplary scores and results generated by a delivery KPI analysis and alerting application according to an embodiment of the present disclosure;

[0023] Figure 6An exemplary interactive dashboard for displaying scores and results generated by a delivery KPI analysis and alerting application according to an embodiment of the present disclosure is shown;

[0024] Figure 7 shows an exemplary segment-level performance screen for displaying scores and results generated by a delivery KPI analysis and alerting application according to an embodiment of the present disclosure;

[0025] Figure 8 shows an exemplary KPI breakdown screen for displaying scores and results generated by a delivery KPI analysis and alerting application according to an embodiment of the present disclosure;

[0026] Figure 9 shows an exemplary customer-level dashboard screen for displaying scores and results generated by a delivery KPI analysis and alerting application in accordance with an embodiment of the present disclosure;

[0027] Figure 10 shows an exemplary KPI Customer Segmentation screen for displaying scores and results generated by a delivery KPI analysis and alerting application according to an embodiment of the present disclosure;

[0028] Figure 11 shows an exemplary country top 10 customers screen for displaying scores and results generated by a delivery KPI analysis and alerting application according to an embodiment of the present disclosure; and

[0029] Figure 12 Exemplary methods are shown that may be used by or with a delivery KPI analysis and alerting application according to embodiments of the present disclosure. DETAILED DESCRIPTION

[0030] This specification and the accompanying drawings illustrate exemplary embodiments of the present disclosure and should not be construed as limiting, and the claims (including equivalents) define the scope of the present disclosure. Various mechanical, compositional, structural, electrical and operational changes may be made without departing from the scope of this specification and the claims (including equivalents). In some cases, well-known structures and techniques are not shown or described in detail to avoid confusing the present disclosure. In addition, elements and related aspects thereof described in detail with reference to one embodiment (as long as feasible) may be included in other embodiments that are not specifically shown or described. For example, if an element is described in detail with reference to one embodiment, but not described with reference to a second embodiment, the element may still be required to be included in the second embodiment.

[0031] Now refer to Figure 1, shows a system 100 according to an embodiment of the present disclosure that can be used by companies and enterprises to monitor customer satisfaction. The customer satisfaction monitoring system 100 described herein can be used by enterprises to monitor various aspects of customer satisfaction, such as pricing, quality control, ease of use, etc., but is specifically configured to monitor customer satisfaction with product and / or service delivery. The customer satisfaction monitoring system 100 applies ML techniques to historical NSSoD survey data and actual customer experience data to identify specific customers and key KPIs that influence customer satisfaction for those customers, resulting in improved NSSoD scores. This allows enterprises to develop mitigation plans tailored to specific customers and their KPIs.

[0032] like Figure 1 As shown, customer satisfaction monitoring system 100 includes a delivery KPI analysis and alert system 102, a customer feedback database 104, an internal delivery KPI database 106, and machine learning and data storage resources, generally designated 108. Delivery KPI analysis and alert system 102 is coupled to receive customer feedback data from database 104 and is also coupled to receive internal delivery KPI data from database 106. These databases 104 and 106 may reside on a private enterprise network, or databases 104 and 106 may be hosted on machine learning and data storage resources 108, or a combination of both options. Delivery KPI analysis and alert system 102 then analyzes the customer feedback data and delivery KPI data using machine learning resources, such as from machine learning and data storage resources 108, to gain insights for improving customer NSSoD. This delivery KPI analysis and alert system 102 can be used by an enterprise to monitor customer satisfaction with product and / or service delivery and, as needed, issue alerts and warnings to one or more users 110 via a wired or wireless communication link 112.

[0033] In some embodiments, the customer feedback data in the customer feedback database 104 may include various types of feedback from customers and potential customers. Examples of customers from whom feedback may be obtained are generally shown at 114 and may include business-to-business (B2B) companies, business-to-end-customer (B2C) companies, wholesale and retail distributors, general contractors and subcontractors, panel builders, original equipment manufacturers (OEMs), system integrators, end users, and the like. The customer feedback data may include responses to general or specific questions, numerical ratings (e.g., 0 to 10), NSSoD scores, free-form natural language feedback, and other suitable feedback known to those skilled in the art. The customer feedback data may be provided directly by the customer, or the feedback data may be obtained from a third-party aggregator of such data.

[0034] In some embodiments, the internal KPI data in the delivery KPI database 106 may be data for a predefined set of internal delivery KPIs tracked by the enterprise. The specific internal delivery KPIs in the predefined set of KPIs may vary depending on the type of enterprise. The data for each KPI may be in the form of a percentage or corresponding decimal value ranging from 0 to 100, or the data may represent an average or other numerical representation, depending on the KPI. Exemplary group internal delivery KPIs are generally shown at 116 and may include: actual lead time (MTS) to stock, actual lead time (MTO) to order, on-time @ CRD (customer request date), on-time at customer door, on-time and early delivery to customer door; order details in mySE lines and EDI lines; number of rescheduling (NoR); actual second promise date (AC2); details of the shift to complete the order; delivery capabilities, such as express line, forward order line, kitting line; large order quantity (LOQ); order grouping line, factory to customer line, precision delivery line; and customer service center (CCC) cases (customers request information from CCC), issues (referring to missing documents or delivery issues), and number of customer visits recorded on the Bridge Front Office (BFO) for information.

[0035] As for ML and data storage resources 108, such resources can be private enterprise computing resources, publicly available cloud-based resources, and / or any other suitable resources capable of providing data storage and execution of machine learning models and algorithms. Examples of suitable ML and data storage resources 108 can be any cloud-based offering from Amazon (i.e., Amazon Web Services), Microsoft (i.e., Microsoft Azure), Google (i.e., Google Cloud), etc.

[0036] According to an embodiment of the present disclosure, the delivery KPI analysis and alert system 102 can use ML and data storage resources 108 to analyze and process customer feedback data in database 104 and delivery KPI data in database 106, as further discussed herein. In particular, this allows the delivery KPI analysis and alert system 102 to identify and cluster similar customer groups based on the customer delivery experience reflected by the internal delivery KPI data. The analysis and alert system 102 can then provide insights into the importance of each delivery KPI to a given customer and derive a KPI score (also known as a customer perception score) for the given customer. The KPI score for a given customer can be obtained using the customer's KPI data for the current month and the KPI data for the previous 12 months (i.e., historical data) of the cluster to which the customer belongs. This arrangement ensures that the KPI score for a given customer closely corresponds to the preferences of the corresponding cluster and its impact on the overall NSSoD.

[0037] In the aforementioned embodiment, if the difference between a given customer's current KPI data and the cluster's historical KPI data (e.g., the previous 12 months) exceeds a certain threshold or range, the delivery KPI analysis and alert system 102 may issue an alert to one or more users 110 to notify and request that the user 110 take appropriate corrective action. The alert may be sent via a wired or wireless connection 112 to a computing device of the user 110, such as a phone, tablet, or other mobile device, or a laptop, desktop, or other computing device. In some embodiments, the delivery KPI analysis and alert system 102 may also automatically perform one or more actions instead of, or in addition to, requesting the user 110 to perform corrective action. Examples of actions that may be automatically performed by the analysis and alert system 102 include requiring the user 110 to manually acknowledge the alert for the customer. For example, the delivery KPI analysis and alert system 102 may lock certain user privileges for the customer, such as access to the customer's account or prohibiting order entry for the customer, until the user 110 takes steps to clear the customer's alert flag.

[0038] It should be noted that while the systems and methods herein are described with respect to delivering KPIs, embodiments of the present disclosure are not limited thereto. In general, the principles and teachings disclosed herein are applicable to any performance metric that can impact customer satisfaction. Examples of performance metrics that do not deliver KPIs include customer inquiry response time, frequency of customer contact, frequency of customer social media postings, targeted customer advertising expenditures, and the like.

[0039] Figure 2 The delivery KPI analysis and alerting system 102 is shown in greater detail, according to some embodiments. The analysis and alerting system 102 in this example includes a processing unit 200, which includes, among other components, at least one processor, a network interface 202, a user interface 204, a display unit 206, and a storage unit 208. The operation of these components is generally well known, and for the sake of economy, only a brief description is provided here. Generally, the processing unit 200 is responsible for the overall operation of the analysis and alerting system 102, including various complex processing operations, data capture and storage operations, corrective action operations, and the like. The network interface 202 allows the analysis and alerting system 102 to communicate with external systems, including local systems and network systems such as cloud-based ML and data storage resources 108. The user interface 204 allows one or more users 110 to use or otherwise interact with the analysis and alerting system 102, while the display unit 206 displays graphs, images, videos, and other media content for the analysis and alerting system 102. The storage unit 208 also stores operating software and data used by and through the processing unit 200 to perform the various operations described above and other functions.

[0040] In this example, storage unit 208 can be any non-transitory storage unit known to those skilled in the art, including volatile memory (e.g., RAM), non-volatile memory (e.g., flash memory), magnetic storage, optical storage, and the like. Storage unit 208 stores thereon a plurality of applications that can be executed by processing unit 200. In some embodiments, these applications can include a delivery KPI analysis and alerting application 210, or computer-readable instructions thereof. Analysis and alerting application 210 can include a plurality of functional components, such as an ML-based customer cluster creation component 212, an ML-based key driver derivation component 214, and a customer KPI score generation component 216. As the name implies, customer cluster creation component 212 can employ an ML-based process to identify and cluster similar customer groups based on delivery KPIs. Key driver derivation component 214 can employ an ML-based process to determine the importance of each KPI for a given customer. And customer KPI score generation component 216 operates to generate a KPI score for each customer.

[0041] Figure 3 An exemplary ML-based process 300 is shown that can be employed by various ML-based components of the delivery KPI analysis and alerting application 210, including the ML-based customer cluster creation component 212 and the ML-based key driver derivation component 214. Generally speaking, the ML-based process 300 can be employed by any component that requires some form of machine learning to analyze data and detect or identify patterns in the data. In the example shown, the ML-based process 300 includes a data input component 302, one or more ML models or algorithms 304, an automatic feedback / correction component 306, a user application 308, a manual feedback / correction component 310, and an analyzer 312. Examples of algorithms that can be used with the ML-based process 300 include K-means clustering, random forests, correlation analysis, natural language processing, and the like.

[0042] In operation, data input component 302 receives data (e.g., customer ratings, customer NSSoD scores, customer survey responses, delivery KPI data, etc.) and, after appropriate preprocessing, feeds the data to one or more ML models 304. ML models 304 use machine learning and neural network processing techniques to extract relevant features from the data based on the training data used to develop the models. Automatic feedback / correction component 306 applies rules and algorithms configured to detect errors in the output received from ML models 304. These errors are used to automatically correct the model output and are fed back to ML models 304 via analyzer 312 to update the processing of ML models 304. The processed output from automatic feedback / correction component 306 is then displayed to the user for confirmation via user application 308. Corrections made by the user are captured by manual feedback / correction component 310 and fed back to ML models 304 via analyzer 312. This allows ML models 304 to continuously improve their assessment and extraction of relevant features from the input data.

[0043] Next reference Figure 4 , shows an exemplary flow chart 400 that graphically illustrates, at a high level, the process implemented by the delivery KPI analysis and alerting application 210. As will be discussed, the process uses data analytics and machine learning techniques to provide insights that can help prioritize actionable items related to the KPIs, thereby increasing the likelihood of improving the NSSoD score.

[0044] As an initial step, customers are divided into segments based on various market factors, such as business type, product type, service type, etc., in a manner known to those skilled in the art. Illustrative customer segments are shown at 402 and may include B2B companies, B2C companies, wholesale and retail distributors, general contractors and subcontractors, panel builders, OEMs, system integrators, end users, etc. Delivery KPI data for each customer cluster is shown at 404 and may be stored, for example, in the delivery KPI database 106. The segment delivery KPI data reflects the average KPI for each customer in the segment.

[0045] Next, the delivery KPI analysis and alerting application 210, via its ML-based customer cluster creation component 212, groups the segment-level customers 402 into clusters based on similarities in their delivery experiences. For purposes herein, each customer belongs to a single cluster. Delivery experience refers to the delivery-related services provided to a customer, as reflected by the customer's delivery KPI data. For example, grouping or clustering can be performed by applying K-means clustering or a similar ML algorithm to the customer's delivery KPI data. Generally speaking, K-means clustering uses vector quantization techniques to partition n observations into k clusters, where each observation belongs to the cluster with the nearest mean. In Figure 4In the example shown in FIG4 , customer cluster creation component 212 has grouped customers in the B2B segment into four clusters, namely clusters 1-4, as shown at 406. The average KPI value for each cluster is shown at 408. In this example, the leftmost column lists the individual KPIs that make up the delivery KPI data for the cluster. Clustering has the effect of grouping customers who share similar delivery experiences together.

[0046] Once clustering is performed, the delivery KPI analysis and alerting application 210, via its ML-based key driver derivation component 214, derives the importance of each KPI to the cluster. Key driver derivation component 214 may perform the derivation of the importance of each KPI by applying, for example, a random forest or similar ML algorithm to the delivery KPI data of the customers in the cluster and the customer-provided NSSoD scores. The random forest algorithm essentially combines the outputs of multiple decision trees to arrive at a single result. Thus, the random forest algorithm can process the delivery KPI data against the NSSoD scores to find the relationship between the delivery KPI data and the NSSoD scores. In some embodiments, the importance of each KPI to the cluster is given as a percentage, such that the various KPI importance values sum to 1.0 (or 100%). In some embodiments, a predefined alternative numerical range may be used instead of 100% to indicate the importance of each KPI, which range may be selected based on the needs of a particular implementation.

[0047] As part of the importance derivation process, the key driver derivation component 214 considers the impact that trends in KPI values may have on NSSoD scores. For example, the key driver derivation component 214 may find that an increasing trend in "Number of Reschedules (NoR)" tends to have a negative correlation or negative impact on customer NSSoD. Conversely, the key driver derivation component 214 may find that a similar increasing trend in "On-time at Customer Door" tends to have a positive correlation or positive impact on customer NSSoD. In some embodiments, the key driver derivation component 214 may use correlation analysis or a similar ML algorithm to determine whether a given KPI trend has a negative or positive impact on NSSoD. Correlation analysis is generally used to identify a correlation that may exist between two variables. In this example, correlation analysis can be used to analyze historical data for each KPI against past NSSoD scores to determine whether an increase or decrease in a particular KPI has a positive or negative impact on NSSoD.

[0048] In some embodiments, an exemplary variable called a logical can be used to capture whether a particular KPI has a positive or negative impact on NSSoD. For a particular KPI, the exemplary logical variable can be assigned a value of -1 if the KPI has a negative impact on NSSoD, a value of +1 if the KPI has a positive impact on NSSoD, and a value of 0 if the impact is neutral.

[0049] Thereafter, the delivery KPI analysis and alerting application 210 calculates the KPI score for each customer in a given cluster and the customer score (also known as the customer perception score) for each customer via its customer KPI score generation component 216. In some embodiments, the score generation component 216 generates a KPI score for each customer based on a month. For this month, the score generation component 216 uses the average KPI percentage for the entire cluster over the previous 12 months and the actual KPI percentage for this month for each customer in the cluster. Thus, if for a particular customer, only half of all deliveries were delivered on time this month, then "on time delivery" for that customer would be 50% (or 0.5). The difference between the 12 month percentage and this month percentage is then calculated and compared to the KPI threshold range, shown generally at 410. In some embodiments, the KPI threshold range may be selected to be 1.5 times the standard deviation of the data collected over the past 12 months for the corresponding KPI, as follows:

[0050] KPI percentage difference = historical average of the KPI for the previous 12 months - current value of the KPI for the current month1)

[0051] KPI threshold range = -1.5*standard deviation to +1.5*standard deviation2)

[0052] In the above, if the calculated KPI percentage difference is within the KPI threshold range, then the value 0 is assigned to the logical variable, whereas if the KPI percentage difference is found to be above or below the KPI threshold range, then the value +1 or -1 is assigned to the logical variable respectively, as follows:

[0053] If KPI Percent Difference < Negative KPI Threshold, then Logic = -1 3)

[0054] If KPI Percent Difference > Positive KPI Threshold, then Logic = +1 4)

[0055] If KPI percentage difference is within KPI threshold range, then logic = 0 5)

[0056] The customer KPI score generation component 216 may then use the KPI scores to calculate a customer score, shown generally at 412 (where the rightmost column shows the customer KPI score), as follows:

[0057] KPI score = logic * KPI importance6)

[0058] Customer score = (sum of KPI scores) / (sum of KPI importance) 7)

[0059] In the above calculation, the KPI importance value is determined by the ML-based key driver derivation component 214 using the random forest algorithm, as previously described. The aforementioned method provides the user with an assessment of how a particular customer reacts to the delivery experience and how important different KPIs are to that customer.

[0060] Figure 5 An exemplary result 500 is shown for a predefined group KPI generated by the delivery KPI analysis and alerting application 210 operating as described above based on exemplary group customer feedback data and exemplary group delivery KPI data. In this example, customers in the B2B segment have been clustered into four clusters, namely, clusters 1-4, as shown at 502. As reflected in the KPI data, customers in each cluster react similarly to one another regarding their delivery experiences, resulting in certain KPIs having similar importance for customers in each cluster. For example, for customers in cluster 1, accurate delivery (1.6), on-time delivery (1.5), on-time + early delivery (1.6), express delivery (1.2), and EDI (1.9) have the highest importance. For customers in cluster 2, NoRe (1.1) and EDI (2.0) have the highest importance, but accurate delivery, on-time delivery, and on-time + early delivery have lower importance compared to customers in cluster 1. For customers in cluster 3, no KPI has a particularly high importance, and for customers in cluster 4, on-time delivery has a lower importance.

[0061] exist Figure 5 In this example, the customers that make up Cluster 4 include Customers A through Z, generally shown at 504. The results for one of these customers, Customer Z, are broken down in detail at 506. These results include the 12-month historical KPI data for Customers A through Z (the "Historical" row), Customer Z's KPI data for this month (the "Current" row), the difference between the first and second rows (the "Difference" row), the values of the threshold ranges (the "Threshold" row), the values of the logical variables (the "Logical" row), the KPI importance values (the "Importance" row), the KPI scores (the "Logical * Importance" row), and the customer scores (the "Customer Score" row). The results show that for Customer Z, the most important KPIs are On-Time Delivery (1.0), On-Time + Early Delivery (1.0), No. of Result (NR) (1.1), and EDI (1.1), as shown in the data in the Importance row. The total importance of all delivery KPIs for Customer Z is 12.11, as shown at 508.

[0062] Furthermore, it can be seen that three of the delivery KPIs (On-Time + Early Delivery, Express Delivery, and Number of Customer Service Cases) have a logical variable value of -1, as shown in the logical row. This means that these three KPIs have a negative impact on Customer Z's NSSoD. The remaining delivery KPIs have a logical variable value of 0, which means that these KPIs have a neutral impact on Customer Z's NSSoD. Calculating the KPI scores for the three negative KPIs as indicated above in equation (6) yields KPI scores of -0.97, -0.40, and -0.54, respectively, as shown in 510, 512, and 514. The customer score can then be calculated using the KPI scores indicated in equation (7) above, resulting in a customer score of -0.16 (or -16%). This means that the three negative KPIs described above reduced the likelihood that Customer Z would improve its NSSoD rating based on its recent delivery experience by 16%. The user can then take appropriate corrective action upon being notified or otherwise informed of Customer Z's three negative KPIs.

[0063] Figure 6 An exemplary dashboard 600 is shown that can be used to inform or otherwise keep a user informed of KPI scores, customer scores, and other aspects of the embodiments discussed herein. Such a dashboard 600 can be displayed on any suitable display, such as a laptop, tablet, mobile device, etc. In some embodiments, the dashboard 600 can include multiple interactive screens that allow the user not only to view data and discern insights, but also to filter data based on various filters, such as by cluster, segment, country, customer, etc., as well as other functions. Scores can also be color-coded based on their value, such that negative scores can be displayed in red, as appropriate, and positive scores can be displayed in green, as appropriate.

[0064] In this embodiment, dashboard 600 has multiple tabs across the top of the screen, each tab representing a different dashboard screen designed to provide different types of information. For example, there is a segment level performance tab 602, a segment breakup by KPI tab 604, a customer level dashboard tab 606, a KPI customer breakup by KPI tab 608, a training and documentation tab 610, and a top 10 customers by country tab 612. The user can use tabs 602-612 to select the appropriate screen based on the desired information.

[0065] Figure 7An exemplary segment-level performance screen 700 is shown, which can be selected via the segment-level performance tab 602. This dashboard screen 700 is in table format, with columns 702 representing segment-level customers and rows 704 representing customer numbers, NSSoDs, and delivery KPIs. Each cell in dashboard screen 700 provides the user with a specific KPI and KPI data for a specific segment. The user can also hover the cursor over or select a cell to display a pop-up window 706 containing more detailed information about the segment and KPI, such as the segment identifier, the segment's KPI score, and the segment's KPI value. Furthermore, each cell can be filled with a different color to indicate the impact of the KPI on that cell. For example, cell 708 can be colored green to indicate that the data in the cell reflects good KPI performance, while cell 710 can be colored red to indicate that the data in the cell reflects poor KPI performance. Different shades of green or red can be used to indicate varying degrees of good or poor KPI performance. Gray or a similar neutral color can be used to indicate neutral KPI performance.

[0066] Figure 8 An exemplary KPI breakdown screen 800 is shown, which can be selected via the KPI breakdown tab 604. This dashboard screen 800 is in the form of a box tree diagram with boxes containing KPI data for a specific segment, in this case, a B2B customer segment called "B2B Distribution." Each box (one of which is shown at 802) displays data for a specific KPI for that segment. The size of each box also reflects the importance and impact of the KPI for that segment, with larger boxes reflecting greater KPI importance and impact, and vice versa. Thus, for example, box 802 is larger than box 804, meaning that the KPI represented by box 802 has greater importance and impact than the KPI represented by box 804. In addition, different colors, such as green or red (or shades thereof), can be used to indicate whether a box reflects good, poor, or neutral KPI performance. In the example shown, box 802 has a red color, reflecting poor KPI performance, while box 804 has a gray color, reflecting neutral KPI performance. In addition, one or more data filters may be provided, generally shown at 806, to allow the user to filter the data, for example, by customer segment, country, region, customer name, and time period. The data filter may be in the form of a drop-down list of filter options from which the user may select.

[0067] Figure 9An exemplary customer-level dashboard screen 900 is shown that can be selected via the customer-level dashboard tab 606. This dashboard screen 900 is also in the form of a table, with columns 902 representing segment-level customers (the "Customer Name" column), the segments to which they belong (the "Segment" column), and customer scores (the "Customer Score" column), while rows 904 contain information and data for each column. Selecting a customer in the Customer Name column brings up and displays the top five KPIs for that customer, generally shown at 906. In some embodiments, one or more data filters, generally shown at 908, in the form of drop-down lists may also be provided to allow the user to filter data, for example, by customer segment, country, region, customer name, and time period.

[0068] Figure 10 An exemplary KPI customer segmentation screen 1000 is shown, which can be selected via the KPI customer segmentation tab 608. Similar to the dashboard screen 800, the dashboard screen 1000 is in the form of a box tree diagram with boxes containing KPI data for a specific customer, in this case, the customer "CNW Electrical Balcatta Balcatta." Each box 1002 again displays data for a specific KPI for that customer. The size of each box again reflects the KPI's importance and impact for that customer, with larger boxes reflecting greater importance and impact, and vice versa. Thus, for example, box 1002 is larger than box 1004, meaning that the KPI represented by box 1002 has greater importance and impact than the KPI represented by box 1004. As previously discussed, different colors, such as green or red (or shades thereof), can be used to indicate whether a box reflects good, poor, or neutral KPI performance. In the example shown, box 1002 has a red color, reflecting poor KPI performance, while box 1004 has a green color, reflecting good KPI performance. Additionally, one or more data filters 1006 may be provided in the form of drop-down lists to allow the user to filter data by, for example, customer segment, country, region, customer name, and time period.

[0069] Figure 11 An exemplary top 10 customers by country screen 1100 is shown, which can be selected via the top 10 customers by country tab 612. As the name suggests, this dashboard screen 1100 provides a list of the top 10 customers sorted by the country to which they deliver. The customers are listed in several columns, generally shown at 1102, including columns for geographic region, country of delivery, customer ranking, customer name, customer segment, overall customer perception (i.e., customer score) and SS, number of responses, and sales order value. In some embodiments, one or more data filters 1104 can be provided in the form of drop-down lists to allow the user to filter the data, for example, by country, region, segment, and time period.

[0070] Thus far, a number of specific embodiments have been shown and described with respect to the delivery KPI analysis and alert system 102 and its application 210. Now following is a description of a general method that may be used by or with the analysis and alert system 102 and its application 210.

[0071] refer to Figure 12 , a flowchart 1200 is shown that represents a method that can be used by or with the delivery KPI analysis and alerting system 102 and its application 210. The method generally begins at block 1202 by obtaining performance data for a predefined set of indicators for a plurality of customers. The performance data includes historical performance data (e.g., 12 months) for the predefined set of indicators and current performance data (e.g., this month) for the predefined set of indicators. In some embodiments, the predefined set of indicators can be a predefined set of delivery KPIs, although alternative performance indicators can be used.

[0072] At block 1204, a net customer satisfaction score for the plurality of customers may be obtained. In some embodiments, the net satisfaction score may be a historical (eg, 12-month) NSSoD score for the plurality of customers.

[0073] At block 1206, cluster analysis is performed based on the historical performance data for the predefined group metric. The cluster analysis groups customers into clusters based on similarities in their experiences, as reflected by the historical performance data for the predefined group metric. In some embodiments, cluster analysis is performed using K-means clustering or a similar ML-based algorithm. In some embodiments, each customer belongs to a single cluster, and clustering is performed based on customer segmentation so that customers from different segments are not mixed together.

[0074] At block 1208, an impact analysis is performed using the historical performance data for the predefined set of indicators and the current performance data for the predefined set of indicators for the selected customer. The impact analysis determines whether each indicator in the predefined set of indicators has a negative impact, a positive impact, or a neutral impact on the selected customer. In some embodiments, the impact analysis can be performed using equations (1)-(5) above.

[0075] At block 1210, a correlation analysis is performed using the historical performance data for the predefined set of indicators and the customer net satisfaction scores. The correlation analysis provides an importance value for each indicator in the predefined set of indicators for the selected customer.

[0076] At block 1212, an indicator score is generated for each indicator in the predefined set of indicators based on the indicator's importance value and the indicator's impact on the selected customer. In some embodiments, the indicator score may be generated using equation (6) above.

[0077] At block 1214, a customer score is generated for the selected customer based on the sum of the indicator scores for the predefined set of indicators and the sum of the importance values for the predefined set of indicators. The customer score provides an indication of the likelihood that the selected customer will increase or decrease their net satisfaction score based on the current performance data for the predefined set of indicators. In some embodiments, the customer score can be generated using equation (7) above.

[0078] At block 1216, one or more interactive displays are generated based on the cluster analysis, correlation analysis, influence analysis, and customer scores. The one or more interactive displays allow a user to selectively display the results of the cluster analysis, correlation analysis, influence analysis, and / or customer scores. In some embodiments, the one or more interactive displays display the influence analysis using boxes of different colors and / or sizes.

[0079] If the current performance data for any metric indicates a negative impact on the selected customer, an alert may be issued and corrective action may be taken at block 1218. In some embodiments, the corrective action may include locking certain user privileges for the selected customer, such as access to the customer's account, or disabling order entry for the customer until the user takes steps to clear the customer's alert flag.

[0080] Although multiple embodiments have been disclosed and described herein, it should be understood that the above description is illustrative and not restrictive. Many other embodiments will be apparent upon reading and understanding the above description, and modifications and variations may be made within the scope of the appended claims. Accordingly, the specification and drawings should be regarded as illustrative and not restrictive. Therefore, the scope of the present disclosure should be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.

Claims

1. A system for improving customer satisfaction, comprising: processor; a display unit coupled to the processor; as well as a memory unit, which is accessed by the processor, the memory unit storing thereon an application that, when executed by the processor, causes the system to: Obtaining performance data of predefined group indicators of multiple customers, the performance data including historical performance data of the predefined group indicators and current performance data of the predefined group indicators; performing an impact analysis using historical performance data of the predefined group of indicators for the plurality of customers and current performance data of the predefined group of indicators for the selected customer, the impact analysis determining whether each indicator in the predefined group of indicators has a negative impact, a positive impact, or a neutral impact on the selected customer; as well as In response to determining that the indicator has a negative impact on the selected customer, a corrective action is performed related to the selected customer.

2. The system according to claim 1, wherein: The plurality of customers constitute a customer cluster, the customer cluster being selected from a plurality of customer clusters, each customer cluster being defined by the application, the application causing the system to perform cluster analysis based on historical performance data of the predefined set of indicators.

3. The system according to claim 2, wherein: The application also enables the system to: obtaining historical customer net satisfaction scores for the plurality of customers, each net satisfaction score having a value within a predefined range of values; and A correlation analysis is performed using the historical performance data of the predefined set of indicators and the net customer satisfaction score, the correlation analysis providing an importance value for each indicator in the predefined set of indicators.

4. The system according to claim 3, wherein: The application further causes the system to generate a metric score for each metric in the predefined set of metrics based on the importance value of the metric and the impact of the metric on the selected customer.

5. The system according to claim 4, wherein: The application also causes the system to generate a customer score for the selected customer based on the sum of the indicator scores for the predefined set of indicators and the sum of the importance values for the predefined set of indicators, the customer score providing an indication of the likelihood that the selected customer will increase or decrease their net satisfaction score based on current performance data for the predefined set of indicators.

6. The system according to claim 5, wherein: The application further causes the system to generate one or more interactive display screens based on the cluster analysis, correlation analysis, influence analysis, and customer scores, the one or more interactive display screens allowing a user to selectively display results of the cluster analysis, correlation analysis, influence analysis, and / or customer scores.

7. The system according to claim 6, wherein: The one or more interactive displays display the impact analysis using boxes having different colors and / or different sizes.

8. A method for improving customer satisfaction, comprising: Obtaining performance data of predefined group indicators of multiple customers, the performance data including historical performance data of the predefined group indicators and current performance data of the predefined group indicators; performing an impact analysis using historical performance data of the predefined group of indicators for the plurality of customers and current performance data of the predefined group of indicators for the selected customer, the impact analysis determining whether each indicator in the predefined group of indicators has a negative impact, a positive impact, or a neutral impact on the selected customer; as well as In response to determining that the indicator has a negative impact on the selected customer, a corrective action is performed related to the selected customer.

9. The method according to claim 8, wherein The plurality of customers constitute a customer cluster, the customer cluster being selected from a plurality of customer clusters, each customer cluster being defined by performing cluster analysis based on historical performance data of the predefined set of indicators.

10. The method according to claim 9, further comprising: obtaining historical customer net satisfaction scores for the plurality of customers, each net satisfaction score having a value within a predefined range of values; and A correlation analysis is performed using the historical performance data of the predefined set of indicators and the net customer satisfaction score, the correlation analysis providing an importance value for each indicator in the predefined set of indicators. 11 . The method of claim 10 , further comprising generating a metric score for each metric in the predefined set of metrics based on the metric's importance value and the metric's impact on the selected customer.

12. The method of claim 11 , further comprising generating a customer score for the selected customer based on the sum of the indicator scores for the predefined set of indicators and the sum of the importance values for the predefined set of indicators, the customer score providing an indication of the likelihood that the selected customer will increase or decrease their net satisfaction score based on current performance data for the predefined set of indicators.

13. The method of claim 12, further comprising generating one or more interactive display screens based on the cluster analysis, correlation analysis, influence analysis, and customer scores, the one or more interactive display screens allowing a user to selectively display results of the cluster analysis, correlation analysis, influence analysis, and / or customer scores.

14. The method of claim 13, further comprising displaying the impact analysis on the one or more interactive displays using boxes having different colors and / or different sizes.

15. A computer-readable medium storing computer-readable instructions for causing a processor to: Obtaining performance data of predefined group indicators of multiple customers, the performance data including historical performance data of the predefined group indicators and current performance data of the predefined group indicators; performing an impact analysis using historical performance data of the predefined group of indicators for the plurality of customers and current performance data of the predefined group of indicators for the selected customer, the impact analysis determining whether each indicator in the predefined group of indicators has a negative impact, a positive impact, or a neutral impact on the selected customer; as well as In response to determining that the indicator has a negative impact on the selected customer, a corrective action is performed related to the selected customer.

16. The computer-readable medium of claim 15, wherein: The plurality of customers constitute a customer cluster, the customer cluster being selected from a plurality of customer clusters, each customer cluster being defined by the computer-readable instructions that perform cluster analysis based on historical performance data of the predefined set of metrics.

17. The computer-readable medium of claim 16, wherein: The computer-readable instructions further cause the processor to: obtaining historical customer net satisfaction scores for the plurality of customers, each net satisfaction score having a value within a predefined range of values; and A correlation analysis is performed using the historical performance data of the predefined set of indicators and the net customer satisfaction score, the correlation analysis providing an importance value for each indicator in the predefined set of indicators.

18. The computer-readable medium of claim 17, wherein: The computer-readable instructions further cause the processor to generate a metric score for each metric in the predefined set of metrics based on the metric's importance value and the metric's impact on the selected customer.

19. The computer-readable medium of claim 18, wherein: The computer-readable instructions further cause the processor to generate a customer score for the selected customer based on the sum of the indicator scores for the predefined set of indicators and the sum of the importance values for the predefined set of indicators, the customer score providing an indication of a likelihood that the selected customer will increase or decrease their net satisfaction score based on current performance data for the predefined set of indicators.

20. The computer-readable medium of claim 19, wherein: The computer-readable instructions further cause the processor to generate one or more interactive display screens based on the cluster analysis, correlation analysis, influence analysis, and customer scores, the one or more interactive display screens allowing a user to selectively display results of the cluster analysis, correlation analysis, influence analysis, and / or customer scores.

Images