A system for estimating the likelihood of a termination event

Abstract

Embodiments of the present disclosure relate to a system for estimating the likelihood of a termination event. In an implementation of the system for estimating the likelihood of a termination event, a computing device implements a termination system to receive observation data describing values ​​of a processing metric and an indication of a termination event. The values ​​of the processing metric are grouped into groups using a mixture model that represents the processing metric as a mixture of distributions. Parameters of the distribution for each of the groups are estimated, and a mixture ratio for each of the groups is also estimated. In response to receiving user input requesting an estimate of the likelihood of a termination event for a particular value of the processing metric, the termination system generates an indication of the estimate of the likelihood of a termination event for the particular value based on the density of the distribution at the particular value for each of the groups and the probability of including the particular value in each of the groups.

Description

Background Art

[0001] The "curse of dimensionality" is a significant challenge faced by computing devices that prevents the computing device's processing system from accurately processing input data that describes a relatively large number of values ​​or dimensions. These processing inaccuracies are indirectly observable in the results output by the computing device, which are also inaccurate. Because many practical applications involve high-cardinality input data, inaccurate results caused by this technical challenge are often observed. In one such example, if a treatment metric has a number of values ​​greater than a threshold number, it is impossible for the computing device to accurately calculate an estimate of the causal effect of the treatment metric on the result. This is because some of the values ​​have a very low probability of occurrence, and the conditional probabilities calculated by the computing device's processing system using these values ​​are inaccurate.

[0002] In order to overcome such accuracy limitations when faced with processing metric data describing several values ​​greater than a threshold, the computing device clusters the values ​​of the processing metric into several bins less than the threshold. For example, the processing system of the computing device is implemented to cluster 10,000 values ​​into 10 bins, so that each of the 10,000 values ​​is included in one of the 10 bins. The computing device then processes the bins of values ​​as input data to calculate estimates of causal effects (e.g., 10 estimates, rather than 10,000). However, once the computing device has clustered the values ​​of the processing metric into bins, conventional systems cannot accurately calculate the causal effect on the outcome of any particular value included in the bin. Instead, conventional computing systems are limited to accurately calculating one causal effect on the outcome of each bin of values, which is a disadvantage of these systems. Summary of the Invention

[0003] Techniques and systems are described for improving the operation of a computing device so that accurate computation can be performed on all values ​​described by input data, even when the number of values ​​is large enough to cause the "curse of dimensionality" using conventional systems. In an example, a computing device implements a termination system to receive observation data describing values ​​of a process metric, values ​​of a covariate metric, and an indication of a termination event. The processing system of the computing device groups the values ​​of the process metric into groups using a mixture model that represents the process metric as a mixture of distributions.

[0004] For example, a processing system of the computing device uses an expectation-maximization algorithm to calculate parameters and a mixture ratio of a distribution for each of the groups. The termination system automatically forms a directed acyclic graph without user intervention, the directed acyclic graph having a node for the treatment metric, a node for the covariate metric, and a node for the termination event. In one example, the termination system receives user input requesting an estimate of the likelihood of the termination event for a particular value of the treatment metric.

[0005] In response to receiving the user input, the termination system implements a processing system to calculate a distribution density at a specific value for each of the groups using the directed acyclic graph and the estimated parameters of the distribution for each of the groups. The processing system also calculates a probability of including the specific value in each of the groups using the directed acyclic graph and the estimated mixing ratio for each of the groups. The termination system generates an indication of an estimate of the likelihood of a termination event for display in a user interface based on the distribution density at the specific value for each of the groups and the probability of including the specific value in each of the groups.

[0006] This Summary introduces some concepts in a simplified form that are further described in the Detailed Description below. As such, this Summary is not intended to identify essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The detailed description is described with reference to the accompanying drawings. The entities shown in the drawings represent one or more entities, and thus reference to the singular or plural forms of the entities may be interchangeably made in the discussion.

[0008] Figure 1 is an illustration of an environment in an example implementation operable to employ digital systems and techniques for estimating the likelihood of a termination event as described herein;

[0009] Figure 2 depicts a system in an example implementation showing operation of a termination module for estimating likelihood of a termination event;

[0010] Figure 3A and Figure 3B An example of preprocessing observation data is shown;

[0011] Figure 4 depicts a system in an example implementation showing operation of an estimation module for estimating distribution parameters and mixture proportions;

[0012] Figure 5 Representations of the estimated distributions and mixture proportions are shown;

[0013] Figure 6A 、 Figure 6B 、 Figure 6C and Figure 6D An example of estimating the likelihood of a termination event is shown;

[0014] Figure 7is a flowchart depicting a procedure in an example implementation in which user input requesting an estimate of the likelihood of a termination event for a particular one of the values ​​of a process metric is received and an indication of the estimate of the likelihood of a termination event for the particular value is generated for display in a user interface;

[0015] Figure 8 An example of validating a system for estimating the likelihood of a termination event on a synthetic dataset is shown; and

[0016] Figure 9 An example system is shown that includes an example computing device, which represents one or more computing systems and / or devices for implementing the various techniques described herein. DETAILED DESCRIPTION

[0017] Overview

[0018] Due to the "curse of dimensionality," a processing system of a computing device cannot accurately process input data in scenarios where the input data describes a relatively large number of values ​​or dimensions. In an example, if a processing metric has a number of values ​​greater than a threshold number, the curse of dimensionality prevents the computing device from accurately calculating an estimate of the causal effect of the processing metric on the outcome. To overcome this, the processor system of the computing device clusters the values ​​of the processing metric into a number of bins less than the threshold number. The computing device then implements the processor system to estimate the causal effect of the bins of values, for example, using a causal discovery algorithm.

[0019] By estimating bins of values ​​rather than causal effects of values, the computing device reduces the dimensionality or number of values ​​described by the input data to avoid the curse of dimensionality. However, once the raw values ​​of the processing metric are clustered into bins, the computing device cannot accurately estimate the causal effects of the raw values. In contrast, the computing device of conventional systems is limited to accurately estimating a single causal effect per bin. This is a shortcoming of conventional systems and techniques.

[0020] To overcome limitations of conventional systems and thereby accurately estimate the causal effect of each process metric value described by high-cardinality input data, techniques and systems for estimating the likelihood of a termination event are described. In an example, a computing device implements a termination system to receive observation data describing a value of a process metric, a value of a covariate metric, and an indication of a termination event. For example, the termination system determines that the number of values ​​of the process metric is greater than a threshold number k. In response to this determination, the termination system implements a processor system of the computing device to group the values ​​of the process metric into groups using a mixture model such that the number of groups is less than the threshold number k.

[0021] The mixture model represents the process metric as a mixture of distributions. Unlike conventional systems that use only groups of values ​​to avoid the "curse of dimensionality," the termination system uses an expectation-maximization algorithm to estimate the parameters of the distributions in the mixture of distributions for each of the groups. The termination system also uses the expectation-maximization algorithm to estimate the mixture ratio for each of the groups. The estimated distribution parameters and mixture ratio for each of the groups can be used to estimate the termination event probability for each value of the process metric included in each of the groups.

[0022] To this end, in one example, the termination system uses a causal discovery algorithm and a group of process metric values ​​to automatically form a directed acyclic graph without user intervention. For example, the termination system uses a fast greedy equivalence search algorithm to form the directed acyclic graph. The directed acyclic graph includes nodes for process metrics, nodes for covariate metrics, and nodes for termination events.

[0023] In response to receiving user input requesting an estimate of the likelihood of a termination event for a specific value of a process metric, the termination system implements a processor system to calculate a distribution density at the specific value for each of the groups using the structure of the directed acyclic graph and the estimated parameters of the distribution for each of the groups. The processor system of the computing device also calculates a probability of including the specific value in each of the groups using the structure of the directed acyclic graph and the estimated mixing ratio for each of the groups. Next, the termination system calculates the estimated component for each of the groups as the product of the distribution density at the specific value and the probability of including the specific value in the group.

[0024] Unlike conventional systems, which are limited to estimating the causal effect of a group containing a specific value, the termination system uses the specific value to calculate an estimated component for each group. Each of these estimated components represents a portion of the causal effect of the specific value that is lost and cannot be recovered using conventional techniques. The termination system sums or combines the estimated components to form an estimate of the likelihood of a termination event for a specific value of the process metric. This estimate of the likelihood of a termination event for a specific value is accurate, and the termination system can accurately estimate the likelihood of a termination event for any value of the process metric, which is not possible using conventional systems.

[0025] By estimating the likelihood of a termination event using the probability of including a particular value in each of the groups, rather than using only one group that includes the particular value, the described system is able to accurately estimate the causal effect of all values ​​of the process metric, even if the number of values ​​is greater than a threshold number k. Furthermore, the described system estimates the likelihood of a termination event with increased accuracy relative to conventional systems. This technical improvement was validated using a real-world dataset and then further validated using a synthetic dataset.

[0026] Terminology Examples

[0027] As used herein, the term "termination event" refers to an event that defines or occurs at the end of a lifecycle, series, relationship, etc. Examples of termination events for a computing device include failure of the computing device, retirement of the computing device, replacement of the computing device, etc. Other examples of termination events include cancellation of service or membership, or loss of connection to a network.

[0028] As used herein, the term "metric" refers to a variable having a value that is expected to affect the occurrence or non-occurrence of a terminal event.

[0029] In the following discussion, an example environment is first described that employs examples of the techniques described herein. Example procedures are also described that can be performed in the example environment and other environments. Therefore, execution of the example procedures is not limited to the example environment, and the example environment is not limited to execution of the example procedures.

[0030] Sample Environment

[0031] Figure 1 1 is a diagram of an environment 100 in an example implementation operable to employ digital systems and techniques as described herein. The illustrated environment 100 includes a computing device 102 connected to a network 104. The computing device 102 can be configured as a desktop computer, a laptop computer, a mobile device (e.g., assuming a handheld configuration such as a tablet or mobile phone), etc. Thus, the computing device 102 can range from a full-resource device with large amounts of memory and processor resources (e.g., a personal computer, a game console) to a low-resource device with limited memory and / or processing resources (e.g., a mobile device). In some examples, the computing device 102 represents multiple different devices, such as multiple servers for performing operations "on the cloud."

[0032] Computing device 102 includes storage device 106 and termination module 108. For example, termination module 108 can access storage device 106 and can transmit and / or receive data via network 104. The illustrated environment 100 also includes a service system 110 and client devices 112, which are also connected to network 104. Although service system 110 is shown as a single service system 110, it should be understood that in some examples, service system 110 represents multiple service systems 110. For example, client device 112 represents multiple client devices 112.

[0033] The service system 110 includes a communication module 114 and the service system 110 implements the communication module 114 to transmit and / or receive data, for example, via the network 104. The data transmitted and / or received by the service system 110 is shown as interaction data 116. For example, the client device 112 is shown as including a communication module 118 that the client device 112 implements to transmit and / or receive interaction data 116 via the network 104.

[0034] In one example, service system 110 provides computing resources, such as storage capacity and / or processing resources, to client device 112 and other client devices 112. In this example, interaction data 116 describes interactions between client device 112 and service system 110 as part of client device 112's use of blocks of storage capacity provided by service system 110 and available to client device 112. For example, interaction data 116 describes blocks of storage capacity requested for use by client device 112, blocks of storage capacity provided by service system 110 and available for use by client device 112, an amount of storage capacity of service system 110 currently used by client device 112, and the like.

[0035] Service system 110 generates observation data 120 based on interaction data 116, and observation data 120 describes the value and indication of a metric for the termination event. For example, the amount of storage capacity of service system 110 currently used by client device 112 is a metric, and the value of the metric is the number of storage capacity blocks. Consider an example in which observation data 120 describes that client device 112 used eight storage capacity blocks on the first day, nine storage capacity blocks on the second day, and eight storage capacity blocks on the third day.

[0036] In this example, the termination event is characterized by the cessation of use of a block of storage capacity previously used by client device 112. Thus, the termination event occurs on the third day when client device 112 uses only eight blocks of storage capacity, rather than on the second day when client device 112 uses nine blocks of storage capacity. Consequently, server system 110 generates observation data 120 that depicts an indication of the absence of a termination event on the first and second days and an indication of a termination event on the third day.

[0037] Service system 110 transmits observation data 120 to computing device 102 via network 104. Computing device 102 implements termination module 108 to receive observation data 120, which is illustrated as being stored on storage device 106. Consider an example in which observation data 120 describes the value of a processing metric, the value of a covariate metric, and an indication of a termination event. The amount of storage capacity of service system 110 currently used by client device 112 is the processing metric, and the termination event is the discontinuation of use of a block of storage capacity previously used by client device 112.

[0038] In this example, for example, the value of the processing metric is eight blocks of storage capacity on the first day, nine blocks of storage capacity on the second day, eight blocks of storage capacity on the third day, and nine blocks of storage capacity for the next 97 days. The cardinality of the processing metric in this example (e.g., 100 possible values) is relatively high compared to the cardinality of the processing metric in the previous example (e.g., three possible values). Continuing with this example, the covariate metric is the average amount of storage capacity of the service system 110 currently being used by a group of client devices 112. The value of the covariate metric is 4.5 blocks of storage capacity on the first day, 5.7 blocks of storage capacity on the second day, 4.4 blocks of storage capacity on the third day, and 5.5 blocks of storage capacity for the next 97 days.

[0039] The termination module 108 groups the values ​​of the treatment metric into groups using a mixture model and also groups the values ​​of the covariate metric into additional groups using an additional mixture model. To this end, in one example, the termination module 108 uses the mixture model to model the distributions within the range of values ​​of the treatment metric, such that each distribution corresponds to one of the groups. To learn the parameters of the distributions and the mixture ratios, the termination module 108 utilizes, for example, an expectation-maximization algorithm.

[0040] In an example, termination module 108 clusters the values ​​of the process metric into groups using k-means clustering, which ensures that the number of groups is less than the number of values ​​of the process metric. In a first example, the values ​​of the process metric are uniformly grouped into groups such that each group in the group includes the same number of values. In a second example, the values ​​of the process metric are not uniformly grouped into groups such that the number of values ​​of the process metric included in a first group of the groups is different from the number of values ​​of the process metric included in a second group of the groups.

[0041] After the values ​​of the process metrics are grouped into groups and the values ​​of the covariate metrics are grouped into additional groups, the termination module 108 forms a directed acyclic graph having nodes for the process metrics, nodes for the covariate metrics, and nodes for the termination events. If the structure of the causal Bayesian network of the observation data 120 is known, the termination module 108 converts the directed acyclic graph into a structure having a causal Bayesian network. If the structure of the causal Bayesian network of the observation data 120 is unknown, the termination module 108 uses a fast greedy equivalence search algorithm to form a directed acyclic graph. The termination module 108 then uses the directed acyclic graph to estimate the conditional probabilities of the nodes of the directed acyclic graph. These conditional probabilities represent the distribution of nodes given the value of the parent node of the node.

[0042] like Figure 1As shown in FIG, the termination module 108 receives input data 122 describing a user input requesting an estimate of the likelihood of a termination event for a particular value in the value of a process metric. The termination module 108 uses the directed acyclic graph, the distribution associated with each of the groups, and the probability of including the particular value in each of the groups to generate estimate data 124. The estimate data 124 describes an indication of the estimate of the likelihood of a termination event for the particular value.

[0043] The termination module 108 disconnects the edges of the directed acyclic graph that are incoming edges to the node for the processing metric and sets the value of the processing metric to be equal to the specific value indicated in the user input. The termination module then performs probabilistic logical sampling, for example, to estimate the required probability. To generate the estimation data 124, the termination module 108 calculates the product of the probability of including the value in each of the groups and the density of the distribution associated with each of the groups at the specific value of the processing metric.

[0044] For example, termination module 108 calculates the estimated component for each group in the group as the product of the distribution density at the specific value and the probability of including the specific value in each group in the group. Termination module 108 then sums or combines the estimated component for each group in the group to determine the estimation of the possibility of the termination event for the specific value. By calculating the estimated component for each group in the group in this way, termination module 108 utilizes each group in the group to determine the estimation of the possibility of the termination event. This is in contrast to the conventional technology that only utilizes the specific group in the group that comprises the specific value to determine the estimation of the possibility of the termination event. By determining the estimation of the possibility of the termination event based on the probability of including the specific value in each group in the group rather than just based on the specific group in the group that comprises the specific value, termination module 108 improves the accuracy of estimation. Even if the cardinality of processing metric is relatively high in this example, this is also possible.

[0045] Figure 2 Depicted is a system 200 in an example implementation illustrating the operation of the termination module 108. The termination module 108 is shown as including a pre-processing module 202, an estimation module 204, a graph module 206, an inference module 208, and a display module 210. The pre-processing module 202 receives the observation data 120 and the input data 122, and the pre-processing module 202 processes the observation data 120 to generate processed data 212.

[0046] Figure 3A and Figure 3B An example of pre-processing observation data 120 is shown. Figure 3A A representation 300 of the observation data 120 is shown. Figure 3B A representation 302 of the processed data 212 is shown. Figure 2 and Figure 3A, the pre-processing module 202 receives observation data 120 , which describes values ​​and indications of metrics of termination events.

[0047] As shown in representation 300, observation data 120 describes a process metric 304. Process metric 304 includes process values ​​t1-t9. Observation data 120 also describes a first covariate metric 306, a second covariate metric 308, and a third covariate metric 310. First covariate metric 306 includes first covariate values ​​x11-x19; second covariate metric 308 includes second covariate values ​​x21-x29; and third covariate metric 310 includes third covariate values ​​x31-x39. Observation data 120 further describes a termination event 312 including termination event indications y1-y9. For example, process metric 304 is a variable of interest, and termination event 312 is an outcome of interest that is affected by the values ​​of first covariate metric, second covariate metric, and third covariate metric 306-310.

[0048] Consider an example in which the number of first covariate values ​​x11-x19 is greater than a threshold k; the number of second covariate values ​​x21-x29 is greater than a threshold k; and the number of third covariate values ​​x31-x39 is greater than a threshold k. In this example, the threshold k represents a number large enough to cause inaccuracies associated with the "curse of dimensionality." For example, some of the first covariate values ​​x11-x19 are associated with very small probabilities of occurrence, and the conditional probabilities estimated based on these low probabilities of occurrence are inaccurate.

[0049] To estimate the likelihood of the termination event 312 for the process values ​​t1-t9 of the process metric 304, the pre-processing module 202 determines that the number of process values ​​t1-t9 is greater than a threshold k; the number of first covariate values ​​x11-x19 is greater than a threshold k; the number of second covariate values ​​x21-x29 is greater than a threshold k; and the number of third covariate values ​​x31-x39 is greater than a threshold k. In one example, the pre-processing module 202 expresses this as:

[0050] V∈(T,X1,X2,X3}

[0051] Wherein: T represents the treatment metric 304; X1 represents the first covariate metric 306; X2 represents the second covariate metric 308; X3 represents the third covariate metric 310.

[0052] For example, the preprocessing module 202 fits a mixture model of V with k components. The preprocessing module 202 then uses the mixture distribution to map all elements of V to a new column d(V) with values ​​{1, ..., k}. Figure 3BAs shown in , representation 302 includes processed values ​​t1-t9 grouped into groups 314-320 such that the number of groups 314-320 is less than or equal to a threshold value k. For example, pre-processing module 202 performs k-means clustering on processed values ​​t1-t9 to cluster processed values ​​t1-t9 into groups 314-320. In this example, each of groups 314-320 is a cluster formed by performing k-means clustering on processed values ​​t1-t9.

[0053] For example, group 314 includes processed values ​​t1 and t2; group 316 includes processed values ​​t3 and t4; group 318 includes processed values ​​t5, t6, and t7; and group 320 includes processed values ​​t8 and t9. The pre-processing module 202 also performs k-means clustering on the first covariate values ​​x11-x19 to cluster the first covariate values ​​x11-x19 into groups 322-326. Similarly, the pre-processing module 202 performs k-means clustering on the second covariate values ​​x21-x29 to cluster the second covariate values ​​x21-x29 into groups 328-332, and the pre-processing module 202 performs k-means clustering on the third covariate values ​​x31-x39 to cluster the third covariate values ​​x31-x39 into groups 334-338. For example, the number of groups 322-326 is less than or equal to a threshold k; the number of groups 328-332 is less than or equal to a threshold k; and the number of groups 334-338 is less than or equal to a threshold k.

[0054] 36 includes third covariate values ​​x33, x34, x35, and x36; and group 338 includes third covariate values ​​x37, x38, and x39. For example, the termination event indicates that y1-y9 are not grouped into groups. The pre-processing module 202 generates processed data 212 that describes the representation 302.

[0055] The estimation module 204 receives the processed data 214 and processes the processed data 214 to generate the distribution data 214 . Figure 4 Depicted is a system 400 in an example implementation showing the operation of the estimation module 204. The estimation module 204 is shown as including a parameter module 402 and an expectation-maximization module 404. The parameter module 402 receives the processed data 212 and processes the processed data 212 to generate parameter data 406.

[0056] Figure 5A representation 500 of the estimated distribution and mixture proportions is shown. As shown, the parameter module 402 processes the processed data 214 to generate parameter models 502-508 for the groups 314-320. For example, the parameter module 402 generates a parameter model 502 for the group 314; a parameter model 504 for the group 316; a parameter model 506 for the group 318; and a parameter model 508 for the group 320. The parameter module 402 generates parameter data 406 that describes the parameter models 502-508 for the groups 314-320.

[0057] refer to Figure 4 and Figure 5 , the expectation maximization module 404 receives the parameter data 406 and processes the parameter data 406 to generate the distribution data 214. To this end, in one example, the expectation maximization module 404 implements an expectation maximization algorithm and uses the expectation maximization algorithm to estimate the distributions 510-516 and the mixing proportions 518-524. As shown, the distribution 510 and the mixing proportions 518 of the treatment values ​​t1 and t2 included in the group 314 are estimated; the distribution 512 and the mixing proportions 520 of the treatment values ​​t3 and t4 included in the group 316 are estimated; the distribution 514 and the mixing proportions 522 of the treatment values ​​t5, t6, and t7 included in the group 318 are estimated; and the distribution 514 and the mixing proportions 524 of the treatment values ​​t8 and t9 included in the group 320 are estimated. The expectation maximization module 404 generates the distribution data 214 describing the distributions 510-516 and the mixing proportions 518-524.

[0058] like Figure 2 As shown in , the graph module 206 receives distribution data 214 and processes the distribution data 214 to generate graph data 216 . Figure 6A 、 Figure 6B 、 Figure 6C and Figure 6D An example of estimating the likelihood of a termination event is shown. Figure 6A A representation 600 of a directed acyclic graph is shown formed from the known structure of a causal Bayesian network. Figure 6B A representation 602 of a directed acyclic graph formed from a known structure of a causal Bayesian network or from performing a fast greedy equivalence search on the processed data 212 is shown. Figure 6C A representation 604 of the calculated estimate components is shown. Figure 6D A representation 606 of an indication of an estimated likelihood of a termination event for a metric value is shown.

[0059] refer to Figure 6AAn example of a known structure of the causal Bayesian network 608 includes a node 304 for the treatment metric, a node 306 for the first covariate metric, a node 308 for the second covariate metric, and a node 310 for the third covariate metric. In this example, the conditional probabilities of the known structure of the causal Bayesian network 608 are also known (e.g., the probability of any node given its parent node is known), and the likelihood 312 of the termination event for treatment values ​​t1-t9 can be directly calculated. If these conditional probabilities are unknown, the graph module 206 generates a modified structure of the causal Bayesian network 610.

[0060] To this end, graph module 206 assumes that the effects of a relatively high-cardinality variable V on its descendants flow through a relatively low-cardinality representation d(V) of the relatively high-cardinality variable V. Based on this assumption, graph module 206 infers a new node d(V) for the known structure of causal Bayesian network 608, such that for any descendant W of V, the edge V→W is equivalent to V→d(V)→W. For example, the new node d(V) is a complete mediator of the causal effects of V on other variables. Graph module 206 also assumes that given V, the relatively low-cardinality representation d(V) of the relatively high-cardinality variable V is independent of V's ancestors. For example, graph module 206 also assumes that V is in the only parent of d(V).

[0061] The graph module 206 adds d(V) as nodes 612-618 to the known structure of the causal Bayesian network 608 to generate a modified structure of the causal Bayesian network 610. For example, the graph module 206 adds a node d(T) 612 to the node that processes the metric 304, so that the node d(T) 612 has only the node that processes the metric 304 as a parent. Similarly, the graph module 206 adds a node d(X1) 614 to the node for the first covariate metric 306, such that the node d(X1) 614 has only the node for the first covariate metric 306 as a parent; and the graph module 206 adds a node d(X2) 616 to the node for the second covariate metric 308, such that the node d(X2) 616 has only the node for the second covariate metric 308 as a parent; and the graph module 206 adds a node d(X3) 618 to the node for the third covariate metric 310, such that the node d(X3) 618 has only the node for the third covariate metric 310 as a parent. For example, node d(T) 612 represents groups 314-320; node d(X1) 614 represents groups 322-326; node d(X2) 616 represents groups 328-332; and node d(X3) 618 represents groups 334-338.

[0062] To calculate the causal effect or likelihood of the terminating event 312, the graph module 206 utilizes the modified structure of the causal Bayesian network 610 to calculate P(Y|do(T=t)). To do this, in one example, the graph module 206 assumes that d(T) takes values ​​in a set {1, ..., k}, which can be expressed as:

[0063]

[0064] Therefore, the graph module 206 uses P(Y|do(d(T)=i)) and P(d(t)=i|T=t) (where i∈{1,...,k}) to calculate the likelihood of the termination event 312. For the second term:

[0065]

[0066] Where: for j∈{1,...,k}, f i represents the density f T|d(t)＝j ; π j represents the probability P(d(T)=j); f is the density function of T.

[0067] The graph module 206 estimates P(Y|do(d(T)=j)), f for all j∈{1,...,k} i and π j , which means estimating P(Y|do(T=t) for all possible values ​​t of T. For example, the graph module 206 models V as a mixture of k distributions (e.g., relatively small and fixed constant distributions) and models d(V) as a variable that captures the index of the distribution from which the sample comes, or a hidden variable of the mixture model. In an example, this can be expressed as:

[0068] For i∈{1,...,k}: V~f i if d(V)=i

[0069] Where: f1, ..., f k are k distributions within the range of values ​​of V; and f i Use a well-known distribution (e.g., Gaussian distribution) and learn its parameters and the mixture ratio π using techniques such as the expectation-maximization algorithm i =P(d(V)=i) to model.

[0070] In order to estimate f j and π j , the graph module 206 processes the distribution data 214 to assume that f j The parametric model is then used to recover f using techniques such as the expectation-maximization algorithm. j and π jFor example, when a node V takes a relatively large number of values, the graph module 206 assumes that V is a node with parameters (μ j , σ j )(where j∈[k]). In this example, the expectation maximization algorithm receives as input the data in the column corresponding to V, and then the expectation maximization algorithm outputs different parameters (μ j , σ j ) and the mixing ratio π j Estimates.

[0071] To estimate P(Y|do(d(T)=j)), the graph module 206 generates graph data 216 describing the formed directed acyclic graph 620. Figure 6B As shown in , if the modified structure of the causal Bayesian network 610 is known, the graph module 206 generates the formed directed acyclic graph 620 by replacing V with d(V) for variables with a number of values ​​greater than k.

[0072] If the modified structure of the causal Bayesian network 610 is unknown, the graph module 206 generates d(V) using a mixture model as previously described, which is applied to the observation data 120 to generate columns corresponding to d(V). In this example, the graph module 206 directly uses a causal discovery algorithm, such as a fast greedy equivalence search algorithm, to learn the formed directed acyclic graph 620. The graph module 206 uses the formed directed acyclic graph 360 to estimate the conditional probability table and then generates graph data 216 describing the formed directed acyclic graph 620 and the conditional probability table.

[0073] like Figure 2 As shown in FIG, the inference module 208 receives the graph data 216 describing the formed directed acyclic graph 360 and the conditional probability table. The inference module 208 also receives the input data 122 describing a user input requesting an estimate of the likelihood of the termination event 312 for a particular value of the process values ​​t1-t9 of the process metric 304. For example, the inference module 208 processes the graph data 216 and / or the input data 122 to generate the estimate data 124.

[0074] To this end, the inference module 208 performs a do-calculus and uses the do operator to disconnect the edges of the formed directed acyclic graph 620, which are the incoming edges of the node d(T) 612. The inference module 208 sets the disconnected edges equal to j in the conditional probability table and uses a probabilistic logical sampling algorithm to estimate the likelihood of the termination event 312 for the specific value of the processing metric 304. The inference module 208 generates the estimate data 124 describing the likelihood of the termination event 312 for the specific value of the processing metric 304.

[0075] For example, rather than estimating the likelihood of the termination event 312 for a particular value of the process metric 304 based on any one of the groups 314-320 containing the particular value of the process metric 304, the reasoning module 208 considers each of the groups 314-320. Figure 6C , the inference module 208 utilizes the distributions 510-516 and the mixture proportions 518-524 to estimate the likelihood of the termination event 312 for a particular value of the process metric 304. To do so, in one example, the inference module 208 determines a density 622 of the distribution 510 of the group 314 at a particular value of the process metric 304; a density 624 of the distribution 512 at a particular value of the process metric 304; a density 626 of the distribution 514 at a particular value of the process metric 304; and a density 628 of the distribution 516 at a particular value of the process metric 304.

[0076] For example, the inference module 208 also determines a probability 630 that the particular value of the process metric 304 is included in the group 314 using the mixture proportion 518; determines a probability 632 that the particular value of the process metric 304 is included in the group 316 using the mixture proportion 520; determines a probability 634 that the particular value of the process metric 304 is included in the group 318 using the mixture proportion 522; and determines a probability 636 that the particular value of the process metric 304 is included in the group 320 using the mixture proportion 524. Next, the inference module 208 calculates estimated components 638-644 for the groups 314-320. The inference module 208 calculates the estimated component 638 of group 314 as the product of density 622 and probability 630; calculates the estimated component 640 of group 316 as the product of density 624 and probability 632; calculates the estimated component 642 of group 318 as the product of density 626 and probability 634; and calculates the estimated component 644 of group 320 as the product of density 628 and probability 636.

[0077] The reasoning module 208 combines or sums the estimate components 638-644 as the likelihood of the termination event 312 for a particular value of the process metric 304. The reasoning module 208 then generates estimate data 124 describing the likelihood of the termination event 312 for the particular value of the process metric 304. The display module 210 receives the estimate data 124 and processes the estimate data 124 to generate an indication of the likelihood of the termination event 312 for the particular value of the process metric 310 for display in a user interface. Although described with respect to one particular value of the process metric 304, it should be understood that in some examples, the reasoning module 208 generates estimate data 124 describing the likelihood of the termination event 312 for all values ​​of the process metric 304. Consider an example in which the input data 122 describes a user input requesting an estimate of the causal effect of the process metric 304 on the termination event 312.

[0078] Continuing with the previous example, reasoning module 208 determines that treatment value t1 is included in group 314; however, in this example, reasoning module 208 considers not only group 314 but also other groups 316-320 when estimating the likelihood of termination event 312 for treatment value t1. To do this, reasoning module 208 calculates the product of the density of distribution 510 of group 314 at treatment value t1 and the probability that treatment value t1 is included in group 314; the product of the density of distribution 512 of group 316 at treatment value t1 and the probability that treatment value t1 is included in group 316; the product of the density of distribution 514 of group 318 at treatment value t1 and the probability that treatment value t1 is included in group 318; and the product of the density of distribution 516 of group 320 at treatment value t1 and the probability that treatment value t1 is included in group 320. For example, reasoning module 208 adds the calculated products as the likelihood of termination event 312 for treatment value t1.

[0079] The reasoning module 208 repeats this process for process values ​​t2, t3, t4, t5, t6, t7, t8, t9 and then generates estimate data 124 describing the likelihood of the termination event 312 for process values ​​t1-t9. The display module 210 receives the estimate data 124 and processes the estimate data 124 to generate an indication of the estimate of the causal effect of the process metric 304 on the termination event 312 for display in the user interface. Examples of these indications for different process metrics are shown in FIG. Figure 6D is shown in .

[0080] As shown, representation 606 includes an estimated causal effect on the termination event 312 for a first processing metric 646 and an estimated causal effect on the termination event 312 for a second processing metric 648. The estimated causal effect on the termination event 312 for the first processing metric 646 includes a first processing metric value 650 and a corresponding probability 652. Results of a baseline method 654 are shown along with the results of the described method 656. The baseline method 654 is limited to estimating metrics at the group level for which the number of possible values ​​is greater than a threshold value k.

[0081] For example, if baseline method 654 is used to estimate the causal effect of treatment metric 304 on termination event 312, the estimate will be limited to the estimated likelihood of termination event 312 for groups 314-320. In this example, the estimated likelihood of termination event 312 using baseline method 654 will be the same for treatment value t1 and treatment value t2. Similarly, using baseline method 654, treatment values ​​t3 and t4 will have the same estimated likelihood; treatment values ​​t5, t6, and t7 will have the same estimated likelihood; and treatment values ​​t8 and t9 will have the same estimated likelihood.

[0082] As shown for the baseline method 654, the first process metric values ​​650 ranging from 40 to approximately 90 each correspond to a probability of approximately 0.265 for the termination event 312. The first process metric values ​​650 ranging from approximately 91 to 140 each correspond to a probability of approximately 0.225 for the termination event 312. As shown for the described method 656, the first process metric values ​​650 are no longer limited to corresponding probabilities 652 for the group of first process metric values ​​650. Instead, for the described method 656, the first process metric values ​​650 ranging from 40 to 140 have corresponding probabilities 652 ranging from 0.265 to approximately 0.225.

[0083] For example, serving system 110 provides analytics support for client device 112 and other client devices 112, and interaction data 116 describes interactions between client device 112 and other client devices 112 and serving system 110 as part of the analytics support. In this example, termination event 312 is termination of analytics support provided by serving system 110 by client device 112. For example, first processing metric 646 is project shares and first processing metric value 650 is the number of project shares.

[0084] Continuing with this example, and because baseline method 654 is limited to estimating the likelihood of a group, the results of baseline method 654 indicate the same likelihood of termination event 312 for numbers of project shares ranging from 40 project shares to approximately 90 project shares. With respect to baseline method 654, increasing the number of project shares from 90 to approximately 91 is illustrated as corresponding to an approximately 15% decrease in the likelihood of termination event 312. Furthermore, this decreased likelihood of termination event 312 does not change for numbers of project shares ranging from approximately 91 to 140. Overall, this indicates that the results produced by baseline method 654 may not be accurate.

[0085] However, the results of the described method 656 indicate that for numbers of project shares in the range of 40 project shares to approximately 110 project shares, the likelihood of the termination event 312 gradually decreases. For numbers of project shares exceeding 110, the results of the described method 656 indicate approximately the same likelihood of the termination event 312 as the results of the baseline method 654 indicate for increases from approximately 90 project shares to approximately 91 project shares. Thus, the results of the described method 656 indicate a realistic scenario in which the likelihood of the termination event 312 gradually decreases as the number of project shares increases until a certain threshold number (e.g., approximately 110 project shares) is reached, above which additional numbers of project shares have little effect on the likelihood of the termination event 312. Therefore, the results of the described method 656 may be more accurate than the results of the baseline method 654.

[0086] The estimated causal effect of the second process metric 648 on the termination event 312 includes a second process metric value 658 and a corresponding probability 660. Results of a baseline method 662 are shown along with the results of the described method 664. The baseline method 662 is limited to estimating metrics at the group level for metrics with a number of possible values ​​greater than a threshold k.

[0087] As shown for the baseline method 662, the second process metric values ​​658 ranging from 0 to approximately 45 each correspond to a probability of approximately 0.274 for the termination event 312. For the baseline method 662, the second process metric values ​​658 ranging from approximately 46 to approximately 110 each correspond to a probability of approximately 0.265 for the termination event 312. As also shown, the second process metric values ​​658 ranging from approximately 111 to approximately 160 each correspond to a probability of approximately 0.267 for the termination event 312, and the second process metric values ​​658 ranging from approximately 161 to approximately 250 each correspond to a probability of approximately 0.243 for the termination event 312.

[0088] Unlike the baseline method 662, the described method 664 is not limited to estimating the likelihood of the termination event 312 for a group. As shown for the described method 664, the probability 660 ranges from approximately 0.274 to approximately 0.243 for second processing metric values ​​658 between 0 and approximately 250. For example, the results of the described method 664 are similar to the results of the baseline method 662; however, the results of the described method 664 provide more information than the results of the baseline method 662 because the baseline method 662 is limited to estimating the likelihood of the termination event 312 for a group.

[0089] Consider an example in which termination event 312 is termination of analytics support provided by service system 110 by client device 112, second processing metric 648 is a unique user identifier (ID), and second processing metric value 658 is the number of unique user IDs. In this example, the results of baseline method 662 indicate that the likelihood of termination event 312 is the same for approximately 46 unique user IDs as for approximately 110 unique user IDs. The results of baseline method 662 also indicate that an increase from approximately 160 unique user IDs to approximately 161 unique user IDs corresponds to an approximately 9% decrease in the likelihood of termination event 312. For example, an approximately 9% change in the likelihood of termination event 312 based on a less than 1% change in the number of unique user IDs indicates that the results of baseline method 662 may be inaccurate.

[0090] The results of described method 664 appear more likely to be accurate than the results of baseline method 662. For example, rather than the approximately 9% decrease with the addition of one unique user ID presented in the results of baseline method 662, the results of described method 664 reflect a similar decrease in the likelihood of termination event 312, but this decrease occurs gradually, starting at approximately 125 unique user IDs and ending at approximately 175 unique user IDs. The results of described method 664 indicate a realistic scenario in which the likelihood of termination event 312 gradually decreases as the number of unique user IDs gradually increases. Therefore, for the second processing metric 648, the results of described method 664 appear more accurate than the results of baseline method 662.

[0091] Although the examples are described with respect to estimating the likelihood of a termination event 312, it should be understood that the described system is not limited to estimating the likelihood of a termination event. For example, the described system can be used to estimate the effect of a new drug on recovery from a particular disease. In another example, the described system can be used to estimate the effect of a collection of fonts on website engagement.

[0092] In general, the functions, features, and concepts associated with the above and following examples are employed in the context of the example processes described in this section. Furthermore, the functions, features, and concepts described with respect to the different figures and examples in this document are interchangeable with each other and are not limited to implementation in the context of a particular figure or process. Furthermore, the blocks associated with the different representative processes and corresponding figures herein may be applied individually, together, and / or in combination in different ways. Therefore, the various functions, features, and concepts described with respect to the different example environments, devices, components, figures, and processes herein may be used in any suitable combination and are not limited to the specific combinations represented by the examples listed in this specification.

[0093] Example Process

[0094] The following discussion describes techniques that can be implemented using the previously described systems and devices. Aspects of each process can be implemented in hardware, firmware, software, or a combination thereof. These processes are shown as a collection of blocks that specify operations performed by one or more devices and are not necessarily limited to the order shown for performing the operations by the respective blocks. In portions of the following discussion, reference is made to Figure 1 -Figure 6. Figure 7 is a flowchart depicting a process 700 in an example implementation in which user input requesting an estimate of the likelihood of a termination event for a particular value in the value of a processing metric is received, and an indication of the estimate of the likelihood of a termination event for the particular value is generated for display in a user interface.

[0095] Observation data describing values ​​of a process metric, values ​​of a covariate metric, and an indication of a termination event is received (block 702). In an example, the computing device 102 implements the termination module 108 to receive the observation data. The values ​​of the process metric are grouped using a mixture model that represents the process metric as a mixture of distributions (block 704). In one example, the termination module 108 groups the values ​​of the process metric into groups.

[0096] An expectation maximization algorithm is used to estimate the parameters of the distribution in the mixture of distributions for each of the groups and the mixture ratio for each of the groups (block 706). For example, the termination module 108 estimates the parameters of the distribution and the mixture ratio for each of the groups in the groups. A directed acyclic graph having a node for the treatment metric, a node for the covariate metric, and a node for the termination event is formed (block 708). In an example, the computing device 102 implements the termination module 108 to form the directed acyclic graph.

[0097] Receive user input requesting an estimate of the likelihood of a termination event for a particular value in the value of the process metric (block 710). In one example, the termination module 108 receives the user input. Calculate the distribution density at the particular value for each group in the group, and calculate the probability of including the particular value in each group in the group (block 712). For example, the termination module 108 calculates the distribution density at the particular value for each group in the group and the probability of including the particular value in each group in the group. Generate an indication of the estimate of the likelihood of a termination event for the particular value based on the distribution density at the particular value for each group in the group and the probability of including the particular value in each group in the group for display in a user interface (block 704). In some examples, the computing device 102 implements the termination module 108 to generate the indication for display in the user interface.

[0098] Figure 8 An example 800 is shown in which a system for estimating the likelihood of a termination event is validated on a synthetic data set. The example 800 includes a directed acyclic graph 802 and a summary of results 804. For example, the synthetic data set is generated using a structural equation model and includes a treatment variable T, a covariate X, and an outcome variable Y. In the example, this can be represented as:

[0099] X:=ε X ; T: =ω XT X+ε T ; T: = Bernoulli(σ(ω XY X+ω TY T+ε Y ))

[0100] Where: X , ε T and ε Yare independent standard Gaussian variables, representing the external noise affecting X, T, and Y respectively; variables X and T are continuous and have a relatively large value range; variable Y is binary; ω XT 、ω XY and ω TY represents the model weights and is randomly selected from a uniform distribution between 0 and 1.

[0101] As shown, the directed acyclic graph 802 includes a node 806 for T; a node 808 for X, and a node 810 for Y. The result summary 804 includes metric values ​​812 and corresponding probabilities 814 for the ground truth 816, the baseline method 818, and the described method 820. The termination module 108 generates 50,000 samples using the directed acyclic graph 802 and uses these samples to estimate the causal effect of T on Y as P(Y=1|do(T=t)). Since the dataset is synthetic, the ground truth 816 is estimated using the do operator, and the edge of the directed acyclic graph 802 is disconnected from X to T, T is set to t, and P(Y=1) is estimated. The termination module 108 groups the metric values ​​812 into 10 groups of equal size. To ensure a fair comparison, d(T) and d(X) use the same value, where k=10. Since X and T are continuous, f i Modeled as a Gaussian N(μ i , σ i ), and the required Gaussian mixture model is to use the provided μ i , σ i , π i (where i∈{1,...,k}) is learned using the expectation-maximization algorithm.

[0102] As shown in the result summary 804, the estimate generated using the described method 820 is closer to the ground truth 816 than the estimate generated using the baseline method 818. This is because the baseline method 818 is limited to estimating only a set of metric values ​​812, and the described method 820 is not limited in this manner. The mean squared error (MSE) between the result of the described method 820 and the ground truth 816 is 9.3e -3 , and the MSE between the results of the baseline method 818 and the ground truth 816 is 15.4e -3 Thus, the described method 820 demonstrates significant improvements over the baseline method 818 on the synthetic dataset.

[0103] Example systems and devices

[0104] Figure 9An example system 900 is shown that includes an example computing device, which represents one or more computing systems and / or devices that can be used to implement the various techniques described herein. This is illustrated by the inclusion of a termination module 108. Computing device 902 includes, for example, a server of a service provider, a device associated with a client (e.g., a client device), a system on a chip, and / or any other suitable computing device or computing system.

[0105] The example computing device 902 shown in the figure includes a processing system 904, one or more computer-readable media 906, and one or more I / O interfaces 908 that are coupled to each other in communication. Although not shown, the computing device 902 also includes a system bus or other data and command transmission system that couples various components to each other. For example, the system bus includes any one or combination of different bus structures, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and / or a processor or local bus that utilizes any of a variety of bus architectures. Various other examples are also contemplated, such as control lines and data lines.

[0106] Processing system 904 represents functionality that performs one or more operations using hardware. Thus, processing system 904 is shown as including hardware elements 910 configured as processors, functional blocks, and the like. This includes example implementations using hardware as application-specific integrated circuits or other logic devices formed using one or more semiconductors. Hardware elements 910 are not limited by the materials from which they are formed or the processing mechanisms employed therein. For example, a processor may include (a plurality of) semiconductors and / or transistors (e.g., an electronic integrated circuit (IC)). In such a context, processor-executable instructions are, for example, electronically executable instructions.

[0107] The computer-readable medium 906 is shown as including a memory / storage device 912. The memory / storage device 912 represents the memory / storage device capacity associated with one or more computer-readable media. In one example, the memory / storage device 912 includes volatile media (such as random access memory (RAM)) and / or non-volatile media (such as read-only memory (ROM), flash memory, optical disks, magnetic disks, etc.). In another example, the memory / storage device 912 includes fixed media (e.g., RAM, ROM, fixed hard drive, etc.) and removable media (e.g., flash memory, removable hard drive, optical disks, etc.). The computer-readable medium 906 can be configured in various other ways, as further described below.

[0108] Input / output interface(s) 908 represent functionality that allows a user to input commands and information to the computing device 902 and also allows information to be presented to the user and / or other components or devices using various input / output devices. Examples of input devices include a keyboard, a cursor control device (e.g., a mouse), a microphone, a scanner, touch functionality (e.g., a capacitive or other sensor configured to detect physical touch), a camera (e.g., which employs visible or invisible wavelengths such as infrared frequencies to recognize motion as gestures that do not involve touch), and the like. Examples of output devices include a display device (e.g., a monitor or projector), speakers, a printer, a network card, a tactile response device, and the like. Thus, the computing device 902 can be configured in various ways to support user interaction, as further described below.

[0109] Various techniques are described herein in the general context of software, hardware elements, or program modules. Typically, such modules include routines, programs, objects, elements, components, data structures, etc. that perform specific tasks or implement specific abstract data types. As used herein, the terms "module," "function," and "component" generally refer to software, firmware, hardware, or a combination thereof. Features of the techniques described herein are platform-independent, meaning that these techniques can be implemented on a variety of commercial computing platforms with a variety of processors.

[0110] An implementation of the described modules and techniques may be stored on or transmitted via some form of computer-readable media. For example, computer-readable media includes various media accessible by computing device 902. By way of example and not limitation, computer-readable media includes "computer-readable storage media" and "computer-readable signal media."

[0111] "Computer-readable storage media" refers to media and / or devices that are capable of storing information persistently and / or non-transitorily, as opposed to simple signal transmissions, carrier waves, or signals themselves. Thus, computer-readable storage media refers to non-signal-bearing media. Computer-readable storage media include hardware, such as volatile and non-volatile removable and non-removable media and / or storage devices implemented in a method or technology suitable for storing information such as computer-readable instructions, data structures, program modules, logic elements / circuits, or other data. Examples of computer-readable storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other storage technology, CD-ROM, digital versatile disks (DVDs) or other optical storage devices, hard disks, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or other storage devices, tangible media, or articles of manufacture suitable for storing desired information and that can be accessed by a computer.

[0112] "Computer-readable signal media" refers to signal-bearing media configured to transmit instructions to the hardware of computing device 902, such as via a network. Signal media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, data signal, or other transport mechanism. Signal media also includes any information delivery media. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media.

[0113] As previously described, hardware elements 910 and computer-readable media 906 represent modules, programmable device logic, and / or fixed device logic implemented in hardware that, in some embodiments, may be used to implement at least some aspects of the technology described herein, such as to execute one or more instructions. Hardware includes integrated circuits or systems on a chip, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), and other implemented components in silicon or other hardware. In this context, hardware operates as a processing device that performs program tasks defined by instructions and / or logic embodied by hardware and hardware for storing instructions for execution, such as the computer-readable storage media previously described.

[0114] Combinations of the foregoing may also be used to implement the various techniques described herein. Thus, software, hardware, or executable modules may be implemented as one or more instructions and / or logic embodied on some form of computer-readable storage medium and / or embodied by one or more hardware elements 910. For example, the computing device 902 is configured to implement specific instructions and / or functions corresponding to the software and / or hardware modules. Thus, the implementation of a module that may be executed as software by the computing device 902 may be at least partially implemented in hardware, for example, by using a computer-readable storage medium and / or hardware elements 910 of the processing system 904. Instructions and / or functions may be executed / operated by one or more articles of manufacture (e.g., one or more computing devices 902 and / or processing systems 904) to implement the techniques, modules, and examples described herein.

[0115] The techniques described herein can be supported by various configurations of computing device 902 and are not limited to the specific examples of the techniques described herein. The functionality can also be implemented in whole or in part through the use of a distributed system, such as through the "cloud" 914 as described below.

[0116] Cloud 914 includes and / or represents a platform 916 of resources 918. Platform 916 abstracts the underlying functionality of the hardware (e.g., servers) and software resources of cloud 914. For example, resources 918 include applications and / or data used when performing computer processing on servers remote from computing device 902. In some examples, resources 918 also include services provided over the Internet and / or over a subscriber network such as a cellular or Wi-Fi network.

[0117] The platform 916 abstracts resources 918 and functionality to connect the computing device 902 with other computing devices. In some examples, the platform 916 is also used to abstract the scaling of resources to provide corresponding scaling levels to meet the demand for resources implemented via the platform. Therefore, in the interconnected device embodiment, the implementation of the functionality described herein can be distributed throughout the system 900. For example, the functionality can be implemented partially on the computing device 902 and via the platform 916 that abstracts the functionality of the cloud 914.

[0118] in conclusion

[0119] Although implementations of a system for estimating the likelihood of a termination event have been described using language specific to structural features and / or methods, it should be understood that the appended claims are not necessarily limited to the specific features or methods described. Rather, specific features and methods are disclosed as example implementations of a system for estimating the likelihood of a termination event, and other equivalent features and methods are intended to fall within the scope of the appended claims. Furthermore, various different examples are described, and it should be understood that each described example can be implemented independently or in combination with one or more other described examples.

Claims

1. A method implemented by a computing device in a digital media environment for estimating the likelihood of a termination event, the method comprising: receiving, by a pre-processing module, observation data describing a value of a treatment metric, a value of a covariate metric, and an indication of said termination event; grouping, by the pre-processing module, the values ​​of the process metric into groups using a mixture model that represents the process metric as a mixture of distributions; estimating, by an estimation module, parameters of a distribution in the mixture of distributions for each of the groups using an expectation-maximization algorithm; estimating, by the estimation module, a mixture ratio for each of the groups using the expectation-maximization algorithm; forming, by a graph module, a directed acyclic graph having a node for the processing metric, a node for the covariate metric, and a node for the termination event; receiving, by a reasoning module, a user input requesting an estimate of the likelihood of the termination event for a particular one of the values ​​of the process metric; calculating, by the inference module, a distribution density at the specific value for each of the groups using the directed acyclic graph and the parameters of the distribution of the mixture of distributions for each of the groups; calculating, by the inference module, a probability of including the specific value in each of the groups using the directed acyclic graph and the mixing ratio for each of the groups; as well as An indication of the estimate of the likelihood of the termination event for the specific value is generated by the reasoning module based on the distribution density at the specific value for each group in the groups and the probability of including the specific value in each group in the groups for display in a user interface. The method of claim 1 , wherein the directed acyclic graph is formed using a causal discovery algorithm. 3 . The method of claim 1 , further comprising grouping the values ​​of the covariate measure into additional groups using additional mixed models, wherein the number of additional groups is equal to the number of groups. The method of claim 1 , further comprising evaluating a conditional probability table for each node of the directed acyclic graph. 5 . The method of claim 1 , further comprising calculating the estimated component for each of the groups as a product of the distribution density at the specific value and the probability of including the specific value in each of the groups.

6. The method of claim 5, further comprising combining the estimate components into the estimate of the likelihood of the termination event for the particular value. The method of claim 1 , wherein grouping the values ​​of the processing metric into the groups comprises performing k-means clustering. The method of claim 1 , wherein the mixture model is a Gaussian mixture model.

9. The method of claim 1, further comprising performing probabilistic logical sampling as part of generating the indication of the estimate of the likelihood of the termination event for a particular value.

10. A system comprising: A pre-processing module, implemented at least in part in hardware of a computing device, to: receiving observational data describing a value of a treatment measure, a value of a covariate measure, and an indication of a termination event; determining that a number of the values ​​of the processing metric is greater than a threshold number; as well as grouping the values ​​of the process metric into groups using a mixture model that represents the process metric as a mixture of distributions, the number of groups being less than the threshold number; an estimation module, implemented at least in part in the hardware of the computing device to: estimating parameters of a distribution in the mixture of distributions for each of the groups using an expectation-maximization algorithm; as well as estimating a mixture ratio for each of the groups using the expectation-maximization algorithm; a graph module implemented at least in part in the hardware of the computing device to form a directed acyclic graph having a node for the processing metric, a node for the covariate metric, and a node for the termination event; as well as an inference module, implemented at least in part in the hardware of the computing device to: receiving user input requesting an estimate of the likelihood of the termination event for a particular one of the values ​​of the process metric; calculating a distribution density at the specific value for each of the groups using the directed acyclic graph and the parameters of the distribution of the mixture of the distributions for each of the groups; calculating a probability of including the specific value in each of the groups using the directed acyclic graph and the mixing ratio for each of the groups; as well as An indication of the estimate of the likelihood of the termination event for the particular value is generated for display in a user interface based on the directed acyclic graph, the distribution associated with each of the groups, and the probability of including the particular value in each of the groups.

11. The system of claim 10, wherein the pre-processing module is further implemented to group the values ​​of the covariate measure into additional groups using additional mixing modules, the number of the additional groups being equal to the number of groups.

12. The system of claim 10, wherein the directed acyclic graph is formed using a causal discovery algorithm.

13. The system of claim 10, wherein the inference module is further implemented to calculate the estimated component for each of the groups as a product of the distribution density at the specific value and the probability of including the specific value in each of the groups.

14. The system of claim 13, wherein the reasoning module is further implemented to combine the estimation components into the estimate of the likelihood of the terminating event for the particular value.

15. The system of claim 10, wherein the reasoning module is further implemented to perform probabilistic logical sampling as part of generating the indication of the estimate of the likelihood of the termination event for a particular value.

16. One or more computer-readable storage media comprising instructions stored thereon that, in response to execution by a computing device, cause the computing device to perform operations comprising: receiving observational data describing a value of a treatment measure, a value of a covariate measure, and an indication of a termination event; determining that a number of the values ​​of the processing metric is greater than a threshold number; grouping the values ​​of the process metric into groups using a mixture model that represents the process metric as a mixture of distributions, the number of groups being less than the threshold number; estimating parameters of a distribution in the mixture of distributions for each of the groups using an expectation-maximization algorithm; estimating a mixture ratio for each of the groups using the expectation-maximization algorithm; forming a directed acyclic graph having a node for the processing metric, a node for the covariate metric, and a node for the termination event; receiving user input requesting an estimate of the likelihood of the termination event for a particular one of the values ​​of the process metric; calculating a distribution density at the specific value for each of the groups using the directed acyclic graph and the parameters of the distribution of the mixture of the distributions for each of the groups; calculating a probability of including the specific value in each of the groups using the directed acyclic graph and the mixing ratio for each of the groups; as well as An indication of the estimate of the likelihood of the termination event for the particular value is generated for display in a user interface based on the distribution density at the particular value for each of the groups and the probability of including the particular value in each of the groups.

17. One or more computer-readable storage media according to claim 16, wherein the operation further comprises calculating the estimated component for each of the groups as the product of the distribution density at the specific value and the probability of including the specific value in each of the groups.

18. The one or more computer-readable storage media of claim 17, wherein the operations further comprise combining the estimate components into the estimate of the likelihood of the termination event for the particular value.

19. The one or more computer-readable storage media of claim 16, wherein the directed acyclic graph is formed using a causal discovery algorithm.

20. The one or more computer-readable storage media of claim 16, wherein grouping the values ​​of the processing metric into the groups comprises performing k-means clustering.

Images