Method of mining behavioral patterns for conformance diagnostics

Abstract

Computer-implemented method of performing conformance diagnostics of traces of an event log with a net representing a model of a process, the process comprising predetermined steps, the process comprising predetermined allowed variants of sequences of activities, the event log comprising traces, each trace being representative of a sequence of observed activities, the traces being representative of actual sequences of activities, each activity being a step of the process, the net being representable by a labeled accepting Petri net, comprising: - Instantiating a predetermined set of candidate constraints out of a library comprising predefined templates of constraints, a template being a function that returns a constraint for each k-tuple of activities, each constraint being defined as an object describing a language; - determining, a set of constraints which are satisfied by all traces in the Petri net's accepted language using a predetermined elation holds; - removing redundant constraints from the set of satisfied constraints by searching for a minimal set of redundancy-free constraints; - verifying the event log by checking whether the minimum set of constraints is satisfied by the traces σ of the event log.

Description

[0001] 1

[0002] Method of Mining Behavioral Patterns for Conformance Diagnostics

[0003] Introduction and Background

[0004] The present invention relates to a method of mining behavioral patterns for conformance diagnostics.

[0005] Process mining [1] is the field of computer science combining traditional process

[0006] 5 science with data science to analyze event data. A core task of process mining is conformance checking [5], which consists of comparing desired behavior (i.e., modeled in some process modeling formalism) and observed behavior (i.e., as captured in the event data) to quantify their differences and identify frequent patterns of deviation. In conformance checking, process models may be of a procedural or declarative

[0007] 10 nature. Procedural models, e.g., BPMN diagrams [9] , describe the exact sequence of steps that are allowed in a process, i.e., a closed-world assumption. Declarative models, e.g., DECLARE

[0024] , specify only the constraints upon which a process must operate, i.e., an open-world assumption.

[0008] Procedural process models are widely available in organizations, e.g., the SAP refer¬

[0009] 15 ence model

[0013] . Organizations maintain large repositories of procedural models that have been designed for enactment or documentation purposes. Declarative models are used less frequently in practice. While their perceived ease of use is graded positively by practitioners

[0026] , their open- world assumption often yields too flexible or incomplete models.

[0010] 20 Both modeling formalisms can be used for conformance checking [2, 16]. In general, conformance checking artifacts based on procedural models, e.g., trace alignments, require a significant cognitive effort from the user to be interpreted . To illustrate that, we consider the purchase process from Figure 2. The process starts with the approval of the purchase (a). After that, the payment must be booked (b) and 2 the goods must be collected (c), which can happen in any order. Only two activity sequences are allowed: (Approve Purchase, Book Payment, Collect Goods') and (Approve Purchase, Collect Goods, Book Payment'), but even for this simple process, state-of-the-art procedural techniques such as trace alignments produce con-

[0011] 5 fusing results. To see that, consider optimal alignments from Table 1.

[0012] Table 1: Example log with all optimal alignments and violating constraints. An optimal alignment returns the minimal number of activity insertions ( - ) and deletions ( / ) needed to make a trace fit the given model.

[0013] Trace Optimal Alignments Violated Constraints

[0014] Trace p ( Approve Purchase, Approve Purchase, Book Payment'), has 2 issues: the purchase is approved twice and the goods are not collected. These are directly captured by constraints C2, and C^'c. Techniques such as trace alignments suggest edits to "fix" constraints C2, and C^'c. Removing one Approve Purchase ($) fixes C£ and 10 inserting a Collect Goods (Q) fixes but these edits can occur at difference places of the trace, leading to 5 possible optimal alignments.

[0015] Similarly, trace t2(Book Payment, Approve Purchase) has two issues: the payment is booked before approval and the goods are not collected. These are indicated by the violated constraints C“, Cf'6, and CgA The trace also has five optimal align-

[0016] 15 ments. It can be "fixed" by either moving a to an earlier point (I • • • >x ( fixes and Cg’6) and inserting <■ ixes Cg’6), as in the first three alignments; or moving b to a later point $ . . . b (fixes and Cg’6) and inserting (fixes Cg’6), as in the last two alignments.

[0017] In the examples above, multiple edit operations might be related to the Hx of a

[0018] 20 single constraint. Similarly, multiple constraints might be fixed by a single edit op- eration. Furthermore, the same edit type might mean different things depending on 3 the context, e.g. the deletion $ for and t2- Consequently, the only way of obtaining insights on the patterns of deviations is by manually inspecting each trace. In sum- mary, the diagnostics provided by trace-alignments are low-level, non-deterministic, and its interpretation depends on context. In contrast, the equivalent declarative

[0019] 5 model is able to directly explain the deviations in a more understandable form by pointing out the violated business constraints.

[0020] In contrast, conformance diagnostics computed for declarative models are more suit- able for direct interpretation, yet, suffer from the aforementioned flexibility and incompleteness problems.

[0021] 10 The most basic form of conformance diagnostics consists of quantifying behav- ioral differences between process models and event logs in terms of a single metric [2, 15, 21, 27, 28]. Some methods go beyond that to provide insights into the nature of deviations. In

[0027] and

[0028] , diagnostics are provided as a set of differences in the behavioral profiles of the log and the model. Techniques based on alignments [2] 15 provide diagnostics as a set of moves (insertions or deletions) on a trace, corresponding to undesired or missing behavior. In

[0021] , decomposition techniques are used to detect regions of the model that are non-conforming. Similarly,

[0015] detects problematic subsets of activities by comparing log and model projections. These techniques offer too low-level feedback, often limited to a set of activities / edits, and do not 20 effectively explain the deviations.

[0022] The issue with low- level feedback has been known for long. In [3], the authors of the alignment technique present a method to detect higher-level deviation patterns via high-level alignments. However, this requires users to specify each pattern to be detected.

[0023] 25 Summary of the Invention

[0024] Therefore, it is an object of the present invention to provide a method of uncovering patterns of deviation of traces of an event log from a process model, which method alleviates at least partially the aforementoiuned problems. 4

[0025] This object is solved by a method according to the claims.

[0026] Thus, the present invention provides a novel conformance checking framework that generates declarative constraints from a procedural input model, which are subse- quently used to present conformance diagnostics in a declarative fashion.

[0027] 5 In comparison to known approaches described above, the method of the present invention requires no further user input besides the model and the log. Similarly, authors in

[0020] propose a set of deviation patterns that can be detected on top of a purpose-built multi-layer synchronous product net using trace alignments. The approach incorporates the resource and privacy perspectives and can provide the

[0028] 10 context in which deviations occurs. Since the technique is still based on alignments, it suffers from some of its limitations such as non-determinism and poor scalability.

[0029] Furthermore, it is unclear how this approach can be extended to add new patterns.

[0030] Short Description of the Figures

[0031] Figure 1 illustrates a model of a purchase process;

[0032] 15 Figure 2 illustrates a schematic overview of the framework of the invention;

[0033] Figure 3 illustrates projected nets; and

[0034] Figure 4 gives an overview over the user interface.

[0035] Detailed Description of the Invention

[0036] Figure 2 presents a schematic overview of an embodiment of the present invention.

[0037] 20 As input, the user provides a procedural model and an event log. The tool internally maintains a constraint template library consisiting of high-level descriptions of constraints, i.e., at the meta-level, to be checked against the reference model. The procedural model is used to compute constraint instances based on the template library (Steps A. and B.). For example, assume that the library contains the tem¬

[0038] 25 plate Init (x) . If the procedural model specifies that some activity, e.g., activity 5 a, should always be executed first, the constraint Init (a) is instantiated. The resuit is an "equivalent" declarative version of the originally procedural model. Then, techniques from the field of declarative process mining are used to prune redundant constraints (C) and verify the log to provide diagnostics (D).

[0039] 5 Using the discovered constraints to analyze the log ensures that the returned diagnostics are on a higher level, and, thus, more understandable than other automated techniques. Furthermore, the approach presents a series of interesting properties such as determinism, monotonicity of the reported deviations, and flexibility to add new patterns. The approach is implemented as a standalone tool and evaluated on two

[0040] 10 real-life datasets, where it is shown to be scalable and to generate nearly complete diagnostics. Some of the main advantages of the present invention are:

[0041] The present invention provides a framework to generate declarative-like conformance diagnostics on top of procedural models. Further, a method is provided for the efficient verification of a subclass of constraint patterns, ensuring the scalability of

[0042] 15 the approach.

[0043] Preliminaries

[0044] First, basic mathematical notations are defined. Familiarity with automata

[0011] and Petri net [6] theories is assumed. We use Uato denote the universe of activity labels and T Uato denote the invisible (empty) label. A trace is a finite sequence of

[0045] 20 activities a G Uf. An erent log is a multi-set of traces L G B(U*^ describing the sequences of observed activities and their frequencies. Similarly, a process model M is a set M G describing the set of allowed variants. For this work, we consider models defined as Petri nets and note that BPMN diagrams can be converted into

[0046] Petri nets.

[0047] 25 Definition 1. (Labeled Accepting Petri Net) A labeled accepting Petri net is a tuple N = (F, T, F, Z, mo, my) where P is the set of places, T the set of transitions, 6 function, and m0, mj G B(P') are its initial and final markings. Transitions t for which l(t) = T are called invisible transitions.

[0048] The state of a labeled accepting Petri net N called marking, is a multiset of places m G S(F). It is possible to move from one state into another

[0049] 5 by firing enabled transitions using the occurrence rule

[0011] . For a firing sequence a (ti, • • • , tn) G T* , we write m\cr)m' to denote that a is enabled at m G B(P') and that firing transitions in a in sequence leads to m! G S(F). The state space of a Petri net is the set of states reachable by firing any enabled firing sequence and its accepted language is the set C(N') = {Z(cr) | a G

[0050] 10 T* , where Z is the concatenation of the labels of transitions in a.

[0051] Figure 2 shows a Petri net N accepting the language {(a, b, c), {a, c, b}} with |7£(7V)| = 6. This work focuses on the class of regular languages, which is the class of languages that can be recognized by a Deterministic Finite Automaton.

[0052] 15 Definition 2. (Deterministic Finite Automaton (DFA)) A Deterministic Finite Automaton (DFA ) is a 5-tuple D = (Q, E, 6, q0, where Q is the set of states, E is the alphabet, 6 : Q x E — > Q is the transition function, q0is the initial state and Qf C Q is the set of final states.

[0053] A minimal DFA is a DEAD (Q, E, 6, qo, Qf) such that there does not exist any

[0054] 20 other DEAD' (Q’ , E, 5’ , q0' , Q'j) such that C(D) = C(D') and |Q| > |Q'|.

[0055] That is, there does not exist another DFA with less states accepting the same lan- guage. "Minimal" here means "minimal number of states Q".

[0056] A trace a («i, • • • , czn) G U* is accepted by aa DDEFAA DD = (Q, E, 5, q0, if there exists a sequence of states and snG F.

[0057] 25 We call the set C(D) = {A G LI* | a is accepted by £>} its language. In general, for a 7

[0058] Petri net N, if |7£(7V)| < oc, i.e., its state space is finite, then £(N) can be expressed by a DFA called its behavioral automaton.

[0059] Definition 3. (Behavioral Automaton) The behavioral automaton of a bounded labeled accepting Petri net N = (P, T, F, Z, m0, my) is the unique (up to state renam¬

[0060] 5 ing) minimal DPA BN= (Q, Xf d,q0, Qj) such that £(N) = £(BNf

[0061] Mining Behavioral Patterns

[0062] This section details the framework steps from Figure 2. For the remainder, we consider Ua= {a,b, c} and the event log a)]. We first formalize the concepts of constraints, constraint templates and constraint template library:

[0063] 10 Definition 4. (Constraint) A constraint c is an object describing a language all constraints as Lie-

[0064] A core idea of the present invention is centered around the automatic instantiation of constraints from a set of constraint templates (called a template library).

[0065] 15 Definition 5. (Constraint Template) A constraint template is a function C : UQ — > Lie returning a constraint for each k-tuple of activities (di, - - - , Qfc) € Llf,

[0066] Definition 6. (Constraint Template Library) Let C be the universe of con- straint templates. A template library L, is a set of templates L C C.

[0067] 20 The present invention focuses on models expressed as bounded Petri nets, i.e., Petri nets with finite state spaces, and constraints expressing regular languages. The DECLARE

[0024] language is an example of a set of constraints that can be expressed using regular expressions. Table 2 shows an example constraint template library 8 containing three constraint templates Lx expressed as regular expressions. It is to be noted that in practice, more extensive libraries are used, x, y denote the constraint parameters in the templates, and k denotes the number of template parameters. Replacing the template parameters x, y with concrete ac-

[0068] 5 tivities a constraint. For example, Ca(a, b) returns G Uc describing all traces where a and b occur in one-to-one succession. Furthermore, each constraint is associated to a verbalization that describes it in human-understandable way.

[0069] The term Lexicographic Order is used as follows:

[0070] Given a set of totally ordered elements E (i.e. each two elements in E can be

[0071] 10 compared). Given two(m + l)-sized sequences of elements in Si, S2 in E. Then, si s2in lexicographical order if and only if there exists an index z such that

[0072] 1 z <= m + 1 such that Si [z] < s2[z] and sx[j] = s2[j] for ally z. That is, there exists an index z such that all both sequences are equal up to index z — 1 and the z-th entry of sxis smaller (according to the total order) than the i-th entry of s^.

[0073] 15 In the present invention: E is the set set of constraints. The total order is the user- provided constraint ordering order. The sequences sxand s2are the permutations of (m + 1) constraints.

[0074] Permutations of constraints are ordered such that permutations with smaller constraints come first.

[0075] 20 Step A: Instantiating Constraints

[0076] The first step of the framework (A) is to instantiate candidate constraints from the template library following an instantiation strategy. The most straightforward instantiation strategy is the brute force instantiation of all templates using all pa- rameter combinations, but notice that more sophisticated instantiation strategies

[0077] 25 are also possible. the brute force instan- tiation yields the set of constraints Inst = J <Rz na

[0078] Wl ? * * * 1 ^2 ? * * * 1 ^2? ^3 1 1 c?6}. 9

[0079] Table 2: Example constraint template library Li = {6i, 62, 63} with templates expressed as regular expressions, x, y denote the constraint parameters and k the number of template parameters.

[0080] Template k Verbalization RegEx

[0081] Ci Init(a?) I The case must start with {: / :} x.*

[0082] 6*2 AtMostl(a?) 1 {: / :} occurs at most [1] times (!x)*x(!x)*

[0083] 6*3 AltSucc(ay y) 2 {$}, {? / } occur in 1:1 alternating succession (!(x|y) | (x(!(x|y))*y))

[0084] Step B: Computing Satisfied Constraints

[0085] From the set of instantiated constraints in the previous step, we are interested in the constraints that are satisfied by the model. A constraint is satisfied by a process model, if and only if the constraint holds for all traces in the model’s accepted

[0086] 5 language. For the Petri net of Figure 2 and the instantiated set Inst , the set of satisfied constraints is Sat

[0087] Definition 7. (Satisfying Constraint) Let c E UCbe a constraint and let M be a process model, c is said to be satisfied by M <t=^- £(M) C 6(c).

[0088] For a bounded Petri net N with behavioral automaton BN and constraint c expressed

[0089] 10 as a regular expression with associated minimal DFA Bcit holds that £(N) C 6(c) <t=^- £(BN) C £(BC). Checking this condition is in (9(|B57V| * \BC\ * |7 / a|)

[0011] . Therefore, one can check if a constraint is satisfied by a model as long as the model and the constraint’s corresponding DFAs can be obtained.

[0090] Scalable Conformance Diagnostics with / -invariant Constraints

[0091] 15 As discussed in the previous section, it is possible to check if a constraint is satisfied by computing the Petri net’s behavioral automaton. This becomes expensive for large

[0092] Petri nets due to the state explosion problem. In this section, we introduce a class of constraints for which constraint checking is significantly accelerated. The method consists of reducing the Petri net’s state space by pruning irrelevant activities. We

[0093] 20 start with the definition of B -invariance for a constraint’s language, which intuitively 10 means that modifying a trace by inserting or removing activities in F does not influence language inclusion.

[0094] Definition 8. (P -Invariance) Let c G Uc and let L C Ua. c is P -invariant if for

[0095] 5 P -invariance is closely related to the notion of stutter-invariance and next-free formulas in model checking

[0023] . In DECLARE, F-invariant constraints are essentially constraints that do not express a directly-follows relation.

[0096] In our running example, constraint Ci) is {&, c}-invariant, i.e., inserting or removing activities b and c to a trace does not change whether C2, holds for that trace. Mean¬

[0097] 10 while, C“ is not F-invariant for any L C Utt. Lemma 1 below shows that one can compute the maximal set F for which a constraint c is F -invariant by looking into the self-loops of c’s corresponding minimal DEA.

[0098] Lemma 1. (P -Invariance and Self Loops) Let c G Uc be a constraint and let

[0099] Proof. The <= direction is straightforward to see. We prove the =>• direction by contradiction. Assuming that c is F-invariant but 3q, q' E Q, q E L,q / f s.t. d(y, 7) = q'. Let x E U* be a prefix leading to state q. Then xy leads to state q'. From the minimality of Dc, it follows that q and q' correspond to different equivalence

[0100] 20 classes of the Nerode equivalence. Therefore, there exists a suffix z E U * such that either But this contradicts F -invariance.

[0101] Definition 9. (Projecting Petri nets) Let N (F, T, F, Z, m0, my) be a labeled 11

[0102] It holds that £(AR) = { / r R | a G Zf(W)}, i.e., the accepted language of the projected Petri net is the projection of the language of the original net in A. Furthermore, oftentimes BNACC BBNN.- We can use this fact to speed up the validity check of P- invariant constraints using the lemma below.

[0103] 5 Lemma 2. (Validity Check for P -invariant Constraints) Let c E UQ be P -

[0104] Proof. =>• follows from £(A^a\p) C £(N\ <= follows from / -hi variance of c.

[0105] Lemma 2 means that we can check if a P -invariant constraint c is satisfied by a Petri net N by checking if it is satisfied by NUa\p. For the template library of Table 2,

[0106] 10 constraint C) is (b, c}-invariant. By projecting Petri net 7VXfrom Figure 2 into {a}, we obtain the Petri net 2Viwfrom Figure 3. By applying language preserving reduction rules

[0022] , we obtain the reduced net . C) is satisfied by 7V-[ . Therefore, it follows from Lemma 2 that it is also satisfied by AR Notice that 6, while 2. This reduction in the state space size can lead to dramatic

[0107] 15 speedups as will be shown in Section 6.

[0108] Step C: Pruning Redundant Constraints

[0109] The result of the previous step is the set of satisfied constraints for the given model. However, some constraints are redundant. For the satisfied constraints Sat, C) ({a} occurs at most [1] times) and C)'b({a}, {&} occur in 1:1 alternating succession)

[0110] 20 imply C*2 ({&} occurs at most [1] times). Redundant constraints produce redundant diagnostics. Therefore, we add a redundancy resolution step.

[0111] Definition 10. (Redundant Constraints) Let C C UG. A constraint c is redun

[0112] We are interested in a minimal set of redundancy-free constraints (in our running

[0113] 25 example, is such a set). Formally, for C C Uc, we want 12

[0114] Algorithm 1: An approximate model minimization algorithm. i input A set of constraints C , a total ordering criterion a maximum intersection size m

[0115] 2 output A set of constraints C C C s.t. C' =>• C

[0116] 3 C G- c-

[0117] 4 foreach

[0118] 5 | if {cx, • • • , Cm} — >• cm+i then C' G- C' \ {cm+i } ;

[0119] 6 return C' to solve the optimizations-t- Vc G C, C — > c. In the declarative process mining literature, there exist approaches to prune redundant constraints [?]. However, existing techniques require the intersection of all constraints’ DFAs, which

[0120] 5 Instead, to ensure the scalability of the pipeline, we use an approximated approach specified in algorithm 1. The algorithm tries to remove as many constraints as possible. The approach orders all constraints according to a predefined order [7] (the order is specific to the template library and hence configured by the tool provider / vendor) and checks all combinations (cx, c2, • • • , <^, (^+1) G C"'1s.t. ct<cc Ccjj for all 1 <

[0121] 10 z < j < m + 1. If cm+1is found to be redundant, it is removed. The algorithm’s correctness follows directly from the fact that we use a total ordering criterion, which ensures that we do not introduce cycles of constraint implications. Furthermore, its complexity is in , i.e. a polynomial with exponent m in the maximum constraint DFA size and the number of constraints.

[0122] In our running example, the set of redundancy free constraints is M' r ria ria ria,b na,ci

[0123] 15 tLvl >Lv2 >Lv3 > °3 G which is the same as the set of constraints from Figure 2.

[0124] Step D: Verifying the Log and Verbalizing Deviations

[0125] The previous step C produces a set of redundancy-free constraints that must be satisfied by all traces in the model. If a trace violates a constraint, the trace is also

[0126] 20 deviant. Hence, we can use the discovered constraints as a way to provide higher- level diagnostics to the user, and use the constraint’s associated verbalization to explain 13 the trace’s deviation. For that, standard constraint monitoring techniques can be leveraged such as replay-based or alignment-based

[0016] . As shown in Section 1, using the set of discovered constraints results in more understandable diagnostics.

[0127] 5 Runtime and Properties

[0128] If a trace violates a constraint c, then it must also violate the model (since £(M) C / 1(c)). Therefore, the approach produces only "correct" diagnostics. Also, given the same template library, the computed set of satisfied constraints is always the same and only depends on £(M\ Therefore, the approach is deterministic. Moreover,

[0129] 10 given models My and M2, with satisfied constraints SatM1and SatM2, it holds that £(7Wi) C £(M2) =r- SatM2C SatM1, i.e., monotonicity holds for the provided set of diagnostics. This last property ensures that the diagnostics are consistent as the user edits the model. Adding / re moving behavior to a model can only prune / extend the set of reported violations. These properties do not hold after the pruning step.

[0130] 15 However, this is not harmful as the set of satisfied constraints can be logically inferred from the set of minimal constraints.

[0131] The method instantiates constraints. The validity check depends on the size of the model’s behavioral automaton, which is worst-case exponential. However, for F -invariant constraints the state-space can be reduced to make it manageable

[0132] 20 in practice. The pruning step performs O(\Ua\mk+k') DEA operations. By limiting the maximum size of a constraint’s DEA, we can ensure a constant time for this operation, therefore the pruning step can be made O(\Ua\mk+k') too. Last, checking the constraints takes is the number of events in the log. At first the exponents look too high, but as shown by the experiments in Section 6

[0133] 25 they are tractable in practice.

[0134] Implementation and Qualitative Evaluation

[0135] The framework of the present invention is implemented as a web app using the PM4Py [?] and automata-lib [?] libraries for the backend (both written in pure 14

[0136] Python). Figure 4 shows an overview of its UI. On the sidebar, the user can set the framework’s parameters. The tool displays the model, the set of discovered constraints, and the log. Furthermore, it has a trace explorer that allows the user to

[0137] 5 drill down on single traces of the log. As a template library, we implement the set of control-flow behavioral patterns presented in

[0025] , which includes all DECLARE

[0024] templates as well as templates with more than two parameters. For the minimization step, we adapt the DECLARE constraint ordering criteria proposed in [7] to this library.

[0138] 10 We demonstrate how the framework can be used to gain insights into the well-known Italian Road Fines

[0017] process. We extract the normative description provided in

[0019] . The process starts with the creation of a fine. In at most 90 days, the fine is sent to the offender’s address. The date on which the offender is notified is inserted into the system. From the moment of notification, the offender has 60 days to appeal

[0139] 15 to the judge or the prefecture, which might dismiss the case. After these 60-days, no further appeal is possible and the offender must pay the fine. If the payment takes too long, a penalty is added to the total amount. Payments are possible at any moment and the case can also terminate at any point if the owned amount is fully paid. Otherwise, if it takes too long, the process also terminates with the case

[0140] 20 being sent for credit collection. The entire case must be completed within a year of the fine creation.

[0141] The original model from

[0019] is designed as a data Petri net. Since our approach can only handle control flow, we enrich the original event log with events to capture temporal marks (90 Days After Creation, 1 Year After Creation, and 60 Days After

[0142] 25 Notification) and data attributes such as dismissal type (Dismissed by Judge and Dismissed by Prefecture) and payment status (Partial Payment and Full Payment). A reference BPMN model is drawn from the process description (model view in Figure 4). For the alignment cost function, we assign very high costs to the removal and addition of temporal marks. Since these are not proper process steps, we do not want to allow them to be edited. The data is loaded into the implemented tool. 15

[0143] We consider a max parameter of two, and the proposed "approximated" constraint minimization method according to step C.

[0144] The most frequently violated constraint is 65: Each {Add Penalty} must be preceded

[0145] 5 by {60 Days After Notification}, occurring in 40% of the cases. The constraint description suggests that penalties are being added too early. To drill down on that, the tool provides a set of frequently co-occurring constraints that positively correlate (i.e., lift >1) with this violation. Among them, constraint 60:Each {Send fine} must be followed by a {90 Days After Creation} suggests a causal relation. If fines are sent 10 too late, then the penalty is also added earlier than it should. This is plausible since the time to add a penalty is computed considering the time elapsed since the fine creation. So a delay in sending the fine also causes the penalty to be added too early.

[0146] This suggests that sending the fines more speedily will also improve other areas of conformance.

[0147] 15 Another interesting violation refers to constraint 3: {Fill Payment} occurs at most [1] times. The enriched log distinguishes between Partial and Full payments, so repeated full payments can only occur in situations where the outstanding amount is fully paid, but further payments are still effectuated. This only happens for 5% of the cases but is arguably a serious problem. To understand its root cause, we again

[0148] 20 look into the constraints that co-occur with it and see that constraint 117:{Full Payment} cannot be followed by {Add Penalty} has a strong correlation with it (lift

[0149] > 10). Constraints 3 and 117 are violated in only 5% and 2.5% of the cases, but constraint 117 is violated with a 50% chance if 3 is also violated. This suggests that double payments mostly happen to pay the added penalty. A possible root

[0150] 25 cause for that are communication delays between the agents adding the penalty and processing the payment.

[0151] Overall, this analysis shows how the high-level diagnostics combined with understandable constraint descriptions help the analyst to make sense of deviations and formulate potential root causes and improvement areas. In principle, similar insights could be obtained by following the methodology proposed in

[0025] . However, 16

[0152] Table 3: Snippet of diagnostics produced by

[0012] (simplified to fit the table) for the enriched road fines dataset. In total, 703 statements are produced.

[0153] SI {Creation + 60 Days } occurs after {Add Penalty} and before {Creation + 1

[0154] Year}

[0155] S2 After {Partial Payment}, {Full Payment} is repeated

[0156] S3 After {Send Fine}, {Full Payment} is repeated

[0157] S4 {Add Penalty} occurs after {Full Payment} and before {Creation + 60 Days } the analyst would need to guess and manually configure each pattern of interest.

[0158] By mining the constraints from the process model, our approach saves the analyst significant time. To the best of our knowledge,

[0012] is the only existing work that gen¬

[0159] 5 erates natural-language diagnostics from procedural models. The approach works by matching a series of hard-coded patterns in the synchronized error-correcting product of the event log and process model.

[0160] Table 3 shows a snippet of the behavioral statements produced by

[0012] , While our approach produces 143 constraints (out of which only 76 are violated),

[0012] pro¬

[0161] 10 duces 703 distinct behavioral statements. The log contains 995 violating variants, i.e. there is almost one unique statement for every violating variant. The high number of distinct statements make it difficult for users to understand common deviation patterns. Many statements are slight variations of each other, e.g. statements S2 and S3 only differ in their "reference point". The tool from

[0012] does not provide

[0162] 15 the frequency of each violation and does correlate violations, making it difficult to drill down through them. Among the returned set of behavioral statements, SI is equivalent to constraint 65 reported by our tool (unfortunately, we could not find an equivalent statement to constraint 60). Similarly, constraints 3 and 117 are (in- directly) reflected by statements S2 / S3 and S4.

[0163] 20 In summary, the method described in

[0012] requires more extensive drill-down efforts (due to the higher number of violations) to obtain similar insights. Furthermore, our approach differs from

[0012] in three key ways: First, we offer a flexible method for defining new patterns; second, we can explain deviations caused by long-term 17 dependencies, e.g. via the "AlternatingSuccession" template; last, we provide better runtime guarantees. A merit of

[0012] is to identify precision issues, i.e. behaviors allowed by the model but absent in the log. This could be achieved in our framework

[0164] 5 by "swapping" the roles of the log and the model.

[0165] Quantitative Evaluation

[0166] This section evaluates the approach for its scalability and completeness. We compare two scenarios: considering all constraints (ALL) and considering only r -invariant constraints (T). For each scenario, we vary the maximum number of template pa¬

[0167] 10 rameters maxp from 2 to 4. The framework is evaluated on two real-world datasets

[0168] : The enriched Italian road fines dataset (RF) described in Section 6 and the BPI Challenge 2015 event log

[0010] (BPI-15) with a model discovered with the Inductive Miner-infrequent variant

[0014] with a noise threshold of 0.9. We filter the BPI-15 log for the municipality 1, subprocess 8, and remove repeated activities. This is needed

[0169] 15 as otherwise the used process discovery method would only return flower constructs.

[0170] The models are of small to middle size. Both contain 20 transitions, and 21 resp. 561 states in their behavioral automata. The experiments are run single-threaded on a

[0171] Ubuntu 22.04 notebook with an Intel Xeon E-2276M processor and 32Gigabytes of mam memory .

[0172] 20 Scalability and Pruning Efficiency

[0173] This experiment evaluates the scalability of the approach and the effectiveness of the pruning step. The results are summarized in Table 4. For both datasets, increasing the maximum number of template parameters max^ from 3 to 4 leads to a significant increase in the number of instantiated constraints (#inst), which causes a significant

[0174] 25 increase in the total runtime for the satisfiability check (tsat). Nevertheless, increasing maxkonly slightly increases the number of satisfied constraints. For the road fines dataset, the average time to check if a constraint is satisfied (= tsat / #inst') is approximately the same for both scenarios (ALL and T). For the BPI-15 dataset, the check of each constraint takes significantly longer for the ALL than for the r sce- 18

[0175] Table 4: Evaluation of constraint check and minimization steps measuring the number of instantiated constraints (#inst), and the number of satisfied / minimal constraints (%sat / %min) and the time in seconds to compute them (tsat / tmin).

[0176] ALL Templates F -Invariant Templates

[0177] DS maxk

[0178] 2 8531 2.58 1295 "PLS 227 3386 0.75 511 7.9 143

[0179] 3 50075 17.0 19653 87.7 223 13772 3.74 1244 13.2 145

[0180] 4 683315 285 336991 1699 454 140420 46.4 21927 830 468

[0181] 2" 9442 16.8 444 6.02 242 3763 0.75 148 1.31 109

[0182] 3 49042 71.3 11147 83.4 1112 13663 3.45 170 1.38 109 m 4 555922 1328 221355 3281 1953 115039 35.7 9632 1075 555 nario. This is explained by its larger state space, for which the inventive techniques using the F -invariants for validity checks described in step B above pay off.

[0183] In all experiment setups, the constraint minimization step takes a significant portion

[0184] 5 of the total time, with up to aann hhoouurr ffoorr BBPPLL1155 wwhheenn m maaxxkk= 4. For comparison, the method proposed in [7] goes out of memory for certain scenarios. Still, the approximated minimization step is very effective at pruning redundant constraints, pruning 99.999% of the satisfied constraints for the road fines dataset in the ALL scenario with maxk4. Finally, when considering the absolute runtime numbers, all

[0185] 10 experimental setups could be computed within two hours, with the majority of them being computed within a minute. Last, notice that the approach is implemented in pure Python. A more careful implementation in a compiled language is expected to bring at least an order of magnitude of improvement, making the approach applicable for interactive scenarios.

[0186] 15 Completeness and Redundancy

[0187] We measure the completeness of the produced diagnostics, i.e., the share of deviations that can be explained, and the amount of redundancy of the returned diagnostics. For each scenario, we align the event log and the process model and pinpoint constraints to model / log moves using the method presented in

[0016] . con¬

[0188] 20 straints associated with a move explain the move, i.e., the constraints justify the insertion / deletion at the given position. Table 5 summarizes the results. 19

[0189] Table 5: Completeness and redundancy results, dex and mox are the numbers of deviant variants and non-synchronous moves. We report the number of traces violating at least one constraint (det), the number of moves explained by at least one constraint (expl), and the average number of constraints per move det expl axg

[0190] DS dex max k=2 k=3 k=4 k=2 k=3 k=4 k=2 k=3 k=4

[0191] ALL 994 995 995 4448 4489 4489 3.79 4.34 9.47

[0192] RF 995 4489 F 992 995 995 39144238 4238 3.49 3.88 8.76

[0193] RF / '+ 992 995 995 995 3.42 3.97 8.14

[0194] ALL 70 70 70 108 108 108 2.34 2.17 2.46

[0195] BPI-15 r 70

[0196] 70 70 70 no 108 108 108 2.17 7.30 5.86

[0197] On the trace level, most non-conforming traces violate at least one discovered constraint. On the alignment move level, the discovered constraints explain most of the moves, with the road fines dataset performing the worst with 13% of its moves

[0198] 5 unexplained for the F scenario with k 2. The experiment also shows that by using

[0199] ALL constraints or increasing the maximum number of template parameters, more moves can be explained.

[0200] Finally, Table 5 also shows the average number of constraints explaining each align- ment move. The method must report as few constraints per move as possible, to not

[0201] 10 overwhelm the user. For the BPI-15 dataset, in all but two scenarios, the average number of constraints per alignment move is below three. Furthermore, we observe only a slight variation in the number of constraints per move when varying maxk.

[0202] In contrast, for the road fines dataset, all but one scenario report over three con- straints per move, with a peak of 9 constraints per move for the ALL scenario with

[0203] 15 maxk= 4.

[0204] In summary, the experiments show the scalability of the method, with F -invariant constraints being particularly efficient for models with large state spaces. This increased efficiency of F -invariant constraints comes at the cost of a slight decrease in completeness. By increasing the maximum number of constraint parameters, more deviations can be explained, but this comes at the expense of an increase in the 20 total runtime. This exposes the tradeoff between scalability, completeness, and redundancy, which the user can select.

[0205] Enriching the Template Library As shown in the section above, the discovered

[0206] 5 constraints provide nearly complete diagnostics. Still, some violations cannot be explained by any constraint. One example is the alignment displayed in Figure 4.

[0207] No constraint explains the insertion of the activity “Send for Credit Collection”.

[0208] However, the template library can be easily extended to become more complete.

[0209] By analyzing the model, one notices that the insertion happens because at least

[0210] 10 one of the activities "Full Payment", "Send for Credit Collection", "Dismissed by

[0211] Judge" or "Dismissed by Prefecture" must occur. By adding the constraint template

[0212] .*(a?i|a?2|a?3|a?4).* to the library, meaning “either aq, aq, aq, or aq must happen in the case ”, all moves are explained (see / '+ in Table 5). Of course, this does not mean that the library is complete, but shows that it can be easily extended to capture

[0213] 15 new patterns.

[0214] Conclusion

[0215] A method to generate understandable conformance diagnostics based on behavioral patterns has been presented. The method automatically instantiates the constraints from the template library, checks for satisfying constraints, and prunes redundant

[0216] 20 constraints. It presents a series of interesting properties such as returning higher level diagnostics, absence of false positives, and monotonicity of the reported diagnostics.

[0217] The qualitative evaluation demonstrates how it can be applied to analyze an event log, uncovering drill-down directions. The quantitative evaluation shows that it can cover most of the deviations in real-world logs in a feasible runtime. The user can

[0218] 25 control the scalability of the method by restricting the set of deviation patterns and varying the maximum number of template parameters at the expense of slightly less complete diagnostics. 21

[0219] References

[0220] 1. van der Aalst, W.: Data Science in Action, pp. 3-23. Springer Berlin Heidelberg, Berlin, Heidelberg (2016)

[0221] 5 2. Adriansyah, A., van Dongen, B. , van der Aalst, W.: Conformance checking using cost-based fitness analysis. In: 2011 IEEE 15th International Enterprise Distributed Object Computing Conference, pp. 55-64 (2011)

[0222] 3. Adriansyah, A. , Van Dongen, B.F. , Zannone, N.: Controlling break-the-glass through alignment. In: 2013 International Conference on Social Computing, pp. 606-611. IEEE (2013)

[0223] 10 4. Alman, A. , Di Ciccio, C. , Haas, D. , Maggi, F.M. , Nolte, A.: Rule mining with rum. In: 2nd International Conference on Process Mining (ICPM). pp. 121-128. IEEE (2020)

[0224] 5. Carmona, J. , van Dongen, B.F. , Solti, A. , Weidlich, M.: Conformance Checking - Relating Processes and Models. Springer (2018)

[0225] 6. Desel, J. , Esparza, J.: Free choice Petri nets. No. 40, Cambridge university press (1995)

[0226] 15 7. Di Ciccio, C. , Maggi, F.M. , Montali, M. , Mendling, J.: Resolving inconsistencies and redundancies in declarative process models. Inf. Sys. 64, 425-446 (2017)

[0227] 8. Di Ciccio, C. , Maggi, F.M. , Montali, M., Mendling, J.: On the relevance of a business constraint to an event log. Information Systems 78, 144-161 (2018)

[0228] 9. Dijkman, R.M. , Dumas, M., Ouyang, C.: Semantics and analysis of business process models in BPMN.

[0229] 20 Inf. Softw. Technol. 50(12) , 1281-1294 (2008)

[0230] 10. van Dongen, B.: BPI challenge 2015 (2015) , data.4tu .nl / collections / {_]75065424 / l

[0231] 11. Esparza, J. , Blondin, M.: Automata theory: An algorithmic approach. MIT Press (2023)

[0232] 12. Garcia-Banuelos, L. , van Beest, N.R. , Dumas, M. , Rosa, M.L. , Mertens, W.: Complete and inter- pretable conformance checking of business processes. IEEE Transactions on Software Engineering

[0233] 25 44(3) , 262-290 (2018)

[0234] 13. Keller, G. , Teufel, T.: SAP R / 3 Process-oriented Implementation: Iterative Process Prototyping. SAP databases, Addison Wesley Longman (1998)

[0235] 14. Leemans, S.J.J. , Fahland, D. , van der Aalst, W.M.P.: Discovering block-structured process models from event logs containing infrequent behaviour. In: Business Process Management Workshops, pp.

[0236] 30 66-78. Springer (2014)

[0237] 15. Leemans, S.J.J. , Fahland, D. , van der Aalst, W.M.P.: Scalable process discovery and conformance checking. Software and Systems Modeling 17, 599 - 631 (2016)

[0238] 16. de Leoni, M. , Maggi, F. , van der Aalst, W.: Aligning event logs and declarative process models for conformance checking, pp. 82-97 (01 2012)

[0239] 35 17. de Leoni, M., Mannhardt, F.: Road traffic fine management process (2 2015)

[0240] 18. Maggi, F.M. , Bose, R.P.J.C. , van der Aalst, W.M.P.: Efficient discovery of understandable declarative process models from event logs. In: Advanced Information Systems Engineering, pp. 270-285. Springer, Berlin, Heidelberg (2012)

[0241] 19. Mannhardt, F. , De Leoni, M. , Reijers, H.A. , Van Der Aalst, W.M.: Balanced multi-perspective checking of process conformance. Computing 98, 407-437 (2016) 22

[0242] 20. Mozafari Mehr, A.S. , M. de Carvalho, R. , van Dongen, B.: Explainable conformance checking: Understanding patterns of anomalous behavior. Engineering Applications of Artificial Intelligence 126, 106827 (2023)

[0243] 5 21. Munoz-Gama, J. , Carmona, J. , van der Aalst, W.: Single-entry single-exit decomposed conformance checking. Information Systems 46, 102-122 (2014)

[0244] 22. Murata, T.: Petri nets: Properties, analysis and applications. Proceedings of the IEEE 77(4) , 541-580 (1989)

[0245] 23. Peled, D. , Wilke, T.: Stutter-invariant temporal properties are expressible without the next-time op-

[0246] 10 erator. Information Processing Letters 63(5) , 243-246 (1997)

[0247] 24. Pesic, M. , Schonenberg, FL , van der Aalst, W.M.P.: Declare: Full support for loosely-structured processes. In: 11th IEEE International Enterprise Distributed Object Computing Conference (EDOC 2007). pp. 287-287 (2007)

[0248] 25. Ramezani, E. , Fahland, D. , van der Aalst, W.M.: Diagnostic information in compliance checking. Tech,

[0249] 15 rep. , Citeseer (2012)

[0250] 26. Reijers, H.A. , Slants, T. , Stahl, C.: Declarative modeling-an academic dream or the future for bpm?

[0251] In: Business Process Management - 11th International Conference, BPM. Lecture Notes in Computer Science, vol. 8094, pp. 307-322. Springer (2013)

[0252] 27. Rozinat, A. , van der Aalst, W.: Conformance checking of processes based on monitoring real behavior.

[0253] 20 Information Systems 33(1) , 64-95 (2008)

[0254] 28. Weidlich, M. , Polyvyanyy, A. , Desai, N., Mendling, J. , Weske, M.: Process compliance analysis based on behavioural profiles. Inf. Syst. 36, 1009-1025 (11 2011)

Claims

23Claims51. A computer-implemented method of performing conformance diagnostics of traces o of an event log L with a net representing a model of a process,10The process comprising predetermined steps, the process comprising predetermined allowed variants of sequences of activities,15 the event log L comprising traces G, each trace being representative of a sequence of observed activities, the traces a being representative of actual sequences of activities a.20 each activity a, being a step of the process, the net being representable by a labeled accepting Petri net TV, whereby the labeled accepting Petri net TV is a tuple N = (P,T,F,l, mo,rnf) where P is the set of places, T the set of transitions, F C (P x T) U (T x P) is the flow relation, I : T ™> MaU {r}25 is the labeling function, and mo, mj E B(P) are its initial and final markings. whereinUarepresenting the universe of activity labels a,24 each transition of the set of transitions T representing a step of the process, the method comprising:5 A step (A) of instantiating a predetermined set of candidate constraints Inst= {G} out of a library comprising predefined templates of constraints, a template being a function that returns a constraint for each Gtuple of activities, k being the number of template parameters,10 each constraint c being defined as an object describing a language w where♦15 * denoting the Kleene operator applied on UaA step (B) of determining, from the set of candidate constraints Inst= {G}, a set of constraints which are satisfied by all traces c in the Petri net’s20 N accepted language using the following relation holds: / * / O \ r )L l I > \ L- 1 > with 5vbeing the behavioral automaton of the Petri net N;A step (C) of25 removing redundant constraints from the set of satisfied constraints by searching for C c Uc a minimal set of redundancy -free constraints C mi»c7,Cc?|C,,|s-t* Vc £ G, Cfc30 a constraint c being denoted redundant in a set of constraints25 a step (D) of verifying the event log L by checking whether the minimum set of constraints C ’ is satisfied by the traces c of the event log L.

52. The method of claim 1, further comprising: if a trace o is checked not to satisfy the minimum set of constraints C ’, labeling the trace c in the event log L as not satisfying the process described by the Petri net N.

103. A computer-implemented method of determining a minimum set of constraints for conformance diagnostics of traces o of an event log L with a net representing a model of a process, the process comprising predetermined steps,15 the traces a being representative of actual sequences of activities a. each activity a, being a step of the process,20 the net being representable by a labeled accepting Petri net TV, whereby the labeled accepting Petri net N is a tuple N = (P,T, where P is the set of places, T the set of transitions, F C (P x T) U (T x P) is the flow relation, I : T — ?• I4aU {r} is the labeling function, and mo, m,j E S(P) are its initial and final markings. wherein25 Uarepresenting the universe of activity labels a, each transition of the set of transitions T representing a step of the process, the method comprising:30 A step (A) of26 instantiating a predetermined set of candidate constraints Inst= {G} out of a library comprising predefined templates of constraints, a template being a function that returns a constraint for each Gtuple of activities, k being the number of template parameters,5 each constraint c being defined as an object describing a languagewhere fift'<110 * denoting the Kleene operator applied on UaA step (B) of determining, from the set of candidate constraints Inst= {G}, a set of constraints which are satisfied by all traces c in the Petri net’s15 TV accepted language using the following relation holds: with 5vbeing the behavioral automaton of the Petri net N;A step (C) of20 removing redundant constraints from the set of satisfied constraints by searching for C c Uc a minimal set of redundancy -free constraints C rain-c,c.c\P,\s-t* Vc e C, O' -4 c25 a constraint c being denoted redundant in a set of constraints4. The method according to one of the preceding claims, wherein in the determining step (B), the set of constraints is determined30 by: for each constraint c of the set of candidate constraints Inst= {Ci},27 computing a maximal set of activities F for which the constraint c is / ’-invariant, and reducing the Petri net A by the set F, thus obtaining the reducedPetri Net Nua\r, and5 checking whether the constraint c is satisfied by the reduced Petri net Nua\r .

5. The method according to one of the preceding claims, wherein the maximal set F of activities for which a constraint c is / ’-invariant is10 determined by collecting all activities y for which the following condition holds:<1 with Q being the set of states of the DFA of the constraint c, q being a state of the set of states, q E Q cO15 5 being the transition function defined as where 27 is the alphabet, with D being the minimal DFA of c defined as quintuple D = (Q,Ua, 8, q0, Qq) such that20 qo being the initial state, Qf being the set of final states.

6. The method according to one of the preceding claims, wherein searching a minimal set of redundancy -free constraints C ’ comprises exe¬25 cuting the following algorithm:Ordering all the constraints of the set of constraints C according to a predefined order criterion, ■C, Ct <cc?- for all 1 < i < j < rn + 130 m being a predetermined maximum intersection size,28Checking all permutations of the ordered constraintsC,ti< >! a” 1 r.-- 1 for redundant constraints,5 such thatIf Cm+i is found to be redundant,X r* removing cm+i from the set of constraints, thus obtaining f ' V f1\ f / '* 1 I10 checking the permutations being performed in lexicographical order of the combinations of the permutations.

7. The method according to one of the preceding claims, wherein a template15 of the set of predefined templates of constraints is a function c. ir - u, the function returning a constraint c for each Btuple of activities (rzi, <22, . .where k is the number of template parameters.20 8. The method according to one of the preceding claims, wherein the library of predefined templates of constraints is held in an electronic database.

9. The method according to one of the preceding claims, wherein25 the behavioral automaton of the labeled accepting Petri net TV is defined as the unique minimal DFA2B, such that , theDFA being defined as / - a quintuple<'• " ' ‘ ‘ ’•* where Q is the set of states, 27 is the al- phabet,uA r '•v is the transition function, go is the initial30 state and Qf c Q isthe set of final states.2910. The method according to one of the preceding claims, wherein the Petri net TV is a bounded Petri net.5 11. The method according to one of the preceding claims, wherein the minimum set of constraints is an approximative minimum set of constraints.

12. A computer-implemented method of performing conformance diagnostics of traces o of an event log L with a net representing a model of a process,10 the process comprising predetermined allowed variants of sequences of activities, the event log L comprising traces G, each trace being representative of a15 sequence of observed activities, the net being representable by a labeled accepting Petri net TV, the method comprising:20 a step (D) of verifying the event log L by checking whether the minimum set of constraints C ’ obtained by the method of one of the claims 3-11 is satisfied by the traces G of the event log L.25 13. The method of claim 12, further comprising: if a trace o is checked not to satisfy the minimum set of constraints C ’, labeling the trace G in the event log L as not satisfying the process described by the Petri net N.30 14. A machine-readable storage medium comprising instructions which when loaded into a computer system controls the computer system to perform the method according to one of the preceding claims.3015. A computer program comprising instructions which, when the program is executed by a computer, cause the computer to perform the method according to one of claims 1-13.5

Images