Method for processing decision data, corresponding device and computer program

Abstract

The invention relates to a device and a method for processing decision data comprising a decision tree (T) and an input path (P), each node (r) of the decision tree being associated with a respective feature (i) of a set of features (F). According to the present disclosure, the method comprises at least one iteration of an inheritance process as a function of a current generic set (U) representing some features that may be independent of interpreting the input path and as a function of a current node (r) and child nodes (s) connected together by edges ((r, s)), the set (U) being contained in a set of features (F), the inheritance process comprises:-determining that the edge ((r, s)) verifies a first criterion relating to the consistency of the edge ((r, s)) with a corresponding edge ((r ', s') of the input path (P), the corresponding edge ((r ', s') of the input path (P) relating to a feature (i) identical to the feature of the current node (r), then performing a first sub-inheritance process as a function of the sub-node (s); and / or-determining that the edge ((r, s)) verifies a second criterion and then performs a second sub-inheritance process as a function of the sub-node (s) and the current general set (U), such that the first sub-inheritance process and the second sub-inheritance process allow features to be included in the set (U) to interpret the input path (P).

Description

Technical Field

[0001] The present invention relates to the field of processing decision data and more precisely to the determination of explanations for machine learning (ML) models.

[0002] Explanations essentially represent the answer to the “Why?” question, i.e., why did the ML model’s predicted outputs give the predicted outputs that we obtained? Background Art

[0003] Such explanations of ML models aim to achieve parsimony through subset minimization (or irreducibility). These explanations are often referred to as PI explanations or abductive explanations (AXp). The cognitive limitations of human decision makers make parsimony one of the key requirements for ML model explanations. Parsimonious explanations are often considered easier to understand by human decision makers and easier to diagnose or debug.

[0004] Decision trees (DTs) are the epitome of so-called interpretable machine learning models, in part because the paths in the tree (which may be short and therefore potentially concise) represent such explanations for the predictions.

[0005] Decision trees have a wide range of practical uses. Moreover, DTs are the most visible example of a collection of machine learning models that have recently been recognized as essential for high-stakes applications. Decision trees are also the epitome of interpretability methods as a function of intrinsic interpretability.

[0006] Given a decision tree, some input, and the resulting prediction, the explanation associated with that prediction is the path in the decision tree that is consistent with the input. This simple observation is partly why decision trees have been considered interpretable for at least two decades, is widely taken for granted, and explains the interest in learning optimal decision trees, especially in recent years, especially when learning optimal (minimal) DTs is well known to be an NP-hard problem.

[0007] The state-of-the-art covers different optimization criteria, some of which are closely related to the simplicity of explanations (e.g., measured as average path length). However, in the prior art, the simplicity of path explanations in DTs cannot be efficiently achieved in practice (or in other words, is NP-hard for most specific cases). Even the smallest DT (called the optimal DT) may exhibit arbitrary explanation redundancy, and explanation redundancy is often observed in practice.

[0008] Comprehensive evidence on the redundancy of path-based interpretations in DT is needed, especially for high-risk and safety-critical applications that can be handled within a reasonable duration.

[0009] The present invention aims to improve this situation. Summary of the invention

[0010] The present invention proposes a new type of method for processing decision data, which does not have the disadvantages of the prior art. The method is used to process decision data, and the method is implemented by an electronic device for processing a data source, and the decision data includes a decision tree. And through the decision tree The decision path (P) of the nodes of the decision tree, each node (r) of the decision tree is associated with a corresponding feature (i) of the feature set (F),

[0011] The method is characterized in that it determines a universal set (U) comprising features that are not actually relevant for explaining the decision path, and wherein the method comprises:

[0012] at least one iteration of a first function, the first function being a function of a current set (U) and of a current node (r) and a child node (s) connected together by an edge ((r, s)), the current set (U) representing some features irrelevant for explaining the decision path, the current set (U) being included in the set of features (F), the first function comprising:

[0013] - determining that the edge ((r, s)) verifies a first criterion, the first criterion being related to the consistency of the edge ((r, s)) with the corresponding edge ((r′, s′)) of the decision path (P), the corresponding edge ((r′, s′)) of the decision path (P) being related to a feature (i) identical to the feature of the current node (r), and satisfying the first criterion triggers execution of a second function that is a function of the child node (s); and / or

[0014] - determining that said edge ((r, s)) verifies a second criterion, the satisfaction of which triggers the execution of a third function which is a function of said child node (s) and said current set (U), said second function and said third function causing the inclusion of a feature in said current set (U), and

[0015] - delivering said universal set (U) for interpreting said input path (P) based on said current set.

[0016] A decision tree can be interpreted as a structure that describes a series of decisions, each of which is associated with a given feature. A decision path, also called an input path (because it is the input to the method), can be derived from a given word (i.e., a series of values), each of which is associated with a feature.

[0017] One of the nodes of a decision tree is the root node, which can be considered as the entry point of the decision tree. Therefore, the input path can be interpreted as a series of connecting edges connecting the root node to the terminal nodes. Each terminal node of the decision tree (i.e., each node without any child nodes) is associated with one of the classes in the class set. The class of the terminal node of an input path is called the prediction of the input path.

[0018] The first function is also referred to as an inheritance process hereinafter. In addition, the second function and the third function are respectively referred to as a first sub-inheritance process and a second sub-inheritance process.

[0019] Therefore, the method allows processing decision trees in an efficient way (e.g. exploration, generation of specific data, etc.) by processing certain edges (or child nodes) only via the first sub-inheritance processing or the second sub-inheritance processing and ignoring other edges (or child nodes), thanks to the first criterion on consistency and the second sub-inheritance processing taking into account the current common set.

[0020] Furthermore, given the first criterion, each edge consistent with the input path is processed. Furthermore, given the input of the second child inheritance process (the input considering the commonality of the features of the current node), each feature considered to be common leads to consideration of any child nodes of the current node.

[0021] These two properties help determine whether a feature is considered irrelevant to explain the input path. It helps determine the features that are relevant to explain the input path, and the minimal set of features that explain the input path forms the abductive explanation of the input path, hereafter referred to as AXp. In other words, the method is able to identify nodes in the decision path that are actually relevant to explain the prediction of the decision path, which is synonymous with determining the universal set.

[0022] The method is advantageous because it can determine the universal set with polynomial running time. This property of belonging to the polynomial complexity class makes it possible to determine the universal set of decision paths in large ML models (i.e., most practical ML models), which is completely impossible for any algorithm belonging to the non-polynomial complexity class.

[0023] According to a particular characteristic of the invention, the second criterion is the negation of the first criterion.

[0024] This type of second criterion enables the successive processing to be performed on each edge of the decision tree without generating redundancy (ie processing the same edge multiple times) or missing edges.

[0025] According to a particular characteristic of the invention, the second criterion is related to the fact that the characteristic (i) of the current node belongs to said current universal set (U).

[0026] This type of second criterion enables special treatment of child nodes of the current node whose features belong to the current common set (ie, whose features may not be relevant for interpreting the input path, regardless of the actual value of the features).

[0027] According to a particular feature of the invention, each node (r) is associated with a corresponding clause (b r ) and

[0028] The second function consists in adding a first inheritance clause (b) to the set of hard constraints (H) representing the set of constraints that must be satisfied for the MaxSAT solver. r →b s ), the first inheritance clause is a clause (b) associated with the current node (r) r ) and the clauses (b) associated with the child node(s) s ) function; and

[0029] - The third function includes adding a second inheritance clause (b) to the hard constraint set (H) r ∧u u →b s ), the second inheritance clause is a clause (b) associated with the current node (r) r ), the clauses (b) associated with the child node(s) s ) and the universal clauses (u i ) function, the general clause (u i ) is associated with the feature (i) of the current node (r).

[0030] Here, the inheritance process constructs a hard constraint set intended to be given to a MaxSAT solver. A MaxSAT solver is an algorithm, program, device, or equivalent that can input a hard constraint set and a soft constraint set, and output the most extensive subset of the soft constraint set whose clauses are satisfied, while all clauses of the hard constraint set are satisfied. In other words, the hard constraint set is the set of clauses whose satisfaction is mandatory, while the soft constraint set is the set of clauses that would ideally be satisfied if it did not conflict with the hard constraint set.

[0031] The set of inheritance clauses produced by the inheritance process can be interpreted as a chain of consequences on nodes (from the current node to the child nodes). Therefore, when inheritance is processed for the set of edges connecting two nodes (an upstream node and a downstream node), and when the clause of the upstream node is set to a certain value (true or false), such a chain of consequences can imply a certain value for the downstream node (or "propagate" the value to the node), depending on the common clauses implied in the chain of consequences.

[0032] According to a particular characteristic of the invention, the method comprises adding to the set of hard constraints (H):

[0033] -Decision Tree The root node Clause

[0034] -Decision Tree Middle category The clause (b) for each terminal node (r) equal to the prediction (c) of the decision path r );as well as

[0035] -Decision Tree Middle category The negation of the clause for each terminal node (r) that differs from the prediction (c) of the decision path

[0036] Adding clauses related to the root node and terminal nodes of the decision tree here can handle nodes that have no parent node (root node) or no child nodes (terminal node).

[0037] Treating terminal nodes whose classes differ from the prediction of the input path differently means that, given that the set of hard constraints is fully satisfied, any chain of consequences (i.e., without a set of inherited clauses) starting from the root node (which must satisfy this clause) will not lead to a contradiction for said terminal nodes whose classes differ from the prediction, if the current general set is chosen appropriately.

[0038] For example, if the current generic set is too broad, such a contradiction may arise such that the value of the clause of the root node (set to true) is "propagated" (through the chain of consequences) to a terminal node whose value is false (or in other words, whose class is different from the prediction, so the negation of the clause of said terminal node must be satisfied).

[0039] According to a particular feature of the invention, the first function further comprises: determining that the feature (i) of the current node (r) is not included in the path feature set (Φ(P)) of the features involved in the decision path (P), and then adding a universal clause (u) associated with the feature (i) to the hard constraint set (H) i ).

[0040] Considering that features not included in the path feature set of the input path are irrelevant to explaining the input path, adding their corresponding universal clauses to the hard constraint set simplifies the calculation of the MaxSAT solver. In fact, the soft constraint set includes universal clauses (of all features). Therefore, including universal clauses for features that are actually irrelevant to explaining the input path in the hard constraint set reduces the number of cases to be calculated / tested by the MaxSAT solver.

[0041] According to a particular characteristic of the invention, the second function and said third function both comprise executing the first function on the child node(s).

[0042] Therefore, if the edge connecting the nodes meets the first criterion or the second criterion, the inheritance process will be recursively applied to the child nodes of the current node. The inheritance process explores the decision tree while not passing through edges that are inconsistent with the input path (the corresponding edge) and are not common to the features tested for the current node. Therefore, the exploration of the decision tree is more efficient than exhaustive exploration.

[0043] According to a particular feature of the invention, the first function further comprises: determining that the current node (r) is a terminal node, and then returning a result that determines that the prediction of the current node (r) is different from the prediction of the decision path (P).

[0044] Therefore, when the inheritance process (i.e., exploration of the tree) is performed on a terminal node, it returns a class for said terminal node that is different from the prediction.

[0045] Therefore, if exploring the decision tree starting from the root node produces a terminal node whose class is different from the prediction, this means that there is an alternative path that is consistent with the input path but produces a different prediction while considering that the features belonging to the current common set are not relevant to explain the input path. Therefore, this means that at least one feature of the current common set is actually relevant to explain the path.

[0046] In contrast, if exploring the decision tree starting from the root node yields only terminal nodes whose class is equal to the prediction, this means that no such alternative path exists and, therefore, each feature of the current general set is irrelevant in explaining the input path.

[0047] According to a particular feature of the invention, the method comprises:

[0048] - for at least one current set (U) executed as a decision tree The root node , thereby returning information indicating that there is an alternative path (Q) that produces a prediction different from the prediction of the decision path and is consistent with the decision path for features not included in the current set; and

[0049] - determining the most universal set (UR) such that the piece of information meets a third criterion relating to the absence of alternative paths (Q) that would lead to a prediction different from the prediction of the decision path (P).

[0050] Here, the inheritance processing produces some processed tree data in response. For example, the processed tree data may contain a Boolean value indicating the existence of an alternative path that produces a prediction different from that of the input path while being consistent with the input path, or the processed tree data may contain some inheritance clauses that can help determine the existence of such a path.

[0051] Thus, based on the processed tree data, a most general set can be determined, or in other words, a set of features in the entire feature set that are considered irrelevant to explaining the input path. The complement of the most general set (in the feature set) is called the abductive explanation of the input path or AXp.

[0052] According to a particular feature of the present invention, for a given current set (U), the decision tree Each node (r) executes the first function no more than once.

[0053] Therefore, there is no redundancy in performing the entire inheritance process (for a given current generic set), thereby speeding up the inheritance process.

[0054] According to a particular feature of the invention, the information indicating the existence of the alternative path (Q) comprises a Boolean value, which is true if such an alternative path exists and false if such an alternative path does not exist, and the third criterion is that the alternative existence value is true.

[0055] Here, once alternative existence values ​​are obtained (e.g., by performing an inheritance process to explore a decision tree), the actual universality of the current universal set can be established.

[0056] According to a particular feature of the invention, determining the widest universal set (UR) comprises:

[0057] -- Initialize the current set (U) to the complement of the path feature set Φ(P) of the features involved in the decision path (P) in the feature set (F);

[0058] -For at least one current feature (i) of the path feature set Φ(P):

[0059] * Add the current feature (i) to the current set (U),

[0060] * Determine if the first function returns true, and the first function is in the root node and based on the current set (U), then removing the current feature (i) from the current set (U); and

[0061] - Returns the current set (U) as the most universal set (UR).

[0062] Here, features that do not belong to the path feature set are considered irrelevant to explaining the input path. Excluding features that do not belong to the path feature set from testing reduces the computation time for the overall most general set determination.

[0063] The remaining features (belonging to the path feature set) are then tested one by one in the following way: one of the features is added to the current universal set and an inheritance process is performed. When the inheritance process returns a Boolean value of true, this means that there is an alternative path that produces a prediction that is different from the prediction of the input path, which means that the tested feature is actually relevant to explaining the input path. In this case, the tested feature is removed from the current universal set. When all features of the path feature set have been tested, the current universal set is returned as the most extensive universal set. The complement of the most extensive universal set is the minimum feature set that explains the input path, i.e., AXp.

[0064] This construction of the most general set has the advantage of being computationally fast, since each feature is tested no more than once, and the testing of the features is done through exploration of decision trees, which is very efficient as described above.

[0065] According to a particular characteristic of the invention, the method comprises:

[0066] -Decision Tree Each node (r) of executes the first function while considering the feature set (F) as the current set (U), thereby returning the hard constraint set (H) as processed tree data; and

[0067] - Using the MaxSAT solver, determine the widest universal set (UR) as the widest subset of the soft constraint set whose clauses can be satisfied when every clause belonging to the hard constraint set (H) is satisfied.

[0068] Here, the inheritance process is a type of adding clauses to the hard constraint set. For each edge, the corresponding inheritance clause is added so that the hard constraint set encodes the entire chain of consequences related to consistency and generality. When building the hard constraint set, the MaxSAT solver can input the hard constraint set and the soft constraint set (containing the general clauses for each feature) and determine the most extensive set of general clauses.

[0069] Considering that each edge implies only one inheritance clause (i.e., inheritance set is executed once per edge (not recursively here)), the computation of the entire set of hard constraints is fast.

[0070] The present disclosure also relates to an electronic device for processing decision data, wherein the decision data includes a decision tree. And through the decision tree The decision path (P) of the nodes of the decision tree, each node (r) of the decision tree is associated with a corresponding feature (i) of the feature set (F),

[0071] The device is characterized in that it is configured to determine a universal set (U) comprising features that are not actually relevant for explaining the decision path, and wherein the device comprises means for processing at least one iteration of a first function, the first function being a function of a current set (U) and of a current node (r) and a child node (s) connected together by an edge ((r, s)), the current set (U) representing some features that are not relevant for explaining the decision path, the current set (U) being contained in the set of features (F), the first function comprising:

[0072] - determining that the edge ((r, s)) verifies a first criterion, the first criterion being related to the consistency of the edge ((r, s)) with the corresponding edge ((r′, s′)) of the decision path (P), the corresponding edge ((r′, s′)) of the decision path (P) being related to a feature (i) identical to the feature of the current node (r), and satisfying the first criterion triggers execution of a second function that is a function of the child node (s); and / or

[0073] - determining that said edge ((r, s)) verifies a second criterion, the satisfaction of which triggers the execution of a third function that is a function of said child node (s) and said current set (U), said second function and third function causing the inclusion of a feature in said current set (U),

[0074] - delivering said universal set (U) for interpreting said input path (P) based on said current set.

[0075] According to one implementation, the different steps of the method according to the present disclosure are implemented by one or more software programs or software module programs, and the one or more software programs or software module programs include software instructions intended to be executed by at least one data processor of a device directly or indirectly connected to an automatic speech recognition system.

[0076] Therefore, another aspect of the present disclosure relates to at least one computer program product that can be downloaded from a communication network and / or recorded on a medium that is readable by a computer and / or executable by a processor, and at least one computer program product includes program code instructions for implementing the method as described above. More specifically, the computer program product includes instructions for commanding the execution of different steps of the method for processing data related to the present disclosure.

[0077] The program may use any programming language and may be in the form of source code, object code or an intermediate code between source code and object code, such as a partially compiled form or any other desired form.

[0078] According to one embodiment, the method / device may be implemented by software components and / or hardware components. In this regard, the term "module" or "unit" in this document may also correspond to a software component and a hardware component or a group of hardware components and software components.

[0079] A software component corresponds to one or more computer programs, one or more subroutines of a program, or more generally to any element or piece of software of a program capable of implementing a function or a set of functions as described below for the relevant module. Such a software component is executed by a data processor of a physical entity (terminal, server, etc.) and is able to access the hardware resources of the physical entity (memory, recording medium, communication bus, input / output electronic board, user interface, etc.).

[0080] Similarly, a hardware component corresponds to any element of a hardware unit capable of implementing a function or a set of functions described below for the relevant module. It can be a programmable hardware component or a component with an integrated processor for executing software, such as an integrated circuit, a smart card, a memory card, an electronic board for executing firmware, etc.

[0081] In addition, the present disclosure also relates to a non-transitory computer-readable medium, which includes a computer program product recorded on the non-transitory computer-readable medium and capable of being executed by a processor, and the computer program product includes program code instructions for implementing the above-mentioned data processing method of the present disclosure.

[0082] Computer-readable storage media used herein are considered to be non-transitory storage media due to the inherent ability to store information therein and to provide the inherent ability to retrieve information therefrom. Computer-readable storage media can be, for example, but not limited to, electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. It should be understood that although more specific examples of computer-readable storage media to which the present principles can be applied are provided below, it is readily understood by those of ordinary skill in the art that the following is merely an exemplary and not exhaustive list: a portable computer floppy disk, a hard disk, a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0083] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the disclosure, as claimed.

[0084] It must also be understood that references in the specification to "one embodiment" or "an embodiment" indicate that the described embodiment may include a particular feature, structure, or characteristic, but not every embodiment may include the particular feature, structure, or characteristic. Furthermore, these phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in conjunction with an embodiment, it should be considered that those skilled in the art are aware that the feature, structure, or characteristic can be affected in conjunction with other embodiments, whether or not explicitly described. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Embodiments of the present disclosure may be better understood with reference to the following description and drawings, which are given by way of example and do not limit the scope of protection, wherein:

[0086] - Figure 1 is an example of a decision tree processed according to the present disclosure;

[0087] - Figure 2 is a second example of a decision tree processed according to the present disclosure;

[0088] - Figure 3 is a schematic block diagram illustrating an example of a process;

[0089] - Figure 4 is a schematic block diagram illustrating an example of a process;

[0090] - Figure 5 is a schematic block diagram illustrating an example of a process;

[0091] - Figure 6 is a schematic block diagram illustrating an example of a process;

[0092] - Figure 7 is a schematic block diagram illustrating an example of a process;

[0093] - Figure 8 A schematic block diagram illustrating an example of an apparatus according to the present disclosure is shown.

[0094] The components in the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the disclosure. DETAILED DESCRIPTION

[0095] 5.1 Public background

[0096] 5.1.1 Principles of Decision Tree

[0097] A decision tree is an acyclic graph. It consists of a set of nodes r and a set of edges (r, s), each edge connecting two nodes r and s. It is defined as directed, that is, each edge is directed, the first node that defines the edge is called the "parent node" of the edge, and the second node that defines the edge is called the "child node" of the edge. Includes at least one node without any parent node, called the root node, denoted by Decision Tree It also includes at least one node without any child nodes, and each such node without child nodes is called a "terminal node".

[0098] Decision Tree The purpose of is to classify the input, that is, to input a given input word v (a vector of values) and output a class k belonging to a set of classes K. Such a word v consists of a series of values, each of which is associated with a feature i. To classify an input word v, the decision tree tests some of the features i of the input word v and determines the class based on these tests. These features i together form a feature set F. Each feature i can take on values ​​that are relevant to that feature. The set of possible values ​​of feature i is called the domain D of the feature i .

[0099] The input word v belongs to the feature space This feature space represents all the words that can be used as input to the decision tree. (i.e. all possible combinations of eigenvalues) must not be confused with the feature set F (i.e. all features).

[0100] Here are a few examples of features and corresponding domains:

[0101] - Feature i is associated with a belonging set, the domain D of said feature i i is {true, false};

[0102] A feature i is associated with a binary value (for example, the binary value of a Boolean variable), the domain D of said feature i i is {true, false};

[0103] A feature i is associated with an integer value (eg an integer value of an integer variable or a real number), the domain D of said feature i being i is {0, 1, 2};

[0104] - Feature i is related to color, and the domain D of said feature i i is {green, dark, light};

[0105] - Feature i is related to color, and the domain D of said feature i i is {red, blue, green};

[0106] - Feature i is related to texture, and the domain D of feature i i is {clear, slightly blurry, blurry};

[0107] - Feature i is related to sound, and the domain D of said feature i i is {crisp, low, dull};

[0108] In order to be able to classify the input word v, each non-terminal node r is associated with feature i. Each edge (r, s) from the parent node r (if the node r is not a terminal node) to the child node s is the feature domain D relative to the feature i of the parent node r. i In other words, More specifically, the edge feature subset ∈((r, s)) associated with the edge (r, s) includes values ​​that, when the value of feature i of input word v is tested at parent node r, will produce the possible values ​​of feature i of child node s. Therefore, non-terminal node r can be regarded as a decision about the value of feature i of input word v, and if the value belongs to the subset ∈((r, s)) of edge (r, s), the next decision made about input word v will be made on child node s (about the value of the corresponding feature of child node s).

[0109] For a given node r, a unique decision about feature i can be ensured by adding the following constraint: given two distinct child nodes s1, s2 of the parent node r, the overlap between the two edge feature subsets ∈((r, s1)), ∈((r, s2)) of the two child nodes is empty: For a given node r, we can ensure that the decision about feature i is taken by adding the following constraint: given the set of child nodes {s1, ..., s n}, the edge feature subsets of all child nodes of the parent node ∈((r, s1))...∈((r, s n )) is equal to the domain of the feature: ∈((r,s1))∪...∪∈((r,s n ))=D i (∈ of parent node).

[0110] Given a path P, the edge feature subset of node r with respect to feature i (φ(r)=i) can be recorded as ρ(i, P).

[0111] Therefore, when considering the classification of input word v, the decision tree Each feature (value) of the input word is tested, starting from the root node (i.e., by testing the feature relative to the root node), then to the child nodes connected to the root node by edges, whose corresponding subsets include the values ​​of the features tested at the root node, and so on, until the terminal node is reached. Once the terminal node is reached, the class of the terminal node is returned as a prediction. The entire process defines a path P in the decision tree, i.e., a series of nodes (or equivalently edges) starting from the root node to the terminal node.

[0112] You can remember this path (or "from The terminal node of path P can be denoted by τ(P). Considering that each terminal node is associated with a category, the function This function takes a given terminal node r and returns its class Using these notations, the prediction c for path P can be expressed as follows The feature i of node r can be recorded as φ(r).

[0113] Ideally, the input word (i.e., the value list) contains as many elements as the number of features in the feature set F, and the i-th element of the input word corresponds to feature i (where i is an integer identifying the feature). It should be noted that in the context of this specification, the identifier of a feature and the feature itself can be used to name the same object, i.e., the feature.

[0114] 5.1.2 First Example Decision Tree

[0115] Figure 1 denoted a first example decision tree 1. The first example decision tree 1 comprises a plurality of nodes 10 connected by edges 20. There are fifteen nodes (seven nodes are non-terminal nodes and eight nodes are terminal nodes) and fourteen edges. The first example decision tree 1 is a binary tree here, i.e. each non-terminal node has exactly two child nodes.

[0116] Each node 10 is identified by a node identifier 12. Each non-terminal node 10 corresponds to a feature 14. The feature 14 is labeled x in the figure. i And in this example, it belongs to a feature set F = {x1, x2, x3, x4, x5} containing five features.

[0117] The edge 20 connects the parent node and the child node. The edge 20 is associated with the corresponding edge feature subset 22. For example:

[0118] - the edge connecting the node with identifier "1" and the node with identifier "2" corresponds to the edge feature subset {0}; and

[0119] -The edge connecting the node with identifier "1" and the node with identifier "3" corresponds to the edge feature subset {1}.

[0120] Some nodes 10 of the example decision tree 1 are terminal nodes 16. A given terminal node 16 corresponds to a class 18. In this example, the class 18 is either 0 or 1, and the set of classes is the set {0, 1}.

[0121] The input word is a list containing as many elements as the number of features, i.e. 5 in this example. Considering that the possible values ​​of a feature are either 0 or 1, the input word is a list containing 5 integers, each of which is either 0 or 1.

[0122] For example, for the input word v = (0, 0, 1, 1, 0), the path derived from the input word includes the following nodes: the identifiers of these nodes are: "1", "2", "4", "7", "11" and the predictions of these nodes (i.e., the class of the terminal node of the derived path) are "1".

[0123] It should be noted that two different input words may produce the same prediction (for example, (0, 0, 1, 1, 0) and (1, 0, 0, 0, 1) give the prediction "1"), or even lead to the same path (for example, (0, 0, 0, 1, 0) and (0, 0, 0, 0, 1) both lead to paths through nodes "1", "2" and "4").

[0124] 5.1.3 Second Example Decision Tree

[0125] Figure 2 Denotes the second example decision tree 2. Unlike the first example decision tree, the second example decision tree is not a binary tree. The second example decision tree contains thirteen nodes, five of which (nodes "1", "2", "3", "6", and "11") are non-terminal nodes, and eight of which (nodes "4", "5", "7", "8", "9", "10", "12", and "13") are terminal nodes. The class set is equal to {Y, N} (yes and no).

[0126] The edge feature subset of edge (1,2) is {1}, the edge feature subset of edge (1,3) is {2}, and the edge feature subset of edge (1,4) is {3}. Similarly, the edge feature subset of edge (6,10) is {1,3}, and the edge feature subset of edge (6,11) is {2}. This example shows that not all decision trees are binary, and not all edge feature subsets are singletons.

[0127] It should also be noted that the feature domains of the features identified by "1", "2", "4", "5", and "6" are the set {1, 2, 3}, and the feature domain of feature "3" is {1, 2}. This shows that the feature domains of features in a decision tree are not necessarily equal, or even of the same size.

[0128] 5.1.4 Abductive Explanation

[0129] A decision tree can also be defined as a classifier, that is, a The function κ to the class set K (called the classification function) has the input word And returns the corresponding class k∈K (equal to the class k of the terminal node of the path caused by the input word v).

[0130] Given a first word v and its prediction κ(v), the explanation of the prediction (also called weak abductive explanation) is a subset χ of the feature set F (i.e., ), so that any word x (whose feature value (x i ) i∈χ is equal to the feature value of the first word v (v i ) i∈χ ) must produce the same prediction κ(v). In other words, a subset x is weakly abductive if the following condition WeakAXp(x) holds:

[0131]

[0132] Thus, the explanation of a first word v and its prediction κ(v) includes features that may be relevant to explaining “why” inputting the first word v meant that the classifier returned such a prediction κ(v).

[0133] The adverb "may" is important here, because some features of the explanation may not actually be useful (or relevant) for explaining the prediction. More precisely, an explanation x may include features u that can be removed from explanation x, but the resulting explanation x\{i} with said removal is still an explanation. Such removals can be called simplifications of an explanation.

[0134] We can also define weak AXp for a path P in a similar way. Given a path P, there exists at least one word v that can lead to the path P. P , then the word v P There is weak AXp. P The weak AXp of is the weak AXp of path P. It should be noted that any weak AXp for a given word v leading to path P is also the weak AXp of all other words leading to the same path P (because the same decision chain, i.e., the same chain of decisions of nodes and edges traversed in the tree, has been used to produce the terminal node and prediction).

[0135] As the name implies, the shorter the explanation, the easier it is for humans to understand. This is especially true when explaining actual decision trees from large machine learning models (such as deep learning models). When an explanation cannot be simplified as defined above, then the explanation can be designated as an abductive explanation of the prediction. Thus, an AXp is the minimal / irreducible (for a word or for a path) of a weak AXp.

[0136] 5.2 General Principles of the Invention

[0137] The main purpose of the present invention is to efficiently calculate such AXp. To this end, the present invention proposes a basic inheritance process of nodes, as described below.

[0138] 5.2.1 Basic inheritance processing of nodes

[0139] The method according to the present invention is implemented by an electronic device capable of processing a data source. The method is intended to process decision data, which includes a decision tree. and decision trees The input path P in .

[0140] The method includes at least one iteration of a succession process, wherein the succession process Figure 3 Schematically represented in .

[0141] The inheritance process S08 can be seen as a function of the current universal set U, the current node r and the child node s. The current node r and the child node s are connected together by the edge (r, s). The universal set U represents some features that may be irrelevant to the interpretation of the input path P. The current universal set U is a subset of the feature set F.

[0142] The inheritance process S08 includes: a step S20 for determining satisfaction of the first criterion; a step S22 for performing a first sub-inheritance process; a step S24 for determining satisfaction of the second criterion; and a step S26 for performing a second sub-inheritance process. During step S20, the first sub-inheritance process is triggered by satisfaction of the first criterion. During step S24, the second sub-inheritance process is triggered by satisfaction of the second criterion. The inheritance process S08 ends at step S28.

[0143] The first criterion is related to the consistency of the corresponding edge (r', s') of the edge (r, s) and the input path P (where r', s' are two consecutive nodes of the input path P). The corresponding edge (r', s') is associated with the same feature i as the feature of the current node r. In other words, the parent node r' of the corresponding edge (r', s') and the current node r are associated with the same feature i (i.e., the same feature i is tested).

[0144] If the corresponding edge feature subsets ∈((r, s)) and ∈((r′, s′)) of edges (r, s) and (r′, s′) have a non-empty intersection, that is, there is at least one feature value x contained in both subsets ∈((r, s)) and ∈((r′, s′)) i (In D i If the path P contains such an edge (r', s'), then the edge (r, s) and (r', s') are considered consistent. If the path P contains such an edge (r', s'), then the edge (r, s) is said to be consistent with the input path P.

[0145] If such consistency is established, this means that given a value x for feature i i Two words x and y, and considering that the first word x leads to the input path P, if the other word y leads to the path P containing the current node r y , then the path must also contain the child node s (i.e. through the edge (r, s)).

[0146] In the example of the present invention, the first criterion is to satisfy the consistency between the edge (r, s) and the corresponding edge (r', s') in the input path P.

[0147] The first sub-inheritance process includes the function of the child node s. The second sub-inheritance process includes the function of the child node s and the current universal set U.

[0148] The first sub-inheritance process S22 and the second sub-inheritance process S26 aim to allow including features in said current universal set U to interpret the input path P. In other words, these first and second sub-inheritance processes contribute to determining AXp of the path P.

[0149] In the example of the present invention, the second criterion is the negation of the first criterion.

[0150] In another example of the present invention, the second criterion is related to whether the feature i of the current node r belongs to the current universal set U.

[0151] 5.2.2 Determining the most general set

[0152] Figure 4 A schematic diagram showing an example of the present invention. In an example of the present invention, the method comprises: for at least one current universal set U, a decision tree At least the root node The inheritance process S08 is performed, thereby returning the processed tree data. The method then comprises determining the widest universal set UR such that the processed tree data satisfies a third criterion related to the absence of an alternative path Q that would produce a prediction different from the prediction of the input path P.

[0153] The most universal set UR is the complement of AXp in the feature set F (ie, the smallest weak AXp).

[0154] In other words, a basic inheritance process is performed on at least the root node so that the widest universal set UR can be determined.

[0155] In the example of the present invention, the decision tree Each node r of performs the inheritance process no more than once. This condition ensures that the complexity for determining the widest universal set UR (ie, AXp) is reasonable.

[0156] 5.3 First example of embodiment of the present disclosure

[0157] 5.3.1 Construction of hard constraint set

[0158] In the example of the present invention, Figure 4 As shown, the method includes constructing two sets of clauses that can be used to determine the most extensive universal set UR. More specifically, these two sets can be input by a MaxSAT solver and determine the most extensive universal set UR.

[0159] A MaxSAT solver is a device, computer program or equivalent that can input a hard constraint set H (which contains clauses that must be satisfied) and a soft constraint set B (which contains clauses that should be satisfied as much as possible) and determine the most extensive subset UR of the soft constraint set B, such that when each clause of the hard constraint set H is satisfied, the clauses of the most extensive subset UR of the soft constraint set are also satisfied. For example, the article Marques-Silva J., Ignatiev A., Morgado A. (2017) "Horn Maximum Satisfiability: Reductions, Algorithms and Applications." In: Oliveira E., Gama J., Vale Z., Lopes Cardoso H. (eds) Progress in Artificial Intelligence. EPIA 2017. Lecture Notes in Computer Science, vol10423. Springer, Cham. https: / / doi.org / 10.1007 / 978-3-319-65340-2_56 describes a MaxSAT solver designed specifically for solving Horn formulas, making its runtime complexity multinomial.

[0160] In this example of the present invention, each node r is associated with a corresponding clause b r The word b r Can be interpreted as a "blocking clause" that involves a chain of consequences caused by a set of hard constraints, as described below. The term "blocking" is used here because when a given node appears to be equal to false (whether through any constraints or through consequences in the consequence chain), the given node "blocks" certain propagation of its children. The nature of the consequence chain will be further described below.

[0161] Each feature i is associated with the common clause u iAssociated, the universal clause indicates whether the clause is actually universal. The set of these universal clauses is the soft constraint set B. Therefore, in this example, the current universal set U is a subset of the soft constraint set B. More specifically:

[0162] - In the process of building the hard constraint set H through the iteration of inheritance process, the current universal set U can be regarded as the soft constraint set B, and

[0163] -In the process of determining the most extensive universal set UR by the MaxSAT solver, the current universal set U is a variable inside the MaxSAT solver that can change between the empty set and the soft constraint set B. The universal set U is the set of u that the solver satisfies i A collection of .

[0164] Here, the first child inheritance process S22 (whose input is the current node r and its child nodes s) includes adding to the hard constraint set H:

[0165] (H1) First inheritance clause b r →b s .

[0166] As its form implies, the first inherited clause is the clause b associated with the current node r. r and clause b associated with child node s s This clause can also be written as It is logically equivalent to b r →b s .

[0167] The second sub-inheritance S26 process (which inputs the current node r, its child nodes s, and the feature i of the current node r) includes adding to the hard constraint set H:

[0168] (H2) Second inheritance clause b r ∧u u →b s .

[0169] The second inheritance clause is the clause b associated with the current node r r , clause b associated with child node s s and the universal clauses u that belong to the current universal set U i The function of the general clause u i is associated with feature i of the current node r. This clause can also be written as It is logically equivalent to b r ∧u u →b s .

[0170] Therefore, the inheritance process constructs a hard constraint set H containing a first inheritance clause and a second inheritance clause. If the inheritance process is applied on each edge of the decision tree, the resulting inheritance clauses together form a series of logical consequences for traversing down the decision tree.

[0171] In an example of the present invention, the method further includes adding the following sentence to the hard constraint set H:

[0172] (H3) Decision Tree The root node Clause

[0173] (H4) Decision Tree Middle category Clause b for each terminal node r that is equal to the prediction c of the input path r ;as well as

[0174] (H5) Decision Tree Middle category The negation of the clause for each terminal node r that is different from the prediction c of the input path

[0175] In this example, each terminal node is labeled with respect to its consistency with the input path (i.e., the equality of its class compared to the class of the terminal nodes of the input path). We can write the clause It is called the "root clause" and the clause b of the terminal node r and It is called a "terminal clause".

[0176] In the example of the present invention, Figure 5 As shown, the method starts with an initialization procedure S04, which includes: in step S40, setting the hard constraint set H to empty, then in step S42, adding a root clause, and further adding a terminal clause (regarding the consistency of the class of the considered terminal node) in steps S44 and S46. The initialization procedure S04 then ends at step S48. The inheritance process S08 can be performed before or after the initialization procedure S04. The symbol "+=" in the figure represents adding an element (i.e., a clause) to the hard constraint set.

[0177] Therefore, the set of hard constraints also includes clauses associated with the root node and each terminal node. These clauses are connected to the consequence chain described above as described below.

[0178] In an alternative implementation, the addition can be made during the inheritance process. In this alternative, when the inheritance process inputs the current node, the inheritance process determines whether the current node is a root node and then adds the root clause to the hard constraint set H, otherwise it determines whether the current node is a terminal node and then adds the corresponding terminal clause (regarding the consistency of the terminal node with the prediction) to the hard constraint set H.

[0179] In an alternative implementation, the clause may be added To replace clause b r By itself, this is logically equivalent. Similarly, we can add clause b r →⊥ to replace clause b r This is also logically equivalent.

[0180] In the example of the present invention, the inheritance process further includes determining that the feature i of the current node r is not included in the path feature set Φ(P) of the features involved in the input path P. Then, if the feature i is not actually in the path feature set Φ(P), add to the hard constraint set H:

[0181] (H6) General clause u associated with the feature i i .

[0182] Therefore, when feature i does not belong to the path feature set Φ(P), the feature is irrelevant to explaining the input path P (i.e., it does not belong to AXp of P). i Adding to the hard constraint set reduces the time required to determine the broadest universal set UR.

[0183] In an example of the present invention, the method includes: Each node r of performs inheritance processing while considering the feature set F as the current universal set U, thereby returning the hard constraint set H as the processed tree data. Then, the third criterion for determining the most extensive universal set UR is that the hard constraint set H is completely satisfied when any universal set U is considered as the soft constraint set B in the MaxSAT solver.

[0184] Thus, in this case, the inheritance process, once applied to each node of the decision tree, produces a set of hard constraints H consisting of consequence chains (inheritance clauses connecting a node to its children), an “initialization clause” (root clause), and at least one terminal clause.

[0185] Therefore, when the MaxSAT solver inputs the processed tree data (ie, the hard constraint set) and the soft constraint set, the most extensive subset of clauses of the soft constraint set B that are satisfied while fully satisfying the hard constraint set H is determined.

[0186] The set of hard constraints can be visualized as follows. As mentioned above, the inheritance clauses (denoted as H1 and H2) together form a "consequence chain" that traverses the entire decision tree, while the root clause (root clause H1) "initializes" the consequence chain.

[0187] Therefore, given the current universal set U — that is, the universal value u i If a list is declared true (i.e. satisfied) and the other values ​​are declared false, this initialization is propagated through the inheritance clauses to the children of the root node, then their corresponding children, and so on. At a given point, this chain of consequences propagates the truth value to the terminal node, which is the path from the root node. No common features are involved (i.e., there are no The feature i of i , i.e., feature i is universal).

[0188] Therefore, when the consequence chain reaches the given terminal node r, the true value Propagating through the inheritance clause chain to the terminal node, there are two cases:

[0189] (i) If the class of the terminal node If is equal to the prediction c (i.e., the class of the terminal node of the input path P), then there is no inconsistency: the clause of the terminal node is satisfied by adding it to the set of hard constraints H and by propagating the truth value to the chain of consequences of the terminal node r.

[0190] (i) If the class Different from prediction c, there is an inconsistency: terminal clause b r cannot both be false (because the negation of the clause of the terminal node has been added to the hard constraint set H, which must satisfy the negation That is, clause b must be satisfied r ) and are both true (because the truth value is derived from the truth value from the root node through the outgoing path propagates to the terminal node r in the chain of consequences). Therefore, this means that the path There is feature i among the features involved, and the common clause u of this feature i Should have different values.

[0191] 5.3.2 Example of Constraint Set

[0192] Given Figure 1 The decision tree shown, and the path (i.e., P 1→15, where "1" and "15" are node identifiers), the hard constraint set H includes the following clauses (where (H1) and (H2) represent inheritance clauses, (H3) represents a root clause, and (H4) and (H5) represent terminal clauses):

[0193]

[0194] The soft constraint set B is as follows: {(u1), (u2), (u3), (u4), (u5)}.

[0195] In the example of adding the above clause (H6), other clauses (common clauses for features not involved in the input path) must be added to the hard constraint set H. In this case, considering the word v = (0, 1, 1, 1, 0), the path P 1→9 , the hard constraint set H also includes (H6): {(u3), (u5)}.

[0196] 5.3.3 Encoding of Constraint Sets

[0197] As described above, the inheritance process provides processed tree data containing a hard constraint set H and a final soft constraint set B. Considering the fact that the soft constraint set B is common to all input paths and is inherent to the decision tree, the soft constraint set B can be output separately. It allows the soft constraint B to be calculated only once (for a given decision tree) instead of for each AXp of different input paths.

[0198] These processed tree data may be in the form of a list of elements, or alternatively in the form of a single-row table (each cell containing a clause). It may also take the form of a multi-row table as described above, with each row being a type of clause (H1, H2, H3, H4, H5, and H6 if the latter exists).

[0199] Atomic clause (i.e., a clause that cannot be split, so that clause b r (each clause is associated with a node r) and a universal clause u i ) can take the form of a string of a specific type.

[0200] For example, clause b associated with node r r can be encoded in the processed tree data as the following string "_b_r_", where r is replaced by the identifier of node r, the underscore string "_" is used as a separator, and the clause u i can be encoded as "_u_i_". More generally, any form that is handled by a MaxSAT solver (or even a SAT solver) can be used.

[0201] Atoms are grouped together, such as enumerating inheritance clauses (using the implication operator →) or terminating clauses (using the negation operator ), can be of any form that the MaxSAT solver can handle. For example, such clauses can be encoded as follows: for clause b1→b2 “IMPLIES(_b_1_,_b_2_)”, or for the equivalent clause "AND(NOR(_b_1_),_b_2_)".

[0202] If necessary, the processed tree data can be stored in the form of clauses as described above and then processed into a grammar suitable for a given SAT solver.

[0203] 5.4. Second Example of Implementation of the Present Disclosure (Tree Traversal)

[0204] 5.4.1. Recursive tree traversal

[0205] In another example of the present invention, Figure 6 As shown, the inheritance process includes recursive tree traversal. More specifically, the first child inheritance process and the second child inheritance process both include performing inheritance processing on the child node s.

[0206] Therefore, when the inheritance process is called, the inheritance process is performed on the current node r, and if any of the child nodes s of the current node r meets the first criterion or the second criterion, the inheritance process is performed on the child node s. Therefore, the inheritance process traverses the decision tree in a recursive manner.

[0207] In the example of the present invention, if both the first criterion and the second criterion are satisfied, the inheritance process is performed only once, which avoids redundancy and thus reduces the computation time required to perform the entire inheritance process.

[0208] In an example of the present invention, the inheritance process includes determining that the current node (to which the inheritance process is applied) is a terminal node. Then, if the determination returns true, the inheritance process includes returning a result of determining that the prediction of the current node r is different from the prediction of the input path P. In other words, if the terminal node is consistent with the prediction c of the input path (i.e., the class of the terminal node r is different from the prediction c of the input path P), the inheritance process returns a result of determining that the prediction of the current node r is different from the prediction c of the input path P. is equal to prediction c), then the inheritance process returns false. Otherwise, the inheritance process returns true. The terminal end portion of the inheritance process helps to end the inheritance process when it reaches a terminal node.

[0209] In this example, the first criterion is the consistency of edge (r, s) with path P. Recalling the explanation of the concept of consistency above, consistency can be expressed as the non-empty intersection of the following:

[0210] - the edge feature subset ∈((r, s)) of edge (r, s), and

[0211] - The set of edge features ρ(i, P) of the edges of the input path P associated with the same feature i (i=φ(r)) as the current node r.

[0212] In this example, the second criterion is that the feature i tested in the current node r (ie, φ(r)=i) belongs to the current universal set U.

[0213] Therefore, the first criterion can be expressed as Therefore, the second criterion can be expressed as (i∈U).

[0214] When the current node r is not a terminal node and no child node s in the terminal node satisfies the first criterion or the second criterion, the inheritance process returns false.

[0215] To summarize the example of inheritance processing described above, it can be described as an example of inheritance processing as follows:

[0216]

[0217] The first part of the above inheritance processing example is the above-mentioned terminal node that ends.

[0218] In the second part of the above inheritance processing example, the first criterion and the second criterion are tested for each child node s of the current node r. If one (or both) of the first criterion and the second criterion is met, the inheritance processing is performed on the child node s.

[0219] Therefore, the inheritance process is performed on each child node of the current node until the terminal node is reached (or a node where there is no child node such that the first criterion or the second criterion is satisfied). The inheritance process then returns a Boolean value (true or false) as the result of the inheritance process.

[0220] The result returned by the inheritance process is the replacement existence value. This value returned once the inheritance process is applied to the current node r represents the decision tree There exists (at least) an alternative path Q that starts from the current node r and produces the same prediction as the input path P. Different predictions For non-universal features, they are consistent with the input path P.

[0221] 5.4.2 Determination of the universality of features

[0222] Thus, the above inheritance process makes it possible to determine whether another feature i is actually universal given a universal feature set U of actually universal features. Finally, it is possible to start from an empty universal feature set U.

[0223] To this end, the method according to the example of the present invention first includes: adding the current feature i to be tested to the current universal set U (including features that are actually universal through previous tests or through other ways of establishing universality). Then, the method includes: And the current universal feature set U performs inheritance processing.

[0224] Here, as described above, the inheritance process returns an alternative existence value, which represents the decision tree There exists (at least) an alternative path Q from the root node Start and generate predictions with input path P Different predictions For non-universal features (i.e., features that do not belong to the current universal set U), they are consistent with the input path P.

[0225] Then, if the value returned by the inheritance process is true, this means that the current feature i is actually relevant to explaining the input path P, because including the current feature i in the current universal set U implies that there is an alternative path Q that is consistent with the input path P but produces a different prediction. In this case, the current feature i cannot be universal (it must be fixed / restricted to its domain in P). The current feature i is removed from the current universal set U.

[0226] Therefore, the method according to the invention can determine, for a given feature i, and given a current universal set U of features that are in fact universal, whether the given feature is universal (or in other words, whether the feature belongs to AXp of the input path P).

[0227] 5.4.3 Tree traversal example

[0228] This section describes an example of a tree traversal given an input path and a given current node. The numbers referenced refer to the nodes they identify. For better readability, the first criterion is referred to here as the consistency criterion (about a certain feature), while the second criterion is referred to as the universality criterion (for a certain feature).

[0229] Considering Figure 1 The decision tree shown, input path P 1→9 =(1, 2, 4, 7, 11) and the root node identified by "1" is used as the current node at which the inheritance process starts.

[0230] Considering the current universal set U = F\Φ(P) = {x5}, i.e., universal features are only features not involved in the input path P, the only child nodes of "1" that meet the first or second criteria are "2", then "4", then "7", then "11". Then, the inheritance process returns false. Obviously, this means that the features not involved in the input path are actually irrelevant to interpreting the input path.

[0231] Now let us consider the current universal set U = {x2, x5}. The inheritance process goes through "1", then "2" (for the consistency criterion on x1), then "4" and "5" (for the universality criterion on x2), and

[0232] - starts with "4", then "7", then "11" (consistency criteria on x3 then consistency criteria on x4), and

[0233] - Start with "5" and then "9" (consistency standard on x4).

[0234] Both terminal nodes here return false (because they are associated with class 1). Therefore, feature x2 is consistent with the explanation input path P. 1→9 Therefore, the input path P 1→9 AXp does not include x2.

[0235] 5.4.4 Determination of the most general set

[0236] exist Figure 7 In the example of the present invention shown, the method comprises the following steps:

[0237] (S201) Initializing the current universal set U as the complement of the path feature set Φ(P) of the features involved in the input path P in the feature set F;

[0238] (S202) For at least one current feature i of the path feature set Φ(P):

[0239] (S202a) adding the current feature i to the current universal set U;

[0240] (S202b) Considering the current universal set U and the input path P, determine the inheritance process once at the root node Returns true after execution.

[0241] (S202c) If the inheritance process returns true, remove the current feature i from the current universal set U;

[0242] (S203) Return the current universal set U as the most extensive universal set UR.

[0243] In an example of the present invention, step (2) is performed for each feature in the path feature set Φ(P).

[0244] This example of the method according to the invention can be summarized as follows (where EXISTSCONSISTENTQPATH is an inheritance process, in Figure 7 EQP), and the most universal set UR is the current universal set U returned by the function UR):

[0245]

[0246] Alternatively, the method may return F\U instead of U, i.e. AXp of the input path P.

[0247] Thus, an example of a method according to the invention comprises testing each feature involved in an input path P to determine whether said feature is actually relevant to interpreting the input path (ie belongs to AXp of the input path P).

[0248] Given that inheritance processing traverses the decision tree much faster than conventional exhaustive tree traversal, the determination of alternative existence values ​​is faster, and thus the computation of AXp (or the widest universal set UR) is faster.

[0249] 5.5. Probabilistic Path Abduction (P2AXp)

[0250] In the preceding examples and explanations, the deterministic computation of abductive explanations (AXp) has been described in a deterministic (i.e., definite) way, where universal features are in fact universal.

[0251] In an alternative example, the method of the present invention includes determining a probabilistic path abductive explanation (P2AXp).

[0252] In the case of a probabilistic path AXp, a given feature is declared universal if the number of paths that go through any node with said feature and lead to a different class (different from the test result) is small enough. In other words, it means that the probability that this given feature is actually not universal is small enough. In this case, one can consider that the probability of a different class is small enough and, therefore, this different class is not considered part of the actual possibilities provided by this feature.

[0253] The goal here is to trade off a little rigor with a smaller size of interpretation (compared to the deterministic computation described above).

[0254] Given the selected features in the explanation (i.e., the set ) and the set of common features (i.e., the set ) makes and Each individual path probability can then be calculated. Therefore,

[0255] Given a target δ∈[0, 1], the goal of computing the probability path AXp is to find the set Make

[0256]

[0257] The inventors note that in practice, the set X calculated using the above method is usually small (although this is not mandatory in rare cases). In addition, the algorithm described in the previous example can still be used even if the starting point of the algorithm is not necessarily deterministic.

[0258] 5.6. Other technical aspects

[0259] Figure 8 A schematic block diagram of an example of a device DD according to an embodiment of the present disclosure is shown. In some embodiments, the device DD is connected to or embedded in a communication device (development environment, database server, cloud-based service) configured for processing a database. In another embodiment, the device DD is implemented on a cloud infrastructure service, for example together with a remote database server. Regardless of which embodiment is considered, the device DD is connected to the database server directly or indirectly through a communication device.

[0260] Refer again Figure 8 , the device DD includes a processor 801, a storage unit 802, an input device 803, an output device 804, and an interface unit 805 connected via a bus 806. Of course, the constituent elements of the device DD may be connected by a connection method other than a bus connection using the bus 806.

[0261] The processor 801 controls the operation of the device DD. The storage unit 802 stores at least one program to be executed by the processor 801, and various data, including, for example, parameters used in the calculations performed by the processor 801, intermediate data of the calculations performed by the processor 801, such as decision trees, decision data, statistical data, etc. The processor 801 is formed by appropriate hardware, or software, or a combination of hardware and software. For example, the processor 801 is formed by dedicated hardware such as a processing circuit or a programmable processing unit such as a CPU (central processing unit) that executes a program stored in its memory.

[0262] The storage unit 802 is formed by any suitable memory or a device capable of storing programs, data, etc. in a computer-readable manner. Examples of the storage unit 802 include non-transitory computer-readable storage media, such as semiconductor memory devices, and magnetic, optical or magneto-optical recording media loaded into a read-write unit. The program causes the processor 801 to execute a method for processing decision data, the method being implemented by an electronic device for processing a data source, the decision data comprising a decision tree (T) and an input path (P) in the decision tree (T), and according to the embodiment of the present disclosure as described above, each node (r) of the decision tree is associated with a corresponding feature (i) in a feature set (F). More specifically, the program causes the processor 801 to perform an iteration of inheritance processing, the inheritance processing being a function of the current universal set (U) and a function of the current node (r) and the child node (s) connected together by an edge ((r, s)), the current universal set (U) representing some features that may be irrelevant to interpreting the input path, the current universal set (U) being included in the feature set (F), as described herein before.

[0263] The input device 803 is formed by, for example, a keyboard or a mouse, as necessary.

[0264] The output device 804 is for example formed by a processing unit configured to determine features and paths allowing explanation of the decisions in the decision tree.

[0265] The interface unit 805 provides an interface between the device DD and external devices and / or systems. The interface unit 805 is typically a communication interface that allows the device to communicate with other devices connected to the network (e.g., a database server). The interface unit 805 can be used to obtain database statistics through a dedicated API. The interface unit 805 can also be used to transmit the calculation results of the path or decision tree.

[0266] although Figure 8 Only one processor 801 is shown in the figure, but it must be understood that such a processor may include different modules and units that embody the functions performed by the device DD according to the embodiments of the present disclosure. These modules and units may also be embodied in multiple processors 801 that communicate and cooperate with each other.

[0267] While the present disclosure has been described with reference to exemplary embodiments, it should be understood that the present disclosure can be embodied in various forms and is not limited to the examples discussed above.

[0268] One or more aspects of the present disclosure may be embodied in computer-usable data or computer-executable instructions, such as embodied in one or more program modules, which are executed by one or more computers or other devices to perform the operations described herein. In general, program modules include routines, programs, objects, components, data structures, etc., which perform specific tasks or implement specific abstract data types when executed by one or more processors in a computer or other data processing device. Computer-executable instructions can be stored as computer-readable instructions on a computer-readable medium (e.g., a hard disk, an optical disk, a removable storage medium, a solid-state memory, a RAM, etc.). In various embodiments, the functions of the program modules can be combined or distributed as needed. In addition, the function can be embodied in firmware or hardware equivalents (e.g., integrated circuits, application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), etc.) in whole or in part. Specific data structures can be used to more effectively implement one or more aspects of the present disclosure, and these data structures are considered to be within the scope of computer-executable instructions and computer-usable data described herein.

[0269] Various aspects described herein may be embodied as one or more computer-readable media storing computer executable instructions, a method, a device, or a computer-readable medium. Therefore, these aspects may be implemented in full hardware, full software, full firmware, or in any combination in the form of an implementation in combination with software, hardware, and firmware. In addition, various signals representing data or events described herein may be transmitted between a source and a destination in the form of light or electromagnetic waves, which travel through a signal-conducting medium (e.g., a metal wire, an optical fiber, or a wireless transmission medium (e.g., air or space)). In general, one or more computer-readable media may be one or more non-transient computer-readable media and / or include one or more non-transient computer-readable media.

Claims

1. A method for processing decision data, the method being implemented by an electronic device for processing a data source, the decision data comprising a decision tree And through the decision tree The decision path (P) of the nodes of the decision tree, each node (r) of the decision tree is associated with a corresponding feature (i) of the feature set (F): The method is characterized in that it determines a universal set (U) comprising features that are substantially irrelevant for explaining the decision path, and in that, The method comprises: at least one iteration of a first function, the first function being a function of a current set (U) and of a current node (r) and a child node (s) connected together by an edge ((r, s)), the current set (U) representing some features irrelevant for explaining the decision path, the current set (U) being included in the set of features (F), the first function comprising: - determining that the edge ((r, s)) verifies a first criterion, the first criterion being related to the consistency of the edge ((r, s)) with the corresponding edge ((r′, s′)) of the decision path (P), the corresponding edge ((r′, s′)) of the decision path (P) being related to a feature (i) identical to the feature of the current node (r), and satisfying the first criterion triggers execution of a second function that is a function of the child node (s); and / or - determining that said edge ((r, s)) verifies a second criterion, the satisfaction of which triggers the execution of a third function as a function of said child node (s) and said current set (U), said second function and said third function causing the inclusion of a feature in said current set (U), and - delivering said universal set (U) for interpreting said input path (P) based on said current set.

2. The method according to claim 1, characterized in that The second criterion is the negation of the first criterion.

3. The method according to claim 1, characterized in that The second criterion is related to whether the feature (i) of the current node belongs to the current universal set (U).

4. The method according to claim 1, wherein: Each node (r) is associated with a corresponding clause (b r ), and where: The second function comprises adding a first inheritance clause (b) to the set of hard constraints (H) representing the set of constraints that must be satisfied for the MaxSAT solver r →b s ), the first inheritance clause is a clause (b) associated with the current node (r) r ) and the clause (b) associated with said child node(s) s ) function; and - the third function comprises adding a second inheritance clause (b) to the hard constraint set (H) r ∧u u →b s ), the second inheritance clause is a clause (b r ), the clause (b) associated with the child node (s) s ) and the universal clauses (u i ) function, the general clause (u i ) is associated with the feature (i) of the current node (r).

5. The method according to claim 4, wherein: The method further comprises adding to the set of hard constraints (H): - The decision tree The root node Clause - The decision tree Middle category The clause (b) for each terminal node (r) equal to the prediction (c) of the decision path r );as well as - The decision tree Middle category The negation of the clause for each terminal node (r) that differs from the prediction (c) of the decision path 6. The method according to claim 4, characterized in that The first function further comprises: determining that the feature (i) of the current node (r) is not included in the path feature set (Φ(P)) of the features involved in the decision path (P), and then adding a universal clause (u) associated with the feature (i) to the hard constraint set (H); i ).

7. The method according to claim 1, characterized in that The second function and the third function both include executing the first function on the child node(s).

8. The method according to claim 7, characterized in that The first function also includes: determining that the current node (r) is a terminal node, and then returning to determine the prediction of the current node (r) An outcome that differs from the prediction (c) of the decision path (P).

9. The method according to claim 8, wherein: The method comprises: - for at least one current set (U) execute as said decision tree The root node , thereby returning information indicating that there is an alternative path (Q) that produces a prediction different from the prediction of the decision path and is consistent with the decision path for features not included in the current set; and - determining the most universal set (UR) such that the piece of information meets a third criterion relating to the absence of alternative paths (Q) that would lead to a prediction different from the prediction of the decision path (P).

10. The method according to claim 9, characterized in that For a given current set (U), the decision tree Each node (r) executes the first function no more than once.

11. The method according to claim 9, characterized in that The piece of information indicating the existence of the alternative path (Q) comprises a Boolean value, the Boolean value being true if such an alternative path exists and the Boolean value being false if such an alternative path does not exist, and wherein the third criterion is that the alternative existence value is true.

12. The method according to claim 11, characterized in that Determining the widest universal set (UR) includes: - Initializing the current set (U) as the complement of the path feature set Φ(P) of the features involved in the decision path (P) in the feature set (F); -For at least one current feature (i) of the path feature set Φ(P): * add the current feature (i) to the current set (U), * Determine that the first function returns true, and the first function is in the root node and based on the current set (U), then removing the current feature (i) from the current set (U); and - Return the current set (U) as the widest universal set (UR).

13. The method according to any one of claims 4 to 6, characterized in that The method comprises: - the decision tree Each node (r) of executes the first function while considering the feature set (F) as the current set (U), thereby returning the hard constraint set (H) as processed tree data; and - Using a MaxSAT solver, a most universal set (UR) is determined as the most extensive subset of a set of soft constraints whose clauses can be satisfied when every clause belonging to the set of hard constraints (H) is satisfied.

14. An electronic device for processing decision data, the decision data comprising a decision tree And through the decision tree The decision path (P) of the nodes of the decision tree, each node (r) of the decision tree is associated with a corresponding feature (i) of the feature set (F), The device is characterized in that it is configured to determine a universal set (U) comprising features that are substantially irrelevant for explaining the decision path, and wherein The device comprises means for processing at least one iteration of a first function, the first function being a function of a current set (U) and of a current node (r) and a child node (s) connected together by an edge ((r, s)), the current set (U) representing some features not relevant for explaining the decision path, the current set (U) being included in the set of features (F), the first function comprising: - determining that the edge ((r, s)) verifies a first criterion, the first criterion being related to the consistency of the edge ((r, s)) with the corresponding edge ((r′, s′)) of the decision path (P), the corresponding edge ((r′, s′)) of the decision path (P) being related to a feature (i) identical to the feature of the current node (r), and satisfying the first criterion triggers execution of a second function that is a function of the child node (s); and / or - determining that said edge ((r, s)) verifies a second criterion, the satisfaction of which triggers the execution of a third function that is a function of said child node (s) and said current set (U), said second function and said third function causing the inclusion of a feature in said current set (U), - delivering said universal set (U) for interpreting said input path (P) based on said current set.

15. A non-transitory computer-readable medium, comprising a computer program product recorded on the non-transitory computer-readable medium and executable by a processor, the computer program product comprising program code instructions for implementing the method according to any one of claims 1 to 13.

Images