Method, system, and computer program product for perturbation-based interpretation of the effects of related features in machine learning models

Abstract

Described are a system, method, and computer program product for perturbation-based interpretation of the effects of related features in machine learning models. The method includes generating a knowledge graph based on features and relationships, the graph including nodes connected edges. The method also includes determining feature values, determining a first feature of a first node, and perturbating a value of the first feature. The method further includes determining a second feature of a second node based on an edge connecting the first node to the second node and perturbating a value of the second feature based on the new value of the first feature. The method further includes inputting the feature values to a decision-based machine learning model and determining a significance of an effect of the first and second features on the decision-based machine learning model based on the new values of the first and second features.

Description

METHOD, SYSTEM, AND COMPUTER PROGRAM PRODUCT FOR PERTURBATION-BASED INTERPRETATION OF THE EFFECTS OF RELATED FEATURES IN MACHINE LEARNING MODELSBACKGROUND1 . Technical Field

[0001] This disclosure relates generally to improved machine learning models and, in non-limiting embodiments or aspects, to methods, systems, and computer program products for interpreting the effects of related features in machine learning models using relational perturbation, to improve model execution.2. Technical Considerations

[0002] Machine learning models may be trained to generate outputs using a plurality of features. However, understanding how features, or which features, affect the output of machine learning models may be difficult. Furthermore, because features may be related to one another, such as through correlation or dependency, it may not be accurate to investigate a single feature’s effect on a model without taking feature relationships into consideration. The ability to explain a feature or model output affects not only the reliability of feature interpretation, but also the accuracy of the associated models.

[0003] It is further important to identity which features have a significant effect on the output of a model, given that the number of features used is related to the number of computations required to execute a machine learning model. By executing a machine learning model using features that do not affect the output of the model, computing resources are wasted (e.g., bandwidth, processing capacity, memory, etc.). Computer resources waste effects are even more apparent with large-scale machine learning models using a high number of features, which may retain a number of irrelevant or redundant features therein.

[0004] There is a need in the art for a technical solution to improve the interpretability of feature effects on the output of machine learning models, and to further reduce the computational cost required to interpret feature effects and execute machine learning models on such features.SUMMARY

[0005] According to some non-limiting embodiments or aspects, provided are systems, methods, and computer program products for perturbation-based interpretation of the effects of related features in machine learning models that overcome some or all of the deficiencies identified above.

[0006] According to some non-limiting embodiments or aspects, provided is a computer-implemented method for perturbation-based interpretation of the effects of related features in machine learning models. The method includes generating, with at least one processor, a knowledge graph based on a plurality of features and a plurality of relationships between features of the plurality of features. The knowledge graph includes a plurality of nodes connected by a plurality of edges. Each node of the plurality of nodes is associated with a feature of the plurality of features. Each edge of the plurality of edges is associated with a relationship of the plurality of relationships between two features of the plurality of features. The method also includes determining, with at least one processor, a plurality of feature values associated with the plurality of features. The method further includes determining, with at least one processor, a first feature associated with a first node of t. / he plurality of nodes. The method further includes perturbating, with at least one processor, a feature value of the plurality of feature values associated with the first feature. Perturbating the feature value associated with the first feature includes updating the feature value associated with the first feature to a new feature value associated with the first feature. The method further includes determining, with at least one processor, a second feature associated with a second node of the plurality of nodes, based on an edge connecting the first node to the second node. The method further includes perturbating, with at least one processor, a feature value of the plurality of feature values associated with the second feature based on the new feature value associated with the first feature. Perturbating the feature value associated with the second feature includes updating the feature value associated with the second feature to a new feature value associated with the second feature, based on the new feature value associated with the first feature. The method further includes inputting, with at least one processor, the plurality of feature values to a decision-based machine learning model and determining, with at least one processor, a significance of an effect of the first feature and the second feature on an output of the decision-based machine learning modelbased on the new feature value associated with the first feature and the new feature value associated with the second feature.

[0007] In some non-limiting embodiments or aspects, the method may include displaying, with at least one processor, the plurality of features in a user interface of a computing device of a user. The method may also include receiving, with at least one processor, user input including an identification of at least one pairing of features of the plurality of features. The method may further include determining, with at least one processor, the plurality of relationships between the features of the plurality of features based at least partly on the at least one pairing of features of the user input.

[0008] In some non-limiting embodiments or aspects, the method may include storing, with at least one processor, a node table associated with the plurality of nodes of the knowledge graph. The node table may identify, for each node of the plurality of nodes, at least one perturbation parameter of the feature associated with the node. Perturbating the feature value of the plurality of feature values associated with the first feature may further include updating the feature value associated with the first feature to a new feature value associated with the first feature based on the node table, wherein the new feature value associated with the first feature satisfies the at least one perturbation parameter of the first feature associated with the first node.

[0009] In some non-limiting embodiments or aspects, the method may include storing, with at least one processor, an edge table associated with the plurality of edges of the knowledge graph. The edge table may identify, for each edge of the plurality of edges, a source node of the plurality of nodes connected by the edge, a target node of the plurality of nodes connected by the edge, and a constraint on a feature value of a feature associated with the target node based on a feature value of a feature associated with the source node. Perturbating the feature value of the plurality of feature values associated with the second feature may further include updating the feature value associated with the second feature to a new feature value associated with the second feature based on the node table and the edge table, wherein the new feature value associated with the second feature satisfies the at least one perturbation parameter of the second feature associated with the second node, and satisfies the constraint on the feature value of the second feature based on the feature value of the first feature.

[0010] In some non-limiting embodiments or aspects, determining the significance of the effect of the first feature and the second feature on the output of the decision-based machine learning model may include generating a first output of the decisionbased machine learning model based on the plurality of feature values associated with the plurality of features before perturbating feature values of the plurality of feature values associated with the first feature and the second feature, generating a second output of the decision-based machine learning model based on the plurality of feature values associated with the plurality of features after perturbating feature values of the plurality of feature values associated with the first feature and the second feature, and comparing the first output to the second output.

[0011] In some non-limiting embodiments or aspects, the method may include reducing, with at least one processor, the plurality of features input to the decisionbased machine learning model based on the significance of the effect of the first feature and the second feature on the output of the decision-based machine learning model, to produce a reduced plurality of features. The method may also include executing, with at least one processor, the decision-based machine learning model based on the reduced plurality of features.

[0012] In some non-limiting embodiments or aspects, reducing the plurality of features input to the decision-based machine learning model based on the significance of the effect of the first feature and the second feature may include determining that the second output is not significantly different than the first output, and removing the first feature and the second feature from the plurality of features.

[0013] In some non-limiting embodiments or aspects, reducing the plurality of features input to the decision-based machine learning model based on the significance of the effect of the first feature and the second feature may include determining that the second output is significantly different than the first output, and removing at least one feature from the plurality of features, wherein the at least one feature is not the first feature nor the second feature.

[0014] In some non-limiting embodiments or aspects, the method may include determining, with at least one processor, a third feature associated with a third node of the plurality of nodes, based on an edge connecting the second node to the third node. The method may also include perturbating, with at least one processor, a feature value of the plurality of feature values associated with the third feature based on the new feature value associated with the second feature. Perturbating the feature value associated with the third feature may include updating the feature value associated with the third feature to a new feature value associated with the thirdfeature, based on the new feature value associated with the second feature. The method may further include determining, with at least one processor, a significance of an effect of the first feature, the second feature, and the third feature on the output of the decision-based machine learning model based on the new feature value associated with the first feature, the new feature value associated with the second feature, and the new feature value associated with the third feature.

[0015] In some non-limiting embodiments or aspects, provided is a system for perturbation-based interpretation of the effects of related features in machine learning models. The system includes at least one processor programmed or configured to generate a knowledge graph based on a plurality of features and a plurality of relationships between features of the plurality of features. The knowledge graph includes a plurality of nodes connected by a plurality of edges. Each node of the plurality of nodes is associated with a feature of the plurality of features. Each edge of the plurality of edges is associated with a relationship of the plurality of relationships between two features of the plurality of features. The at least one processor is also programmed or configured to determine a plurality of feature values associated with the plurality of features. The at least one processor is further programmed or configured to determine a first feature associated with a first node of the plurality of nodes. The at least one processor is further programmed or configured to perturbate a feature value of the plurality of feature values associated with the first feature. When perturbating the feature value associated with the first feature, the at least one processor is programmed or configured to update the feature value associated with the first feature to a new feature value associated with the first feature. The at least one processor is further programmed or configured to determine a second feature associated with a second node of the plurality of nodes, based on an edge connecting the first node to the second node. The at least one processor is further programmed or configured to perturbate a feature value of the plurality of feature values associated with the second feature based on the new feature value associated with the first feature. When perturbating the feature value associated with the second feature, the at least one processor is programmed or configured to update the feature value associated with the second feature to a new feature value associated with the second feature, based on the new feature value associated with the first feature. The at least one processor is further programmed or configured to input the plurality of feature values to a decision-based machine learning model and determine a significance ofan effect of the first feature and the second feature on an output of the decision-based machine learning model based on the new feature value associated with the first feature and the new feature value associated with the second feature.

[0016] In some non-limiting embodiments or aspects, the at least one processor may be further programmed or configured to display the plurality of features in a user interface of a computing device of a user. The at least one processor may further be programmed or configured to receive user input including an identification of at least one pairing of features of the plurality of features. The at least one processor may further be programmed or configured to determine the plurality of relationships between the features of the plurality of features based at least partly on the at least one pairing of features of the user input.

[0017] In some non-limiting embodiments or aspects, the at least one processor may be further programmed or configured to store a node table associated with the plurality of nodes of the knowledge graph. The node table may identify, for each node of the plurality of nodes, at least one perturbation parameter of the feature associated with the node. When perturbating the feature value of the plurality of feature values associated with the first feature, the at least one processor may be further programmed or configured to update the feature value associated with the first feature to a new feature value associated with the first feature based on the node table, wherein the new feature value associated with the first feature satisfies the at least one perturbation parameter of the first feature associated with the first node.

[0018] In some non-limiting embodiments or aspects, the at least one processor may be further programmed or configured to store an edge table associated with the plurality of edges of the knowledge graph. The edge table may identify, for each edge of the plurality of edges, a source node of the plurality of nodes connected by the edge, a target node of the plurality of nodes connected by the edge, and a constraint on a feature value of a feature associated with the target node based on a feature value of a feature associated with the source node. When perturbating the feature value of the plurality of feature values associated with the second feature, the at least one processor may be programmed or configured to update the feature value associated with the second feature to a new feature value associated with the second feature based on the node table and the edge table, wherein the new feature value associated with the second feature satisfies the at least one perturbation parameter of the secondfeature associated with the second node, and satisfies the constraint on the feature value of the second feature based on the feature value of the first feature.

[0019] In some non-limiting embodiments or aspects, when determining the significance of the effect of the first feature and the second feature on the output of the decision-based machine learning, the at least one processor may be programmed or configured to generate a first output of the decision-based machine learning model based on the plurality of feature values associated with the plurality of features before perturbating feature values of the plurality of feature values associated with the first feature and the second feature, generate a second output of the decision-based machine learning model based on the plurality of feature values associated with the plurality of features after perturbating feature values of the plurality of feature values associated with the first feature and the second feature, and compare the first output to the second output.

[0020] In some non-limiting embodiments or aspects, the at least one processor may be further programmed or configured to reduce the plurality of features input to the decision-based machine learning model based on the significance of the effect of the first feature and the second feature on the output of the decision-based machine learning model, to produce a reduced plurality of features, and execute the decisionbased machine learning model based on the reduced plurality of features.

[0021] In some non-limiting embodiments or aspects, when reducing the plurality of features input to the decision-based machine learning model based on the significance of the effect of the first feature and the second feature, the at least one processor may be programmed or configured to determine that the second output is not significantly different than the first output and remove the first feature and the second feature from the plurality of features.

[0022] In some non-limiting embodiments or aspects, when reducing the plurality of features input to the decision-based machine learning model based on the significance of the effect of the first feature and the second feature, the at least one processor may be programmed or configured to determine that the second output is significantly different than the first output, and remove at least one feature from the plurality of features, wherein the at least one feature is not the first feature nor the second feature.

[0023] In some non-limiting embodiments or aspects, the at least one processor may be further programmed or configured to determine a third feature associated witha third node of the plurality of nodes, based on an edge connecting the second node to the third node; perturbate a feature value of the plurality of feature values associated with the third feature based on the new feature value associated with the second feature, wherein, when perturbating the feature value associated with the third feature, the at least one processor is programmed or configured to: update the feature value associated with the third feature to a new feature value associated with the third feature, based on the new feature value associated with the second feature; and determine a significance of an effect of the first feature, the second feature, and the third feature on the output of the decision-based machine learning model based on the new feature value associated with the first feature, the new feature value associated with the second feature, and the new feature value associated with the third feature.

[0024] According to some non-limiting embodiments or aspects, provided is a computer program product perturbation-based interpretation of the effects of related features in machine learning models. The computer program product includes at least one non-transitory computer-readable medium including one or more instructions that, when executed by at least one processor, cause the at least one processor to generate a knowledge graph based on a plurality of features and a plurality of relationships between features of the plurality of features. The knowledge graph includes a plurality of nodes connected by a plurality of edges. Each node of the plurality of nodes is associated with a feature of the plurality of features. Each edge of the plurality of edges is associated with a relationship of the plurality of relationships between two features of the plurality of features. The one or more instructions also cause the at least one processor to determine a plurality of feature values associated with the plurality of features. The one or more instructions further cause the at least one processor to determine a first feature associated with a first node of the plurality of nodes. The one or more instructions further cause the at least one processor to perturbate a feature value of the plurality of feature values associated with the first feature. The one or more instructions that cause the at least one processor to perturbate the feature value associated with the first feature cause the at least one processor to: update the feature value associated with the first feature to a new feature value associated with the first feature; determine a second feature associated with a second node of the plurality of nodes, based on an edge connecting the first node to the second node; perturbate a feature value of the plurality of feature values associated with the second feature based on the new feature value associated withthe first feature, wherein the one or more instructions that cause the at least one processor to perturbate the feature value associated with the second feature cause the at least one processor to: update the feature value associated with the second feature to a new feature value associated with the second feature, based on the new feature value associated with the first feature; input the plurality of feature values to a decision-based machine learning model; and determine a significance of an effect of the first feature and the second feature on an output of the decision-based machine learning model based on the new feature value associated with the first feature and the new feature value associated with the second feature.

[0025] In some non-limiting embodiments or aspects, the one or more instructions may cause the at least one processor to display the plurality of features in a user interface of a computing device of a user, receive user input including an identification of at least one pairing of features of the plurality of features, and determine the plurality of relationships between the features of the plurality of features based at least partly on the at least one pairing of features of the user input.

[0026] In some non-limiting embodiments or aspects, the one or more instructions may further cause the at least one processor to store a node table associated with the plurality of nodes of the knowledge graph. The node table may identify, for each node of the plurality of nodes, at least one perturbation parameter of the feature associated with the node. The one or more instructions that cause the at least one processor to perturbate the feature value of the plurality of feature values associated with the first feature may cause the at least one processor to update the feature value associated with the first feature to a new feature value associated with the first feature based on the node table, wherein the new feature value associated with the first feature satisfies the at least one perturbation parameter of the first feature associated with the first node.

[0027] In some non-limiting embodiments or aspects, the one or more instructions may further cause the at least one processor to store an edge table associated with the plurality of edges of the knowledge graph. The edge table may identify, for each edge of the plurality of edges, a source node of the plurality of nodes connected by the edge, a target node of the plurality of nodes connected by the edge, and a constraint on a feature value of a feature associated with the target node based on a feature value of a feature associated with the source node. Wherein the one or more instructions that cause the at least one processor to perturbate the feature value of theplurality of feature values associated with the second feature may cause the at least one processor to update the feature value associated with the second feature to a new feature value associated with the second feature based on the node table and the edge table, wherein the new feature value associated with the second feature satisfies the at least one perturbation parameter of the second feature associated with the second node, and satisfies the constraint on the feature value of the second feature based on the feature value of the first feature.

[0028] In some non-limiting embodiments or aspects, the one or more instructions that cause the at least one processor to determine the significance of the effect of the first feature and the second feature on the output of the decision-based machine learning may cause the at least one processor to generate a first output of the decisionbased machine learning model based on the plurality of feature values associated with the plurality of features before perturbating feature values of the plurality of feature values associated with the first feature and the second feature, generate a second output of the decision-based machine learning model based on the plurality of feature values associated with the plurality of features after perturbating feature values of the plurality of feature values associated with the first feature and the second feature, and compare the first output to the second output.

[0029] In some non-limiting embodiments or aspects, the one or more instructions may further cause the at least one processor to reduce the plurality of features input to the decision-based machine learning model based on the significance of the effect of the first feature and the second feature on the output of the decision-based machine learning model, to produce a reduced plurality of features, and execute the decisionbased machine learning model based on the reduced plurality of features.

[0030] In some non-limiting embodiments or aspects, the one or more instructions that cause the at least one processor to reduce the plurality of features input to the decision-based machine learning model based on the significance of the effect of the first feature and the second feature may cause the at least one processor to determine that the second output is not significantly different than the first output, and remove the first feature and the second feature from the plurality of features.

[0031] In some non-limiting embodiments or aspects, the one or more instructions that cause the at least one processor to reduce the plurality of features input to the decision-based machine learning model based on the significance of the effect of the first feature and the second feature may cause the at least one processor to determinethat the second output is significantly different than the first output, and remove at least one feature from the plurality of features, wherein the at least one feature is not the first feature nor the second feature.

[0032] In some non-limiting embodiments or aspects, the one or more instructions may further cause the at least one processor to determine a third feature associated with a third node of the plurality of nodes, based on an edge connecting the second node to the third node. The one or more instructions may further cause the at least one processor to perturbate a feature value of the plurality of feature values associated with the third feature based on the new feature value associated with the second feature. The one or more instructions that cause the at least one processor to perturbate the feature value associated with the third feature cause the at least one processor to update the feature value associated with the third feature to a new feature value associated with the third feature, based on the new feature value associated with the second feature. The one or more instructions may further cause the at least one processor to determine a significance of an effect of the first feature, the second feature, and the third feature on the output of the decision-based machine learning model based on the new feature value associated with the first feature, the new feature value associated with the second feature, and the new feature value associated with the third feature.

[0033] Other non-limiting embodiments or aspects will be set forth in the following numbered clauses:

[0034] Clause 1 : A computer-implemented method comprising: generating, with at least one processor, a knowledge graph based on a plurality of features and a plurality of relationships between features of the plurality of features, the knowledge graph comprising a plurality of nodes connected by a plurality of edges, each node of the plurality of nodes being associated with a feature of the plurality of features, and each edge of the plurality of edges being associated with a relationship of the plurality of relationships between two features of the plurality of features; determining, with at least one processor, a plurality of feature values associated with the plurality of features; determining, with at least one processor, a first feature associated with a first node of the plurality of nodes; perturbating, with at least one processor, a feature value of the plurality of feature values associated with the first feature, wherein perturbating the feature value associated with the first feature comprises: updating the feature value associated with the first feature to a new feature value associated with the firstfeature; determining, with at least one processor, a second feature associated with a second node of the plurality of nodes, based on an edge connecting the first node to the second node; perturbating, with at least one processor, a feature value of the plurality of feature values associated with the second feature based on the new feature value associated with the first feature, wherein perturbating the feature value associated with the second feature comprises: updating the feature value associated with the second feature to a new feature value associated with the second feature, based on the new feature value associated with the first feature; inputting, with at least one processor, the plurality of feature values to a decision-based machine learning model; and determining, with at least one processor, a significance of an effect of the first feature and the second feature on an output of the decision-based machine learning model based on the new feature value associated with the first feature and the new feature value associated with the second feature.

[0035] Clause 2: The method of clause 1 , further comprising: displaying, with at least one processor, the plurality of features in a user interface of a computing device of a user; receiving, with at least one processor, user input comprising an identification of at least one pairing of features of the plurality of features; and determining, with at least one processor, the plurality of relationships between the features of the plurality of features based at least partly on the at least one pairing of features of the user input.

[0036] Clause 3: The method of clause 1 or clause 2, further comprising: storing, with at least one processor, a node table associated with the plurality of nodes of the knowledge graph, wherein the node table identifies, for each node of the plurality of nodes, at least one perturbation parameter of the feature associated with the node; and wherein perturbating the feature value of the plurality of feature values associated with the first feature further comprises: updating the feature value associated with the first feature to a new feature value associated with the first feature based on the node table, wherein the new feature value associated with the first feature satisfies the at least one perturbation parameter of the first feature associated with the first node.

[0037] Clause 4: The method of any of clauses 1 -3, further comprising: storing, with at least one processor, an edge table associated with the plurality of edges of the knowledge graph, wherein the edge table identifies, for each edge of the plurality of edges, a source node of the plurality of nodes connected by the edge, a target node of the plurality of nodes connected by the edge, and a constraint on a feature value of a feature associated with the target node based on a feature value of a featureassociated with the source node; and wherein perturbating the feature value of the plurality of feature values associated with the second feature further comprises: updating the feature value associated with the second feature to a new feature value associated with the second feature based on the node table and the edge table, wherein the new feature value associated with the second feature satisfies the at least one perturbation parameter of the second feature associated with the second node, and satisfies the constraint on the feature value of the second feature based on the feature value of the first feature.

[0038] Clause 5: The method of any of clauses 1 -4, wherein determining the significance of the effect of the first feature and the second feature on the output of the decision-based machine learning model comprises: generating a first output of the decision-based machine learning model based on the plurality of feature values associated with the plurality of features before perturbating feature values of the plurality of feature values associated with the first feature and the second feature; generating a second output of the decision-based machine learning model based on the plurality of feature values associated with the plurality of features after perturbating feature values of the plurality of feature values associated with the first feature and the second feature; and comparing the first output to the second output.

[0039] Clause 6: The method of any of clauses 1 -5, further comprising: reducing, with at least one processor, the plurality of features input to the decision-based machine learning model based on the significance of the effect of the first feature and the second feature on the output of the decision-based machine learning model, to produce a reduced plurality of features; and executing, with at least one processor, the decision-based machine learning model based on the reduced plurality of features.

[0040] Clause 7: The method of any of clauses 1 -6, wherein reducing the plurality of features input to the decision-based machine learning model based on the significance of the effect of the first feature and the second feature comprises: determining that the second output is not significantly different than the first output; and removing the first feature and the second feature from the plurality of features.

[0041] Clause 8: The method of any of clauses 1 -6, wherein reducing the plurality of features input to the decision-based machine learning model based on the significance of the effect of the first feature and the second feature comprises: determining that the second output is significantly different than the first output; andremoving at least one feature from the plurality of features, wherein the at least one feature is not the first feature nor the second feature.

[0042] Clause 9: The method of any of clauses 1 -8, further comprising: determining, with at least one processor, a third feature associated with a third node of the plurality of nodes, based on an edge connecting the second node to the third node; perturbating, with at least one processor, a feature value of the plurality of feature values associated with the third feature based on the new feature value associated with the second feature, wherein perturbating the feature value associated with the third feature comprises: updating the feature value associated with the third feature to a new feature value associated with the third feature, based on the new feature value associated with the second feature; and determining, with at least one processor, a significance of an effect of the first feature, the second feature, and the third feature on the output of the decision-based machine learning model based on the new feature value associated with the first feature, the new feature value associated with the second feature, and the new feature value associated with the third feature.

[0043] Clause 10: A system comprising at least one processor programmed or configured to: generate a knowledge graph based on a plurality of features and a plurality of relationships between features of the plurality of features, the knowledge graph comprising a plurality of nodes connected by a plurality of edges, each node of the plurality of nodes being associated with a feature of the plurality of features, and each edge of the plurality of edges being associated with a relationship of the plurality of relationships between two features of the plurality of features; determine a plurality of feature values associated with the plurality of features; determine a first feature associated with a first node of the plurality of nodes; perturbate a feature value of the plurality of feature values associated with the first feature, wherein, when perturbating the feature value associated with the first feature, the at least one processor is programmed or configured to: update the feature value associated with the first feature to a new feature value associated with the first feature; determine a second feature associated with a second node of the plurality of nodes, based on an edge connecting the first node to the second node; perturbate a feature value of the plurality of feature values associated with the second feature based on the new feature value associated with the first feature, wherein, when perturbating the feature value associated with the second feature, the at least one processor is programmed or configured to: update the feature value associated with the second feature to a new feature value associatedwith the second feature, based on the new feature value associated with the first feature; input the plurality of feature values to a decision-based machine learning model; and determine a significance of an effect of the first feature and the second feature on an output of the decision-based machine learning model based on the new feature value associated with the first feature and the new feature value associated with the second feature.

[0044] Clause 1 1 : The system of clause 10, wherein the at least one processor is further programmed or configured to: display the plurality of features in a user interface of a computing device of a user; receive user input comprising an identification of at least one pairing of features of the plurality of features; and determine the plurality of relationships between the features of the plurality of features based at least partly on the at least one pairing of features of the user input.

[0045] Clause 12: The system of clause 10 or clause 1 1 , wherein the at least one processor is further programmed or configured to: store a node table associated with the plurality of nodes of the knowledge graph, wherein the node table identifies, for each node of the plurality of nodes, at least one perturbation parameter of the feature associated with the node; and wherein, when perturbating the feature value of the plurality of feature values associated with the first feature, the at least one processor is further programmed or configured to: update the feature value associated with the first feature to a new feature value associated with the first feature based on the node table, wherein the new feature value associated with the first feature satisfies the at least one perturbation parameter of the first feature associated with the first node.

[0046] Clause 13: The system of any of clauses 10-12, wherein the at least one processor is further programmed or configured to: store an edge table associated with the plurality of edges of the knowledge graph, wherein the edge table identifies, for each edge of the plurality of edges, a source node of the plurality of nodes connected by the edge, a target node of the plurality of nodes connected by the edge, and a constraint on a feature value of a feature associated with the target node based on a feature value of a feature associated with the source node; and wherein, when perturbating the feature value of the plurality of feature values associated with the second feature, the at least one processor is programmed or configured to: update the feature value associated with the second feature to a new feature value associated with the second feature based on the node table and the edge table, wherein the new feature value associated with the second feature satisfies the at least one perturbationparameter of the second feature associated with the second node, and satisfies the constraint on the feature value of the second feature based on the feature value of the first feature.

[0047] Clause 14: The system of any of clauses 10-13, wherein, when determining the significance of the effect of the first feature and the second feature on the output of the decision-based machine learning, the at least one processor is programmed or configured to: generate a first output of the decision-based machine learning model based on the plurality of feature values associated with the plurality of features before perturbating feature values of the plurality of feature values associated with the first feature and the second feature; generate a second output of the decision-based machine learning model based on the plurality of feature values associated with the plurality of features after perturbating feature values of the plurality of feature values associated with the first feature and the second feature; and compare the first output to the second output.

[0048] Clause 15: The system of any of clauses 10-14, wherein the at least one processor is further programmed or configured to: reduce the plurality of features input to the decision-based machine learning model based on the significance of the effect of the first feature and the second feature on the output of the decision-based machine learning model, to produce a reduced plurality of features; and execute the decisionbased machine learning model based on the reduced plurality of features.

[0049] Clause 16: The system of any of clauses 10-15, wherein, when reducing the plurality of features input to the decision-based machine learning model based on the significance of the effect of the first feature and the second feature, the at least one processor is programmed or configured to: determine that the second output is not significantly different than the first output; and remove the first feature and the second feature from the plurality of features.

[0050] Clause 17: The system of any of clauses 10-15, wherein, when reducing the plurality of features input to the decision-based machine learning model based on the significance of the effect of the first feature and the second feature, the at least one processor is programmed or configured to: determine that the second output is significantly different than the first output; and remove at least one feature from the plurality of features, wherein the at least one feature is not the first feature nor the second feature.

[0051] Clause 18: The system of any of clauses 10-17, wherein the at least one processor is further programmed or configured to: determine a third feature associated with a third node of the plurality of nodes, based on an edge connecting the second node to the third node; perturbate a feature value of the plurality of feature values associated with the third feature based on the new feature value associated with the second feature, wherein, when perturbating the feature value associated with the third feature, the at least one processor is programmed or configured to: update the feature value associated with the third feature to a new feature value associated with the third feature, based on the new feature value associated with the second feature; and determine a significance of an effect of the first feature, the second feature, and the third feature on the output of the decision-based machine learning model based on the new feature value associated with the first feature, the new feature value associated with the second feature, and the new feature value associated with the third feature.

[0052] Clause 19: A computer program product comprising at least one non- transitory computer-readable medium comprising one or more instructions that, when executed by at least one processor, cause the at least one processor to: generate a knowledge graph based on a plurality of features and a plurality of relationships between features of the plurality of features, the knowledge graph comprising a plurality of nodes connected by a plurality of edges, each node of the plurality of nodes being associated with a feature of the plurality of features, and each edge of the plurality of edges being associated with a relationship of the plurality of relationships between two features of the plurality of features; determine a plurality of feature values associated with the plurality of features; determine a first feature associated with a first node of the plurality of nodes; perturbate a feature value of the plurality of feature values associated with the first feature, wherein the one or more instructions that cause the at least one processor to perturbate the feature value associated with the first feature cause the at least one processor to: update the feature value associated with the first feature to a new feature value associated with the first feature; determine a second feature associated with a second node of the plurality of nodes, based on an edge connecting the first node to the second node; perturbate a feature value of the plurality of feature values associated with the second feature based on the new feature value associated with the first feature, wherein the one or more instructions that cause the at least one processor to perturbate the feature value associated with the second feature cause the at least one processor to: update the feature value associated withthe second feature to a new feature value associated with the second feature, based on the new feature value associated with the first feature; input the plurality of feature values to a decision-based machine learning model; and determine a significance of an effect of the first feature and the second feature on an output of the decision-based machine learning model based on the new feature value associated with the first feature and the new feature value associated with the second feature.

[0053] Clause 20: The computer program product of clause 19, wherein the one or more instructions cause the at least one processor to: display the plurality of features in a user interface of a computing device of a user; receive user input comprising an identification of at least one pairing of features of the plurality of features; and determine the plurality of relationships between the features of the plurality of features based at least partly on the at least one pairing of features of the user input.

[0054] Clause 21 : The computer program product of clause 19 or clause 20, wherein the one or more instructions further cause the at least one processor to: store a node table associated with the plurality of nodes of the knowledge graph, wherein the node table identifies, for each node of the plurality of nodes, at least one perturbation parameter of the feature associated with the node; and wherein the one or more instructions that cause the at least one processor to perturbate the feature value of the plurality of feature values associated with the first feature cause the at least one processor to: update the feature value associated with the first feature to a new feature value associated with the first feature based on the node table, wherein the new feature value associated with the first feature satisfies the at least one perturbation parameter of the first feature associated with the first node.

[0055] Clause 22: The computer program product of any of clauses 19-21 , wherein the one or more instructions further cause the at least one processor to: store an edge table associated with the plurality of edges of the knowledge graph, wherein the edge table identifies, for each edge of the plurality of edges, a source node of the plurality of nodes connected by the edge, a target node of the plurality of nodes connected by the edge, and a constraint on a feature value of a feature associated with the target node based on a feature value of a feature associated with the source node; and wherein the one or more instructions that cause the at least one processor to perturbate the feature value of the plurality of feature values associated with the second feature cause the at least one processor to: update the feature value associated with the second feature to a new feature value associated with the secondfeature based on the node table and the edge table, wherein the new feature value associated with the second feature satisfies the at least one perturbation parameter of the second feature associated with the second node, and satisfies the constraint on the feature value of the second feature based on the feature value of the first feature.

[0056] Clause 23: The computer program product of any of clauses 19-22, wherein the one or more instructions that cause the at least one processor to determine the significance of the effect of the first feature and the second feature on the output of the decision-based machine learning cause the at least one processor to: generate a first output of the decision-based machine learning model based on the plurality of feature values associated with the plurality of features before perturbating feature values of the plurality of feature values associated with the first feature and the second feature; generate a second output of the decision-based machine learning model based on the plurality of feature values associated with the plurality of features after perturbating feature values of the plurality of feature values associated with the first feature and the second feature; and compare the first output to the second output.

[0057] Clause 24: The computer program product of any of clauses 19-23, wherein the one or more instructions further cause the at least one processor to: reduce the plurality of features input to the decision-based machine learning model based on the significance of the effect of the first feature and the second feature on the output of the decision-based machine learning model, to produce a reduced plurality of features; and execute the decision-based machine learning model based on the reduced plurality of features.

[0058] Clause 25: The computer program product of any of clauses 19-24, wherein the one or more instructions that cause the at least one processor to reduce the plurality of features input to the decision-based machine learning model based on the significance of the effect of the first feature and the second feature cause the at least one processor to: determine that the second output is not significantly different than the first output; and remove the first feature and the second feature from the plurality of features.

[0059] Clause 26: The computer program product of any of clauses 19-24, wherein the one or more instructions that cause the at least one processor to reduce the plurality of features input to the decision-based machine learning model based on the significance of the effect of the first feature and the second feature cause the at least one processor to: determine that the second output is significantly different than thefirst output; and remove at least one feature from the plurality of features, wherein the at least one feature is not the first feature nor the second feature.

[0060] Clause 27: The computer program product of any of clauses 19-26, wherein the one or more instructions further cause the at least one processor to: determine a third feature associated with a third node of the plurality of nodes, based on an edge connecting the second node to the third node; perturbate a feature value of the plurality of feature values associated with the third feature based on the new feature value associated with the second feature, wherein the one or more instructions that cause the at least one processor to perturbate the feature value associated with the third feature cause the at least one processor to: update the feature value associated with the third feature to a new feature value associated with the third feature, based on the new feature value associated with the second feature; and determine a significance of an effect of the first feature, the second feature, and the third feature on the output of the decision-based machine learning model based on the new feature value associated with the first feature, the new feature value associated with the second feature, and the new feature value associated with the third feature.

[0061] These and other features and characteristics of the present disclosure, as well as the methods of operation and functions of the related elements of structures and the combination of parts and economies of manufacture, will become more apparent upon consideration of the following description and the appended claims with reference to the accompanying drawings, all of which form a part of this specification, wherein like reference numerals designate corresponding parts in the various figures. It is to be expressly understood, however, that the drawings are for the purpose of illustration and description only and are not intended as a definition of the limits of the present disclosure. As used in the specification and the claims, the singular form of “a,” “an,” and “the” include plural referents unless the context clearly dictates otherwise.BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Additional advantages and details of the disclosure are explained in greater detail below with reference to the exemplary embodiments or aspects that are illustrated in the accompanying schematic figures, in which:

[0063] FIG. 1 is a diagram of a non-limiting embodiment or aspect of an environment in which systems, devices, products, apparatus, and / or methods,described herein, may be implemented, according to the principles of the present disclosure;

[0064] FIG. 2 is a diagram of one or more components, devices, and / or systems, according to some non-limiting embodiments or aspects;

[0065] FIG. 3 is a flowchart of a method for perturbation-based interpretation of the effects of related features in machine learning models, according to some non-limiting embodiments or aspects;

[0066] FIG. 4 is a flow diagram of a method for perturbation-based interpretation of the effects of related features in machine learning models, according to some nonlimiting embodiments or aspects;

[0067] FIG. 5 is an illustration of a knowledge graph, for use in a method for perturbation-based interpretation of the effects of related features in machine learning models, according to some non-limiting embodiments or aspects;

[0068] FIG. 6 is an illustration of a node table, for use in a method for perturbationbased interpretation of the effects of related features in machine learning models, according to some non-limiting embodiments or aspects; and

[0069] FIG. 7 is an illustration of an edge table, for use in a method for perturbationbased interpretation of the effects of related features in machine learning models, according to some non-limiting embodiments or aspects.DETAILED DESCRIPTION

[0070] For purposes of the description hereinafter, the terms “upper”, “lower”, “right”, “left”, “vertical”, “horizontal”, “top”, “bottom”, “lateral”, “longitudinal,” and derivatives thereof shall relate to non-limiting embodiments or aspects as they are oriented in the drawing figures. However, it is to be understood that non-limiting embodiments or aspects may assume various alternative variations and step sequences, except where expressly specified to the contrary. It is also to be understood that the specific devices and processes illustrated in the attached drawings, and described in the following specification, are simply exemplary embodiments or aspects. Hence, specific dimensions and other physical characteristics related to the embodiments or aspects disclosed herein are not to be considered as limiting.

[0071] No aspect, component, element, structure, act, step, function, instruction, and / or the like used herein should be construed as critical or essential unless explicitlydescribed as such. Also, as used herein, the articles “a” and “an” are intended to include one or more items and may be used interchangeably with “one or more” and “at least one.” Furthermore, as used herein, the term “set” is intended to include one or more items (e.g., related items, unrelated items, a combination of related and unrelated items, etc.) and may be used interchangeably with “one or more” or “at least one.” Where only one item is intended, the term “one” or similar language is used. Also, as used herein, the terms “has,” “have,” “having,” or the like are intended to be open-ended terms. Further, the phrase “based on” is intended to mean “based at least partially on” unless explicitly stated otherwise. The phase “based on” may also mean “in response to” where appropriate.

[0072] Some non-limiting embodiments or aspects are described herein in connection with thresholds. As used herein, satisfying a threshold may refer to a value being greater than the threshold, more than the threshold, higher than the threshold, greater than or equal to the threshold, less than the threshold, fewer than the threshold, lower than the threshold, less than or equal to the threshold, equal to the threshold, and / or the like.

[0073] As used herein, the term “acquirer institution” may refer to an entity licensed and / or approved by a transaction service provider to originate transactions (e.g., payment transactions) using a payment device associated with the transaction service provider. The transactions the acquirer institution may originate may include payment transactions (e.g., purchases, original credit transactions (OCTs), account funding transactions (AFTs), and / or the like). In some non-limiting embodiments or aspects, an acquirer institution may be a financial institution, such as a bank. As used herein, the term “acquirer system” may refer to one or more computing devices operated by or on behalf of an acquirer institution, such as a server computer executing one or more software applications.

[0074] As used herein, the term “account identifier” may include one or more primary account numbers (PANs), tokens, or other identifiers associated with a customer account. The term “token” may refer to an identifier that is used as a substitute or replacement identifier for an original account identifier, such as a PAN. Account identifiers may be alphanumeric or any combination of characters and / or symbols. Tokens may be associated with a PAN or other original account identifier in one or more data structures (e.g., one or more databases, and / or the like) such that they may be used to conduct a transaction without directly using the original accountidentifier. In some examples, an original account identifier, such as a PAN, may be associated with a plurality of tokens for different individuals or purposes.

[0075] As used herein, the term “communication” may refer to the reception, receipt, transmission, transfer, provision, and / or the like of data (e.g., information, signals, messages, instructions, commands, and / or the like). For one unit (e.g., a device, a system, a component of a device or system, combinations thereof, and / or the like) to be in communication with another unit means that the one unit is able to directly or indirectly receive information from and / or transmit information to the other unit. This may refer to a direct or indirect connection (e.g., a direct communication connection, an indirect communication connection, and / or the like) that is wired and / or wireless in nature. Additionally, two units may be in communication with each other even though the information transmitted may be modified, processed, relayed, and / or routed between the first and second unit. For example, a first unit may be in communication with a second unit even though the first unit passively receives information and does not actively transmit information to the second unit. As another example, a first unit may be in communication with a second unit if at least one intermediary unit processes information received from the first unit and communicates the processed information to the second unit.

[0076] As used herein, the term “computing device” may refer to one or more electronic devices configured to process data. A computing device may, in some examples, include the necessary components to receive, process, and output data, such as a processor, a display, a memory, an input device, a network interface, and / or the like. A computing device may be a mobile device. As an example, a mobile device may include a cellular phone (e.g., a smartphone or standard cellular phone), a portable computer, a wearable device (e.g., watches, glasses, lenses, clothing, and / or the like), a personal digital assistant (PDA), and / or other like devices. A computing device may also be a desktop computer or other form of non-mobile computer.

[0077] As used herein, the terms “electronic wallet” and “electronic wallet application” refer to one or more electronic devices and / or software applications configured to initiate and / or conduct payment transactions. For example, an electronic wallet may include a mobile device executing an electronic wallet application, and may further include server-side software and / or databases for maintaining and providing transaction data to the mobile device. An “electronic wallet provider” may include an entity that provides and / or maintains an electronic wallet for a customer, such asGoogle Pay®, Android Pay®, Apple Pay®, Samsung Pay®, and / or other like electronic payment systems. In some non-limiting examples, an issuer bank may be an electronic wallet provider.

[0078] As used herein, the term “issuer institution” may refer to one or more entities, such as a bank, that provide accounts to customers for conducting transactions (e.g., payment transactions), such as initiating credit and / or debit payments. For example, an issuer institution may provide an account identifier, such as a PAN, to a customer that uniquely identifies one or more accounts associated with that customer. The account identifier may be embodied on a portable financial device, such as a physical financial instrument, e.g., a payment card, and / or may be electronic and used for electronic payments. The term “issuer system” refers to one or more computer devices operated by or on behalf of an issuer institution, such as a server computer executing one or more software applications. For example, an issuer system may include one or more authorization servers for authorizing a transaction.

[0079] As used herein, the term “merchant” may refer to an individual or entity that provides goods and / or services, or access to goods and / or services, to customers based on a transaction, such as a payment transaction. The term “merchant” or “merchant system” may also refer to one or more computer systems operated by or on behalf of a merchant, such as a server computer executing one or more software applications.

[0080] As used herein, a “point-of-sale (POS) device” may refer to one or more devices, which may be used by a merchant to conduct a transaction (e.g., a payment transaction) and / or process a transaction. For example, a POS device may include one or more client devices. Additionally or alternatively, a POS device may include peripheral devices, card readers, scanning devices (e.g., code scanners), Bluetooth® communication receivers, near-field communication (NFC) receivers, radio frequency identification (RFID) receivers, and / or other contactless transceivers or receivers, contact-based receivers, payment terminals, and / or the like. As used herein, a “point- of-sale (POS) system” may refer to one or more client devices and / or peripheral devices used by a merchant to conduct a transaction. For example, a POS system may include one or more POS devices and / or other like devices that may be used to conduct a payment transaction. In some non-limiting embodiments or aspects, a POS system (e.g., a merchant POS system) may include one or more server computersprogrammed or configured to process online payment transactions through webpages, mobile applications, and / or the like.

[0081] As used herein, the terms “client” and “client device” may refer to one or more client-side devices or systems (e.g., remote from a transaction service provider) used to initiate or facilitate a transaction (e.g., a payment transaction). As an example, a “client device” may refer to one or more POS devices used by a merchant, one or more acquirer host computers used by an acquirer, one or more mobile devices used by a user, one or more computing devices used by a payment device provider system, and / or the like. In some non-limiting embodiments or aspects, a client device may be an electronic device configured to communicate with one or more networks and initiate or facilitate transactions. For example, a client device may include one or more computers, portable computers, laptop computers, tablet computers, mobile devices, cellular phones, wearable devices (e.g., watches, glasses, lenses, clothing, and / or the like), PDAs, and / or the like. Moreover, a “client” may also refer to an entity (e.g., a merchant, an acquirer, and / or the like) that owns, utilizes, and / or operates a client device for initiating transactions (e.g., for initiating transactions with a transaction service provider).

[0082] As used herein, the term “payment device” may refer to a payment card (e.g., a credit or debit card), a gift card, a smartcard, smart media, a payroll card, a healthcare card, a wristband, a machine-readable medium containing account information, a keychain device or fob, an RFID transponder, a retailer discount or loyalty card, a cellular phone, an electronic wallet mobile application, a PDA, a pager, a security card, a computing device, an access card, a wireless terminal, a transponder, and / or the like. In some non-limiting embodiments or aspects, the payment device may include volatile or non-volatile memory to store information (e.g., an account identifier, a name of the account holder, and / or the like).

[0083] As used herein, the term “server” may refer to or include one or more computing devices that are operated by or facilitate communication and processing for multiple parties in a network environment, such as the internet, although it will be appreciated that communication may be facilitated over one or more public or private network environments and that various other arrangements are possible. Further, multiple computing devices (e.g., servers, POS devices, mobile devices, etc.) directly or indirectly communicating in the network environment may constitute a “system.” Reference to “a server” or “a processor,” as used herein, may refer to a previouslyrecited server and / or processor that is recited as performing a previous step or function, a different server and / or processor, and / or a combination of servers and / or processors. For example, as used in the specification and the claims, a first server and / or a first processor that is recited as performing a first step or function may refer to the same or different server and / or a processor recited as performing a second step or function.

[0084] As used herein, the term “transaction service provider” may refer to an entity that receives transaction authorization requests from merchants or other entities and provides guarantees of payment, in some cases through an agreement between the transaction service provider and an issuer institution. For example, a transaction service provider may include a payment network such as Visa® or any other entity that processes transactions. The term “transaction processing system” may refer to one or more computer systems operated by or on behalf of a transaction service provider, such as a transaction processing server executing one or more software applications. A transaction processing server may include one or more processors and, in some non-limiting embodiments or aspects, may be operated by or on behalf of a transaction service provider.

[0085] The systems, methods, and computer program products described herein provide numerous technical advantages in systems for machine learning model interpretation and execution. First, by generating a knowledge graph based on a plurality of features and a plurality of relationships between those features, the effects that features have on each other and model output can be captured more accurately in perturbation-based analysis. The interpretability of feature effects is greatly improved given that correlations and dependencies between features are captured in the perturbation-based analysis, which yields cascading improvements to model accuracy and runtime computational cost. Furthermore, the described systems and methods provide processes for relational perturbation, which is more accurate than single-feature perturbation. For example, interpretability is greatly improved by the described steps for determining a first feature associated with a first node, perturbating the value of the first feature, determining a second feature that is related to the first feature based on an edge connecting a first and second node in the knowledge graph, and then perturbating the value of the second feature based, at least partly, on the new value of the first feature. Such techniques can be repeated for a number of related features (e.g., a path of connected nodes), so as to sequentially perturbate featurevalues of related nodes and update feature values based on dependencies to other features. The above-described techniques greatly improve the accuracy of perturbation-based feature analysis for machine learning model output interpretation and may be applicable to many perturbation analysis frameworks, contributing to enhanced deployability due to model agnosticism.

[0086] Improving the accuracy and interpretability of perturbation-based techniques is also directly linked to the performance of the technical environment of modeling systems. In particular, when executing a machine learning model, the number of features input to the machine learning model is related to the computer resources required to execute the machine learning model (e.g., bandwidth, processing capacity, memory, storage capacity, etc.). The more features that are retained for model execution, the more computer resources are required to execute the model. The described systems and methods produce a leaner execution of a machine learning model in a modeling system, by allowing insignificant features to be excluded from model execution. For example, the described systems and methods of related-feature perturbation analysis may be used to identify features that do or do not have a significant effect (e.g., a statistically significant influence) on the output of a model. Features having a significant effect may be retained for model execution, while features not having a significant effect may be excluded from model execution. In this manner, the computer resources required to execute an acceptably reliable and accurate model are directly reduced through the reduction of the feature set. The above-described technical advantages, among others, are made apparent in view of the following figures and accompanying description.

[0087] Referring now to FIG. 1 , FIG. 1 is a diagram of an example environment 100 in which devices, systems, and / or methods, described herein, may be implemented. As shown in FIG. 1 , environment 100 may include modeling system 102, memory 104, computing device 106, and communication network 108. Modeling system 102, memory 104, and computing device 106 may interconnect (e.g., establish a connection to communicate) via wired connections, wireless connections, or a combination of wired and wireless connections. In some non-limiting embodiments or aspects, environment 100 may further include a natural language processing system, an advertising system, a fraud detection system, a transaction processing system, a network security monitoring system, a merchant system, an acquirer system, an issuer system, and / or a payment device.

[0088] Modeling system 102 may include one or more computing devices configured to communicate with memory 104 and / or computing device 106 at least partly over communication network 108. Modeling system 102 may be configured to generate a knowledge graph having nodes associated with features (e.g., model input features) and edges associated with relationships (e.g., correlations, dependencies, etc.) between features. Modeling system 102 may be further configured to use the relationships between features (e.g., associated with edges between nodes) to perform perturbation analysis on the effect of features on model output. Modeling system 102 may include or be in communication with memory 104. Modeling system 102 may be associated with, or included in a same system as, a natural language processing system, a fraud detection system, an advertising system, a network security monitoring system, and / or a transaction processing system.

[0089] Memory 104 may include one or more computing devices configured to communicate with modeling system 102 and / or computing device 106 at least partly over communication network 108. Memory 104 may be configured to store data associated with decision-based machine learning models (e.g., states, features, feature values, layers, gradients, training data, etc.) in one or more non-transitory computer readable storage media. Memory 104 may communicate with and / or be included in modeling system 102.

[0090] Computing device 106 may include one or more processors that are configured to communicate with modeling system 102 and / or memory 104 at least partly over communication network 108. Computing device 106 may be associated with a user and may include at least one user interface for transmitting data to and receiving data from modeling system 102 and / or memory 104. For example, computing device 106 may show, on a display of computing device 106, one or more outputs of decision-based machine learning models executed by modeling system 102. By way of further example, one or more inputs for decision-based machine learning models (e.g., features, feature values, relationships between features, perturbation parameters, constraints, etc.) may be determined or received by modeling system 102 via a user interface of computing device 106. In some non-limiting embodiments or aspects, computing device 106 may be a mobile device.

[0091] Communication network 108 may include one or more wired and / or wireless networks over which the systems and devices of environment 100 may communicate. For example, communication network 108 may include a cellular network (e.g., a long-term evolution (LTE®) network, a third generation (3G) network, a fourth generation (4G) network, a fifth generation (5G) network, a code division multiple access (CDMA) network, etc.), a public land mobile network (PLMN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a telephone network (e.g., the public switched telephone network (PSTN)), a private network, an ad hoc network, an intranet, the Internet, a fiber optic-based network, a cloud computing network, and / or the like, and / or a combination of these or other types of networks.

[0092] The number and arrangement of devices and networks shown in FIG. 1 are provided as an example. There may be additional devices and / or networks, fewer devices and / or networks, different devices and / or networks, or differently arranged devices and / or networks than those shown in FIG. 1. Furthermore, two or more devices shown in FIG. 1 may be implemented within a single device, or a single device shown in FIG. 1 may be implemented as multiple, distributed devices. Additionally or alternatively, a set of devices (e.g., one or more devices) of environment 100 may perform one or more functions described as being performed by another set of devices of environment 100.

[0093] In some non-limiting embodiments or aspects, at least one processor of modeling system 102 may be programmed or configured to generate a knowledge graph based on a plurality of features (e.g., individual measurable properties used for model input) and a plurality of relationships (e.g., correlations, dependencies, etc.) between features of the plurality of features. The knowledge graph may define the relations of entities (e.g., features) in a schema, which may include a plurality of nodes (e.g., vertices) connected by a plurality of edges (e.g., links). See, e.g., knowledge graph 500 shown in FIG. 5. Each node of the plurality of nodes may be associated with a feature of the plurality of features. For example, each node may be representative of a feature that may be used to train and / or execute a decision-based machine learning model (e.g., a machine learning model trained to output a value and / or categorization used for making a decision, such as an assessment of transaction fraud, an association with an advertising group, etc.). See, e.g., features associated with nodes represented in node table 600, shown in FIG. 6. Each edge of the plurality of edges may be associated with a relationship (e.g., correlation, dependency, etc.) of the plurality of relationships between two features of the plurality of features. For example, each edge may be representative of a dependency of a value of a second feature associated with a second node on a value of a first featureassociated with a first node, where the second node and first node are connected by the edge. See, e.g., relationships associated with edges represented in edge table 700, shown in FIG. 7.

[0094] In some non-limiting embodiments or aspects, at least one processor of modeling system 102 may be further programmed or configured to determine a plurality of feature values associated with the plurality of features. For example, modeling system 102 may determine an initial feature value for each feature of the plurality of features (e.g., in an initialization process). To determine an initial feature value for each feature, modeling system 102 may reference a node table (e.g., a feature data table stored in memory 104) associated with the plurality of nodes of the knowledge graph. The node table may identify, for each node of the plurality of nodes, at least one perturbation parameter (e.g., one or more rules, limits, definitions, ranges, etc., describing acceptable values for the feature). The at least one perturbation parameter for the values of a feature may include, but is not limited to, an upper bound, a lower bound, a set of acceptable values (e.g., a binary set of 0 or 1 , a list of values, etc.), a data type (e.g., numerical, such as an integer, decimal, date, time, and the like; categorical, such as a merchant category code, country code, transaction type, and the like; etc.), or any combination thereof. It will be appreciated that the node table may also be referred to as a feature data table, because each node of the knowledge graph is associated with a feature. To initialize the feature values of the plurality of features, modeling system 102 may generate a random feature value for each feature based on the at least one perturbation parameter associated with each feature. Additionally or alternatively, user input from computing device 106 may be used to set initial feature values for the plurality of features.

[0095] In some non-limiting embodiments or aspects, at least one processor of modeling system 102 may determine a first feature associated with a first node of the plurality of nodes. For example, modeling system 102 may determine the first feature based on user input from computing device 106, wherein a user identifies a first feature and / or a first node for interpretation. Additionally or alternatively, modeling system 102 may randomly identify the first feature from the plurality of features or the plurality of nodes, or identify the first feature from a list of the plurality of features or the plurality of nodes. Additionally or alternatively, modeling system 102 may determine the first feature based on global statistical analysis showing high variability in the first feature.Determination of the first feature is useful to begin identifying and perturbating feature values of related features in sequence.

[0096] In some non-limiting embodiments or aspects, at least one processor of modeling system 102 may perturbate a feature value (e.g., of the plurality of feature values) that is associated with the first feature. Modeling system 102 may perturbate the feature value associated with the first feature by updating the feature value associated with the first feature to a new feature value associated with the first feature. In some non-limiting embodiments or aspects, modeling system 102 may reference a node table and update the feature to have a feature value that satisfies the at least one perturbation parameter (e.g., comports with the one or more rules, limits, definitions, ranges, etc.) of the first feature. For example, if the first feature has perturbation parameters that specify the first feature may be a numerical value in the range of 0 to 10,000, and the first feature was initialized with a starting value of 5,000, then the first feature may be perturbated by having its feature value updated to a new value that satisfies the range and data type (e.g., 6,000). The magnitude (e.g., size, intensity, etc.) of the perturbation may also be limited by a predetermined hyperparameter (e.g., 5% perturbation, 10% perturbation, 20% perturbation, etc.), to prevent over-tuning (e.g., a new value of 5,400 would satisfy a perturbation hyperparameter of <10% change).

[0097] In some non-limiting embodiments or aspects, at least one processor of modeling system 102 may determine a second feature associated with a second node of the plurality of nodes, based on an edge connecting the first node to the second node. Because the knowledge graph embeds the relationships of features in a node- and-edge schema, a related feature to the first perturbated feature may be identified by determining a second node that is connected to the first node via an edge (e.g., representative of a relationship between the first feature and the second feature). Modeling system 102 may then perturbate a feature value (e.g., of the plurality of feature values) associated with the second feature based on the new feature value associated with the first feature. For example, modeling system 102 may update the feature value associated with the second feature to a new feature value associated with the second feature, based on the new value associated with the first feature and the relationship between the first feature and the second feature. For example, if the second feature’s value is dependent on the first feature’s value, then whatever newvalue is determined for the second feature may be dependent on the new value that is determined for the first feature.

[0098] In some non-limiting embodiments or aspects, modeling system 102 may perturbate the value of the second feature based at least partly by referencing an edge table (e.g., a relationship data table stored in memory 104). The edge table may be associated with the plurality of edges of the knowledge graph and may identify, for each edge of the plurality of edges, a source node (e.g., an independent node, a controlling node, a first node, etc.) connected by the edge, a target node (e.g., a dependent node, a controlled node, a second node, etc.) connected by the edge, and a constraint (e.g., a conditional parameter) on a feature value of a feature associated with the target node based on a feature value of a feature associated with the source node. In this manner, the edge table may define how the feature values for target nodes may change based on the feature values for source nodes that are connected to the target nodes, thereby capturing the relationship (e.g., correlation, dependency, etc.) of associated features.

[0099] Information about relationships between features, which may include the data of the edge table, may be determined using global statistical analysis, machine learning analysis, input by skilled personnel, and / or the like. In some non-limiting embodiments or aspects, at least one processor of modeling system 102 may be programmed or configured to display (e.g., cause to be displayed, by transmitting display data) the plurality of features in a user interface of computing device 106 of a user. A user may, using computing device 106, identify one or more related features, including data about those relationships. For example, the identification provided by a user may include, but is not limited to, at least one pairing of features of the plurality of features that are related, data defining a constraint on one feature of the pairing based on the other feature of the pairing, a feature being a source node (e.g., independent variable), a feature being a target node (e.g., dependent variable), and / or the like. Modeling system 102 may receive the user input including the identification of at least one pairing of features of the plurality of features and may determine the plurality of relationships between the features at least partly based on the at least one pairing of features. Additionally or alternatively, modeling system 102 may generate an edge table based, at least in part, on the user input.

[0100] When perturbating the feature value (e.g., of the plurality of feature values) associated with the second feature, at least one processor of modeling system 102may be programmed or configured to update the feature value associated with the second feature to a new feature value associated with the second feature based on the node table and the edge table, wherein the new feature value associated with the second feature satisfies the at least one perturbation parameter of the second feature associated with the second node, and satisfies the constraint on the feature value of the second feature based on the feature value of the first feature. For example, the first feature may have at least one perturbation parameter (e.g., in the node table) that specifies that the first feature must be a binary value of 0 or 1 . The first feature may be perturbated from an initial value of 0 to a new value of 1 . The second feature may have at least one perturbation parameter (e.g., in the node table) that specifies that the second feature must also be a binary value of 0 or 1 . Furthermore, the second feature may have a constraint (e.g., in the edge table) that specifies that the value of the second feature is conditional to be consistent with the value of the first feature, such that if the new value of the first feature is 1 , so too must be the new value of the second feature. As such, the second feature may be perturbated to a new value of 1 . In this manner, perturbation of the second feature abides by the at least one perturbation parameter of the second feature, as well as the constraint on the value of the second feature.

[0101] In some non-limiting embodiments or aspects, at least one processor of modeling system 102 may be programmed or configured to input the plurality of feature values to a decision-based machine learning model. For example, modeling system 102 may input the plurality of feature values, as initialized and before the perturbations of one or more feature values, to decision-based machine learning model to obtain a first model output. By way of further example, modeling system 102 may input the plurality of feature values, after perturbations of one or more feature values, to decision-based machine learning model to obtain a second (or higher order) model output. Modeling system 102 may obtain outputs of decision-based machine learning model by inputting the plurality of feature values to the model after each perturbation, after all perturbations, and / or the like.

[0102] In some non-limiting embodiments or aspects, at least one processor of modeling system 102 may be programmed or configured to determine a significance (e.g., magnitude, intensity, statistical significance, degree, etc.) of an effect of the first feature, the second feature, and / or additional features on an output of the decisionbased machine learning model based at least partly on the new feature valueassociated with the first feature and the new feature value associated with the second feature. For example, modeling system 102 may generate a first output of the decision-based machine learning model based on the plurality of feature values associated with the plurality of features before perturbating feature values of the plurality of feature values associated with the first feature and the second feature. Modeling system 102 may also generate a second output of the decision-based machine learning model based on the plurality of feature values associated with the plurality of features after perturbating feature values of the plurality of feature values associated with the first feature and the second feature. Modeling system 102 may further compare the first output to the second output to determine the significance of the effect of the first feature and the second feature on the output of modeling system 102 (e.g., using statistical analysis, a perturbation-based analysis framework, such as local interpretable model-agnostic explanations (LIME), Shapley additive explanations (SHAP), TimeSHAP, etc.).

[0103] By way of further example, suppose modeling system 102 is analyzing a decision-based machine learning model that outputs a binary categorization based on a numerical scale of 0 to 1. If the model output before perturbation is 0.857 (suggesting a categorization of 1 ) and the model output after perturbation is not statistically significantly different (e.g., 0.859), then modeling system 102 may determine that the effect of the first feature and / or the second feature on the output of the model is not significant. In contrast, if the model output before perturbation is 0.857 (suggesting a categorization of 1 ) and the model output after perturbation is statistically significantly different (e.g., 0.412, suggesting a categorization more likely being 0), then modeling system 102 may determine that the effect of the first feature and / or the second feature on the output of the model is significant. In some nonlimiting embodiments or aspects, if the effect of the first feature and the second feature on the output of the model is not significant, then modeling system 102 may loop through another iteration of perturbations after adjusting the hyperparameter to allow for larger perturbations (e.g., adjusting the hyperparameter from 5% to 10%), and reevaluate the output of the model to determine if the larger perturbation yielded a statistically significant difference in model output. Modeling system 102 may update its determination of the effects of first feature and / or second feature based on the outcome of the additional iteration(s) of perturbations.

[0104] In some non-limiting embodiments or aspects, at least one processor of modeling system 102 may be programmed or configured to reduce the plurality of features input to the decision-based machine learning model based on the significance of the effect of the first feature and the second feature on the output of the decisionbased machine learning model, to produce a reduced plurality of features. For example, modeling system 102 may determine that the second output is not significantly different (e.g., not statistically significantly different at a p-value of 0.05) than the first output and may remove the first feature and the second feature from the plurality of features. By way of another example, modeling system 102 may determine that the second output is significantly different than the first output and may remove at least one feature from the plurality of features that is not the first feature nor the second feature. Modeling system 102 may then execute the decision-based machine learning model based on the reduced plurality of features. In this manner, the plurality of features that are used to execute the decision-based machine learning model are reduced to a set of features that are determined to be more influential on the output of the machine learning model, and features that are not influential are excluded from being used, thereby reducing computer resource requirements for execution.

[0105] It will be appreciated that the above-described processes related to perturbating a first feature and a second feature may be extended to additional features that are connected by edge(s) to the first feature, the second feature, or other features with a connected path in the knowledge graph (e.g., a series of nodes connected by edges). For example, at least one processor of modeling system 102 may be programmed or configured to determine a third feature associated with a third node of the plurality of nodes, based on an edge connecting the second node to the third node. Modeling system 102 may then perturbate a feature value of the plurality of feature values associated with the third feature based on the new feature value associated with the second feature. When perturbating the feature value associated with the third feature, modeling system 102 may update the feature value associated with the third feature to a new feature value associated with the third feature, based on the new feature value associated with the second feature. For example, modeling system 102 may update the value of the third feature to a value that satisfies at least one perturbation parameter for the third feature and at least one constraint on the value of the third feature based on the value of the second feature. Modeling system 102 may then determine a significance of an effect of the first feature, the secondfeature, and / or the third feature on the output of the decision-based machine learning model based on the new feature value associated with the first feature, the new feature value associated with the second feature, and the new feature value associated with the third feature. It will be appreciated that the above process may be repeated for one or more additional features associated with the first feature, the second feature, or other features, based on nodes connected by edges in the knowledge graph.

[0106] Referring now to FIG. 2, FIG. 2 is a diagram of example components of a device 200, according to some non-limiting embodiments or aspects. Device 200 may correspond to one or more devices of modeling system 102, memory 104, computing device 106, and / or communication network 108 as shown in FIG. 1. In some nonlimiting embodiments or aspects, such systems or devices may include at least one device 200 and / or at least one component of device 200.

[0107] As shown in FIG. 2, device 200 may include bus 202, processor 204, memory 206, storage component 208, input component 210, output component 212, and communication interface 214. Bus 202 may include a component that permits communication among the components of device 200. In some non-limiting embodiments or aspects, processor 204 may be implemented in hardware, firmware, or a combination of hardware and software. For example, processor 204 may include a processor (e.g., a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), etc.), a microprocessor, a digital signal processor (DSP), and / or any processing component (e.g., a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), etc.) that can be programmed to perform a function. Memory 206 may include random access memory (RAM), read only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic memory, optical memory, etc.) that stores information and / or instructions for use by processor 204.

[0108] Storage component 208 may store information and / or software related to the operation and use of device 200. For example, storage component 208 may include a hard disk (e.g., a magnetic disk, an optical disk, a magneto-optic disk, a solid state disk, etc.) and / or another type of computer-readable medium.

[0109] Input component 210 may include a component that permits device 200 to receive information, such as via user input (e.g., a touch screen display, a keyboard, a keypad, a mouse, a button, a switch, a microphone, etc.). Additionally, or alternatively, input component 210 may include a sensor for sensing information (e.g.,a global positioning system (GPS) component, an accelerometer, a gyroscope, an actuator, etc.). Output component 212 may include a component that provides output information from device 200 (e.g., a display, a speaker, one or more light-emitting diodes (LEDs), etc.).

[0110] Communication interface 214 may include a transceiver-like component (e.g., a transceiver, a separate receiver and transmitter, etc.) that enables device 200 to communicate with other devices, such as via a wired connection, a wireless connection, or a combination of wired and wireless connections. Communication interface 214 may permit device 200 to receive information from another device and / or provide information to another device. For example, communication interface 214 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi® interface, a cellular network interface, and / or the like.

[0111] Device 200 may perform one or more processes described herein. Device 200 may perform these processes based on processor 204 executing software instructions stored by a computer-readable medium, such as memory 206 and / or storage component 208. A computer-readable medium (e.g., a non-transitory computer-readable medium) is defined herein as a non-transitory memory device. A memory device includes memory space located inside of a single physical storage device or memory space spread across multiple physical storage devices.

[0112] Software instructions may be read into memory 206 and / or storage component 208 from another computer-readable medium or from another device via communication interface 214. When executed, software instructions stored in memory 206 and / or storage component 208 may cause processor 204 to perform one or more processes described herein. Additionally, or alternatively, hardwired circuitry may be used in place of or in combination with software instructions to perform one or more processes described herein. Thus, embodiments or aspects described herein are not limited to any specific combination of hardware circuitry and software.

[0113] The number and arrangement of components shown in FIG. 2 are provided as an example. In some non-limiting embodiments or aspects, device 200 may include additional components, fewer components, different components, or differently arranged components than those shown in FIG. 2. Additionally or alternatively, a set of components (e.g., one or more components) of device 200 may perform one ormore functions described as being performed by another set of components of device 200.

[0114] Referring now to FIG. 3, FIG. 3 is a flowchart of a non-limiting embodiment or aspect of a process 300 for perturbation-based interpretation of the effects of related features in machine learning models, according to some non-limiting embodiments or aspects. The steps shown in FIG. 3 are for example purposes only. It will be appreciated that additional, fewer, different, and / or a different order of steps may be used in non-limiting embodiments or aspects. In some non-limiting embodiments or aspects, one or more of the steps of process 300 may be performed (e.g., completely, partially, and / or the like) by modeling system 102. In some non-limiting embodiments or aspects, one or more of the steps of process 300 may be performed (e.g., completely, partially, and / or the like) by another system, another device, another group of systems, or another group of devices, separate from or including modeling system 102.

[0115] As shown in FIG. 3, at step 302, process 300 may include generating a knowledge graph based on a plurality of features and a plurality of relationships. For example, modeling system 102 may generate a knowledge graph based on a plurality of features and a plurality of relationships between features of the plurality of features. The knowledge graph may include a plurality of nodes connected by a plurality of edges. Each node of the plurality of nodes may be associated with a feature of the plurality of features. Each edge of the plurality of edges may be associated with a relationship of the plurality of relationships between two features of the plurality of features.

[0116] In some non-limiting embodiments or aspects, the knowledge graph may be associated with (e.g., represented as) a node table and an edge table. For example, modeling system 102 may store a node table in memory 104 associated with the plurality of nodes of the knowledge graph. The node table may identify, for each node of the plurality of nodes, an identifier of the node, an identifier of a feature associated with the node, an initial feature value associated with the node, at least one perturbation parameter associated with the feature of the node, one or more nodes connected to the node by edge, or any combination thereof. By way of further example, modeling system 102 may store an edge table in memory 104 associated with the plurality of edges of the knowledge graph. The edge table may identify, for each edge of the plurality of edges, an identifier of a source node of the plurality ofnodes connected by the edge, an identifier of a target node of the plurality of nodes connected by the edge, a constraint on a feature value of a feature associated with the target node based on a feature value of a feature associated with the source node, or any combination thereof.

[0117] In some non-limiting embodiments or aspects, modeling system 102 may determine the plurality of relationships associated with the plurality of edges based on user input. For example, modeling system 102 may display the plurality of features in a user interface of computing device 106 of a user. Modeling system 102 may receive user input including an identification of at least one pairing of features of the plurality of features. Modeling system 102 may then determine the plurality of relationships between the features of the plurality of features based at least partly on the at least one pairing of features of the user input.

[0118] As shown in FIG. 3, at step 304, process 300 may include determining a plurality of feature values. For example, modeling system 102 may determine a plurality of feature values associated with the plurality of features. In some non-limiting embodiments or aspects, modeling system 102 may determine each feature value of the plurality of feature values by randomly initializing a feature value for each feature based on at least one perturbation parameter for the feature in the node table.

[0119] As shown in FIG. 3, at step 306, process 300 may include determining a first feature associated with a first node. For example, modeling system 102 may determine a first feature associated with a first node of the plurality of nodes of the knowledge graph. In some non-limiting embodiments or aspects, modeling system 102 may determine the first feature based on user input from computing device 106 of a user (e.g., in a request message from computing device 106 to provide the interpretation of the effect of the first feature on the output of the model). In some nonlimiting embodiments or aspects, modeling system 102 may randomly identify a node in the plurality of nodes to begin perturbation analysis.

[0120] As shown in FIG. 3, at step 308, process 300 may include perturbating a feature value associated with the first feature. For example, modeling system 102 may perturbate a feature value of the plurality of feature values associated with the first feature. When perturbating the feature value associated with the first feature, modeling system 102 may update the feature value associated with the first feature to a new feature value associated with the first feature.

[0121] In some non-limiting embodiments or aspects, when perturbating the feature value of the plurality of feature values associated with the first feature, modeling system 102 may update the feature value associated with the first feature to a new feature value associated with the first feature based on the node table. For example, the new feature value associated with the first feature may satisfy at least one perturbation parameter of the first feature associated with the first node.

[0122] As shown in FIG. 3, at step 310, process 300 may include determining a second feature associated with a second node based on an edge. For example, modeling system 102 may determine a second feature associated with a second node of the plurality of nodes of the knowledge graph, based on an edge connecting the first node to the second node. In some non-limiting embodiments or aspects, modeling system 102 may reference the node table and / or edge table to determine a second node connected to the first node by an edge.

[0123] As shown in FIG. 3, at step 312, process 300 may include perturbating a feature value associated with the second feature. For example, modeling system 102 may perturbate a feature value of the plurality of feature values associated with the second feature based on the new feature value associated with the first feature. When perturbating the feature value associated with the second feature, modeling system 102 may update the feature value associated with the second feature to a new feature value associated with the second feature, based on the new feature value associated with the first feature.

[0124] In some non-limiting embodiments or aspects, when perturbating the feature value of the plurality of feature values associated with the second feature, modeling system 102 may update the feature value associated with the second feature to a new feature value associated with the second feature based on the node table and the edge table. For example, the new feature value associated with the second feature may satisfy at least one perturbation parameter of the second feature associated with the second node, and may further satisfy the constraint on the feature value of the second feature based on the feature value of the first feature.

[0125] As shown in FIG. 3, at step 314, process 300 may include inputting the plurality of feature values to a decision-based machine learning model. For example, modeling system 102 may input the plurality of feature values to a decision-based machine learning model. In some non-limiting embodiments or aspects, modeling system 102 may input the plurality of feature values to the decision-based machinelearning model before perturbating feature values, during perturbating feature values, and / or after perturbating feature values.

[0126] As shown in FIG. 3, at step 316, process 300 may include determining a significance of an effect of the first feature and the second feature on an output of the decision-based machine learning model. For example, modeling system 102 may determine a significance of an effect of the first feature and the second feature on an output of the decision-based machine learning model based on the new feature value associated with the first feature and the new feature value associated with the second feature.

[0127] In some non-limiting embodiments or aspects, modeling system 102 may generate a first output of the decision-based machine learning model based on the plurality of feature values associated with the plurality of features before perturbating feature values of the plurality of features associated with the first feature and the second feature. Modeling system 102 may also generate a second output of the decision-based machine learning model based on the plurality of feature values associated with the plurality of features after perturbating feature values of the plurality of feature values associated with the first feature and the second feature. Modeling system 102 may further compare the first output to the second output to determine the significance of the effect of the first feature and the second feature on the output of the decision-based machine learning model.

[0128] In some non-limiting embodiments or aspects, modeling system 102 may reduce the plurality of features input to the decision-based machine learning model based on the significance of the effect of the first feature and the second feature on the output of the decision-based machine learning model to produce a reduced plurality of features. For example, modeling system 102 may determine that the second output is not significantly different than the first output and may remove the first feature and / or the second feature from the plurality of features. By way of another example, modeling system 102 may determine that the second output is significantly different than the first output and may remove at least one feature from the plurality of features that is not the first feature nor the second feature. After producing the reduced plurality of features, modeling system 102 may execute the decision-based machine learning model based on the reduced plurality of features (e.g., to output a decision, such as part of a transaction fraud detection system, a targeted advertisement system, a network security system, etc.).

[0129] In some non-limiting embodiments or aspects, modeling system 102 may repeat steps 310, 312, 314, and / or 316 for additional features / nodes of the knowledge graph. For example, modeling system 102 may, in a repetition of step 310, determine a third feature associated with a third node of the plurality of nodes, based on an edge connecting the third node to the first node and / or the second node. Modeling system 102 may further, in a repetition of step 312, perturbate a feature value of the plurality of feature values associated with the third feature based on the new feature value associated with the first feature and / or the second feature, wherein perturbating the feature value associated with the third feature includes updating the feature value associated with the third feature to a new feature value associated with the third feature, based on the new feature value associated with the first feature and / or the second feature. Modeling system 102 may, in step 314, input the plurality of feature values to the decision-based machine learning model and, in step 316, determine a significance of an effect of the first feature, the second feature, and / or the third feature on the output of the decision-based machine learning model based on the new feature value associated with the first feature, the new feature value associated with the second feature, and / or the new feature value associated with the third feature. It will be appreciated that modeling system 102 may repeat the above-described steps for a fourth feature, a fifth feature, etc.

[0130] Referring now to FIG. 4, FIG. 4 is a flow diagram of a method for perturbation-based interpretation of the effects of related features in machine learning models, according to some non-limiting embodiments or aspects. The steps shown in FIG. 4 are for example purposes only. It will be appreciated that additional, fewer, different, and / or a different order of steps may be used in non-limiting embodiments or aspects. In some non-limiting embodiments or aspects, one or more of the steps of process 400 may be performed (e.g., completely, partially, and / or the like) by modeling system 102. In some non-limiting embodiments or aspects, one or more of the steps of process 400 may be performed (e.g., completely, partially, and / or the like) by another system, another device, another group of systems, or another group of devices, separate from or including modeling system 102.

[0131] As shown in FIG. 4, at step 402, process 400 may include receiving a data instance of a modeled environment. For example, in an electronic payment processing network, the data instance may be related to transaction data of one or more processed transactions. By way of another example, in a networkedenvironment, the data instance may be related to network activity data of one or more network access requests. In some non-limiting embodiments or aspects, modeling system 102 may retrieve the data instance from memory 104 and map the data instance to a look-up table 401 , where each row is a data record and each column is a data field. Modeling system 102 may construct a node table based on the look-up table 401 , wherein each row of the node table is associated with a feature of the data instance. See FIG. 6 for further detailed description of a node table.

[0132] As shown in FIG. 4, at step 404, process 400 may include generating a knowledge graph based on the data instance. For example, modeling system 102 may generate the knowledge graph based on the look-up table 401 . Modeling system 102 may generate the node table before or during generation of the knowledge graph (denoted in FIG. 4 by clusters of circles connected by lines) based on the look-up table. Each node (denoted in FIG. 4 by circles) of the knowledge graph may be associated with a data feature. Each edge 405 (denoted in FIG. 4 by lines connecting circles) of the knowledge graph may be associated with a relationship between two nodes. The information required to assign features to nodes and relationships to edges 405 may be determined by modeling system 102 from global statistical analysis of the environment, human knowledge input (e.g., from computing device 104), or a combination thereof. Upon generation, each node of the knowledge graph is a value- unassigned node 403 (denoted in FIG. 4 by the white-filled circles). A value- unassigned node 403 is associated with a feature that has not been associated with an initial value yet.

[0133] As shown in FIG. 4, at step 406, process 400 may include determining a plurality of feature values associated with the plurality of features associated with the plurality of nodes of the knowledge graph. For example, modeling system 102 may update each node of the knowledge graph such that the associated feature of the node is assigned an initial feature value. The initial feature value may be based on at least one perturbation parameter of the feature that is defined in the node table. After determining the plurality of feature values, each node of the knowledge graph is an initialized node 407. An initialized node 407 is associated with a feature that has been associated with an initial value. Modeling system 102 may use the plurality of initial feature values as input to at least one decision-based machine learning model, to generate a first output prior to perturbation.

[0134] As shown in FIG. 4, at step 408, process 400 may include determining a first node 409 of the plurality of nodes and perturbating the first node 409. For example, modeling system 102 may determine a first node 409 (denoted in FIG. 4 as the black-filled nodes marked with a letter “A”) to begin the perturbation process. To perturbate the first node 409, modeling system 102 may update a feature value of a first feature associated with a first node 409 to a new feature value. In some nonlimiting embodiments or aspects, modeling system 102 may update the feature value of the first feature to a new feature value based on at least one perturbation parameter associated with the first feature in the node table. As such, the first node 409 and its associated feature is perturbated in a first round of perturbations.

[0135] As shown in FIG. 4, at step 410, process 400 may include determining one or more second nodes 41 1 of the plurality of nodes and perturbating the one or more second nodes 41 1 . For example, modeling system 102 may determine one or more second nodes 41 1 (denoted in FIG. 4 as the black-filled nodes marked with a letter “B”) that are connected to the first node 409 by at least one edge, to continue the perturbation process. To perturbate the one or more second nodes 41 1 , modeling system 102 may, for each of the second nodes 41 1 , update a feature value of a second feature associated with the second node 411 to a new feature value based at least partly on the new feature value of the first feature associated with the first node 409. In some non-limiting embodiments or aspects, modeling system 102 may update the feature value of one or more second features based on at least one perturbation parameter for the second feature, as defined in the node table, and further based on at least one constraint for the second feature, as defined in the edge table. The constraint may condition the feature value of the second feature based on the new feature value of the first feature, which was updated in step 408.

[0136] As shown in FIG. 4, at step 412, process 400 may include determining one or more third nodes 413 of the plurality of nodes and perturbating the one or more third nodes 413. For example, modeling system 102 may determine one or more third nodes 413 (denoted in FIG. 4 as the black-filled nodes marked with a letter “C”) that are connected to a second node 411 by at least one edge, to continue the perturbation process. To perturbate the one or more third nodes 413, modeling system 102 may, for each of the third nodes 413, update a feature value of a third feature associated with the third node 413 to a new feature value based at least partly on the new feature value of the second feature associated with the connected second node 41 1. In somenon-limiting embodiments or aspects, modeling system 102 may update the feature value of one or more third features based on at least one perturbation parameter for the third feature, as defined in the node table, and further based on at least one constraint for the third feature, as defined in the edge table. The constraint may condition the feature value of the third feature based on the new feature value of the related second feature, which was updated in step 410. It will be appreciated that modeling system 102 may continue the above steps for fourth nodes, fifth nodes, etc., by traversing a path (e.g., of connected edges) from each source node to each target node, perturbating the target nodes based on at least one perturbation parameter associated with the target node, the new value of the source node, and at least one constraint on the feature value of the target node based on the new value of the source node.

[0137] As shown in FIG. 4, at step 414, process 400 may include using the perturbated data instance 415 for perturbation-based interpretation of one or more decision-based machine learning models. For example, the plurality of feature values, after perturbation (e.g., step 412), may be input to a decision-based machine learning model to generate a second output that is based on post-perturbation feature values. Modeling system 102 may the compare the first output (e.g., generated preperturbation) to the second output (e.g., generated post-perturbation) to determine a significance of the effect of one or more features on the output of the decision-based machine learning model. For example, a metric (e.g., percentile difference, categorical difference, proportion of change in output in comparison to proportion of change in input, etc.) may be used to evaluate a statistical difference between the first output and the second output. If the statistical difference in outputs is significant (e.g., statistically significant), then the effect of the perturbated features may also be concluded to be significant. It will be appreciated that the relationship-based perturbation process described herein may be used to generate a perturbated data instance 415 that may be used in a number of different perturbation-based analysis frameworks, such as LIME, SHAP, TimeSHAP, etc., to determine if the effects of the perturbated features are significant on the output of the model. This provides a degree of framework agnosticism, allowing the described methods to be used and integrated to a system flexibly.

[0138] Referring now to FIG. 5, FIG. 5 is an illustration of a knowledge graph 500, for use in a method for perturbation-based interpretation of the effects of relatedfeatures in machine learning models, according to some non-limiting embodiments or aspects. The knowledge graph 500 shown in FIG. 5 is for example purposes only. In some non-limiting embodiments or aspects, a method making use of the exemplary knowledge graph 500 may be performed (e.g., completely, partially, and / or the like) by modeling system 102. In some non-limiting embodiments or aspects, the method making use of the exemplary knowledge graph 500 may be performed (e.g., completely, partially, and / or the like) by another system, another device, another group of systems, or another group of devices, separate from or including modeling system 102.

[0139] As shown in FIG. 5, each icon represents a different data feature of a data instance that may be received and encoded into the knowledge graph 500 schema by modeling system 102. Each icon is associated with a node of the knowledge graph 500. The exemplary depicted knowledge graph 500 is for a data instance of transaction data associated with one or more transactions of a payment device, for modeling in an electronic payment processing network (e.g., using a fraud detection and mitigation system). The following data features are associated with nodes of the knowledge graph 500 shown in FIG. 5: “tA”, a numerical feature corresponding to a transaction amount; “tUA”, a numerical feature corresponding to a transaction amount in United States dollars (USD); “tin”, a categorical feature corresponding to a travel indicator, associated with a transaction made during travel; “meed”, a categorical feature corresponding to a merchant country code, associated with the country code of a merchant involved in a transaction; “teed”, a categorical feature corresponding to a transaction currency code, associated with an identifier of a currency type for a transaction; “xi”, a categorical feature corresponding to a cross border indicator, associated with a cross-border transaction; “d1 m”, a numerical feature corresponding to a number of declines for a payment device involved in a transaction, over the last minute; “d5m”, a numerical feature corresponding to a number of declines for a payment device involved in a transaction, over the last five minutes; “d11”, a numerical feature corresponding to a number of declines for a payment device involved in a transaction, in the last transaction; “d5t”, a numerical feature corresponding to a number of declines for a payment device involved in a transaction, in the last five transactions; “tg”, a numerical feature corresponding to a number of seconds between the current transaction and a last transaction; “nohistory”, a categorical feature corresponding to whether the transactions is the first for a payment device; and “mdA”,a numerical feature corresponding to a maximum declined amount in the past day. The nodes of the knowledge graph 500 may be described in node table 600, stored in memory 104 (see, e.g., FIG. 6). The above features are exemplary and for the purpose of illustration only, and are not meant to be limiting on the understanding of a knowledge graph.

[0140] As shown in FIG. 5, the nodes are connected in the knowledge graph 500 by a number of edges, which represent relationships between the features. For example, “tccd” is connected to four other features: “tUA”, “meed”, “tA”, and “tin”, indicating that there is a relationship between the values of the “tccd” feature and the values of the connected features. By way of another example, “d1t” is connected to “d5t”, indicating that the value of “d1t” would be related to the value of “d5t”. This comports with the logic that at least the number of declines in the last one minute would be included in the number of declines in the last five minutes. Therefore, if perturbating the value of “d1t”, the value of “d5t” should be perturbated as well, to account for the relationship between the features. The relationships of knowledge graph 500 may be described in edge table 700, stored in memory 104 (see, e.g., FIG. 7).

[0141] Referring now to FIG. 6, FIG. 6 is an illustration of node table 600, for use in a method for perturbation-based interpretation of the effects of related features in machine learning models, according to some non-limiting embodiments or aspects. Node table 600 shown in FIG. 6 is for example purposes only, and corresponds generally to illustrative knowledge graph 500 shown in FIG. 5. In some non-limiting embodiments or aspects, a method making use of exemplary node table 600 may be performed (e.g., completely, partially, and / or the like) by modeling system 102. In some non-limiting embodiments or aspects, the method making use of exemplary node table 600 may be performed (e.g., completely, partially, and / or the like) by another system, another device, another group of systems, or another group of devices, separate from or including modeling system 102.

[0142] As shown in FIG. 6, node table 600 may include, but is not limited to, a row associated with each node and / or feature of knowledge graph 500. Node table 600 may further include at least one perturbation parameter for each node and / or feature of knowledge graph 500. Each perturbation parameter may include, but is not limited to, a range, an upper bound, a lower bound, a data type, a mathematical method of perturbation, a set of acceptable values, etc. In some non-limiting embodiments oraspects, and for illustrative purposes, node table 600 includes a column to show an example value of each feature of knowledge graph 500.

[0143] For example, the “tA” feature has two perturbation parameters, including a range and a mathematical method of perturbation. Any new value for “tA” must be between 0 and a maximum value and may be generated using a probability distribution that is based on historic transaction data. The “tUA” feature has one perturbation parameter, including a mathematical method of perturbation. Any new value for “tUA” must be determined from “tA” and “tccd”, where the transaction amount and country code can be used to convert to the amount in USD. The “tin”, “xi”, and “no history” features each have one perturbation parameter, including a set of acceptable values, which define any new value for these features as being “0” or “1 ”. The “meed” feature has one perturbation parameter, including a set of acceptable values. Any new value for “meed” must be selected from a country code list. The “tccd” feature has one perturbation parameter, including a set of acceptable values. Any new value for “tccd” must be “USD” or related directly to the “meed” value. The “di m”, “d5m”, “d1 t”, “d5t”, and “tg” features have two perturbation parameters each, including a data type and a range. Any new value for “d1 m”, “d5m”, “d1 t”, “d5t”, and “tg” must be an integer value between 0 and a maximum value (e.g., set as 1 for “d1 t” and 5 for “d5t”). Finally, the “mdA” feature has one perturbation parameter, including a range. Any new value for “mdA” must be between 0 and a mean value based on historic transaction data. Modeling system 102 may determine perturbation parameters based on user input and / or global statistical data.

[0144] Referring now to FIG. 7, FIG. 7 is an illustration of edge table 700, for use in a method for perturbation-based interpretation of the effects of related features in machine learning models, according to some non-limiting embodiments or aspects. Edge table 700 shown in FIG. 7 is for example purposes only and corresponds generally to illustrative knowledge graph 500 shown in FIG. 5. In some non-limiting embodiments or aspects, a method making use of exemplary edge table 700 may be performed (e.g., completely, partially, and / or the like) by modeling system 102. In some non-limiting embodiments or aspects, the method making use of exemplary edge table 700 may be performed (e.g., completely, partially, and / or the like) by another system, another device, another group of systems, or another group of devices, separate from or including modeling system 102.

[0145] As shown in FIG. 7, edge table 700 includes definitions for the relationships between pairings of nodes in knowledge graph 500. Each pairing of nodes includes a source node and a target node. Each pairing of nodes also includes a constraint that, in part, defines the relationship between the source node and the target node. A constraint, in part, defines how the new value of a perturbated target node may be changed based on the new value of a perturbated source node. In some non-limiting embodiments or aspects, the target node’s feature value may be said to be dependent on the source node’s feature value. Example edge table 700 illustrates some, but not all, of the constrained relationships of knowledge graph 500.

[0146] For example, if the “tA” node is perturbated, then the perturbation of related node “tUA” will be constrained by the new value for “tA”. As shown, the value (V) of the target node “tUA” is a function of the value of the source node “tA”, given the value of “tccd”. By way of further example, if the “tin” node is perturbated, then the perturbation of related node “xi” will be constrained by the new value for “tin”. As shown, if the value of the source node “tin” is 0, then so too must be the value of the target node “xi.” By way of further example, if the “meed” node is perturbated, then the perturbation of related node “tccd” will be constrained by the new value for “meed”. As shown, if there is a change (A) in “meed”, then there must be a change in “tccd” that is conditional (|) upon “meed”. By way of further example, if the “di m” or “d1 t” nodes are perturbated, then the perturbations of related nodes “d5m” and “d5t” will be constrained by the new values for “di m” and “d1 t”, respectively. As shown, each target node’s new value is constrained as being at least the new value of the source node or larger, but less than some maximum value. By way of further example, if the “di m” node is perturbated, then the perturbation of related node “tg” will be constrained by the new value for “d1 m”. As shown, if “d1 m” has a new value that is greater than 0, then the value for “tg” must be less than 60 seconds. By way of further example, if the “nohistory” node is perturbated, then the perturbation for related node “tg” will be constrained by the new value for “nohistory”. As shown, if the value for “nohistory” is 1 , then the value for “tg” will be set to -1 , since there will necessarily be no time gap since last decline. Likewise, if “di m” or “d1 t” are perturbated, then the value for “nohistory” will be constrained, such that if “d1 m” or d1t” are greater than 1 , “nohistory” will be set to 0, since there must be a transaction history if there are past declined transactions.

[0147] If a target node is constrained by the value of more than one source node during perturbation, then modeling system 102 may operate to provide a new value of target node that abides by all constraints and perturbation parameters. If a source node constrains more than one target node, then modeling system 102 may operate to provide a new value for each constrained target node in the next layer of perturbation.

[0148] Although the disclosure has been described in detail for the purpose of illustration based on what is currently considered to be the most practical and preferred embodiments or aspects, it is to be understood that such detail is solely for that purpose and that the disclosure is not limited to the disclosed embodiments or aspects, but, on the contrary, is intended to cover modifications and equivalent arrangements that are within the spirit and scope of the appended claims. For example, it is to be understood that the present disclosure contemplates that, to the extent possible, one or more features of any embodiment or aspect can be combined with one or more features of any other embodiment or aspect, and one or more steps may be taken in a different order than presented in the present disclosure.

Claims

WHAT IS CLAIMED IS:1 . A computer-implemented method comprising: generating, with at least one processor, a knowledge graph based on a plurality of features and a plurality of relationships between features of the plurality of features, the knowledge graph comprising a plurality of nodes connected by a plurality of edges, each node of the plurality of nodes being associated with a feature of the plurality of features, and each edge of the plurality of edges being associated with a relationship of the plurality of relationships between two features of the plurality of features; determining, with at least one processor, a plurality of feature values associated with the plurality of features; determining, with at least one processor, a first feature associated with a first node of the plurality of nodes; perturbating, with at least one processor, a feature value of the plurality of feature values associated with the first feature, wherein perturbating the feature value associated with the first feature comprises: updating the feature value associated with the first feature to a new feature value associated with the first feature; determining, with at least one processor, a second feature associated with a second node of the plurality of nodes, based on an edge connecting the first node to the second node; perturbating, with at least one processor, a feature value of the plurality of feature values associated with the second feature based on the new feature value associated with the first feature, wherein perturbating the feature value associated with the second feature comprises: updating the feature value associated with the second feature to a new feature value associated with the second feature, based on the new feature value associated with the first feature; inputting, with at least one processor, the plurality of feature values to a decision-based machine learning model; and determining, with at least one processor, a significance of an effect of the first feature and the second feature on an output of the decision-based machinelearning model based on the new feature value associated with the first feature and the new feature value associated with the second feature.

2. The method of claim 1 , further comprising: displaying, with at least one processor, the plurality of features in a user interface of a computing device of a user; receiving, with at least one processor, user input comprising an identification of at least one pairing of features of the plurality of features; and determining, with at least one processor, the plurality of relationships between the features of the plurality of features based at least partly on the at least one pairing of features of the user input.

3. The method of claim 1 , further comprising: storing, with at least one processor, a node table associated with the plurality of nodes of the knowledge graph, wherein the node table identifies, for each node of the plurality of nodes, at least one perturbation parameter of the feature associated with the node; and wherein perturbating the feature value of the plurality of feature values associated with the first feature further comprises: updating the feature value associated with the first feature to a new feature value associated with the first feature based on the node table, wherein the new feature value associated with the first feature satisfies the at least one perturbation parameter of the first feature associated with the first node.

4. The method of claim 3, further comprising: storing, with at least one processor, an edge table associated with the plurality of edges of the knowledge graph, wherein the edge table identifies, for each edge of the plurality of edges, a source node of the plurality of nodes connected by the edge, a target node of the plurality of nodes connected by the edge, and a constraint on a feature value of a feature associated with the target node based on a feature value of a feature associated with the source node; and wherein perturbating the feature value of the plurality of feature values associated with the second feature further comprises:updating the feature value associated with the second feature to a new feature value associated with the second feature based on the node table and the edge table, wherein the new feature value associated with the second feature satisfies the at least one perturbation parameter of the second feature associated with the second node, and satisfies the constraint on the feature value of the second feature based on the feature value of the first feature.

5. The method of claim 1 , wherein determining the significance of the effect of the first feature and the second feature on the output of the decisionbased machine learning model comprises: generating a first output of the decision-based machine learning model based on the plurality of feature values associated with the plurality of features before perturbating feature values of the plurality of feature values associated with the first feature and the second feature; generating a second output of the decision-based machine learning model based on the plurality of feature values associated with the plurality of features after perturbating feature values of the plurality of feature values associated with the first feature and the second feature; and comparing the first output to the second output.

6. The method of claim 5, further comprising: reducing, with at least one processor, the plurality of features input to the decision-based machine learning model based on the significance of the effect of the first feature and the second feature on the output of the decision-based machine learning model, to produce a reduced plurality of features; and executing, with at least one processor, the decision-based machine learning model based on the reduced plurality of features.

7. The method of claim 6, wherein reducing the plurality of features input to the decision-based machine learning model based on the significance of the effect of the first feature and the second feature comprises: determining that the second output is not significantly different than the first output; andremoving the first feature and the second feature from the plurality of features.

8. The method of claim 6, wherein reducing the plurality of features input to the decision-based machine learning model based on the significance of the effect of the first feature and the second feature comprises: determining that the second output is significantly different than the first output; and removing at least one feature from the plurality of features, wherein the at least one feature is not the first feature nor the second feature.

9. The method of claim 1 , further comprising: determining, with at least one processor, a third feature associated with a third node of the plurality of nodes, based on an edge connecting the second node to the third node; perturbating, with at least one processor, a feature value of the plurality of feature values associated with the third feature based on the new feature value associated with the second feature, wherein perturbating the feature value associated with the third feature comprises: updating the feature value associated with the third feature to a new feature value associated with the third feature, based on the new feature value associated with the second feature; and determining, with at least one processor, a significance of an effect of the first feature, the second feature, and the third feature on the output of the decisionbased machine learning model based on the new feature value associated with the first feature, the new feature value associated with the second feature, and the new feature value associated with the third feature.

10. A system comprising at least one processor programmed or configured to: generate a knowledge graph based on a plurality of features and a plurality of relationships between features of the plurality of features, the knowledge graph comprising a plurality of nodes connected by a plurality of edges, each node of the plurality of nodes being associated with a feature of the plurality of features, andeach edge of the plurality of edges being associated with a relationship of the plurality of relationships between two features of the plurality of features; determine a plurality of feature values associated with the plurality of features; determine a first feature associated with a first node of the plurality of nodes; perturbate a feature value of the plurality of feature values associated with the first feature, wherein, when perturbating the feature value associated with the first feature, the at least one processor is programmed or configured to: update the feature value associated with the first feature to a new feature value associated with the first feature; determine a second feature associated with a second node of the plurality of nodes, based on an edge connecting the first node to the second node; perturbate a feature value of the plurality of feature values associated with the second feature based on the new feature value associated with the first feature, wherein, when perturbating the feature value associated with the second feature, the at least one processor is programmed or configured to: update the feature value associated with the second feature to a new feature value associated with the second feature, based on the new feature value associated with the first feature; input the plurality of feature values to a decision-based machine learning model; and determine a significance of an effect of the first feature and the second feature on an output of the decision-based machine learning model based on the new feature value associated with the first feature and the new feature value associated with the second feature.1 1. The system of claim 10, wherein, when determining the significance of the effect of the first feature and the second feature on the output of the decision-based machine learning, the at least one processor is programmed or configured to: generate a first output of the decision-based machine learning model based on the plurality of feature values associated with the plurality of features beforeperturbating feature values of the plurality of feature values associated with the first feature and the second feature; generate a second output of the decision-based machine learning model based on the plurality of feature values associated with the plurality of features after perturbating feature values of the plurality of feature values associated with the first feature and the second feature; and compare the first output to the second output.

12. The system of claim 1 1 , wherein the at least one processor is further programmed or configured to: reduce the plurality of features input to the decision-based machine learning model based on the significance of the effect of the first feature and the second feature on the output of the decision-based machine learning model, to produce a reduced plurality of features; and execute the decision-based machine learning model based on the reduced plurality of features.

13. The system of claim 12, wherein, when reducing the plurality of features input to the decision-based machine learning model based on the significance of the effect of the first feature and the second feature, the at least one processor is programmed or configured to: determine that the second output is not significantly different than the first output; and remove the first feature and the second feature from the plurality of features.

14. The system of claim 12, wherein, when reducing the plurality of features input to the decision-based machine learning model based on the significance of the effect of the first feature and the second feature, the at least one processor is programmed or configured to: determine that the second output is significantly different than the first output; and remove at least one feature from the plurality of features, wherein the at least one feature is not the first feature nor the second feature.

15. The system of claim 10, wherein the at least one processor is further programmed or configured to: determine a third feature associated with a third node of the plurality of nodes, based on an edge connecting the second node to the third node; perturbate a feature value of the plurality of feature values associated with the third feature based on the new feature value associated with the second feature, wherein, when perturbating the feature value associated with the third feature, the at least one processor is programmed or configured to: update the feature value associated with the third feature to a new feature value associated with the third feature, based on the new feature value associated with the second feature; and determine a significance of an effect of the first feature, the second feature, and the third feature on the output of the decision-based machine learning model based on the new feature value associated with the first feature, the new feature value associated with the second feature, and the new feature value associated with the third feature.

16. A computer program product comprising at least one non- transitory computer-readable medium comprising one or more instructions that, when executed by at least one processor, cause the at least one processor to: generate a knowledge graph based on a plurality of features and a plurality of relationships between features of the plurality of features, the knowledge graph comprising a plurality of nodes connected by a plurality of edges, each node of the plurality of nodes being associated with a feature of the plurality of features, and each edge of the plurality of edges being associated with a relationship of the plurality of relationships between two features of the plurality of features; determine a plurality of feature values associated with the plurality of features; determine a first feature associated with a first node of the plurality of nodes; perturbate a feature value of the plurality of feature values associated with the first feature, wherein the one or more instructions that cause the at least oneprocessor to perturbate the feature value associated with the first feature cause the at least one processor to: update the feature value associated with the first feature to a new feature value associated with the first feature; determine a second feature associated with a second node of the plurality of nodes, based on an edge connecting the first node to the second node; perturbate a feature value of the plurality of feature values associated with the second feature based on the new feature value associated with the first feature, wherein the one or more instructions that cause the at least one processor to perturbate the feature value associated with the second feature cause the at least one processor to: update the feature value associated with the second feature to a new feature value associated with the second feature, based on the new feature value associated with the first feature; input the plurality of feature values to a decision-based machine learning model; and determine a significance of an effect of the first feature and the second feature on an output of the decision-based machine learning model based on the new feature value associated with the first feature and the new feature value associated with the second feature.

17. The computer program product of claim 16, wherein the one or more instructions that cause the at least one processor to determine the significance of the effect of the first feature and the second feature on the output of the decisionbased machine learning cause the at least one processor to: generate a first output of the decision-based machine learning model based on the plurality of feature values associated with the plurality of features before perturbating feature values of the plurality of feature values associated with the first feature and the second feature; generate a second output of the decision-based machine learning model based on the plurality of feature values associated with the plurality of features after perturbating feature values of the plurality of feature values associated with the first feature and the second feature; and compare the first output to the second output.

18. The computer program product of claim 17, wherein the one or more instructions further cause the at least one processor to: reduce the plurality of features input to the decision-based machine learning model based on the significance of the effect of the first feature and the second feature on the output of the decision-based machine learning model, to produce a reduced plurality of features; and execute the decision-based machine learning model based on the reduced plurality of features.

19. The computer program product of claim 18, wherein the one or more instructions that cause the at least one processor to reduce the plurality of features input to the decision-based machine learning model based on the significance of the effect of the first feature and the second feature cause the at least one processor to: determine that the second output is not significantly different than the first output; and remove the first feature and the second feature from the plurality of features.

20. The computer program product of claim 18, wherein the one or more instructions that cause the at least one processor to reduce the plurality of features input to the decision-based machine learning model based on the significance of the effect of the first feature and the second feature cause the at least one processor to: determine that the second output is significantly different than the first output; and remove at least one feature from the plurality of features, wherein the at least one feature is not the first feature nor the second feature.

Images