Method, system, and computer program product for analysis of machine learning models based on sequential counterfactual actions

Abstract

Methods, systems, and computer program products are provided for analysis of machine learning models based on sequential counterfactual actions. A method includes receiving a data record including fields associated with features. A feature dependence matrix associated with a dependence between each feature and every other feature is received. The data record is inputted to a machine learning model to generate an initial output. At least one tuple including an action vector and a sequence vector is determined based on the data record and feature dependence matrix. Each action vector is associated with perturbations of fields for which inputting a perturbed data record based thereon to the machine learning model generates a different output than the initial output. Each sequence vector is associated with a sequence of the perturbations. A sequence loss is determined for each tuple. At least one communication based on the tuples and the sequence losses is communicated.

Description

METHOD, SYSTEM, AND COMPUTER PROGRAM PRODUCT FOR ANALYSIS OF MACHINE LEARNING MODELS BASED ON SEQUENTIAL COUNTERFACTUAL ACTIONSCROSS REFERENCE TO RELATED APPLICATION

[0001] This application claims priority to United States Provisional Patent Application No. 63 / 542,646, filed October 5, 2023, the disclosure of which is hereby incorporated by reference in its entirety.BACKGROUND1. Field

[0002] This disclosure relates generally to analysis of machine learning models and, in some non-limiting embodiments or aspects, to methods, systems, and computer program products for analysis of machine learning models based on sequential counterfactual actions.2. Technical Considerations

[0003] Machine learning models are successful in performing many real-world tasks. However, it is difficult to explain why a machine learning model generated a particular output and / or to interpret the behavior of a machine learning model. This difficulty is exacerbated (e.g., increases) as machine learning models become more complex and / or deeper (e.g., deep learning), and opacity of such models can be problematic. For example, in certain high-stakes scenarios such as financial investment and policy making, interpreting the behaviors of machine learning models is vital for humans to properly understand the outputs of the models prior to making important decisions.

[0004] Certain techniques attempting to explain and / or interpret machine learning models, such as feature attribution, may highlight features that contribute to an output, but such techniques lack reasoning capability since such techniques only involve the output (e.g., predicted label) for interpretation. This limits the understanding provided by such techniques because, to provide human-interpretable reasoning, it is not sufficient to just identify which features are important, and such techniques fail to explain how such features impact the outcome. Other techniques, such as counterfactual explanation, use a hypothetical data sample to flip the output of a machine learning model. However, such hypothetical data samples include all features (or changes to features) simultaneously, and as such, these techniques fail to consider timing or order of including features (or charges to features). Also, thehypothetical data samples may not be feasible. For example, when trying to explain why an individual’s loan application was rejected, using a hypothetical data in which a hypothetical individual having all other attributes the same as the original individual except for suddenly having a much higher level of education (e.g., without getting any older) is neither feasible nor meaningful to the original individual.SUMMARY

[0005] Accordingly, provided are improved methods, systems, and computer program products for analysis of machine learning models based on sequential counterfactual actions (e.g., that overcome some or all of the deficiencies identified above).

[0006] According to non-limiting embodiments or aspects, provided is a computer- implemented method for analysis of machine learning models based on sequential counterfactual actions. An example method may include receiving a data record including a plurality of fields. Each respective field of the plurality of fields may be associated with a respective feature of a plurality of features. A feature dependence matrix associated with a dependence between each feature of the plurality of features and every other feature of the plurality of features may be received. The data record may be inputted to a machine learning model to generate an initial output. At least one tuple may be determined based on the data record and the feature dependence matrix. Each tuple of the at least one tuple may include an action vector and a sequence vector. Each action vector may be associated with a plurality of perturbations of respective fields of the plurality of fields for which inputting a perturbed data record based on the data record and the plurality of perturbations to the machine learning model generates a perturbed output different than the initial output. Each sequence vector may be associated with a sequence of the plurality of perturbations for the action vector. A sequence loss may be determined for each tuple of the at least one tuple. At least one communication may be communicated based on the at least one tuple and the sequence loss for each tuple of the at least one tuple.

[0007] In some non-limiting embodiments or aspects, receiving the feature dependence matrix may include receiving data associated with domain knowledge of the dependence between each feature of the plurality of features and every other feature of the plurality of features and / or determining the feature dependence matrix based on the data associated with domain knowledge.

[0008] In some non-limiting embodiments or aspects, receiving the feature dependence matrix may include extracting the feature dependence matrix based on the machine learning model.

[0009] In some non-limiting embodiments or aspects, extracting the feature dependence matrix may include determining the feature dependence matrix based on the machine learning model and a Shapley value.

[0010] In some non-limiting embodiments or aspects, determining the at least one tuple may include determining the at least one tuple based on the data record and the feature dependence matrix using mixed integer programming to solve a bi-objective framework associated with the action vector and the sequence vector.

[0011] In some non-limiting embodiments or aspects, the at least one tuple may include a plurality of tuples. The method may further include ranking the plurality of tuples based on the sequence loss for each tuple.

[0012] In some non-limiting embodiments or aspects, communicating the at least one communication may include selecting one or more tuples from the plurality of tuples based on ranking the plurality of tuples and / or generating the communication based on the one or more tuples.

[0013] In some non-limiting embodiments or aspects, communicating the at least one communication may include determining an explanation associated with the machine learning model based on the at least one tuple. The at least one communication may include the explanation.

[0014] According to non-limiting embodiments or aspects, provided is a system for analysis of machine learning models based on sequential counterfactual actions. An example system may include at least one processor and at least one non-transitory computer readable medium including one or more instructions that, when executed by the at least one processor, cause the at least one processor to receive a data record including a plurality of fields. Each respective field of the plurality of fields may be associated with a respective feature of a plurality of features. A feature dependence matrix associated with a dependence between each feature of the plurality of features and every other feature of the plurality of features may be received. The data record may be inputted to a machine learning model to generate an initial output. At least one tuple may be determined based on the data record and the feature dependence matrix. Each tuple of the at least one tuple may include an action vector and a sequence vector. Each action vector may be associated with a plurality ofperturbations of respective fields of the plurality of fields for which inputting a perturbed data record based on the data record and the plurality of perturbations to the machine learning model generates a perturbed output different than the initial output. Each sequence vector may be associated with a sequence of the plurality of perturbations for the action vector. A sequence loss may be determined for each tuple of the at least one tuple. At least one communication may be communicated based on the at least one tuple and the sequence loss for each tuple of the at least one tuple.

[0015] In some non-limiting embodiments or aspects, receiving the feature dependence matrix may include receiving data associated with domain knowledge of the dependence between each feature of the plurality of features and every other feature of the plurality of features and / or determining the feature dependence matrix based on the data associated with domain knowledge.

[0016] In some non-limiting embodiments or aspects, receiving the feature dependence matrix may include extracting the feature dependence matrix based on the machine learning model.

[0017] In some non-limiting embodiments or aspects, extracting the feature dependence matrix may include determining the feature dependence matrix based on the machine learning model and a Shapley value.

[0018] In some non-limiting embodiments or aspects, determining the at least one tuple may include determining the at least one tuple based on the data record and the feature dependence matrix using mixed integer programming to solve a bi-objective framework associated with the action vector and the sequence vector.

[0019] In some non-limiting embodiments or aspects, the at least one tuple may include a plurality of tuples. The one or more instructions, when executed by the at least one processor, may further cause the at least one processor to rank the plurality of tuples based on the sequence loss for each tuple.

[0020] In some non-limiting embodiments or aspects, communicating the at least one communication may include selecting one or more tuples from the plurality of tuples based on ranking the plurality of tuples and / or generating the communication based on the one or more tuples.

[0021] In some non-limiting embodiments or aspects, communicating the at least one communication may include determining an explanation associated with the machine learning model based on the at least one tuple. The at least one communication may include the explanation.

[0022] According to non-limiting embodiments or aspects, provided is a computer program product for analysis of machine learning models based on sequential counterfactual actions. An example computer program product may include at least one non-transitory computer-readable medium comprising one or more instructions that, when executed by at least one processor, cause the at least one processor to receive a data record comprising a plurality of fields. Each respective field of the plurality of fields may be associated with a respective feature of a plurality of features. A feature dependence matrix associated with a dependence between each feature of the plurality of features and every other feature of the plurality of features may be received. The data record may be inputted to a machine learning model to generate an initial output. At least one tuple may be determined based on the data record and the feature dependence matrix. Each tuple of the at least one tuple may include an action vector and a sequence vector. Each action vector may be associated with a plurality of perturbations of respective fields of the plurality of fields for which inputting a perturbed data record based on the data record and the plurality of perturbations to the machine learning model generates a perturbed output different than the initial output. Each sequence vector may be associated with a sequence of the plurality of perturbations for the action vector. A sequence loss may be determined for each tuple of the at least one tuple. At least one communication may be communicated based on the at least one tuple and the sequence loss for each tuple of the at least one tuple.

[0023] In some non-limiting embodiments or aspects, receiving the feature dependence matrix may include extracting the feature dependence matrix based on the machine learning model. Extracting the feature dependence matrix may include determining the feature dependence matrix based on the machine learning model and a Shapley value.

[0024] In some non-limiting embodiments or aspects, determining the at least one tuple may include determining the at least one tuple based on the data record and the feature dependence matrix using mixed integer programming to solve a bi-objective framework associated with the action vector and the sequence vector.

[0025] In some non-limiting embodiments or aspects, the at least one tuple may include a plurality of tuples. The one or more instructions, when executed by the at least one processor, may further cause the at least one processor to rank the plurality of tuples based on the sequence loss for each tuple. Communicating the at least one communication may include selecting one or more tuples from the plurality of tuplesbased on ranking the plurality of tuples and / or generating the communication based on the one or more tuples.

[0026] Further non-limiting embodiments or aspects will be set forth in the following numbered clauses:

[0027] Clause 1 : A computer-implemented method, comprising: receiving, with at least one processor, a data record comprising a plurality of fields, each respective field of the plurality of fields associated with a respective feature of a plurality of features; receiving, with at least one processor, a feature dependence matrix associated with a dependence between each feature of the plurality of features and every other feature of the plurality of features; inputting, with at least one processor, the data record to a machine learning model to generate an initial output; determining, with at least one processor, at least one tuple based on the data record and the feature dependence matrix, each tuple of the at least one tuple comprising an action vector and a sequence vector, each action vector associated with a plurality of perturbations of respective fields of the plurality of fields for which inputting a perturbed data record based on the data record and the plurality of perturbations to the machine learning model generates a perturbed output different than the initial output, each sequence vector associated with a sequence of the plurality of perturbations for the action vector; determining, with at least one processor, a sequence loss for each tuple of the at least one tuple; and communicating, with at least one processor, at least one communication based on the at least one tuple and the sequence loss for each tuple of the at least one tuple.

[0028] Clause 2: The method of clause 1 , wherein receiving the feature dependence matrix comprises: receiving data associated with domain knowledge of the dependence between each feature of the plurality of features and every other feature of the plurality of features; and determining the feature dependence matrix based on the data associated with domain knowledge.

[0029] Clause 3: The method of clause 1 or clause 2, wherein receiving the feature dependence matrix comprises extracting the feature dependence matrix based on the machine learning model.

[0030] Clause 4: The method of any of clauses 1 -3, wherein extracting the feature dependence matrix comprises determining the feature dependence matrix based on the machine learning model and a Shapley value.

[0031] Clause 5: The method of any of clauses 1 -4, wherein determining the at least one tuple comprises determining the at least one tuple based on the data recordand the feature dependence matrix using mixed integer programming to solve a biobjective framework associated with the action vector and the sequence vector.

[0032] Clause 6: The method of any of clauses 1 -5, wherein the at least one tuple comprises a plurality of tuples, the method further comprising: ranking, with at least one processor, the plurality of tuples based on the sequence loss for each tuple.

[0033] Clause 7: The method of any of clauses 1 -6, wherein communicating the at least one communication comprises: selecting one or more tuples from the plurality of tuples based on ranking the plurality of tuples; and generating the communication based on the one or more tuples.

[0034] Clause 8: The method of any of clauses 1 -7, wherein communicating the at least one communication comprises: determining an explanation associated with the machine learning model based on the at least one tuple, wherein the at least one communication comprises the explanation.

[0035] Clause 9: A system, comprising: at least one processor; and at least one non-transitory computer readable medium comprising one or more instructions that, when executed by the at least one processor, cause the at least one processor to: receive a data record comprising a plurality of fields, each respective field of the plurality of fields associated with a respective feature of a plurality of features; receive a feature dependence matrix associated with a dependence between each feature of the plurality of features and every other feature of the plurality of features; input the data record to a machine learning model to generate an initial output; determine at least one tuple based on the data record and the feature dependence matrix, each tuple of the at least one tuple comprising an action vector and a sequence vector, each action vector associated with a plurality of perturbations of respective fields of the plurality of fields for which inputting a perturbed data record based on the data record and the plurality of perturbations to the machine learning model generates a perturbed output different than the initial output, each sequence vector associated with a sequence of the plurality of perturbations for the action vector; determine a sequence loss for each tuple of the at least one tuple; and communicate at least one communication based on the at least one tuple and the sequence loss for each tuple of the at least one tuple.

[0036] Clause 10: The system of clause 9, wherein receiving the feature dependence matrix comprises: receiving data associated with domain knowledge of the dependence between each feature of the plurality of features and every otherfeature of the plurality of features; and determining the feature dependence matrix based on the data associated with domain knowledge.

[0037] Clause 1 1 : The system of clause 9 or clause 10, wherein receiving the feature dependence matrix comprises extracting the feature dependence matrix based on the machine learning model.

[0038] Clause 12: The system of any of clauses 9-1 1 , wherein extracting the feature dependence matrix comprises determining the feature dependence matrix based on the machine learning model and a Shapley value.

[0039] Clause 13: The system of any of clauses 9-12, wherein determining the at least one tuple comprises determining the at least one tuple based on the data record and the feature dependence matrix using mixed integer programming to solve a biobjective framework associated with the action vector and the sequence vector.

[0040] Clause 14: The system of any of clauses 9-13, wherein the at least one tuple comprises a plurality of tuples, wherein the one or more instructions, when executed by the at least one processor, further cause the at least one processor to: rank the plurality of tuples based on the sequence loss for each tuple.

[0041] Clause 15: The system of any of clauses 9-14, wherein communicating the at least one communication comprises: selecting one or more tuples from the plurality of tuples based on ranking the plurality of tuples; and generating the communication based on the one or more tuples.

[0042] Clause 16: The system of any of clauses 9-15, wherein communicating the at least one communication comprises: determining an explanation associated with the machine learning model based on the at least one tuple, wherein the at least one communication comprises the explanation.

[0043] Clause 17: A computer program product, the computer program product comprising at least one non-transitory computer-readable medium comprising one or more instructions that, when executed by at least one processor, cause the at least one processor to: receive a data record comprising a plurality of fields, each respective field of the plurality of fields associated with a respective feature of a plurality of features; receive a feature dependence matrix associated with a dependence between each feature of the plurality of features and every other feature of the plurality of features; input the data record to a machine learning model to generate an initial output; determine at least one tuple based on the data record and the feature dependence matrix, each tuple of the at least one tuple comprising an action vectorand a sequence vector, each action vector associated with a plurality of perturbations of respective fields of the plurality of fields for which inputting a perturbed data record based on the data record and the plurality of perturbations to the machine learning model generates a perturbed output different than the initial output, each sequence vector associated with a sequence of the plurality of perturbations for the action vector; determine a sequence loss for each tuple of the at least one tuple; and communicate at least one communication based on the at least one tuple and the sequence loss for each tuple of the at least one tuple.

[0044] Clause 18: The computer program product of clause 17, wherein receiving the feature dependence matrix comprises extracting the feature dependence matrix based on the machine learning model, wherein extracting the feature dependence matrix comprises determining the feature dependence matrix based on the machine learning model and a Shapley value.

[0045] Clause 19: The computer program product of clause 17 or clause 18, wherein determining the at least one tuple comprises determining the at least one tuple based on the data record and the feature dependence matrix using mixed integer programming to solve a bi-objective framework associated with the action vector and the sequence vector.

[0046] Clause 20: The computer program product of any of clauses 17-19, wherein the at least one tuple comprises a plurality of tuples, wherein the one or more instructions, when executed by the at least one processor, further cause the at least one processor to: rank the plurality of tuples based on the sequence loss for each tuple, wherein communicating the at least one communication comprises: selecting one or more tuples from the plurality of tuples based on ranking the plurality of tuples; and generating the communication based on the one or more tuples.

[0047] These and other features and characteristics of the present disclosure, as well as the methods of operation and functions of the related elements of structures and the combination of parts and economies of manufacture, will become more apparent upon consideration of the following description and the appended claims with reference to the accompanying drawings, all of which form a part of this specification, wherein like reference numerals designate corresponding parts in the various figures. It is to be expressly understood, however, that the drawings are for the purpose of illustration and description only and are not intended as a definition of the limits of the disclosed subject matter.BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Additional advantages and details are explained in greater detail below with reference to the non-limiting, exemplary embodiments that are illustrated in the accompanying schematic figures, in which:

[0049] FIG. 1 is a schematic diagram of an example system for analysis of machine learning models based on sequential counterfactual actions, according to some nonlimiting embodiments or aspects;

[0050] FIG. 2 is a flow diagram for an example process for analysis of machine learning models based on sequential counterfactual actions, according to some nonlimiting embodiments or aspects;

[0051] FIG. 3A is a diagram of a feature dependence matrix according to an example implementation of a process for analysis of machine learning models based on sequential counterfactual actions, according to some non-limiting embodiments or aspects;

[0052] FIG. 3B is a diagram of a sequence of actions according to an example implementation of a process for analysis of machine learning models based on sequential counterfactual actions, according to some non-limiting embodiments or aspects;

[0053] FIG. 4 is a diagram of an algorithm according to an example implementation of a process for analysis of machine learning models based on sequential counterfactual actions, according to some non-limiting embodiments or aspects;

[0054] FIG. 5 is a diagram of an example environment in which methods, systems, and / or computer program products, described herein, may be implemented, according to some non-limiting embodiments or aspects;

[0055] FIG. 6 is a schematic diagram of example components of a device used in connection with some non-limiting embodiments or aspects.DESCRIPTION

[0056] For purposes of the description hereinafter, the terms “end,” “upper,” “lower,” “right,” “left,” “vertical,” “horizontal,” “top,” “bottom,” “lateral,” “longitudinal,” and derivatives thereof shall relate to the embodiments as they are oriented in the drawing figures. However, it is to be understood that the present disclosure may assume various alternative variations and step sequences, except where expressly specifiedto the contrary. It is also to be understood that the specific devices and processes illustrated in the attached drawings, and described in the following specification, are simply exemplary and non-limiting embodiments or aspects of the disclosed subject matter. Hence, specific dimensions and other physical characteristics related to the embodiments or aspects disclosed herein are not to be considered as limiting.

[0057] Some non-limiting embodiments or aspects are described herein in connection with thresholds. As used herein, satisfying a threshold may refer to a value being greater than the threshold, more than the threshold, higher than the threshold, greater than or equal to the threshold, less than the threshold, fewer than the threshold, lower than the threshold, less than or equal to the threshold, equal to the threshold, etc.

[0058] No aspect, component, element, structure, act, step, function, instruction, and / or the like used herein should be construed as critical or essential unless explicitly described as such. Also, as used herein, the articles “a” and “an” are intended to include one or more items and may be used interchangeably with “one or more” and “at least one.” Furthermore, as used herein, the term “set” is intended to include one or more items (e.g., related items, unrelated items, a combination of related and unrelated items, and / or the like) and may be used interchangeably with “one or more” or “at least one.” Where only one item is intended, the term “one” or similar language is used. Also, as used herein, the terms “has,” “have,” “having,” or the like are intended to be open-ended terms. Further, the phrase “based on” is intended to mean “based at least partially on” unless explicitly stated otherwise. In addition, reference to an action being “based on” a condition may refer to the action being “in response to” the condition. For example, the phrases “based on” and “in response to” may, in some non-limiting embodiments or aspects, refer to a condition for automatically triggering an action (e.g., a specific operation of an electronic device, such as a computing device, a processor, and / or the like).

[0059] As used herein, the term “acquirer institution” may refer to an entity licensed and / or approved by a transaction service provider to originate transactions (e.g., payment transactions) using a payment device associated with the transaction service provider. The transactions the acquirer institution may originate may include payment transactions (e.g., purchases, original credit transactions (OCTs), account funding transactions (AFTs), and / or the like). In some non-limiting embodiments or aspects, an acquirer institution may be a financial institution, such as a bank. As used herein,the term “acquirer system” may refer to one or more computing devices operated by or on behalf of an acquirer institution, such as a server computer executing one or more software applications.

[0060] As used herein, the term “account identifier” may include one or more PANs, tokens, or other identifiers associated with a customer account. The term “token” may refer to an identifier that is used as a substitute or replacement identifier for an original account identifier, such as a PAN. Account identifiers may be alphanumeric or any combination of characters and / or symbols. Tokens may be associated with a PAN or other original account identifier in one or more data structures (e.g., one or more databases, and / or the like) such that they may be used to conduct a transaction without directly using the original account identifier. In some examples, an original account identifier, such as a PAN, may be associated with a plurality of tokens for different individuals or purposes.

[0061] As used herein, the term “communication” may refer to the reception, receipt, transmission, transfer, provision, and / or the like of data (e.g., information, signals, messages, instructions, commands, and / or the like). For one unit (e.g., a device, a system, a component of a device or system, combinations thereof, and / or the like) to be in communication with another unit means that the one unit is able to directly or indirectly receive information from and / or transmit information to the other unit. This may refer to a direct or indirect connection (e.g., a direct communication connection, an indirect communication connection, and / or the like) that is wired and / or wireless in nature. Additionally, two units may be in communication with each other even though the information transmitted may be modified, processed, relayed, and / or routed between the first and second unit. For example, a first unit may be in communication with a second unit even though the first unit passively receives information and does not actively transmit information to the second unit. As another example, a first unit may be in communication with a second unit if at least one intermediary unit processes information received from the first unit and communicates the processed information to the second unit. In some non-limiting embodiments or aspects, a message may refer to a network packet (e.g., a data packet and / or the like) that includes data. It will be appreciated that numerous other arrangements are possible.

[0062] As used herein, the term “computing device” may refer to one or more electronic devices configured to process data. A computing device may, in someexamples, include the necessary components to receive, process, and output data, such as a processor, a display, a memory, an input device, a network interface, and / or the like. A computing device may be a mobile device. As an example, a mobile device may include a cellular phone (e.g., a smartphone or standard cellular phone), a portable computer, a wearable device (e.g., watches, glasses, lenses, clothing, and / or the like), a personal digital assistant (PDA), and / or other like devices. A computing device may also be a desktop computer or other form of non-mobile computer.

[0063] As used herein, the terms “electronic wallet” and “electronic wallet application” refer to one or more electronic devices and / or software applications configured to initiate and / or conduct payment transactions. For example, an electronic wallet may include a mobile device executing an electronic wallet application, and may further include server-side software and / or databases for maintaining and providing transaction data to the mobile device. An “electronic wallet provider” may include an entity that provides and / or maintains an electronic wallet for a customer, such as Google Pay®, Android Pay®, Apple Pay®, Samsung Pay®, and / or other like electronic payment systems. In some non-limiting examples, an issuer bank may be an electronic wallet provider.

[0064] As used herein, the term “issuer institution” may refer to one or more entities, such as a bank, that provide accounts to customers for conducting transactions (e.g., payment transactions), such as initiating credit and / or debit payments. For example, an issuer institution may provide an account identifier, such as a primary account number (PAN), to a customer that uniquely identifies one or more accounts associated with that customer. The account identifier may be embodied on a portable financial device, such as a physical financial instrument, e.g., a payment card, and / or may be electronic and used for electronic payments. The term “issuer system” refers to one or more computer devices operated by or on behalf of an issuer institution, such as a server computer executing one or more software applications. For example, an issuer system may include one or more authorization servers for authorizing a transaction.

[0065] As used herein, the term “merchant” may refer to an individual or entity that provides goods and / or services, or access to goods and / or services, to customers based on a transaction, such as a payment transaction. The term “merchant” or “merchant system” may also refer to one or more computer systems operated by oron behalf of a merchant, such as a server computer executing one or more software applications.

[0066] As used herein, a “point-of-sale (POS) device” may refer to one or more devices, which may be used by a merchant to conduct a transaction (e.g., a payment transaction) and / or process a transaction. For example, a POS device may include one or more client devices. Additionally or alternatively, a POS device may include peripheral devices, card readers, scanning devices (e.g., code scanners), Bluetooth® communication receivers, near-field communication (NFC) receivers, radio frequency identification (RFID) receivers, and / or other contactless transceivers or receivers, contact-based receivers, payment terminals, and / or the like. As used herein, a “point- of-sale (POS) system” may refer to one or more client devices and / or peripheral devices used by a merchant to conduct a transaction. For example, a POS system may include one or more POS devices and / or other like devices that may be used to conduct a payment transaction. In some non-limiting embodiments or aspects, a POS system (e.g., a merchant POS system) may include one or more server computers configured to process online payment transactions through webpages, mobile applications, and / or the like.

[0067] As used herein, the terms “client” and / or “client device” may refer to one or more client-side devices or systems (e.g., remote from a transaction service provider system) used to initiate or facilitate a transaction (e.g., a payment transaction). As an example, a “client device” may refer to one or more POS devices used by a merchant, one or more acquirer host computers used by an acquirer, one or more mobile devices used by a user, and / or the like. In some non-limiting embodiments or aspects, a client device may be an electronic device configured to communicate with one or more networks and initiate or facilitate transactions. For example, a client device may include one or more computers, portable computers, laptop computers, tablet computers, mobile devices, cellular phones, wearable devices (e.g., watches, glasses, lenses, clothing, and / or the like), PDAs, and / or the like. Moreover, a “client” may also refer to an entity (e.g., a merchant, an acquirer, and / or the like) that owns, utilizes, and / or operates a client device for initiating transactions (e.g., for initiating transactions with a transaction service provider).

[0068] As used herein, the term “payment device” may refer to a payment card (e.g., a credit or debit card), a gift card, a smartcard, smart media, a payroll card, a healthcare card, a wristband, a machine-readable medium containing accountinformation, a keychain device or fob, an RFID transponder, a retailer discount or loyalty card, a cellular phone, an electronic wallet mobile application, a personal digital assistant (PDA), a pager, a security card, a computing device, an access card, a wireless terminal, a transponder, and / or the like. In some non-limiting embodiments or aspects, the payment device may include volatile or non-volatile memory to store information (e.g., an account identifier, a name of the account holder, and / or the like).

[0069] As used herein, the term “payment gateway” may refer to an entity and / or a payment processing system operated by or on behalf of such an entity (e.g., a merchant service provider, a payment service provider, a payment facilitator, a payment facilitator that contracts with an acquirer, a payment aggregator, and / or the like), which provides payment services (e.g., transaction service provider payment services, payment processing services, and / or the like) to one or more merchants. The payment services may be associated with the use of portable financial devices managed by a transaction service provider. As used herein, the term “payment gateway system” may refer to one or more computer systems, computer devices, servers, groups of servers, and / or the like, operated by or on behalf of a payment gateway.

[0070] As used herein, the term “server” may refer to or include one or more computing devices that are operated by or facilitate communication and processing for multiple parties in a network environment, such as the Internet, although it will be appreciated that communication may be facilitated over one or more public or private network environments and that various other arrangements are possible. Further, multiple computing devices (e.g., servers, point-of-sale (POS) devices, mobile devices, etc.) directly or indirectly communicating in the network environment may constitute a “system.”

[0071] As used herein, the term “system” may refer to one or more computing devices or combinations of computing devices (e.g., processors, servers, client devices, software applications, components of such, and / or the like). Reference to “a device,” “a server,” “a processor,” and / or the like, as used herein, may refer to a previously-recited device, server, or processor that is recited as performing a previous step or function, a different device, server, or processor, and / or a combination of devices, servers, and / or processors. For example, as used in the specification and the claims, a first device, a first server, or a first processor that is recited as performing afirst step or a first function may refer to the same or different device, server, or processor recited as performing a second step or a second function.

[0072] As used herein, the term “transaction service provider” may refer to an entity that receives transaction authorization requests from merchants or other entities and provides guarantees of payment, in some cases through an agreement between the transaction service provider and an issuer institution. For example, a transaction service provider may include a payment network such as Visa® or any other entity that processes transactions. The term “transaction processing system” may refer to one or more computer systems operated by or on behalf of a transaction service provider, such as a transaction processing server executing one or more software applications. A transaction processing server may include one or more processors and, in some non-limiting embodiments or aspects, may be operated by or on behalf of a transaction service provider.

[0073] Non-limiting embodiments or aspects of the disclosed subject matter are directed to methods, systems, and computer program products for analysis of machine learning models, including, but not limited to, analysis of machine learning models based on sequential counterfactual actions. For example, non-limiting embodiments or aspects of the disclosed subject matter provide receiving a data record including a plurality of fields, each respective field associated with a respective feature of a plurality of features. A feature dependence matrix associated with a dependence between each feature and every other feature of the plurality of features may be received and / or determined. The data record may be inputted to a machine learning model to generate an initial output. At least one tuple may be determined based on the data record and the feature dependence matrix, and each tuple may include an action vector and a sequence vector. Each action vector may be associated with a plurality of perturbations (e.g., actions) of respective fields of the plurality of fields for which inputting a perturbed data record based on the data record and the plurality of perturbations to the machine learning model generates a perturbed output different than the initial output. Each sequence vector may be associated with a sequence of the plurality of perturbations for the action vector. A sequence loss may be determined for each tuple. At least one communication based on the tuple(s) and the sequence loss for each tuple may be communicated. As such, the disclosed subject matter allows for taking into account not just actions but also sequence information and feature dependency to improve generating counterfactual examples that are both validand actionable (e.g., as opposed to counterfactual examples that are not actually feasible). Also, by virtue of using the feature dependence matrix, the disclosed subject matter allows for taking into account asymmetrical feature dependence (e.g., the influence of a first feature on a second feature is not necessarily the same as the influence of the second feature on the first feature). Additionally, the disclosed subject matter allows for either determining the feature dependence matrix based on domain knowledge (e.g., of an expert in the domain) or automatically generating (e.g., extracting) the feature dependence matrix based on the machine learning model itself. For example, non-limiting embodiments or aspects of the disclosed subject matter provide determining the feature dependence matrix based on the machine learning model and a Shapley value. Such automated determination of feature dependence using the Shapley value (e.g., modified Shapley value estimation, as described herein) enables using the disclosed techniques even when expert domain knowledge is unavailable. In addition, non-limiting embodiments or aspects of the disclosed subject matter provide determining the tuple(s) based on solving a bi-objective framework associated with the action vector and the sequence vector (e.g., using mixed integer programming). By virtue of this the bi-objective optimization that takes into account both action and sequence, the disclosed subject matter achieves a better tradeoff between explanation validity and actionability. Moreover, non-limiting embodiments or aspects of the disclosed subject matter provide ranking plurality of tuples based on the sequence loss for each tuple and / or selecting the tuple(s) for the communication based on the ranking and / or sequence loss. Such use of the sequence loss and / or ranking based thereon enables comparison of different potential counterfactual examples to help identify which is the most relevant, thereby improving the accuracy and efficacy of explanations of machine learning model behavior. Furthermore, nonlimiting embodiments or aspects of the disclosed subject matter provide determining an explanation associated with the machine learning model based on the tuple(s) and / or including the explanation in the communication. As such, valid and actionable explanations may be provided (e.g., to a user, requester, and / or the like).

[0074] Referring now to FIG. 1 , depicted is an example system 100 for analysis of machine learning models based on sequential counterfactual actions, according to some non-limiting embodiments or aspects. For example, system 100 may include model analysis system 102, database 104, user device 106, and / or communication network 1 12.

[0075] Model analysis system 102 may include one or more devices capable of receiving information from and / or communicating information to database 104 and / or user device 106 (e.g., directly via wired or wireless communication connection, indirectly via communication network 112, and / or the like). For example, model analysis system 102 may include a computing device, such as a server, a group of servers, a desktop computer, a portable computer, a mobile device, and / or other like devices. In some non-limiting embodiments or aspects, model analysis system 102 may be in communication with a data storage device (e.g., database 104), which may be local or remote to model analysis system 102. In some non-limiting embodiments or aspects, model analysis system 102 may be capable of receiving information from, storing information in, communicating information to, or searching information stored in the data storage device (e.g., database 104).

[0076] Database 104 may include one or more devices capable of receiving information from and / or communicating information to model analysis system 102 and / or user device 106 (e.g., directly via wired or wireless communication connection, indirectly via communication network 1 12, and / or the like). For example, database 104 may include a computing device, such as a server, a group of servers, a desktop computer, a portable computer, a mobile device, and / or other like devices. In some non-limiting embodiments or aspects, database 104 may include a data storage device. In some non-limiting embodiments or aspects, database 104 may be capable of receiving information from, storing information in, communicating information to, or searching information stored in the data storage device. In some non-limiting embodiments or aspects, database 104 may be part of model analysis system 102 and / or part of the same system as model analysis system 102.

[0077] User device 106 may include one or more devices capable of receiving information from and / or communicating information to model analysis system 102 and / or database 104 (e.g., directly via wired or wireless communication connection, indirectly via communication network 112, and / or the like). For example, user device 106 may include a computing device, such as a mobile device, a portable computer, a desktop computer, and / or other like devices. Additionally or alternatively, each user device 106 may include a device capable of receiving information from and / or communicating information to other user devices 106 (e.g., directly via wired or wireless communication connection, indirectly via communication network 1 12, and / or the like). In some non-limiting embodiments or aspects, user device 106 may be partof model analysis system 102 and / or part of the same system as model analysis system 102. For example, model analysis system 102, database 104, and user device 106 may all be (and / or be part of) a single system and / or a single computing device.

[0078] Communication network 1 12 may include one or more wired and / or wireless networks. For example, communication network 1 12 may include a cellular network (e.g., a long-term evolution (LTE®) network, a third generation (5G) network, a fourth generation (4G) network, a fifth generation (5G) network, a code division multiple access (CDMA) network, and / or the like), a public land mobile network (PLMN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a telephone network (e.g., the public switched telephone network (PSTN)), a private network (e.g., a private network associated with a transaction service provider), an ad hoc network, an intranet, the Internet, a fiber optic-based network, a cloud computing network, and / or the like, and / or a combination of these or other types of networks.

[0079] The number and arrangement of systems and devices shown in FIG. 1 are provided as an example. There may be additional systems and / or devices, fewer systems and / or devices, different systems and / or devices, and / or differently arranged systems and / or devices than those shown in FIG. 1. Furthermore, two or more systems or devices shown in FIG. 1 may be implemented within a single system or device, or a single system or device shown in FIG. 1 may be implemented as multiple, distributed systems or devices. Additionally or alternatively, a set of systems (e.g., one or more systems) or a set of devices (e.g., one or more devices) of system 100 may perform one or more functions described as being performed by another set of systems or another set of devices of system 100.

[0080] Referring now to FIG. 2, shown is a process 200 for analysis of machine learning models based on sequential counterfactual actions, according to some nonlimiting embodiments or aspects. The steps shown in FIG. 2 are for example purposes only. It will be appreciated that additional, fewer, different, and / or a different order of steps may be used in non-limiting embodiments or aspects. In some non-limiting embodiments or aspects, a step may be automatically performed in response to performance and / or completion of a prior step. In some non-limiting embodiments or aspects, one or more of the steps of process 200 may be performed (e.g., completely, partially, and / or the like) by model analysis system 102 (e.g., one or more devices of model analysis system 102). In some non-limiting embodiments or aspects, one or more of the steps of process 200 may be performed (e.g., completely, partially, and / orthe like) by another system, another device, another group of systems, or another group of devices, separate from or including model analysis system 102, such as database 104 and / or user device 106.

[0081] As shown in FIG. 2, at step 202, process 200 may include receiving a data record. For example, model analysis system 102 may receive (and / or retrieve) a data record (e.g., from database 104 and / or user device 106).

[0082] In some non-limiting embodiments or aspects, the data record may include a plurality of fields. For example, each respective field of the plurality of fields may be associated with a respective feature of a plurality of features.

[0083] In some non-limiting embodiments or aspects, model analysis system 102 may receive (and / or retrieve) a machine learning model (e.g., from database 104 and / or user device 106).

[0084] As shown in FIG. 2, at step 204, process 200 may include receiving and / or determining a feature dependence matrix. For example, model analysis system 102 may receive and / or determine the feature dependence matrix. In some non-limiting embodiments or aspects, the feature dependence matrix may be associated with a dependence between each feature of the plurality of features and every other feature of the plurality of features.

[0085] In some non-limiting embodiments or aspects, model analysis system 102 may receive (e.g., from user device 106 and / or database 104) data associated with domain knowledge of the dependence between each feature of the plurality of features and every other feature of the plurality of features. For example, an expert in the domain may provide, generate, and / or store (e.g., using user device 106 and / or database 104) such data associated with domain knowledge. In some non-limiting embodiments or aspects, model analysis system 102 may determine the feature dependence matrix based on the data associated with domain knowledge. In some non-limiting embodiments or aspects, the expert may provide (e.g., using user device 106 and / or database 104) the feature dependence matrix.

[0086] In some non-limiting embodiments or aspects, model analysis system 102 may extract the feature dependence matrix based on the machine learning model (e.g., extract the feature dependence matrix from the machine learning model itself).

[0087] In some non-limiting embodiments or aspects, model analysis system 102 may extract the feature dependence matrix by determining the feature dependencematrix based on the machine learning model and a Shapley value (e.g., a modified Shapley value, as described herein).

[0088] For the purpose of illustration, referring now to FIG. 3A and with continued reference to FIG. 2, shown is a diagram of a feature dependence matrix according to an example implementation of a process for analysis of machine learning models based on sequential counterfactual actions, according to some non-limiting embodiments or aspects. In some non-limiting embodiments or aspects, one or more of the steps of the implementation may be performed (e.g., completely, partially, and / or the like) by model analysis system 102 (e.g., one or more devices of model analysis system 102). In some non-limiting embodiments or aspects, one or more of the steps of the implementation may be performed (e.g., completely, partially, and / or the like) by another system, another device, another group of systems, or another group of devices, separate from or including model analysis system 102, such as database 104 and / or user device 106.

[0089] As shown in FIG. 3A, the feature dependence matrix may include a row for each feature and a column for each feature. As such, along the diagonal of the feature dependence matrix, values are all equal to 1 (e.g., the influence of a feature on itself is 1 ). The other values in the matrix are between 0 and 1 , where 0 would denote no influence and 1 would denote complete influence. In some non-limiting embodiments or aspects, the dependency (e.g., influence of one feature on another) may be asymmetrical. For example, the influence of the first feature one a third feature may be 0 (e.g., row 1 , column 3), while the influence of the third feature on the first feature may be 0.8 (e.g., row 3, column 1 ).

[0090] With continued reference to FIG. 2, in some non-limiting embodiments or aspects, the feature dependence matrix may be part of a bi-objective framework associated with an action vector and a sequence vector, as described herein. For example, a bi-objective framework may be expressed by the following equation: Equation 1where a and s are the action vector and sequence vector, respectively (e.g., for a counterfactual explanation), a*, s* are the optimal action vector and sequence vector, A is the set of all possible actions, S is the set of all possible sequences, respectively (e.g., the solution to the equation), yo is the initial output of the machine learning model f based on the data record x, y* is the perturbed output of the machine learning model based on the action vector a (e.g., perturbations) and the data record x (e.g., combined to form a perturbed input (e.g., counterfactual instance) to the machine learning model f), d is the maximum sequence length (e.g., the total number of features for perturbation), M is the feature dependence matrix, Lds is the distance loss between the data record x and perturbed input (e.g., x+a), and Lsqis the sequence loss for the sequence of actions in order (e.g., in the order according to the sequence vector).

[0091] In some non-limiting embodiments or aspects, since feature dependency may be asymmetrical, Shapley values may be used to extract the feature dependence matrix from the machine learning model. In some non-limiting embodiments or aspects, the Shapley values may be modified to a conditional bivariate form. For example, this may be expressed by the following equation:

[0092] The i-th column of M may be calculated as E(vi) by defining its value functionVi as set forth in the following equation:Equation 3 v(B) if i e BVt : PdIR, where V[(B) , WB e Pd. 0, if i B where B indicates a subset of the power set Pdon the whole feature space. In some non-limiting embodiments or aspects, E(vi) may determine the feature dependence conditioned on the presence of feature i.

[0093] In some non-limiting embodiments or aspects, the sequence loss Lsqmay be determined based on the action vector a, the sequence vector s, and the feature dependence matrix M. For example, assuming linear dependence among features, the sequence loss Lsqmay be expressed based on the following equation: Equation 4SUBSTITUTE SHEET (RULE 26)

[0094] In some non-limiting embodiments or aspects, actions may be expressed based on the following equation:

[0095] In some non-limiting embodiments or aspects, according to the induction, the following equation may be derived: Equation 6

[0096] In some non-limiting embodiments or aspects, based on the above, the sequence loss Lsq may be expressed based on the following equation:Equation 7where e7- is a non-negative scaling coefficient for an action (e.g., perturbation) at step

[0097] As shown in FIG. 2, at step 206, process 200 may include inputting the data record to a machine learning model to generate an initial output. For example, model analysis system 102 may input the data record to the machine learning model to generate an initial output.

[0098] As shown in FIG. 2, at step 208, process 200 may include determining at least one tuple (e.g., including an action vector and sequence vector). For example,23SUBSTITUTE SHEET (RULE 26)model analysis system 102 may determine at least one tuple based on the data record and the feature dependence matrix.

[0099] In some non-limiting embodiments or aspects, each tuple may include an action vector and a sequence vector. Each action vector may be associated with a plurality of perturbations of respective fields of the plurality of fields (e.g., a plurality of actions) for which inputting a perturbed data record based on the data record and the perturbations to the machine learning model generates a perturbed output different than the initial output (e.g., changes the output of the model). Each sequence vector may be associated with a sequence of the perturbations for the action vector.

[0100] For the purpose of illustration, referring now to FIG. 3B and with continued reference to FIG. 2, shown is a diagram of a sequence of actions according to an example implementation of a process for analysis of machine learning models based on sequential counterfactual actions, according to some non-limiting embodiments or aspects. In some non-limiting embodiments or aspects, one or more of the steps of the implementation may be performed (e.g., completely, partially, and / or the like) by model analysis system 102 (e.g., one or more devices of model analysis system 102). In some non-limiting embodiments or aspects, one or more of the steps of the implementation may be performed (e.g., completely, partially, and / or the like) by another system, another device, another group of systems, or another group of devices, separate from or including model analysis system 102, such as database 104 and / or user device 106.

[0101] As shown in FIG. 3B, data record 301 may include a plurality of fields. For example, data record 301 may include five fields: age, workclass (e.g., type of employment, such as private, self-employment, or public), education (e.g., level of highest degree earned), hours (e.g., hours worked per week), and marriage (e.g., marital status). A first sequence vector 321 may indicate 0 for every field thereof, e.g., to denote that no perturbations (e.g., actions) have been taken.

[0102] The fields of data record 301 may be perturbed to generate one or more perturbed data records. For example, a first field of data record 301 (e.g., age) may be perturbed (e.g., value changed from 22 to 27) to generate first perturbed data record 31 1. This perturbation may be the first action in a sequence. As such, sequence vector 332 may include a value of 1 in a first field thereof (e.g., indicating that the first field was perturbed first).

[0103] Another field of first perturbed data record 31 1 may be perturbed to generate another perturbed data record. For example, a third field of perturbed data record 31 1 (e.g., education) may be perturbed (e.g., value changed from high school graduate (HS-Grad) to college graduate (College)) to generate second perturbed data record 312. This perturbation may be the second action in the sequence. As such, sequence vector 333 may include a value of 2 in a third field thereof (e.g., indicating that the third field was perturbed second).

[0104] Another field of second perturbed data record 312 may be perturbed to generate another perturbed data record. For example, a fifth field of second perturbed data record 312 (e.g., marriage) may be perturbed (e.g., value changed from single to married) to generate third perturbed data record 313. This perturbation may be the third action in the sequence. As such, sequence vector 334 may include a value of 3 in a fifth field thereof (e.g., indicating that the fifth field was perturbed third). The output generated by the machine learning model based on the third perturbed data record 313 as input may be different than the output generated by the machine learning model based on the initial data record 301 (e.g., the model may have flipped its output, as indicated by the checkmark in the Prediction column associated with Model Flip).

[0105] With continued reference to FIG. 2, determining the tuple(s) may include determining the tuple(s) based on the data record and the feature dependence matrix using mixed integer programming (MIP) to solve a bi-objective framework associated with the action vector and the sequence vector. For example, the bi-objective framework may be expressed by Equation 1 , as described herein. In some nonlimiting embodiments or aspects, MIP may include initially solving the bi-objective framework equation (e.g., Equation 1 ) without constraining variables to be integers to provide initial (e.g., non-integer) values for each variable. The values for the variables may be rounded down to the nearest integer, and a solution value may be determined based on those rounded-down integer values for the variables. Upper and lower bounds may be determined based on solution values using variable values without and with constraining the variables to be integers (e.g., the upper bound may be the initial solution value when the variables were not constrained, as described above, and the lower bound may be the solution value when the variables were rounded down, as described above). Then, starting with whichever variable has a greater fractional part from the initial solution with the variable values unconstrained (e.g., not constrained to integers), two branches may be created, with a first branch being basedon that variable being rounded down and a second branch being based on that variable being rounded up. In each branch, the solution to the bi-objective framework equation may be determined with other variable unconstrained to provide a first solution value for that other variable, and then the other variable may be rounded down to the nearest integer to provide a second solution value. These solution values may be used as the upper and lower bounds of each branch. Two new branches may be created from whichever current branch has the greatest upper bound, and the process may repeat for each of the two new branches until at least one stopping criterion is satisfied. For example, the stopping criteria may be when the upper and lower bounds of a branch are equal and both variables have integer values. As such, by satisfying the stopping criteria, the solution based on MIP may have been obtained.

[0106] As shown in FIG. 2, at step 210, process 200 may include determining a sequence loss for each tuple. For example, model analysis system 102 may determine a sequence loss for each tuple of the at least one tuple.

[0107] In some non-limiting embodiments or aspects, the sequence loss may be determined based on the equations set forth herein.

[0108] In some non-limiting embodiments or aspects, there may be a plurality of tuples. In some non-limiting embodiments or aspects, model analysis system 102 may rank the plurality of tuples based on the sequence loss for each tuple (e.g., rank the tuples from lowest to highest sequence loss, from highest to lowest sequence loss, and / or the like).

[0109] As shown in FIG. 2, at step 212, process 200 may include communicating at least one communication. For example, model analysis system 102 may communicate at least one communication based on the at least one tuple and / or the sequence loss for each tuple of the at least one tuple.

[0110] In some non-limiting embodiments or aspects, communicating the at least one communication may include selecting one or more tuples from the plurality of tuples based on the ranking of the plurality of tuples. Additionally or alternatively, the communication may be generated based on the selection of the tuple(s).

[0111] In some non-limiting embodiments or aspects, communicating the at least one communication may include determining an explanation associated with the machine learning model based on the at least one tuple. For example, the explanation may be a message (e.g., a narrative) explaining which actions (e.g., perturbations) could be taken in which order (e.g., sequence) in order to cause the machine learningmodel to generate a different output (e.g., make a different decision). In some nonlimiting embodiments or aspects, the communication(s) may include the explanation.

[0112] In some non-limiting embodiments or aspects, the data record may be associated with a credit (e.g., credit card, loan, and / or the like) application. For example, the credit application may have been rejected, and the applicant of the credit application may desire to know why the credit application was rejected. In some nonlimiting embodiments or aspects, the communication (e.g., including the explanation based on the tuple(s)) may be communicated to the applicant of the rejected credit application. This communication may help the applicant of the rejected credit application understand what circumstances need to be changed in order to successfully have a credit application approved in the future.

[0113] In some non-limiting embodiments or aspects, the data record may be associated with a payment transaction. For example, the payment transaction may have been rejected and / or flagged as fraudulent, and a user (e.g., cardholder, fraud investigator, and / or the like) of the transaction may desire to know why the transaction was rejected and / or flagged as fraudulent. In some non-limiting embodiments or aspects, the communication (e.g., including the explanation based on the tuple(s)) may be communicated to the user. This communication may help the user understand why the payment transaction was rejected and / or flagged as fraudulent.

[0114] Referring now to FIG. 4, shown is a diagram of an algorithm 400 according to an example implementation of a process for analysis of machine learning models based on sequential counterfactual actions, according to some non-limiting embodiments or aspects. In some non-limiting embodiments or aspects, one or more of the steps of algorithm 400 may be performed (e.g., completely, partially, and / or the like) by model analysis system 102 (e.g., one or more devices of model analysis system 102). In some non-limiting embodiments or aspects, one or more of the steps of algorithm 400 may be performed (e.g., completely, partially, and / or the like) by another system, another device, another group of systems, or another group of devices, separate from or including model analysis system 102, such as database 104 and / or user device 106.

[0115] As shown in FIG. 4, the inputs for the algorithm may include a data record (e.g., query instance x), a machine learning model (e.g., target model f), a sequence length and / or a number of features of the data record (e.g., d), and, optionally, domain knowledge K.

[0116] As shown in FIG. 4, at line 1 , the availability of domain knowledge K is checked. At lines 2-4, if domain knowledge K is not available (e.g., K is NULL), then the feature dependence matrix M may be estimated, as described herein (e.g., based on the equations above, including Equation 3). At lines 5-6, if domain knowledge K is available, then the feature dependence matrix M may be determined based on domain knowledge, as described herein.

[0117] As shown in FIG. 4, at lines 7-8, the data record (e.g., query instance x) and feature dependence matrix M are inputted into a bi-objective framework (e.g., based on the equations above, including Equation 1 ), and the bi-objective framework may be solved (e.g., based on MIP) to determine (e.g., generate) one or more tuples, as described herein. Each tuple may include an action vector and a sequence vector, as described herein. At line 9, all tuples that have been determined (e.g., in the solution set S, which may be a set of all such tuples) may be ranked based on the sequence loss thereof, as described herein.

[0118] As shown in FIG. 4, at lines 10-1 1 , if no tuples have been determined (e.g., the solution set is NULL), then a communication may not be sent or a communication indicating that no tuples have been determined may be sent. At lines 12-13, a selected number (e.g., N) of the top ranked tuples (e.g., N tuples with the lowest sequence loss) may be communicated, as described herein. Additionally or alternatively, at line 14, the top-ranked tuple (e.g., the tuple with the lowest sequence loss) may be selected to determine (e.g., generate) an explanation, as described herein.

[0119] Referring now to FIG. 5, depicted is a diagram of a non-limiting embodiment or aspect of an exemplary environment 500 in which systems, products, and / or methods, as described herein, may be implemented. As shown in FIG. 5, environment 500 may include transaction service provider system 502, issuer system 504, customer device 506, merchant system 508, acquirer system 510, and communication network 512. In some non-limiting embodiments or aspects, each of model analysis system 102, database 104, and / or user device 106 of FIG. 1 may be implemented by (e.g., part of) transaction service provider system 502. In some non-limiting embodiments or aspects, at least one of model analysis system 102, database 104, and / or user device 106 of FIG. 1 may be implemented by (e.g., part of) another system, another device, another group of systems, or another group of devices, separate from or including transaction service provider system 502, such as issuer system 504, customer device 506, merchant system 508, acquirer system 510, and / or the like.

[0120] Transaction service provider system 502 may include one or more devices capable of receiving information from and / or communicating information to issuer system 504, customer device 506, merchant system 508, and / or acquirer system 510 via communication network 512. For example, transaction service provider system 502 may include a computing device, such as a server (e.g., a transaction processing server), a group of servers, and / or other like devices. In some non-limiting embodiments or aspects, transaction service provider system 502 may be associated with a transaction service provider as described herein. In some non-limiting embodiments or aspects, transaction service provider system 502 may be in communication with a data storage device, which may be local or remote to transaction service provider system 502. In some non-limiting embodiments or aspects, transaction service provider system 502 may be capable of receiving information from, storing information in, communicating information to, or searching information stored in the data storage device.

[0121] Issuer system 504 may include one or more devices capable of receiving information and / or communicating information to transaction service provider system 502, customer device 506, merchant system 508, and / or acquirer system 510 via communication network 512. For example, issuer system 504 may include a computing device, such as a server, a group of servers, and / or other like devices. In some non-limiting embodiments or aspects, issuer system 504 may be associated with an issuer institution as described herein. For example, issuer system 504 may be associated with an issuer institution that issued a credit account, debit account, credit card, debit card, and / or the like to a user associated with customer device 506.

[0122] Customer device 506 may include one or more devices capable of receiving information from and / or communicating information to transaction service provider system 502, issuer system 504, merchant system 508, and / or acquirer system 510 via communication network 512. Additionally or alternatively, each customer device 506 may include a device capable of receiving information from and / or communicating information to other customer devices 506 via communication network 512, another network (e.g., an ad hoc network, a local network, a private network, a virtual private network, and / or the like), and / or any other suitable communication technique. For example, customer device 506 may include a client device and / or the like. In some non-limiting embodiments or aspects, customer device 506 may or may not be capable of receiving information (e.g., from merchant system 508 or from another customerdevice 506) via a short-range wireless communication connection (e.g., an NFC communication connection, an RFID communication connection, a Bluetooth® communication connection, a Zigbee® communication connection, and / or the like), and / or communicating information (e.g., to merchant system 508) via a short-range wireless communication connection.

[0123] Merchant system 508 may include one or more devices capable of receiving information from and / or communicating information to transaction service provider system 502, issuer system 504, customer device 506, and / or acquirer system 510 via communication network 512. Merchant system 508 may also include a device capable of receiving information from customer device 506 via communication network 512, a communication connection (e.g., an NFC communication connection, an RFID communication connection, a Bluetooth® communication connection, a Zigbee® communication connection, and / or the like) with customer device 506, and / or the like, and / or communicating information to customer device 506 via communication network 512, the communication connection, and / or the like. In some non-limiting embodiments or aspects, merchant system 508 may include a computing device, such as a server, a group of servers, a client device, a group of client devices, and / or other like devices. In some non-limiting embodiments or aspects, merchant system 508 may be associated with a merchant as described herein. In some non-limiting embodiments or aspects, merchant system 508 may include one or more client devices. For example, merchant system 508 may include a client device that allows a merchant to communicate information to transaction service provider system 502. In some non-limiting embodiments or aspects, merchant system 508 may include one or more devices, such as computers, computer systems, and / or peripheral devices capable of being used by a merchant to conduct a transaction with a user. For example, merchant system 508 may include a POS device and / or a POS system.

[0124] Acquirer system 510 may include one or more devices capable of receiving information from and / or communicating information to transaction service provider system 502, issuer system 504, customer device 506, and / or merchant system 508 via communication network 512. For example, acquirer system 510 may include a computing device, a server, a group of servers, and / or the like. In some non-limiting embodiments or aspects, acquirer system 510 may be associated with an acquirer as described herein.

[0125] Communication network 512 may include one or more wired and / or wirelessnetworks. For example, communication network 512 may include a cellular network (e.g., a long-term evolution (LTE®) network, a third generation (5G) network, a fourth generation (4G) network, a fifth generation (5G) network, a code division multiple access (CDMA) network, and / or the like), a public land mobile network (PLMN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a telephone network (e.g., the public switched telephone network (PSTN)), a private network (e.g., a private network associated with a transaction service provider), an ad hoc network, an intranet, the Internet, a fiber optic-based network, a cloud computing network, and / or the like, and / or a combination of these or other types of networks.

[0126] For the purpose of illustration, processing a transaction (e.g., a payment transaction) may include generating a transaction message (e.g., authorization request and / or the like) based on an account identifier of a customer (e.g., accountholder associated with customer device 506 and / or the like) and / or transaction data associated with the transaction. For example, merchant system 508 (e.g., a client device of merchant system 508, a POS device of merchant system 508, and / or the like) may initiate the transaction, e.g., by generating an authorization request (e.g., in response to receiving the account identifier from a payment device and / or a portable financial device of the customer and / or the like). Merchant system 508 may communicate the authorization request to a payment gateway system and / or acquirer system 510. In some non-limiting embodiments or aspects, payment gateway system may communicate the authorization request to acquirer system 510 and / or transaction service provider system 502. Additionally or alternatively, acquirer system 510 (and / or the payment gateway system) may communicate the authorization request to transaction service provider system 502. After receiving the authorization request from merchant system 508 that identifies the account identifier of the customer (e.g., the accountholder associated with consumer device 506 and / or the account identifier), transaction service provider system 502 may communicate the authorization request to issuer system 504 (e.g., the issuer system that issued the payment device and / or account identifier). Issuer system 504 may determine an authorization decision (e.g., approve, deny, and / or the like) based on the authorization request, and / or issuer system 504 may generate an authorization response based on the authorization decision and / or the authorization request. Issuer system 504 may communicate the authorization response to transaction service provider system 502. Transaction service provider system 502 may communicate the authorization response to acquirersystem 510 and / or payment gateway system. In some non-limiting embodiments or aspects, acquirer system 510 may communicate the authorization response to the payment gateway system and / or merchant system 508. Additionally or alternatively, the payment gateway system (and / or acquirer system 510) may communicate the authorization response to merchant system 508.

[0127] In some non-limiting embodiments or aspects, transaction service provider system 502 and / or issuer system 504 may include at least one machine learning model (e.g., at least one of a fraud detection model, a risk detection model, a transaction authorization model, a credit approval model, a product recommendation model, a classifier model, an anomaly detection model, an authentication model, any combination thereof, and / or the like). For example, the machine learning model(s) may be trained, finetuned, and / or analyzed based on counterfactual instances, as described herein. Transaction service provider system 502 and / or issuer system 504 may perform at least one task (e.g., generate a prediction, generate a communication, and / or the like) based on the authorization request and the machine learning model(s). For example, performing the task(s) may include generating at least one prediction associated with fraud detection, risk detection, transaction authorization, credit approval, product recommendation, classification, anomaly detection, authentication, any combination thereof, and / or the like. In some non-limiting embodiments or aspects, transaction service provider system 502 may communicate at least one message based on performing the task (e.g., generating the prediction, generating the communication, and / or the like) to issuer system 504 (e.g., along with the authorization request). In some non-limiting embodiments or aspects, issuer system 504 may determine the authorization decision (e.g., approve, deny, and / or the like) based on the authorization request and the performance of the task (e.g., generation of the prediction and / or generation of the embedding).

[0128] For the purpose of illustration, clearing and / or settlement of a transaction may include generating a message (e.g., clearing message and / or the like) based on an account identifier of a customer (e.g., associated with customer device 506 and / or the like) and / or transaction data associated with the transaction. For example, merchant system 508 may generate at least one clearing message (e.g., a plurality of clearing messages, a batch of clearing messages, and / or the like). Merchant system 508 may communicate the clearing message(s) to acquirer system 510 (and / or the payment gateway system, which may communicate the clearing message(s) toacquirer system 510). Acquirer system 510 may communicate the clearing message(s) to transaction service provider system 502. Transaction service provider system 502 may communicate the clearing message(s) to issuer system 504. Issuer system 504 may generate at least one settlement message based on the clearing message(s). In some non-limiting embodiments or aspects, issuer system 504 may communicate the settlement message(s) and / or funds to transaction service provider system 502 (and / or a settlement bank system associated with transaction service provider system 502), and transaction service provider system 502 (and / or the settlement bank system) may communicate the settlement message(s) and / or funds to acquirer system 510. Additionally or alternatively, issuer system 504 may communicate the settlement message(s) and / or funds to acquirer system 504. In some non-limiting embodiments or aspects, acquirer system 504 may communicate settlement message(s) and / or funds to merchant system 508 (and / or an account associated with merchant system 508).

[0129] The number and arrangement of systems, devices, and / or networks shown in FIG. 5 are provided as an example. There may be additional systems, devices, and / or networks; fewer systems, devices, and / or networks; different systems, devices, and / or networks; and / or differently arranged systems, devices, and / or networks than those shown in FIG. 5. Furthermore, two or more systems or devices shown in FIG. 5 may be implemented within a single system or device, or a single system or device shown in FIG. 5 may be implemented as multiple, distributed systems or devices. Additionally or alternatively, a set of systems (e.g., one or more systems) or a set of devices (e.g., one or more devices) of environment 500 may perform one or more functions described as being performed by another set of systems or another set of devices of environment 500.

[0130] Referring now to FIG. 6, shown is a diagram of example components of a device 600 according to non-limiting embodiments or aspects. Device 600 may correspond to model analysis system 102, database 104, and / or user device 106 of FIG. 1 and / or transaction service provider system 502, issuer system 504, customer device 506, merchant system 508, acquirer system 510, and / or communication network 512 of FIG. 5, as examples. In some non-limiting embodiments or aspects, such systems or devices may include at least one device 600 and / or at least one component of device 600. The number and arrangement of components shown are provided as an example. In some non-limiting embodiments or aspects, device 600may include additional components, fewer components, different components, or differently arranged components than those shown in FIG. 6. Additionally or alternatively, a set of components (e.g., one or more components) of device 600 may perform one or more functions described as being performed by another set of components of device 600.

[0131] As shown in FIG. 6, device 600 may include a bus 602, a processor 604, memory 606, a storage component 608, an input component 610, an output component 612, and a communication interface 614. Bus 602 may include a component that permits communication among the components of device 600. In some non-limiting embodiments or aspects, processor 604 may be implemented in hardware, firmware, or a combination of hardware and software. For example, processor 604 may include a processor (e.g., a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), etc.), a microprocessor, a digital signal processor (DSP), and / or any processing component (e.g., a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), etc.) that can be programmed to perform a function. Memory 606 may include random access memory (RAM), read only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic memory, optical memory, etc.) that stores information and / or instructions for use by processor 604.

[0132] With continued reference to FIG. 6, storage component 608 may store information and / or software related to the operation and use of device 600. For example, storage component 608 may include a hard disk (e.g., a magnetic disk, an optical disk, a magneto-optic disk, a solid state disk, etc.) and / or another type of computer-readable medium. Input component 610 may include a component that permits device 600 to receive information, such as via user input (e.g., a touch screen display, a keyboard, a keypad, a mouse, a button, a switch, a microphone, etc.). Additionally or alternatively, input component 610 may include a sensor for sensing information (e.g., a global positioning system (GPS) component, an accelerometer, a gyroscope, an actuator, etc.). Output component 612 may include a component that provides output information from device 600 (e.g., a display, a speaker, one or more light-emitting diodes (LEDs), etc.). Communication interface 614 may include a transceiver-like component (e.g., a transceiver, a separate receiver and transmitter, etc.) that enables device 600 to communicate with other devices, such as via a wired connection, a wireless connection, or a combination of wired and wirelessconnections. Communication interface 614 may permit device 600 to receive information from another device and / or provide information to another device. For example, communication interface 614 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi® interface, a cellular network interface, and / or the like.

[0133] Device 600 may perform one or more processes described herein. Device 600 may perform these processes based on processor 604 executing software instructions stored by a computer-readable medium, such as memory 606 and / or storage component 608. A computer-readable medium may include any non- transitory memory device. A memory device includes memory space located inside of a single physical storage device or memory space spread across multiple physical storage devices. Software instructions may be read into memory 606 and / or storage component 608 from another computer-readable medium or from another device via communication interface 614. When executed, software instructions stored in memory 606 and / or storage component 608 may cause processor 604 to perform one or more processes described herein. Additionally, or alternatively, hardwired circuitry may be used in place of or in combination with software instructions to perform one or more processes described herein. Thus, embodiments described herein are not limited to any specific combination of hardware circuitry and software. The term “configured to,” as used herein, may refer to an arrangement of software, device(s), and / or hardware for performing and / or enabling one or more functions (e.g., actions, processes, steps of a process, and / or the like). For example, “a processor configured to” may refer to a processor that executes software instructions (e.g., program code) that cause the processor to perform one or more functions.

[0134] Although embodiments have been described in detail for the purpose of illustration, it is to be understood that such detail is solely for that purpose and that the disclosure is not limited to the disclosed embodiments or aspects, but, on the contrary, is intended to cover modifications and equivalent arrangements that are within the spirit and scope of the appended claims. For example, it is to be understood that the present disclosure contemplates that, to the extent possible, one or more features of any embodiment or aspect can be combined with one or more features of any other embodiment or aspect.

Claims

WHAT IS CLAIMED IS:1 . A computer-implemented method, comprising: receiving, with at least one processor, a data record comprising a plurality of fields, each respective field of the plurality of fields associated with a respective feature of a plurality of features; receiving, with at least one processor, a feature dependence matrix associated with a dependence between each feature of the plurality of features and every other feature of the plurality of features; inputting, with at least one processor, the data record to a machine learning model to generate an initial output; determining, with at least one processor, at least one tuple based on the data record and the feature dependence matrix, each tuple of the at least one tuple comprising an action vector and a sequence vector, each action vector associated with a plurality of perturbations of respective fields of the plurality of fields for which inputting a perturbed data record based on the data record and the plurality of perturbations to the machine learning model generates a perturbed output different than the initial output, each sequence vector associated with a sequence of the plurality of perturbations for the action vector; determining, with at least one processor, a sequence loss for each tuple of the at least one tuple; and communicating, with at least one processor, at least one communication based on the at least one tuple and the sequence loss for each tuple of the at least one tuple.

2. The method of claim 1 , wherein receiving the feature dependence matrix comprises: receiving data associated with domain knowledge of the dependence between each feature of the plurality of features and every other feature of the plurality of features; and determining the feature dependence matrix based on the data associated with domain knowledge.

3. The method of claim 1 , wherein receiving the feature dependence matrix comprises extracting the feature dependence matrix based on the machine learning model.

4. The method of claim 3, wherein extracting the feature dependence matrix comprises determining the feature dependence matrix based on the machine learning model and a Shapley value.

5. The method of claim 1 , wherein determining the at least one tuple comprises determining the at least one tuple based on the data record and the feature dependence matrix using mixed integer programming to solve a bi-objective framework associated with the action vector and the sequence vector.

6. The method of claim 1 , wherein the at least one tuple comprises a plurality of tuples, the method further comprising: ranking, with at least one processor, the plurality of tuples based on the sequence loss for each tuple.

7. The method of claim 6, wherein communicating the at least one communication comprises: selecting one or more tuples from the plurality of tuples based on ranking the plurality of tuples; and generating the communication based on the one or more tuples.

8. The method of claim 1 , wherein communicating the at least one communication comprises: determining an explanation associated with the machine learning model based on the at least one tuple, wherein the at least one communication comprises the explanation.

9. A system, comprising: at least one processor; andat least one non-transitory computer readable medium comprising one or more instructions that, when executed by the at least one processor, cause the at least one processor to: receive a data record comprising a plurality of fields, each respective field of the plurality of fields associated with a respective feature of a plurality of features; receive a feature dependence matrix associated with a dependence between each feature of the plurality of features and every other feature of the plurality of features; input the data record to a machine learning model to generate an initial output; determine at least one tuple based on the data record and the feature dependence matrix, each tuple of the at least one tuple comprising an action vector and a sequence vector, each action vector associated with a plurality of perturbations of respective fields of the plurality of fields for which inputting a perturbed data record based on the data record and the plurality of perturbations to the machine learning model generates a perturbed output different than the initial output, each sequence vector associated with a sequence of the plurality of perturbations for the action vector; determine a sequence loss for each tuple of the at least one tuple; and communicate at least one communication based on the at least one tuple and the sequence loss for each tuple of the at least one tuple.

10. The system of claim 9, wherein receiving the feature dependence matrix comprises: receiving data associated with domain knowledge of the dependence between each feature of the plurality of features and every other feature of the plurality of features; and determining the feature dependence matrix based on the data associated with domain knowledge.1 1 . The system of claim 9, wherein receiving the feature dependence matrix comprises extracting the feature dependence matrix based on the machine learning model.

12. The system of claim 1 1 , wherein extracting the feature dependence matrix comprises determining the feature dependence matrix based on the machine learning model and a Shapley value.

13. The system of claim 9, wherein determining the at least one tuple comprises determining the at least one tuple based on the data record and the feature dependence matrix using mixed integer programming to solve a bi-objective framework associated with the action vector and the sequence vector.

14. The system of claim 9, wherein the at least one tuple comprises a plurality of tuples, wherein the one or more instructions, when executed by the at least one processor, further cause the at least one processor to: rank the plurality of tuples based on the sequence loss for each tuple.

15. The system of claim 14, wherein communicating the at least one communication comprises: selecting one or more tuples from the plurality of tuples based on ranking the plurality of tuples; and generating the communication based on the one or more tuples.

16. The system of claim 9, wherein communicating the at least one communication comprises: determining an explanation associated with the machine learning model based on the at least one tuple, wherein the at least one communication comprises the explanation.

17. A computer program product, the computer program product comprising at least one non-transitory computer-readable medium comprising one or more instructions that, when executed by at least one processor, cause the at least one processor to:receive a data record comprising a plurality of fields, each respective field of the plurality of fields associated with a respective feature of a plurality of features; receive a feature dependence matrix associated with a dependence between each feature of the plurality of features and every other feature of the plurality of features; input the data record to a machine learning model to generate an initial output; determine at least one tuple based on the data record and the feature dependence matrix, each tuple of the at least one tuple comprising an action vector and a sequence vector, each action vector associated with a plurality of perturbations of respective fields of the plurality of fields for which inputting a perturbed data record based on the data record and the plurality of perturbations to the machine learning model generates a perturbed output different than the initial output, each sequence vector associated with a sequence of the plurality of perturbations for the action vector; determine a sequence loss for each tuple of the at least one tuple; and communicate at least one communication based on the at least one tuple and the sequence loss for each tuple of the at least one tuple.

18. The computer program product of claim 17, wherein receiving the feature dependence matrix comprises extracting the feature dependence matrix based on the machine learning model, wherein extracting the feature dependence matrix comprises determining the feature dependence matrix based on the machine learning model and a Shapley value.

19. The computer program product of claim 17, wherein determining the at least one tuple comprises determining the at least one tuple based on the data record and the feature dependence matrix using mixed integer programming to solve a bi-objective framework associated with the action vector and the sequence vector.

20. The computer program product of claim 17, wherein the at least one tuple comprises a plurality of tuples, wherein the one or more instructions, when executed by the at least one processor, further cause the at least one processor to:rank the plurality of tuples based on the sequence loss for each tuple, wherein communicating the at least one communication comprises: selecting one or more tuples from the plurality of tuples based on ranking the plurality of tuples; and generating the communication based on the one or more tuples.

Images