Systems, methods and program products for integrating knowledge from complex models into simple models

By integrating the knowledge of more complex models into a simpler payment industry model, using loss functions to train the intermediate layer output, the problems of insufficient generalization ability and interpretation of existing models are solved, and better fraud detection performance and model ease of use are achieved. sex.

CN111428872BActive Publication Date: 2024-12-06VISA INTERNATIONAL SERVICE ASSOCIATION
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010009567.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2019-01-10
Filing Date
2020-01-06
Publication Date
2024-12-06
Estimated Expiration
2040-01-06

AI Technical Summary

Technical Problem

The existing risk or fraud detection models in the payment industry are simple in structure and limited generalization capabilities, and cannot effectively capture complex risk models, resulting in false rejection of legal transactions, and poor interpretation of the model, affecting user trust.

Method used

By integrating knowledge from more complex models into simpler models, using loss functions to rely on intermediate layer output to train simpler models, improving its generalization capabilities while maintaining the ease of deployment and interpretability of the model.

Benefits of technology

It achieves better performance in binary classification tasks in the payment industry, improves the generalization ability of the model, maintains the ease of deployment and interpretability of the model, and avoids major changes to the data processor and system architecture.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111428872B_ABST
    Figure CN111428872B_ABST
Patent Text Reader

Abstract

The present invention provides a system, method and computer program product for incorporating knowledge from a more complex model into a simpler model. The method may include obtaining first training data associated with a first set of features and second training data associated with a second set of features different from the first set of features; training a first model based on the first training data and the second training data; and training a second model based on the second training data using a loss function, the loss function depending on the output of an intermediate layer of the first model and the output of the second model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure generally relates to incorporating knowledge from more complex models into simpler models, and in some embodiments or aspects, to systems, methods, and computer program products for training, providing, and / or using predictive models that incorporate knowledge from more complex models. Background Art

[0002] Existing risk or fraud detection models used in the payments industry use relatively simple models such as logistic regression, decision trees, feed-forward neural networks, and / or their equivalents. For example, the . The fraud detection model uses a feed-forward neural network, which provides a relatively simple analytical model architecture focused on efficiency, low latency, and low memory usage.

[0003] Simpler existing models may be easier to train and / or maintain than more complex models. For example, once trained, simpler existing models often use a smaller memory footprint and fewer computational resources at run time, making such simpler existing models well suited for deployment to a large number of users who may have more stringent requirements for latency and computational resource usage. For example, latency and computational resource usage requirements may be particularly stringent in the payments industry, where predictive models affect millions of consumers and are used to make real-time decisions.

[0004] Existing simpler models may also be easier to explain than more complex models. For example, model explainability can be used to assess the trust associated with the model's predictions. Existing risk or fraud detection models deployed in the payment industry may use many handcrafted features that incorporate domain expert knowledge collected over many years. These existing risk or fraud detection models are generally relatively easy to explain and well accepted by users. For example, taking action based on predictions from a model may become a tendency in financial decisions such as credit assessment, risk assessment, and / or the like that involve consumer rights, and the explainability of the model may affect whether the model is used for such financial decisions. For example, the effect of model explainability can be seen in the General Data Protection Regulation (GDPR) issued by the European Union, which came into effect as law across the European Union in May 2018 and created a "right to explanation" whereby consumers can request an explanation of algorithmic decisions made about consumers.

[0005] However, existing simpler models may have inherent limitations. For example, due to stricter requirements on latency and computing resources, existing simpler models may not use too many features and / or complex features, regardless of whether such features may be good indicators of risk or fraud. In addition, the structure of existing simpler models has more limited capacity than more complex models, which may weaken the generalization ability of existing simpler models. In this way, existing simpler models may fail to capture potential and more complex risk patterns, and / or erroneously reject legitimate transactions, which may result in revenue loss and / or poor consumer experience. For example, although major data processors and issuing banks have deployed fraud detection models, such as Fraud detection models are used to protect consumer transactions, but the losses caused by fraudulent transactions and wrongly rejected transactions are relatively large. For example, on average, in the payment industry, there may be $9 of fraudulent transactions and $520 of rejected transactions for every $10,000 of approved transactions, and a large portion of the $520 of rejected transactions may be wrongly rejected.

[0006] Deep learning has had a relatively large impact on a wide variety of applications, especially in the fields of computer vision and speech recognition, where deep neural networks including multiple layers are used. These existing deep learning models may include millions of parameters and may be trained using billions of examples, which may provide greater capacity and better generalization performance than existing simpler models. Existing ensemble methods that combine a set of existing models to make predictions may also produce more complex existing models with relatively stronger generalization capabilities.

[0007] However, despite having relatively stronger generalization performance, existing more complex models are not very suitable for applications with stricter requirements on latency and computing resource utilization. For example, existing more complex models may not be able to generate decisions in real time. For example, Netflix awarded a developer team a $1 million bonus in 2009 for an integrated model that increased the accuracy of the company's recommendation engine by 10%, but Netflix's engineering blog disclosed in 2012 that Netflix had never deployed the winning integrated model in a production system because the increase in accuracy provided by the winning improvement "did not seem to justify the engineering effort necessary to introduce the improvement into the production environment."

[0008] Existing more complex models may also become less interpretable. As more complex models are involved in decisions, it becomes more difficult for users to perceive the underlying reasoning that led to the decisions made by more complex models. However, model explanations are likely to remain an important issue in the payments industry for years to come, because if users do not trust the model or its decisions, they will not implement it.

[0009] Therefore, there is a need in the field to improve the generalization ability of the model while maintaining the ease of deployment and interpretability of the model. Summary of the invention

[0010] Thus, systems, methods, and computer program products are provided for improving the generalization ability of a model by incorporating knowledge from a more complex model into a simpler model while maintaining ease of deployment and interpretability of the model.

[0011] According to some non-limiting embodiments or aspects, a computer-implemented method is provided, comprising: obtaining, by at least one processor, first training data associated with a first set of features and second training data associated with a second set of features different from the first set of features; training, by at least one processor, a first model based on the first training data and the second training data; and training, by at least one processor, a second model based on the second training data using a loss function, wherein the loss function depends on an output of an intermediate layer of the first model and an output of the second model.

[0012] In some non-limiting embodiments or aspects, the second model includes at least one first layer and at least one second layer, the output of the second model includes the output of the at least one first layer, and training the second model also includes: modifying one or more parameters of the at least one first layer of the second model using a loss function that depends on the output of the intermediate layer of the first model and the output of the second model including the output of the first layer; and training the at least one second layer based on the output of the at least one first layer.

[0013] In some non-limiting embodiments or aspects, the first model includes at least one of: a deep neural network, a recurrent neural network, an ensemble of multiple neural networks, or any combination thereof; the first layer of the second model includes a recurrent neural network, and the second layer of the second model includes a logistic regression model.

[0014] In some non-limiting embodiments or aspects, the method further includes: determining, by at least one processor, a plurality of information values ​​of a plurality of intermediate layers of the first model; and selecting, by at least one processor, an intermediate layer from the plurality of intermediate layers based on the plurality of information values.

[0015] In some non-limiting embodiments or aspects, the first set of features includes complex features and the second set of features includes interpretable features.

[0016] In some non-limiting embodiments or aspects, the first model includes a greater number of parameters than the second model.

[0017] In some non-limiting embodiments or aspects, the method further includes: providing a trained second model through at least one processor; obtaining input data associated with at least one transaction through at least one processor; and processing the input data through at least one processor and using the trained second model to generate output data, wherein the output data includes a prediction of whether the at least one transaction is a fraudulent transaction.

[0018] According to some non-limiting embodiments or aspects, a computing system comprising at least one processor is provided, wherein the at least one processor is programmed and / or configured to: obtain first training data associated with a first set of features and second training data associated with a second set of features different from the first set of features; train a first model based on the first training data and the second training data; and train the second model based on the second training data using a loss function, wherein the loss function depends on an output of an intermediate layer of the first model and an output of the second model.

[0019] In some non-limiting embodiments or aspects, the second model includes at least one first layer and at least one second layer, the output of the second model includes the output of the at least one first layer, and the at least one processor is further programmed and / or configured to train the second model in the following manner: modifying one or more parameters of the at least one first layer of the second model using a loss function that depends on the output of the intermediate layer of the first model and the output of the second model including the output of the first layer; and training the at least one second layer based on the output of the at least one first layer.

[0020] In some non-limiting embodiments or aspects, the first model includes at least one of: a deep neural network, a recurrent neural network, an ensemble of multiple neural networks, or any combination thereof; the first layer of the second model includes a recurrent neural network, and the second layer of the second model includes a logistic regression model.

[0021] In some non-limiting embodiments or aspects, the at least one processor is further programmed and / or configured to: determine a plurality of information values ​​for a plurality of intermediate layers of the first model; and select an intermediate layer from the plurality of intermediate layers based on the plurality of information values.

[0022] In some non-limiting embodiments or aspects, the first set of features includes complex features and the second set of features includes interpretable features.

[0023] In some non-limiting embodiments or aspects, the first model includes a greater number of parameters than the second model.

[0024] In some non-limiting embodiments or aspects, the at least one processor is further programmed and / or configured to: provide a trained second model; obtain input data associated with at least one transaction; and process the input data using the trained second model to generate output data, the output data comprising a prediction of whether the at least one transaction is a fraudulent transaction.

[0025] According to some non-limiting embodiments or aspects, there is provided a computer program product comprising at least one non-transitory computer-readable medium, the at least one non-transitory computer-readable medium comprising program instructions, which, when executed by at least one processor, causes the at least one processor to: obtain first training data associated with a first set of features and second training data associated with a second set of features different from the first set of features; train a first model based on the first training data and the second training data; and train the second model based on the second training data using a loss function, the loss function depending on an output of an intermediate layer of the first model and an output of the second model.

[0026] In some non-limiting embodiments or aspects, the second model includes at least one first layer and at least one second layer, the output of the second model includes the output of the at least one first layer, and the instructions also cause the at least one processor to train the second model in the following manner: modifying one or more parameters of the at least one first layer of the second model using a loss function that depends on the output of the intermediate layer of the first model and the output of the second model including the output of the first layer; and training the at least one second layer based on the output of the at least one first layer.

[0027] In some non-limiting embodiments or aspects, the instructions further cause the at least one processor to: determine a plurality of information values ​​for a plurality of intermediate layers of the first model; and select an intermediate layer from the plurality of intermediate layers based on the plurality of information values.

[0028] In some non-limiting embodiments or aspects, the first set of features includes complex features and the second set of features includes interpretable features.

[0029] In some non-limiting embodiments or aspects, the first model includes a greater number of parameters than the second model.

[0030] In some non-limiting embodiments or aspects, the instructions further cause the at least one processor to: provide a trained second model; obtain input data associated with at least one transaction; and process the input data using the trained second model to generate output data, the output data comprising a prediction of whether the at least one transaction is a fraudulent transaction.

[0031] Other embodiments or aspects are described in the following numbered clauses:

[0032] Article 1 A computer-implemented method comprising: obtaining, by at least one processor, first training data associated with a first set of features and second training data associated with a second set of features different from the first set of features; training, by at least one processor, a first model based on the first training data and the second training data; and training, by at least one processor, a second model based on the second training data using a loss function, wherein the loss function depends on an output of an intermediate layer of the first model and an output of the second model.

[0033] Article 2. A computer-implemented method according to Article 1, wherein the second model comprises at least one first layer and at least one second layer, wherein the output of the second model comprises the output of the at least one first layer, and wherein training the second model further comprises: modifying one or more parameters of the at least one first layer of the second model using a loss function that depends on the output of an intermediate layer of the first model and the output of the second model including the output of the first layer; and training the at least one second layer based on the output of the at least one first layer.

[0034] Clause 3. The computer-implemented method of clause 1 or clause 2, wherein the first model comprises at least one of: a deep neural network, a recurrent neural network, an ensemble of multiple neural networks, or any combination thereof, wherein the first layer of the second model comprises a recurrent neural network, and wherein the second layer of the second model comprises a logistic regression model.

[0035] Article 4 The computer-implemented method according to any one of Articles 1 to 3 further includes: determining, by at least one processor, a plurality of information values ​​of a plurality of intermediate layers of the first model; and selecting, by at least one processor, an intermediate layer from the plurality of intermediate layers based on the plurality of information values.

[0036] Clause 5. The computer-implemented method of any of Clauses 1 to 4, wherein the first set of features comprises complex features, and wherein the second set of features comprises interpretable features.

[0037] Clause 6. The computer-implemented method of any of Clauses 1 to 5, wherein the first model comprises a greater number of parameters than the second model.

[0038] Article 7 The computer-implemented method of any one of Articles 1 to 6 further includes: providing a trained second model through at least one processor; obtaining input data associated with at least one transaction through at least one processor; and processing the input data through at least one processor and using the trained second model to generate output data, wherein the output data includes a prediction of whether the at least one transaction is a fraudulent transaction.

[0039] Article 8 A computing system comprising at least one processor, wherein the at least one processor is programmed and / or configured to: obtain first training data associated with a first set of features and second training data associated with a second set of features different from the first set of features; train a first model based on the first training data and the second training data; and train the second model based on the second training data using a loss function, wherein the loss function depends on an output of an intermediate layer of the first model and an output of the second model.

[0040] Article 9. A computing system according to Article 8, wherein the second model comprises at least one first layer and at least one second layer, wherein the output of the second model comprises the output of the at least one first layer, and wherein the at least one processor is further programmed and / or configured to train the second model by: modifying one or more parameters of the at least one first layer of the second model using a loss function that depends on the output of an intermediate layer of the first model and the output of the second model including the output of the first layer; and training the at least one second layer based on the output of the at least one first layer.

[0041] Article 10. The computing system of Article 8 or Article 9, wherein the first model comprises at least one of: a deep neural network, a recurrent neural network, an ensemble of multiple neural networks, or any combination thereof, wherein the first layer of the second model comprises a recurrent neural network, and wherein the second layer of the second model comprises a logistic regression model.

[0042] Clause 11. A computing system according to any one of Clauses 8 to 10, wherein the at least one processor is further programmed and / or configured to: determine a plurality of information values ​​of a plurality of intermediate layers of the first model; and select an intermediate layer from the plurality of intermediate layers based on the plurality of information values.

[0043] Clause 12. The computing system of any of Clauses 8 to 11, wherein the first set of features comprises complex features, and wherein the second set of features comprises interpretable features.

[0044] Clause 13. The computing system of any of Clauses 8 to 12, wherein the first model includes a greater number of parameters than the second model.

[0045] Article 14. A computing system as described in any of Articles 8 to 13, wherein the at least one processor is further programmed and / or configured to: provide a trained second model; obtain input data associated with at least one transaction; and process the input data using the trained second model to generate output data, wherein the output data includes a prediction of whether the at least one transaction is a fraudulent transaction.

[0046] Article 15 A computer program product comprising at least one non-transitory computer-readable medium, the at least one non-transitory computer-readable medium comprising program instructions, which, when executed by at least one processor, cause the at least one processor to: obtain first training data associated with a first set of features and second training data associated with a second set of features different from the first set of features; train a first model based on the first training data and the second training data; and train the second model based on the second training data using a loss function, the loss function depending on an output of an intermediate layer of the first model and an output of the second model.

[0047] Article 16. A computer program product according to Article 15, wherein the second model comprises at least one first layer and at least one second layer, wherein the output of the second model comprises the output of the at least one first layer, and wherein the instructions further cause the at least one processor to train the second model by: modifying one or more parameters of the at least one first layer of the second model using a loss function that depends on the output of an intermediate layer of the first model and the output of the second model including the output of the first layer; and training the at least one second layer based on the output of the at least one first layer.

[0048] Clause 17. The computer program product of Clause 15 or Clause 16, wherein the instructions further cause the at least one processor to: determine a plurality of information values ​​for a plurality of intermediate layers of the first model; and select an intermediate layer from the plurality of intermediate layers based on the plurality of information values.

[0049] Clause 18. A computer program product according to any of Clauses 15 to 17, wherein the first set of features comprises complex features and wherein the second set of features comprises interpretable features.

[0050] Clause 19. The computer program product of any of Clauses 15 to 18, wherein the first model comprises a greater number of parameters than the second model.

[0051] Article 20. A computer program product according to any one of Articles 15 to 19, wherein the instructions further cause the at least one processor to: provide a trained second model; obtain input data associated with at least one transaction; and process the input data using the trained second model to generate output data, wherein the output data includes a prediction of whether the at least one transaction is a fraudulent transaction.

[0052] These and other features and characteristics of the present disclosure and methods of operation and functions of combinations of related structural elements and parts and economies of manufacture will become more apparent after considering the following description and the appended claims with reference to the accompanying drawings, all of which form part of this specification, wherein like reference numerals indicate corresponding parts in the various figures. However, it should be expressly understood that the various figures are for illustration and description purposes only and are not intended as definitions of limitations. As used in the specification and in the claims, the singular forms "a", "an" and "the" include plural referents unless the context clearly dictates otherwise. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Additional advantages and details of non-limiting embodiments or aspects are explained in more detail below with reference to exemplary embodiments shown in the attached schematic drawings, in which:

[0054] Figure 1 are diagrams of non-limiting embodiments or aspects of environments in which the systems, apparatus, products, devices, and / or methods described herein may be implemented;

[0055] Figure 2 yes Figure 1 Figures of non-limiting embodiments or aspects of one or more devices and / or components of one or more systems;

[0056] Figure 3 is a flow chart illustrating a non-limiting embodiment or aspect of a process for training, providing, and / or using a predictive model that incorporates knowledge from a more complex model;

[0057] Figure 4 is a diagram of an embodiment of a non-limiting example or aspect of a process disclosed herein for training, providing, and / or using a predictive model that incorporates knowledge from a more complex model;

[0058] Figure 5A is a diagram of an implementation of a non-limiting example or aspect of a process disclosed herein for training, providing, and / or using a predictive model that incorporates knowledge from a more complex model; and

[0059] Figure 5B Yes Figure 5A A graph of information values ​​for a non-limiting example or aspect of an implementation of a process for training, providing, and / or using a predictive model that incorporates knowledge from more complex models as shown in . DETAILED DESCRIPTION

[0060] It should be understood that, except where expressly specified to the contrary, the present disclosure may employ various alternative variations and step sequences. It should also be understood that the specific devices and processes shown in the accompanying drawings and described in the following specification are merely exemplary and non-limiting embodiments or aspects. Therefore, specific dimensions and other physical characteristics associated with the embodiments or aspects disclosed herein should not be considered limiting.

[0061] Aspects, parts, elements, structures, actions, steps, functions, instructions and / or the like used herein should not be understood as critical or necessary unless explicitly described as such. In addition, as used herein, the article "one" is intended to include one or more items, and can be used interchangeably with "one or more" and "at least one". In addition, as used herein, the term "group" is intended to include one or more items (e.g., related items, unrelated items, a combination of related items and unrelated items, etc.), and can be used interchangeably with "one or more" or "at least one". In the case of only one project, the term "one" or similar language is used. In addition, as used herein, the term "having" or the like is intended to be an open term. In addition, unless otherwise explicitly stated, the phrase "based on" is intended to mean "based at least in part on".

[0062] For the purposes of the following description, the terms "end," "upper," "lower," "right," "left," "vertical," "horizontal," "top," "bottom," "lateral," "longitudinal," and derivatives thereof shall relate to the non-limiting embodiments or aspects as they are oriented in the accompanying drawings. However, it shall be understood that the non-limiting embodiments or aspects may employ various alternative variations and step sequences, except where expressly specified to the contrary. It shall also be understood that the specific devices and processes shown in the accompanying drawings and described in the following specification are merely exemplary embodiments or aspects. Therefore, unless otherwise indicated, specific dimensions and other physical characteristics associated with the embodiments or aspects of the embodiments disclosed herein shall not be considered limiting.

[0063] As used herein, the terms "communication" and "communication" may refer to the reception, acceptance, transmission, transfer, provision and / or the like of information (e.g., data, signals, messages, instructions, commands and / or the like). A unit (e.g., a device, a system, a component of a device or system, a combination thereof and / or the like) communicating with another unit means that the unit is able to directly or indirectly receive information from the other unit and / or transmit information to the other unit. This may refer to a direct or indirect connection (e.g., a direct communication connection, an indirect communication connection and / or the like) that is wired and / or wireless in nature. In addition, the two units may communicate with each other even if the transmitted information may be modified, processed, relayed and / or routed between the first unit and the second unit. For example, the first unit may communicate with the second unit even if the first unit passively receives information and does not actively transmit the information to the second unit. As another example, the first unit may communicate with the second unit if at least one intermediate unit (e.g., a third unit located between the first unit and the second unit) processes the information received from the first unit and communicates the processed information to the second unit. In some non-limiting embodiments, a message may refer to a network packet (eg, a data packet and / or the like) that includes data. It should be appreciated that many other arrangements are possible.

[0064] Some non-limiting embodiments or aspects are described herein in conjunction with a threshold value. As used herein, satisfying a threshold value may refer to a value greater than a threshold value, more than a threshold value, above a threshold value, greater than or equal to a threshold value, less than a threshold value, less than a threshold value, below a threshold value, less than or equal to a threshold value, equal to a threshold value, etc.

[0065] As used herein, the term "transaction service provider" may refer to an entity that receives transaction authorization requests from merchants or other entities and, in some cases, provides payment assurance through an agreement between the transaction service provider and an issuer organization. For example, a transaction service provider may include a payment network, such as or any other entity that processes transactions. The term "transaction processing system" may refer to one or more computer systems operated by or on behalf of a transaction service provider, such as a transaction processing server that executes one or more software applications. A transaction processing server may include one or more processors and, in some non-limiting embodiments, may be operated by or on behalf of a transaction service provider.

[0066] As used herein, the term "issuer institution" may refer to one or more entities, such as a bank, that provide accounts to customers to conduct transactions (e.g., payment transactions), such as initiating credit and / or debit payments. For example, an issuer institution may provide customers with account identifiers, such as a primary account number (PAN), that uniquely identify one or more accounts associated with the customer. The account identifier may be implemented on a portable financial device, such as a physical financial instrument, such as a payment card, and / or may be electronic and used for electronic payments. The term "issuer system" refers to one or more computer systems operated by or on behalf of an issuer institution, such as a server computer that executes one or more software applications. For example, an issuer system may include one or more authorization servers for authorizing transactions.

[0067] As used herein, the term "acquirer institution" may refer to an entity that is authorized and / or approved by a transaction service provider to initiate transactions (e.g., payment transactions) using a portable financial device associated with the transaction service provider. Transactions that may be initiated by an acquirer institution may include payment transactions (e.g., purchases, original letter of credit transactions (OCTs), account funding transactions (AFTs), and / or the like). In some non-limiting embodiments, the acquirer institution may be a financial institution, such as a bank. As used herein, the term "acquirer system" may refer to one or more computer systems, computer devices, software applications, and / or the like operated by or on behalf of an acquirer institution.

[0068] As used herein, the term "account identifier" may include one or more PANs, tokens, or other identifiers associated with a customer account. The term "token" may refer to an identifier used as a replacement or alternative identifier for an original account identifier such as a PAN. An account identifier may be alphanumeric, or any combination of characters and / or symbols. A token may be associated with a PAN or another original account identifier in one or more data structures (e.g., one or more databases and / or the like) so that the token can be used to conduct transactions without directly using the original account identifier. In some examples, an original account identifier, such as a PAN, may be associated with multiple tokens for different individuals or purposes. As used herein, the term "account data" refers to any data about one or more accounts of one or more users. Account data may include, for example, one or more account identifiers, user identifiers, transaction history, balances, credit limits, issuer institution identifiers, and / or the like.

[0069] As used herein, the term "merchant" may refer to a person or entity that provides goods and / or services, or the right to use goods and / or services, to a customer based on a transaction such as a payment transaction. The term "merchant" or "merchant system" may also refer to one or more computer systems operated by or on behalf of a merchant, such as a server computer that executes one or more software applications. As used herein, a "point of sale (POS) system" may refer to one or more computers and / or peripheral devices used by a merchant to conduct payment transactions with customers, including one or more card readers, near field communication (NFC) receivers, RFID receivers and / or other contactless transceivers or receivers, contact-based receivers, payment terminals, computers, servers, input devices, and / or other similar devices that can be used to initiate payment transactions.

[0070] As used herein, the term "mobile device" may refer to one or more portable electronic devices configured to communicate with one or more networks. For example, a mobile device may include a cellular phone (e.g., a smart phone or a standard cellular phone), a portable computer (e.g., a tablet computer, a laptop computer, etc.), a wearable device (e.g., a watch, glasses, lenses, clothing, and / or the like), a personal digital assistant (PDA), and / or other similar devices. As used herein, the terms "client device" and "user device" refer to any electronic device configured to communicate with one or more servers or remote devices and / or systems. Client devices or user devices may include mobile devices, network-enabled devices (e.g., network-enabled televisions, refrigerators, thermostats, and / or the like), computers, POS systems, and / or any other devices or systems capable of communicating with a network.

[0071] As used herein, the term "computing device" may refer to one or more electronic devices configured to communicate directly or indirectly with one or more networks or to communicate on one or more networks. The computing device may be a mobile device, a desktop computer, etc. In addition, the term "computer" may refer to any computing device including necessary components for receiving, processing and outputting data and typically including a display, a processor, a memory, an input device, and a network interface. An "application" or "application programming interface" (API) refers to computer code or other data sorted on a computer-readable medium, which can be executed by a processor to facilitate interaction between software components, such as a client-side front end and / or a server-side back end for receiving data from a client. An "interface" refers to a generated display, such as one or more graphical user interfaces (GUIs) with which a user can interact directly or indirectly (e.g., via a keyboard, a mouse touch screen, etc.).

[0072] As used herein, the terms "electronic wallet" and "electronic wallet application" refer to one or more electronic devices and / or software applications configured to initiate and / or conduct payment transactions. For example, an electronic wallet may include a mobile device executing an electronic wallet application, and may also include server-side software and / or databases for maintaining and providing transaction data to the mobile device. An "electronic wallet provider" may include an entity that provides and / or maintains electronic wallets for customers, such as Google Wallet. TM 、Android Apple Samsung And / or other similar electronic payment systems. In some non-limiting examples, the issuing bank may be an electronic wallet provider.

[0073] As used herein, the term "portable financial device" may refer to a payment card (e.g., a credit or debit card), a gift card, a smart card, smart media, a payroll card, a healthcare card, a wristband, a machine-readable medium containing account information, a keychain device or fob, an RFID transponder, a retailer discount or loyalty card, a cellular telephone, an electronic wallet mobile application, a personal digital assistant (PDA), a security card, a computer, an access card, a wireless terminal and / or a transponder, and / or the like. In some non-limiting embodiments, a portable financial device may include volatile or non-volatile memory to store information (e.g., an account identifier, an account holder's name, and / or the like).

[0074] As used herein, the term "server" may refer to or include one or more processors or computers, storage devices, or similar computer arrangements that operate or facilitate communication and processing by multiple parties in a network environment such as the Internet, but it should be understood that communication can be facilitated by one or more public or private network environments, and various other arrangements are possible. In addition, multiple computers, such as servers, or other computerized devices, such as POS devices, that communicate directly or indirectly in a network environment may constitute a "system", such as a merchant's POS system.

[0075] As used herein, the term "payment gateway" may refer to an entity and / or a payment processing system operated by or on behalf of such an entity that provides payment services (e.g., transaction service provider payment services, payment processing services, and / or the like) to one or more merchants (e.g., transaction service provider payment services, payment processing services, and / or the like). The payment service may be associated with the use of a portable financial device managed by a transaction service provider. As used herein, the term "payment gateway system" may refer to one or more computer systems, computer devices, servers, server groups, and / or the like operated by or on behalf of a payment gateway.

[0076] Improved systems, methods, and computer program products are provided for incorporating knowledge from more complex models into simpler models.

[0077] Bucila, C., Caruana, R., and Niculescu-Mizil, A., in a paper titled "Model Compression" in the proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2006), the entire contents of which are hereby incorporated by reference herein, describe a "model compression" approach to model training in which knowledge acquired by a more cumbersome or complex model, including a large ensemble of models, can be transferred to a single smaller or simpler model. For example, this prior model compression approach uses the output of the final layer of the more complex model to train the simpler model, which is logits with dimension six that are input to a softmax, rather than the probabilities produced by the softmax.

[0078] Hinton, G., Vinyals, O., and Dean, J., in a paper titled "Distilling the Knowledge in a Neural Network" at the Deep Learning and Representation Learning Workshop (NIPS 2014), the entire contents of which are hereby incorporated by reference herein, describe a "knowledge distillation" approach for model training to transfer knowledge from a more cumbersome or complex model to a smaller or simpler model. For example, this prior art knowledge distillation approach uses a more complex model including an ensemble of deep neural network models and trains the simpler neural network model to match the output of the final layer of the more complex model, which is a variation of the class probabilities produced using a softmax with an annealing parameter T. Prior art knowledge distillation approaches have been shown to perform well in classification tasks that include multiple classes, such as image and speech recognition tasks that typically involve hundreds and / or thousands of classes (e.g., in speech recognition experiments conducted using prior art knowledge distillation approaches, the softmax contained 14,000 labels).

[0079] However, existing model compression methods and existing knowledge distillation methods may not perform well on classification tasks that are commonly found in the payment industry, such as classification tasks that are binary or only include two values ​​(such as approval / rejection, fraud / non-fraud, etc.), because using these existing methods for such classification tasks will result in the use of lower dimensional vectors associated with the final layer of a more cumbersome or complex model, and the lower dimensional vectors do not have sufficient ability to fully transfer knowledge from the more cumbersome or complex model to a smaller or simpler model. For example, the simpler model may only capture the information provided by the true classification label without learning the finer structure learned by the more complex model. For example, when the dimension of the feature vector is too small, the logits input to the softmax and the class probability output from the softmax cannot express the true data distribution. In this way, the existing computer system and method for model training have no mechanism to train the simpler model using a loss function that directly depends on the output of the intermediate layer of the more complex model. For example, the existing computer system and method for model training have no mechanism to transfer or transmit the true data distribution from the more complex model to the simpler model for binary or only two value classification tasks. Therefore, existing computer systems and methods for model training do not perform well on classification tasks that are commonly found in the payment industry and are binary or include only two values ​​(e.g., approved / rejected, fraudulent / non-fraudulent, etc.).

[0080] Non-limiting embodiments or aspects of the present disclosure relate to systems, methods, and computer program products for training, providing, and / or using a predictive model that incorporates knowledge from a more complex model by: obtaining first training data associated with a first set of features and second training data associated with a second set of features different from the first set of features; training the first model based on the first training data and the second training data; and training the second model based on the second training data using a loss function that depends on an output of an intermediate layer of the first model and an output of the second model.

[0081] In this way, embodiments or aspects of the present disclosure may enable training of simpler models using a loss function that depends directly on the output of an intermediate layer of a more complex model, which may enable (i) a simpler model that learns the finer structure learned by the more complex model; (ii) better model performance on classification tasks that are commonly found in the payment industry, such as classification tasks that are binary or include only two values ​​(e.g., approved / rejected, fraudulent / non-fraudulent, etc.); (iii) improving the generalization ability of the model while maintaining the ease of deployment and interpretability of the model without requiring payment data processors and / or issuing banks to change existing system architectures and data feeds to accommodate more complex models; and / or the like.

[0082] Reference Figure 1 , Figure 1 1 is a diagram of an example environment 100 in which the apparatus, systems, methods, and / or products described herein may be implemented. Figure 1 As shown in FIG. 1 , environment 100 includes a transaction processing network 101, which may include a merchant system 102, a payment gateway 104, an acquirer system 106, a transaction service provider system 108, and / or an issuer system 110, a user device 112, and / or a communication network 114. The transaction processing network 101, the merchant system 102, the payment gateway 104, the acquirer system 106, the transaction service provider system 108, the issuer system 110, and / or the user device 112 may be interconnected (e.g., establish a connection to communicate) via a wired connection, a wireless connection, or a combination of wired and wireless connections.

[0083] The merchant system 102 may include one or more devices that are capable of receiving information from the payment gateway 104, the acquirer system 106, the transaction service provider system 108, the issuer system 110, and / or the user device 112 via the communication network 114, and / or transmitting information to the payment gateway 104, the acquirer system 106, the transaction service provider system 108, the issuer system 110, and / or the user device 112 via the communication network 114. The merchant system 102 may include a communication connection (e.g., an NFC communication connection, an RFID communication connection, In some non-limiting embodiments or aspects, the merchant system 102 may include one or more devices, such as computers, computer systems, and / or peripheral devices, that can be used by merchants to conduct payment transactions with users. For example, the merchant system 102 may include a POS device and / or a POS system.

[0084] The payment gateway 104 may include one or more devices capable of receiving information from the merchant system 102, the acquirer system 106, the transaction service provider system 108, the issuer system 110, and / or the user device 112 via the communication network 114, and / or transmitting information to the merchant system 102, the acquirer system 106, the transaction service provider system 108, the issuer system 110, and / or the user device 112 via the communication network 114. For example, the payment gateway 104 may include a computing device, such as a server, a server group, and / or other similar devices. In some non-limiting embodiments or aspects, the payment gateway 104 is associated with the payment gateway described herein.

[0085] The acquirer system 106 may include one or more devices capable of receiving information from the merchant system 102, the payment gateway 104, the transaction service provider system 108, the issuer system 110, and / or the user device 112 via the communication network 114, and / or communicating information to the merchant system 102, the payment gateway 104, the transaction service provider system 108, the issuer system 110, and / or the user device 112 via the communication network 114. For example, the acquirer system 106 may include a computing device, such as a server, a server group, and / or other similar devices. In some non-limiting embodiments or aspects, the acquirer system 106 may be associated with an acquirer as described herein.

[0086] The transaction service provider system 108 may include one or more devices capable of receiving information from the merchant system 102, the payment gateway 104, the acquirer system 106, the issuer system 110, and / or the user device 112 via the communication network 114, and / or communicating information to the merchant system 102, the payment gateway 104, the acquirer system 106, the issuer system 110, and / or the user device 112 via the communication network 114. For example, the transaction service provider system 108 may include a computing device, such as a server (e.g., a transaction processing server), a server group, and / or other similar devices. In some non-limiting embodiments or aspects, the transaction service provider system 108 may be associated with a transaction service provider described herein.

[0087] The issuer system 110 may include one or more devices that are capable of receiving information from the merchant system 102, the payment gateway 104, the acquirer system 106, the transaction service provider system 108, and / or the user device 112 via the communication network 114, and / or communicating information to the merchant system 102, the payment gateway 104, the acquirer system 106, the transaction service provider system 108, and / or the user device 112 via the communication network 114. For example, the issuer system 110 may include a computing device, such as a server, a server group, and / or other similar devices. In some non-limiting embodiments or aspects, the issuer system 110 may be associated with an issuer organization described herein. For example, the issuer system 110 may be associated with an issuer organization that issues payment accounts or instruments (e.g., credit accounts, debit accounts, credit cards, debit cards, etc.) to users (e.g., users associated with the user device 112, etc.).

[0088] In some non-limiting embodiments or aspects, the transaction processing network 101 includes multiple systems in a communication path for processing transactions. For example, the transaction processing network 101 may include a merchant system 102, a payment gateway 104, an acquirer system 106, a transaction service provider system 108, and / or an issuer system 110 in a communication path (e.g., a communication path, a communication channel, a communication network, etc.) for processing electronic payment transactions. For example, the transaction processing network 101 can process (e.g., initiate, conduct, authorize, etc.) electronic payment transactions through the communication path between the merchant system 102, the payment gateway 104, the acquirer system 106, the transaction service provider system 108, and / or the issuer system 110.

[0089] The user device 112 may include one or more devices that are capable of receiving information from the merchant system 102, the payment gateway 104, the acquirer system 106, the transaction service provider system 108, and / or the issuer system 110 via the communication network 114, and / or transmitting information to the merchant system 102, the payment gateway 104, the acquirer system 106, the transaction service provider system 108, and / or the issuer system 110 via the communication network 114. For example, the user device 112 may include a client device and / or the like. In some non-limiting embodiments or aspects, the user device 112 is capable of communicating with the merchant system 102, the payment gateway 104, the acquirer system 106, the transaction service provider system 108, and / or the issuer system 110 via a short-range wireless communication connection (e.g., an NFC communication connection, an RFID communication connection, In some non-limiting embodiments or aspects, the user device 112 may include an application associated with the user device 112, such as an application stored on the user device 112, a mobile application stored and / or executed on the user device 112 (e.g., a mobile device application, a native application of the mobile device, a mobile cloud application of the mobile device, an electronic wallet application, and / or the like).

[0090] The communication network 114 may include one or more wired and / or wireless networks. For example, the communication network 114 may include a cellular network (e.g., a long term evolution (LTE) network, a third generation (3G) network, a fourth generation (4G) network, a code division multiple access (CDMA) network, etc.), a public land mobile network (PLMN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a telephone network (e.g., a public switched telephone network (PSTN), a private network, an ad hoc network, an intranet, the Internet, a fiber-optic-based network, a cloud computing network, and / or the like, and / or a combination of these or other types of networks.

[0091] Provided as an example Figure 1 The number and arrangement of devices, systems, and networks shown in FIG. There may be additional devices, systems, and / or networks, fewer devices, systems, and / or networks, different devices, systems, and / or networks, or devices, systems, and / or networks in a similar manner. Figure 1 The devices, systems and / or networks shown in the figure may be arranged in different ways. In addition, the invention may be implemented in a single device and / or system. Figure 1 Two or more devices and / or systems shown, or Figure 1 The single device and / or system shown may be implemented as multiple distributed devices and / or systems. Additionally or alternatively, a set of devices and / or systems (e.g., one or more devices or systems) of environment 100 may perform one or more functions described as being performed by another set of devices or systems of environment 100.

[0092] Reference Figure 2 , Figure 2 is a diagram of example components of device 200. Device 200 may correspond to one or more devices of transaction processing network 101, one or more devices of merchant system 102, one or more devices of payment gateway 104, one or more devices of acquirer system 106, one or more devices of transaction service provider system 108, one or more devices of issuer system 110, one or more devices of user device 112, and / or one or more devices of communication network 114. In some non-limiting embodiments or aspects, one or more devices of transaction processing network 101, one or more devices of merchant system 102, one or more devices of payment gateway 104, one or more devices of acquirer system 106, one or more devices of transaction service provider system 108, one or more devices of issuer system 110, one or more devices of user device 112, and / or one or more devices of communication network 114 may include at least one device 200 and / or at least one component of device 200. As Figure 2 As shown in , device 200 may include a bus 202 , a processor 204 , a memory 206 , a storage component 208 , an input component 210 , an output component 212 , and a communication interface 214 .

[0093] The bus 202 may include components that permit communication between components of the device 200. In some non-limiting embodiments or aspects, the processor 204 may be implemented in hardware, firmware, or a combination of hardware and software. For example, the processor 204 may include a processor (e.g., a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), etc.), a microprocessor, a digital signal processor (DSP), and / or any processing component that can be programmed to perform a function (e.g., a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc.). The memory 206 may include a random access memory (RAM), a read-only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic memory, optical memory, etc.) that stores information and / or instructions for use by the processor 204.

[0094] The storage component 208 may store information and / or software related to the operation and use of the device 200. For example, the storage component 208 may include a hard disk (e.g., a magnetic disk, an optical disk, a magneto-optical disk, a solid-state disk, etc.), a compact disk (CD), a digital versatile disk (DVD), a floppy disk, a cassette, a magnetic tape, and / or another type of computer-readable medium, and a corresponding drive.

[0095] Input components 210 may include components that permit device 200 to receive information, such as through user input (e.g., a touch screen display, a keyboard, a keypad, a mouse, buttons, switches, a microphone, a camera, an electroencephalogram (EEG) monitor, etc.). Additionally or alternatively, input components 210 may include sensors for sensing information (e.g., a global positioning system (GPS) component, an accelerometer, a gyroscope, an actuator, etc.). Output components 212 may include components that provide output information from device 200 (e.g., a display, a speaker, one or more light emitting diodes (LEDs), and / or the like).

[0096] The communication interface 214 may include transceiver-like components (e.g., transceivers, separate receivers and transmitters, etc.) that enable the device 200 to communicate with other devices, such as through a wired connection, a wireless connection, or a combination of wired and wireless connections. The communication interface 214 may permit the device 200 to receive information from another device and / or provide information to another device. For example, the communication interface 214 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, interface, cellular network interface, and / or the like.

[0097] The device 200 may perform one or more processes described herein. The device 200 may perform these processes based on the processor 204 executing software instructions stored by a computer-readable medium such as the memory 206 and / or the storage component 208. A computer-readable medium (e.g., a non-transitory computer-readable medium) is defined herein as a non-transitory memory device. A non-transitory memory device includes a memory space located within a single physical memory device or a memory space spread across multiple physical memory devices.

[0098] The software instructions may be read into the memory 206 and / or storage component 208 from other computer-readable media or from another device via the communication interface 214. When executed, the software instructions stored in the memory 206 and / or storage component 208 may cause the processor 204 to perform one or more processes described herein. Additionally or alternatively, hard-wired circuitry may be used in place of or in combination with software instructions to perform one or more processes described herein. Thus, the embodiments or aspects described herein are not limited to any specific combination of hardware circuitry and software.

[0099] The memory 206 and / or the storage component 208 may include a data storage device or one or more data structures (e.g., a database, etc.). The device 200 can receive information from, store information in, communicate information to, or search for information stored in the data storage device or one or more data structures in the memory 206 and / or the storage component 208. For example, the information may include training data, first training data, second training data, input data, output data, transaction data, account data, or any combination thereof.

[0100] Provided as an example Figure 2 In some non-limiting embodiments or aspects, the device 200 may include additional components, fewer components, different components, or components in a manner similar to that of the device 200. Figure 2 Additionally or alternatively, one set of components (eg, one or more components) of the device 200 may perform one or more functions described as being performed by another set of components of the device 200.

[0101] Reference Figure 3 , Figure 3 300 for training, providing, and / or using a predictive model that incorporates knowledge from a more complex model. In some non-limiting embodiments or aspects, one or more steps in the process 300 may be performed (e.g., in whole, in part, etc.) by a transaction service provider system 108 (e.g., one or more devices of the transaction service provider system 108). In some non-limiting embodiments or aspects, one or more steps in the process 300 may be performed (e.g., in whole, in part, etc.) by another device or group of devices that is independent of or includes the transaction service provider system 108, such as a merchant system 102 (e.g., one or more devices of the merchant system 102), a payment gateway 104 (e.g., one or more devices of the payment gateway 104), an acquirer system 106 (e.g., one or more devices of the acquirer system 106), an issuer system 110 (e.g., one or more devices of the issuer system 110), and / or a user device 112.

[0102] like Figure 3As shown in , at step 302, process 300 includes obtaining first training data and second training data. For example, the transaction service provider system 108 may obtain the first training data and the second training data. For example, the transaction service provider system 108 may obtain first training data (e.g., more complex training data, etc.) associated with a first set of features (e.g., complex features, more complex features, etc.) and second training data (e.g., simpler training data, etc.) associated with a second set of features different from the first set of features (e.g., simpler features, explainable features, etc.). In some non-limiting embodiments or aspects, the first training data and / or the second training data may be determined based on parameters of a transaction stored in a transaction data log or database.

[0103] In some non-limiting embodiments or aspects, the first set of features includes a greater number of features than the second set of features. In some non-limiting embodiments or aspects, the first set of features includes complex features and the second set of features includes interpretable features.

[0104] In some non-limiting embodiments or aspects, the transaction service provider system 108 may generate first training data including a first set of features based on the transaction data. For example, and also referring to Figure 4 , which is a diagram of an implementation 400 of a non-limiting example or aspect of a process for training, providing, and / or using a predictive model that incorporates knowledge from a more complex model as described herein, in a training phase, at reference numeral 402, the transaction service provider system 108 may learn by representing complex features (e.g., as described in Bengio, Y. Foundations and 2: No. 1, 2009, pp. 1-127, the entire contents of which are hereby incorporated herein by reference, as disclosed in a paper entitled “Learning Deep Architectures for AI” by Mikolov, T., Sutskever, I., Chen, K., Corrado, G., and Dean J., in a paper entitled “Distributed Representations of Words and Phrases and their Compositionality” in the proceedings of the 26th International Conference on Neural Information Processing Systems (NIPS), 2013, pp. 3111-3119, the entire contents of which are hereby incorporated herein by reference, etc.) applied to the transaction data to generate a first set of features. For example, applying complex feature representation learning to the transaction data may generate complex features from the transaction data. In such examples, the complex features may represent one or more attributes or parameters associated with one or more transactions.

[0105] In some non-limiting embodiments or aspects, the transaction service provider system 108 may generate second training data including a second set of features based on the transaction data. For example, and also with reference to Figure 4 In the training phase, at reference numeral 404, the transaction service provider system 108 can generate a second set of features by applying one or more existing feature engineering processes to the transaction data. For example, applying the feature engineering process to the transaction data can generate interpretable features from the transaction data. In such examples, the interpretable features can represent one or more attributes or parameters associated with one or more transactions.

[0106] In some non-limiting embodiments or aspects, the transaction data includes transaction parameters associated with a transaction, such as a payment transaction initiated and / or conducted through an e-wallet application, and / or the like. Non-limiting examples of transaction parameters include: e-wallet card data, decision data, authorization data, PAN, transaction amount, transaction date and time, currency conversion rate, merchant type, acquirer country, PAN country, response code, merchant name / location, currency type, and / or the like. The response code may refer to successful approval / completion of the transaction, rejection due to reported lost or stolen card, non-acceptance, partial approval, VIP approval (VIP program), amount exceeds maximum, insufficient funds, incorrect PIN, suspected fraud, exceeded activity, exceeded allowable PIN entry attempts, and / or the like. In some non-limiting embodiments or aspects, the transaction data includes classification labels associated with transactions (e.g., previous transactions, etc.), such as true classification labels (e.g., labels indicating that the transaction is associated with fraud, labels indicating that the transaction is not associated with fraud, labels for multiple sets of positive and negative training examples or samples, etc.).

[0107] In some non-limiting embodiments or aspects, the electronic wallet card data includes one or more of the following data: data associated with an identifier regarding a portable financial device to be provided to an electronic wallet application, data associated with an identifier of an issuer associated with a portable financial device to be provided to an electronic wallet application, data associated with an identifier of a transaction service provider associated with a portable financial device to be provided to an electronic wallet application, data associated with a name of a user associated with a portable financial device to be provided to an electronic wallet application, data associated with an account identifier of an account associated with a portable financial device to be provided to an electronic wallet application, and / or the like.

[0108] like Figure 3 As shown in , at step 304, process 300 includes training a first model based on first training data and second training data. For example, the transaction service provider system 108 may train the first model based on the first training data and the second training data. For example, the transaction service provider system 108 may train a first model (e.g., a more complex model, a teacher model, etc.) based on the first training data and the second training data.

[0109] In some non-limiting embodiments or aspects, the first model includes a greater number of parameters than the second model (eg, the first model includes a more complex model than the second model, etc.). For example, and also with reference to Figure 4In the training phase, at reference numeral 406, the transaction service provider system 108 can train a more complex model for the complex features and the explainable features. In this example, the transaction service provider system 108 can train the more complex model with the true classification labels of the transactions (e.g., previous transactions, etc.) represented by the complex features and the explainable features.

[0110] In some non-limiting embodiments or aspects, the first model includes at least one of the following: a deep neural network, a recurrent neural network, an integration of multiple neural networks, or any combination thereof. For example, the first model may include a recurrent neural network model, the recurrent neural network model includes multiple long short-term memory (LSTM) layers (e.g., two LSTM layers, etc.), and each LSTM layer may include multiple hidden nodes (e.g., each LSTM layer may include 256 hidden nodes, etc.).

[0111] like Figure 3 As shown in , at step 306, the process 300 includes training the second model based on the second training data using a loss function that depends on the output of the intermediate layer of the first model and the output of the second model. For example, the transaction service provider system 108 may train the second model based on the second training data using a loss function that depends on the output of the intermediate layer of the first model and the output of the second model. For example, the transaction service provider system 108 may train the second model (e.g., a simpler model, a student model, etc.) based on the second training data using a loss function that depends on the output of the intermediate layer of the first model and the output of the second model.

[0112] In some non-limiting embodiments or aspects, the intermediate layer comprises a hidden layer of the neural network. For example, the output of the intermediate layer of the first model may include an output provided directly from the hidden layer of the neural network of the first model. For example, the output of the intermediate layer of the first model may include a hidden state vector. In such examples, the intermediate layer may be located between an input layer and an output or final layer of the neural network. In some non-limiting embodiments or aspects, the output or final layer of the neural network may include a classification layer (e.g., a softmax layer, etc.), a logits layer, a final or last neuron layer, and / or the like of the neural network.

[0113] In some non-limiting embodiments or aspects, the second model is trained using only the second training data combined with the knowledge learned from the first model. For example, and also referring to Figure 4 In the training phase, at reference numeral 408, the transaction service provider system 108 may train a simpler model using only the interpretable features in combination with knowledge learned from the more complex model.

[0114] In some non-limiting embodiments or aspects, the second model includes at least one first layer and at least one second layer. In some non-limiting embodiments or aspects, the first layer of the second model includes a recurrent neural network, and the second layer of the second model includes a logistic regression model. For example, the transaction service provider system 108 may use a simple recurrent neural network model to learn to match the output of an intermediate layer from a more complex model. For example, the intermediate layer of the more complex model may have a much higher dimension than the final layer of the more complex model. In such examples, the transaction service provider system 108 may use a recurrent neural network to generate a feature vector used as an input to the logistic regression model, and the combination of the recurrent neural network and the logistic regression model may form a simpler model.

[0115] In some non-limiting embodiments or aspects, the transaction service provider system 108 may determine multiple information values ​​for multiple intermediate layers of the first model, and select an intermediate layer from the multiple intermediate layers based on the multiple information values. For example, the information value is a metric used to rank features according to their predictive power. For example, a larger feature information value may indicate a more predictive feature, while a smaller feature information value may indicate a less predictive feature. In such examples, each hidden state of the first model (e.g., a more complex model, a teacher model, etc.) may be considered a feature and divided into M categories or buckets, and the information value IV may be calculated according to the following formula (1):

[0116]

[0117]

[0118] In equation (1), F i and NF i are the number of fraud and non-fraud in category i, and and are the number of fraud and non-fraud in the whole population respectively. It should be noted that IV is a non-negative number and is positive if and only if F i / F and NF i / NF is zero when i=1, 2, ..., and M are the same.

[0119] In some non-limiting embodiments or aspects, the first model may include a recurrent neural network model including two LSTM hidden layers. Figure 5A and 5B, the two figures illustrate a non-limiting example or aspect of an implementation scheme 500 for determining the information value of a process for training, providing and / or using a predictive model disclosed herein that incorporates knowledge from a more complex model, where the upper LSTM layer hidden states from a recurrent neural network model can be used as a target for training a second model because the upper LSTM layer hidden states are more predictive than other network states in the recurrent neural network model as measured by information value. For example, such a recurrent neural network model including two LSTM hidden layers may be extremely complex while providing a fairly significant performance improvement over a simpler baseline model. For example, as shown in FIG. 5 , the information value of the cell state and the hidden state in the two LSTM layers is Figure 5B As shown in , the upper LSTM state has a higher IV value than the lower LSTM state, which indicates that the upper LSTM network state should be more predictive. However, non-limiting embodiments or aspects are not limited to the first model including such a recurrent neural network model including two LSTM hidden layers, and the first model may include a more complex model (e.g., an integration of multiple recurrent neural networks, etc.) to provide a stronger teacher model.

[0120] In some non-limiting embodiments or aspects, the output of the second model includes the output of at least one first layer, and training the second model further includes modifying one or more parameters of the at least one first layer of the second model using a loss function that depends on the output of the intermediate layer of the first model and the output of the second model including the output of the first layer. For example, the transaction service provider system 108 may extract a vector from the intermediate layer of the first model to serve as a target for the first layer of the second model. For example, the transaction service provider system 108 may train the first layer of the second model by optimizing a loss function (e.g., L2 loss, etc.) between the output of the first layer of the second model and the vector output from the intermediate layer of the first model.

[0121] like Figure 5A As shown in , the first model may include a recurrent neural network model including two LSTM hidden layers each having 256 hidden nodes. For example, the transaction service provider system 108 may extract a hidden state vector denoted as h(t) in the following equation (2) from the second LSTM layer of the recurrent neural network, which is a 256-dimensional vector serving as a target for training the second model, which may include a feedforward recurrent neural network denoted as FRNN in the following equation (2) having one hidden layer. In such an example, the second model may be trained by minimizing the L2 loss between the output of the second model and the 256-dimensional target vector from the first model according to the following equation (2):

[0122]

[0123] In equation (2), W1 is the weight matrix between the input feature vector x and the hidden layer of the recurrent neural network, W 2 is the weight matrix from the hidden layer to the target unit, and N is the number of training examples. The parameters W can be updated using the standard back-propagation algorithm with momentum term 1 and W 2 Still refer to Figure 5A , the hidden state vector h(t) from the recurrent neural network is bounded between -1 and 1, because the hidden state vector h(t) is calculated as h(t) = o(t)⊙tanh(c(t)), where o(t) is the output from the output gate in the recurrent neural network, which has a value from 0 to 1, and c(t) is an unbounded cell state. The operation ⊙ represents an element-wise product, such as a TANH layer vector, which can be used as an activation function in a recurrent neural network model to ensure that the output from the second model has a value between -1 and 1. After training, the parameters of the recurrent neural network can be fixed. For each training example x(t), the recurrent neural network can produce a feature vector, which can be used as an input to a logistic regression model.

[0124] In some non-limiting embodiments or aspects, training the second model further comprises training at least one second layer based on the output of at least one first layer. For example, the transaction service provider system 108 may train the second layer of the second model using input from the first layer of the second model with true classification labels (e.g., with positive and negative examples or samples, etc.) of transactions in the transaction data log or database (e.g., previous transactions, etc.). For example, a logistic regression model may be trained using input (e.g., feature vectors, etc.) output from a regression neural network model with true classification labels of transactions in the transaction data log or database.

[0125] like Figure 3 As shown in , at step 308, process 300 includes providing a trained second model. For example, transaction service provider system 108 may provide a trained second model. For example, transaction service provider system 108 may provide (e.g., via communication network 114) a trained second model (e.g., a trained simpler model, a trained student model, etc.) for use at merchant system 102, payment gateway 104, acquirer system 106, transaction service provider system 108, issuer system 110, and / or user device 112. In such examples, the trained second model may include a hierarchical model including a regression neural network as a first layer and a logistic regression model as a second layer.

[0126] In some non-limiting embodiments or aspects, the transaction service provider system 108 provides interpretable features (e.g., one or more feature engineering processes or functions for generating interpretable features based on transaction data, etc.) and a trained second model. For example, and also with reference to Figure 4 , at reference number 410, the transaction service provider system 108 provides (e.g., stores, transmits, etc.) interpretable features (e.g., one or more feature engineering processes or functions for generating interpretable features based on transaction data, etc.) and a trained second model (e.g., a trained simpler model, etc.).

[0127] like Figure 3 As shown in , at step 310, process 300 includes obtaining input data. For example, transaction service provider system 108 may obtain input data. For example, transaction service provider system 108 may obtain input data associated with at least one transaction.

[0128] In some non-limiting embodiments or aspects, the transaction service provider system 108 may generate input data based on the transaction data. For example, and also with reference to Figure 4 In the prediction phase, at reference numeral 410, the transaction service provider system 108 may generate interpretable features associated with the at least one transaction by applying one or more existing feature engineering processes to the transaction data associated with the at least one transaction. For example, when a new transaction is received in the transaction processing network 101, the transaction service provider system 108 may generate interpretable features associated with the new transaction for input to a trained second model (e.g., a trained simpler model, etc.).

[0129] like Figure 3 As shown in , at step 312, process 300 includes processing the input data using the trained second model to generate output data. For example, the transaction service provider system 108 may process the input data using the trained second model to generate output data. For example, the transaction service provider system 108 may process the input data using the trained second model to generate output data (e.g., prediction, probability, certainty, likelihood, percentage, binary output, yes-no output, etc.).

[0130] In some non-limiting embodiments or aspects, the output data includes a prediction of whether the at least one transaction is a fraudulent transaction (e.g., a risk prediction score, etc.). For example, the transaction service provider system 108 may determine a probability (e.g., certainty, likelihood, percentage, binary output, yes-no output, etc.) that the at least one transaction is a fraudulent transaction by processing the input data using the trained second model to generate output data including the prediction. For example, the transaction service provider system 108 generates a probability that the at least one transaction is a fraudulent transaction by receiving input data associated with the at least one transaction as input to the trained second model and providing a probability that the at least one transaction is a fraudulent transaction as an output from the trained second model.

[0131] In some non-limiting embodiments or aspects, the transaction service provider system 108 may generate output data using only interpretable features associated with new transactions. For example, and also with reference to Figure 4 , in the prediction stage, at reference numeral 412, the transaction service provider system 108 may generate a prediction of whether the new transaction is a fraudulent transaction by receiving interpretable features (e.g., interpretable feature vectors, etc.) associated with the new transaction as inputs to a simpler model at reference numeral 414 and providing a risk prediction score as an output from the simpler model including a probability (e.g., certainty, likelihood, percentage, binary output, yes-no output, etc.) that the new transaction is a fraudulent transaction.

[0132] Table 1 below provides a comparison of the performance results of the implementation scheme 500 and the benchmark model. For example, Table 1 shows a teacher model including a recursive neural network model having two LSTM layers with 256 hidden nodes each as described herein with respect to the implementation scheme 500, a student model including a recursive neural network as a first layer and a logistic regression model as a second layer as described with respect to the implementation scheme 500, and a benchmark model including a simple neural network model having the same parameters as the student model but without knowledge transfer from the teacher model, measured according to the area under the curve (AUC). For example, the higher the AUC value, the better the performance of the model.

[0133] Model AUC teacher 0.961 student 0.944 Benchmarks 0.931

[0134] Table 1

[0135] In such examples, the teacher model may include about 1.8 million parameters and the student model may include about 290,000 parameters. Therefore, the teacher model may be considered as only an extremely complex model, but such an extremely complex model is sufficient to illustrate the performance improvement that can be achieved by a non-limiting embodiment or aspect of the process for training, providing and / or using a prediction model that incorporates knowledge from a more complex model. For example, the complexity of the teacher model can be increased (e.g., by using an ensemble of a set of models, etc.) to improve the teacher's generalization ability for training the student model. In such examples, the teacher model can be trained offline because the teacher model may not operate in real time, which enables the teacher model to use a larger data set and a larger amount of computing resources. For example, the complexity of the teacher model can be infinite due to the decoupling of model training and model deployment, which can achieve the use of a much more complex model with a large capacity and strong generalization ability as a teacher model to transfer knowledge to a student model that can be easily deployed in an existing system architecture for real-time prediction (e.g., for real-time fraud prediction during receiving and / or processing new transactions in the transaction processing network 101, etc.).

[0136] Although the above methods, systems, and computer program products have been described in detail based on what are currently considered to be the most practical and preferred embodiments or aspects for the purpose of illustration, it should be understood that such details are only for the purpose of illustration, and the present disclosure is not limited to the described embodiments or aspects; on the contrary, the present disclosure is intended to cover modifications and equivalent arrangements within the spirit and scope of the appended claims. For example, it should be understood that the present disclosure contemplates that, to the extent possible, one or more features of any embodiment or aspect may be combined with one or more features of any other embodiment or aspect.

Claims

1. A computer-implemented method comprising: obtaining, by at least one processor, first training data associated with a first set of features and second training data associated with a second set of features different from the first set of features; training, by at least one processor, a first model based on the first training data and the second training data; as well as training, by at least one processor, a second model based on the second training data using a loss function that depends on an output of an intermediate layer of the first model and an output of the second model, The first model includes a recurrent neural network, the recurrent neural network includes two LSTM hidden layers, the two LSTM hidden layers include an intermediate layer of the first model, wherein the output of the intermediate layer of the first model includes a hidden state vector h(t) from a second LSTM hidden layer of the two LSTM hidden layers, wherein the second model includes a feedforward neural network FRNN, The loss function between the output of the second model and the output from the intermediate layer of the first model is minimized according to the following formula: Wherein, W1 is the weight matrix between the input feature vector x and the hidden layer of the recurrent neural network, W2 is the weight matrix between the hidden layer to the target unit, and N is the number of training examples, and x(t) is the training example.

2. The computer-implemented method of claim 1 , wherein the second model comprises at least one first layer and at least one second layer, wherein the output of the second model comprises an output of the at least one first layer, and wherein training the second model further comprises: modifying one or more parameters of the at least one first layer of the second model using the loss function that depends on the output of the intermediate layer of the first model and the output of the second model, wherein the output of the second model includes the output of the first layer; as well as The at least one second layer is trained based on the output of the at least one first layer.

3. The computer-implemented method of claim 2, wherein the first model further comprises at least one of: a deep neural network, an ensemble of multiple neural networks, or any combination thereof, wherein the first layer of the second model comprises a regression neural network, and wherein the second layer of the second model comprises a logistic regression model.

4. The computer-implemented method of claim 1 , further comprising: determining, by at least one processor, a plurality of information values ​​of a plurality of intermediate layers of the first model; as well as The intermediate layer is selected, by at least one processor, from among the plurality of intermediate layers based on the plurality of information values.

5. The computer-implemented method of claim 1, wherein the first set of features comprises complex features, and wherein the second set of features comprises interpretable features. The computer-implemented method of claim 1 , wherein the first model includes a greater number of parameters than the second model.

7. The computer-implemented method of claim 1 , further comprising: providing, by at least one processor, a trained second model; obtaining, by at least one processor, input data associated with at least one transaction; as well as The input data is processed by at least one processor and using the trained second model to generate output data, wherein the output data includes a prediction of whether the at least one transaction is a fraudulent transaction.

8. A computing system comprising: at least one processor programmed and / or configured to: obtaining first training data associated with a first set of features and second training data associated with a second set of features different from the first set of features; training a first model based on the first training data and the second training data; and training a second model based on the second training data using a loss function that depends on an output of an intermediate layer of the first model and an output of the second model, The first model includes a recurrent neural network, the recurrent neural network includes two LSTM hidden layers, the two LSTM hidden layers include an intermediate layer of the first model, wherein the output of the intermediate layer of the first model includes a hidden state vector h(t) from a second LSTM hidden layer of the two LSTM hidden layers, wherein the second model includes a feedforward neural network FRNN, The loss function between the output of the second model and the output from the intermediate layer of the first model is minimized according to the following formula: Wherein, W1 is the weight matrix between the input feature vector x and the hidden layer of the recurrent neural network, W2 is the weight matrix between the hidden layer to the target unit, and N is the number of training examples, and x(t) is the training example.

9. The computing system of claim 8, wherein the second model comprises at least one first layer and at least one second layer, wherein the output of the second model comprises the output of the at least one first layer, and wherein the at least one processor is further programmed and / or configured to train the second model by: modifying one or more parameters of the at least one first layer of the second model using the loss function that depends on the output of the intermediate layer of the first model and the output of the second model, wherein the output of the second model includes the output of the first layer; and The at least one second layer is trained based on the output of the at least one first layer.

10. The computing system of claim 9, wherein the first model further comprises at least one of: a deep neural network, an ensemble of multiple neural networks, or any combination thereof, wherein the first layer of the second model comprises a regression neural network, and wherein the second layer of the second model comprises a logistic regression model.

11. The computing system of claim 8, wherein the at least one processor is further programmed and / or configured to: determining a plurality of information values ​​of a plurality of intermediate layers of the first model; and The intermediate layer is selected from the plurality of intermediate layers based on the plurality of information values.

12. The computing system of claim 8, wherein the first set of features comprises complex features, and wherein the second set of features comprises interpretable features.

13. The computing system of claim 8, wherein the first model includes a greater number of parameters than the second model.

14. The computing system of claim 8, wherein the at least one processor is further programmed and / or configured to: providing a trained second model; obtaining input data associated with at least one transaction; and The input data is processed using a trained second model to generate output data, wherein the output data includes a prediction of whether the at least one transaction is a fraudulent transaction.

15. A computer program product comprising at least one non-transitory computer readable medium, the at least one non-transitory computer readable medium comprising program instructions that, when executed by at least one processor, cause the at least one processor to: obtaining first training data associated with a first set of features and second training data associated with a second set of features different from the first set of features; training a first model based on the first training data and the second training data; and training a second model based on the second training data using a loss function that depends on an output of an intermediate layer of the first model and an output of the second model, The first model includes a recurrent neural network, the recurrent neural network includes two LSTM hidden layers, the two LSTM hidden layers include an intermediate layer of the first model, wherein, The output of the intermediate layer of the first model includes a hidden state vector h(t) from a second LSTM hidden layer of the two LSTM hidden layers, wherein the second model includes a feed-forward neural network FRNN, The loss function between the output of the second model and the output from the intermediate layer of the first model is minimized according to the following formula: Wherein, W1 is the weight matrix between the input feature vector x and the hidden layer of the recurrent neural network, W2 is the weight matrix between the hidden layer to the target unit, and N is the number of training examples, and x(t) is the training example.

16. The computer program product of claim 15, wherein the second model comprises at least one first layer and at least one second layer, wherein the output of the second model comprises an output of the at least one first layer, and wherein the instructions further cause the at least one processor to train the second model by: modifying one or more parameters of the at least one first layer of the second model using the loss function that depends on the output of the intermediate layer of the first model and the output of the second model, wherein the output of the second model includes the output of the first layer; and The at least one second layer is trained based on the output of the at least one first layer.

17. The computer program product of claim 15, wherein the instructions further cause the at least one processor to: determining a plurality of information values ​​of a plurality of intermediate layers of the first model; and The intermediate layer is selected from the plurality of intermediate layers based on the plurality of information values.

18. The computer program product of claim 15, wherein the first set of features comprises complex features, and wherein the second set of features comprises interpretable features.

19. The computer program product of claim 15, wherein the first model includes a greater number of parameters than the second model.

20. The computer program product of claim 15, wherein the instructions further cause the at least one processor to: providing a trained second model; obtaining input data associated with at least one transaction; and The input data is processed using a trained second model to generate output data, wherein the output data includes a prediction of whether the at least one transaction is a fraudulent transaction.