Systems, methods, and computer program products for adaptive feature optimization during unsupervised training of classification models

Abstract

This invention provides a system comprising a processor configured to: receive a training dataset comprising a plurality of data records; compute feature projection errors and classification scores for the plurality of data records using a machine learning model; determine a distribution of features based on the feature projection errors; apply a scaling function to each feature value of a feature in a false positive classification distribution to provide a scaled-down set of feature values; apply a scaling function to each feature value of a feature in a false negative classification distribution to provide a scaled-up set of feature values; combine the scaled-down set of feature values ​​and the scaled-up set of feature values ​​with the training dataset to provide an updated training dataset; and train a machine learning model using the updated training dataset to provide an updated trained machine learning model. Methods and computer program products are also provided.

Description

Technical Field

[0001] This disclosure generally relates to feature optimization of machine learning models, and in some non-limiting embodiments or aspects, relates to systems, methods, and computer program products for adaptive feature optimization of machine learning models. Background Technology

[0002] Machine learning can be a field of computer science that uses statistical techniques to enable computer systems to learn from data (e.g., progressively improve performance) for tasks without explicitly programming the computer system to perform those tasks. In some cases, machine learning models can be developed for datasets that can perform tasks related to that dataset (e.g., tasks associated with prediction).

[0003] Features of a machine learning model can include attributes (e.g., features, properties, etc.) shared by all independent units of the dataset that the machine learning model will analyze. These features can have values ​​associated with the attributes, such as numerical values. Additionally, feature importance can refer to a measure of a feature's contribution to the analytical output (such as prediction or classification) of the machine learning model. Feature importance can be used to understand the behavior of machine learning models, detect errors in the dataset to avoid potential failures during the implementation of machine learning models, and validate governance processes associated with machine learning models.

[0004] In some cases, the importance of a feature in a machine learning model can be obtained by changing the value of the feature in each data record of the dataset (e.g., by removing the feature value to give the feature a value of 0 in each data record of the dataset, or by replacing the feature value in each data record of the dataset with a single value such as the value 1), to provide a modified dataset, which can then be used to retrain the machine learning model. The performance of the retrained machine learning model for that feature can be determined based on the evaluation data.

[0005] However, retraining a machine learning model can be time-consuming when every single feature in a dataset changes. This is especially true for deep learning models, which in some cases require extensive training and rely heavily on a large number of input features. Furthermore, when every single feature in a dataset changes, the machine learning model may need to be retrained more than once to ensure correct results. Summary of the Invention

[0006] Therefore, systems, devices, products, apparatuses and / or methods for adaptive feature optimization of machine learning models are disclosed, which overcome some or all of the defects of the prior art.

[0007] According to some non-limiting embodiments or aspects, a system is provided, comprising: at least one processor programmed or configured to: receive a training dataset comprising a plurality of data records, each data record comprising a plurality of feature values ​​of a plurality of features; compute a feature projection error for each feature among the plurality of features in each of the plurality of data records using a trained machine learning model; compute a classification score for each of the plurality of data records using the trained machine learning model; and determine a distribution of features based on the classification score of each data record and according to the feature projection error of each feature among the plurality of features in each data record, wherein the distribution of features according to the feature projection error includes: a false positive classification distribution of the features, the false positive classification distribution including the feature values ​​of each feature among the features in each data record having a false positive classification. The distribution of feature projection error for each of the multiple features; and the false negative classification distribution of the features, which includes the distribution of feature projection error for each of the multiple features in each data record having a false negative classification; applying a scaling function to each feature value of the feature with the highest projection error value in the false positive classification distribution to provide a scaled feature value set; applying a scaling function to each feature value of the feature with the lowest projection error value in the false negative classification distribution to provide a scaled feature value set; and combining the scaled feature value set and the scaled feature value set with the training dataset to provide an updated training dataset; and using the updated training dataset to train the training machine learning model to provide an updated training machine learning model.

[0008] According to some non-limiting embodiments or aspects, a computer-implemented method is provided, comprising: receiving, using at least one processor, a training dataset including a first plurality of data records, each data record including a plurality of feature values ​​of a plurality of features; using a trained machine learning model, using the at least one processor to calculate a feature projection error for each of the plurality of features in each of the second plurality of data records; using the trained machine learning model, using the at least one processor to calculate a classification score for the second plurality of data records; and using the at least one processor to determine a feature distribution based on the classification score of each of the second plurality of data records, according to the feature projection error of each of the plurality of features in the second plurality of data records, wherein the feature distribution according to the feature projection error includes: a false positive classification distribution of the features, the false positive classification distribution including a distribution based on the feature projection error of each of the plurality of features in each data record having a false positive classification. The distribution of feature projection error for each feature in the feature set; and the false negative classification distribution of the feature, which includes the distribution of feature projection error for each of the plurality of features in each data record having a false negative classification; applying a scaling function to each feature value of the feature having the highest projection error value in the false positive classification distribution using at least one processor to provide a scaled feature value set; applying a scaling function to each feature value of the feature having the lowest projection error value in the false negative classification distribution using at least one processor to provide a scaled feature value set; combining the scaled feature value set and the scaled feature value set with the training dataset using at least one processor to provide an updated training dataset; and training the trained machine learning model using the updated training dataset using at least one processor to provide an updated trained machine learning model.

[0009] According to some non-limiting embodiments or aspects, a computer program product is provided comprising at least one non-transient computer-readable medium, the at least one non-transient computer-readable medium comprising one or more instructions, which, when executed by at least one processor, cause the at least one processor to: receive a training dataset comprising a plurality of data records, each data record comprising a plurality of feature values ​​of a plurality of features; compute a feature projection error for each of the plurality of features in each of the plurality of data records using a trained machine learning model; compute a classification score for each of the plurality of data records using the trained machine learning model; and determine a distribution of features based on the classification score of each data record and according to the feature projection error of each of the plurality of features in each data record, wherein the distribution of feature projection errors includes: a false positive classification distribution of the features, the false positive classification distribution... The method includes: a distribution of features representing the projection error of each of the plurality of features in each data record having a false positive classification; a false negative classification distribution of features, the false negative classification distribution including the distribution of features representing the projection error of each of the plurality of features in each data record having a false negative classification; applying a scaling function to each feature value of the feature having the highest projection error value in the false positive classification distribution to provide a scaled feature value set; applying a scaling function to each feature value of the feature having the lowest projection error value in the false negative classification distribution to provide a scaled feature value set; combining the scaled feature value set and the scaled feature value set with a training dataset to provide an updated training dataset; and using the updated training dataset to train the trained machine learning model to provide an updated trained machine learning model.

[0010] Other implementation schemes are described in the following numbered clauses:

[0011] Clause 1: A system comprising: at least one processor programmed or configured to: receive a training dataset comprising a plurality of data records, each data record comprising a plurality of feature values ​​of a plurality of features; compute a feature projection error for each of the plurality of features in each of the plurality of data records using a trained machine learning model; compute a classification score for each of the plurality of data records using the trained machine learning model; and determine a distribution of features based on the classification score of each data record and according to the feature projection error of each of the plurality of features in each data record, wherein the distribution of features according to the feature projection error includes: a false positive classification distribution of the features, the false positive classification distribution including each of the plurality of features in each data record having a false positive classification. The distribution of feature projection error; and the false negative classification distribution of features, which includes the distribution of feature projection error of each of the plurality of features in each data record having a false negative classification; applying a scaling function to each feature value of the feature with the highest projection error value in the false positive classification distribution to provide a scaled feature value set; applying a scaling function to each feature value of the feature with the lowest projection error value in the false negative classification distribution to provide a scaled feature value set; and combining the scaled feature value set and the scaled feature value set with the training dataset to provide an updated training dataset; and using the updated training dataset to train the trained machine learning model to provide an updated trained machine learning model.

[0012] Clause 2: The system as described in Clause 1, wherein the at least one processor is further programmed or configured to: determine a performance metric of the updated trained machine learning model; and determine, based on the performance metric, whether further training of the updated trained machine learning model is necessary.

[0013] Clause 3: A system as described in Clause 1 or Clause 2, wherein the scaling reduction function includes a lower limit scalar value and an upper limit scalar value, and wherein the scaling reduction function is configured such that: the characteristic values ​​between the lower limit scalar value and the upper limit scalar value remain unchanged; characteristic values ​​below the lower limit scalar value are changed to the lower limit scalar value; and characteristic values ​​above the upper limit scalar value are changed to the upper limit scalar value.

[0014] Clause 4: The system of any one of Clauses 1 to 3, wherein the scaling function comprises a lower limit scalar value, an upper limit scalar value, and an intermediate value, and wherein the scaling function is configured such that: feature values ​​below the lower limit scalar value and above the upper limit scalar value remain unchanged; feature values ​​between the lower limit scalar value and the intermediate value are changed to the lower limit scalar value; and feature values ​​between the upper limit scalar value and the intermediate value are changed to the upper limit scalar value.

[0015] Clause 5: The system of any one of Clauses 1 to 4, wherein the at least one processor is further programmed or configured to: determine the lower limit scalar value and the upper limit scalar value of the scaling down function; and determine the lower limit scalar value, the intermediate value, and the upper limit scalar value of the scaling up function.

[0016] Clause 6: The system as described in any one of Clauses 1 to 5, wherein when determining the lower limit scalar value and the upper limit scalar value of the scaling reduction function, the at least one processor is programmed or configured to: determine the lower limit scalar value and the upper limit scalar value of the scaling reduction function based on the Mann-Whitney test; and wherein when determining the lower limit scalar value, the intermediate value, and the upper limit scalar value of the scaling magnification function, the at least one processor is programmed or configured to: determine the lower limit scalar value, the intermediate value, and the upper limit scalar value of the scaling magnification function based on the Mann-Whitney test.

[0017] Clause 7: A system as described in any one of Clauses 1 to 6, wherein the trained machine learning model is an unsupervised binary classification machine learning model, and wherein the unsupervised binary classification machine learning model is an autoencoder.

[0018] Clause 8: A computer-implemented method comprising: receiving, using at least one processor, a training dataset comprising a first plurality of data records, each data record comprising a plurality of feature values ​​of a plurality of features; using a trained machine learning model, using the at least one processor, calculating a feature projection error for each of the plurality of features in each of the second plurality of data records; using the trained machine learning model, using the at least one processor, calculating a classification score for the second plurality of data records; using the at least one processor, determining a feature distribution based on the classification score of each of the second plurality of data records, according to the feature projection error of each of the plurality of features in the second plurality of data records, wherein the feature distribution according to the feature projection error comprises: a false positive classification distribution of the features, the false positive classification distribution comprising, according to, the feature projection error of each of the plurality of features in each data record having a false positive classification. The distribution of features of projection error; and the distribution of false negative classification of features, the false negative classification distribution comprising the distribution of features of projection error of each of the plurality of features in each data record having a false negative classification; applying a scaling function to each feature value of the feature having the highest projection error value in the false positive classification distribution using at least one processor to provide a scaled feature value set; applying a scaling function to each feature value of the feature having the lowest projection error value in the false negative classification distribution using at least one processor to provide a scaled feature value set; combining the scaled feature value set and the scaled feature value set with a training dataset using at least one processor to provide an updated training dataset; and training the trained machine learning model using the updated training dataset using at least one processor to provide an updated trained machine learning model.

[0019] Clause 9: The computer-implemented method as described in Clause 8 further includes: determining a performance metric for the updated trained machine learning model; and determining, based on the performance metric, whether further training procedures for the updated trained machine learning model are necessary.

[0020] Clause 10: A computer-implemented method as described in Clause 8 or 9, wherein the scaling function includes a lower limit scalar value and an upper limit scalar value, and wherein the scaling function is configured such that: characteristic values ​​between the lower limit scalar value and the upper limit scalar value remain unchanged; characteristic values ​​below the lower limit scalar value are changed to the lower limit scalar value; and characteristic values ​​above the upper limit scalar value are changed to the upper limit scalar value.

[0021] Clause 11: A computer-implemented method as described in any one of Clauses 8 to 10, wherein the scaling function comprises a lower limit scalar value, an upper limit scalar value, and an intermediate value, and wherein the scaling function is configured such that: feature values ​​below the lower limit scalar value and above the upper limit scalar value remain unchanged; feature values ​​between the lower limit scalar value and the intermediate value are changed to the lower limit scalar value; and feature values ​​between the upper limit scalar value and the intermediate value are changed to the upper limit scalar value.

[0022] Clause 12: The computer-implemented method as described in any one of Clauses 8 to 11 further comprises: determining a lower limit scalar value and an upper limit scalar value of the scaling down function; and determining a lower limit scalar value, an intermediate value, and an upper limit scalar value of the scaling up function.

[0023] Clause 13: A computer-implemented method as described in any one of Clauses 8 to 12, wherein determining the lower limit scalar value and the upper limit scalar value of the scaling reduction function comprises: determining the lower limit scalar value and the upper limit scalar value of the scaling reduction function based on a Mann-Whitney test; and wherein determining the lower limit scalar value, the intermediate value, and the upper limit scalar value of the scaling magnification function comprises: determining the lower limit scalar value, the intermediate value, and the upper limit scalar value of the scaling magnification function based on the Mann-Whitney test.

[0024] Clause 14: A computer-implemented method as described in any one of Clauses 8 to 13, wherein the trained machine learning model is an unsupervised binary classification machine learning model, and wherein the unsupervised binary classification machine learning model is an autoencoder.

[0025] Clause 15: A computer program product comprising at least one non-transient computer-readable medium, the at least one non-transient computer-readable medium comprising one or more instructions, which, when executed by at least one processor, cause the at least one processor to: receive a training dataset comprising a plurality of data records, each data record comprising a plurality of feature values ​​of a plurality of features; compute a feature projection error for each of the plurality of features in each of the plurality of data records using a trained machine learning model; compute a classification score for each of the plurality of data records using the trained machine learning model; and determine a distribution of features based on the classification score of each data record and according to the feature projection error of each of the plurality of features in each data record, wherein the distribution of feature projection errors includes: a false positive classification distribution of features, the false positive classification distribution including a distribution based on the false positive classification error of features. The distribution of feature projection error for each of the plurality of features in each data record with a positive classification; and the false negative classification distribution of the features, which includes the distribution of feature projection error for each of the plurality of features in each data record with a false negative classification; applying a scaling function to each feature value of the feature with the highest projection error value in the false positive classification distribution to provide a scaled feature value set; applying a scaling function to each feature value of the feature with the lowest projection error value in the false negative classification distribution to provide a scaled feature value set; and combining the scaled feature value set and the scaled feature value set with the training dataset to provide an updated training dataset; and using the updated training dataset to train the trained machine learning model to provide an updated trained machine learning model.

[0026] Clause 16: A computer program product as described in Clause 15, wherein one or more instructions further cause the at least one processor to: determine a performance metric of the updated trained machine learning model; and determine, based on the performance metric, whether further training of the updated trained machine learning model is necessary.

[0027] Clause 17: A computer program product as described in Clause 15 or 16, wherein the scaling function includes a lower limit scalar value and an upper limit scalar value, and wherein the scaling function is configured such that: characteristic values ​​between the lower limit scalar value and the upper limit scalar value remain unchanged; characteristic values ​​below the lower limit scalar value are changed to the lower limit scalar value; and characteristic values ​​above the upper limit scalar value are changed to the upper limit scalar value.

[0028] Clause 18: A computer program product as described in any one of Clauses 15 to 17, wherein the scaling function comprises a lower limit scalar value, an upper limit scalar value, and an intermediate value, and wherein the scaling function is configured such that: feature values ​​below the lower limit scalar value and above the upper limit scalar value remain unchanged; feature values ​​between the lower limit scalar value and the intermediate value are changed to the lower limit scalar value; and feature values ​​between the upper limit scalar value and the intermediate value are changed to the upper limit scalar value.

[0029] Clause 19: A computer program product as described in any one of Clauses 15 to 18, wherein one or more of the instructions further cause the at least one processor to: determine a lower limit scalar value and an upper limit scalar value of the scaling down function; and determine a lower limit scalar value, an intermediate value, and an upper limit scalar value of the scaling up function.

[0030] Clause 20: A computer program product as described in any one of Clauses 15 to 19, wherein the trained machine learning model is an unsupervised binary classification machine learning model, and wherein the unsupervised binary classification machine learning model is an autoencoder.

[0031] These and other features and characteristics of this disclosure, as well as the operational methods and manufacturing economies of combinations of related structural elements and parts, will become more apparent when considered in conjunction with the accompanying drawings, all of which form part of this specification, wherein similar reference numerals in the drawings denote corresponding parts. However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to be construed as limiting the scope of this disclosure. Unless the context clearly requires otherwise, the singular forms “a” and “above” as used in this specification and claims include plural indicators. Attached Figure Description

[0032] Additional advantages and details of this disclosure are explained in more detail below with reference to exemplary embodiments illustrated in the accompanying drawings, in which:

[0033] Figure 1 These are illustrations of non-limiting embodiments or aspects of environments in which the systems, devices, products, apparatuses and / or methods described herein can be implemented based on the principles of this disclosure;

[0034] Figure 2 yes Figure 1 A non-limiting illustration of an embodiment or aspect of a component of one or more devices;

[0035] Figure 3 It is a flowchart of a non-limiting implementation or aspect of the process for adaptive feature optimization of a machine learning model; and

[0036] Figures 4A to 4GIt is a diagram of a non-limiting implementation or aspect of the process for adaptive feature optimization of a machine learning model. Detailed Implementation

[0037] For descriptive purposes, the terms “end,” “upper,” “lower,” “right,” “left,” “vertical,” “horizontal,” “top,” “bottom,” “lateral,” “longitudinal,” and their derivatives are intended to refer to the orientation of this disclosure as shown in the accompanying drawings. However, it should be understood that this disclosure may take various alternative variations and sequences of steps, except where expressly specified otherwise. It should also be understood that the specific apparatus and processes illustrated in the drawings and described in the following description are merely exemplary embodiments or aspects of this disclosure. Therefore, unless otherwise indicated, specific dimensions and other physical characteristics associated with embodiments or aspects of embodiments disclosed herein should not be considered limiting.

[0038] The terms "aspect," "component," "element," "structure," "action," "step," "function," and "instruction" used herein should not be construed as critical or necessary unless explicitly stated otherwise. Furthermore, as used herein, the article "a" is intended to include one or more items and is interchangeable with "one or more" and "at least one." Additionally, as used herein, the term "set" is intended to include one or more items (e.g., related items, unrelated items, a combination of related and unrelated items, etc.) and is interchangeable with "one or more" or "at least one." Where only one item is desired, the term "a" or similar language is used. Also, as used herein, the terms "having" and similar terms are intended to be open-ended terms. Furthermore, unless explicitly stated otherwise, the phrase "based on" is intended to mean "at least partially based on." Where appropriate, the phrase "based on" may also mean "in response to."

[0039] As used herein, the terms "communication" and "transmission" can refer to the receiving, accepting, sending, transferring, providing, etc., of information (e.g., data, signals, messages, instructions, commands, etc.). Communication between one unit (e.g., apparatus, system, component of an apparatus or system, combination thereof, etc.) and another unit means that the first unit is able to receive information directly or indirectly from and / or transmit (e.g., send) information to the other unit. This can refer to a direct or indirect connection that is inherently wired and / or wireless. Furthermore, although the transmitted information may be modified, processed, relayed, and / or routed between the first and second units, the two units can also communicate with each other. For example, the first unit can communicate with the second unit even if it passively receives information and does not actively send information to the second unit. As another example, the first unit can communicate with the second unit if at least one intermediate unit (e.g., a third unit located between the first and second units) processes information received from the first unit and sends the processed information to the second unit. In some non-limiting embodiments or aspects, a message can refer to a network packet (e.g., a data packet, etc.) that includes data.

[0040] As used herein, the terms “issuer,” “issuer institution,” “issuer bank,” or “payment device issuer” can refer to one or more entities that provide accounts to individuals (e.g., users, customers, etc.) for making payment transactions such as credit payment transactions and / or debit payment transactions. For example, an issuer institution may provide a customer with an account identifier, such as a primary account number (PAN), that uniquely identifies one or more accounts associated with said customer. In some non-limiting embodiments or aspects, an issuer may be associated with a bank identification number (BIN) that uniquely identifies the issuer institution. As used herein, “issuer system” can refer to one or more computer systems operated by or on behalf of the issuer, such as servers executing one or more software applications. For example, an issuer system may include one or more authorization servers for authorizing transactions.

[0041] As used herein, the term "transaction service provider" can refer to an entity that receives transaction authorization requests from merchants or other entities and, in some cases, provides payment guarantees through an agreement between the transaction service provider and the issuing institution. For example, a transaction service provider may include payment networks, such as... American Or any other entity that processes transactions. As used herein, the term "trading service provider system" can refer to one or more computer systems operated by or on behalf of a trading service provider, such as a trading service provider system executing one or more software applications. A trading service provider system may include one or more processors and, in some non-limiting embodiments or aspects, may be operated by or on behalf of a trading service provider.

[0042] As used herein, the term "merchant" can refer to one or more entities (e.g., operators of retail businesses) that provide goods and / or services and / or access to goods and / or services to users (e.g., customers, consumers) based on transactions such as payment transactions. As used herein, "merchant system" can refer to one or more computer systems operated by or on behalf of a merchant, such as servers executing one or more software applications. As used herein, the term "product" can refer to one or more goods and / or services provided by a merchant.

[0043] As used herein, the term "acquiring party" can refer to an entity authorized and approved by a transaction service provider to initiate transactions (e.g., payment transactions) involving payment devices associated with the transaction service provider. As used herein, the term "acquiring party system" can also refer to one or more computer systems, computer devices, etc., operated by or on behalf of the acquiring party. Transactions that an acquiring party can initiate can include payment transactions (e.g., purchases, Original Credit Transactions (OCT), Account Funds Transactions (AFT), etc.). In some non-restrictive implementations or aspects, the acquiring party may be authorized by the transaction service provider to enter into contracts with merchants or service providers to initiate transactions involving payment devices associated with the transaction service provider. The acquiring party may enter into contracts with payment service providers to enable the payment service providers to provide sponsorships to merchants. The acquiring party may monitor the compliance of payment service providers in accordance with the transaction service provider's regulations. The acquiring party may conduct due diligence on payment service providers and ensure that appropriate due diligence occurs before entering into contracts with sponsored merchants. The acquiring party may be responsible for all transaction service provider programs operated or sponsored by the acquiring party. The acquiring party can be responsible for the actions of the acquiring payment service provider, merchants sponsored by the acquiring payment service provider, and so on. In some non-restrictive implementation schemes or aspects, the acquiring party can be a financial institution, such as a bank.

[0044] As used herein, the term "payment gateway" can refer to an entity and / or a payment processing system operated by or on behalf of such an entity (e.g., a merchant service provider, payment service provider, payment servicer, payment aggregator, payment servicer contracted with an acquirer, payment aggregator, etc.) that provides payment services (e.g., transaction service provider payment services, payment processing services, etc.) to one or more merchants. Payment services may be associated with the use of portable financial devices managed by a transaction service provider. As used herein, the term "payment gateway system" can refer to one or more computer systems, computer devices, servers, server clusters, etc., operated by or on behalf of a payment gateway.

[0045] As used herein, the terms "client" and "client device" can refer to one or more computing devices, such as processors, storage devices, and / or similar computer components that access services available from a server. In some non-limiting embodiments or aspects, a client device can include computing devices configured to communicate with one or more networks and / or facilitate transactions, such as, but not limited to, one or more desktop computers, one or more portable computers (e.g., tablet computers), one or more mobile devices (e.g., cellular phones, smartphones, personal digital assistants, wearable devices such as watches, glasses, lenses, and / or clothing), and / or other similar devices. Furthermore, the term "client" can also refer to an entity that owns, uses, and / or operates a client device to facilitate transactions with another entity.

[0046] As used herein, the term "server" can refer to one or more computing devices, such as processors, storage devices, and / or similar computer components, that communicate with client devices and / or other computing devices on a network such as the Internet or a private network, and in some instances, facilitate communication between other servers and / or client devices.

[0047] As used herein, the term "system" may refer to one or more computing devices or combinations of computing devices, such as, but not limited to, processors, servers, client devices, software applications, and / or other similar components. Furthermore, as used herein, references to "server" or "processor" may refer to the server and / or processor previously stated to perform the preceding steps or functions, different servers and / or processors, and / or combinations of servers and / or processors. For example, as used in the specification and claims, a first server and / or first processor stated to perform a first step or function may refer to the same or different servers and / or processors stated to perform a second step or function.

[0048] This document describes some non-limiting implementations or aspects in conjunction with thresholds. As used herein, satisfying a threshold can refer to a value that is greater than, more than, higher than, greater than or equal to, less than, less than, lower than, less than or equal to, or equal to a threshold.

[0049] This disclosure relates to systems, methods, and computer program products for adaptive feature optimization of machine learning models. In some non-limiting embodiments or aspects, the feature management system may include at least one processor programmed or configured to receive a training dataset comprising a plurality of data records, each data record including a plurality of feature values ​​of a plurality of features; compute a feature projection error for each of the plurality of features in each of the plurality of data records using a trained machine learning model; compute a classification score for each of the plurality of data records using the trained machine learning model; and determine a feature distribution based on the classification score of each data record and the feature projection error of each of the plurality of features in each data record, wherein the feature distribution based on the feature projection error includes: a false positive classification distribution of the features, the false positive classification distribution including the feature values ​​of each of the plurality of features in each data record having a false positive classification. The training model comprises: a distribution of the feature projection error of each feature in the dataset; a false negative classification distribution comprising the distribution of the feature projection error of each of the plurality of features in each data record having a false negative classification; applying a scaling function to each feature value of the feature having the highest projection error value in the false positive classification distribution to provide a scaled feature value set; applying a scaling function to each feature value of the feature having the lowest projection error value in the false negative classification distribution to provide a scaled feature value set; combining the scaled feature value set and the scaled feature value set with the training dataset to provide an updated training dataset; and using the updated training dataset to train the trained machine learning model to provide an updated trained machine learning model. In some non-limiting embodiments or aspects, the trained machine learning model is an unsupervised binary classification machine learning model.

[0050] In some non-limiting embodiments or aspects, the feature management system is further programmed or configured to determine the performance metric of the updated trained machine learning model and, based on the performance metric, whether further training procedures for the updated trained machine learning model are necessary. In some non-limiting embodiments or aspects, the scaling function includes a lower bound scalar value and an upper bound scalar value, wherein the scaling function is configured such that: feature values ​​between the lower bound scalar value and the upper bound scalar value remain unchanged; feature values ​​below the lower bound scalar value are changed to the lower bound scalar value; and feature values ​​above the upper bound scalar value are changed to the upper bound scalar value. In some non-limiting embodiments or aspects, the scaling function includes a lower bound scalar value, an upper bound scalar value, and an intermediate value, wherein the scaling function is configured such that: feature values ​​below the lower bound scalar value and above the upper bound scalar value remain unchanged; feature values ​​between the lower bound scalar value and the intermediate value are changed to the lower bound scalar value; and feature values ​​between the upper bound scalar value and the intermediate value are changed to the upper bound scalar value.

[0051] In some non-limiting embodiments or aspects, the feature management system is further programmed or configured to determine the lower and upper scalar values ​​of both the scaling reduction function and the scaling magnification function. In some non-limiting embodiments or aspects, when determining the lower and upper scalar values ​​of both the scaling reduction function and the scaling magnification function, the feature management system is programmed or configured to determine the lower and upper scalar values ​​of both the scaling reduction function and the scaling magnification function based on the Mann-Whitney test.

[0052] In this way, the feature management system can provide feature optimization for at least one feature of the machine learning model, enabling the machine learning model to be trained with improved performance without having to retrain the model multiple times for each of the many features in the dataset. Furthermore, it can reduce the network resources used to train the machine learning model and improve its accuracy, while also reducing the runtime of one or more actions performed using the machine learning model based on the improved performance.

[0053] Now for reference Figure 1 , Figure 1 This is an illustration of an exemplary environment 100 in which the devices, systems, methods, and / or methods described herein may be implemented. Figure 1 As shown, environment 100 may include a feature management system 102, a transaction service provider system 104, user equipment 106, and a communication network 108. The feature management system 102, the transaction service provider system 104, and / or the user equipment 106 may be interconnected (e.g., establishing a connection for communication) via wired connections, wireless connections, or a combination of wired and wireless connections.

[0054] The feature management system 102 may include one or more devices configured to communicate with the transaction service provider system 104 and / or user equipment 106 via a communication network 108. For example, the feature management system 102 may include a server, server cluster, and / or other similar devices. In some non-limiting embodiments or aspects, the feature management system 102 may be associated with the transaction service provider system (e.g., it may be operated by a transaction service provider as part of the transaction service provider system, or it may be operated by a transaction service provider independent of the transaction service provider system, etc.), as described herein. Additionally or alternatively, the feature management system 102 may generate (e.g., train, validate, retrain, etc.), store, and / or implement (e.g., operate, provide input to one or more machine learning models, and / or output from one or more machine learning models, etc.) one or more machine learning models. For example, the feature management system 102 may generate one or more machine learning models by fitting (e.g., validating) one or more machine learning models against data used for training (e.g., training data). In some non-limiting embodiments or aspects, the feature management system 102 may generate, store, and / or implement one or more machine learning models, such as one or more machine learning models for a production environment (e.g., a real-time or runtime environment for providing inference based on data in a real-world situation). In some non-limiting embodiments or aspects, the feature management system 102 may communicate with a data storage device, which may be local or remote to the feature management system 102. In some non-limiting embodiments or aspects, the feature management system 102 may be able to receive information from the data storage device, store information in the data storage device, transfer information to the data storage device, and / or search for information stored in the data storage device.

[0055] Transaction service provider system 104 may include one or more devices configured to communicate with feature management system 102 and / or user equipment 106 via communication network 108. For example, transaction service provider system 104 may include computing devices such as servers, server clusters, and / or other similar devices. In some non-limiting embodiments or aspects, transaction service provider system 104 may be associated with a transaction service provider system as described herein. In some non-limiting embodiments or aspects, feature management system 102 may be a component of transaction service provider system 104.

[0056] User equipment 106 may include computing devices configured to communicate with feature management system 102 and / or transaction service provider system 104 via communication network 108. For example, user equipment 106 may include computing devices such as desktop computers, portable computers (e.g., tablet computers, laptop computers, etc.), mobile devices (e.g., cellular phones, smartphones, personal digital assistants, wearable devices, etc.) and / or other similar devices. In some non-limiting embodiments or aspects, user equipment 106 may be associated with a user (e.g., an individual operating user equipment 106).

[0057] The communication network 108 may include one or more wired and / or wireless networks. For example, the communication network 108 may include a cellular network (e.g., LTE). Networks, third-generation (3G) networks, fourth-generation (4G) networks, fifth-generation (5G) networks, code division multiple access (CDMA) networks, public terrestrial mobile networks (PLMN), local area networks (LAN), wide area networks (WAN), metropolitan area networks (MAN), telephone networks (e.g., public switched telephone network (PSTN), etc.), private networks, self-organizing networks, intranets, the Internet, fiber-optic networks, cloud computing networks, and / or combinations of some or all of these or other types of networks.

[0058] supply Figure 1 The number and layout of devices and networks shown are for illustrative purposes only. It is possible that... Figure 1 The devices and / or networks shown are those that, compared to additional devices and / or networks, fewer devices and / or networks, different devices and / or networks, or devices and / or networks arranged differently. Furthermore, Figure 1 The two or more devices shown can be implemented within a single device, or Figure 1 The single device shown may be implemented as multiple distributed devices. Alternatively, a group of devices in environment 100 (e.g., one or more devices) may perform one or more functions described as being performed by another group of devices in environment 100.

[0059] Now for reference Figure 2 , Figure 2 This is a diagram of example components of device 200. Device 200 may correspond to feature management system 102 (e.g., one or more devices of feature management system 102), transaction service provider system 104 (e.g., one or more devices of transaction service provider system 104), and / or user device 106. In some non-limiting embodiments or aspects, feature management system 102, transaction service provider system 104, and / or user device 106 may include at least one device 200 and / or at least one component of device 200. Figure 2As shown, device 200 may include bus 202, processor 204, memory 206, storage unit 208, input unit 210, output unit 212, and communication interface 214.

[0060] Bus 202 may include components that enable communication between parts of device 200. In some non-limiting embodiments or aspects, processor 204 may be implemented in hardware, software, or a combination of hardware and software. For example, processor 204 may include a processor (e.g., a central processing unit (CPU), graphics processing unit (GPU), accelerated processing unit (APU), etc.), microprocessor, digital signal processor (DSP), and / or any processing component that can be programmed to perform functions (e.g., a field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), etc.). Memory 206 may include random access memory (RAM), read-only memory (ROM), and / or another type of dynamic or static memory (e.g., flash memory, magnetic memory, optical memory, etc.) that stores information and / or instructions for use by processor 204.

[0061] Storage component 208 may store information and / or software associated with the operation and use of device 200. For example, storage component 208 may include hard disk (e.g., magnetic disk, optical disk, magneto-optical disk, solid-state disk, etc.), compressed optical disk (CD), digital versatile optical disk (DVD), floppy disk, cassette tape, magnetic tape and / or another type of computer-readable medium, and corresponding drives.

[0062] Input component 210 may include components that allow device 200 to receive information, such as through user input (e.g., touchscreen display, keyboard, keypad, mouse, button, switch, microphone, etc.). Alternatively, input component 210 may include sensors for sensing information (e.g., Global Positioning System (GPS) components, accelerometers, gyroscopes, actuators, etc.). Output component 212 may include components that provide output information from device 200 (e.g., display, speaker, one or more light-emitting diodes (LEDs), etc.).

[0063] Communication interface 214 may include transceiver-like components (e.g., transceivers, separate receivers, and transmitters) that enable device 200 to communicate with other devices, for example, via wired connections, wireless connections, or a combination of wired and wireless connections. Communication interface 214 may allow device 200 to receive information from another device and / or provide information to another device. For example, communication interface 214 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, etc. Interfaces, cellular network interfaces, etc.

[0064] Device 200 can perform one or more processes described herein. Device 200 can perform these processes based on software instructions stored in a computer-readable medium such as memory 206 and / or storage unit 208, executed by processor 204. Computer-readable media (e.g., non-transitory computer-readable media) are defined herein as non-transitory memory devices. Memory devices include memory space located within a single physical memory device or memory space extending across multiple physical memory devices.

[0065] Software instructions may be read from another computer-readable medium or from another device into memory 206 and / or storage component 208 via communication interface 214. When executed, the software instructions stored in memory 206 and / or storage component 208 cause processor 204 to perform one or more processes described herein. Alternatively or additionally, hard-wired circuitry may be used in place of or in conjunction with the software instructions to perform one or more processes described herein. Therefore, the embodiments described herein are not limited to any particular combination of hardware circuitry and software.

[0066] Figure 2 The number and arrangement of components shown are provided as examples. In some non-limiting embodiments or aspects, device 200 may include additional components, fewer components, different components, or components that are compatible with... Figure 2 The components are arranged in different ways as shown. Alternatively, a group of components (e.g., one or more components) of device 200 may perform one or more functions described as being performed by another group of components of device 200.

[0067] Now for reference Figure 3 , Figure 3 This is a flowchart of a non-limiting embodiment or aspect of a process 300 for adaptive feature optimization of a machine learning model. In some non-limiting embodiments or aspects, one or more steps of process 300 may be performed (e.g., entirely, partially, etc.) by the feature management system 102 (e.g., one or more devices of the feature management system 102). In some non-limiting embodiments or aspects, one or more steps of process 300 may be performed (e.g., entirely, partially, etc.) by another device or set of devices that are separate from or include the feature management system 102 (e.g., one or more devices of the feature management system 102), the transaction service provider system 104 (e.g., one or more devices of the transaction service provider system 104), and / or the user device 106.

[0068] like Figure 3As shown, at step 302, process 300 includes calculating classification scores for multiple data records in a training dataset using a machine learning model. For example, feature management system 102 may use a machine learning model to calculate classification scores for multiple data records in a dataset (e.g., a training dataset). Each data record may include multiple feature values ​​for multiple features. In some non-limiting embodiments or aspects, each data record may include observations (e.g., observed values) for each of the multiple features. In some non-limiting embodiments or aspects, feature management system 102 may use a machine learning model to calculate a classification score for each of the multiple data records. In some non-limiting embodiments or aspects, a data record may include unlabeled (e.g., may not have classification labels, such as binary classification) data regarding a potential classification provided by the machine learning model. In some non-limiting embodiments or aspects, a data record may include only unlabeled data.

[0069] In some non-limiting embodiments or aspects, the machine learning model may include a trained machine learning model, such as one trained on a dataset different from the dataset that the feature management system 102 can use to compute classification scores. In some non-limiting embodiments or aspects, the machine learning model may include an unsupervised binary classification machine learning model. For example, the machine learning model may include an unsupervised binary classification machine learning model that has been trained on unlabeled data.

[0070] In some non-limiting embodiments or aspects, the machine learning model can be configured to provide an output that includes a prediction indicating whether an input (such as a data record) is classified into a first group (such as normal) or a second group (such as anomalous). For example, the machine learning model can be configured to provide an output (e.g., including a risk score) that includes a prediction indicating whether the input is classified as normal (e.g., the input is classified as having low risk) or risky (e.g., the input is classified as having high risk).

[0071] In some non-limiting embodiments or aspects, the machine learning model may be configured to provide an output that includes a projection of each feature corresponding to each feature of the input. For example, the machine learning model may include an autoencoder configured to provide an output vector that includes the projection of each feature (e.g., a projection of each feature corresponding to each feature of the input vector).

[0072] In some non-limiting embodiments, the feature management system 102 may use a trained machine learning model to compute the feature projection error for each of a plurality of features in each of a plurality of data records. In one example, a data record may include multiple feature values ​​of multiple features, and the feature management system 102 may provide the data record as input to the machine learning model. The feature management system 102 may generate the output of the machine learning model based on the input, wherein the output includes a projection value for each feature corresponding to the feature values ​​of each feature in the input. The feature management system 102 may determine the value of the projection error for each feature based on the feature values ​​of the plurality of features in the input and the projection values ​​corresponding to the plurality of features in the input. For example, the feature management system 102 may compare the feature values ​​of the plurality of features in the input with the projection values ​​of the plurality of features (e.g., the projection values ​​corresponding to the plurality of features in the input), and the feature management system 102 may determine the value of the projection error for each feature based on the difference between each feature value of the plurality of features in the input and each projection value of the plurality of features.

[0073] In some non-limiting embodiments or aspects, the feature management system 102 may use a machine learning model to calculate a classification score for a data record. For example, the feature management system 102 may calculate a classification score for a data record based on the projection error of each of a plurality of features of the data record. In such an example, the feature management system 102 may calculate a classification score for a data record based on a range of projection error values. Thus, the feature management system 102 may assign a first classification score to a data record based on the projection error of each of a plurality of features of the data record being included in a first range of projection error values, assign a second classification score to a data record based on the projection error of each of a plurality of features of the data record being included in a second range of projection error values, assign a third classification score to a data record based on the projection error of each of a plurality of features of the data record being included in a third range of projection error values, and so on.

[0074] In some non-limiting embodiments or aspects, the feature management system 102 may calculate the average and / or median of projection errors based on the projection error values ​​of each of a plurality of features, and the feature management system 102 may determine the classification score of a data record based on the average and / or median of projection errors (e.g., based on the average and / or median of projection errors included in the range of projection error values).

[0075] In some non-limiting embodiments or aspects, the feature value may correspond to the value of transaction data associated with the payment transaction. For example, the feature value may correspond to the value of a data field in a payment transaction record. In some non-limiting embodiments or aspects, multiple features may include one or more features corresponding to transaction data associated with the payment transaction (e.g., parameters of the transaction data, such as data fields in a payment transaction record). For example, multiple features may include transaction amount associated with the amount of the payment transaction (e.g., fees associated with the payment transaction, transaction amount, total transaction amount, fees for one or more products involved in the payment transaction, etc.), transaction value associated with the quantity of the payment transaction, transaction time associated with the time interval between the occurrence of the payment transaction (e.g., time of day, day of week, date of month, month of year, predetermined time of day such as morning, afternoon, evening, etc., predetermined time of week such as weekday, weekend, etc., predetermined time of year such as first quarter, second quarter, etc.), transaction type of the payment transaction (e.g., online transaction, card payment transaction, face-to-face transaction, e-commerce indicator, settlement mark of the payment transaction, etc.), account identifier of the account involved in the payment transaction (e.g., PAN), merchant identifier of the merchant involved in the payment transaction (e.g., merchant name), merchant category code of the merchant involved in the payment transaction, etc.

[0076] In some non-limiting embodiments or aspects, the transaction data associated with a payment transaction may include values ​​of multiple data fields associated with the payment transaction. These multiple data field values ​​may include values ​​of one or more transaction amount data fields associated with the amount of the payment transaction (e.g., fees associated with the payment transaction, transaction amount, total transaction amount, fees for one or more products involved in the payment transaction, etc.), values ​​of one or more transaction time data fields associated with the time interval in which the payment transaction occurred (e.g., time of day, day of week, date of month, month of year, predetermined daytime such as morning, afternoon, evening, night, etc., predetermined weektime such as weekday, weekend, etc., predetermined yeartime such as first quarter, second quarter, etc.), and values ​​of one or more transaction type data fields associated with the transaction type of the payment transaction (e.g., online transaction, card payment, face-to-face transaction, e-commerce indicator, settlement indicator of the payment transaction, etc.), etc.

[0077] In some non-limiting embodiments or aspects, the feature management system 102 may receive datasets. In some non-limiting embodiments or aspects, the feature management system 102 may receive datasets from external systems such as issuer systems, merchant systems, transaction service provider systems (e.g., transaction service provider system 104), etc.

[0078] In some non-limiting embodiments or aspects, the feature management system 102 may perform feature engineering procedures on the dataset. In some non-limiting embodiments or aspects, the feature engineering procedures may include numerical transformation procedures (e.g., scaling), category encoder procedures, clustering procedures, procedures for grouping aggregated values, principal component analysis procedures, and / or feature construction procedures. In one example, the feature management system 102 may receive an initial dataset with an initial number of features, and the feature management system 102 may perform feature engineering procedures (e.g., feature selection) on the initial dataset to provide a revised dataset with a revised number of features. In some non-limiting embodiments or aspects, the revised number of features is less than the initial number of features.

[0079] like Figure 3 As shown, in step 304, process 300 includes generating a distribution of features for misclassified data records. For example, feature management system 102 may generate a distribution of features for multiple misclassified data records (e.g., data records with incorrect classifications). In some non-limiting embodiments or aspects, feature management system 102 may generate a distribution of features based on feature projection errors of multiple misclassified data records. For example, feature management system 102 may generate a distribution of features based on feature projection errors of data records with false positive and false negative classifications. In some non-limiting embodiments or aspects, the distribution of features based on feature projection errors may include a histogram comprising the frequencies of feature values ​​of multiple features included in one or more data records, plotted relative to feature projection error values ​​of multiple features. For example, the distribution of features based on feature projection errors may include a histogram comprising the frequencies of feature values ​​of multiple features included in data records with false positive and false negative classifications, plotted relative to feature projection error values ​​of multiple features.

[0080] In some non-limiting embodiments or aspects, the distribution of features based on feature projection errors may include the distribution of features based on feature projection errors for each of a plurality of features in each data record having false positive classifications (e.g., the false positive classification distribution of features), and the distribution of features based on feature projection errors for each of a plurality of features in each data record having false negative classifications (e.g., the false negative classification distribution of features).

[0081] In some non-limiting embodiments or aspects, the feature management system 102 can determine the distribution of features based on the classification score of each data record, according to the feature projection error of each of a plurality of features in data records with false positive classifications and data records with false negative classifications. For example, the feature management system 102 can determine the projection error value of each of a plurality of features based on the feature values ​​of a plurality of features in each of a plurality of data records. The feature management system 102 can calculate the classification score of each data record based on the projection error values ​​of a plurality of features in each data record, and the feature management system 102 can determine the classification (e.g., normal classification or abnormal classification) of each data record based on the classification score of each data record. In some non-limiting embodiments or aspects, the feature management system 102 can determine the classification of a data record by comparing the classification score of the data record with a threshold (e.g., a threshold for the classification score).

[0082] In such examples, if a data record's classification score meets a threshold, the feature management system 102 can determine that the data record's classification is incorrect. Conversely, if a data record's classification score does not meet a threshold, the feature management system 102 can determine that the data record's classification is correct. In some non-limiting embodiments or aspects, the feature management system 102 can compare the classifications of multiple data records with known classifications of multiple data records, and the feature management system 102 can determine a set of data records with false positive classifications and a set of data records with false negative classifications.

[0083] In the example above, the feature management system 102 can generate a feature distribution based on the feature projection error value of each of multiple features in each data record of the set of data records with false positive classification. Additionally, the feature management system 102 can generate a feature distribution based on the feature projection error value of each of multiple features in each data record of the set of data records with false negative classification.

[0084] like Figure 3 As shown, in step 306, process 300 includes applying a scaling function to features in the distribution of features of misclassified data records. For example, feature management system 102 may apply a scaling function to features in the distribution of features of misclassified data records. In some non-limiting embodiments or aspects, the scaling function may include a scaling up function and / or a scaling down function.

[0085] In some non-limiting embodiments or aspects, the feature management system 102 may apply a scaling function to features in a distribution of features for false positive classification. For example, the feature management system 102 may apply the scaling function to features in a false positive classification distribution (e.g., a distribution of features with feature projection errors for each of multiple features in each data record having a false positive classification). In some non-limiting embodiments or aspects, the feature management system 102 may apply the scaling function to each feature value of a feature having a high projection error in the false positive classification distribution (e.g., the highest projection error value, a value within the range of highest projection error values, etc.). In some non-limiting embodiments or aspects, the feature management system 102 may apply the scaling function to each feature value of a feature having a high projection error in the false positive classification distribution to provide an additional set of feature values ​​(e.g., a scaled-down set of feature values).

[0086] In some non-limiting embodiments or aspects, the scaling function may include multiple parameters. For example, the scaling function may include a lower bound scalar value and an upper bound scalar value. In some non-limiting embodiments or aspects, the scaling function is configured such that: eigenvalues ​​between the lower bound scalar value and the upper bound scalar value remain unchanged; eigenvalues ​​below the lower bound scalar value are changed to the lower bound scalar value; and eigenvalues ​​above the upper bound scalar value are changed to the upper bound scalar value.

[0087] In some non-limiting embodiments or aspects, the feature management system 102 may apply a scaling function to features in a distribution of features for false negative classification. For example, the feature management system 102 may apply the scaling function to features in a false negative classification distribution of features (e.g., the distribution of features based on the feature projection error of each of a plurality of features in each data record having a false negative classification). In some non-limiting embodiments or aspects, the feature management system 102 may apply the scaling function to each feature value of a feature having a low projection error (e.g., the lowest projection error value, a value within the lowest projection error range, etc.) in the false negative classification distribution of features. In some non-limiting embodiments or aspects, the feature management system 102 may apply a scaling reduction function to each feature value of a feature having a low projection error in the false negative classification distribution to provide an additional set of feature values ​​(e.g., a scaled-up set of feature values).

[0088] In some non-limiting embodiments or aspects, the scaling function may include multiple parameters. For example, the scaling function may include a lower limit scalar value, an upper limit scalar value, and an intermediate value. In some non-limiting embodiments or aspects, the scaling function is configured such that: eigenvalues ​​below the lower limit scalar value and above the upper limit scalar value remain unchanged; eigenvalues ​​between the lower limit scalar value and the intermediate value are changed to the lower limit scalar value; and eigenvalues ​​between the upper limit scalar value and the intermediate value are changed to the upper limit scalar value.

[0089] In some non-limiting embodiments or aspects, the feature management system 102 can determine the parameters of the scaling reduction function and / or scaling amplification function. In one example, the feature management system 102 can determine the lower bound scalar value and / or upper bound scalar value of the scaling reduction function. In some non-limiting embodiments or aspects, the feature management system 102 can determine the lower bound scalar value and / or upper bound scalar value of the scaling reduction function based on the Mann-Whitney test (e.g., an application of the Mann-Whitney test, which may include calculating a U statistic that quantifies the distributional difference between two sample sets having a first category such as normal and a second category such as abnormal). Alternatively or additionally, the feature management system 102 can determine the lower bound scalar value, upper bound scalar value, and / or intermediate value (e.g., intermediate scalar value) of the scaling amplification function. For example, the feature management system 102 can determine the lower bound scalar value, upper bound scalar value, and intermediate value of the scaling amplification function based on the Mann-Whitney test.

[0090] In some non-limiting embodiments or aspects, the feature management system 102 can determine the parameters of the scaling function and / or scaling function based on a sample set of data records, the sample set of data records including data that can be represented as s i,Norm The first subset of s with the first category (e.g., normal) can be represented as s using the Mann-Whitney test. i,Risk The second subset having a second classification (e.g., abnormal, risk, etc.). In some non-limiting embodiments or aspects, the feature management system 102 can... i,Risk and s i,Norm The feature management system 102 combines and ranks the feature values ​​from low to high, such that the lowest value has a rank of 1, the next lowest value has a rank of 2, and so on. Identical feature values ​​may have the same rank. In some non-limiting embodiments or aspects, the feature management system 102 can... i,Risk and s i,Norm The rankings are summed, and the U statistic is calculated using the following formula:

[0091] U=min[n1n2+0.5n1(n1+1)-T1,n1n2+0.5n2(n2+1)-T2]

[0092] Where n1 and n2 are respectively s i,Risk and s i,Norm The number of data records in the dataset, and T1 and T2 are s i,Risk and s i,Norm Overall ranking. U-statistics quantifications i,Risk and s i,Norm The distributional differences between them mean that the smaller the U statistic, the better. i,Risk and s i,Norm The greater the difference between them.

[0093] In some non-limiting embodiments or aspects, the feature management system 102 may use different values ​​of the lower bound scalar value and / or upper bound scalar value for the scaling reduction function to perform the calculation of the U statistic in order to minimize the value of the U statistic. In some non-limiting embodiments or aspects, the feature management system 102 may use different values ​​of the lower bound scalar value, upper bound scalar value, and / or intermediate value of the scaling magnification function to perform the calculation of the U statistic in order to minimize the value of the U statistic.

[0094] In some non-limiting embodiments or aspects, the feature management system 102 can be configured by setting W(parameter combinations) = [W1, ..., W...]. M The M-threshold parameter is determined as part of the scaling function (e.g., a scaling down or scaling up function) and w is... n =[w 1,n , ..., w M,n The arbitrary sample set that can be used to determine W determines one or more parameters of the scaling reduction function and / or scaling up function.

[0095] In some non-limiting embodiments or aspects, the feature management system 102 can obtain feature values ​​for a first set of data records having a first category (e.g., risk) and feature values ​​for a second set of data records having a second category (e.g., normal), which can be represented as and As illustrated in the example above, each possible set of data records can be combined using the given parameter values ​​w. n Confirmed. Using the Mann-Whitney test, the feature management system 102 can determine the associated U statistic. Quantification of the U statistic and The differences between them.

[0096] The feature management system 102 can handle all possible values ​​w n The upper iteration is used to provide the minimum U statistic (e.g., ) of wn The value of w is given in this way as the minimum U statistic. n The value of can be the optimal parameter value for the scaling down function and / or scaling up function.

[0097] In some non-limiting embodiments or aspects, the feature management system 102 may determine one or more parameters of the scaling function and / or scaling function based on hyperparameter tuning techniques such as grid search, random search and / or Bayesian optimization.

[0098] like Figure 3 As shown, in step 308, process 300 includes combining scaled feature values ​​with the training dataset. For example, feature management system 102 may combine multiple scaled feature values ​​with the training dataset. In some non-limiting embodiments or aspects, the multiple scaled feature values ​​may include a scaled-down set of feature values ​​and / or a scaled-up set of feature values. In some non-limiting embodiments or aspects, feature management system 102 may combine the scaled-down set of feature values ​​and / or the scaled-up set of feature values ​​with the training dataset to provide an updated training dataset. In some non-limiting embodiments or aspects, feature management system 102 may combine the scaled-down set of feature values ​​and / or the scaled-up set of feature values ​​with the training dataset based on features in the distribution of features for which a scaling function is applied to misclassified data records.

[0099] like Figure 3 As shown, in step 310, process 300 includes training a machine learning model using an updated training dataset. For example, feature management system 102 may use an updated training dataset to train a machine learning model (e.g., a trained machine learning model), the updated training dataset comprising a set of scaled-down feature values ​​and a set of scaled-up feature values. In some non-limiting embodiments or aspects, feature management system 102 may use an updated training dataset to train a machine learning model to provide an updated trained machine learning model. In some non-limiting embodiments or aspects, feature management system 102 may use an updated training dataset to train a machine learning model based on combining scaled feature values ​​with the training dataset.

[0100] In some non-limiting embodiments or aspects, the feature management system 102 may determine performance metrics for the updated trained machine learning model. For example, the feature management system 102 may determine one or more performance metrics for the updated trained machine learning model. In some non-limiting embodiments or aspects, the feature management system 102 may determine accuracy metrics, precision metrics, recall metrics, etc., for the updated trained machine learning model.

[0101] In some non-limiting embodiments or aspects, the feature management system 102 may determine whether further training of the updated trained machine learning model is necessary based on a performance metric. For example, the feature management system 102 may determine the value of a performance metric for the updated trained machine learning model and compare the value of the performance metric to a threshold (e.g., a threshold for the performance metric). If the feature management system 102 determines that the value of the performance metric meets the threshold, then the feature management system 102 may determine that further training of the updated trained machine learning model is unnecessary. If the feature management system 102 determines that the value of the performance metric does not meet the threshold, then the feature management system 102 may determine that further training of the updated trained machine learning model is necessary.

[0102] In some non-limiting embodiments or aspects, the feature management system 102 may perform further training procedures on the updated trained machine learning model. For example, the feature management system 102 may perform further training procedures on the updated trained machine learning model by repeating the following process: calculating classification scores for multiple data records in the updated training dataset using the updated trained machine learning model; generating a distribution of features for misclassified data records; applying one or more scaling functions to the features in the distribution of features for misclassified data records; combining the scaled feature values ​​with the updated training dataset to provide two updated training datasets; and using the two updated training datasets to train the updated trained machine learning model (e.g., similar to steps 302 to 310 of process 300). In some non-limiting embodiments or aspects, the feature management system 102 may perform multiple further training procedures on the updated trained machine learning model as necessary (e.g., until the updated trained machine learning model meets a threshold of a performance metric).

[0103] In some non-limiting embodiments or aspects, the feature management system 102 may receive a request for inference from production machine learning, and the feature management system 102 may generate inference based on the request. In some non-limiting embodiments or aspects, the production machine learning model may include a machine learning model that has been trained and / or validated (e.g., tested) and can be used to generate inferences (e.g., predictions) such as real-time inference, runtime inference, etc. In some non-limiting embodiments or aspects, the production machine learning model may include an updated trained machine learning model.

[0104] In some non-limiting embodiments or aspects, inference requests may be associated with tasks for which production machine learning models can provide inferences. In some non-limiting embodiments or aspects, inference requests may be associated with financial services tasks. For example, inference requests may be associated with token servicing tasks, authentication tasks (e.g., 3D security authentication tasks), fraud detection tasks, etc.

[0105] In some non-limiting embodiments or aspects, the inference request may include runtime input data. In some non-limiting embodiments or aspects, runtime input data may include data samples received in real-time by a trained machine learning model relative to runtime input data being generated. For example, runtime input data may be generated by a data source (e.g., a customer executing a transaction) and subsequently received in real-time by a trained machine learning model. Runtime (e.g., production) may refer to inputting runtime data (e.g., runtime datasets, real-world data, real-world observations, etc.) into one or more trained machine learning models (e.g., one or more trained machine learning models of feature management system 102) and / or generating inference (e.g., generating inference using feature management system 102 or another machine learning system).

[0106] In some non-limiting implementations or aspects, the runtime may be executed during a phase that may occur after the training phase, after the testing phase, and / or after the machine learning model is deployed to a production environment. During the time period associated with the runtime phase, the machine learning model (e.g., a production machine learning model) may process runtime input data to generate inferences (e.g., real-time inference, real-time prediction, etc.).

[0107] Now for reference Figures 4A to 4G , Figures 4A to 4G This is a diagram of a non-limiting embodiment or aspect of an implementation 400 of a process (e.g., process 300) for adaptive feature optimization of a machine learning model. Figures 4A to 4G As shown, implementation 400 may include steps of a process performed by the feature management system 102 (e.g., a process identical or similar to process 300). In some non-limiting embodiments or aspects, one or more steps of the process may be performed (e.g., entirely, partially, etc.) by another device or set of devices that are separate from or include the feature management system 102 (e.g., one or more devices of the feature management system 102) such as the transaction service provider system 104 (e.g., one or more devices of the transaction service provider system 104) and / or the user device 106. In some non-limiting embodiments or aspects, Figures 4A to 4G The steps of the process shown can be associated with a training procedure used for a trained machine learning model.

[0108] like Figure 4A As shown by reference numeral 405 in the accompanying drawings, the feature management system 102 can receive a training dataset comprising multiple data records. In some non-limiting embodiments or aspects, each data record may include multiple feature values ​​of multiple features. In some non-limiting embodiments or aspects, each data record may include observations (e.g., observed values) of each of the multiple features. In some non-limiting embodiments or aspects, the data record may include unlabeled (e.g., may not have classification labels, such as binary classification) data regarding a potential classification provided by a machine learning model.

[0109] like Figure 4B As shown by reference numeral 410 in the accompanying drawings, the feature management system 102 can calculate a classification score for each of a plurality of data records. For example, the feature management system 102 can use a trained machine learning model to calculate a classification score for each of the plurality of data records. In some non-limiting embodiments or aspects, the trained machine learning model may include an autoencoder configured to provide an output vector that includes a projection of each feature (e.g., a projection of each feature corresponding to each feature of the input vector).

[0110] like Figure 4C As shown by reference numeral 415 in the accompanying drawings, the feature management system 102 can determine the distribution of features based on the feature projection error of each of a plurality of features in a data record. In some non-limiting embodiments or aspects, the feature management system 102 can generate the distribution of features based on the feature projection errors of data records having false positive and false negative classifications. In some non-limiting embodiments or aspects, the distribution of features based on feature projection errors may include a histogram comprising the frequencies of feature values ​​of a plurality of features included in one or more data records, plotted relative to the feature projection error values ​​of the plurality of features. For example, the distribution of features based on feature projection errors may include a histogram comprising the frequencies of feature values ​​of a plurality of features included in data records having false positive and false negative classifications, plotted relative to the feature projection error values ​​of the plurality of features.

[0111] In some non-limiting embodiments or aspects, the distribution of features based on feature projection errors may include the distribution of features based on feature projection errors for each of a plurality of features in each data record having false positive classifications (e.g., the false positive classification distribution of features), and the distribution of features based on feature projection errors for each of a plurality of features in each data record having false negative classifications (e.g., the false negative classification distribution of features).

[0112] like Figure 4DAs indicated by reference numeral 420 in the accompanying drawings, the feature management system 102 can apply a scaling function to each feature value of a feature in the false positive classification distribution. In some non-limiting embodiments or aspects, the feature management system 102 can apply the scaling function to each feature value of a feature in the false positive classification distribution that has a high projection error (e.g., the highest projection error value). In some non-limiting embodiments or aspects, the scaling function is configured such that: feature values ​​between the lower and upper scalar values ​​remain unchanged; feature values ​​below the lower scalar value are changed to the lower scalar value; and feature values ​​above the upper scalar value are changed to the upper scalar value.

[0113] like Figure 4D As further shown by reference numeral 425 in the accompanying drawings, the feature management system 102 can generate a scaled-down set of feature values. For example, the feature management system 102 can generate a scaled-down set of feature values ​​based on each feature value of a feature that has a high projection error in the false positive classification distribution, by applying a scale reduction function to it.

[0114] like Figure 4E As shown by reference numeral 430 in the accompanying drawings, the feature management system 102 can apply a scaling function to each feature value of a feature in a false negative classification distribution. In some non-limiting embodiments or aspects, the feature management system 102 can apply the scaling function to each feature value of a feature having a low projection error (e.g., the lowest projection error value) in the false negative classification distribution of the feature. In some non-limiting embodiments or aspects, the scaling function is configured such that: feature values ​​below the lower limit scalar value and above the upper limit scalar value remain unchanged; feature values ​​between the lower limit scalar value and the intermediate value are changed to the lower limit scalar value; and feature values ​​between the upper limit scalar value and the intermediate value are changed to the upper limit scalar value.

[0115] like Figure 4E As further shown by reference numeral 435 in the figure, the feature management system 102 can generate a set of scaled feature values ​​for a second subset of features among a plurality of features. For example, the feature management system 102 can generate a set of scaled feature values ​​based on each feature value of the feature that has the lowest projection error value in the false negative classification distribution by applying a scaling function.

[0116] like Figure 4F As shown by reference numeral 440 in the accompanying drawings, the feature management system 102 can combine the scaled-down feature set and the scaled-up feature set with the training dataset to provide an updated training dataset. In some non-limiting embodiments or aspects, the feature management system 102 can combine the scaled-down feature set and the scaled-up feature set with the training dataset by replacing one or more feature values ​​of a feature with the scaled-down feature set and / or the scaled-up feature set.

[0117] like Figure 4G As shown by reference numeral 445 in the accompanying drawings, the feature management system 102 can train (e.g., retrain) a trained machine learning model based on an updated training dataset. In some non-limiting embodiments or aspects, the feature management system 102 can determine whether further training of the updated trained machine learning model is necessary based on a performance metric. For example, the feature management system 102 can determine the value of a performance metric for the updated trained machine learning model and compare the value of the performance metric to a threshold (e.g., a threshold for the performance metric). If the feature management system 102 determines that the value of the performance metric meets the threshold, then the feature management system 102 can determine that further training of the updated trained machine learning model is unnecessary. If the feature management system 102 determines that the value of the performance metric does not meet the threshold, then the feature management system 102 can determine that further training of the updated trained machine learning model is necessary.

[0118] Although this disclosure has been described in detail for illustrative purposes based on embodiments or aspects currently considered most practical and preferred, it should be understood that such details are for the purposes described only, and that this disclosure is not limited to the disclosed embodiments or aspects, but rather is intended to cover modifications and equivalent arrangements within the spirit and scope of the appended claims. For example, it should be understood that the invention contemplates that, to the extent possible, one or more features of any embodiment may be combined with one or more features of any other embodiment.

Claims

1. A feature management system, comprising: At least one processor, said at least one processor being programmed or configured to: The system receives a training dataset comprising multiple data records from an external system. Each data record includes multiple feature values ​​for multiple features, and the multiple feature values ​​correspond to values ​​in transaction data associated with a payment transaction. The multiple features include transaction data associated with a payment transaction. The external system includes one or more of an issuer system, a merchant system, and a transaction service provider system. A trained machine learning model is used to calculate the feature projection error of each of the multiple features in each of the multiple data records; The trained machine learning model is used to calculate a classification score for each of the plurality of data records; Based on the classification score for each data record, the distribution of features is determined according to the feature projection error of each of the plurality of features in each data record, wherein the distribution of features according to the feature projection error includes: The false positive classification distribution of features, said false positive classification distribution comprising the distribution of features based on the feature projection error of each of the plurality of features in each data record having a false positive classification; and The false negative classification distribution of features, wherein the false negative classification distribution includes the distribution of features based on the feature projection error of each of the plurality of features in each data record having a false negative classification; The scaling function is applied to each feature value of the feature that has the highest projection error value in the false positive classification distribution to provide a scaled feature value set; The scaling function is applied to each feature value of the feature that has the lowest projection error value in the false negative classification distribution to provide a scaled feature value set; The scaled-down feature set and the scaled-up feature set are combined with the training dataset to provide an updated training dataset; and The updated training dataset is used to train the trained machine learning model to provide an updated trained machine learning model. The feature management system is capable of generating inferences in real time in response to a request for inferences in production machine learning. The request for inferences is generated by a data source and associated with one or more of a token service task, an authentication task, and a fraud detection task. The data source includes the client executing the transaction. The production machine learning model is used to generate the inference, and the production machine learning model includes one or more of the trained machine learning model and the updated trained machine learning model.

2. The system of claim 1, wherein the at least one processor is further programmed or configured to: Determine the performance metric of the updated trained machine learning model; and The performance metric is used to determine whether further training of the updated, trained machine learning model is necessary.

3. The system of claim 1, wherein the scaling function comprises a lower bound scalar value and an upper bound scalar value, and wherein the scaling function is configured such that: The characteristic values ​​between the lower limit scalar value and the upper limit scalar value remain unchanged; Feature values ​​that are below the lower limit scalar value are changed to the lower limit scalar value; as well as Feature values ​​higher than the upper limit scalar value are changed to the upper limit scalar value.

4. The system of claim 1, wherein the scaling function includes a lower limit scalar value, an upper limit scalar value, and an intermediate value, and wherein the scaling function is configured such that: Feature values ​​that are below the lower limit scalar value and above the upper limit scalar value remain unchanged; The characteristic value between the lower limit scalar value and the intermediate value is changed to the lower limit scalar value; as well as The characteristic value between the upper limit scalar value and the intermediate value is changed to the upper limit scalar value.

5. The system of claim 1, wherein the at least one processor is further programmed or configured to: Determine the lower and upper scalar values ​​of the scaling function; and Determine the lower limit scalar value, the middle value, and the upper limit scalar value of the scaling function.

6. The system of claim 5, wherein when determining the lower limit scalar value and the upper limit scalar value of the scaling function, the at least one processor is programmed or configured to: The lower and upper scalar values ​​of the scaling function are determined based on the Mann-Whitney test; and When determining the lower limit scalar value, the intermediate value, and the upper limit scalar value of the scaling function, the at least one processor is programmed or configured to: Based on the sample set of the data records and by applying the Mann-Whitney test, the lower limit scalar value, the median value, and the upper limit scalar value of the scaling function are determined; wherein, The sample set of data records includes a first subset with a first classification and a second subset with a second classification, wherein the first classification indicates that the data records it includes are normal and the second classification indicates that the data records it includes are abnormal or risky.

7. The system of claim 1, wherein the trained machine learning model is an unsupervised binary classification machine learning model, and wherein the unsupervised binary classification machine learning model is an autoencoder.

8. A computer-implemented method, comprising: The feature management system receives a training dataset from an external system comprising a plurality of data records, each data record comprising a plurality of feature values ​​corresponding to values ​​of transaction data associated with a payment transaction; wherein the plurality of features comprises transaction data associated with a payment transaction; and wherein the external system comprises one or more of an issuer system, a merchant system, and a transaction service provider system. Using a trained machine learning model, the feature projection error of each of the plurality of features in each of the second plurality of data records is calculated using at least one processor. Using the trained machine learning model, at least one processor is used to compute classification scores for a second set of data records. Using at least one processor, based on the classification score of each data record in the second plurality of data records, a distribution of features is determined according to the feature projection error of each of the plurality of features in the second plurality of data records, wherein the distribution of features according to the feature projection error includes: The false positive classification distribution of features, said false positive classification distribution comprising the distribution of features based on the feature projection error of each of the plurality of features in each data record having a false positive classification; and The false negative classification distribution of features, wherein the false negative classification distribution includes the distribution of features based on the feature projection error of each of the plurality of features in each data record having a false negative classification; The scaling function is applied to each feature value of the feature with the highest projection error value in the false positive classification distribution using at least one processor to provide a scaled feature value set; The scaling function is applied to each feature value of the feature that has the lowest projection error value in the false negative classification distribution using at least one processor to provide a scaled feature value set; The scaled-down feature set and the scaled-up feature set are combined with the training dataset using at least one processor to provide an updated training dataset; and The trained machine learning model is trained using at least one processor with the updated training dataset to provide an updated trained machine learning model. The feature management system is capable of generating inferences in real time in response to a request for inferences in production machine learning. The request for inferences is generated by a data source and associated with one or more of a token service task, an authentication task, and a fraud detection task. The data source includes the client executing the transaction. The production machine learning model is used to generate the inference, and the production machine learning model includes one or more of the trained machine learning model and the updated trained machine learning model.

9. The computer-implemented method of claim 8, further comprising: Determine the performance metric for the updated trained machine learning model; as well as The performance metric is used to determine whether further training of the updated, trained machine learning model is necessary.

10. The computer-implemented method of claim 8, wherein the scaling function includes a lower limit scalar value and an upper limit scalar value, and wherein the scaling function is configured such that: The characteristic values ​​between the lower limit scalar value and the upper limit scalar value remain unchanged; Feature values ​​that are below the lower limit scalar value are changed to the lower limit scalar value; as well as Feature values ​​higher than the upper limit scalar value are changed to the upper limit scalar value.

11. The computer-implemented method of claim 8, wherein the scaling function includes a lower limit scalar value, an upper limit scalar value, and an intermediate value, and wherein the scaling function is configured such that: Feature values ​​that are below the lower limit scalar value and above the upper limit scalar value remain unchanged; The characteristic value between the lower limit scalar value and the intermediate value is changed to the lower limit scalar value; as well as The characteristic value between the upper limit scalar value and the intermediate value is changed to the upper limit scalar value.

12. The computer-implemented method of claim 8, further comprising: Determine the lower and upper scalar values ​​of the scaling function; as well as Determine the lower limit scalar value, the middle value, and the upper limit scalar value of the scaling function.

13. The computer-implemented method of claim 12, wherein determining the lower limit scalar value and the upper limit scalar value of the scaling function comprises: The lower limit scalar value and the upper limit scalar value of the scaling function are determined based on the Mann-Whitney test; and Determining the lower limit scalar value, the intermediate value, and the upper limit scalar value of the scaling function includes: Based on a sample set of the data records and by applying the Mann-Whitney test, the lower limit scalar value, the median value, and the upper limit scalar value of the scaling function are determined; wherein the sample set of data records includes a first subset with a first classification and a second subset with a second classification, the first classification indicating that the data records it includes are normal, and the second classification indicating that the data records it includes are abnormal or risky.

14. The computer-implemented method of claim 8, wherein the trained machine learning model is an unsupervised binary classification machine learning model, and wherein the unsupervised binary classification machine learning model is an autoencoder.

15. A computer program product comprising at least one non-transitory computer-readable medium, the at least one non-transitory computer-readable medium comprising one or more instructions, the one or more instructions causing the at least one processor, when executed by at least one processor of a feature management system, to: The system receives a training dataset comprising multiple data records from an external system. Each data record includes multiple feature values ​​for multiple features, and the multiple feature values ​​correspond to values ​​in transaction data associated with a payment transaction. The multiple features include transaction data associated with a payment transaction. The external system includes one or more of an issuer system, a merchant system, and a transaction service provider system. A trained machine learning model is used to calculate the feature projection error of each of the multiple features in each of the multiple data records; The trained machine learning model is used to calculate a classification score for each of the plurality of data records; Based on the classification score for each data record, the distribution of features is determined according to the feature projection error of each of the plurality of features in each data record, wherein the distribution of the feature projection error includes: The false positive classification distribution of features, said false positive classification distribution comprising the distribution of features based on the feature projection error of each of the plurality of features in each data record having a false positive classification; and The false negative classification distribution of features, wherein the false negative classification distribution includes the distribution of features based on the feature projection error of each of the plurality of features in each data record having a false negative classification; The scaling function is applied to each feature value of the feature that has the highest projection error value in the false positive classification distribution to provide a scaled feature value set; The scaling function is applied to each feature value of the feature that has the lowest projection error value in the false negative classification distribution to provide a scaled feature value set; The scaled-down feature set and the scaled-up feature set are combined with the training dataset to provide an updated training dataset; and The updated training dataset is used to train the trained machine learning model to provide an updated trained machine learning model. The feature management system is capable of generating inferences in real time in response to a request for inferences in production machine learning. The request for inferences is generated by a data source and associated with one or more of a token service task, an authentication task, and a fraud detection task. The data source includes the client executing the transaction. The production machine learning model is used to generate the inference, and the production machine learning model includes one or more of the trained machine learning model and the updated trained machine learning model.

16. The computer program product of claim 15, wherein the one or more instructions further cause the at least one processor to perform the following operations: Determine the performance metric of the updated trained machine learning model; and The performance metric is used to determine whether further training of the updated, trained machine learning model is necessary.

17. The computer program product of claim 15, wherein the scaling function includes a lower limit scalar value and an upper limit scalar value, and wherein the scaling function is configured such that: The characteristic values ​​between the lower limit scalar value and the upper limit scalar value remain unchanged; Feature values ​​that are below the lower limit scalar value are changed to the lower limit scalar value; as well as Feature values ​​higher than the upper limit scalar value are changed to the upper limit scalar value.

18. The computer program product of claim 15, wherein the scaling function includes a lower limit scalar value, an upper limit scalar value, and an intermediate value, and wherein the scaling function is configured such that: Feature values ​​that are below the lower limit scalar value and above the upper limit scalar value remain unchanged; The characteristic value between the lower limit scalar value and the intermediate value is changed to the lower limit scalar value; as well as The characteristic value between the upper limit scalar value and the intermediate value is changed to the upper limit scalar value.

19. The computer program product of claim 15, wherein the one or more instructions further cause the at least one processor to perform the following operations: Determine the lower and upper scalar values ​​of the scaling function; and Based on the sample set of the data records, and by applying the Mann-Whitney test, the lower limit scalar value, median value, and upper limit scalar value of the scaling function are determined; wherein, The sample set of data records includes a first subset with a first classification and a second subset with a second classification, wherein the first classification indicates that the data records it includes are normal and the second classification indicates that the data records it includes are abnormal or risky.

20. The computer program product of claim 15, wherein the trained machine learning model is an unsupervised binary classification machine learning model, and wherein the unsupervised binary classification machine learning model is an autoencoder.

Images