Method, system, and computer program product for enhancing synthetic oversampling for supervised anomaly detection

Abstract

Methods, systems, and computer program products may formulate an iterative data mix problem into a Markov decision process (MDP) using a customized reward signal to guide a learning process. To address MDP, a depth deterministic actor-evaluator framework may be modified to accommodate discrete continuous decision spaces for training data augmentation policies.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 397,719, filed on August 12, 2022, the disclosure of which is hereby incorporated by reference in its entirety. Technical Field

[0003] The present disclosure relates to synthetic oversampling and, in some non-limiting embodiments or aspects, to methods, systems, and computer program products for synthetic oversampling for enhanced supervised anomaly detection. Background Art

[0004] Training anomaly detectors can be challenging due to label sparsity and / or different distributions of known anomalies. Existing methods typically use unsupervised or semi-supervised learning to try to alleviate these problems. However, semi-supervised learning may directly adopt limited label information, which may cause the model to overfit on existing anomalies, while unsupervised learning may ignore label information, which may lead to low accuracy. Summary of the invention

[0005] Thus, improved systems, apparatus, products, devices, and / or methods for synthetic oversampling for enhanced supervised anomaly detection are provided.

[0006] According to some non-limiting embodiments or aspects, a method is provided, the method comprising: obtaining a training data set X using at least one processor 训练 , the training data set includes a plurality of source samples, the plurality of source samples include a plurality of labeled normal samples and a plurality of labeled abnormal samples; and executing a training segment using the at least one processor by: (i) initializing a timestamp t; (ii) receiving an action vector a of the timestamp t from the actor network π of an actor-evaluator framework including an actor network π and an evaluator network Q t , where the actor network π is configured based on the state s t To generate the action vector a t , where the state s t is determined based on a current pair of source samples in the plurality of source samples, and wherein the action vector a t comprising a nearest neighbor size k, a composition ratio α, an oversampling number n and a termination probability ε; (iii) combining the current pair of source samples according to the composition ratio α and the oversampling number n to generate a sample with the label y syn The associated labeled synthetic sample x syn ; (iv) using the labeled synthetic sample x syn and the marker y synto train the machine learning classifier φ; (v) obtaining the labeled synthetic sample x based on the nearest neighbor size k syn k source samples in the nearest neighborhood; (vi) for the labeled synthetic sample x syn The k nearest neighbors of the source samples and validation dataset X val The training dataset X 训练 (vii) from the labeled synthetic samples x syn (viii) storing the state s in a memory buffer; t , the action vector a t , next state s t+1 and reward r t , where the next state s t+1 is determined based on the next pair of source samples, and wherein the reward r t is determined based on the multiple classifier outputs; (ix) determining whether the termination probability ε satisfies a termination threshold; (x) in response to determining that the termination probability ε fails to satisfy the termination threshold, incrementing the timestamp t for multiple training steps S: according to the state s t , the action vector a t and the reward r t to train the evaluator network Q according to an evaluator loss function of ; and to train the actor network π according to an actor loss function that depends on the output of the evaluator network, and after training the actor network π and the evaluator network Q for the multiple training steps S, return to step (ii), wherein the next pair of source samples is used as the current pair of source samples; (xi) in response to determining that the termination probability ε satisfies the termination threshold, determine whether the number of executed training segments satisfies the threshold number of training segments; (xii) in response to determining that the number of executed training segments fails to satisfy the threshold number of training segments, return to step (i) to execute the next training segment; and (xiii) in response to determining that the number of executed training segments satisfies the threshold number of training segments, provide the machine learning classifier φ.

[0007] In some non-limiting embodiments or aspects, the current pair of source samples are combined according to the composition ratio α to generate the labeled synthetic sample x according to the following equation: syn :

[0008] x syn =α*x0+(1-α)*x1

[0009]

[0010] Where x0 is the first sample in the current pair of samples, x1 is the second sample in the current pair of samples, and y syn is the labeled synthetic sample x syn of hard marks, y0 is the first hard mark value, and y1 is the second hard mark value.

[0011] In some non-limiting embodiments or aspects, the reward r t is determined according to the following equation:

[0012]

[0013] Where M is the evaluation metric, ΔM(φ t ) measures the trained classifier φ t Performance improvements, X val is the validation dataset, y val is the labeled set of the training dataset, where is the baseline at timestamp t, m is a hyperparameter defining the size of the buffer used to form the baseline, and C(φt|st,at) evaluates the trained classifier φ t is the model confidence, P is the model exploration function, and k is the action vector a t Specifies the nearest neighbor size, x i is the labeled synthetic sample x at timestamp t syn k nearest neighbors, and y i is x i Mark.

[0014] In some non-limiting embodiments or aspects, the actor loss function is defined according to the following equation:

[0015]

[0016] Where N is the number of transitions, π(s i |θ2) is the state s i The expected action, and Q(s i ,π(s i )|θ2) is the evaluator network's response to the expected action π(S i |θ2) and state s i The output of , and wherein the evaluator loss function is defined according to the following equation:

[0017] L Q (θ2)=[Q(s t ,a t )-b t ]2

[0018] where b t =R(s t ,a t )+γQ(s t+1 ,π(s t+1 |θ1)|θ2),π(s t+1 |θ1) is the action specified by the actor network, and γ is a decimal factor.

[0019] In some non-limiting embodiments or aspects, the plurality of source samples are associated with a plurality of transactions in a transaction processing network, wherein the plurality of labeled normal samples are associated with a plurality of non-fraudulent transactions among the plurality of transactions, and wherein the plurality of labeled abnormal samples are associated with a plurality of fraudulent transactions among the plurality of transactions.

[0020] In some non-limiting embodiments or aspects, the method further includes: using the at least one processor to receive transaction data associated with a transaction currently being processed in the transaction processing network; using the at least one processor to process the transaction data using the trained machine learning classifier to classify the transaction as a fraudulent transaction or a non-fraudulent transaction; and in response to classifying the transaction as a fraudulent transaction, using the at least one processor to refuse to authorize the transaction in the transaction processing network.

[0021] In some non-limiting embodiments or aspects, the method further comprises: before executing the training segment: using the at least one processor to use the training data set X 训练 To train the machine learning classifier φ; and for the training data set X 训练 For each source sample in the plurality of source samples, using the at least one processor to pre-compute each k-nearest neighbor.

[0022] According to some non-limiting embodiments or aspects, a system is provided, the system comprising: at least one processor, the at least one processor being programmed and / or configured to: obtain a training data set X 训练 , the training data set includes a plurality of source samples, the plurality of source samples include a plurality of labeled normal samples and a plurality of labeled abnormal samples; and a training segment is performed by: (i) initializing a timestamp t; (ii) receiving an action vector a of the timestamp t from the actor network π of an actor-critic framework including an actor network π and a critic network Q t , where the actor network π is configured based on the state s t To generate the action vector a t , where the state st is determined based on a current pair of source samples in the plurality of source samples, and wherein the action vector a t comprising a nearest neighbor size k, a composition ratio α, an oversampling number n and a termination probability ε; (iii) combining the current pair of source samples according to the composition ratio α and the oversampling number n to generate a sample with the label y syn The associated labeled synthetic sample x syn ; (iv) using the labeled synthetic sample x syn and the marker y syn to train the machine learning classifier φ; (v) obtaining the labeled synthetic sample x based on the nearest neighbor size k syn k source samples in the nearest neighborhood; (vi) for the labeled synthetic sample x syn The k nearest neighbors of the source samples and validation dataset X val The training dataset X 训练 (vii) from the labeled synthetic samples x syn (viii) storing the state s in a memory buffer; t , the action vector a t , next state s t+1 and reward r t , where the next state s t+1 is determined based on the next pair of source samples, and wherein the reward r t is determined based on the multiple classifier outputs; (ix) determining whether the termination probability ε satisfies a termination threshold; (x) in response to determining that the termination probability ε fails to satisfy the termination threshold, incrementing the timestamp t for multiple training steps S: according to the state s t , the action vector a t and the reward r tto train the evaluator network Q according to an evaluator loss function of ; and to train the actor network π according to an actor loss function that depends on the output of the evaluator network, and after training the actor network π and the evaluator network Q for the multiple training steps S, return to step (ii), wherein the next pair of source samples is used as the current pair of source samples; (xi) in response to determining that the termination probability ε satisfies the termination threshold, determine whether the number of executed training segments satisfies the threshold number of training segments; (xii) in response to determining that the number of executed training segments fails to satisfy the threshold number of training segments, return to step (i) to execute the next training segment; and (xiii) in response to determining that the number of executed training segments satisfies the threshold number of training segments, provide the machine learning classifier φ.

[0023] In some non-limiting embodiments or aspects, the current pair of source samples are combined according to the composition ratio α to generate the labeled synthetic sample x according to the following equation: syn :

[0024] x syn =*x0+(1-α)*x1

[0025]

[0026] Where x0 is the first sample in the current pair of samples, x1 is the second sample in the current pair of samples, and y syn is the labeled synthetic sample x syn of hard marks, y0 is the first hard mark value, and y1 is the second hard mark value.

[0027] In some non-limiting embodiments or aspects, the reward r t is determined according to the following equation:

[0028]

[0029] Where M is the evaluation metric, ΔM(φ t ) measures the trained classifier φ t Performance improvements, X val is the validation dataset, y val is the labeled set of the training dataset, s is the baseline at timestamp t, m is a hyperparameter defining the size of the buffer used to form the baseline, and C(φt|st,at) evaluates the trained classifier φ t is the model confidence, P is the model exploration function, and k is the action vector a t Specifies the nearest neighbor size, x iis the labeled synthetic sample x at timestamp t syn k nearest neighbors, and y i is x i Mark.

[0030] In some non-limiting embodiments or aspects, the actor loss function is defined according to the following equation:

[0031]

[0032] Where N is the number of transitions, π(s i |θ2) is the state s i The expected action, and Q(s i ,π(s i )|θ2) is the evaluator network's response to the expected action π(s i |θ2) and state s i The output of , and wherein the evaluator loss function is defined according to the following equation:

[0033] L Q (θ2)=[Q(S t ,a t )-b t ] 2

[0034] where b t =R(s t ,a t )+γQ(s t+1 ,π(s t+1 |θ1)|θ2),π(s t+1 |θ1) is the action specified by the actor network, and γ is a decimal factor.

[0035] In some non-limiting embodiments or aspects, the plurality of source samples are associated with a plurality of transactions in a transaction processing network, wherein the plurality of labeled normal samples are associated with a plurality of non-fraudulent transactions among the plurality of transactions, and wherein the plurality of labeled abnormal samples are associated with a plurality of fraudulent transactions among the plurality of transactions.

[0036] In some non-limiting embodiments or aspects, the at least one processor is further programmed and / or configured to: receive transaction data associated with a transaction currently being processed in the transaction processing network; process the transaction data using the trained machine learning classifier to classify the transaction as a fraudulent transaction or a non-fraudulent transaction; and in response to classifying the transaction as a fraudulent transaction, refuse to authorize the transaction in the transaction processing network.

[0037] In some non-limiting embodiments or aspects, the at least one processor is further programmed and / or configured to: before executing the training segment: using the training data set X 训练 To train the machine learning classifier φ; and for the training data set X 训练 For each source sample in the plurality of source samples, each k-nearest neighbor is pre-computed.

[0038] According to some non-limiting embodiments or aspects, a computer program product is provided, which includes a non-transitory computer-readable medium, wherein the non-transitory computer-readable medium includes program instructions, and when the program instructions are executed by at least one processor, the at least one processor is caused to: obtain a training data set X 训练 , the training data set includes a plurality of source samples, the plurality of source samples include a plurality of labeled normal samples and a plurality of labeled abnormal samples; and a training segment is performed by: (i) initializing a timestamp t; (ii) receiving an action vector a of the timestamp t from the actor network π of an actor-critic framework including an actor network π and a critic network Q t , where the actor network π is configured based on the state s t To generate the action vector a t , where the state s t is determined based on a current pair of source samples in the plurality of source samples, and wherein the action vector a t comprising a nearest neighbor size k, a composition ratio α, an oversampling number n and a termination probability ε; (iii) combining the current pair of source samples according to the composition ratio α and the oversampling number n to generate a sample with the label y syn The associated labeled synthetic sample x syn ; (iv) using the labeled synthetic sample x syn and the marker y syn to train the machine learning classifier φ; (v) obtaining the labeled synthetic sample x based on the nearest neighbor size k syn k source samples in the nearest neighborhood; (vi) for the labeled synthetic sample x syn The k nearest neighbors of the source samples and validation dataset X val The training dataset X 训练 (vii) from the labeled synthetic samples x syn (viii) storing the state s in a memory buffer; t , the action vector a t , next state st+1 and reward r t , where the next state s t+1 is determined based on the next pair of source samples, and wherein the reward r t is determined based on the multiple classifier outputs; (ix) determining whether the termination probability ε satisfies a termination threshold; (x) in response to determining that the termination probability ε fails to satisfy the termination threshold, incrementing the timestamp t for multiple training steps S: according to the state s t , the action vector a t and the reward r t to train the evaluator network Q according to an evaluator loss function of ; and to train the actor network π according to an actor loss function that depends on the output of the evaluator network, and after training the actor network π and the evaluator network Q for the multiple training steps S, return to step (ii), wherein the next pair of source samples is used as the current pair of source samples; (xi) in response to determining that the termination probability ε satisfies the termination threshold, determine whether the number of executed training segments satisfies the threshold number of training segments; (xii) in response to determining that the number of executed training segments fails to satisfy the threshold number of training segments, return to step (i) to execute the next training segment; and (xiii) in response to determining that the number of executed training segments satisfies the threshold number of training segments, provide the machine learning classifier φ.

[0039] In some non-limiting embodiments or aspects, the current pair of source samples are combined according to the composition ratio α to generate the labeled synthetic sample x according to the following equation: syn :

[0040] x syn =α*x0+(1-α)*x1

[0041]

[0042] Where x0 is the first sample in the current pair of samples, x1 is the second sample in the current pair of samples, and y syn is the labeled synthetic sample x syn of hard marks, y0 is the first hard mark value, and y1 is the second hard mark value.

[0043] In some non-limiting embodiments or aspects, the reward r t is determined according to the following equation:

[0044]

[0045] Where M is the evaluation metric, ΔM(φ t ) measures the trained classifier φt Performance improvements, X val is the validation dataset, y val is the labeled set of the training dataset, where is the baseline at timestamp t, m is a hyperparameter defining the size of the buffer used to form the baseline, and C(φt|st,at) evaluates the trained classifier φ t is the model confidence, P is the model exploration function, and k is the action vector a t Specifies the nearest neighbor size, x i is the labeled synthetic sample x at timestamp t syn k nearest neighbors, and y i is x i Mark.

[0046] In some non-limiting embodiments or aspects, the actor loss function is defined according to the following equation:

[0047]

[0048] Where N is the number of transitions, π(s i |θ2) is the state s i The expected action, and Q(s i ,π(s i )|θ2) is the evaluator network's response to the expected action π(s i |θ2) and state s i The output of , and wherein the evaluator loss function is defined according to the following equation:

[0049] L Q (θ2)=[Q(s t ,a t )-b t ] 2

[0050] where b t =R(s t ,a t )+γQ(s t+1 ,π(s t+1 |θ1)|θ2),π(s t+1 |θ1) is the action specified by the actor network, and γ is a decimal factor.

[0051] In some non-limiting embodiments or aspects, the plurality of source samples are associated with a plurality of transactions in a transaction processing network, wherein the plurality of labeled normal samples are associated with a plurality of non-fraudulent transactions among the plurality of transactions, and wherein the plurality of labeled abnormal samples are associated with a plurality of fraudulent transactions among the plurality of transactions.

[0052] In some non-limiting embodiments or aspects, the program instructions, when executed by at least one processor, further cause the at least one processor to: receive transaction data associated with a transaction currently being processed in the transaction processing network; process the transaction data using the trained machine learning classifier to classify the transaction as a fraudulent transaction or a non-fraudulent transaction; and in response to classifying the transaction as a fraudulent transaction, refuse to authorize the transaction in the transaction processing network.

[0053] In some non-limiting embodiments or aspects, the program instructions, when executed by at least one processor, further cause the at least one processor to: before executing the training segment: use the training data set X 训练 To train the machine learning classifier φ; and for the training data set X 训练 For each source sample in the plurality of source samples, each k nearest neighbor is pre-computed.

[0054] Additional embodiments or aspects are set forth in the following numbered clauses:

[0055] Clause 1: A method comprising: obtaining, using at least one processor, a training data set X 训练 , the training data set includes a plurality of source samples, the plurality of source samples include a plurality of labeled normal samples and a plurality of labeled abnormal samples; and executing a training segment using the at least one processor by: (i) initializing a timestamp t; (ii) receiving an action vector a of the timestamp t from the actor network π of an actor-evaluator framework including an actor network π and an evaluator network Q t , where the actor network π is configured based on the state s t To generate the action vector a t , where the state s t is determined based on a current pair of source samples in the plurality of source samples, and wherein the action vector a t comprising a nearest neighbor size k, a composition ratio α, an oversampling number n and a termination probability ε; (iii) combining the current pair of source samples according to the composition ratio α and the oversampling number n to generate a sample with the label y syn The associated labeled synthetic sample x syn ; (iv) using the labeled synthetic sample x syn and the marker y syn to train the machine learning classifier φ; (v) obtaining the labeled synthetic sample x based on the nearest neighbor size k syn k source samples in the nearest neighborhood; (vi) for the labeled synthetic sample x synThe k nearest neighbors of the source samples and validation dataset X val The training dataset X 训练 (vii) from the labeled synthetic samples x syn (viii) storing the state s in a memory buffer; t , the action vector a t , next state s t+1 and reward r t , where the next state s t+1 is determined based on the next pair of source samples, and wherein the reward r t is determined based on the multiple classifier outputs; (ix) determining whether the termination probability ε satisfies a termination threshold; (x) in response to determining that the termination probability ε fails to satisfy the termination threshold, incrementing the timestamp t for multiple training steps S: according to the state s t , the action vector a t and the reward r t to train the evaluator network Q according to an evaluator loss function of ; and to train the actor network π according to an actor loss function that depends on the output of the evaluator network, and after training the actor network π and the evaluator network Q for the multiple training steps S, return to step (ii), wherein the next pair of source samples is used as the current pair of source samples; (xi) in response to determining that the termination probability ε satisfies the termination threshold, determine whether the number of executed training segments satisfies the threshold number of training segments; (xii) in response to determining that the number of executed training segments fails to satisfy the threshold number of training segments, return to step (i) to execute the next training segment; and (xiii) in response to determining that the number of executed training segments satisfies the threshold number of training segments, provide the machine learning classifier φ.

[0056] Clause 2: The method of clause 1, wherein the current pair of source samples is combined according to the composition ratio α to generate the labeled synthetic sample x according to the following equation syn :

[0057] x syn =α*x0+(1-α)*x1

[0058]

[0059] Where x0 is the first sample in the current pair of samples, x1 is the second sample in the current pair of samples, and y synis the labeled synthetic sample x syn of hard marks, y0 is the first hard mark value, and y1 is the second hard mark value.

[0060] Clause 3: A method according to clause 1 or 2, wherein the reward r t is determined according to the following equation:

[0061]

[0062] Where M is the evaluation metric, ΔM(φ t ) measures the trained classifier φ t Performance improvements, X val is the validation dataset, y val is the labeled set of the training dataset, is the baseline at timestamp t, m is a hyperparameter defining the size of the buffer used to form the baseline, and C(φt|st,at) evaluates the trained classifier φ t is the model confidence, P is the model exploration function, and k is the action vector a t Specifies the nearest neighbor size, x i is the labeled synthetic sample x at timestamp t syn k nearest neighbors, and y i is x i Mark.

[0063] Clause 4: A method according to any one of clauses 1 to 3, wherein the actor loss function is defined according to the following equation:

[0064]

[0065] Where N is the number of transitions, π(s i |θ2) is the state s i The expected action, and Q(s i ,π(s i )|θ2) is the evaluator network's response to the expected action π(s i |θ2) and state s i The output of

[0066] The evaluator loss function is defined according to the following equation:

[0067] L Q (θ2)=[Q(s r ,a t )-b t ] 2

[0068] where bt =R(s t ,a t )+γQ(s t+1 ,π(s t+1 |θ1)|θ2),π(s t+1 |θ1) is the action specified by the actor network, and γ is a decimal factor.

[0069] Clause 5: A method according to any one of clauses 1 to 4, wherein the plurality of source samples are associated with a plurality of transactions in a transaction processing network, wherein the plurality of labeled normal samples are associated with a plurality of non-fraudulent transactions among the plurality of transactions, and wherein the plurality of labeled abnormal samples are associated with a plurality of fraudulent transactions among the plurality of transactions.

[0070] Clause 6: A method according to any one of clauses 1 to 5, further comprising: using the at least one processor to receive transaction data associated with a transaction currently being processed in the transaction processing network; using the at least one processor to process the transaction data using the trained machine learning classifier to classify the transaction as a fraudulent transaction or a non-fraudulent transaction; and in response to classifying the transaction as a fraudulent transaction, using the at least one processor to refuse to authorize the transaction in the transaction processing network.

[0071] Clause 7: The method according to any one of clauses 1 to 6, further comprising: before executing the training segment: using the at least one processor to use the training data set X 训练 To train the machine learning classifier φ; and for the training data set X 训练 For each source sample in the plurality of source samples, using the at least one processor to pre-compute each k-nearest neighbor.

[0072] Clause 8: A system comprising: at least one processor, the at least one processor being programmed and / or configured to: obtain a training data set X 训练 , the training data set includes a plurality of source samples, the plurality of source samples include a plurality of labeled normal samples and a plurality of labeled abnormal samples; and a training segment is performed by: (i) initializing a timestamp t; (ii) receiving an action vector a of the timestamp t from the actor network π of an actor-critic framework including an actor network π and a critic network Q t , where the actor network π is configured based on the state s t To generate the action vector a t , where the state s t is determined based on a current pair of source samples in the plurality of source samples, and wherein the action vector at comprising a nearest neighbor size k, a composition ratio α, an oversampling number n and a termination probability ε; (iii) combining the current pair of source samples according to the composition ratio α and the oversampling number n to generate a sample with the label y syn The associated labeled synthetic sample x syn ; (iv) using the labeled synthetic sample x syn and the marker y syn to train the machine learning classifier φ; (v) obtaining the labeled synthetic sample x based on the nearest neighbor size k syn k source samples in the nearest neighborhood; (vi) for the labeled synthetic sample x syn The k nearest neighbors of the source samples and validation dataset X val The training dataset X 训练 (vii) from the labeled synthetic samples x syn (viii) storing the state s in a memory buffer; t , the action vector a t , next state s t+1 and reward r t , where the next state s t+1 is determined based on the next pair of source samples, and wherein the reward r t is determined based on the multiple classifier outputs; (ix) determining whether the termination probability ε satisfies a termination threshold; (x) in response to determining that the termination probability ε fails to satisfy the termination threshold, incrementing the timestamp t for multiple training steps S: according to the state s t , the action vector a t and the reward r t to train the evaluator network Q according to an evaluator loss function of ; and to train the actor network π according to an actor loss function that depends on the output of the evaluator network, and after training the actor network π and the evaluator network Q for the multiple training steps S, return to step (ii), wherein the next pair of source samples is used as the current pair of source samples; (xi) in response to determining that the termination probability ε satisfies the termination threshold, determine whether the number of executed training segments satisfies the threshold number of training segments; (xii) in response to determining that the number of executed training segments fails to satisfy the threshold number of training segments, return to step (i) to execute the next training segment; and (xiii) in response to determining that the number of executed training segments satisfies the threshold number of training segments, provide the machine learning classifier φ.

[0073] Clause 9: The system of clause 8, wherein the current pair of source samples are combined according to the composition ratio α to generate the labeled synthetic sample x according to the following equation syn :

[0074] x syn =α*x0+(1-α)*x1

[0075]

[0076] Where x0 is the first sample in the current pair of samples, x1 is the second sample in the current pair of samples, and y syn is the labeled synthetic sample x syn of hard marks, y0 is the first hard mark value, and y1 is the second hard mark value.

[0077] Clause 10: A system according to clause 8 or 9, wherein the reward r t is determined according to the following equation:

[0078]

[0079] Where M is the evaluation metric, ΔM(φ t ) measures the trained classifier φ t Performance improvements, X val is the validation dataset, y val is the labeled set of the training dataset, where is the baseline at timestamp t, m is a hyperparameter defining the size of the buffer used to form the baseline, and C(φt|st,at) evaluates the trained classifier φ t is the model confidence, P is the model exploration function, and k is the action vector a t Specifies the nearest neighbor size, x i is the labeled synthetic sample x at timestamp t syn k nearest neighbors, and y i is x i Mark.

[0080] Clause 11: The system of any of clauses 8 to 10, wherein the actor loss function is defined according to the following equation:

[0081]

[0082] Where N is the number of transitions, π(s i |θ2) is the state s i The expected action, and Q(s i ,π(s i)|θ2) is the evaluator network's response to the expected action π(s i |θ2) and state s i The output of

[0083] The evaluator loss function is defined according to the following equation:

[0084] L Q (θ2)=[Q(s t ,a t )-b t ] 2

[0085] where b t =R(s t ,a t )+γQ(s t+1 ,π(s t+1 |θ1)|θ2),π(s t+1 |θ1) is the action specified by the actor network, and γ is a decimal factor.

[0086] Clause 12: A system according to any one of clauses 8 to 11, wherein the plurality of source samples are associated with a plurality of transactions in a transaction processing network, wherein the plurality of labeled normal samples are associated with a plurality of non-fraudulent transactions among the plurality of transactions, and wherein the plurality of labeled abnormal samples are associated with a plurality of fraudulent transactions among the plurality of transactions.

[0087] Clause 13: A system according to any one of clauses 8 to 12, wherein the at least one processor is further programmed and / or configured to: receive transaction data associated with a transaction currently being processed in the transaction processing network; process the transaction data using the trained machine learning classifier to classify the transaction as a fraudulent transaction or a non-fraudulent transaction; and in response to classifying the transaction as a fraudulent transaction, refuse to authorize the transaction in the transaction processing network.

[0088] Clause 14: A system according to any one of clauses 8 to 13, wherein the at least one processor is further programmed and / or configured to: before executing the training segment: use the training data set X 训练 To train the machine learning classifier φ; and for the training data set X 训练 For each source sample in the plurality of source samples, each k-nearest neighbor is pre-computed.

[0089] Clause 15: A computer program product comprising a non-transitory computer readable medium comprising program instructions which, when executed by at least one processor, cause the at least one processor to: obtain a training data set X 训练 , the training data set includes a plurality of source samples, the plurality of source samples include a plurality of labeled normal samples and a plurality of labeled abnormal samples; and a training segment is performed by: (i) initializing a timestamp t; (ii) receiving an action vector a of the timestamp t from the actor network π of an actor-critic framework including an actor network π and a critic network Q t , where the actor network π is configured based on the state s t To generate the action vector a t , where the state s t is determined based on a current pair of source samples in the plurality of source samples, and wherein the action vector a t comprising a nearest neighbor size k, a composition ratio α, an oversampling number n and a termination probability ε; (iii) combining the current pair of source samples according to the composition ratio α and the oversampling number n to generate a sample with the label y syn The associated labeled synthetic sample x syn ; (iv) using the labeled synthetic sample x syn and the marker y syn to train the machine learning classifier φ; (v) obtaining the labeled synthetic sample x based on the nearest neighbor size k syn k source samples in the nearest neighborhood; (vi) for the labeled synthetic sample x syn The k nearest neighbors of the source samples and validation dataset X val The training dataset X 训练 (vii) from the labeled synthetic samples x syn (viii) storing the state s in a memory buffer; t , the action vector a t , next state s t+1 and reward r t , where the next state s t+1 is determined based on the next pair of source samples, and wherein the reward r tis determined based on the multiple classifier outputs; (ix) determining whether the termination probability ε satisfies a termination threshold; (x) in response to determining that the termination probability ε fails to satisfy the termination threshold, incrementing the timestamp t for multiple training steps S: according to the state s t , the action vector a t and the reward r t to train the evaluator network Q according to an evaluator loss function of ; and to train the actor network π according to an actor loss function that depends on the output of the evaluator network, and after training the actor network π and the evaluator network Q for the multiple training steps S, return to step (ii), wherein the next pair of source samples is used as the current pair of source samples; (xi) in response to determining that the termination probability ε satisfies the termination threshold, determine whether the number of executed training segments satisfies the threshold number of training segments; (xii) in response to determining that the number of executed training segments fails to satisfy the threshold number of training segments, return to step (i) to execute the next training segment; and (xiii) in response to determining that the number of executed training segments satisfies the threshold number of training segments, provide the machine learning classifier φ.

[0090] Clause 16: The computer program product of clause 15, wherein the current pair of source samples is combined according to the composition ratio α to generate the labeled synthetic sample x according to the following equation syn :

[0091] x syn =α*x0+(1-α)*x1

[0092]

[0093] Where x0 is the first sample in the current pair of samples, x1 is the second sample in the current pair of samples, and y syn is the labeled synthetic sample x syn of hard marks, y0 is the first hard mark value, and y1 is the second hard mark value.

[0094] Clause 17: A computer program product according to claim 15 or 16, wherein the reward r t is determined according to the following equation:

[0095]

[0096] Where M is the evaluation metric, ΔM(φ t ) measures the trained classifier φ t Performance improvements, X val is the validation dataset, y valis the labeled set of the training dataset, where is the baseline at timestamp t, m is a hyperparameter defining the size of the buffer used to form the baseline, and C(φt|st,at) evaluates the trained classifier φ t is the model confidence, P is the model exploration function, and k is the action vector a t Specifies the nearest neighbor size, x i is the labeled synthetic sample x at timestamp t syn k nearest neighbors, and y i is x i Mark.

[0097] Clause 18: A computer program product according to any one of clauses 15 to 17, wherein the actor loss function is defined according to the following equation:

[0098]

[0099] Where N is the number of transitions, π(s i |θ2) is the state s i The expected action, and Q(s i ,π(s i )|θ2) is the evaluator network's response to the expected action π(s i |θ2) and state s i The output of

[0100] The evaluator loss function is defined according to the following equation:

[0101] L Q (θ2)=[Q(s t ,a t )-b t ] 2

[0102] where b t =R(s t ,a t )+γQ(s t+1 ,π(s t+1 |θ1)|θ2),π(s t+1 |θ1) is the action specified by the actor network, and γ is a decimal factor.

[0103] Clause 19: A computer program product according to any one of clauses 15 to 18, wherein the plurality of source samples are associated with a plurality of transactions in a transaction processing network, wherein the plurality of labeled normal samples are associated with a plurality of non-fraudulent transactions among the plurality of transactions, and wherein the plurality of labeled abnormal samples are associated with a plurality of fraudulent transactions among the plurality of transactions.

[0104] Clause 20: A computer program product according to any one of clauses 15 to 19, wherein the program instructions, when executed by at least one processor, further cause the at least one processor to: receive transaction data associated with a transaction currently being processed in the transaction processing network; process the transaction data using the trained machine learning classifier to classify the transaction as a fraudulent transaction or a non-fraudulent transaction; and in response to classifying the transaction as a fraudulent transaction, refuse to authorize the transaction in the transaction processing network.

[0105] Clause 21: A computer program product according to any one of clauses 15 to 20, wherein the program instructions, when executed by at least one processor, further cause the at least one processor to: before executing the training segment: use the training data set X 训练 To train the machine learning classifier φ; and for the training data set X 训练 For each source sample in the plurality of source samples, each k nearest neighbor is pre-computed.

[0106] These and other features and characteristics of the present disclosure, as well as methods of operation and functions of combinations of related structural elements and parts and economies of manufacture, will become more apparent when considering the following description and the appended claims with reference to the accompanying drawings, all of which form a part of this specification, wherein like reference numerals indicate corresponding parts in the various figures. However, it should be expressly understood that the various figures are for illustration and description purposes only and are not intended as definitions of limitations. Unless the context clearly dictates otherwise, when used in this specification and claims, the singular forms "a", "an", and "the" include plural referents. BRIEF DESCRIPTION OF THE DRAWINGS

[0107] Additional advantages and details are explained in more detail below with reference to exemplary embodiments shown in the schematic drawings, in which:

[0108] Figure 1 are illustrations of non-limiting embodiments or aspects of environments in which the systems, apparatus, products, devices, and / or methods described herein may be implemented;

[0109] Figure 2 yes Figure 1 Illustrations of non-limiting embodiments or aspects of one or more devices and / or components of one or more systems;

[0110] Figures 3A-3C is a flow chart of a non-limiting embodiment or aspect of a process for enhancing synthetic oversampling for supervised anomaly detection;

[0111] Figure 4 are graphs providing comparisons between a random mixture of anomalies (left) and a random mixture of anomalies and normal samples (right);

[0112] Figure 5 A non-limiting embodiment or aspect of an iterative mixing process is shown;

[0113] Figure 6 Non-limiting embodiments or aspects of a network architecture for synthetic oversampling for enhanced supervised anomaly detection are presented;

[0114] Figure 7 Non-limiting examples or implementations of aspects of a training procedure for a process of synthetic oversampling for enhanced supervised anomaly detection are presented;

[0115] Figure 8 is a table summarizing the statistics of a dataset;

[0116] Fig. 9 is a table summarizing a horizontal comparison between a framework and representative data augmentation methods according to non-limiting embodiments or aspects of the present disclosure;

[0117] Fig.10 is a table outlining a longitudinal comparison between a framework and a detection algorithm for tag notifications according to a non-limiting embodiment or aspect of the present disclosure; and

[0118] Fig.11 is a table of macro-average scores for frameworks according to non-limiting embodiments or aspects of the present disclosure. DETAILED DESCRIPTION

[0119] It should be understood that, except where expressly specified to the contrary, the present disclosure may employ various alternative variations and step sequences. It should also be understood that the specific devices and processes shown in the accompanying drawings and described in the following specification are merely exemplary and non-limiting embodiments or aspects. Therefore, specific dimensions and other physical characteristics associated with the embodiments or aspects disclosed herein should not be considered limiting.

[0120] The aspects, components, elements, structures, actions, steps, functions, instructions, etc. used herein should not be understood as being critical or necessary unless explicitly described as such. Also, as used herein, the articles "a" and "an" are intended to include one or more items and can be used interchangeably with "one or more" and "at least one". In addition, as used herein, the term "set" is intended to include one or more items (e.g., related items, unrelated items, a combination of related items and unrelated items, etc.), and can be used interchangeably with "one or more" or "at least one". In the case of intending to have only one item, the term "one" or similar language is used. Moreover, as used herein, the term "having" or its analog is intended to be an open term. In addition, unless otherwise explicitly stated, the phrase "based on" is intended to mean "based at least in part on".

[0121] As used herein, the term "communication" may refer to the reception, acceptance, transmission, transfer, provision, etc. of data (e.g., information, signals, messages, instructions, commands, etc.). A unit (e.g., a device, a system, a component of a device or system, a combination thereof, and / or the like) communicating with another unit means that the unit is able to directly or indirectly receive information from the other unit and / or send information to the other unit. This may refer to a direct or indirect connection (e.g., a direct communication connection, an indirect communication connection, and / or the like) that is wired and / or wireless in nature. In addition, although the information sent may be modified, processed, relayed, and / or routed between the first unit and the second unit, the two units may also communicate with each other. For example, even if the first unit passively receives information and does not actively send information to the second unit, the first unit may communicate with the second unit. As another example, if at least one intermediate unit processes the information received from the first unit and transmits the processed information to the second unit, the first unit may communicate with the second unit.

[0122] Obviously, the systems and / or methods described herein can be implemented in different forms of hardware, software, or a combination of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods does not limit the implementation. Therefore, the operation and behavior of the systems and / or methods are described herein without reference to specific software codes, and it should be understood that software and hardware can be designed to implement the systems and / or methods based on the description herein.

[0123] Some non-limiting embodiments or aspects are described herein in conjunction with a threshold value. As used herein, satisfying a threshold value may refer to a value greater than a threshold value, more than a threshold value, above a threshold value, greater than or equal to a threshold value, less than a threshold value, less than a threshold value, below a threshold value, less than or equal to a threshold value, equal to a threshold value, etc.

[0124] As used herein, the term "transaction service provider" may refer to an entity that receives transaction authorization requests from merchants or other entities and, in some cases, provides payment assurance through an agreement between the transaction service provider and the issuer organization. For example, a transaction service provider may include, for example The term "transaction processing system" may refer to one or more computing devices operated by or on behalf of a transaction service provider, such as a transaction processing server executing one or more software applications. A transaction processing system may include one or more processors and, in some non-limiting embodiments, may be operated by or on behalf of a transaction service provider.

[0125] As used herein, the term "account identifier" may include one or more primary account numbers (PANs), tokens, or other identifiers associated with a customer account. The term "token" may refer to an identifier used as a replacement or substitute identifier for an original account identifier such as a PAN. An account identifier may be alphanumeric, or any combination of characters and / or symbols. A token may be associated with a PAN or other original account identifier in one or more data structures (e.g., one or more databases, etc.) such that the token can be used to conduct transactions without directly using the original account identifier. In some instances, an original account identifier such as a PAN may be associated with multiple tokens for different individuals or purposes.

[0126] As used herein, the terms "issuer institution", "portable financial device issuer", "issuer" or "issuer bank" may refer to one or more entities that provide one or more accounts to a user (e.g., a customer, consumer, organization, etc.) to conduct transactions (e.g., payment transactions), such as initiating a credit card payment transaction and / or a debit card payment transaction. For example, an issuer institution may provide an account identifier such as a PAN to a user that uniquely identifies one or more accounts associated with the user. The account identifier may be embodied on a portable financial device such as a physical financial instrument (e.g., a payment card), and / or may be electronic and used for electronic payments. In some non-limiting embodiments or aspects, the issuer institution may be associated with a bank identification number (BIN) that uniquely identifies the issuer institution. As used herein, the term "issuer institution system" may refer to one or more computer systems operated by or on behalf of the issuer institution, such as a server computer that executes one or more software applications. For example, an issuer institution system may include one or more authorization servers for authorizing payment transactions.

[0127] As used herein, the term "merchant" may refer to an individual or entity that provides goods and / or services or access to goods and / or services to a user (e.g., a customer) based on a transaction (e.g., a payment transaction). As used herein, the term "merchant" or "merchant system" may also refer to one or more computer systems, computing devices, and / or software applications operated by or on behalf of a merchant, such as a server computer that executes one or more software applications. As used herein, a "point of sale (POS) system" may refer to one or more computers and / or peripheral devices used by a merchant to conduct payment transactions with a user, including one or more card readers, near field communication (NFC) receivers, radio frequency identification (RFID) receivers, and / or other contactless transceivers or receivers, contact-based receivers, payment terminals, computers, servers, input devices, and / or other similar devices that can be used to initiate payment transactions. A POS system may be part of a merchant system. A merchant system may also include a merchant plug-in for facilitating online Internet-based transactions through a merchant webpage or software application. A merchant plug-in may include software that runs on a merchant server or is hosted by a third party to facilitate such online transactions.

[0128] As used herein, the term "mobile device" may refer to one or more portable electronic devices configured to communicate with one or more networks. As an example, a mobile device may include a cellular phone (e.g., a smart phone or a standard cellular phone), a portable computer (e.g., a tablet computer, a laptop computer, etc.), a wearable device (e.g., a watch, glasses, lenses, clothing, etc.), a personal digital assistant (PDA), and / or other similar devices. As used herein, the terms "client device" and "user device" refer to any electronic device configured to communicate with one or more servers or remote devices and / or systems. Client devices or user devices may include mobile devices, network-enabled appliances (e.g., network-enabled TVs, refrigerators, thermostats, etc.), computers, POS systems, and / or any other device or system capable of communicating with a network.

[0129] As used herein, the term "computing device" may refer to one or more electronic devices configured to process data. In some instances, a computing device may include the necessary components to receive, process, and output data, such as a processor, a display, a memory, an input device, a network interface, and / or the like. The computing device may be a mobile device. As an example, a mobile device may include a cellular phone (e.g., a smartphone or a standard cellular phone), a portable computer, a wearable device (e.g., a watch, glasses, lenses, clothing, and / or the like), a PDA, and / or other similar devices. The computing device may also be a desktop computer or other form of non-mobile computer.

[0130] As used herein, the term "payment device" may refer to a portable financial device, an electronic payment device, a payment card (e.g., a credit or debit card), a gift card, a smart card, smart media, a payroll card, a healthcare card, a wristband, a machine-readable medium containing account information, a key chain device or pendant, an RFID transponder, a retailer discount or membership card, a cellular phone, an electronic wallet mobile application, a PDA, a pager, a security card, a computer, an access card, a wireless terminal, a transponder, etc. In some non-limiting embodiments or aspects, a payment device may include volatile or non-volatile memory to store information (e.g., an account identifier, an account holder's name, etc.).

[0131] As used herein, the terms "server" and / or "processor" may refer to or include one or more computing devices operated by or facilitating communications and processing by multiple parties in a network environment such as the Internet, but it should be understood that communications may be facilitated through one or more public or private network environments, and that there may be various other arrangements. In addition, multiple computing devices (e.g., servers, POS devices, mobile devices, etc.) that communicate directly or indirectly in a network environment may constitute a "system." As used herein, references to "server" or "processor" may refer to previously described servers and / or processors, different servers and / or processors, and / or combinations of servers and / or processors that are stated to implement previous steps or functions. For example, as used in the specification and claims, a first server and / or first processor stated to perform a first step or function may refer to the same or different servers and / or processors stated to perform a second step or function.

[0132] As used herein, the term "acquirer" may refer to an entity that is licensed by a transaction service provider and / or approved by a transaction service provider to initiate a transaction using a portable financial device of a transaction service provider. An acquirer may also refer to one or more computer systems operated by or on behalf of an acquirer, such as a server computer (e.g., "acquirer server") that executes one or more software applications. An "acquirer" may be a merchant bank, or in some cases, a merchant system may be an acquirer. The transaction may include an original credit transaction (OCT) and an account funds transaction (AFT). A transaction service provider may authorize an acquirer to sign a merchant of a service provider to initiate a transaction using a portable financial device of a transaction service provider. An acquirer may sign a contract with a payment service provider to enable the service provider to sponsor a merchant. An acquirer may monitor the compliance of a payment service provider according to the regulations of a transaction service provider. An acquirer may perform due diligence on a payment service provider and ensure that appropriate due diligence is performed before signing a sponsored merchant. An acquirer may be responsible for all transaction service provider programs that they operate or sponsor. An acquirer can be held responsible for the actions of its payment facilitator and the merchants it or its payment facilitator sponsors.

[0133] As used herein, the term "payment gateway" may refer to an entity and / or a payment processing system operated by or on behalf of such an entity, the entity (e.g., merchant service provider, payment service provider, payment service provider, payment service provider with a contract with an acquirer, payment aggregator, etc.) providing payment services (e.g., transaction service provider payment services, payment processing services, etc.) to one or more merchants. The payment service may be associated with the use of a portable financial device managed by a transaction service provider. As used herein, the term "payment gateway system" may refer to one or more computer systems, computer devices, servers, server groups, etc. operated by or on behalf of a payment gateway.

[0134] As used herein, the term "authentication system" may refer to one or more computing devices that authenticate users and / or accounts, such as, but not limited to, transaction processing systems, merchant systems, issuer systems, payment gateways, third-party authentication services, etc.

[0135] As used herein, the terms "request," "response," "request message," and "response message" may refer to one or more messages, data packets, signals, and / or data structures used to transfer data between two or more components or units.

[0136] As used herein, the term "application programming interface" (API) may refer to computer code that allows communication between different systems or (hardware and / or software) system components. For example, an API may include function calls, functions, subroutines, communication protocols, fields, etc. that can be used and / or accessed by other systems or other (hardware and / or software) system components.

[0137] As used herein, the term "user interface" or "graphical user interface" refers to a generated display, such as one or more graphical user interfaces (GUIs), with which a user can interact directly or indirectly (e.g., via keyboard, mouse, touch screen, etc.).

[0138] Anomaly detection is widely used in a wide range of fields, such as intrusion detection in cybersecurity, fault detection in manufacturing, and fraud detection in finance. Anomalies in data often come from different factors, resulting in multiple abnormal behaviors with significantly different characteristics. For example, different fraudulent transactions can embody completely different behaviors. By definition, anomalous behaviors usually occur rarely in a dataset and are unpredictable. Therefore, it is difficult, if not impossible, to obtain well-labeled training data. For example, given a set of transactions, it takes less effort to distinguish fraudulent transactions from normal transactions than to classify fraudulent transactions, especially when there is no clear definition of the behavior categories. This often results in a limited amount of single-type labeled information with different semantic meanings and is therefore not suitable for supervised training of anomaly detectors.

[0139] Still, label information plays an important role in improving detection performance. In order to better utilize the sparsity of label information, existing efforts focus on weak / semi-supervised learning and data augmentation to overcome the label sparsity problem. Weak / semi-supervised learning methods attempt to extract additional information from given labels with customized loss functions or scoring functions. Although weak / semi-supervised learning is able to capture label information, weak / semi-supervised learning focuses on learning knowledge from limited anomaly samples and therefore ignores supervisory signals of possible anomalies in unlabeled data. To overcome this limitation, some data augmentation methods focus on synthetic oversampling to synthesize minority class species (e.g., anomalies) to create a balanced dataset for training supervised classifiers. However, anomaly behaviors are different, and synthetic anomalies based on two arbitrary anomalies may introduce noise into the training dataset.

[0140] For example, there are mainly two existing strategies to leverage limited labeled information to solve the anomaly detection problem: label-informed anomaly detection and data augmentation.

[0141] Weakly / semi-supervised anomaly detection is accessible as two main existing strategies to solve the problem in scenarios with labeled normal or anomalous samples. To leverage a large number of labeled normal samples, SAnDCat selects the top K representative samples from the dataset as references to evaluate anomaly scores based on a model learned from pairwise distances between labeled normal instances. On the other hand, to leverage a limited number of labeled anomalies, DevNet implements anomaly scores for individual data instances to fit a one-sided Gaussian distribution, thereby leveraging prior knowledge of labeled normal samples and anomalies. PRO introduces a two-stream ordered regression network to learn pairwise relations between two data samples, which is assumption-free for the probability distribution of anomaly scores.

[0142] Recently, several efforts have further generalized the anomaly detection problem of label notification to a semi-supervised classification setting, which has access to a limited number of normal and abnormal samples. The main underlying assumption assumes that similar points may be of the same kind and are therefore densely distributed in the same high-density region of a low-dimensional feature space. XGBOD extracts feature representations based on the anomaly scores of multiple unsupervised anomaly detectors used to train supervised gradient boosted tree classifiers. DeepSAD points out that the semi-supervised assumption is only applicable to normal samples, and further develops a class of classification frameworks to cluster labeled normal samples while maximizing the distance between labeled anomalies and clusters in high-dimensional space. However, weak / semi-supervised learning methods focus on modeling given label information without considering the relationship between two labeled instances. Therefore, it is not feasible to generalize the label information when the abnormal behavior is different. By considering the correlation between labeled samples and generating beneficial training data accordingly, non-limiting embodiments or aspects of the present disclosure may be able to generalize the label information used to train any classifier.

[0143] Data augmentation has been widely studied for various data types to expand the training data size and generalize the model decision boundary, thereby improving performance and robustness. To address the imbalanced classification problem, there are mainly two different categories: algorithm-by-algorithm and data-by-data methods. Algorithm-by-algorithm methods directly customize the loss function of the classification model to better fit the data distribution. However, modifying the loss function only helps to fit the labeled information well and may be affected by the generalization of the labeled information when the minority class behaves differently. Data-by-data methods generate new samples into the dataset of the minority class or remove existing samples from the dataset of the majority class. Synthetic Minority Oversampling (SMOTE) generates new minority samples by linearly combining minority samples with their k nearest minority instances with a manually selected neighborhood size and the number of synthetic instances. A series of advances in SMOTE further introduced density estimation, data distribution-aware sampling to address the class imbalance problem without manually selecting the neighborhood size and the number of synthetic instances.

[0144] Instead of sampling synthetic data for a single category, mixing achieves significant improvements in the image domain by linearly combining two random samples from different categories at a given combination ratio and creating soft labels for training neural networks to synthesize data points. Since mixing assumes that all categories are uniformly distributed for image classification tasks, it is not applicable when the category distribution is skewed. To address this limitation, mixing-enhancement introduces a skewed probability distribution to sample the combination ratio for linearly combining two heterogeneous samples. However, the unbalanced classification problem assumes that minority class samples are clustered in the feature space, which may not be true when the minority class category is abnormal. To this end, a non-limiting embodiment or aspect of the present disclosure may consider the attributes of a pair of normal samples and abnormal samples for jointly identifying the best k nearest neighbors and combination ratios. A non-limiting embodiment or aspect of the present disclosure may generate synthetic samples at a combination ratio and identify the next pair of samples within the k nearest neighbors. In this way, a non-limiting embodiment or aspect of the present disclosure may be able to exploit label information while exploring abnormalities of different distributions.

[0145] Motivated by the recent success of domain-agnostic data mixing techniques in image domain and imbalanced classification problems, a preliminary study is performed on a toy dataset to compare randomly mixed anomalies with randomly mixed anomalies and normal samples. The dataset simulates different anomaly behaviors. Figure 4 As shown, this is a graph providing a comparison between randomly mixing anomalies (left) and randomly mixing anomalies and normal samples (right), in which the black data points are synthetic samples and the edge colors are their assigned labels (red as anomalies and blue as normal samples). Randomly mixing anomalies produces synthetic samples that appear to be no different from normal samples, labeled as anomalies (reducing the F1-score from 0.73 to 0.66). However, with proper control of the composition ratio of anomalies and normal samples, the noise of the synthetic samples generated by mixing anomalies with normal samples is significantly smaller, and therefore benefits the base classifier (improving the F1-score from 0.73 to 0.86). However, choosing an appropriate composition ratio is not an easy task, and therefore, different pairs of anomalies and normal samples may require different mixing strategies for high-quality synthetic samples.

[0146] In order to solve the above problems, it may be necessary to develop an integrated framework to generalize the knowledge of labeled anomalies of any classifier, with the goal of advancing supervised anomaly detection. Specifically, given a set of labeled samples, the goal can be to identify a data augmentation strategy that mixes labeled normal samples with anomalies. In this way, the prior knowledge of the labeling information can be generalized, and the resulting synthetic samples can be used to train the classifier to achieve maximum performance improvement. In order to achieve the goal, the non-limiting embodiments or aspects of the present disclosure can learn a sample-by-sample strategy that maps the characteristic attributes of each data sample to the data augmentation strategy. At the same time, the state of model training can be used as a reference to guide data augmentation.

[0147] However, developing such a framework can be very challenging for the following reasons. First, since existing data augmentation techniques only create synthetic samples based on feature distributions, there is no existing technique that simultaneously considers both feature distributions and the model state used to synthesize new samples. Second, even if the model state can be viewed as creating synthetic samples, the model may not necessarily converge when synthesizing samples. In this way, when the model has not yet converged, the generated synthetic samples may not be beneficial. Third, the augmentation strategy can be composed of discrete values ​​and continuous values, and learning such a mapping can be challenging. For example, the composition ratio is a continuous number, where the oversampling number is a discrete number.

[0148] The non-limiting embodiments or aspects of the present disclosure may obtain a training data set X 训练 , the training data set includes a plurality of source samples, the plurality of source samples include a plurality of labeled normal samples and a plurality of labeled abnormal samples; and a training segment is performed by: (i) initializing a timestamp t; (ii) receiving an action vector a of the timestamp t from the actor network π of an actor-critic framework including an actor network π and a critic network Q t , where the actor network π is configured based on the state s t To generate the action vector a t , where the state s t is determined based on a current pair of source samples in the plurality of source samples, and wherein the action vector a t comprising a nearest neighbor size k, a composition ratio α, an oversampling number n and a termination probability ε; (iii) combining the current pair of source samples according to the composition ratio α and the oversampling number n to generate a sample with the label y syn The associated labeled synthetic sample x syn ; (iv) using the labeled synthetic sample x syn and the marker y syn to train the machine learning classifier φ; (v) obtaining the labeled synthetic sample x based on the nearest neighbor size k synk source samples in the nearest neighborhood; (vi) for the labeled synthetic sample x syn The k nearest neighbors of the source samples and validation dataset X val The training dataset X 训练 (vii) from the labeled synthetic samples x syn (viii) storing the state s in a memory buffer; t , the action vector a t , next state s t+1 and reward r t , where the next state s t+1 is determined based on the next pair of source samples, and wherein the reward r t is determined based on the multiple classifier outputs; (ix) determining whether the termination probability ε satisfies a termination threshold; (x) in response to determining that the termination probability ε fails to satisfy the termination threshold, incrementing the timestamp t for multiple training steps S: according to the state s t , the action vector a t and the reward r t to train the evaluator network Q according to an evaluator loss function of ; and to train the actor network π according to an actor loss function that depends on the output of the evaluator network, and after training the actor network π and the evaluator network Q for the multiple training steps S, return to step (ii), wherein the next pair of source samples is used as the current pair of source samples; (xi) in response to determining that the termination probability ε satisfies the termination threshold, determine whether the number of executed training segments satisfies the threshold number of training segments; (xii) in response to determining that the number of executed training segments fails to satisfy the threshold number of training segments, return to step (i) to execute the next training segment; and (xiii) in response to determining that the number of executed training segments satisfies the threshold number of training segments, provide the machine learning classifier φ.

[0149] In this way, non-limiting embodiments or aspects of the present disclosure may formulate the synthetic oversampling procedure as a Markov decision process and / or customize the exploration reward function for learning the data amplifier by exploring the uncertainty of the underlying supervised classifier. For example, by traversing the feature space of the original data set under the guidance of model performance and model uncertainty, the generated synthetic samples may follow the original data distribution and / or contain information that is not reflected in the original data set but may be beneficial to improving model performance. For example, non-limiting embodiments or aspects of the present disclosure may train a "data mixer" that generalizes the label information into synthetic data points for training classifiers. For example, in each step, a pair of data samples with different labels may be sampled as input to the data mixer, and / or a "mixing" or composition ratio may be output to create a synthetic sample for the training data, and / or the output k of the data mixer may be used to determine the next pair of samples based on the k nearest neighbors of the created synthetic samples. At the same time, the ε output by the data mixer may be used to draw probabilities to stop the synthetic oversampling process to suppress or prevent the model from overfitting the synthetic data samples. At each step, a combined reward signal can be used, which aims to improve the classification performance on the validation dataset while exploring the uncertainty of the base classifier.

[0150] Therefore, non-limiting embodiments or aspects of the present disclosure may formulate feature space traversal as a Markov decision process to solve the problem as a sequential decision problem with a deep reinforcement learning algorithm. In this way, instead of having a unified strategy for creating synthetic data samples, non-limiting embodiments or aspects of the present disclosure may customize the synthesis strategy for individual data points and different base classifiers to create fine-grained synthetic samples that provide useful information for enhancing the performance of anomaly detection. Further, non-limiting embodiments of the present disclosure may provide a reward function that focuses on the improvement of classification performance rather than the performance itself. In this way, even if the base classifier may not converge during the training procedure, the classifier can still provide meaningful feedback for training the data mixer. Still further, the reward function according to non-limiting embodiments or aspects of the present disclosure can explore model uncertainty, which can enable the data to identify potential beneficial information lost in the original data set for further creation of synthetic samples.

[0151] Reference now Figure 1 , Figure 1 1 is a diagram of an example environment 100 in which the apparatus, systems, methods, and / or products described herein may be implemented. Figure 1As shown in , environment 100 includes transaction processing network 101, user device 112 and / or communication network 116, and transaction processing network may include merchant system 102, payment gateway system 104, acquirer system 106, transaction service provider system 108, issuer system 110. Transaction processing network 101, merchant system 102, payment gateway system 104, acquirer system 106, transaction service provider system 108, issuer system 110 and / or user device 112 may be interconnected (e.g., establish connection for communication, etc.) via wired connection, wireless connection or a combination of wired connection and wireless connection.

[0152] The merchant system 102 may include one or more devices that are capable of receiving information and / or data from the payment gateway system 104, the acquirer system 106, the transaction service provider system 108, the issuer system 110, and / or the user device 112 (e.g., via a communication network 116, etc.) and / or transmitting information and / or data to the payment gateway system 104, the acquirer system 106, the transaction service provider system 108, the issuer system 110, and / or the user device 112 (e.g., via a communication network 116, etc.). The merchant system 102 may include a communication connection (e.g., an NFC communication connection, an RFID communication connection, The merchant system 102 may include a device that receives information and / or data from the user device 112 and / or transmits information and / or data to the user device 112 through the communication connection (communication connection, etc.). For example, the merchant system 102 may include a computing device, such as a server, a server group, a client device, a client device group, and / or other similar devices. In some non-limiting embodiments or aspects, the merchant system 102 may be associated with a merchant described herein. In some non-limiting embodiments or aspects, the merchant system 102 may include one or more devices that can be used by a merchant to conduct payment transactions with a user, such as a computer, a computer system, and / or a peripheral device. For example, the merchant system 102 may include a POS device and / or a POS system.

[0153] The payment gateway system 104 may include one or more devices that are capable of receiving information and / or data from the merchant system 102, the acquirer system 106, the transaction service provider system 108, the issuer system 110, and / or the user device 112 (e.g., via a communication network 116, etc.) and / or transmitting information and / or data to the merchant system 102, the acquirer system 106, the transaction service provider system 108, the issuer system 110, and / or the user device 112 (e.g., via a communication network 116, etc.). For example, the payment gateway system 104 may include a computing device, such as a server, a server group, and / or other similar devices. In some non-limiting embodiments or aspects, the payment gateway system 104 is associated with the payment gateway described herein.

[0154] The acquirer system 106 may include one or more devices that are capable of receiving information and / or data from the merchant system 102, the payment gateway system 104, the transaction service provider system 108, the issuer system 110, and / or the user device 112 (e.g., via a communication network 116, etc.) and / or transmitting information and / or data to the merchant system 102, the payment gateway system 104, the transaction service provider system 108, the issuer system 110, and / or the user device 112 (e.g., via a communication network 116, etc.). For example, the acquirer system 106 may include a computing device, such as a server, a server group, and / or other similar devices. In some non-limiting embodiments or aspects, the acquirer system 106 may be associated with an acquirer as described herein.

[0155] The transaction service provider system 108 may include one or more devices that are capable of receiving information and / or data from the merchant system 102, the payment gateway system 104, the acquirer system 106, the issuer system 110, and / or the user device 112 (e.g., via a communication network 116, etc.), and / or transmitting information and / or data to the merchant system 102, the payment gateway system 104, the acquirer system 106, the issuer system 110, and / or the user device 112 (e.g., via a communication network 116, etc.). For example, the transaction service provider system 108 may include a computing device, such as a server (e.g., a transaction processing server, etc.), a server group, and / or other similar devices. In some non-limiting embodiments or aspects, the transaction service provider system 108 may be associated with a transaction service provider described herein. In some non-limiting embodiments or aspects, the transaction service provider 108 may include and / or access one or more internal and / or external databases that include transaction data.

[0156] The issuer system 110 may include one or more devices that are capable of receiving information and / or data from the merchant system 102, the payment gateway system 104, the acquirer system 106, the transaction service provider system 108, and / or the user device 112 (e.g., via a communication network 116, etc.) and / or transmitting information and / or data to the merchant system 102, the payment gateway system 104, the acquirer system 106, the transaction service provider system 108, and / or the user device 112 (e.g., via a communication network 116, etc.). For example, the issuer system 110 may include a computing device, such as a server, a server group, and / or other similar devices. In some non-limiting embodiments or aspects, the issuer system 110 may be associated with an issuer organization described herein. For example, the issuer system 110 may be associated with an issuer organization that issues payment accounts or instruments (e.g., credit accounts, debit accounts, credit cards, debit cards, etc.) to users (e.g., users associated with the user device 112, etc.).

[0157] In some non-limiting embodiments or aspects, the transaction processing network 101 includes multiple systems in a communication path for processing transactions. For example, the transaction processing network 101 may include a merchant system 102, a payment gateway system 104, an acquirer system 106, a transaction service provider system 108, and / or an issuer system 110 in a communication path (e.g., a communication path, a communication channel, a communication network, etc.) for processing electronic payment transactions. For example, the transaction processing network 101 may process (e.g., initiate, conduct, authorize, etc.) electronic payment transactions via the communication path between the merchant system 102, the payment gateway system 104, the acquirer system 106, the transaction service provider system 108, and / or the issuer system 110.

[0158] The user device 112 may include one or more devices that are capable of receiving information and / or data from the merchant system 102, the payment gateway system 104, the acquirer system 106, the transaction service provider system 108, and / or the issuer system 110 (e.g., via a communication network 116, etc.) and / or transmitting information and / or data to the merchant system 102, the payment gateway system 104, the acquirer system 106, the transaction service provider system 108, and / or the issuer system 110 (e.g., via a communication network 116, etc.). For example, the user device 112 may include a client device, etc. In some non-limiting embodiments or aspects, the user device 112 is capable of communicating with the merchant system 102, the payment gateway system 104, the acquirer system 106, the transaction service provider system 108, and / or the issuer system 110 via a short-range wireless communication connection (e.g., an NFC communication connection, an RFID communication connection, The user device 112 may receive information via a short-range wireless communication connection (e.g., from the merchant system 102, etc.), and / or transmit information via a short-range wireless communication connection (e.g., to the merchant system 102). In some non-limiting embodiments or aspects, the user device 112 may include an application associated with the user device 112, such as an application stored on the user device 112, a mobile application stored and / or executed on the user device 112 (e.g., a mobile device application, a native application for the mobile device, a mobile cloud application for the mobile device, an electronic wallet application, an issuer bank application, etc.). In some non-limiting embodiments or aspects, for one or more transactions in the payment network, the user device 112 may be associated with a sender account and / or a receiving account in the payment network.

[0159] The communication network 116 may include one or more wired and / or wireless networks. For example, the communication network 116 may include a cellular network (e.g., a long term evolution (LTE) network, a third generation (3G) network, a fourth generation (4G) network, a fifth generation (5G) network, a code division multiple access (CDMA) network, etc.), a public land mobile network (PLMN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a telephone network (e.g., a public switched telephone network (PSTN), a private network, an ad hoc network, an intranet, the Internet, a fiber-optic-based network, a cloud computing network, etc., and / or a combination of these or other types of networks.

[0160] supply Figure 1 The number and arrangement of devices and systems shown are examples. Figure 1 Additional devices and / or systems, fewer devices and / or systems, different devices and / or systems, and / or differently arranged devices and / or systems than those shown. Furthermore, the present invention may be implemented within a single device and / or system. Figure 1 two or more of the devices and / or systems shown in, or Figure 1 The single device and / or system shown in the environment 100 may be implemented as multiple distributed devices and / or systems. Additionally or alternatively, a group of devices and / or systems (e.g., one or more devices or systems) of the environment 100 may perform one or more functions described as being performed by another group of devices and / or systems of the environment 100.

[0161] Reference now Figure 2 , Figure 2is a diagram of example components of a device 200. Device 200 may correspond to one or more devices of a merchant system 102, one or more devices of a payment gateway system 104, one or more devices of an acquirer system 106, one or more devices of a transaction service provider system 108, one or more devices of an issuer system 110, and / or a user device 112 (e.g., one or more devices of a system of a user device 112, etc.). In some non-limiting embodiments or aspects, one or more devices of a merchant system 102, one or more devices of a payment gateway system 104, one or more devices of an acquirer system 106, one or more devices of a transaction service provider system 108, one or more devices of an issuer system 110, and / or a user device 112 (e.g., one or more devices of a system of a user device 112, etc.) may include at least one device 200 and / or at least one component of a device 200. As Figure 2 As shown, apparatus 200 may include a bus 202 , a processor 204 , a memory 206 , a storage component 208 , an input component 210 , an output component 212 , and a communication interface 214 .

[0162] The bus 202 may include components that permit communication between components of the device 200. In some non-limiting embodiments or aspects, the processor 204 may be implemented in hardware, software, or a combination of hardware and software. For example, the processor 204 may include a processor (e.g., a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), etc.), a microprocessor, a digital signal processor (DSP), and / or any processing component that can be programmed to perform a function (e.g., a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc.). The memory 206 may include a random access memory (RAM), a read-only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic memory, optical memory, etc.) that stores information and / or instructions for use by the processor 204.

[0163] The storage component 208 may store information and / or software associated with the operation and use of the device 200. For example, the storage component 208 may include a hard disk (e.g., a magnetic disk, an optical disk, a magneto-optical disk, a solid-state disk, etc.), a compact disk (CD), a digital versatile disk (DVD), a floppy disk, a cassette, a magnetic tape, and / or another type of computer-readable medium, and a corresponding drive.

[0164] Input components 210 may include components that permit device 200 to receive information, for example, via user input (e.g., a touch screen display, a keyboard, a keypad, a mouse, buttons, switches, a microphone, etc.). Additionally or alternatively, input components 210 may include sensors for sensing information (e.g., a global positioning system (GPS) component, an accelerometer, a gyroscope, an actuator, etc.). Output components 212 may include components that provide output information from device 200 (e.g., a display, a speaker, one or more light emitting diodes (LEDs), etc.).

[0165] The communication interface 214 may include a transceiver-type component (e.g., a transceiver, a separate receiver and transmitter, etc.) that enables the device 200 to communicate with other devices, for example, via a wired connection, a wireless connection, or a combination of a wired connection and a wireless connection. The communication interface 214 may permit the device 200 to receive information from another device and / or provide information to another device. For example, the communication interface 214 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, Interface, cellular network interface, etc.

[0166] The device 200 can perform one or more processes described herein. The device 200 can perform these processes based on the processor 204 (e.g., central processing unit (CPU), graphics processing unit (GPU), etc.) executing software instructions stored by a computer-readable medium such as memory 206 and / or storage component 208. Computer-readable media (e.g., non-transitory computer-readable media) are defined herein as non-transitory memory devices. Non-transitory memory devices include memory space located within a single physical storage device or memory space spread across multiple physical storage devices.

[0167] The software instructions may be read into the memory 206 and / or storage component 208 from another computer-readable medium or from another device via the communication interface 214. When executed, the software instructions stored in the memory 206 and / or storage component 208 may cause the processor 204 to perform one or more processes described herein. Additionally or alternatively, hard-wired circuitry may be used in place of or in combination with software instructions to perform one or more processes described herein. Thus, the embodiments or aspects described herein are not limited to any specific combination of hardware circuitry and software.

[0168] The memory 206 and / or storage component 208 may include a data storage device or one or more data structures (e.g., a database, etc.). The device 200 can receive information from the data storage device or one or more data structures in the memory 206 and / or storage component 208, store information in the data storage device or one or more data structures, transmit information to the data storage device or one or more data structures, or search for information stored therein.

[0169] supply Figure 2 The number and arrangement of components shown in FIG. 2 are provided as examples. In some non-limiting embodiments or aspects, the apparatus 200 may include a Figure 2 Components shown in the figure may include additional components, fewer components, different components, or differently arranged components than those shown in the figure. Additionally or alternatively, a set of components (e.g., one or more components) of the device 200 may perform one or more functions described as being performed by another set of components of the device 200.

[0170] Reference now Figures 3A-3C , Figures 3A-3C is a flow chart of a non-limiting embodiment or aspect of a process 300 for synthetic oversampling for enhanced supervised anomaly detection. In some non-limiting embodiments or aspects, one or more steps in the process 300 may be performed (e.g., in whole, in part, etc.) by a transaction service provider system 108 (e.g., one or more devices of the transaction service provider system 108, etc.). In some non-limiting embodiments or aspects, one or more steps in the process 300 may be performed (e.g., in whole, in part, etc.) by another device or group of devices that is independent of or includes the transaction service provider system 108, such as a merchant system 102 (e.g., one or more devices of the merchant system 102), a payment gateway system 104 (e.g., one or more devices of the payment gateway system 104), an acquirer system 106 (e.g., one or more devices of the acquirer system 106), an issuer system 110 (e.g., one or more devices of the issuer system 110), and / or a user device 112 (e.g., one or more devices of a system of the user device 112).

[0171] like Figure 3A As shown, at step 302, process 300 includes obtaining a training data set, wherein the training data set includes a plurality of source samples, wherein the plurality of source samples include a plurality of labeled normal samples and a plurality of labeled abnormal samples. For example, the transaction service provider system may obtain a training data set X 训练 , the training data set includes multiple source samples, and the multiple source samples include multiple labeled normal samples and multiple labeled abnormal samples. For example, after the setting of semi-supervised anomaly detection, the training data set X for supervised anomaly detection 训练={x1,x2,...,x n+m} can be composed of a set of labeled non-anomalous or normal N = {x1, x2, ..., x n} and a set of labeled anomalies A = {x n+1 ,x n+2 ,...,x n+m}, where n>>m. The goal of supervised anomaly detection can be to learn a classifier It evaluates the probability of individual data points being anomalies in a given dataset X by leveraging the prior knowledge of labeled normal N and anomalies A, so that when x i ∈A and x j ∈N,φ(x i )>φ(x j ). In some non-limiting embodiments or aspects, the plurality of source samples may be associated with a plurality of transactions in the transaction processing network 101, the plurality of labeled normal samples may be associated with a plurality of non-fraudulent transactions in the plurality of transactions, and / or the plurality of labeled abnormal samples may be associated with a plurality of fraudulent transactions in the plurality of transactions.

[0172] To better generalize the knowledge from labeled information, the problem of strategic data augmentation can be defined as follows: Given a dataset X 训练 ={N,A}, where In the case of a supervised classifier φ, non-limiting embodiments or aspects of the present disclosure may have a synthetic dataset X according to φ. syn Augmented dataset X 训练 The goal or purpose of the synthetic dataset Generated by mixing samples from N with samples from A. For example, a non-limiting embodiment or aspect of the present disclosure may aim to appropriately sample pairs of data instances of N and A at corresponding mixing ratios α to create synthetic instances x syn ∈X syn , so that it can be 训练 ={X 训练 ∪X syn} to improve or maximize the performance of φ.

[0173] In order to utilize labeled information from two different categories, disclosed in the paper titled “Mixup: Beyond empirical risk minimization” by Hongyi Zhang, Moustapha Cisse, Yann NDauphin, and David Lopez-Paz, 2017 (arXiv preprint arXiv:1710.09412), the disclosure of which is hereby incorporated by reference in its entirety, mixing performs synthetic data generation on two samples from different categories, which has been widely studied to augment image and text data. The idea of ​​mixing is to linearly combine two samples according to the following equation (1):

[0174] x syn =α*x0+(1-α)*x1 (1)

[0175] where α∈[0.0,1.0] controls x syn Although existing work generates x for imbalanced classification problems in the same way syn , but different abnormal behaviors lead to similar labels with high granularity on different synthetic samples, which may indicate that the model is overfitting the noisy synthetic labels. To this end, instead of generating soft labels, non-limiting embodiments or aspects of the present disclosure may synthesize for x according to the following equation (2): syn Hard Marks:

[0176]

[0177] Due to different abnormal behaviors, any mixture from the dataset X 训练 The two random source samples of may result in noisy samples. To solve this problem, the non-limiting embodiments or aspects of the present disclosure may seek to identify a meaningful pair of samples for synthesizing new samples. Since normal samples are usually concentrated in the latent space and boundary samples are more informative, the non-limiting embodiments or aspects of the present disclosure may traverse X under the guidance of the decision boundary of the synthetic oversampling model φ. 训练 For example, given a pair of arbitrary source samples, non-limiting embodiments or aspects of the present disclosure may consider the properties of the two samples to identify the features used to generate a set of x syn ∈X syn The corresponding α and oversampling amount. At the same time, the optimal range for uniformly sampling the next pair of source samples can be identified according to the model state used to iterate to the next round of the mixing process. The intuition behind uniform sampling is to consider the properties between the properties of the source sample and its entire neighborhood information, rather than focusing on a certain sample in the neighborhood. Also refer to Figure 5 , Figure 5Non-limiting embodiments or aspects of an iterative mixing process are presented. Figure 5 In , the background color indicates the model decision boundary. The attributes of the source samples (purple circles) are considered to specify the composition ratio α of oversampling the synthetic samples (black dots) n times with the corresponding markers (outer circle of black dots). The k nearest neighbors are identified to randomly sample the next pair of source samples. The process iterates to the next round with a new pair of source samples. For example, Figure 5 As shown, based on the properties of the source samples x0 and x1, the composition ratio α=4 and the oversampling number n=1 are specified to generate the synthetic sample x syn . Identify the k nearest neighbors to randomly sample the next pair of source samples x2 and x3 for the next round of mixing.

[0178] The iterative mixing process according to non-limiting embodiments or aspects of the present disclosure can have several desired characteristics. The iterative mixing process can make personalized decisions. For example, more samples can be generated for some instances, and fewer samples can be generated for other instances. The iterative mixing process can incorporate various information to guide the mixing process. For example, data attributes and model states can be considered and used as guidance for generating samples. By considering both the model state and the feature distribution, the iterative mixing process can directly generate information that is missing in the original data set but is beneficial to model training.

[0179] Still refer to Figure 5 , and also refer to Figure 6 , which presents a non-limiting embodiment or aspect of a network architecture for synthetic oversampling for enhanced supervised anomaly detection, the iterative mixing process according to the non-limiting embodiment or aspect of the present disclosure can be formulated as a Markov decision process (MDP) with a five-tuple in is a finite set of states, is a finite set of actions, is the state transition function that maps the current state s and action a to the next state s', is an immediate reward function that reflects the quality of action a in state s, and γ is a decimal factor that gradually considers future transitions. To customize the mixing strategy on a sample-by-sample basis, a data mixer can be learned that maps the attributes of the source data samples to an augmentation strategy for creating synthetic samples while exploring the model decision boundary. The MDP can be defined as follows:

[0180] State Space At each timestamp t, the state can be defined as in is the concatenation of the two-dimensional feature vectors of the two source samples. Therefore, the state space can be defined as

[0181] Action Space At each timestamp t, the action where a t =(k,α,n,ε) can be a vector consisting of the size of the neighborhood k, the composition ratio α, the number of oversampling n, and the termination probability of the iterative mixing process ε. Therefore, the action space can be defined as a discrete continuous space

[0182] Transformation function Given state and action a t =(k,α,n,ε), the transformation function can be used to transform x using equations (1) and (2) with α syn Oversampling is performed n times. The resulting synthetic sample X can be used syn To train the classifier, and generate the classifier φ at timestamp t t The transition function can switch to the next state in From x syn The k nearest neighbors are randomly sampled, and Identified as having The closest data points to different markers.

[0183] Reward Function The reward signal r at each timestamp t t can be designed to encourage performance improvement while exploring the decision boundary of the classifier φ. Therefore, the reward function can be defined according to the following equation (3):

[0184]

[0185] where λ is a hyperparameter that defines the strength of the reward signal, is the evaluation metric, and Measuring φ t C(φ t |s t ,a t ) evaluates model confidence to encourage exploration of φ t In this way, the reward signal can drive the data mixer to explore the classifier while achieving the greatest improvement with the newly synthesized data samples.

[0186] To solve the MDP, parameterize the policy π θ can be defined as a data mixer to maximize the reward signal of the MDP, where the ultimate goal is to learn to maximize the cumulative reward The best strategy However, the action space of the iterative hybrid process is a discrete continuous vector and is caused by the underfitting φ t The generated reward signal may be unstable. To this end, non-limiting embodiments or aspects of the present disclosure may adopt deep deterministic policy gradient (DDPG), as disclosed in the paper titled "Continuous control with deep reinforcement learning" by Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra, 2015 (arXiv preprint arXiv:1509.02971), the disclosure of which is hereby incorporated by reference in its entirety, which is an actor-critic framework equipped with two independent networks: an actor and a critic. The critic network (s t ,a t |θ1) approximates the reward signal for the state-action pair from the MDP, while the actor network (s t |θ2) aims to learn a given state s based on the evaluator network t Additionally or alternatively, an advanced actor-critic framework, such as a soft actor-critic, such as disclosed in Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine, 2018, entitled "Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor", in International Conference on Machine Learning. PMLR, 1861-1870, the disclosure of which is hereby incorporated by reference in its entirety, can be employed to learn a policy

[0187] To perform continuous actions, DDPG learns an actor network (s t |θ1), the actor network deterministically transforms a given state s t Mapped to action vector a t , and train the network by maximizing the approximate cumulative reward generated by the critic network (.|θ2). For example, given N transitions, we can generate an expected action (s t|θ1) is used as the input of the evaluator to minimize the loss function defined according to the following equation (4):

[0188]

[0189] The action π(s i ) is a 4-dimensional real vector. In order to fully utilize the expressive power of deep neural networks during training and output discrete continuous vectors for MDP, the continuous action vector (s) can be transformed using the sigmoid function. i ), and generate the action vector where w specifies the value constraints for individual entries. For example, if the maximum values ​​of k and n are 10 and 5, then w = [10, 1, 5, 1], since α and ε are expected to be in the range of 0 to 1. The results of k and n may be rounded to the nearest integer.

[0190] To address the problem of unstable reward signals, DDPG approximates the reward signal with a critic network (.|θ2) and trains the network off-policy. This introduces a replay buffer to store history and randomly sample transitions to minimize the temporal correlation between two transitions for learning across a set of uncorrelated transitions. For example, the critic network (.|θ2) can map state-action pairs to real-valued y by minimizing the loss function defined according to the following equation (5): t middle:

[0191] L Q (θ2)=[Q(S r ,a t )-bt] 2 (5)

[0193] in is a signal derived from the Bayman equation, which considers the recursive relationship between the current true reward signal and the future approximate reward signal to maximize the cumulative reward, where π(s t+1 |θ1) is the action specified by the actor network, and γ is the decimal factor.

[0194] like Figure 3A As shown, at step 304, process 300 includes initializing a timestamp. For example, transaction service provider system 108 may initialize a timestamp t. For example, and also referring to Figure 7, which illustrates a non-limiting example or aspect of a training procedure for a process of synthetic oversampling for enhanced supervised anomaly detection, the transaction service provider system 108 may initialize a timestamp t when executing a training segment e of the number E of training segments to be executed. In such an example, the transaction service provider system 108 may initialize the timestamp t in response to executing each training segment e of the number E of training segments to be executed (e.g., in response to executing a new or next training segment, etc.).

[0195] like Figure 3A As shown, at step 306, process 300 includes receiving a time-stamped action vector from an actor network of an actor-evaluator framework including an actor network and an evaluator network. For example, and again referring to Figure 6 and 7 , the transaction service provider system 108 may receive an action vector a with a timestamp t from an actor network π of an actor-evaluator framework including an actor network π and an evaluator network Q t The actor network π can be configured based on the state s t Generate action vector a t The state s may be determined based on a current pair of source samples in the plurality of source samples. t The action vector α t It may include the nearest neighbor size k, the composition ratio α, the oversampling number n, and the termination probability ε.

[0196] like Figure 3A As shown, at step 308, the process 300 includes combining the current pair of source samples according to the composition ratio and the oversampling number to generate a labeled synthetic sample associated with the label. For example, the transaction service provider system 108 can combine the current pair of source samples according to the composition ratio α and the oversampling number n to generate a labeled synthetic sample associated with the label y syn The associated labeled synthetic sample x syn .

[0197] In some non-limiting embodiments or aspects, the current pair of source samples may be combined according to a composition ratio α to generate a labeled synthetic sample x according to equations (1) and (2): syn , where x0 is the first sample in the current pair of samples, x1 is the second sample in the current pair of samples, and y syn is a labeled synthetic sample x syn of hard marks, y0 is the first hard mark value, and y1 is the second hard mark value.

[0198] like Figure 3A As shown, at step 310, the process 300 includes training a machine learning classifier using the labeled synthetic samples and the labels. For example, the transaction service provider system 108 may use the labeled synthetic samples xsyn and mark y syn To train the machine learning classifier φ. For example, the transaction service provider system 108 may train the machine learning classifier φ for the labeled synthetic samples x according to the machine learning classifier φ. syn The output and label y syn The objective function is to transform the labeled synthetic sample x syn Provided as input to the machine learning classifier φ, and modifying one or more parameters of the machine learning classifier φ. For example, and again referring to Figure 6 ,In each step, a pair of normal samples and anomalies can be input into the data mixer, the corresponding actions can be generated to create synthetic samples, and the synthetic samples are employed to train the classifier and generate a reward signal for updating the data mixer.

[0199] like Figure 3A As shown, at step 312, the process 300 includes obtaining source samples in the k nearest neighbors of the labeled synthetic sample based on the nearest neighbor size. For example, the transaction service provider system 108 may obtain the labeled synthetic sample x based on the nearest neighbor size k. syn For example, the transaction service provider system 108 may use a k-nearest neighbor algorithm to calculate the labeled synthetic sample x syn The source samples in the k nearest neighbors of .

[0200] In some non-limiting embodiments or aspects, the transaction service provider system 102 may use the training data set X before executing the training segment (eg, before executing any training segment, etc.). 训练 To train the machine learning classifier φ; and for the training data set X 训练 Each of the k nearest neighbors is pre-calculated for each of the plurality of source samples in the training data set X. The transaction service provider system 102 may store each of the pre-calculated k nearest neighbors for each of the plurality of source samples in the training data set X. 训练 In this way, non-limiting embodiments or aspects of the present disclosure can reduce the computational cost during the training procedure.

[0201] like Figure 3A As shown, at step 314, the process 300 includes generating a plurality of classifier outputs using a machine learning classifier for a subset of the plurality of source samples in the training dataset and the k-nearest neighbors of the labeled synthetic sample. For example, the transaction service provider system 108 may generate a plurality of classifier outputs for the labeled synthetic sample x syn The source samples and validation dataset X in the k nearest neighbors val The training dataset X in 训练The transaction service provider system 108 may use the machine learning classifier φ to generate a plurality of classifier outputs. syn The source samples in the k nearest neighbors of the training dataset X and the validation dataset Xval 训练 A subset of the plurality of source samples is provided as input to a machine learning classifier φ, and a plurality of classifier outputs is received as output from the machine learning classifier φ.

[0202] like Figure 3B As shown, at step 316, the process 300 includes selecting a next pair of source samples from the source samples in the k nearest neighborhood of the labeled synthetic sample. For example, the transaction service provider system 108 may select a next pair of source samples from the labeled synthetic sample x syn Select (eg, randomly select, etc.) the next pair of source samples from the source samples in the k nearest neighbors of .

[0203] like Figure 3B As shown, at step 318, process 300 includes storing the state, action vector, next state, and reward in a memory buffer. For example, the transaction service provider system 108 may store the state s in a memory buffer. t , action vector a t , next state s t+1 and reward r t The next state s can be determined based on the next pair of source samples t+1 A reward r may be determined based on the plurality of classifier outputs. t .

[0204] In some non-limiting embodiments or aspects, the reward r may be determined according to the following equations (6) and (7): t :

[0205]

[0206] Where M is the evaluation metric, ΔM(φ t ) measures the trained classifier φ t Performance improvements, X val is the validation dataset, y val is the labeled set of the training dataset, where is the baseline at timestamp t, m is a hyperparameter defining the size of the buffer used to form the baseline, and C(φt|st,at) evaluates the trained classifier φ t is the model confidence, P is the model exploration function, and k is the action vector a t Specifies the nearest neighbor size, x i is the labeled synthetic sample x at timestamp t synk nearest neighbors, and y i is x i Mark.

[0207] For example, the reward signal may include a stimulus component to improve and the model exploration component P(φ t |s t ,a t ). In order to learn the best policy for the target task, existing solutions directly use the performance on the validation dataset as the reward signal. However, since the convergence of the base classifier is not guaranteed, directly learning a policy with performance on the validation set may result in a noisy reward signal. Therefore, non-limiting embodiments or aspects of the present disclosure are not to use the performance phi of the current model. t , but rather provide improvement incentives to maximize model improvements on the validation set using the baseline performance according to (6). In such instances, synthetic samples can be created by mixing normal samples and anomalies while iteratively training the classifier φ, while exploring the model decision boundary to create beneficial samples and reduce or prevent the generation of noisy samples. Therefore, a model exploration signal for quantifying the instance-by-instance prediction uncertainty can be defined according to equation (7) to encourage the data mixer to explore uncertain regions in the feature space.

[0208] like Figure 3B As shown, at step 320, the process 300 includes determining whether the termination probability satisfies a termination threshold. For example, the transaction service provider system 108 may determine whether the termination probability ε satisfies a termination threshold (e.g., a predetermined termination threshold, a hyperparameter set by a user, etc.). For example, and again referring to Figure 7 , the transaction service provider system 108 may iteratively train the classifier φ until the termination probability ε satisfies the termination threshold.

[0209] like Figure 3B As shown, at step 322, process 300 includes incrementing the timestamp in response to determining in step 320 that the termination probability fails to satisfy the termination threshold. For example, the transaction service provider system 108 may increment the timestamp t in response to determining in step 320 that the termination probability ε fails to satisfy the termination threshold.

[0210] like Figure 3B As shown, at step 324, the process 300 includes training the evaluator network according to an evaluator loss function that depends on the state, the action vector, and the reward. For example, the transaction service provider system 108 may train the evaluator network according to an evaluator loss function that depends on the state s t , action vector a t and reward r t The critic loss function is used to train the critic network Q. For example, the critic loss function can be defined according to equation (5), where b t=R(s t ,a t )+γQ(s t+1 ,π(s t+1 |θ1)|θ2),π(s t+1 |θ1) is the action specified by the actor network, and γ is the decimal factor.

[0211] like Figure 3B As shown, at step 326, the process 300 includes training the actor network according to an actor loss function that depends on the output of the evaluator network. For example, the transaction service provider system 108 can train the actor network π according to the actor loss function that depends on the output of the evaluator network. For example, the actor function can be defined according to equation (4), where N is the number of transitions, π(s i |θ2) is the state s i The expected action, and Q(s i ,π(s i )|θ2) is the evaluator network's response to the expected action π(s i |θ2) and state s i Output.

[0212] like Figure 3B As shown, at step 328, the process 300 includes determining whether the actor network and the evaluator network have been trained for multiple training steps. For example, the transaction service provider system 108 can determine whether the actor network π and the evaluator network Q have been trained for multiple training steps S.

[0213] In response to determining in step 328 that the actor network and the evaluator network have not been trained for the plurality of training steps, the process may return to step 324 to train the actor network and the evaluator network in the next training step. For example, the transaction service provider system 108 may return the process to step 324 in response to determining in step 328 that the actor network π and the evaluator network Q have not been trained for the plurality of training steps S to train the actor network π and the evaluator network Q in the next training step.

[0214] In response to determining in step 328 that the actor network and the evaluator network have been trained for the plurality of training steps, the process may return to step 306 with the next pair of source samples as the current pair of source samples. For example, the transaction service provider system 108 may return to step 306 with the next pair of source samples as the current pair of source samples in response to determining in step 328 that the actor network π and the evaluator network Q have been trained for the plurality of training steps S.

[0215] like Figure 3BAs shown, at step 330, the process 300 includes determining whether the number of executed training segments meets the threshold number of training segments in response to determining that the termination probability satisfies the termination threshold in step 320. For example, the transaction service provider system 108 may determine whether the number of executed training segments meets the threshold number of training segments in response to determining that the termination probability ε satisfies the termination threshold in step 320.

[0216] In response to determining that the number of training segments executed fails to meet the threshold number of training segments in step 330, the process may return to step 304 to execute the next training segment. For example, the transaction service provider system 108 may return the process to step 304 to execute the next training segment in response to determining that the number of training segments executed fails to meet the threshold number of training segments.

[0217] like Figure 3B As shown, at step 332, the process 300 includes providing a machine learning classifier in response to determining that the number of training segments executed meets the threshold number of training segments. For example, the transaction service provider system 108 may provide the machine learning classifier φ (e.g., provide a trained machine learning classifier, etc.) in response to determining that the number of training segments executed meets the threshold number of training segments.

[0218] like Figure 3C As shown, at step 334, process 300 includes receiving transaction data associated with a transaction currently being processed in the transaction processing network. For example, transaction service provider system 108 may receive transaction data associated with a transaction currently being processed in transaction processing network 101.

[0219] In some non-limiting embodiments or aspects, the transaction data may include parameters associated with the transaction, such as an account identifier (e.g., PAN, etc.), transaction amount, transaction date and time, type of product and / or service associated with the transaction, currency conversion rate, currency type, merchant type, merchant name, merchant location, transaction approval (and / or rejection) rate, etc.

[0220] like Figure 3C As shown, at step 336, process 300 includes processing transaction data using a trained machine learning classifier to classify a transaction as a fraudulent transaction or a non-fraudulent transaction. For example, the transaction service provider system 108 may process transaction data using a trained machine learning classifier to classify a transaction as a fraudulent transaction or a non-fraudulent transaction. For example, the transaction service provider system 108 may provide one or more parameters of the transaction data as input to the trained machine learning classifier, and receive a prediction (e.g., probability, likelihood, yes / no, etc.) from the trained machine learning classifier as an output that the transaction is a fraudulent transaction or a non-fraudulent transaction.

[0221] like Figure 3C As shown, at step 338, process 300 includes authorizing or denying a transaction in the transaction processing network based on the classification. For example, the transaction service provider system 108 may authorize or deny a transaction in the transaction processing network based on the classification. As an example, the transaction service provider system 108 may deny authorization of a transaction in the transaction processing network 101 in response to classifying the transaction as a fraudulent transaction. For example, the transaction service provider system 108 may authorize a transaction in the transaction processing network 101 in response to classifying the transaction as a non-fraudulent transaction.

[0222] experiment

[0223] The following discusses a general data mixer with supervised anomaly detection according to non-limiting embodiments or aspects of the present disclosure (e.g., Figure 9-11 Experiments to evaluate the proposed framework (referred to as “AnoMix” in

[17] ) against state-of-the-art tag-informed anomaly detection algorithms and a number of data augmentation methods. The following experiments aim to answer the following questions: RQ: How does the proposed framework AnoMix compare to existing data augmentation methods?; RQ2: How does the proposed framework AnoMix compare to existing tag-informed anomaly detection methods?; and RQ3: How does each component contribute to the performance of the proposed framework AnoMix?

[0224] A horizontal analysis was conducted to compare the framework (e.g., "AnoMix") according to non-limiting embodiments or aspects of the present disclosure with data augmentation methods for three different classifiers. A vertical analysis was also conducted to compare the proposed framework with a label-informed anomaly detector. Figure 8 is a table summarizing the statistics of the dataset used for the experiment. Figure 8 As shown, the following five benchmark datasets from different domains are used: Japanese vowel dataset, Annthyroid dataset, mammography dataset, satellite dataset, and SMTP dataset.

[0225] The Japanese vowel dataset contains utterances of / ae / recorded from nine speakers with 12 LPC cepstral coefficients. The goal is to identify outlier speakers.

[0226] The Annthyroid dataset is a set of clinical records that records 21 physical attributes of over 7200 patients. The goal is to identify patients who may have hypothyroidism.

[0227] The mammography dataset consists of 6 features extracted from the images including shape, edge, density, etc. The goal is to identify malignant cases that may lead to breast cancer.

[0228] The satellite dataset contains remote sensing data for 6,435 areas, where each area is split into 3x 3 square neighborhood areas and monitored by 4 different light wavelengths captured from satellite images of the earth. The goal is to identify areas with abnormal soil conditions.

[0229] SMTP has 95,156 server connections with 41 connection attributes including duration, src_byte, dst_byte, etc. The task is to identify malicious attacks from the connection logs.

[0230] Each dataset is publicly available in OpenML. The widely adopted protocols proposed by the ODDS library are used to process the data.

[0231] Fig. 9 is a table summarizing a horizontal comparison between a framework (e.g., "AnoMix") according to a non-limiting embodiment or aspect of the present disclosure and the following representative data augmentation methods performed on 3 different classifiers (i.e., KNN, XGBoost, MLP). Random is a basic baseline that generates synthetic anomalies by randomly averaging two existing anomalies. SMOTE linearly combines existing anomalies with their K nearest anomalies through a combination ratio of random sampling. BorderlineSMOTE identifies the boundary between anomalies and normal samples, where each anomaly has K nearest neighbors. SMOTE is then performed on anomalies in the boundary area. SVMSMOTE introduces a support vector classifier to identify the boundary between anomalies and normal samples, and performs SMOTE on anomalies close to the boundary. ADASYN uses an estimated local distribution to calculate multiple synthetic samples generated by individual anomalies, and then performs SMOTE on them. MixBoost uses a combination ratio of random sampling to mix anomalies and normals. Weighted sampling is performed on the number of oversampled individual anomalies based on the entropy of the base classifier.

[0232] Fig.10 is a table outlining a longitudinal comparison between a framework (e.g., "AnoMix") and a label-informed detection algorithm according to a non-limiting embodiment or aspect of the present disclosure to study the effectiveness of synthetic oversampling. XGBOD utilizes anomaly scores of individual data points generated by multiple unsupervised algorithms as input features to the underlying XGBoost classifier. DeepSAD develops a semi-supervised loss function that classifies known normal samples and unlabeled samples into uniform clusters and deviates known anomalies from the clusters. DevNet uses label information with prior probabilities to perform anomaly scoring for significant deviations from the majority of normal samples.

[0233] The macro average precision, recall and F1-score, which calculates the score of each category separately and averages the scores, is adopted as the evaluation scheme. The intuition behind this is to balance the importance of anomaly detection and normal sample classification, because minimum false positives are also a key evaluation criterion. 5-fold cross validation is performed using 80% of the data for training and 20% for testing. In addition, 40% of the training data is further split into a validation set for the framework to generate reward signals or for executing baseline methods for model tuning. The average performance on the test set is reported.

[0234] For the horizontal analysis, a KNN classifier with k=15, an XGBoost classifier with a linear kernel, and an Adam optimizer with a relu activation function for a 128-64 multilayer perceptron classifier were used. For the longitudinal analysis, XGBOD from PyOD and publicly available implementations of DevNet and DeepSAD were used. Since the output of Dev and DeepSAD is an anomaly score, the thresholds of both methods are searched from {0.5x, 1.0x, 1.5x, 2.0x} of the anomaly ratio to perform classification and report the best result. For the framework according to non-limiting embodiments or aspects of the present disclosure, a maximum neighborhood size K=15, a reward coefficient λ=10.0, a window size m=25 for the baseline, and a reward signal The macro-average F1-score of .

[0235] Reference again Fig. 9 To answer RQ1, a framework according to a non-limiting embodiment or aspect of the present disclosure (e.g., “AnoMix”) is compared with a cutting edge synthetic oversampling method for anomaly detection. Fig. 9 As shown, the macro-average precision, recall, and F1-score of each data augmentation method across 3 different classifiers are tabulated. Performance without data augmentation on the original classifier is also reported to show insights into how different classifiers affect performance. In general, the framework according to non-limiting embodiments or aspects of the present disclosure is able to significantly outperform all baseline augmentation methods and achieve at least 8.5% and at most 29.4% improvement in F1-score. Based on Fig. 9 , the following observations can be made.

[0236] First, by comparing the baseline augmentation method with the classifier without data augmentation, it can be observed that the performance of the baseline augmentation method is generally lower than that of the classifier trained without data augmentation. Specifically, the average F1-score of the baseline augmentation method is consistently lower than that of the vanilla classifier on the five datasets. The only exception is Mixboost with a KNN classifier, which is due to its outstanding performance on the SMTP dataset. Further investigation of this phenomenon shows that when anomalies are very rare, randomly mixing normal samples with anomalies is able to produce beneficial synthetic normal samples that form the decision boundary. This supports the claim that existing data augmentation methods cannot handle the different behaviors of anomalies and may result in noisy synthetic samples, but generalizing the labeled anomaly information by mixing normal samples with anomalies can alleviate the problem.

[0237] Secondly, by comparing the framework according to the non-limiting embodiments or aspects of the present disclosure (e.g., "AnoMix") with a vanilla classifier without data augmentation, it is observed that the framework according to the non-limiting embodiments or aspects of the present disclosure consistently outperforms all vanilla classifiers. On five data sets, the framework according to the non-limiting embodiments or aspects of the present disclosure improves the F1-scores of the KNN, XGBoost, and MLP classifiers by 17.1%, 16.7%, and 12.8%, respectively. This phenomenon indicates that the framework according to the non-limiting embodiments or aspects of the present disclosure is capable of adaptively creating synthetic samples for different classifiers to achieve performance improvements. In addition, it is also observed that the KNN classifier has the greatest improvement, which indicates that the nearest neighbor exploration of the transition function is beneficial to classifiers with similar properties. Another interesting observation is that the more complex the model, the less improvement. A possible explanation is that complex models tend to be too trusting of predictions, which can lead to noisy prediction uncertainty rewards and thus mislead the learning procedure of the augmentation strategy.

[0238] Third, by comparing the framework (e.g., "AnoMix") according to the non-limiting embodiments or aspects of the present disclosure with all other data augmentation methods, it is observed that the framework (e.g., "AnoMix") according to the non-limiting embodiments or aspects of the present disclosure is superior to the scores of all baselines with three classifiers in macro-F1 scores. Specifically, the framework (e.g., "AnoMix") according to the non-limiting embodiments or aspects of the present disclosure is 8.5%, 18.2%, and 18.9% better than the F1-scores of the second best augmentation methods with KNN, XGBoost, and MLP classifiers on five data sets, respectively. Because Mixboost is similar to AnoMix with a random mixing strategy, this means that the proposed framework can learn customized mixing strategies for different classifiers and data samples. It can also be observed that although the framework (e.g., "AnoMix") according to the non-limiting embodiments or aspects of the present disclosure may not always be superior to all other baselines with respect to precision and recall, the F1-score is always the best. This phenomenon shows that the framework according to the non-limiting embodiments or aspects of the present disclosure is able to balance the trade-off between precision and recall, and therefore produces excellent F1-scores in all settings. The reason behind this is that the framework according to the non-limiting embodiments or aspects of the present disclosure can adopt Macro-F1 to form a reward signal. Users can also customize their own metrics (e.g., precision, recall, custom metrics, etc.) to obtain an anomaly detector that meets their requirements.

[0239] Fourth, by making a detailed comparison between the framework according to a non-limiting embodiment or aspect of the present disclosure (e.g., “AnoMix”) and SVM SMOTE and Borderline SMOTE, it can be observed that, with three different classifiers, the F1-score of the framework according to a non-limiting embodiment or aspect of the present disclosure is at least 17.1%, 18.2%, and 18.9% better than the scores of the two baselines. This phenomenon suggests that the instance-by-instance prediction uncertainty in the reward function is a better way to generate customized beneficial synthetic samples for different classifiers. The reason behind this is that the two baselines identify category boundaries in the label space and hyperspace of the SVM, where the framework according to a non-limiting embodiment or aspect of the present disclosure (e.g., “AnoMix”) identifies the boundaries directly defined by the base classifier. By encouraging the strategy to generate samples on the boundaries defined by the classifier, it is more likely to create beneficial information that cannot be observed from the original feature space or hyperspace of another classifier.

[0240] Reference again Fig.10 ,To answer RQ2, a framework according to a non-limiting embodiment or aspect of the present disclosure (e.g., “AnoMix”) is compared with a state-of-the-art label-informed anomaly detection algorithm and a baseline augmentation method. Fig.10The macro-average precision, recall, and F1-score of each method are presented on the five datasets. The “best baseline” refers to the horizontal baseline with the best F1-score on the individual dataset. The following two observations can be made based on Fig.10 get.

[0241] First, data augmentation methods generally outperform label-informed anomaly detectors. Comparing the best data augmentation baseline with the three label-informed methods, the best data augmentation baseline outperforms the best label-informed algorithm by 6.2%. This suggests that data augmentation methods may be a more effective way to generalize label information when combined with appropriate strategies. In addition, frameworks according to non-limiting embodiments or aspects of the present disclosure (e.g., "AnoMix") with appropriately learned strategies achieve excellent performance, which further verifies the above suggestions.

[0242] Second, the method of labeling notifications achieves better precision and lower recall. Fig. 9 From the cross-comparison, we can observe that the average precision of label-informed methods is usually better than that of data augmentation methods on all three classifiers, while the average recall is usually lower than the recall of those baselines. A possible explanation is that label-informed methods can only exploit the labels themselves, while data augmentation methods are able to explore potentially beneficial information from limited label information. This suggests that label-informed methods tend to overfit to the existing labels and are therefore suboptimal.

[0243] Reference now Fig.11 , which is a table of macro-average scores for a framework (e.g., “AnoMix”) according to a non-limiting embodiment or aspect of the present disclosure, with KNN classifiers and ablations on a Japanese vowel dataset, an ablation study was performed on the Japanese vowel dataset with a KNN classifier to answer question RQ3. Since the reward signal may be the most critical guide for the optimal augmentation strategy, the reward function is ablated to study the contribution of individual components. It should be noted that the random reward generates a reward signal with a random floating point number between 0 and 1, which may produce a random augmentation strategy. The following observations can be made based on Fig.11 get.

[0244] First, the proposed MDP is solvable and the customized RL agent is able to learn the optimal policy. By comparing the random reward baseline with the framework (e.g., "AnoMix") according to a non-limiting embodiment or aspect of the present disclosure, it can be observed that all three scores have significant improvements. Since both ablations on equations (6) and (7) significantly outperform the random reward baseline, this shows that the customized RL agent is able to solve the MDP to the optimal augmentation policy.

[0245] Second, the two components in the proposed reward signal play an important role in the optimal augmentation strategy. Specifically, both components can increase the utilization of label information and thus significantly improve the precision. On the one hand, since the classifier may experience underfitting during the training procedure, learning an augmentation strategy without equation (6) may lead to significant performance degradation. On the other hand, without considering the model state via equation (7), it is unlikely to identify potentially beneficial information of the underlying classifier and thus produce a lower recall.

[0246] Therefore, non-limiting embodiments or aspects of the present disclosure may provide a general data mixer that is capable of incorporating different classifiers to utilize and explore potentially beneficial information from labeled information for supervised anomaly detection by: using an iterative mixing process to simultaneously consider feature distributions and model states, formulating the iterative mixing as a Markov decision process (MDP), and / or providing a reward function to guide the policy learning procedure when the classifier is underfit. In order to solve the MDP, non-limiting embodiments or aspects of the present disclosure provide a deep actor-evaluator framework that optimizes on a discrete continuous action space. In this way, non-limiting embodiments or aspects of the present disclosure may generalize the labeled information by simultaneously traversing the feature space while considering the model state, formulating the iterative combination as a Markov decision process, and designing a combined reward signal to guide the combination process and / or customizing a deep reinforcement learning algorithm to solve a discrete continuous action space for learning the best combined strategy.

[0247] Although various embodiments or aspects have been described in detail for purposes of illustration and description, it will be understood that such detail is for that purpose only and that the embodiments or aspects are not limited to the disclosed embodiments or aspects, but, on the contrary, are intended to cover modifications and equivalent arrangements within the spirit and scope of the appended claims. For example, it will be understood that the present disclosure contemplates that, to the extent possible, one or more features of any embodiment or aspect may be combined with one or more features of any other embodiment or aspect. In fact, any of these features may be combined in a manner not specifically recited in the claims and / or disclosed in the specification. Although each dependent claim listed below may be directly dependent on only one claim, the disclosure of possible implementations includes each dependent claim in combination with each other claim in the claim set.

Claims

1. A method comprising: Using at least one processor to obtain a training data set X 训练 , the training data set includes a plurality of source samples, and the plurality of source samples include a plurality of labeled normal samples and a plurality of labeled abnormal samples; as well as Executing, with the at least one processor, a training segment by: (i) Initialize timestamp t; (ii) receiving the action vector a at the timestamp t from the actor-critic framework comprising an actor network π and a critic network Q t , where the actor network π is configured based on the state s t To generate the action vector a t , where the state s t is determined based on a current pair of source samples in the plurality of source samples, and wherein the action vector a t Including the nearest neighbor size k, composition ratio α, oversampling number n and termination probability ∈; (iii) combining the current pair of source samples according to the composition ratio α and the oversampling number n to generate a syn The associated labeled synthetic sample x syn ; (iv) Using the labeled synthetic sample x syn and the marker y syn To train the machine learning classifier φ; (v) Obtaining the labeled synthetic sample x based on the nearest neighbor size k syn Source samples in the k nearest neighbors of ; (vi) For the labeled synthetic sample x syn The k nearest neighbors of the source samples and validation dataset X val The training dataset X 训练 a subset of the plurality of source samples, using the machine learning classifier φ to generate a plurality of classifier outputs; (vii) From the labeled synthetic sample x syn Select the next pair of source samples from the source samples in the k nearest neighbors; (viii) storing the state s in a memory buffer t , the action vector a t , next state s t+1 and reward r t , where the next state s t+1 is determined based on the next pair of source samples, and wherein the reward r t is determined based on the plurality of classifier outputs; (ix) determining whether the termination probability ∈ satisfies a termination threshold; (x) in response to determining that the termination probability ∈ fails to satisfy the termination threshold, incrementing the timestamp t for a plurality of training steps S: Depending on the state s t , the action vector a t and the reward r t The evaluator loss function is used to train the evaluator network Q; as well as The actor network π is trained according to an actor loss function that depends on the output of the critic network, and After training the actor network π and the critic network Q for the plurality of training steps S, returning to step (ii), wherein the next pair of source samples is used as the current pair of source samples; (xi) in response to determining that the termination probability ∈ satisfies the termination threshold, determining whether the number of training segments executed satisfies a threshold number of training segments; (xii) in response to determining that the number of training segments executed fails to satisfy the threshold number of training segments, returning to step (i) to execute a next training segment; as well as (xiii) in response to determining that the number of training segments executed meets the threshold number of training segments, providing the machine learning classifier φ.

2. The method according to claim 1, wherein the current pair of source samples is combined according to the composition ratio α to generate the labeled synthetic sample x according to the following equation syn : x syn =α*x0+(1-α)*x1 Where x0 is the first sample in the current pair of samples, x1 is the second sample in the current pair of samples, and y syn is the labeled synthetic sample x syn of hard marks, y0 is the first hard mark value, and y1 is the second hard mark value.

3. The method according to claim 1, wherein the reward r t is determined according to the following equation: Where M is the evaluation metric, ΔM(φ t ) measures the trained classifier φ t Performance improvements, X val is the validation dataset, y val is the labeled set of the training dataset, is the baseline at timestamp t, m is a hyperparameter defining the size of the buffer used to form the baseline, and C(φt|st,at) evaluates the trained classifier φ t is the model confidence, P is the model exploration function, and k is the action vector a t Specifies the nearest neighbor size, x i is the labeled synthetic sample x at timestamp t syn k nearest neighbors, and y i is x i Mark.

4. The method of claim 1, wherein the actor loss function is defined according to the following equation: Where N is the number of transitions, π(s i |θ2) is the state s i The expected action, and Q(s i ,π(s i )|θ2) is the evaluator network's response to the expected action π(S i |θ2) and state s i The output of The evaluator loss function is defined according to the following equation: where b t =R(s t ,a t )+γQ(s t+1 ,π(s t+1 |θ1)|θ2),π(s t+1 |θ1) is the action specified by the actor network, and γ is a decimal factor.

5. The method of claim 1, wherein the plurality of source samples are associated with a plurality of transactions in a transaction processing network, wherein the plurality of labeled normal samples are associated with a plurality of non-fraudulent transactions among the plurality of transactions, and wherein the plurality of labeled abnormal samples are associated with a plurality of fraudulent transactions among the plurality of transactions.

6. The method according to claim 5, further comprising: receiving, with the at least one processor, transaction data associated with a transaction currently being processed in the transaction processing network; processing, with the at least one processor, the transaction data using the trained machine learning classifier to classify the transaction as a fraudulent transaction or a non-fraudulent transaction; as well as Responsive to classifying the transaction as a fraudulent transaction, refusing, using the at least one processor, to authorize the transaction in the transaction processing network.

7. The method according to claim 1, further comprising: Before executing the training segment: Using the at least one processor to train the training data set X 训练 To train the machine learning classifier φ; as well as For the training data set X 训练 For each source sample in the plurality of source samples, using the at least one processor to pre-compute each k-nearest neighbor.

8. A system comprising: at least one processor programmed and / or configured to: Get the training data set X 训练 , the training data set includes a plurality of source samples, and the plurality of source samples include a plurality of labeled normal samples and a plurality of labeled abnormal samples; and Perform a training session by: (i) Initialize timestamp t; (ii) receiving the action vector a at the timestamp t from the actor-critic framework comprising an actor network π and a critic network Q t , where the actor network π is configured based on the state s t To generate the action vector a t , where the state s t is determined based on a current pair of source samples in the plurality of source samples, and wherein the action vector a t Including the nearest neighbor size k, composition ratio α, oversampling number n and termination probability ∈; (iii) combining the current pair of source samples according to the composition ratio α and the oversampling number n to generate a syn The associated labeled synthetic sample x syn ; (iv) Using the labeled synthetic sample x syn and the marker y syn To train the machine learning classifier φ; (v) Obtaining the labeled synthetic sample x based on the nearest neighbor size k syn Source samples in the k nearest neighbors of ; (vi) For the labeled synthetic sample x syn The k nearest neighbors of the source samples and validation dataset X val The training dataset X 训练 a subset of the plurality of source samples, using the machine learning classifier φ to generate a plurality of classifier outputs; (vii) From the labeled synthetic sample x syn Select the next pair of source samples from the source samples in the k nearest neighbors; (viii) storing the state s in a memory buffer t , the action vector a t , next state s t+1 and reward r t , where the next state s t+1 is determined based on the next pair of source samples, and wherein the reward r t is determined based on the plurality of classifier outputs; (ix) determining whether the termination probability ∈ satisfies a termination threshold; (x) in response to determining that the termination probability ∈ fails to satisfy the termination threshold, incrementing the timestamp t for a plurality of training steps S: Depending on the state s t , the action vector a t and the reward r t The evaluator loss function is used to train the evaluator network Q; as well as The actor network π is trained according to an actor loss function that depends on the output of the critic network, and After training the actor network π and the critic network Q for the plurality of training steps S, returning to step (ii), wherein the next pair of source samples is used as the current pair of source samples; (xi) in response to determining that the termination probability ∈ satisfies the termination threshold, determining whether the number of training segments executed satisfies a threshold number of training segments; (xii) in response to determining that the number of training segments executed fails to satisfy the threshold number of training segments, Return to step (i) to perform the next training segment; as well as (xiii) in response to determining that the number of training segments executed meets the threshold number of training segments, providing the machine learning classifier φ.

9. The system of claim 8, wherein the current pair of source samples are combined according to the composition ratio α to generate the labeled synthetic sample x according to the following equation syn : x syn =α*x0++(1-α)*x1 Where x0 is the first sample in the current pair of samples, x1 is the second sample in the current pair of samples, and y syn is the labeled synthetic sample x syn of hard marks, y0 is the first hard mark value, and y1 is the second hard mark value.

10. The system according to claim 8, wherein the reward r t is determined according to the following equation: Where M is the evaluation metric, ΔM(φ t ) measures the trained classifier φ t Performance improvements, X val is the validation dataset, y val is the labeled set of the training dataset, where is the baseline at timestamp t, m is a hyperparameter defining the size of the buffer used to form the baseline, and C(φt|st,at) evaluates the trained classifier φ t is the model confidence, P is the model exploration function, and k is the action vector a t Specifies the nearest neighbor size, x i is the labeled synthetic sample x at timestamp t syn k nearest neighbors, and y i is x i Mark.

11. The system of claim 8, wherein the actor loss function is defined according to the following equation: Where N is the number of transitions, π(s i |θ2) is the state s i The expected action, and Q(s i ,π(s i )|θ2) is the evaluator network's response to the expected action π(s i |θ2) and state s i The output of The evaluator loss function is defined according to the following equation: where b t =R(s t ,a t )+γQ(s t+1 ,π(s t+1 |θ1)|θ2),π(s t+1 |θ1) is the action specified by the actor network, and γ is a decimal factor.

12. The system of claim 8, wherein the plurality of source samples are associated with a plurality of transactions in a transaction processing network, wherein the plurality of labeled normal samples are associated with a plurality of non-fraudulent transactions among the plurality of transactions, and wherein the plurality of labeled abnormal samples are associated with a plurality of fraudulent transactions among the plurality of transactions.

13. The system of claim 12, wherein the at least one processor is further programmed and / or configured to: receiving transaction data associated with a transaction currently being processed in the transaction processing network; processing the transaction data using the trained machine learning classifier to classify the transactions as fraudulent transactions or non-fraudulent transactions; and Responsive to classifying the transaction as a fraudulent transaction, denying authorization for the transaction in the transaction processing network.

14. The system of claim 8, wherein the at least one processor is further programmed and / or configured to: Before executing the training segment: Using the training dataset X 训练 to train the machine learning classifier φ; and For the training data set X 训练 For each source sample in the plurality of source samples, each k nearest neighbor is pre-computed.

15. A computer program product comprising a non-transitory computer readable medium comprising program instructions that, when executed by at least one processor, cause the at least one processor to: Get the training data set X 训练 , the training data set includes a plurality of source samples, and the plurality of source samples include a plurality of labeled normal samples and a plurality of labeled abnormal samples; and A training session is performed by: (i) Initialize timestamp t; (ii) receiving the action vector a at the timestamp t from the actor-critic framework comprising an actor network π and a critic network Q t , where the actor network π is configured based on the state s t To generate the action vector a t , where the state s t is determined based on a current pair of source samples in the plurality of source samples, and wherein the action vector a t Including the nearest neighbor size k, composition ratio α, oversampling number n and termination probability ∈; (iii) combining the current pair of source samples according to the composition ratio α and the oversampling number n to generate a syn The associated labeled synthetic sample x syn ; (iv) Using the labeled synthetic sample x syn and the marker y syn To train the machine learning classifier φ; (v) Obtaining the labeled synthetic sample x based on the nearest neighbor size k syn Source samples in the k nearest neighbors of ; (vi) For the labeled synthetic sample x syn The k nearest neighbors of the source samples and validation dataset X val The training dataset X 训练 a subset of the plurality of source samples, using the machine learning classifier φ to generate a plurality of classifier outputs; (vii) From the labeled synthetic sample x syn Select the next pair of source samples from the source samples in the k nearest neighbors; (viii) storing the state s in a memory buffer t , the action vector a t , next state s t+1 and reward r t , where the next state s t+1 is determined based on the next pair of source samples, and wherein the reward r t is determined based on the plurality of classifier outputs; (ix) determining whether the termination probability ∈ satisfies a termination threshold; (x) in response to determining that the termination probability ∈ fails to satisfy the termination threshold, incrementing the timestamp t for a plurality of training steps S: Depending on the state s t , the action vector a t and the reward r t The evaluator loss function is used to train the evaluator network Q; as well as The actor network π is trained according to an actor loss function that depends on the output of the critic network, and After training the actor network π and the critic network Q for the plurality of training steps S, returning to step (ii), wherein the next pair of source samples is used as the current pair of source samples; (xi) in response to determining that the termination probability ∈ satisfies the termination threshold, determining whether the number of training segments executed satisfies a threshold number of training segments; (xii) in response to determining that the number of training segments executed fails to satisfy the threshold number of training segments, returning to step (i) to execute a next training segment; as well as (xiii) in response to determining that the number of training segments executed meets the threshold number of training segments, providing the machine learning classifier φ.

16. The computer program product of claim 15, wherein the current pair of source samples are combined according to the composition ratio α to generate the labeled synthetic sample x according to the following equation syn : x syn =α*x0+(1-α)*x1 Where x0 is the first sample in the current pair of samples, x1 is the second sample in the current pair of samples, and y syn is the labeled synthetic sample x syn of hard marks, y0 is the first hard mark value, and y1 is the second hard mark value.

17. The computer program product of claim 15, wherein the reward t is determined according to the following equation: Where M is the evaluation metric, ΔM(φ t ) measures the trained classifier φ t Performance improvements, X val is the validation dataset, y val is the labeled set of the training dataset, where is the baseline at timestamp t, m is a hyperparameter defining the size of the buffer used to form the baseline, and C(φt|st,at) evaluates the trained classifier φ t is the model confidence, P is the model exploration function, and k is the action vector a t Specifies the nearest neighbor size, x i is the labeled synthetic sample x at timestamp t syn k nearest neighbors, and y i is x i Mark.

18. The computer program product of claim 15, wherein the actor loss function is defined according to the following equation: Where N is the number of transitions, π(s i |θ2) is the state s i The expected action, and Q(s i ,π(s i )|θ2) is the evaluator network's response to the expected action π(s i |θ2) and state s i The output of The evaluator loss function is defined according to the following equation: where b t =R(s t ,a t )+γQ(s t+1 ,π(s t+1 |θ1)|θ2),π(s t+1 |θ1) is the action specified by the actor network, and γ is a decimal factor.

19. The computer program product of claim 15, wherein the plurality of source samples are associated with a plurality of transactions in a transaction processing network, wherein the plurality of labeled normal samples are associated with a plurality of non-fraudulent transactions in the plurality of transactions, wherein the plurality of labeled abnormal samples are associated with a plurality of fraudulent transactions in the plurality of transactions, and wherein the program instructions, when executed by at least one processor, further cause the at least one processor to: receiving transaction data associated with a transaction currently being processed in the transaction processing network; processing the transaction data using the trained machine learning classifier to classify the transactions as fraudulent transactions or non-fraudulent transactions; and Responsive to classifying the transaction as a fraudulent transaction, denying authorization for the transaction in the transaction processing network.

20. The computer program product of claim 15, wherein the program instructions, when executed by at least one processor, further cause the at least one processor to: Before executing the training segment: Using the training dataset X 训练 to train the machine learning classifier π; and For the training data set X 训练 For each source sample in the plurality of source samples, each k nearest neighbor is pre-computed.

Images