System, method, and computer program product for improved machine learning models for generating tabular data

Abstract

Systems, methods, and computer program products are provided for improved machine learning models for generating tabular data. An example system includes at least one processor configured to receive tabular data associated with at least one data record including values associated with features. Each feature has a feature type. Mask data associated with masking at least one value is received. A stochastic latent space embedding for each data record is generated based on the tabular data and the mask data using at least one encoder machine learning model. At least one generated data record is generated based on the stochastic latent space embedding and the feature type using at least one decoder machine learning model including a plurality of output distributions. Each generated value of the plurality of each generated data record is generated by one of the output distributions based on the feature type.

Description

SYSTEM, METHOD, AND COMPUTER PROGRAM PRODUCT FOR IMPROVED MACHINE LEARNING MODELS FOR GENERATING TABULAR DATACROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims the benefit of U.S. Provisional Patent Application No. 63,555,242, filed February 19, 2024, the disclosure of which is hereby incorporated by reference in its entirety.BACKGROUND1 . Technical Field

[0002] This disclosure relates generally to generative machine learning models and, in some non-limiting embodiments or aspects, to systems, methods, and computer program products for improved machine learning models for generating tabular data.2. Technical Considerations

[0003] Machine learning models are successful in performing many real-world tasks. For example, a machine learning model may be trained to receive an input and perform a task (e.g., generate an output, such as a prediction, a classification, or the like). Generative machine learning models (and / or generative artificial intelligence models) may by trained based on a particular type of data to generate new data of that type. For example, a large language model (LLM) or a natural language processing (NLP) model may be trained based on text to generate new text (e.g., predict the next word(s), next sentence(s), and / or the like). An image generation model may be trained based on images to generate new images. A music generation model may be trained based on audio data to generate new audio. For each of these types of generative models, the data is homogeneous (e.g., all of the same type), and the dependencies and / or correlations between data items are also homogeneous (and / or of a uniform type that is predictable). For example, text data (e.g., natural language text) may have local dependencies and sequential dependencies (e.g., between words in a sentence and / or the like). Image data may have spatial correlations (e.g., between pixels and / or patches of an image and / or the like). Music / audio data may have sequential dependencies (e.g., between audio samples and / or the like). Additionally, the outputs of these generative models are known to be (and in fact, often designed and / or desired to be) creative and / or fanciful.

[0004] However, tabular data presents unique challenges to machine learning models that are not present for machine learning tasks based on other types of data. For example, tabular data includes heterogeneous types of features (e.g., integer / count features, real number features, categorical features, text features, etc.). Additionally, tabular data includes complex and / or varying types of relationships between features. Also, tabular data includes highly domain-specific sets of features. As such, machine learning models and / or techniques based thereon that are not tailored to tabular data may have difficulty handling these different types of features, may neglect and / or fail to learn the complex feature relationships (e.g., quantify such relationships, disentangle correlation and causation in such relationships, etc.). Additionally, when generating new tabular data, it may be undesirable to generate data that is completely creative or fanciful. For example, when generating tabular data for use cases such as data imputation, upsampling, or transfer learning, it may be desirable for the generated data to reflect real-world data and / or fill in gaps in real- world data.SUMMARY

[0005] Accordingly, provided are improved systems, methods, and computer program products for improved machine learning models for generating tabular data (e.g., that overcome some or all of the deficiencies identified above).

[0006] According to non-limiting embodiments or aspects, provided is a system for improved machine learning models for generating tabular data. An example system may include at least one processor configured to receive tabular data associated with at least one data record. Each data record of the at least one data record may include a plurality of values associated with a plurality of features. Each respective value of the plurality of values may be associated with a respective feature of the plurality of features. Each feature of the plurality of features may have a feature type of a plurality of feature types. Mask data associated with masking at least one value of the plurality of values may be received. A stochastic latent space embedding may be generated for each data record based on the tabular data and the mask data using at least one encoder machine learning model. The stochastic latent space embedding may be associated with a mean and a variance of each dimension of a plurality of dimensions of a latent space. At least one generated data record may be generated based on the stochastic latent space embedding for each data record and the feature type of each feature of each data record using at least one decoder machine learning modelincluding a plurality of output distributions including a respective output distribution for each respective feature type of the plurality of feature types. Each generated data record of the at least one generated data record may include a plurality of generated values associated with the plurality of features. Each generated value of the plurality of generated values may be generated by an output distribution of the plurality of output distributions based on the feature type.

[0007] In some non-limiting embodiments or aspects, the at least one data record may include a plurality of data records. The mask data may be associated with masking at least one respective value of the plurality of values for each respective data record of the plurality of data records.

[0008] In some non-limiting embodiments or aspects, generating the stochastic latent space embedding for each data record may include generating a plurality of a stochastic latent space embeddings including a respective stochastic latent space embedding for each respective data record of the plurality of data records using the at least one encoder machine learning model.

[0009] In some non-limiting embodiments or aspects, at least one autoencoder may include the at least one encoder machine learning model and the at least one decoder machine learning model.

[0010] In some non-limiting embodiments or aspects, the at least one autoencoder may include at least one of a variational autoencoder (VAE), a beta VAE ([3-VAE), a multi-modal VAE (MVAE), an importance weighted autoencoder (IWAE), or any combination thereof.

[0011] In some non-limiting embodiments or aspects, the at least one autoencoder may include a plurality of MVAEs comprising a first MVAE associated with an importance-sampled (IS) evidence lower bound (ELBO), a second MVAE associated with importance weighted sampling (IWS), and a third MVAE associated with a doubly reparametrised gradient estimator (DReG). Generating the stochastic latent space embedding for each data record may include generating the stochastic latent space embedding for each data record based on the tabular data, the mask data, a first encoder machine learning model of the first MVAE, a second encoder machine learning model of the second MVAE, and a third encoder machine learning model of the third MVAE. Generating the at least one generated data record may include generating the at least one generated data record based on the stochastic latent space embedding for each data record, the feature type of each feature of each data record,a first decoder machine learning model of the first MVAE, a second decoder machine learning model of the second MVAE, and a third decoder machine learning model of the third MVAE.

[0012] In some non-limiting embodiments or aspects, the at least one processor may be further configured to train the at least one autoencoder based on the tabular data. Training may include inputting each respective data record of the tabular data to the at least one encoder machine learning model to generate a respective stochastic latent space embedding, inputting the respective stochastic latent space embedding for each respective data record of the tabular data to the at least one decoder machine learning model to generate a respective reconstructed data record, determining a loss based on each respective data record of the tabular data and the respective reconstructed data record, and / or updating parameters of the at least one autoencoder based on the loss.

[0013] In some non-limiting embodiments or aspects, training further may include repeating inputting each respective data record of the tabular data to the at least one encoder machine learning model, inputting the respective stochastic latent space embedding for each respective data record of the tabular data to the at least one decoder machine learning model, determining the loss, and / or updating the parameters of the at least one autoencoder until a termination condition is satisfied.

[0014] In some non-limiting embodiments or aspects, the plurality of feature types may include a count feature type, a real number feature type, and a categorical feature type. The plurality of output distributions may include a gamma distribution for the count feature type, a normal distribution for the real number feature type, and a multinomial distribution for the categorical feature type.

[0015] In some non-limiting embodiments or aspects, the at least one processor may be further configured to train at least one other machine learning model based on the at least one generated data record and / or generate at least one prediction based on the at least one other machine learning model.

[0016] In some non-limiting embodiments or aspects, the tabular data may include transaction data associated with at least one payment transaction. The at least one data record may include at least one transaction record. The at least one other machine learning model may include at least one of a fraud detection model, a risk detection model, a transaction authorization model, a credit approval model, a productrecommendation model, a classifier model, an anomaly detection model, an authentication model, or any combination thereof.

[0017] According to non-limiting embodiments or aspects, provided is a computer- implemented method for improved machine learning models for generating tabular data. An example method may include receiving tabular data associated with at least one data record. Each data record of the at least one data record may include a plurality of values associated with a plurality of features. Each respective value of the plurality of values may be associated with a respective feature of the plurality of features. Each feature of the plurality of features may have a feature type of a plurality of feature types. Mask data associated with masking at least one value of the plurality of values may be received. A stochastic latent space embedding may be generated for each data record based on the tabular data and the mask data using at least one encoder machine learning model. The stochastic latent space embedding may be associated with a mean and a variance of each dimension of a plurality of dimensions of a latent space. A least one generated data record may be generated based on the stochastic latent space embedding for each data record and the feature type of each feature of each data record using at least one decoder machine learning model including a plurality of output distributions including a respective output distribution for each respective feature type of the plurality of feature types. Each generated data record of the at least one generated data record may include a plurality of generated values associated with the plurality of features. Each generated value of the plurality of generated values may be generated by an output distribution of the plurality of output distributions based on the feature type.

[0018] In some non-limiting embodiments or aspects, the at least one data record may include a plurality of data records. The mask data may be associated with masking at least one respective value of the plurality of values for each respective data record of the plurality of data records. Generating the stochastic latent space embedding for each data record may include generating a plurality of a stochastic latent space embeddings including a respective stochastic latent space embedding for each respective data record of the plurality of data records using the at least one encoder machine learning model.

[0019] In some non-limiting embodiments or aspects, at least one autoencoder may include the at least one encoder machine learning model and the at least one decoder machine learning model.

[0020] In some non-limiting embodiments or aspects, the at least one autoencoder may include at least one of a variational autoencoder (VAE), a beta VAE ([3-VAE), a multi-modal VAE (MVAE), an importance weighted autoencoder (IWAE), or any combination thereof.

[0021] In some non-limiting embodiments or aspects, the at least one autoencoder may include a plurality of MVAEs comprising a first MVAE associated with an importance-sampled (IS) evidence lower bound (ELBO), a second MVAE associated with importance weighted sampling (IWS), and a third MVAE associated with a doubly reparametrised gradient estimator (DReG). Generating the stochastic latent space embedding for each data record may include generating the stochastic latent space embedding for each data record based on the tabular data, the mask data, a first encoder machine learning model of the first MVAE, a second encoder machine learning model of the second MVAE, and a third encoder machine learning model of the third MVAE. Generating the at least one generated data record may include generating the at least one generated data record based on the stochastic latent space embedding for each data record, the feature type of each feature of each data record, a first decoder machine learning model of the first MVAE, a second decoder machine learning model of the second MVAE, and a third decoder machine learning model of the third MVAE.

[0022] In some non-limiting embodiments or aspects, the at least one autoencoder may be trained based on the tabular data. For example, training may include inputting each respective data record of the tabular data to the at least one encoder machine learning model to generate a respective stochastic latent space embedding, inputting the respective stochastic latent space embedding for each respective data record of the tabular data to the at least one decoder machine learning model to generate a respective reconstructed data record, determining a loss based on each respective data record of the tabular data and the respective reconstructed data record, updating parameters of the at least one autoencoder based on the loss, and / or repeating inputting each respective data record of the tabular data to the at least one encoder machine learning model, inputting the respective stochastic latent space embedding for each respective data record of the tabular data to the at least one decoder machine learning model, determining the loss, and updating the parameters of the at least one autoencoder until a termination condition is satisfied.

[0023] In some non-limiting embodiments or aspects, the plurality of feature types may include a count feature type, a real number feature type, and a categorical feature type. The plurality of output distributions may include a gamma distribution for the count feature type, a normal distribution for the real number feature type, and a multinomial distribution for the categorical feature type.

[0024] In some non-limiting embodiments or aspects, at least one other machine learning model may be trained based on the at least one generated data record. At least one prediction may be generated based on the at least one other machine learning model.

[0025] In some non-limiting embodiments or aspects, the tabular data may include transaction data associated with at least one payment transaction. The at least one data record may include at least one transaction record. The at least one other machine learning model may include at least one of a fraud detection model, a risk detection model, a transaction authorization model, a credit approval model, a product recommendation model, a classifier model, an anomaly detection model, an authentication model, or any combination thereof.

[0026] According to non-limiting embodiments or aspects, provided is a computer program product for improved machine learning models for generating tabular data. An example computer program product may include at least one non-transitory computer-readable medium including program instructions that, when executed by at least one processor, cause the at least one processor to receive tabular data associated with at least one data record. Each data record of the at least one data record may include a plurality of values associated with a plurality of features. Each respective value of the plurality of values may be associated with a respective feature of the plurality of features. Each feature of the plurality of features may have a feature type of a plurality of feature types. Mask data associated with masking at least one value of the plurality of values may be received. A stochastic latent space embedding for each data record may be generated based on the tabular data and the mask data using at least one encoder machine learning model. The stochastic latent space embedding may be associated with a mean and a variance of each dimension of a plurality of dimensions of a latent space. At least one generated data record may be generated based on the stochastic latent space embedding for each data record and the feature type of each feature of each data record using at least one decoder machine learning model including a plurality of output distributions including arespective output distribution for each respective feature type of the plurality of feature types. Each generated data record of the at least one generated data record may include a plurality of generated values associated with the plurality of features. Each generated value of the plurality of generated values may be generated by an output distribution of the plurality of output distributions based on the feature type.

[0027] Further non-limiting embodiments or aspects are set forth in the following numbered clauses:

[0028] Clause 1 : A system, comprising: at least one processor configured to: receive tabular data associated with at least one data record, each data record of the at least one data record comprising a plurality of values associated with a plurality of features, each respective value of the plurality of values associated with a respective feature of the plurality of features, each feature of the plurality of features having a feature type of a plurality of feature types; receive mask data associated with masking at least one value of the plurality of values; generate a stochastic latent space embedding for each data record based on the tabular data and the mask data using at least one encoder machine learning model, the stochastic latent space embedding associated with a mean and a variance of each dimension of a plurality of dimensions of a latent space; and generate at least one generated data record based on the stochastic latent space embedding for each data record and the feature type of each feature of each data record using at least one decoder machine learning model comprising a plurality of output distributions comprising a respective output distribution for each respective feature type of the plurality of feature types, each generated data record of the at least one generated data record comprising a plurality of generated values associated with the plurality of features, each generated value of the plurality of generated values generated by an output distribution of the plurality of output distributions based on the feature type.

[0029] Clause 2: The system of clause 1 , wherein the at least one data record comprises a plurality of data records, wherein the mask data is associated with masking at least one respective value of the plurality of values for each respective data record of the plurality of data records.

[0030] Clause 3: The system of clause 1 or clause 2, wherein generating the stochastic latent space embedding for each data record comprises generating a plurality of a stochastic latent space embeddings comprising a respective stochasticlatent space embedding for each respective data record of the plurality of data records using the at least one encoder machine learning model.

[0031] Clause 4: The system of any of clauses 1 -3, wherein at least one autoencoder comprises the at least one encoder machine learning model and the at least one decoder machine learning model.

[0032] Clause 5: The system of any of clauses 1 -4, wherein the at least one autoencoder comprises at least one of a variational autoencoder (VAE), a beta VAE ([3-VAE), a multi-modal VAE (MVAE), an importance weighted autoencoder (IWAE), or any combination thereof.

[0033] Clause 6: The system of any of clauses 1 -5, wherein the at least one autoencoder comprises a plurality of MVAEs comprising a first MVAE associated with an importance-sampled (IS) evidence lower bound (ELBO), a second MVAE associated with importance weighted sampling (IWS), and a third MVAE associated with a doubly reparametrised gradient estimator (DReG), wherein generating the stochastic latent space embedding for each data record comprises generating the stochastic latent space embedding for each data record based on the tabular data, the mask data, a first encoder machine learning model of the first MVAE, a second encoder machine learning model of the second MVAE, and a third encoder machine learning model of the third MVAE, and wherein generating the at least one generated data record comprises generating the at least one generated data record based on the stochastic latent space embedding for each data record, the feature type of each feature of each data record, a first decoder machine learning model of the first MVAE, a second decoder machine learning model of the second MVAE, and a third decoder machine learning model of the third MVAE.

[0034] Clause 7: The system of any of clauses 1 -6, wherein the at least one processor is further configured to: train the at least one autoencoder based on the tabular data, wherein training comprises: inputting each respective data record of the tabular data to the at least one encoder machine learning model to generate a respective stochastic latent space embedding; inputting the respective stochastic latent space embedding for each respective data record of the tabular data to the at least one decoder machine learning model to generate a respective reconstructed data record; determining a loss based on each respective data record of the tabular data and the respective reconstructed data record; and updating parameters of the at least one autoencoder based on the loss.

[0035] Clause 8: The system of any of clauses 1 -7, wherein training further comprises repeating inputting each respective data record of the tabular data to the at least one encoder machine learning model, inputting the respective stochastic latent space embedding for each respective data record of the tabular data to the at least one decoder machine learning model, determining the loss, and updating the parameters of the at least one autoencoder until a termination condition is satisfied.

[0036] Clause 9: The system of any of clauses 1 -8, wherein the plurality of feature types comprises a count feature type, a real number feature type, and a categorical feature type, wherein the plurality of output distributions comprises a gamma distribution for the count feature type, a normal distribution for the real number feature type, and a multinomial distribution for the categorical feature type.

[0037] Clause 10: The system of any of clauses 1 -9, wherein the at least one processor is further configured to: train at least one other machine learning model based on the at least one generated data record; and generate at least one prediction based on the at least one other machine learning model.

[0038] Clause 1 1 : The system of any of clauses 1 -10, wherein the tabular data comprises transaction data associated with at least one payment transaction, the at least one data record comprises at least one transaction record, and wherein the at least one other machine learning model comprises at least one of a fraud detection model, a risk detection model, a transaction authorization model, a credit approval model, a product recommendation model, a classifier model, an anomaly detection model, an authentication model, or any combination thereof.

[0039] Clause 12: A computer-implemented method, comprising: receiving, with at least one processor, tabular data associated with at least one data record, each data record of the at least one data record comprising a plurality of values associated with a plurality of features, each respective value of the plurality of values associated with a respective feature of the plurality of features, each feature of the plurality of features having a feature type of a plurality of feature types; receiving, with at least one processor, mask data associated with masking at least one value of the plurality of values; generating, with at least one processor, a stochastic latent space embedding for each data record based on the tabular data and the mask data using at least one encoder machine learning model, the stochastic latent space embedding associated with a mean and a variance of each dimension of a plurality of dimensions of a latent space; and generating, with at least one processor, at least one generated data recordbased on the stochastic latent space embedding for each data record and the feature type of each feature of each data record using at least one decoder machine learning model comprising a plurality of output distributions comprising a respective output distribution for each respective feature type of the plurality of feature types, each generated data record of the at least one generated data record comprising a plurality of generated values associated with the plurality of features, each generated value of the plurality of generated values generated by an output distribution of the plurality of output distributions based on the feature type.

[0040] Clause 13: The method of clause 12, wherein the at least one data record comprises a plurality of data records, wherein the mask data is associated with masking at least one respective value of the plurality of values for each respective data record of the plurality of data records, wherein generating the stochastic latent space embedding for each data record comprises generating a plurality of a stochastic latent space embeddings comprising a respective stochastic latent space embedding for each respective data record of the plurality of data records using the at least one encoder machine learning model.

[0041] Clause 14: The method of clause 12 or clause 13, wherein at least one autoencoder comprises the at least one encoder machine learning model and the at least one decoder machine learning model.

[0042] Clause 15: The method of any of clauses 12-14, wherein the at least one autoencoder comprises at least one of a variational autoencoder (VAE), a beta VAE ([3-VAE), a multi-modal VAE (MVAE), an importance weighted autoencoder (IWAE), or any combination thereof.

[0043] Clause 16: The method of any of clauses 12-15, wherein the at least one autoencoder comprises a plurality of MVAEs comprising a first MVAE associated with an importance-sampled (IS) evidence lower bound (ELBO), a second MVAE associated with importance weighted sampling (IWS), and a third MVAE associated with a doubly reparametrised gradient estimator (DReG), wherein generating the stochastic latent space embedding for each data record comprises generating the stochastic latent space embedding for each data record based on the tabular data, the mask data, a first encoder machine learning model of the first MVAE, a second encoder machine learning model of the second MVAE, and a third encoder machine learning model of the third MVAE, and wherein generating the at least one generated data record comprises generating the at least one generated data record based on thestochastic latent space embedding for each data record, the feature type of each feature of each data record, a first decoder machine learning model of the first MVAE, a second decoder machine learning model of the second MVAE, and a third decoder machine learning model of the third MVAE.

[0044] Clause 17: The method of any of clauses 12-16, further comprising: training, with at least one processor, the at least one autoencoder based on the tabular data, wherein training comprises: inputting each respective data record of the tabular data to the at least one encoder machine learning model to generate a respective stochastic latent space embedding; inputting the respective stochastic latent space embedding for each respective data record of the tabular data to the at least one decoder machine learning model to generate a respective reconstructed data record; determining a loss based on each respective data record of the tabular data and the respective reconstructed data record; updating parameters of the at least one autoencoder based on the loss; and repeating inputting each respective data record of the tabular data to the at least one encoder machine learning model, inputting the respective stochastic latent space embedding for each respective data record of the tabular data to the at least one decoder machine learning model, determining the loss, and updating the parameters of the at least one autoencoder until a termination condition is satisfied.

[0045] Clause 18: The method of any of clauses 12-17, wherein the plurality of feature types comprises a count feature type, a real number feature type, and a categorical feature type, wherein the plurality of output distributions comprises a gamma distribution for the count feature type, a normal distribution for the real number feature type, and a multinomial distribution for the categorical feature type.

[0046] Clause 19: The method of any of clauses 12-18, further comprising: training, with at least one processor, at least one other machine learning model based on the at least one generated data record; and generating, with at least one processor, at least one prediction based on the at least one other machine learning model, wherein the tabular data comprises transaction data associated with at least one payment transaction, the at least one data record comprises at least one transaction record, and wherein the at least one other machine learning model comprises at least one of a fraud detection model, a risk detection model, a transaction authorization model, a credit approval model, a product recommendation model, a classifier model, an anomaly detection model, an authentication model, or any combination thereof.

[0047] Clause 20: A computer program product comprising at least one non- transitory computer-readable medium including program instructions that, when executed by at least one processor, cause the at least one processor to: receive tabular data associated with at least one data record, each data record of the at least one data record comprising a plurality of values associated with a plurality of features, each respective value of the plurality of values associated with a respective feature of the plurality of features, each feature of the plurality of features having a feature type of a plurality of feature types; receive mask data associated with masking at least one value of the plurality of values; generate a stochastic latent space embedding for each data record based on the tabular data and the mask data using at least one encoder machine learning model, the stochastic latent space embedding associated with a mean and a variance of each dimension of a plurality of dimensions of a latent space; and generate at least one generated data record based on the stochastic latent space embedding for each data record and the feature type of each feature of each data record using at least one decoder machine learning model comprising a plurality of output distributions comprising a respective output distribution for each respective feature type of the plurality of feature types, each generated data record of the at least one generated data record comprising a plurality of generated values associated with the plurality of features, each generated value of the plurality of generated values generated by an output distribution of the plurality of output distributions based on the feature type.

[0048] These and other features and characteristics of the present disclosure, as well as the methods of operation and functions of the related elements of structures and the combination of parts and economies of manufacture, will become more apparent upon consideration of the following description and the appended claims with reference to the accompanying drawings, all of which form a part of this specification, wherein like reference numerals designate corresponding parts in the various figures. It is to be expressly understood, however, that the drawings are for the purpose of illustration and description only and are not intended as a definition of the limits of the disclosed subject matter.BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Additional advantages and details are explained in greater detail below with reference to the non-limiting, exemplary embodiments that are illustrated in the accompanying schematic figures, in which:

[0050] FIG. 1 is a schematic diagram of an example system for improved machine learning models for generating tabular data, according to some non-limiting embodiments or aspects;

[0051] FIG. 2 is a flow diagram of an example method for improved machine learning models for generating tabular data, according to some non-limiting embodiments or aspects;

[0052] FIG. 3 is a diagram of an example payment processing network in which systems, methods, and / or computer program products, described herein, may be implemented, according to some non-limiting embodiments or aspects;

[0053] FIG. 4 is a schematic diagram of example components of one or more devices of FIG. 1 and / or FIG. 3, according to some non-limiting embodiments or aspects;

[0054] FIG. 5 is a schematic diagram of an example implementation of a machine learning models for generating tabular data, according to some non-limiting embodiments or aspects; and

[0055] FIGS. 6A-6D are bar charts of actual and generated distributions of tabular data based on an example implementation of a method for improved machine learning models for generating tabular data, according to some non-limiting embodiments or aspects.DETAILED DESCRIPTION

[0056] For purposes of the description hereinafter, the terms “end,” “upper,” “lower,” “right,” “left,” “vertical,” “horizontal,” “top,” “bottom,” “lateral,” “longitudinal,” and derivatives thereof shall relate to the embodiments as they are oriented in the drawing figures. However, it is to be understood that the present disclosure may assume various alternative variations and step sequences, except where expressly specified to the contrary. It is also to be understood that the specific devices and processes illustrated in the attached drawings, and described in the following specification, are simply exemplary and non-limiting embodiments or aspects of the disclosed subject matter. Hence, specific dimensions and other physical characteristics related to the embodiments or aspects disclosed herein are not to be considered as limiting.

[0057] Some non-limiting embodiments or aspects may be described herein in connection with thresholds. As used herein, satisfying a threshold may refer to a value being greater than the threshold, more than the threshold, higher than the threshold, greater than or equal to the threshold, less than the threshold, fewer than thethreshold, lower than the threshold, less than or equal to the threshold, equal to the threshold, etc.

[0058] No aspect, component, element, structure, act, step, function, instruction, and / or the like used herein should be construed as critical or essential unless explicitly described as such. Also, as used herein, the articles “a” and “an” are intended to include one or more items and may be used interchangeably with “one or more” and “at least one.” Furthermore, as used herein, the term “set” is intended to include one or more items (e.g., related items, unrelated items, a combination of related and unrelated items, and / or the like) and may be used interchangeably with “one or more” or “at least one.” Where only one item is intended, the term “one” or similar language is used. Also, as used herein, the terms “has,” “have,” “having,” or the like are intended to be open-ended terms. Further, the phrase “based on” is intended to mean “based at least partially on” unless explicitly stated otherwise. In addition, reference to an action being “based on” a condition may refer to the action being “in response to” the condition. For example, the phrases “based on” and “in response to” may, in some non-limiting embodiments or aspects, refer to a condition for automatically triggering an action (e.g., a specific operation of an electronic device, such as a computing device, a processor, and / or the like).

[0059] As used herein, the term “acquirer institution” may refer to an entity licensed and / or approved by a transaction service provider to originate transactions (e.g., payment transactions) using a payment device associated with the transaction service provider. The transactions the acquirer institution may originate may include payment transactions (e.g., purchases, original credit transactions (OCTs), account funding transactions (AFTs), and / or the like). In some non-limiting embodiments or aspects, an acquirer institution may be a financial institution, such as a bank. As used herein, the term “acquirer system” may refer to one or more computing devices operated by or on behalf of an acquirer institution, such as a server computer executing one or more software applications.

[0060] As used herein, the term “account identifier” may include one or more primary account numbers (PANs), tokens, or other identifiers associated with a customer account. The term “token” may refer to an identifier that is used as a substitute or replacement identifier for an original account identifier, such as a PAN. Account identifiers may be alphanumeric or any combination of characters and / or symbols. Tokens may be associated with a PAN or other original account identifier inone or more data structures (e.g., one or more databases, and / or the like) such that they may be used to conduct a transaction without directly using the original account identifier. In some examples, an original account identifier, such as a PAN, may be associated with a plurality of tokens for different individuals or purposes.

[0061] As used herein, the terms “client” and “client device” may refer to one or more client-side devices or systems (e.g., remote from a transaction service provider) used to initiate or facilitate a transaction (e.g., a payment transaction). As an example, a “client device” may refer to one or more POS devices used by a merchant, one or more acquirer host computers used by an acquirer, one or more mobile devices used by a user, and / or the like. In some non-limiting embodiments or aspects, a client device may be an electronic device configured to communicate with one or more networks and initiate or facilitate transactions. For example, a client device may include one or more computers, portable computers, laptop computers, tablet computers, mobile devices, cellular phones, wearable devices (e.g., watches, glasses, lenses, clothing, and / or the like), PDAs, and / or the like. Moreover, a “client” may also refer to an entity (e.g., a merchant, an acquirer, and / or the like) that owns, utilizes, and / or operates a client device for initiating transactions (e.g., for initiating transactions with a transaction service provider).

[0062] As used herein, the term “communication” may refer to the reception, receipt, transmission, transfer, provision, and / or the like of data (e.g., information, signals, messages, instructions, commands, and / or the like). For one unit (e.g., a device, a system, a component of a device or system, combinations thereof, and / or the like) to be in communication with another unit means that the one unit is able to directly or indirectly receive information from and / or transmit information to the other unit. This may refer to a direct or indirect connection (e.g., a direct communication connection, an indirect communication connection, and / or the like) that is wired and / or wireless in nature. Additionally, two units may be in communication with each other even though the information transmitted may be modified, processed, relayed, and / or routed between the first and second unit. For example, a first unit may be in communication with a second unit even though the first unit passively receives information and does not actively transmit information to the second unit. As another example, a first unit may be in communication with a second unit if at least one intermediary unit processes information received from the first unit and communicates the processed information to the second unit. In some non-limiting embodiments oraspects, a message may refer to a network packet (e.g., a data packet and / or the like) that includes data. It will be appreciated that numerous other arrangements are possible.

[0063] As used herein, the term “computing device” may refer to one or more electronic devices configured to process data. A computing device may, in some examples, include the necessary components to receive, process, and output data, such as a processor, a display, a memory, an input device, a network interface, and / or the like. A computing device may be a mobile device. As an example, a mobile device may include a cellular phone (e.g., a smartphone or standard cellular phone), a portable computer, a wearable device (e.g., watches, glasses, lenses, clothing, and / or the like), a personal digital assistant (PDA), and / or other like devices. A computing device may also be a desktop computer or other form of non-mobile computer.

[0064] As used herein, the terms “electronic wallet” and “electronic wallet application” refer to one or more electronic devices and / or software applications configured to initiate and / or conduct payment transactions. For example, an electronic wallet may include a mobile device executing an electronic wallet application, and may further include server-side software and / or databases for maintaining and providing transaction data to the mobile device. An “electronic wallet provider” may include an entity that provides and / or maintains an electronic wallet for a customer, such as Google Pay®, Android Pay®, Apple Pay®, Samsung Pay®, and / or other like electronic payment systems. In some non-limiting examples, an issuer bank may be an electronic wallet provider.

[0065] As used herein, the term “issuer institution” may refer to one or more entities, such as a bank, that provide accounts to customers for conducting transactions (e.g., payment transactions), such as initiating credit and / or debit payments. For example, an issuer institution may provide an account identifier, such as a PAN, to a customer that uniquely identifies one or more accounts associated with that customer. The account identifier may be embodied on a portable financial device, such as a physical financial instrument, e.g., a payment card, and / or may be electronic and used for electronic payments. The term “issuer system” refers to one or more computer devices operated by or on behalf of an issuer institution, such as a server computer executing one or more software applications. For example, an issuer system may include one or more authorization servers for authorizing a transaction.

[0066] As used herein, the term “merchant” may refer to an individual or entity that provides goods and / or services, or access to goods and / or services, to customers based on a transaction, such as a payment transaction. The term “merchant” or “merchant system” may also refer to one or more computer systems operated by or on behalf of a merchant, such as a server computer executing one or more software applications.

[0067] As used herein, a “point-of-sale (POS) device” may refer to one or more devices, which may be used by a merchant to conduct a transaction (e.g., a payment transaction) and / or process a transaction. For example, a POS device may include one or more client devices. Additionally or alternatively, a POS device may include peripheral devices, card readers, scanning devices (e.g., code scanners), Bluetooth® communication receivers, near-field communication (NFC) receivers, radio frequency identification (RFID) receivers, and / or other contactless transceivers or receivers, contact-based receivers, payment terminals, and / or the like. As used herein, a “point- of-sale (POS) system” may refer to one or more client devices and / or peripheral devices used by a merchant to conduct a transaction. For example, a POS system may include one or more POS devices and / or other like devices that may be used to conduct a payment transaction. In some non-limiting embodiments or aspects, a POS system (e.g., a merchant POS system) may include one or more server computers programmed or configured to process online payment transactions through webpages, mobile applications, and / or the like.

[0068] As used herein, the term “payment device” may refer to an electronic payment device, a portable financial device, a payment card (e.g., a credit or debit card), a gift card, a smartcard, smart media, a payroll card, a healthcare card, a wristband, a machine-readable medium containing account information, a keychain device or fob, an RFID transponder, a retailer discount or loyalty card, a cellular phone, an electronic wallet mobile application, a personal digital assistant (PDA), a pager, a security card, a computing device, an access card, a wireless terminal, a transponder, and / or the like. In some non-limiting embodiments or aspects, the payment device may include volatile or non-volatile memory to store information (e.g., an account identifier, a name of the account holder, and / or the like).

[0069] As used herein, the term “payment gateway” may refer to an entity and / or a payment processing system operated by or on behalf of such an entity (e.g., a merchant service provider, a payment service provider, a payment facilitator, apayment facilitator that contracts with an acquirer, a payment aggregator, and / or the like), which provides payment services (e.g., transaction service provider payment services, payment processing services, and / or the like) to one or more merchants. The payment services may be associated with the use of portable financial devices managed by a transaction service provider. As used herein, the term “payment gateway system” may refer to one or more computer systems, computer devices, servers, groups of servers, and / or the like, operated by or on behalf of a payment gateway.

[0070] As used herein, the term “server” may refer to or include one or more computing devices that are operated by or facilitate communication and processing for multiple parties in a network environment, such as the Internet, although it will be appreciated that communication may be facilitated over one or more public or private network environments and that various other arrangements are possible. Further, multiple computing devices (e.g., servers, point-of-sale (POS) devices, mobile devices, etc.) directly or indirectly communicating in the network environment may constitute a “system.”

[0071] As used herein, the term “system” may refer to one or more computing devices or combinations of computing devices and / or components of such (e.g., processors, servers, client devices, software applications, and / or the like). Reference to “a device,” “a server,” “a processor,” and / or the like, as used herein, may refer to a previously-recited device, server, or processor that is recited as performing a previous step or function, a different device, server, or processor, and / or a combination of devices, servers, and / or processors. For example, as used in the specification and the claims, a first device, a first server, or a first processor that is recited as performing a first step or a first function may refer to the same or different device, server, or processor recited as performing a second step or a second function.

[0072] As used herein, the term “transaction service provider” may refer to an entity that receives transaction authorization requests from merchants or other entities and provides guarantees of payment, in some cases through an agreement between the transaction service provider and an issuer institution. For example, a transaction service provider may include a payment network such as Visa® or any other entity that processes transactions. The term “transaction processing system” may refer to one or more computer systems operated by or on behalf of a transaction service provider, such as a transaction processing server executing one or more softwareapplications. A transaction processing server may include one or more processors and, in some non-limiting embodiments or aspects, may be operated by or on behalf of a transaction service provider.

[0073] Non-limiting embodiments or aspects of the disclosed subject matter are directed to systems, methods, and computer program products for improved machine learning models for generating tabular data. For example, non-limiting embodiments or aspects of the disclosed subject matter provide receiving tabular data associated with at least one data record (e.g., with values associated with a plurality of features of a plurality of feature types), receiving mask data associated with masking at least one value of the tabular data, and generating a stochastic latent space embedding for each data record of the tabular data based on the tabular data and the mask data using at least one encoder machine learning model. The stochastic latent space embedding may be associated with a mean and a variance of each dimension of a plurality of dimensions of a latent space. At least one generated data record may be generated based on the stochastic latent space embedding for each data record and the feature type of each feature of each data record using at least one decoder machine learning model, which may include a plurality of output distributions including a respective output distribution for each respective feature type of the plurality of feature types. Each generated data record may include a plurality of generated values associated with the plurality of features, and each generated value may be generated based on one of the output distributions based on the feature type. Such embodiments or aspects enable generation of tabular data in a manner that addresses unique problems that arise with tabular data, such as heterogeneous types of data (e.g., categorical, integer / count, and real numbers), complex and / or varying types of relationships between features (e.g., local and / or global correlation), difficulty in quantifying relationships between features, and disentangling correlation and causation, for example, because the data generated based on the stochastic latent space embeddings (which are associated with the mean and variance of the dimensions of the latent space) and the different output distributions allow for different treatment of the different types of data. Additionally, such embodiments or aspects allow for specific use cases such as data imputation (e.g., predicting the values of missing data from an existing data set), upsampling (e.g., when there is sparse data), and transfer learning (e.g., generated tabular data can be used for training other machine learning models), for example, because the data generated based on thestochastic latent space embeddings (which are associated with the mean and variance of the dimensions of the latent space) and the different output distributions reflect real- world data. For example, the generated tabular data has greater accuracy (e.g., for reconstructing the original input data, including imputing values for missing data) and / or has limited “creativity” (e.g., compared to generative models for natural language, images, or music) such that the generated data reflects the real world.

[0074] Additionally, non-limiting embodiments or aspects of the disclosed subject matter provide a plurality of multi-modal variational autoencoders (MVAEs) including a first MVAE associated with an importance-sampled (IS) evidence lower bound (ELBO), a second MVAE associated with importance weighted sampling (IWS), and a third MVAE associated with a doubly reparametrised gradient estimator (DReG). Generating the stochastic latent space embedding for each data record may include generating the stochastic latent space embedding for each data record based on the tabular data, the mask data, a first encoder machine learning model of the first MVAE, a second encoder machine learning model of the second MVAE, and a third encoder machine learning model of the third MVAE, and generating the at least one generated data record may include generating the at least one generated data record based on the stochastic latent space embedding for each data record, the feature type of each feature of each data record, a first decoder machine learning model of the first MVAE, a second decoder machine learning model of the second MVAE, and a third decoder machine learning model of the third MVAE. Such embodiments or aspects provide even further improved performance (e.g., accuracy and / or handling of the heterogeneous types of data and complex / or varying types of relationships between features), for example, because multiple different MVAEs with different properties are utilized together such that improved performance by one or more types of MVAE with respect to the others can be reflected in the generated data.

[0075] Furthermore, non-limiting embodiments or aspects of the disclosed subject matter provide training at least one other machine learning model based on the at least one generated data record and performing a task (e.g., generating at least one prediction) based on the other machine learning model. Such embodiments or aspects provide transfer learning, for example, because the other machine learning model is trained (e.g., learns) based on the generated data. This transfer learning may be especially useful where real-world data is sparse, for example, because the generated data enables upsampling of the original data so that the combined dataset (e.g.,including the original data and the generated data) is sufficient to train the other machine learning model. In addition, such embodiments or aspects enable performance of real-world tasks with the other machine learning model, such as fraud detection, risk detection, transaction authorization, credit approval, product recommendation, classification, anomaly detection, authentication, any combination thereof, and / or the like. For example, the tabular data may include transaction data, and the at least one other machine learning model may include at least one of a fraud detection model, a risk detection model, a transaction authorization model, a credit approval model, a product recommendation model, a classifier model, an anomaly detection model, an authentication model, any combination thereof, and / or the like.

[0076] For the purpose of illustration, in the following description, while the presently disclosed subject matter is described with respect to systems, methods, and computer program products for improved machine learning models for generating tabular data, one skilled in the art will recognize that the disclosed subject matter is not limited to the illustrative embodiments or aspects. For example, the systems, methods, and computer program products described herein may be used with a wide variety of settings, such as generating formatted data (e.g., data with a specific structure, syntax, and / or the like) for any suitable type of machine learning task, e.g., prediction, recommendation, fraud detection, anomaly detection, risk detection, credit default prediction, credit application approval, transaction approval, and / or the like.

[0077] Referring now to FIG. 1 , shown is an example system 100 for improved machine learning models for generating tabular data, according to some non-limiting embodiments or aspects. As shown in FIG. 1 , system 100 may include machine learning model system 102, data storage system 104, user device 106, and / or communication network 1 12.

[0078] Machine learning model system 102 may include one or more devices capable of receiving information from and / or communicating information to data storage system 104 and / or user device 106 (e.g., directly via wired or wireless communication connection, indirectly via communication network 112, and / or the like). For example, machine learning model system 102 may include a computing device, such as a server, a group of servers, a desktop computer, a portable computer, a mobile device, and / or other like devices. In some non-limiting embodiments or aspects, machine learning model system 102 may be in communication with a data storage device (e.g., data storage system 104), which may be local or remote tomachine learning model system 102. In some non-limiting embodiments or aspects, machine learning model system 102 may be capable of receiving information from, storing information in, communicating information to, or searching information stored in the data storage device (e.g., data storage system 104).

[0079] Data storage system 104 may include one or more devices capable of receiving information from and / or communicating information to machine learning model system 102 and / or user device 106 (e.g., directly via wired or wireless communication connection, indirectly via communication network 112, and / or the like). For example, data storage system 104 may include a computing device, such as a server, a group of servers, a desktop computer, a portable computer, a mobile device, and / or other like devices. In some non-limiting embodiments or aspects, data storage system 104 may include at least one data storage device (e.g., memory, a storage component, and / or the like). In some non-limiting embodiments or aspects, data storage system 104 may be capable of receiving information from, storing information in, communicating information to, or searching information stored in the data storage device. In some non-limiting embodiments or aspects, data storage system 104 may be part of machine learning model system 102 and / or part of the same system as machine learning model system 102. In some non-limiting embodiments or aspects, data storage system 104 may include and / or implement a database.

[0080] User device 106 may include one or more devices capable of receiving information from and / or communicating information to machine learning model system 102 and / or data storage system 104 (e.g., directly via wired or wireless communication connection, indirectly via communication network 1 12, and / or the like). For example, user device 106 may include a computing device, such as a mobile device, a portable computer, a desktop computer, and / or other like devices. Additionally or alternatively, each user device 106 may include a device capable of receiving information from and / or communicating information to other user devices 106 (e.g., directly via wired or wireless communication connection, indirectly via communication network 1 12, and / or the like). In some non-limiting embodiments or aspects, user device 106 may be part of machine learning model system 102 and / or part of the same system as machine learning model system 102. For example, machine learning model system 102, data storage system 104, and user device 106 may all be (and / or be part of) a single system and / or a single computing device.

[0081] Communication network 1 12 may include one or more wired and / or wireless networks. For example, communication network 1 12 may include a cellular network (e.g., a long-term evolution (LTE®) network, a third generation (5G) network, a fourth generation (4G) network, a fifth generation (5G) network, a code division multiple access (CDMA) network, and / or the like), a public land mobile network (PLMN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a telephone network (e.g., the public switched telephone network (PSTN)), a private network (e.g., a private network associated with a transaction service provider), an ad hoc network, an intranet, the Internet, a fiber optic-based network, a cloud computing network, and / or the like, and / or a combination of these or other types of networks.

[0082] The number and arrangement of systems and devices shown in FIG. 1 are provided as an example. There may be additional systems and / or devices, fewer systems and / or devices, different systems and / or devices, and / or differently arranged systems and / or devices than those shown in FIG. 1. Furthermore, two or more systems or devices shown in FIG. 1 may be implemented within a single system or device, or a single system or device shown in FIG. 1 may be implemented as multiple, distributed systems or devices. Additionally or alternatively, a set of systems (e.g., one or more systems) or a set of devices (e.g., one or more devices) of system 100 may perform one or more functions described as being performed by another set of systems or another set of devices of system 100.

[0083] Referring now to FIG. 2, shown is a flow diagram for an example method 200 for improved machine learning models for generating tabular data, according to some non-limiting embodiments or aspects. The steps shown in FIG. 2 are for example purposes only. It will be appreciated that additional, fewer, different, and / or a different order of steps may be used in some non-limiting embodiments or aspects. In some non-limiting embodiments or aspects, a step may be automatically performed in response to performance and / or completion of a prior step. In some non-limiting embodiments or aspects, one or more of the steps of method 200 may be performed (e.g., completely, partially, and / or the like) by machine learning model system 102 (e.g., one or more devices of machine learning model system 102). In some nonlimiting embodiments or aspects, one or more of the steps of method 200 may be performed (e.g., completely, partially, and / or the like) by another system, another device, another group of systems, or another group of devices, separate from orincluding machine learning model system 102, such as data storage system 104 and / or user device 106.

[0084] As shown in FIG. 2, at step 202, method 200 may include receiving tabular data. For example, machine learning model system 102 may receive tabular data associated with at least one data record (e.g., a plurality of data records).

[0085] In some non-limiting embodiments or aspects, each data record may include a plurality of values associated with a plurality of features. For example, each respective value may be associated with a respective feature of the plurality of features. Each feature may have a feature type of a plurality of feature types.

[0086] In some non-limiting embodiments or aspects, the at least one data record may include a plurality of data records.

[0087] In some non-limiting embodiments or aspects, the plurality of feature types may include a count feature type, a real number feature type, and a categorical feature type.

[0088] As shown in FIG. 2, at step 204, method 200 may include receiving (and / or generating) mask data. For example, machine learning model system 102 may receive mask data associated with masking at least one value of the plurality of values.

[0089] In some non-limiting embodiments or aspects, the at least one data record may include a plurality of data records. The mask data may be associated with masking at least one respective value of the plurality of values for each respective data record of the plurality of data records.

[0090] As shown in FIG. 2, at step 206, method 200 may include generating at least one stochastic latent space embedding. For example, machine learning model system 102 may generate a stochastic latent space embedding for each data record based on the tabular data and the mask data using at least one encoder machine learning model. In some non-limiting embodiments or aspects, the stochastic latent space embedding may be associated with a mean and a variance of each dimension of a plurality of dimensions of a latent space.

[0091] In some non-limiting embodiments or aspects, the at least one data record may include a plurality of data records. Generating the stochastic latent space embedding for each data record may include generating a plurality of a stochastic latent space embeddings comprising a respective stochastic latent space embedding for each respective data record of the plurality of data records using the at least one encoder machine learning model.

[0092] In some non-limiting embodiments or aspects, at least one autoencoder may include the encoder machine learning model(s) (e.g., an encoder network of the autoencoder) and the decoder machine learning model(s) (e.g., a decoder network of the autoencoder). In other words, the encoder machine learning model(s) and the decoder machine learning model(s) may be parts of at least one autoencoder. In some non-limiting embodiments or aspects, the autoencoder(s) may include at least one of a variational autoencoder (VAE), a beta VAE ([3-VAE), a multi-modal VAE (MVAE), an importance weighted autoencoder (IWAE), any combination thereof, and / or the like.

[0093] In some non-limiting embodiments or aspects, the autoencoder(s) may include a plurality of MVAEs. For example, the MVAEs may include a first MVAE associated with an importance-sampled (IS) evidence lower bound (ELBO), a second MVAE associated with importance weighted sampling (IWS), and a third MVAE associated with a doubly reparametrised gradient estimator (DReG). In some nonlimiting embodiments or aspects, IS, IWS, and DReG each may be associated with a respective estimate of a true ELBO.

[0094] In some non-limiting embodiments or aspects, generating the stochastic latent space embedding for each data record may include generating the stochastic latent space embedding for each data record based on the tabular data, the mask data, a first encoder machine learning model (e.g., first encoder network) of the first MVAE (e.g., associated with IS), a second encoder machine learning model (e.g., second encoder network) of the second MVAE (e.g., associated with IWS), and a third encoder machine learning model (e.g., third encoder network) of the third MVAE (e.g., associated with DReG).

[0095] In some non-limiting embodiments or aspects, machine learning model system 102 may train the autoencoder(s). For example, machine learning model system 102 may train the autoencoder(s) based on tabular data (e.g., the same tabular data described above, other tabular data such as a training dataset, a historical dataset, and / or the like of tabular data, any combination thereof, and / or the like).

[0096] In some non-limiting embodiments or aspects, training may include inputting each respective data record of the tabular data to the encoder machine learning model(s) (e.g., encoder network(s) of the autoencoder(s)) to generate a respective stochastic latent space embedding and / or inputting the respective stochastic latent space embedding for each respective data record of the tabular data to the decoder machine learning model(s) (e.g., decoder network(s) of the autoencoder(s)) togenerate a respective reconstructed data record (e.g., forward propagation). A loss may be determined based on each respective data record of the tabular data and the respective reconstructed data record. For example, the losses may be calculated based on a difference between each respective data record and the respective reconstructed data record, a loss function based on each respective data record and the respective reconstructed data record, an error, a mean error, a mean squared error (MSE), any combination thereof, and / or the like. The parameters of the autoencoder(s) may be updated based on the loss. For example, machine learning model system 102 may update (e.g., adjust) the parameters of the autoencoder(s) based on back propagation (e.g., of the loss(es)), gradient calculations (e.g., based on the loss(es)), any combination thereof, and / or the like.

[0097] In some non-limiting embodiments or aspects, training may further include repeating the training process (e.g., repeating inputting each respective data record of the tabular data to the encoder machine learning model(s), inputting the respective stochastic latent space embedding for each respective data record of the tabular data to the decoder machine learning model(s), determining the loss, and updating the parameters) until a termination condition is satisfied. For example, the termination condition may include a threshold level of accuracy, and the repetition may continue until the threshold level of accuracy is satisfied. Additionally or alternatively, the termination condition may include a target number of repetitions (e.g., a target number of epochs and / or the like), and the repetition may continue until the target number of repetitions is completed. In some non-limiting embodiments or aspects, there may be a plurality of termination conditions, and the repetition may continue until at least one of the plurality of termination conditions is satisfied.

[0098] As shown in FIG. 2, at step 208, method 200 may include generating at least one generated data record based on the stochastic latent space embedding(s) and feature types. For example, machine learning model system 102 may generate at least one generated data record (e.g., a plurality of generated data records) based on the stochastic latent space embedding for each data record and the feature type of each feature of each data record using at least one decoder machine learning model

[0099] In some non-limiting embodiments or aspects, the decoder machine learning model(s) may include a plurality of output distributions including a respective output distribution for each respective feature type of the plurality of feature types. Each generated data record may include a plurality of generated values associatedwith the plurality of features. For example, each generated value may be generated by an output distribution of the plurality of output distributions based on the feature type.

[0100] In some non-limiting embodiments or aspects, autoencoder(s) may include a plurality of MVAEs, as described herein. For example, generating the at least one generated data record may include generating the at least one generated data record based on the stochastic latent space embedding for each data record, the feature type of each feature of each data record, a first decoder machine learning model (e.g., a first decoder network) of the first MVAE (e.g., associated with IS), a second decoder machine learning model (e.g., a second decoder network) of the second MVAE (e.g., associated with IWS), and a third decoder machine learning model (e.g., a third decoder network) of the third MVAE (e.g., associated with DReG). For example, the generated data record(s) may include an ensemble (e.g., average, weighted average, majority vote, weighted vote, aggregation, any combination thereof, and / or the like) of the outputs of the three MVAEs (e.g., the output generated by each MVAE based on the stochastic latent space embedding(s), the feature type(s), and the output distribution(s)).

[0101] In some non-limiting embodiments or aspects, each of the first MVAE associated with IS, the second MVAE associated with IWS, and the third MVAE associated with DReG may perform better than one or more of the others in different situations. By using all three MVAEs (e.g., an ensemble of the outputs thereof), performance may be improved and / or may be robust in various different settings.

[0102] In some non-limiting embodiments or aspects, the plurality of feature types may include a count feature type, a real number feature type, and a categorical feature type. The plurality of output distributions may include a gamma distribution for the count feature type, a normal distribution for the real number feature type, and a multinomial distribution for the categorical feature type.

[0103] In some non-limiting embodiments or aspects, machine learning model system 102 may train at least one other machine learning model based on the at least one generated data record. Additionally or alternatively, machine learning model system 102 may perform at least one task (e.g., generate at least one prediction) based on the other machine learning model(s).

[0104] In some non-limiting embodiments or aspects, training may include inputting each respective generated data record to the other machine learning model togenerate a respective output (e.g., a respective prediction) based on each respective generated data record. A loss may be determined based on each respective output (e.g., each respective prediction). For example, the losses may be calculated based on a loss function based on the respective output, an error, a mean error, an MSE, any combination thereof, and / or the like. The parameters of the other machine learning model may be updated based on the loss. For example, machine learning model system 102 may update (e.g., adjust) the parameters of the other machine learning model based on back propagation (e.g., of the loss(es)), gradient calculations (e.g., based on the loss(es)), any combination thereof, and / or the like. In some non-limiting embodiments or aspects, training may further include repeating the training process (e.g., repeating inputting each respective generated data record, determining the loss, and updating the parameters) until a termination condition is satisfied. For example, the termination condition may include a threshold level of accuracy, and the repetition may continue until the threshold level of accuracy is satisfied. Additionally or alternatively, the termination condition may include a target number of repetitions (e.g., a target number of epochs and / or the like), and the repetition may continue until the target number of repetitions is completed. In some non-limiting embodiments or aspects, there may be a plurality of termination conditions, and the repetition may continue until at least one of the plurality of termination conditions is satisfied.

[0105] In some non-limiting embodiments or aspects, the tabular data may include transaction data associated with at least one payment transaction, and / or the data record(s) may include at least one transaction record. The other machine learning model(s) may include at least one of a fraud detection model, a risk detection model, a transaction authorization model, a credit approval model, a product recommendation model, a classifier model, an anomaly detection model, an authentication model, any combination thereof, and / or the like.

[0106] In some non-limiting embodiments or aspects, performing the task(s) may include generating at least one prediction associated with fraud detection, risk detection, transaction authorization, credit approval, product recommendation, classification, anomaly detection, authentication, any combination thereof, and / or the like (e.g., based on the fraud detection model, the risk detection model, the transaction authorization model, the credit approval model, the product recommendation model, the classifier model, the anomaly detection model, the authentication model, any combination thereof, and / or the like, respectively).

[0107] Referring now to FIG. 3, depicted is a diagram of an example payment processing network 300, according to non-limiting embodiments or aspects. In some non-limiting embodiments or aspects, payment processing network 300 may be used in conjunction with the systems, methods, and / or computer program products described herein, and / or the systems, methods, and / or computer program products described herein may be implemented in payment processing network 300. As shown in FIG. 3, payment processing network 300 may include transaction processing system 301 , payment gateway system 302, merchant system 304, issuer system 306, acquirer system 308, and / or consumer device 310. In some non-limiting embodiments or aspects, each of machine learning model system 102, data storage system 104, and / or user device 106 of FIG. 1 may be implemented by (e.g., part of) transaction processing system 301 . In some non-limiting embodiments or aspects, at least one of machine learning model system 102, data storage system 104, and / or user device 106 of FIG. 1 may be implemented by (e.g., part of) another system, another device, another group of systems, or another group of devices, separate from or including transaction processing system 301 , such as merchant system 304, issuer system 306, acquirer system 308, consumer device 310, and / or the like. For example, user device 106 may be implemented by (e.g., part of) at least one of payment gateway system 302, merchant system 304, issuer system 306, acquirer system 308, and / or consumer device 310. Additionally or alternatively, for example, data storage system 104 may be implemented by (e.g., part of) at least one of payment gateway system 302, merchant system 304, issuer system 306, acquirer system 308, and / or consumer device 310.

[0108] Transaction processing system 301 may include one or more devices capable of receiving information from and / or communicating information to payment gateway system 302, merchant system 304, issuer system 306, acquirer system 308, consumer device 310, and / or the like (e.g., directly, indirectly, via a public and / or private communication network connection, and / or the like). For example, as shown in FIG. 3, transaction processing system 301 may be in communication with one or more issuer systems (e.g., issuer system 306), one or more acquirer systems (e.g., acquirer system 308), and / or one or more payment gateway systems (e.g., payment gateway system 302). Although only a single issuer system 306, single acquirer system 308, and single payment gateway system 302 are shown, it will be appreciated that transaction processing system 301 may be in communication with a plurality ofissuer systems, a plurality of acquirer systems, and / or a plurality of payment gateways. In some non-limiting embodiments or aspects, transaction processing system 301 may include a computing device, such as a server (e.g., a transaction processing server), a group of servers, and / or other like devices. In some non-limiting embodiments or aspects, transaction processing system 301 may be in communication with a data storage device, which may be local or remote to transaction processing system 301. In some non-limiting embodiments or aspects, transaction processing system 301 may be capable of receiving information from, storing information in, communicating information to, or searching information stored in the data storage device. In some non-limiting embodiments or aspects, transaction processing system 301 may be associated with a transaction service provider, as described herein. In some nonlimiting embodiments or aspects, transaction processing system 301 may also operate as an issuer system such that both transaction processing system 301 and issuer system 306 are a single system and / or controlled by a single entity.

[0109] Payment gateway system 302 may include one or more devices capable of receiving information from and / or communicating information to transaction processing system 301 , merchant system 304, issuer system 306, acquirer system 308, consumer device 310, and / or the like (e.g., directly, indirectly, via a public and / or private communication network connection, and / or the like). For example, as shown in FIG. 3, payment gateway system 302 may be in communication with one or more merchant systems (e.g., merchant system 304), one or more acquirer systems (e.g., acquirer system 308), and / or one or more transaction processing systems (e.g., transaction processing system 301 ). Although only a single merchant system 304, single acquirer system 308, and single transaction processing system 301 are shown, it will be appreciated that payment gateway system 302 may be in communication with a plurality of merchant systems, a plurality of acquirer systems, and / or a plurality of transaction processing systems. In some non-limiting embodiments or aspects, payment gateway system 302 may include a computing device, such as a server, a group of servers, and / or other like devices. In some non-limiting embodiments or aspects, payment gateway system 302 may be associated with a payment gateway, as described herein.

[0110] Merchant system 304 may include one or more devices capable of receiving information from and / or communicating information to transaction processing system 301 , payment gateway system 302, issuer system 306, acquirer system 308,consumer device 310, and / or the like (e.g., directly, indirectly, via a public and / or private communication network connection, and / or the like). For example, as shown in FIG. 3, merchant system 304 may be in communication with one or more payment gateway systems (e.g., payment gateway system 302), one or more acquirer systems (e.g., acquirer system 308), and / or one or more consumer devices (e.g., consumer device 310). Although only a single payment gateway system 302, single acquirer system 308, and single consumer device 310 are shown, it will be appreciated that merchant system 304 may be in communication with a plurality of payment gateway systems, a plurality of acquirer systems, and / or a plurality of consumer devices. In some non-limiting embodiments or aspects, merchant system 304 may include a computing device, such as a server, a group of servers, a client device, a group of client devices, a POS device, a POS system, computers, computer systems, peripheral devices, and / or other like devices. In some non-limiting embodiments or aspects, merchant system 304 may be associated with a merchant, as described herein. In some non-limiting embodiments or aspects, merchant system 304 may include a device capable of receiving information from and / or communicating information to consumer device 310 via a short range communication connection (e.g., an NFC communication connection, an RFID communication connection, a Bluetooth® communication connection, a Zigbee® communication connection, and / or the like) with consumer device 310 and / or the like. In some non-limiting embodiments or aspects, merchant system 304 may include one or more client devices. For example, merchant system 304 may include a client device that allows a merchant to communicate information to transaction processing system 301 (e.g., via at least one of acquirer system 308 and / or payment gateway system 302). In some non-limiting embodiments or aspects, merchant system 304 (e.g., a client device thereof, a POS device thereof, and / or the like) may also operate as a payment gateway system such that both merchant system 304 and payment gateway system 302 are a single system and / or controlled by a single entity.

[0111] Issuer system 306 may include one or more devices capable of receiving information and / or communicating information to transaction processing system 301 , payment gateway system 302, merchant system 304, acquirer system 308, consumer device 310, and / or the like (e.g., directly, indirectly, via a public and / or private communication network connection, and / or the like). For example, as shown in FIG. 3, issuer system 306 may be in communication with one or more transactionprocessing systems (e.g., transaction processing system 301 ) and / or one or more consumer devices (e.g., consumer device 310). Although only a single transaction processing system 301 and a single consumer device 310 are shown, it will be appreciated that issuer system 306 may be in communication with a plurality of transaction processing systems and / or a plurality of consumer devices 310. In some non-limiting embodiments or aspects, issuer system 306 may include a computing device, such as a server, a group of servers, and / or other like devices. In some nonlimiting embodiments or aspects, issuer system 306 may be associated with an issuer institution, as described herein. For example, issuer system 306 may be associated with an issuer institution that issued a credit account, debit account, credit card, debit card, a payment device, and / or the like to a user associated with consumer device 310.

[0112] Acquirer system 308 may include one or more devices capable of receiving information from and / or communicating information to transaction processing system 301 , payment gateway system 302, merchant system 304, issuer system 306, consumer device 310, and / or the like (e.g., directly, indirectly, via a public and / or private communication network connection, and / or the like). For example, as shown in FIG. 3, acquirer system 308 may be in communication with one or more transaction processing systems (e.g., transaction processing system 301 ), one or more payment gateway systems (e.g., payment gateway system 302), and / or one or more merchant systems (e.g., merchant system 304). Although only a single transaction processing system 301 , a single payment gateway system 302, and a single merchant system 304 are shown, it will be appreciated that acquirer system 308 may be in communication with a plurality of transaction processing systems, a plurality of payment gateway systems, and / or a plurality of merchant systems. In some nonlimiting embodiments or aspects, acquirer system 308 may include a computing device, such as a server, a group of servers, and / or other like devices. In some nonlimiting embodiments or aspects, acquirer system 308 may be associated with an acquirer institution, as described herein.

[0113] Consumer device 310 may include one or more devices capable of receiving information from and / or communicating information to transaction processing system 301 , payment gateway system 302, merchant system 304, issuer system 306, acquirer system 308, and / or the like (e.g., directly, indirectly, via a public and / or private communication network connection, and / or the like). For example, as shown in FIG.3, consumer device 310 may be in communication with one or more merchant systems (e.g., merchant system 304) and / or one or more issuer systems (e.g., issuer system 306). Although only a single merchant system 304 and a single issuer system 306 are shown, it will be appreciated that consumer device 310 may be in communication with a plurality of merchant systems and / or a plurality of issuer systems. In some nonlimiting embodiments or aspects, consumer device 310 may be associated with a user to whom a credit account, a debit account, a credit card, a debit card, a payment device, and / or the like has been issued. In some non-limiting embodiments or aspects, user device 310 may include a computing device, such as a computer, a portable computer, a laptop computer, a tablet computers, a mobile device, a cellular phone, a smartphone, a wearable device (e.g., watches, glasses, lenses, clothing, and / or the like), a PDA, a client device, and / or other like devices. In some non-limiting embodiments or aspects, user device 310 may include a payment device, as described herein. In some non-limiting embodiments or aspects, consumer device 310 may include a device capable of receiving information from and / or communicating information to other customer devices 310 (e.g., directly, indirectly, via a public and / or private communication network connection, a short range communication connection, and / or the like). In some non-limiting embodiments or aspects, consumer device 310 may include a device capable of receiving information from and / or communicating information to merchant system 304 via a short range communication connection (e.g., an NFC communication connection, an RFID communication connection, a Bluetooth® communication connection, a Zigbee® communication connection, and / or the like) with merchant system 304 and / or the like. In some non-limiting embodiments or aspects, consumer device 310 may include a client device.

[0114] In some non-limiting embodiments or aspects, transaction processing system 301 may communicate with merchant system 304 directly (e.g., via a public and / or private communication network connection and / or the like). Additionally or alternatively, transaction processing system 301 may communicate with merchant system 304 through payment gateway 302 and / or acquirer system 308. In some nonlimiting embodiments or aspects, an acquirer system 308 associated with merchant system 304 may operate as payment gateway 302 to facilitate the communication of transaction messages (e.g., authorization requests) from merchant system 304 to transaction processing system 301. In some non-limiting embodiments or aspects, merchant system 304 may communicate with payment gateway 302 directly (e.g., viaa public and / or private communication network connection and / or the like). For example, a merchant system 304 that includes a physical POS device may communicate with payment gateway 302 through a public or private network to conduct card-present transactions. As another example, a merchant system 304 that includes a server (e.g., a web server) may communicate with payment gateway 302 through a public or private network, such as the Internet, to conduct card-not-present transactions.

[0115] For the purpose of illustration, processing a transaction (e.g., a payment transaction) may include generating a transaction message (e.g., authorization request and / or the like) based on an account identifier of a customer (e.g., accountholder associated with customer device 310 and / or the like) and / or transaction data associated with the transaction. For example, merchant system 304 (e.g., a client device of merchant system 304, a POS device of merchant system 304, and / or the like) may initiate the transaction, e.g., by generating an authorization request (e.g., in response to receiving the account identifier from a payment device and / or a portable financial device of the customer and / or the like). Merchant system 304 may communicate the authorization request to payment gateway 302 and / or acquirer system 308. In some non-limiting embodiments or aspects, payment gateway 302 may communicate the authorization request to acquirer system 308 and / or transaction processing system 301. Additionally or alternatively, acquirer system 308 (and / or payment gateway 302) may communicate the authorization request to transaction processing system 301. After receiving the authorization request from merchant system 304 that identifies the account identifier of the customer (e.g., the accountholder associated with consumer device 310 and / or the account identifier), transaction processing system 301 may communicate the authorization request to issuer system 306 (e.g., the issuer system that issued the payment device and / or account identifier). Issuer system 306 may determine an authorization decision (e.g., approve, deny, and / or the like) based on the authorization request, and / or issuer system 306 may generate an authorization response based on the authorization decision and / or the authorization request. Issuer system 306 may communicate the authorization response to transaction processing system 301 . Transaction processing system 301 may communicate the authorization response to acquirer system 308 and / or payment gateway 302. In some non-limiting embodiments or aspects, acquirer system 308 may communicate the authorization response to payment gateway 302and / or merchant system 304. Additionally or alternatively, payment gateway 302 (and / or acquirer system 308) may communicate the authorization response to merchant system 304.

[0116] In some non-limiting embodiments or aspects, transaction processing system 301 and / or issuer system 306 may include at least one machine learning model (e.g., at least one of a fraud detection model, a risk detection model, a transaction authorization model, a credit approval model, a product recommendation model, a classifier model, an anomaly detection model, an authentication model, any combination thereof, and / or the like). For example, the machine learning model(s) may be trained based on generated data records, as described herein. Transaction processing system 301 and / or issuer system 306 may perform at least one task (e.g., generate at least one prediction) based on the authorization request and the machine learning model(s). For example, performing the task(s) may include generating at least one prediction associated with fraud detection, risk detection, transaction authorization, credit approval, product recommendation, classification, anomaly detection, authentication, any combination thereof, and / or the like. In some nonlimiting embodiments or aspects, transaction processing system 301 may communicate at least one message based on performing the task (e.g., generating the prediction) to issuer system 306 (e.g., along with the authorization request). In some non-limiting embodiments or aspects, issuer system 306 may determine the authorization decision (e.g., approve, deny, and / or the like) based on the authorization request and the performance of the task (e.g., generation of the prediction).

[0117] For the purpose of illustration, clearing and / or settlement of a transaction may include generating a message (e.g., clearing message and / or the like) based on an account identifier of a customer (e.g., associated with customer device 310 and / or the like) and / or transaction data associated with the transaction. For example, merchant system 304 may generate at least one clearing message (e.g., a plurality of clearing messages, a batch of clearing messages, and / or the like). Merchant system 304 may communicate the clearing message(s) to acquirer system 308 (and / or payment gateway 302, which may communicate the clearing message(s) to acquirer system 308). Acquirer system 308 may communicate the clearing message(s) to transaction processing system 301. Transaction processing system 301 may communicate the clearing message(s) to issuer system 306. Issuer system 306 may generate at least one settlement message based on the clearing message(s). In somenon-limiting embodiments or aspects, issuer system 306 may communicate the settlement message(s) and / or funds to transaction processing system 301 (and / or a settlement bank system associated with transaction processing system 301 ), and transaction processing system 301 (and / or the settlement bank system) may communicate the settlement message(s) and / or funds to acquirer system 308. Additionally or alternatively, issuer system 306 may communicate the settlement message(s) and / or funds to acquirer system 308. In some non-limiting embodiments or aspects, acquirer system 308 may communicate settlement message(s) and / or funds to merchant system 304 (and / or an account associated with merchant system 304).

[0118] The systems and / or devices of FIG. 3 may communicate via one or more wired and / or wireless communication networks. For example, the communication network(s) may include a cellular network (e.g., a long-term evolution (LTE®) network, a third generation (3G) network, a fourth generation (4G) network, a fifth generation (5G) network, a code division multiple access (CDMA) network, and / or the like), a public land mobile network (PLMN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a telephone network (e.g., the public switched telephone network (PSTN)), a private network (e.g., a private network associated with a transaction service provider), an ad hoc network, an intranet, the Internet, a fiber optic-based network, a cloud computing network, and / or the like, and / or a combination of these or other types of networks.

[0119] The number and arrangement of systems, devices, and / or networks shown in FIG. 3 are provided as an example. There may be additional systems, devices, and / or networks; fewer systems, devices, and / or networks; different systems, devices, and / or networks; and / or differently arranged systems, devices, and / or networks than those shown in FIG. 3. Furthermore, two or more systems or devices shown in FIG. 3 may be implemented within a single system or device, or a single system or device shown in FIG. 3 may be implemented as multiple, distributed systems or devices. Additionally or alternatively, a set of systems (e.g., one or more systems) or a set of devices (e.g., one or more devices) of payment processing network 300 may perform one or more functions described as being performed by another set of systems or another set of devices of payment processing network 300.

[0120] Referring now to FIG. 4, shown is a diagram of example components of a device 400 according to non-limiting embodiments. Device 400 may correspond tomachine learning model system 102, data storage system 104, and / or user device 106 of FIG. 1 and / or transaction processing system 301 , payment gateway system 302, merchant system 304, issuer system 306, acquirer system 308, and / or consumer device 310 of FIG. 3, as an example. In some non-limiting embodiments, such systems or devices may include at least one device 400 and / or at least one component of device 400. The number and arrangement of components shown are provided as an example. In some non-limiting embodiments, device 400 may include additional components, fewer components, different components, or differently arranged components than those shown. Additionally or alternatively, a set of components (e.g., one or more components) of device 400 may perform one or more functions described as being performed by another set of components of device 400.

[0121] As shown in FIG. 4, device 400 may include bus 402, processor 404, memory 406, storage component 408, input component 410, output component 412, and communication interface 414. Bus 402 may include a component that permits communication among the components of device 400. In some non-limiting embodiments, processor 404 may be implemented in hardware, firmware, or a combination of hardware and software. For example, processor 404 may include a processor (e.g., a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), etc.), a microprocessor, a digital signal processor (DSP), and / or any processing component (e.g., a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), etc.) that can be programmed to perform a function. Memory 406 may include random access memory (RAM), read only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic memory, optical memory, etc.) that stores information and / or instructions for use by processor 404.

[0122] With continued reference to FIG. 4, storage component 408 may store information and / or software related to the operation and use of device 400. For example, storage component 408 may include a hard disk (e.g., a magnetic disk, an optical disk, a magneto-optic disk, a solid-state disk, etc.) and / or another type of computer-readable medium. Input component 410 may include a component that permits device 400 to receive information, such as via user input (e.g., a touch screen display, a keyboard, a keypad, a mouse, a button, a switch, a microphone, etc.). Additionally or alternatively, input component 410 may include a sensor for sensing information (e.g., a global positioning system (GPS) component, an accelerometer, agyroscope, an actuator, etc.). Output component 412 may include a component that provides output information from device 400 (e.g., a display, a speaker, one or more light-emitting diodes (LEDs), etc.). Communication interface 414 may include a transceiver-like component (e.g., a transceiver, a separate receiver and transmitter, etc.) that enables device 400 to communicate with other devices, such as via a wired connection, a wireless connection, or a combination of wired and wireless connections. Communication interface 414 may permit device 400 to receive information from another device and / or provide information to another device. For example, communication interface 414 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi® interface, a cellular network interface, and / or the like.

[0123] Device 400 may perform one or more processes described herein. Device 400 may perform these processes based on processor 404 executing software instructions stored by a computer-readable medium, such as memory 406 and / or storage component 408. A computer-readable medium may include any non-transitory memory device. A memory device includes memory space located inside of a single physical storage device or memory space spread across multiple physical storage devices. Software instructions may be read into memory 406 and / or storage component 408 from another computer-readable medium or from another device via communication interface 414. When executed, software instructions stored in memory 406 and / or storage component 408 may cause processor 404 to perform one or more processes described herein. Additionally or alternatively, hardwired circuitry may be used in place of or in combination with software instructions to perform one or more processes described herein. Thus, embodiments described herein are not limited to any specific combination of hardware circuitry and software. The term “configured to,” as used herein, may refer to an arrangement of software, device(s), and / or hardware for performing and / or enabling one or more functions (e.g., actions, processes, steps of a process, and / or the like). For example, “a processor configured to” may refer to a processor that executes software instructions (e.g., program code) that cause the processor to perform one or more functions.

[0124] Referring now to FIG. 5, shown is a schematic diagrams of an example implementation of a machine learning model 500 for generating tabular data, according to some non-limiting embodiments or aspects. The number andarrangement of components shown in FIG. 5 are provided as an example. There may be additional components, fewer components, different components, and / or differently arranged components than those shown in FIG. 5. In some non-limiting embodiments or aspects, machine learning model 500 may be implemented (e.g., completely, partially, and / or the like) by machine learning model system 102 (e.g., one or more devices of machine learning model system 102). In some non-limiting embodiments or aspects, machine learning model 500 may be implemented (e.g., completely, partially, and / or the like) by another system, another device, another group of systems, or another group of devices, separate from or including machine learning model system 102, such as data storage system 104 and / or user device 106. For example, machine learning model 500 (and / or the parameters and / or hyperparameters thereof) may be stored (e.g., completely, partially, and / or the like) by data storage system 104.

[0125] Machine learning model 500 may include encoder 510 (e.g., an encoder machine learning model, an encoder network, and / or the like), latent space 520, and decoder 530 (e.g., a decoder machine learning model, a decoder network, and / or the like), as described herein. In some non-limiting embodiments or aspects, machine learning model 500 (e.g., encoder 510) may receive multi-modal input data 512 as input. For example, multi-modal input data 512 may include tabular data associated with at least one data record (e.g., a plurality of data records) including values associated features of a plurality of feature types (e.g., a plurality of modes), as described herein. In some non-limiting embodiments or aspects, machine learning model 500 (e.g., decoder 510) may generate reconstructed data 538. For example, reconstructed data 538 may include at least one generated data record, as described herein.

[0126] In some non-limiting embodiments or aspects, encoder 510 may include a plurality of hidden layers (e.g., first hidden layer 514-1 and second hidden layer 514- 2, individually referred to as “hidden layer 514”, and collectively referred to as “hidden layers 514”). Each hidden layer 514 may include a plurality of nodes (e.g., neurons, perceptrons, computational units, and / or the like). As shown in FIG. 5, each circle may represent a node of the respective hidden layer 514, and arrows connecting the circles may represent connections between (e.g., weights connecting) nodes. The number of nodes for each hidden layer 514 may be the same as, greater than, or less than the number of feature of multi-modal input data 512 and / or the number of nodes of each other hidden layer 514. In some non-limiting embodiments or aspects, multi-modalinput data 512 may be connected to first hidden layer 514-1. In some non-limiting embodiments or aspects, the nodes of each hidden layer 514 may be connected to the nodes of a subsequent hidden layer 514 (e.g., the nodes of first hidden layer 514- 1 may be connected to the nodes of second hidden layer 514-2). In some non-limiting embodiments or aspects, the intermediate outputs of the last hidden layer (e.g., second hidden layer 514-2) may be associated with mean 516 and variance 518 of each dimension of a plurality of dimensions of latent space 520. In some non-limiting embodiments or aspects, stochastic latent space embedding 522 may be associated with mean 516 and a variance 518 of each dimension of a plurality of dimensions of latent space 520, as described herein.

[0127] In some non-limiting embodiments or aspects, decoder 530 may include a plurality of hidden layers (e.g., first hidden layer 532-1 and second hidden layer 532- 2, individually referred to as “hidden layer 532”, and collectively referred to as “hidden layers 532”). As shown in FIG. 5, each circle may represent a node of the respective hidden layer 532, and arrows connecting the circles may represent connections between (e.g., weights connecting) nodes. In some non-limiting embodiments or aspects, stochastic latent space embedding 522 may be connected to the hidden layers (e.g., first hidden layer 532-1 ) of decoder 530. In some non-limiting embodiments or aspects, the nodes of each hidden layer 532 may be connected to the nodes of a subsequent hidden layer 532 (e.g., the nodes of first hidden layer 532- 1 may be connected to the nodes of second hidden layer 532-2). In some non-limiting embodiments or aspects, the intermediate outputs of the last hidden layer (e.g., second hidden layer 532-2) may be connected to distribution-aware parameter learning (sub-)system 534.

[0128] In some non-limiting embodiments or aspects, distribution-aware parameter learning (sub-)system 534 may include a plurality of output distributions (e.g., first output distribution 536-1 , second output distribution 536-2, and third output distribution 536-3, individually referred to as “output distribution 536”, and collectively referred to as “output distributions 536”) including a respective output distribution 536 for each respective feature type of the plurality of feature types, as described herein. Each generated data record of reconstructed data 538 (e.g., each generated data record) may include a plurality of generated values associated with the plurality of features, and each generated value may be generated by one of the output distributions based on the feature type, as described herein. For example, the plurality of feature types(e.g., of multi-modal data 512 and / or reconstructed data 538) may include a count feature type (“count”), a real number feature type (“real”), and a categorical feature type (“categ”), and the plurality of output distributions may include a gamma distribution (e.g., first output distribution 536-1 ) for the count feature type, a normal distribution (e.g., second output distribution 536-2) for the real number feature type, and a multinomial distribution (e.g., third output distribution 536-3) for the categorical feature type, as described herein.

[0129] Referring now to FIGS. 6A-6D, shown are bar charts of actual and generated distributions of tabular data based on an example implementation of a method for improved machine learning models for generating tabular data, according to some non-limiting embodiments or aspects.

[0130] For the purpose of illustration, as shown in FIGS. 6A-6D, tabular data may include data from the Bank Marketing dataset (Bank Marketing, UC Irvine Machine Learning Repository (2012), https: / / archive.ics.uci.edu / dataset / 222 / bank-i-marketing), which is incorporated by reference herein in its entirety. The tabular data may include a plurality of features of a plurality of feature types including a count / integer feature type, a categorical feature type, a binary feature type, a date feature type, etc. For example, the feature “education” may be a categorical feature associated with education level with seven possible values (e.g., “basic.4y”, “basic.6y”, “basic.9y”, “high. school”, “illiterate”, “professional. course”, “university.degree”). An eighth possible value (e.g., “unknown”) may be associated with mask data (e.g., missing data or data being masked, as described herein). Additionally, the feature “age” may be a count / integer feature associated with the age of an individual. Also, “y” may be a binary feature associated with a classification to be predicted (e.g., a target classification), with binary values associated with whether an individual subscribed a term deposit or not.

[0131] FIGS. 6A and 6B show the distributions of the “age” feature and the “education” feature, respectively, for the original data and for generated data that was generated based on a VAE. FIGS. 6C and 6D show the distributions of the “age” feature and the “education” feature, respectively, for the original data and for generated data that was generated based on techniques described herein. As shown in FIGS. 6A-6D, the distributions of generated data that was generated based on techniques described herein more closely resembles the distributions of the original data than the distributions of generated data that was generated based on a VAE.

[0132] Table 1 shows accuracy metrics (e.g., area under the receiver operating characteristic curve (“ROC-AUC”), area under the precision recall curve (“Pr-AUC”), Precision (“Prec”), and recall (“Recall”). The “non-imputed” metrics represent the accuracy of classification (e.g., prediction of the target “y”) based on the original data. The “imputed” metrics represent the accuracy of classification (e.g., prediction of the target “y”) when the techniques described herein are used to generate values for missing (e.g., masked) data (e.g., data imputation by generating generated data records including values for the masked / missing data). As shown in Table 1 , using the techniques described herein to generate missing values (e.g., data imputation) before inputting the full data (e.g., the original data with the generated values inserted for the missing / mased values) improves accuracy of the classification.Table 1

[0133] Although embodiments have been described in detail for the purpose of illustration, it is to be understood that such detail is solely for that purpose and that the disclosure is not limited to the disclosed embodiments or aspects, but, on the contrary, is intended to cover modifications and equivalent arrangements that are within the spirit and scope of the appended claims. For example, it is to be understood that the present disclosure contemplates that, to the extent possible, one or more features of any embodiment or aspect can be combined with one or more features of any other embodiment or aspect.

Claims

WHAT IS CLAIMED IS:1 . A system, comprising: at least one processor configured to: receive tabular data associated with at least one data record, each data record of the at least one data record comprising a plurality of values associated with a plurality of features, each respective value of the plurality of values associated with a respective feature of the plurality of features, each feature of the plurality of features having a feature type of a plurality of feature types; receive mask data associated with masking at least one value of the plurality of values; generate a stochastic latent space embedding for each data record based on the tabular data and the mask data using at least one encoder machine learning model, the stochastic latent space embedding associated with a mean and a variance of each dimension of a plurality of dimensions of a latent space; and generate at least one generated data record based on the stochastic latent space embedding for each data record and the feature type of each feature of each data record using at least one decoder machine learning model comprising a plurality of output distributions comprising a respective output distribution for each respective feature type of the plurality of feature types, each generated data record of the at least one generated data record comprising a plurality of generated values associated with the plurality of features, each generated value of the plurality of generated values generated by an output distribution of the plurality of output distributions based on the feature type.

2. The system of claim 1 , wherein the at least one data record comprises a plurality of data records, wherein the mask data is associated with masking at least one respective value of the plurality of values for each respective data record of the plurality of data records.

3. The system of claim 2, wherein generating the stochastic latent space embedding for each data record comprises generating a plurality of a stochastic latent space embeddings comprising a respective stochastic latent space embedding for each respective data record of the plurality of data records using the at least one encoder machine learning model.

4. The system of claim 1 , wherein at least one autoencoder comprises the at least one encoder machine learning model and the at least one decoder machine learning model.

5. The system of claim 4, wherein the at least one autoencoder comprises at least one of a variational autoencoder (VAE), a beta VAE ([3-VAE), a multi-modal VAE (MVAE), an importance weighted autoencoder (IWAE), or any combination thereof.

6. The system of claim 5, wherein the at least one autoencoder comprises a plurality of MVAEs comprising a first MVAE associated with an importance-sampled (IS) evidence lower bound (ELBO), a second MVAE associated with importance weighted sampling (IWS), and a third MVAE associated with a doubly reparametrised gradient estimator (DReG), wherein generating the stochastic latent space embedding for each data record comprises generating the stochastic latent space embedding for each data record based on the tabular data, the mask data, a first encoder machine learning model of the first MVAE, a second encoder machine learning model of the second MVAE, and a third encoder machine learning model of the third MVAE, and wherein generating the at least one generated data record comprises generating the at least one generated data record based on the stochastic latent space embedding for each data record, the feature type of each feature of each data record, a first decoder machine learning model of the first MVAE, a second decoder machine learning model of the second MVAE, and a third decoder machine learning model of the third MVAE.

7. The system of claim 4, wherein the at least one processor is further configured to:train the at least one autoencoder based on the tabular data, wherein training comprises: inputting each respective data record of the tabular data to the at least one encoder machine learning model to generate a respective stochastic latent space embedding; inputting the respective stochastic latent space embedding for each respective data record of the tabular data to the at least one decoder machine learning model to generate a respective reconstructed data record; determining a loss based on each respective data record of the tabular data and the respective reconstructed data record; and updating parameters of the at least one autoencoder based on the loss.

8. The system of claim 7, wherein training further comprises repeating inputting each respective data record of the tabular data to the at least one encoder machine learning model, inputting the respective stochastic latent space embedding for each respective data record of the tabular data to the at least one decoder machine learning model, determining the loss, and updating the parameters of the at least one autoencoder until a termination condition is satisfied.

9. The system of claim 1 , wherein the plurality of feature types comprises a count feature type, a real number feature type, and a categorical feature type, wherein the plurality of output distributions comprises a gamma distribution for the count feature type, a normal distribution for the real number feature type, and a multinomial distribution for the categorical feature type.

10. The system of claim 1 , wherein the at least one processor is further configured to: train at least one other machine learning model based on the at least one generated data record; and generate at least one prediction based on the at least one other machine learning model.1 1. The system of claim 10, wherein the tabular data comprises transaction data associated with at least one payment transaction, the at least one data record comprises at least one transaction record, and wherein the at least one other machine learning model comprises at least one of a fraud detection model, a risk detection model, a transaction authorization model, a credit approval model, a product recommendation model, a classifier model, an anomaly detection model, an authentication model, or any combination thereof.

12. A computer-implemented method, comprising: receiving, with at least one processor, tabular data associated with at least one data record, each data record of the at least one data record comprising a plurality of values associated with a plurality of features, each respective value of the plurality of values associated with a respective feature of the plurality of features, each feature of the plurality of features having a feature type of a plurality of feature types; receiving, with at least one processor, mask data associated with masking at least one value of the plurality of values; generating, with at least one processor, a stochastic latent space embedding for each data record based on the tabular data and the mask data using at least one encoder machine learning model, the stochastic latent space embedding associated with a mean and a variance of each dimension of a plurality of dimensions of a latent space; and generating, with at least one processor, at least one generated data record based on the stochastic latent space embedding for each data record and the feature type of each feature of each data record using at least one decoder machine learning model comprising a plurality of output distributions comprising a respective output distribution for each respective feature type of the plurality of feature types, each generated data record of the at least one generated data record comprising a plurality of generated values associated with the plurality of features, each generated value of the plurality of generated values generated by an output distribution of the plurality of output distributions based on the feature type.

13. The method of claim 12, wherein the at least one data record comprises a plurality of data records, wherein the mask data is associated with masking at least one respective value of the plurality of values for each respectivedata record of the plurality of data records, wherein generating the stochastic latent space embedding for each data record comprises generating a plurality of a stochastic latent space embeddings comprising a respective stochastic latent space embedding for each respective data record of the plurality of data records using the at least one encoder machine learning model.

14. The method of claim 12, wherein at least one autoencoder comprises the at least one encoder machine learning model and the at least one decoder machine learning model.

15. The method of claim 14, wherein the at least one autoencoder comprises at least one of a variational autoencoder (VAE), a beta VAE ([3-VAE), a multi-modal VAE (MVAE), an importance weighted autoencoder (IWAE), or any combination thereof.

16. The method of claim 15, wherein the at least one autoencoder comprises a plurality of MVAEs comprising a first MVAE associated with an importance-sampled (IS) evidence lower bound (ELBO), a second MVAE associated with importance weighted sampling (IWS), and a third MVAE associated with a doubly reparametrised gradient estimator (DReG), wherein generating the stochastic latent space embedding for each data record comprises generating the stochastic latent space embedding for each data record based on the tabular data, the mask data, a first encoder machine learning model of the first MVAE, a second encoder machine learning model of the second MVAE, and a third encoder machine learning model of the third MVAE, and wherein generating the at least one generated data record comprises generating the at least one generated data record based on the stochastic latent space embedding for each data record, the feature type of each feature of each data record, a first decoder machine learning model of the first MVAE, a second decoder machine learning model of the second MVAE, and a third decoder machine learning model of the third MVAE.

17. The method of claim 14, further comprising: training, with at least one processor, the at least one autoencoder based on the tabular data, wherein training comprises: inputting each respective data record of the tabular data to the at least one encoder machine learning model to generate a respective stochastic latent space embedding; inputting the respective stochastic latent space embedding for each respective data record of the tabular data to the at least one decoder machine learning model to generate a respective reconstructed data record; determining a loss based on each respective data record of the tabular data and the respective reconstructed data record; updating parameters of the at least one autoencoder based on the loss; and repeating inputting each respective data record of the tabular data to the at least one encoder machine learning model, inputting the respective stochastic latent space embedding for each respective data record of the tabular data to the at least one decoder machine learning model, determining the loss, and updating the parameters of the at least one autoencoder until a termination condition is satisfied.

18. The method of claim 12, wherein the plurality of feature types comprises a count feature type, a real number feature type, and a categorical feature type, wherein the plurality of output distributions comprises a gamma distribution for the count feature type, a normal distribution for the real number feature type, and a multinomial distribution for the categorical feature type.

19. The method of claim 12, further comprising: training, with at least one processor, at least one other machine learning model based on the at least one generated data record; and generating, with at least one processor, at least one prediction based on the at least one other machine learning model, wherein the tabular data comprises transaction data associated with at least one payment transaction, the at least one data record comprises at least one transaction record, and wherein the at least one other machine learning modelcomprises at least one of a fraud detection model, a risk detection model, a transaction authorization model, a credit approval model, a product recommendation model, a classifier model, an anomaly detection model, an authentication model, or any combination thereof.

20. A computer program product comprising at least one non- transitory computer-readable medium including program instructions that, when executed by at least one processor, cause the at least one processor to: receive tabular data associated with at least one data record, each data record of the at least one data record comprising a plurality of values associated with a plurality of features, each respective value of the plurality of values associated with a respective feature of the plurality of features, each feature of the plurality of features having a feature type of a plurality of feature types; receive mask data associated with masking at least one value of the plurality of values; generate a stochastic latent space embedding for each data record based on the tabular data and the mask data using at least one encoder machine learning model, the stochastic latent space embedding associated with a mean and a variance of each dimension of a plurality of dimensions of a latent space; and generate at least one generated data record based on the stochastic latent space embedding for each data record and the feature type of each feature of each data record using at least one decoder machine learning model comprising a plurality of output distributions comprising a respective output distribution for each respective feature type of the plurality of feature types, each generated data record of the at least one generated data record comprising a plurality of generated values associated with the plurality of features, each generated value of the plurality of generated values generated by an output distribution of the plurality of output distributions based on the feature type.

Images