Systems, methods, and computer program products for encoding feature interactions based on tabular data using machine learning

Abstract

Methods, systems, and computer program products are provided for encoding feature interactions based on tabular data. An example method includes receiving a data set in a table format including a plurality of rows and a plurality of columns. Each column is indexed to generate a position embedding matrix. Each column is grouped based on at least one tree model to generate a domain embedding matrix. An input vector is generated based on the data set, the position embedding matrix, and the domain embedding matrix. The input vector is input into a first multilayer perceptron (MLP) model to generate a first output vector, and the first output vector is transposed to generate a transposed vector. The transposed vector is input into a second MLP model to generate a second output vector, and the second output vector is input into at least one classifier model to generate at least one prediction.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 345,599, filed on May 25, 2022, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] The present disclosure generally relates to encoding features using machine learning models and, in some non-limiting embodiments or aspects, to systems, methods, and computer program products for encoding feature interactions based on tabular data and machine learning models. Background Art

[0004] Tabular data (e.g., table, data sheet, tabular data set, etc.) may include an arrangement of data (e.g., information) in the form of rows and / or columns including elements (e.g., values). Rows of tabular data may be ordered or unordered. Columns of tabular data may include identifiers, such as the names of fields (e.g., parameters, features, etc.) that may apply to each element of the column.

[0005] Tabular data can be used to analyze data using machine learning techniques. Due to the structure and organization of data in a tabular format, some systems can use machine learning models to learn tabular data. For example, deep learning models (such as deep neural networks (DNNs)) can be trained using tabular data to learn features and make predictions.

[0006] However, systems using machine learning become difficult to scale to analyze tabular data with thousands of features including millions of data instances. When the entire tabular data set is stored in the memory of a computing device, the number of data instances and features in the data table may cause a computational bottleneck. Additionally, deep learning models used to learn features in tabular data may not scale to large tabular data sets and may result in poor performance. In some cases, systems with poor performance may not be sufficient to train machine learning models on the fly and / or meet the requirements of online services, such as latency or memory requirements. Summary of the invention

[0007] Accordingly, improved systems, methods, and computer program products for encoding feature interactions based on tabular data and machine learning are provided that overcome some or all of the deficiencies identified above.

[0008] According to non-limiting embodiments or aspects, a computer-implemented method for encoding feature interactions based on tabular data is provided. In some non-limiting embodiments or aspects, the computer-implemented method may include receiving a data set in a tabular format including a plurality of rows and a plurality of columns. Each row in the plurality of rows may represent a corresponding data instance in a plurality of data instances. Each column in the plurality of columns may represent a corresponding feature in a plurality of features. Each data instance in the plurality of data instances may include a plurality of values, the plurality of values ​​including a corresponding value associated with each corresponding feature in the plurality of features. The computer-implemented method may also include indexing each of the plurality of columns to generate a position embedding matrix including a plurality of position embedding vectors. Each position embedding matrix row in the position embedding matrix may include a corresponding position embedding vector associated with a corresponding column in the plurality of columns in the plurality of positions embedding vectors. The computer-implemented method may also include grouping each of the plurality of columns based on at least one tree model to generate a domain embedding matrix including a plurality of domain embedding vectors. The computer-implemented method may also include generating an input vector based on the data set, the position embedding matrix, and the domain embedding matrix. The computer-implemented method may also include inputting the input vector into a first multilayer perceptron (MLP) model to generate a first output vector. The computer-implemented method may also include transposing the first output vector to generate a transposed vector. The computer-implemented method may also include inputting the transposed vector into a second MLP model to generate a second output vector. The computer-implemented method may also include inputting the second output vector into at least one classifier model to generate at least one prediction.

[0009] In some non-limiting embodiments or aspects, at least one prediction can include at least one predicted label.

[0010] In some non-limiting embodiments or aspects, the plurality of data instances may include a plurality of payment transaction records. In some non-limiting embodiments or aspects, at least one predicted tag may indicate that a corresponding payment transaction record in the plurality of payment transaction records is predicted to be fraudulent.

[0011] In some non-limiting embodiments or aspects, generating the input vector may include concatenating at least one row of the data set, at least one position embedding vector of the position embedding matrix, and at least one domain embedding vector of the domain embedding matrix to produce the input vector.

[0012] In some non-limiting embodiments or aspects, the computer-implemented method may further include embedding each value in the plurality of values ​​to generate a dense embedding matrix. Each corresponding dense embedding matrix row of the dense embedding matrix may include a low-dimensional representation of the corresponding value.

[0013] In some non-limiting embodiments or aspects, generating the input vector may include generating the input vector based on a dense embedding matrix, a position embedding matrix, and a domain embedding matrix.

[0014] In some non-limiting embodiments or aspects, generating the input vector may include concatenating at least one row of a dense embedding matrix, at least one position embedding vector of a position embedding matrix, and at least one domain embedding vector of a domain embedding matrix to produce the input vector.

[0015] In some non-limiting embodiments or aspects, each of the plurality of values ​​may include one of a discrete value or a continuous value. In some non-limiting embodiments or aspects, embedding each discrete value may include encoding the discrete value using an independent embedding. In some non-limiting embodiments or aspects, embedding each continuous value may include encoding the continuous value based on scaling the continuous value using a shared embedding.

[0016] In some non-limiting embodiments or aspects, the computer-implemented method may also include modifying the input vector by replacing one or more values ​​of the input vector to produce a modified input vector. The computer-implemented method may also include inputting the modified input vector into the first MLP model to generate a first modified output vector. The computer-implemented method may also include transposing the first modified output vector to generate a modified transposed vector. The computer-implemented method may also include inputting the modified transposed vector into the second MLP model to generate a second modified output vector. The computer-implemented method may also include adjusting parameters of at least one of the first MLP model, the second MLP model, or any combination thereof based on at least one of the first modified output vector, the second modified output vector, the modified input vector, or any combination thereof.

[0017] In some non-limiting embodiments or aspects, the computer-implemented method may further include normalizing the input vector based on layer normalization to generate a normalized input vector. In some non-limiting embodiments or aspects, inputting the input vector into the first MLP model may include inputting the normalized input vector into the first MLP model.

[0018] In some non-limiting embodiments or aspects, grouping each column based on at least one tree model to generate the domain embedding matrix may include grouping each column based on a gradient boosted decision tree.

[0019] According to non-limiting embodiments or aspects, a system for encoding feature interactions based on tabular data is provided. In some non-limiting embodiments or aspects, the system may include at least one processor and at least one non-transitory computer-readable medium storing instructions, which, when executed by at least one processor, cause the at least one processor to receive a data set in a tabular format including multiple rows and multiple columns. Each of the multiple rows may represent a corresponding data instance in a plurality of data instances. Each of the multiple columns may represent a corresponding feature in a plurality of features. Each data instance of the multiple data instances may include multiple values, including corresponding values ​​associated with each corresponding feature in the plurality of features. Each of the multiple columns may be indexed to generate a position embedding matrix including multiple position embedding vectors. Each position embedding matrix row in the position embedding matrix may include a corresponding position embedding vector associated with a corresponding column in the plurality of columns in the plurality of position embedding vectors. Each of the multiple columns may be grouped based on at least one tree model to generate a domain embedding matrix including multiple domain embedding vectors. An input vector may be generated based on the data set, the position embedding matrix, and the domain embedding matrix. The input vector may be input into a first MLP model to generate a first output vector. The first output vector can be transposed to generate a transposed vector. The transposed vector can be input into a second MLP model to generate a second output vector. The second output vector can be input into at least one classifier model to generate at least one prediction.

[0020] In some non-limiting embodiments or aspects, at least one prediction can include at least one predicted label.

[0021] In some non-limiting embodiments or aspects, the plurality of data instances may include a plurality of payment transaction records. In some non-limiting embodiments or aspects, at least one predicted tag may indicate that a corresponding payment transaction record in the plurality of payment transaction records is predicted to be fraudulent.

[0022] In some non-limiting embodiments or aspects, generating the input vector may include concatenating at least one row of the data set, at least one position embedding vector of the position embedding matrix, and at least one domain embedding vector of the domain embedding matrix to produce the input vector.

[0023] In some non-limiting embodiments or aspects, each value in the plurality of values ​​may be embedded to generate a dense embedding matrix. Each corresponding dense embedding matrix row of the dense embedding matrix may include a low-dimensional representation of the corresponding value.

[0024] In some non-limiting embodiments or aspects, generating the input vector may include generating the input vector based on a dense embedding matrix, a position embedding matrix, and a domain embedding matrix.

[0025] In some non-limiting embodiments or aspects, generating the input vector may include concatenating at least one row of a dense embedding matrix, at least one position embedding vector of a position embedding matrix, and at least one domain embedding vector of a domain embedding matrix to produce the input vector.

[0026] In some non-limiting embodiments or aspects, each of the plurality of values ​​may include one of a discrete value or a continuous value. In some non-limiting embodiments or aspects, embedding each discrete value may include encoding the discrete value using an independent embedding. In some non-limiting embodiments or aspects, embedding each continuous value may include encoding the continuous value based on scaling the continuous value using a shared embedding.

[0027] In some non-limiting embodiments or aspects, the input vector can be modified by replacing one or more values ​​of the input vector to produce a modified input vector. The modified input vector can be input into the first MLP model to generate a first modified output vector. The first modified output vector can be transposed to generate a modified transposed vector. The modified transposed vector can be input into the second MLP model to generate a second modified output vector. The parameters of at least one of the first MLP model, the second MLP model, or any combination thereof can be adjusted based on at least one of the first modified output vector, the second modified output vector, the modified input vector, or any combination thereof.

[0028] In some non-limiting embodiments or aspects, the input vector may be normalized based on layer normalization to generate a normalized input vector. In some non-limiting embodiments or aspects, inputting the input vector into the first MLP model may include inputting the normalized input vector into the first MLP model.

[0029] In some non-limiting embodiments or aspects, grouping each column based on at least one tree model to generate the domain embedding matrix may include grouping each column based on a gradient boosted decision tree.

[0030] According to non-limiting embodiments or aspects, a computer program product for encoding feature interactions based on tabular data is provided. In some non-limiting embodiments or aspects, the computer program product includes at least one non-transitory computer-readable medium, the at least one non-transitory computer-readable medium including program instructions, when executed by at least one processor, the program instructions cause the at least one processor to receive a data set in a tabular format including multiple rows and multiple columns. Each row in the multiple rows can represent a corresponding data instance in a plurality of data instances. Each column in the multiple columns can represent a corresponding feature in a plurality of features. Each data instance of the multiple data instances can include multiple values, the multiple values ​​including a corresponding value associated with each corresponding feature in the multiple features. Each column in the multiple columns can be indexed to generate a position embedding matrix including multiple position embedding vectors. Each position embedding matrix row in the position embedding matrix can include a corresponding position embedding vector associated with a corresponding column in the multiple columns in the multiple position embedding vectors. Each column in the multiple columns can be grouped based on at least one tree model to generate a domain embedding matrix including multiple domain embedding vectors. An input vector can be generated based on a data set, a position embedding matrix, and a domain embedding matrix. The input vector may be input into a first MLP model to generate a first output vector. The first output vector may be transposed to generate a transposed vector. The transposed vector may be input into a second MLP model to generate a second output vector. The second output vector may be input into at least one classifier model to generate at least one prediction.

[0031] In some non-limiting embodiments or aspects, at least one prediction can include at least one predicted label.

[0032] In some non-limiting embodiments or aspects, the plurality of data instances may include a plurality of payment transaction records. In some non-limiting embodiments or aspects, at least one predicted tag may indicate that a corresponding payment transaction record in the plurality of payment transaction records is predicted to be fraudulent.

[0033] In some non-limiting embodiments or aspects, generating the input vector may include concatenating at least one row of the data set, at least one position embedding vector of the position embedding matrix, and at least one domain embedding vector of the domain embedding matrix to produce the input vector.

[0034] In some non-limiting embodiments or aspects, each value in the plurality of values ​​may be embedded to generate a dense embedding matrix. Each corresponding dense embedding matrix row of the dense embedding matrix may include a low-dimensional representation of the corresponding value.

[0035] In some non-limiting embodiments or aspects, generating the input vector may include generating the input vector based on a dense embedding matrix, a position embedding matrix, and a domain embedding matrix.

[0036] In some non-limiting embodiments or aspects, generating the input vector may include concatenating at least one row of a dense embedding matrix, at least one position embedding vector of a position embedding matrix, and at least one domain embedding vector of a domain embedding matrix to produce the input vector.

[0037] In some non-limiting embodiments or aspects, each of the plurality of values ​​may include one of a discrete value or a continuous value. In some non-limiting embodiments or aspects, embedding each discrete value may include encoding the discrete value using an independent embedding. In some non-limiting embodiments or aspects, embedding each continuous value may include encoding the continuous value based on scaling the continuous value using a shared embedding.

[0038] In some non-limiting embodiments or aspects, the input vector can be modified by replacing one or more values ​​of the input vector to produce a modified input vector. The modified input vector can be input into the first MLP model to generate a first modified output vector. The first modified output vector can be transposed to generate a modified transposed vector. The modified transposed vector can be input into the second MLP model to generate a second modified output vector. The parameters of at least one of the first MLP model, the second MLP model, or any combination thereof can be adjusted based on at least one of the first modified output vector, the second modified output vector, the modified input vector, or any combination thereof.

[0039] In some non-limiting embodiments or aspects, the input vector may be normalized based on layer normalization to generate a normalized input vector. In some non-limiting embodiments or aspects, inputting the input vector into the first MLP model may include inputting the normalized input vector into the first MLP model.

[0040] In some non-limiting embodiments or aspects, grouping each column based on at least one tree model to generate the domain embedding matrix may include grouping each column based on a gradient boosted decision tree.

[0041] According to non-limiting embodiments or aspects, a system for encoding feature interactions based on tabular data is provided. In some non-limiting embodiments or aspects, the system includes at least one processor and at least one non-transitory computer-readable medium storing instructions that, when executed by the at least one processor, cause the at least one processor to perform any of the methods described herein.

[0042] According to non-limiting embodiments or aspects, a computer program product for encoding feature interactions based on tabular data is provided. In some non-limiting embodiments or aspects, the computer program product includes at least one non-transitory computer-readable medium, the at least one non-transitory computer-readable medium including program instructions that, when executed by at least one processor, cause the at least one processor to perform any of the methods described herein.

[0043] Other non-limiting embodiments or aspects are set forth in the following numbered clauses:

[0044] Clause 1: A computer-implemented method comprising: receiving, with at least one processor, a data set in a tabular format comprising a plurality of rows and a plurality of columns, wherein each of the plurality of rows represents a corresponding data instance in a plurality of data instances, wherein each of the plurality of columns represents a corresponding feature in a plurality of features, wherein each of the plurality of data instances comprises a plurality of values, the plurality of values ​​comprising a corresponding value associated with each corresponding feature in the plurality of features; and indexing, with at least one processor, each of the plurality of columns to generate a position embedding matrix comprising a plurality of position embedding vectors, wherein each position embedding matrix row in the position embedding matrix comprises a corresponding position in the plurality of position embedding vectors associated with a corresponding column in the plurality of columns. The method comprises the steps of: generating an embedding vector for a plurality of columns; grouping each of a plurality of columns based on at least one tree model using at least one processor to generate a domain embedding matrix including a plurality of domain embedding vectors; generating an input vector based on the data set, the position embedding matrix, and the domain embedding matrix using at least one processor; inputting the input vector into a first multilayer perceptron (MLP) model using at least one processor to generate a first output vector; transposing the first output vector using at least one processor to generate a transposed vector; inputting the transposed vector into a second MLP model using at least one processor to generate a second output vector; and inputting the second output vector into at least one classifier model using at least one processor to generate at least one prediction.

[0045] Clause 2: The computer-implemented method of Clause 1, wherein at least one prediction comprises at least one predicted label.

[0046] Clause 3: A computer-implemented method as described in clause 1 or 2, wherein the plurality of data instances include a plurality of payment transaction records, wherein at least one predicted label indicates that a corresponding payment transaction record in the plurality of payment transaction records is predicted to be fraudulent.

[0047] Clause 4: A computer-implemented method as described in any of clauses 1 to 3, wherein generating the input vector comprises concatenating at least one row of the data set, at least one position embedding vector of the position embedding matrix, and at least one domain embedding vector of the domain embedding matrix to produce the input vector.

[0048] Clause 5: A computer-implemented method as described in any of clauses 1 to 4, further comprising: embedding each value of the plurality of values ​​using at least one processor to generate a dense embedding matrix, each corresponding dense embedding matrix row of the dense embedding matrix comprising a low-dimensional representation of the corresponding value.

[0049] Clause 6: A computer-implemented method as described in any of clauses 1 to 5, wherein generating the input vector comprises generating the input vector based on a dense embedding matrix, a position embedding matrix, and a domain embedding matrix.

[0050] Clause 7: A computer-implemented method as described in any of clauses 1 to 6, wherein generating the input vector comprises: concatenating at least one row of a dense embedding matrix, at least one position embedding vector of a position embedding matrix, and at least one domain embedding vector of a domain embedding matrix to produce the input vector.

[0051] Clause 8: A computer-implemented method as described in any of clauses 1 to 7, wherein each of the plurality of values ​​comprises one of a discrete value or a continuous value, and wherein embedding each discrete value comprises encoding the discrete value using an independent embedding, and wherein embedding each continuous value comprises encoding the continuous value based on scaling the continuous value using a shared embedding.

[0052] Clause 9: A computer-implemented method as described in any of Clauses 1 to 8, further comprising: utilizing at least one processor to modify an input vector by replacing one or more values ​​of the input vector to produce a modified input vector; utilizing at least one processor to input the modified input vector into a first MLP model to generate a first modified output vector; utilizing at least one processor to transpose the first modified output vector to generate a modified transposed vector; utilizing at least one processor to input the modified transposed vector into a second MLP model to generate a second modified output vector; and utilizing at least one processor to adjust parameters of at least one of the first MLP model, the second MLP model, or any combination thereof based on at least one of the first modified output vector, the second modified output vector, the modified input vector, or any combination thereof.

[0053] Clause 10: A computer-implemented method as described in any of clauses 1 to 9, further comprising: utilizing at least one processor to normalize an input vector based on layer normalization to generate a normalized input vector, wherein inputting the input vector into the first MLP model comprises inputting the normalized input vector into the first MLP model.

[0054] Clause 11: The computer-implemented method of any of clauses 1 to 10, wherein grouping each column based on at least one tree model to generate a domain embedding matrix comprises grouping each column based on a gradient boosted decision tree.

[0055] Clause 12: A system comprising: at least one processor; and at least one non-transitory computer-readable medium storing instructions that, when executed by the at least one processor, cause the at least one processor to: receive a data set in a tabular format comprising a plurality of rows and a plurality of columns, wherein each of the plurality of rows represents a corresponding data instance in a plurality of data instances, wherein each of the plurality of columns represents a corresponding feature in a plurality of features, wherein each of the plurality of data instances comprises a plurality of values, the plurality of values ​​comprising a corresponding value associated with each corresponding feature in the plurality of features; index each of the plurality of columns to generate a position embedding matrix comprising a plurality of position embedding vectors, wherein Each position embedding matrix row in the position embedding matrix includes a corresponding position embedding vector in the plurality of position embedding vectors associated with a corresponding column in the plurality of columns; each of the plurality of columns is grouped based on at least one tree model to generate a domain embedding matrix including a plurality of domain embedding vectors; an input vector is generated based on the data set, the position embedding matrix, and the domain embedding matrix; the input vector is input into a first multilayer perceptron (MLP) model to generate a first output vector; the first output vector is transposed to generate a transposed vector; the transposed vector is input into a second MLP model to generate a second output vector; and the second output vector is input into at least one classifier model to generate at least one prediction.

[0056] Clause 13: The system of clause 12, wherein at least one prediction comprises at least one predicted label, wherein the plurality of data instances comprises a plurality of payment transaction records, and wherein at least one predicted label indicates that a corresponding payment transaction record in the plurality of payment transaction records is predicted to be fraudulent.

[0057] Clause 14: A system as described in clause 12 or 13, wherein generating the input vector includes: concatenating at least one row of the data set, at least one position embedding vector of the position embedding matrix, and at least one domain embedding vector of the domain embedding matrix to produce the input vector.

[0058] Clause 15: A system as described in any of clauses 12 to 14, wherein when executed by at least one processor, the instructions further cause the at least one processor to: embed each value of a plurality of values ​​to generate a dense embedding matrix, each corresponding dense embedding matrix row of the dense embedding matrix comprising a low dimensional representation of the corresponding value, wherein generating an input vector comprises generating the input vector based on the dense embedding matrix, the position embedding matrix, and the domain embedding matrix.

[0059] Clause 16: A system as described in any of clauses 12 to 15, wherein generating the input vector includes: concatenating at least one row of a dense embedding matrix, at least one position embedding vector of a position embedding matrix, and at least one domain embedding vector of a domain embedding matrix to produce the input vector.

[0060] Clause 17: A system as described in any of clauses 12 to 16, wherein each of the plurality of values ​​comprises one of a discrete value or a continuous value, and wherein embedding each discrete value comprises encoding the discrete value using an independent embedding, and wherein embedding each continuous value comprises encoding the continuous value based on scaling the continuous value using a shared embedding.

[0061] Clause 18: A system as described in any of clauses 12 to 17, wherein when executed by at least one processor, the instructions further cause the at least one processor to: modify an input vector by replacing one or more values ​​of the input vector to produce a modified input vector; input the modified input vector into a first MLP model to generate a first modified output vector; transpose the first modified output vector to generate a modified transposed vector; input the modified transposed vector into a second MLP model to generate a second modified output vector; and adjust parameters of at least one of the first MLP model, the second MLP model, or any combination thereof based on at least one of the first modified output vector, the second modified output vector, the modified input vector, or any combination thereof.

[0062] Clause 19: The system of any of clauses 12 to 18, wherein grouping each column based on at least one tree model to generate a domain embedding matrix comprises grouping each column based on a gradient boosted decision tree.

[0063] Clause 20: A computer program product comprising at least one non-transitory computer-readable medium, the at least one non-transitory computer-readable medium comprising program instructions that, when executed by at least one processor, cause the at least one processor to: receive a data set in a tabular format comprising a plurality of rows and a plurality of columns, wherein each of the plurality of rows represents a corresponding data instance of a plurality of data instances, wherein each of the plurality of columns represents a corresponding feature of a plurality of features, wherein each of the plurality of data instances comprises a plurality of values, the plurality of values ​​comprising a corresponding value associated with each corresponding feature of the plurality of features; index each of the plurality of columns to generate a position embedding vector comprising a plurality of position embedding vectors; The invention relates to a method for generating a domain embedding matrix comprising a plurality of domain embedding vectors, wherein each position embedding matrix row in the position embedding matrix includes a corresponding position embedding vector of a plurality of position embedding vectors associated with a corresponding column of a plurality of columns; grouping each of the plurality of columns based on at least one tree model to generate a domain embedding matrix including a plurality of domain embedding vectors; generating an input vector based on the dataset, the position embedding matrix, and the domain embedding matrix; inputting the input vector into a first multi-layer perceptron (MLP) model to generate a first output vector; transposing the first output vector to generate a transposed vector; inputting the transposed vector into a second MLP model to generate a second output vector; and inputting the second output vector into at least one classifier model to generate at least one prediction.

[0064] Clause 21: A system comprising: at least one processor; and at least one non-transitory computer-readable medium storing instructions that, when executed by the at least one processor, cause the at least one processor to perform the method of any of clauses 1 to 11.

[0065] Clause 22: A computer program product comprising at least one non-transitory computer-readable medium containing program instructions which, when executed by at least one processor, cause the at least one processor to perform the method of any one of clauses 1 to 11.

[0066] These and other features and characteristics of the present disclosure, as well as methods of operation and functions of the related structural elements and combinations of parts and economies of manufacture, will become more apparent when considering the following description and appended claims with reference to the accompanying drawings, all of which form a part of this specification, wherein like reference numerals designate corresponding parts in the various figures. It is to be expressly understood, however, that the drawings are for purposes of illustration and description only and are not intended as a definition of the limits of the disclosed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Additional advantages and details are explained in more detail below with reference to a non-limiting exemplary embodiment shown in the schematic drawings, in which:

[0068] Figure 1 is a schematic diagram of a system for encoding feature interactions based on tabular data and machine learning according to some non-limiting embodiments or aspects;

[0069] Figure 2 is a flow chart of a method for encoding feature interactions based on tabular data and machine learning according to some non-limiting embodiments or aspects;

[0070] Figure 3 is a diagram of an exemplary environment in which methods, systems, and / or computer program products described herein may be implemented according to some non-limiting embodiments or aspects;

[0071] Figure 4 According to some non-limiting embodiments or aspects Figure 1 and / or Figure 3 a schematic diagram of example components of one or more devices; and

[0072] Figure 5 is a schematic diagram of an implementation of a system and method for encoding feature interactions based on tabular data and machine learning according to some non-limiting embodiments or aspects. DETAILED DESCRIPTION

[0073] For the purposes of the following description, the terms "end," "upper," "lower," "right," "left," "vertical," "horizontal," "top," "bottom," "lateral," "longitudinal," and their derivatives shall relate to the orientation of the embodiments in the accompanying drawings. However, it shall be understood that the embodiments may employ various alternative variations and step sequences, except where expressly specified to the contrary. It shall also be understood that the specific devices and processes illustrated in the drawings and described in the following specification are merely exemplary embodiments or aspects of the disclosed subject matter. Accordingly, specific dimensions and other physical characteristics relating to the embodiments or aspects disclosed herein shall not be considered limiting.

[0074] The aspects, parts, elements, structures, actions, steps, functions, instructions, etc. used herein should not be understood as critical or necessary unless explicitly described as such. Moreover, as used herein, the article "one" is intended to include one or more items, and can be used interchangeably with "one or more" and "at least one". In addition, as used herein, the term "set" is intended to include one or more items (e.g., related items, unrelated items, a combination of related items and unrelated items, and / or the like), and can be used interchangeably with "one or more" or "at least one". In the case of intending to have only one item, the term "one" or similar language is used. Moreover, as used herein, the term "having" and / or the like is intended to be an open term. In addition, unless otherwise explicitly stated, the phrase "based on" is intended to mean "based at least in part on".

[0075] As used herein, the term "acquirer institution" may refer to an entity that is licensed and / or approved by a transaction service provider to initiate transactions (e.g., payment transactions) using a payment device associated with the transaction service provider. Transactions that may be initiated by an acquirer institution may include payment transactions (e.g., purchases, original letter of credit transactions (OCTs), account funding transactions (AFTs), and / or the like). In some non-limiting embodiments or aspects, an acquirer institution may be a financial institution, such as a bank. As used herein, the term "acquirer system" may refer to one or more computing devices operated by or on behalf of an acquirer institution, such as a server computer executing one or more software applications.

[0076] As used herein, the term "account identifier" may include one or more primary account numbers (PANs), payment tokens, or other identifiers associated with a customer account. The term "payment token" may refer to an identifier used as a replacement or alternative identifier for an original account identifier such as a PAN. An account identifier may be alphanumeric or any combination of characters and / or symbols. A payment token may be associated with a PAN or other original account identifier in one or more data structures (e.g., one or more databases, etc.) so that the payment token can be used to conduct transactions without directly using the original account identifier. In some instances, an original account identifier such as a PAN may be associated with multiple payment tokens for different individuals or purposes.

[0077] As used herein, the term "communication" may refer to the reception, acceptance, transmission, transfer, provision, etc. of data (e.g., information, signals, messages, instructions, commands, etc.). A unit (e.g., a device, a system, a component of a device or system, a combination thereof, and / or the like) communicating with another unit means that the unit is able to directly or indirectly receive information from the other unit and / or send information to the other unit. This may refer to a direct or indirect connection (e.g., a direct communication connection, an indirect communication connection, and / or the like) that is wired and / or wireless in nature. In addition, although the information sent may be modified, processed, relayed, and / or routed between the first unit and the second unit, the two units may also communicate with each other. For example, even if the first unit passively receives information and does not actively send information to the second unit, the first unit may communicate with the second unit. As another example, if at least one intermediate unit processes the information received from the first unit and transmits the processed information to the second unit, the first unit may communicate with the second unit.

[0078] As used herein, the term "computing device" may refer to one or more electronic devices configured to process data. In some examples, a computing device may include the necessary components for receiving, processing, and outputting data, such as a processor, a display, a memory, an input device, a network interface, and / or the like. A computing device may be a mobile device. As an example, a mobile device may include a cellular phone (e.g., a smartphone or a standard cellular phone), a portable computer, a wearable device (e.g., a watch, glasses, lenses, clothing, etc.), a personal digital assistant (PDA), and / or other similar devices. A computing device may also be a desktop computer or other form of non-mobile computer.

[0079] As used herein, the terms "electronic wallet" and "electronic wallet application" refer to one or more electronic devices and / or software applications configured to initiate and / or conduct payment transactions. For example, an electronic wallet may include a mobile device executing an electronic wallet application, and may also include server-side software and / or databases for maintaining and providing transaction data to the mobile device. An "electronic wallet provider" may include an entity that provides and / or maintains electronic wallets for customers, such as Google Android Apple Samsung And / or other similar electronic payment systems. In some non-limiting examples, the issuing bank may be an electronic wallet provider.

[0080] As used herein, the term "issuer institution" may refer to one or more entities, such as a bank, that provide accounts to customers for conducting transactions (e.g., payment transactions), such as initiating credit and / or debit payments. For example, an issuer institution may provide customers with an account identifier, such as a PAN, that uniquely identifies one or more accounts associated with the customer. The account identifier may be embodied on a portable financial device, such as a physical financial instrument (e.g., a payment card), and / or may be electronic and used for electronic payments. The term "issuer system" refers to one or more computer devices operated by or on behalf of an issuer institution, such as a server computer executing one or more software applications. For example, an issuer system may include one or more authorization servers for authorizing transactions.

[0081] As used herein, the term "merchant" may refer to a person or entity that provides goods and / or services, or access to goods and / or services, to a customer based on a transaction, such as a payment transaction. The term "merchant" or "merchant system" may also refer to one or more computer systems operated by or on behalf of a merchant, such as a server computer that executes one or more software applications.

[0082] As used herein, a "point of sale (POS) device" may refer to one or more devices that may be used by a merchant to conduct a transaction (e.g., a payment transaction) and / or process a transaction. For example, a POS device may include one or more client devices. Additionally or alternatively, a POS device may include a peripheral device, a card reader, a scanning device (e.g., a code scanner), Communication receiver, near field communication (NFC) receiver, radio frequency identification (RFID) receiver and / or other contactless transceiver or receiver, contact-based receiver, payment terminal and / or the like. As used herein, a "point of sale (POS) system" may refer to one or more client devices and / or peripheral devices used by a merchant to conduct transactions. For example, a POS system may include one or more POS devices, and / or other similar devices that can be used to conduct payment transactions. In some non-limiting embodiments or aspects, a POS system (e.g., a merchant POS system) may include one or more server computers that are programmed or configured to process online payment transactions through web pages, mobile applications, etc.

[0083] As used herein, the terms "client" and "client device" may refer to one or more client-side devices or systems (e.g., at a remote location of a transaction service provider) for initiating or facilitating a transaction (e.g., a payment transaction). As an example, a "client device" may refer to one or more POS devices used by a merchant, one or more acquirer host computers used by an acquirer, one or more mobile devices used by a user, and / or the like. In some non-limiting embodiments or aspects, a client device may be an electronic device configured to communicate with one or more networks and initiate or facilitate a transaction. For example, a client device may include one or more computers, portable computers, laptop computers, tablet computers, mobile devices, cellular phones, wearable devices (e.g., watches, glasses, lenses, clothing, and / or the like), PDAs, and / or the like. In addition, a "client" may also refer to an entity (e.g., a merchant, an acquirer, etc.) that owns, utilizes, and / or operates a client device for initiating a transaction (e.g., for initiating a transaction with a transaction service provider).

[0084] As used herein, the term "payment device" may refer to a payment card (e.g., a credit or debit card), a gift card, a smart card, smart media, a payroll card, a healthcare card, a wristband, a machine-readable medium containing account information, a key chain device or fob, an RFID transponder, a retailer discount or membership card, a cellular phone, an electronic wallet mobile application, a personal digital assistant (PDA), a pager, a security card, a computing device, an access card, a wireless terminal, a transponder, etc. In some non-limiting embodiments or aspects, a payment device may include volatile or non-volatile memory to store information (e.g., an account identifier, an account holder's name, etc.).

[0085] As used herein, the term "payment gateway" may refer to an entity and / or a payment processing system operated by or on behalf of such an entity that provides payment services (e.g., transaction service provider payment services, payment processing services, and / or the like) to one or more merchants (e.g., transaction service provider payment services, payment processing services, and / or the like). The payment services may be associated with the use of a portable financial device managed by a transaction service provider. As used herein, the term "payment gateway system" may refer to one or more computer systems, computer devices, servers, server groups, etc. operated by or on behalf of a payment gateway.

[0086] As used herein, the term "server" may refer to or include one or more computing devices operated by or facilitating communications and processing by multiple parties in a network environment such as the Internet, but it should be understood that communications may be facilitated through one or more public or private network environments, and that various other arrangements are possible. In addition, multiple computing devices (e.g., servers, point of sale (POS) devices, mobile devices, etc.) that communicate directly or indirectly in a network environment may constitute a "system." As used herein, references to a "server" or "processor" may refer to a previously described server and / or processor stated to perform a previous step or function, a different server and / or processor, and / or a combination of servers and / or processors. For example, as used in the specification and claims, a first server and / or first processor stated to perform a first step or function may refer to the same or different server and / or processor stated to perform a second step or function.

[0087] As used herein, the term "transaction service provider" may refer to an entity that receives transaction authorization requests from merchants or other entities and, in some cases, provides payment assurance through an agreement between the transaction service provider and the issuer organization. For example, a transaction service provider may include, e.g. The term "transaction processing system" may refer to one or more computer systems operated by or on behalf of a transaction service provider, such as a transaction processing server that executes one or more software applications. A transaction processing server may include one or more processors and, in some non-limiting embodiments or aspects, may be operated by or on behalf of a transaction service provider.

[0088] Non-limiting embodiments or aspects of the disclosed subject matter relate to systems, methods, and computer program products for encoding feature interactions (including, but not limited to, encoding feature interactions based on tabular data and machine learning). For example, non-limiting embodiments or aspects of the disclosed subject matter provide for receiving a data set in a tabular format including a plurality of rows and a plurality of columns. Each row in a plurality of rows may represent a corresponding data instance in a plurality of data instances. Each column in a plurality of columns may represent a corresponding feature in a plurality of features. Each data instance in a plurality of data instances may include a plurality of values, the plurality of values ​​including a corresponding value associated with each corresponding feature in a plurality of features. Each column in a plurality of columns may be indexed to generate a position embedding matrix including a plurality of position embedding vectors. Each position embedding matrix row in the position embedding matrix may include a corresponding position embedding vector associated with a corresponding column in a plurality of columns in a plurality of position embedding vectors. Each column in a plurality of columns may be grouped based on at least one tree model to generate a domain embedding matrix including a plurality of domain embedding vectors. An input vector may be generated based on a data set, a position embedding matrix, and a domain embedding matrix. The input vector may be input into a first multilayer perceptron (MLP) model to generate a first output vector. The first output vector can be transposed to generate a transposed vector. The transposed vector can be input into a second MLP model to generate a second output vector. The second output vector can be input into at least one classifier model to generate at least one prediction. These embodiments or aspects provide a method and system for encoding feature interactions based on tabular data, which achieves improved performance and efficiency. Non-limiting embodiments or aspects can allow scaling systems to analyze tabular data with thousands of features including millions of data instances, while maintaining or improving performance (e.g., accuracy) and greatly improving efficiency. Additionally, when a larger (e.g., millions of data instances) tabular data set is used as an input (e.g., used as a training input and / or runtime input) of a machine learning model, non-limiting embodiments or aspects can reduce memory usage and computational bottlenecks. Further, non-limiting embodiments for learning features in tabular data can be scaled for larger tabular data sets without sacrificing performance. Non-limiting embodiments or aspects can allow instant training of machine learning models, and ignore explicit calculations of similarities between a pair of features. These non-limiting embodiments or aspects can improve the ability of machine learning models to generalize feature interactions and classification tasks with improved efficiency.

[0089] Figure 1 A system 100 for encoding feature interactions based on tabular data is depicted according to some non-limiting embodiments or aspects. The system 100 may include a feature encoding system 102 and a machine learning model 104.

[0090] The feature encoding system 102 may include a computing device, such as a server (e.g., a single server), a server group, and / or other similar devices. In some non-limiting embodiments or aspects, the feature encoding system 102 may include at least one processor (e.g., a multi-core processor), such as a graphics processing unit (GPU), a central processing unit (CPU), an accelerated processing unit (APU), a microprocessor, etc. In some non-limiting embodiments or aspects, the feature encoding system 102 may include a memory, one or more storage components, one or more input components, one or more output components, and one or more communication interfaces, as described herein.

[0091] The machine learning model 104 may include one or more machine learning models. For example, the machine learning model 104 may include one or more convolutional neural networks (CNNs), feedforward artificial neural networks (ANNs), such as multilayer perceptrons (MLPs), deep neural networks (DNNs), decision trees (e.g., gradient boosting decision trees), etc. In some non-limiting embodiments or aspects, the machine learning model 104 may be trained based on the techniques described herein. In some non-limiting embodiments or aspects, the machine learning model 104 may be used to generate predictions as described herein. In some non-limiting embodiments or aspects, the machine learning model 104 may communicate with the feature encoding system 102. In some non-limiting embodiments or aspects, the machine learning model 104 may be implemented by the feature encoding system 102 (e.g., a portion thereof). In some non-limiting embodiments or aspects, the machine learning model 104 may be implemented by another system, another device, another system group, or another device group (e.g., a portion thereof) that is separate from the feature encoding system 102 or includes the feature encoding system.

[0092] Figure 1 The number and arrangement of systems and devices shown in the FIG. are provided as examples. Figure 1 There may be additional systems and / or devices, fewer systems and / or devices, different systems and / or devices, and / or systems and / or devices arranged in a different manner than the systems and / or devices shown in the drawings. Furthermore, a single system and / or device may be implemented Figure 1 Two or more systems or devices shown in Figure 1 The single system or device shown in the embodiment may be implemented as multiple distributed systems or devices. Additionally or alternatively, a group of systems (e.g., one or more systems) and / or a group of devices (e.g., one or more devices) of system 100 may perform one or more functions described as being performed by another group of systems or another group of devices of system 100.

[0093] Reference now Figure 2, illustrates a process 200 for encoding feature interactions based on tabular data, according to some non-limiting embodiments or aspects. Figure 2 The steps shown are for exemplary purposes only. It should be understood that additional, fewer, different and / or different order steps may be used in non-limiting embodiments or aspects.

[0094] like Figure 2 As shown, at step 202, process 200 may include receiving a data set in a tabular format. For example, feature encoding system 102 may receive a data set in a tabular format including a plurality of rows and a plurality of columns. In some non-limiting embodiments or aspects, each of the plurality of rows may represent a corresponding data instance in a plurality of data instances. In some non-limiting embodiments or aspects, each of the plurality of columns may represent a corresponding feature in a plurality of features. In some non-limiting embodiments or aspects, each data instance of the plurality of data instances may include a plurality of values ​​including a corresponding value associated with each corresponding feature in the plurality of features. In some non-limiting embodiments or aspects, the plurality of data instances may include a plurality of payment transaction records.

[0095] In some non-limiting embodiments or aspects, the feature encoding system 102 may embed each of the plurality of values ​​to generate a dense embedding matrix. In some non-limiting embodiments or aspects, each corresponding dense embedding matrix row of the dense embedding matrix may include a low-dimensional representation of the corresponding value.

[0096] In some non-limiting embodiments or aspects, each value in the plurality of values ​​may include one of a discrete value or a continuous value. In some non-limiting embodiments or aspects, the feature encoding system 102 may embed each discrete value by encoding the discrete value using an independent embedding. In some non-limiting embodiments or aspects, the feature encoding system 102 may embed each continuous value by encoding the continuous value based on scaling the continuous value using a shared embedding.

[0097] like Figure 2 As shown, at step 204, process 200 may include indexing the columns to generate a position embedding matrix. For example, feature encoding system 102 may index each of the plurality of columns to generate a position embedding matrix including a plurality of position embedding vectors. In some non-limiting embodiments or aspects, each position embedding matrix row in the position embedding matrix may include a corresponding position embedding vector of the plurality of position embedding vectors associated with a corresponding column of the plurality of columns.

[0098] like Figure 2As shown, at step 206, process 200 may include grouping the columns to generate a domain embedding matrix. For example, feature encoding system 102 may group each of the plurality of columns based on at least one tree model to generate a domain embedding matrix including a plurality of domain embedding vectors. In some non-limiting embodiments or aspects, feature encoding system 102 may group each of the plurality of columns based on a gradient boosting decision tree (e.g., XGBoost, etc.).

[0099] like Figure 2 As shown, at step 208, process 200 may include generating an input vector. For example, feature encoding system 102 may generate the input vector based on the data set, the position embedding matrix, and the domain embedding matrix. In some non-limiting embodiments or aspects, feature encoding system 102 may generate the input vector by concatenating at least one row of the data set, at least one position embedding vector of the position embedding matrix, and at least one domain embedding vector of the domain embedding matrix to generate the input vector.

[0100] In some non-limiting embodiments or aspects, the feature encoding system 102 may generate an input vector based on a dense embedding matrix, a position embedding matrix, and a domain embedding matrix. In some non-limiting embodiments or aspects, the feature encoding system 102 may generate an input vector by concatenating at least one row of the dense embedding matrix, at least one position embedding vector of the position embedding matrix, and at least one domain embedding vector of the domain embedding matrix to produce the input vector.

[0101] In some non-limiting embodiments or aspects, the feature encoding system 102 may modify the input vector. For example, the feature encoding system 102 may modify the input vector by replacing one or more values ​​of the input vector (e.g., removing, replacing with a 0 value, replacing with a default value, etc.) to generate a modified input vector. In some non-limiting embodiments or aspects, the feature encoding system 102 may normalize the input vector. For example, the feature encoding system 102 may normalize the input vector based on layer normalization to generate a normalized input vector.

[0102] like Figure 2As shown, at step 210, process 200 may include inputting an input vector to generate a first output vector. For example, feature encoding system 102 may input the input vector into a first multilayer perceptron (MLP) model (e.g., its machine learning model 104) to generate a first output vector. In some non-limiting embodiments or aspects, feature encoding system 102 may input a modified input vector to generate a first modified output vector. For example, feature encoding system 102 may input the modified input vector into a first MLP model to generate a first modified output vector. In some non-limiting embodiments or aspects, feature encoding system 102 may input the input vector into a first MLP model by inputting a normalized input vector into the first MLP model.

[0103] like Figure 2 As shown, at step 212, process 200 may include transposing the first output vector to generate a transposed vector. For example, feature encoding system 102 may transpose the first output vector to generate a transposed vector. In some non-limiting embodiments or aspects, feature encoding system 102 may transpose the first modified output vector to generate a modified transposed vector.

[0104] like Figure 2 As shown, at step 214, process 200 may include inputting the transposed vector to generate a second output vector. For example, feature encoding system 102 may input the transposed vector into a second MLP model (e.g., its machine learning model 104) to generate a second output vector. In some non-limiting embodiments or aspects, feature encoding system 102 may input the modified transposed vector to generate a second modified output vector. For example, feature encoding system 102 may input the modified transposed vector into a second MLP model to generate a second modified output vector.

[0105] In some non-limiting embodiments or aspects, the feature encoding system 102 can adjust parameters of the MLP model. For example, the feature encoding system 102 can adjust parameters of at least one of the first MLP model, the second MLP model, or any combination thereof based on at least one of the first modified output vector, the second modified output vector, the modified input vector, or any combination thereof.

[0106] like Figure 2As shown, at step 216, process 200 may include inputting the second output vector to generate a prediction. For example, feature encoding system 102 may input the second output vector into at least one classifier model (e.g., its machine learning model 104) to generate at least one prediction. In some non-limiting embodiments or aspects, at least one prediction may include at least one predicted label. In some non-limiting embodiments or aspects, at least one predicted label may indicate that a corresponding payment transaction record in a plurality of payment transaction records is predicted to be fraudulent.

[0107] Reference now Figure 3 , Figure 3 3 is a diagram of a non-limiting embodiment or aspect of an exemplary environment 300 in which systems, products, and / or methods as described herein may be implemented. Figure 3 As shown, environment 300 includes transaction service provider system 302, issuer system 304, client device 306, merchant system 308, acquirer system 310, and communication network 312. In some non-limiting embodiments or aspects, each of feature encoding system 102 and / or machine learning model 104 may be implemented by transaction service provider system 302 (e.g., a portion thereof). In some non-limiting embodiments or aspects, at least one of each of feature encoding system 102 and / or machine learning model 104 may be implemented by another system, another device, another system group, or another device group (e.g., a portion thereof) that is separate from or includes transaction service provider system 302 (such as issuer system 304, client device 306, merchant system 308, acquirer system 310, etc.).

[0108] The transaction service provider system 302 may include one or more devices capable of receiving information from an issuer system 304, a client device 306, a merchant system 308, and / or an acquirer system 310 via a communication network 312 and / or transmitting information to the issuer system, the client device, the merchant system, and / or the acquirer system. For example, the transaction service provider system 302 may include a computing device, such as a server (e.g., a transaction processing server, etc.), a server group, and / or other similar devices. In some non-limiting embodiments or aspects, the transaction service provider system 302 may be associated with a transaction service provider described herein. In some non-limiting embodiments or aspects, the transaction service provider system 302 may communicate with a data storage device, which may be local or remote to the transaction service provider system 302. In some non-limiting embodiments or aspects, the transaction service provider system 302 may receive information from the data storage device, store information in the data storage device, transmit information to the data storage device, or search for information stored in the data storage device.

[0109] The issuer system 304 may include one or more devices capable of receiving information from and / or transmitting information to the transaction service provider system 302, the client device 306, the merchant system 308, and / or the acquirer system 310 via a communication network 312. For example, the issuer system 304 may include a computing device, such as a server, a server group, and / or other similar devices. In some non-limiting embodiments or aspects, the issuer system 304 may be associated with an issuer organization described herein. For example, the issuer system 304 may be associated with an issuer organization that issues a credit account, a debit account, a credit card, a debit card, and / or the like to a user associated with the client device 306.

[0110] The client devices 306 may include one or more devices capable of receiving information from and / or transmitting information to the transaction service provider system 302, the issuer system 304, the merchant system 308, and / or the acquirer system 310 via a communication network 312. Additionally or alternatively, each client device 306 may include a device capable of receiving information from and / or transmitting information to other client devices 306 via the communication network 312, another network (e.g., a temporary network, a local network, a private network, a virtual private network, and / or the like), and / or any other suitable communication technology. For example, the client devices 306 may include client devices and / or the like. In some non-limiting embodiments or aspects, the client devices 306 may or may not be capable of communicating via a short-range wireless communication connection (e.g., an NFC communication connection, an RFID communication connection, Communication connection, communication connection and / or the like) (e.g., from merchant system 308 or from another client device 306), and / or transmit information via a short-range wireless communication connection (e.g., to merchant system 308).

[0111] The merchant system 308 may include one or more devices capable of receiving information from and / or transmitting information to the transaction service provider system 302, the issuer system 304, the customer device 306, and / or the acquirer system 310 via the communication network 312. The merchant system 308 may also include a communication connection (e.g., an NFC communication connection, an RFID communication connection, a NFC communication connection, a ... Communication connection, In some non-limiting embodiments or aspects, the merchant system 308 may include a computing device, such as a server, a server group, a client device, a client device group, and / or other similar devices. In some non-limiting embodiments or aspects, the merchant system 308 may be associated with the merchant described herein. In some non-limiting embodiments or aspects, the merchant system 308 may include one or more client devices. For example, the merchant system 308 may include a client device that allows the merchant to transmit information to the transaction service provider system 302. In some non-limiting embodiments or aspects, the merchant system 308 may include one or more devices that can be used by merchants to conduct transactions with users, such as computers, computer systems, and / or peripheral devices. For example, the merchant system 308 may include a POS device and / or a POS system.

[0112] Acquirer system 310 may include one or more devices capable of receiving information from and / or transmitting information to transaction service provider system 302, issuer system 304, client device 306, and / or merchant system 308 via communication network 312. For example, acquirer system 310 may include a computing device, a server, a server group, and / or the like. In some non-limiting embodiments or aspects, acquirer system 310 may be associated with an acquirer as described herein.

[0113] The communication network 312 may include one or more wired and / or wireless networks. For example, the communication network 312 may include a cellular network (e.g., Long Term Evolution). networks, third generation (3G) networks, fourth generation (4G) networks, fifth generation (5G) networks, code division multiple access (CDMA) networks, etc.), public land mobile networks (PLMNs), local area networks (LANs), wide area networks (WANs), metropolitan area networks (MANs), telephone networks (e.g., public switched telephone networks (PSTNs)), private networks (e.g., private networks associated with transaction service providers), ad hoc networks, intranets, the Internet, fiber-optic-based networks, cloud computing networks, etc., and / or combinations of these or other types of networks.

[0114] Provided as an example Figure 3 The number and arrangement of systems, devices, and / or networks shown are not intended to be construed as limiting the number and arrangement of systems, devices, and / or networks shown. Additional systems, devices, and / or networks may exist; fewer systems, devices, and / or networks; different systems, devices, and / or networks; and / or networks in a manner similar to that shown. Figure 3The systems, devices and / or networks shown in the figure may be arranged in different ways. In addition, the system or device may be implemented in a single system or device. Figure 3 Two or more systems or devices shown, or Figure 3 The single system or device shown may be implemented as multiple distributed systems or devices. Additionally or alternatively, a set of systems (e.g., one or more systems) and / or a set of devices (e.g., one or more devices) of environment 300 may perform one or more functions described as being performed by another set of systems or another set of devices of environment 300.

[0115] Reference now Figure 4 , a diagram showing example components of a device 400 according to a non-limiting embodiment or aspect. As an example, the device 400 may correspond to Figure 1 At least one of the feature encoding system 102 and / or the machine learning model 104 and / or Figure 3 At least one of the transaction service provider system 302, the issuer system 304, the customer device 306, the merchant system 308 and / or the acquirer system 310. In some non-limiting embodiments or aspects, Figure 1 or Figure 3 Such a system or device in may include at least one device 400 and / or at least one component of a device 400. Figure 4 The number and arrangement of the components shown in FIG. are provided as examples. In some non-limiting embodiments or aspects, Figure 4 Device 400 may include additional components, fewer components, different components, or components arranged in a different manner than those shown in . Additionally or alternatively, a set of components (e.g., one or more components) of device 400 may perform one or more functions described as being performed by another set of components of device 400.

[0116] like Figure 4As shown, the device 400 may include a bus 402, a processor 404, a memory 406, a storage component 408, an input component 410, an output component 412, and a communication interface 414. The bus 402 may include components that permit communication between components of the device 400. In some non-limiting embodiments or aspects, the processor 404 may be implemented in hardware, firmware, or a combination of hardware and software. For example, the processor 404 may include a processor (e.g., a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), etc.), a microprocessor, a digital signal processor (DSP), and / or any processing component that can be programmed to perform a function (e.g., a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc.). The memory 406 may include a random access memory (RAM), a read-only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic memory, optical memory, etc.) that stores information and / or instructions for use by the processor 404.

[0117] Continue to refer Figure 4 , the storage component 408 may store information and / or software related to the operation and use of the device 400. For example, the storage component 408 may include a hard disk (e.g., a magnetic disk, an optical disk, a magneto-optical disk, a solid-state disk, etc.) and / or another type of computer-readable medium. The input component 410 may include a component that allows the device 400 to receive information, for example, through user input (e.g., a touch screen display, a keyboard, a keypad, a mouse, a button, a switch, a microphone, etc.). In addition or alternatively, the input component 410 may include a sensor for sensing information (e.g., a global positioning system (GPS) component, an accelerometer, a gyroscope, an actuator, etc.). The output component 412 may include a component that provides output information from the device 400 (e.g., a display, a speaker, one or more light-emitting diodes (LEDs), etc.). The communication interface 414 may include a transceiver-like component (e.g., a transceiver, a separate receiver and a transmitter, etc.) that enables the device 400 to communicate with other devices, for example, via a wired connection, a wireless connection, or a combination of wired and wireless connections. The communication interface 414 may allow the device 400 to receive information from another device and / or provide information to another device. For example, the communication interface 414 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, interface, cellular network interface, and / or the like.

[0118] Device 400 can perform one or more processes described herein. Device 400 can perform these processes based on processor 404 executing software instructions stored by computer-readable media such as memory 406 and / or storage component 408. Computer-readable media can include any non-transient memory device. The memory device includes a memory space located inside a single physical storage device or a memory space distributed on multiple physical storage devices. The software instructions can be read from another computer-readable medium or from another device to memory 406 and / or storage component 408 through communication interface 414. When executed, the software instructions stored in memory 406 and / or storage component 408 can make processor 404 perform one or more processes described herein. In addition or alternatively, hard-wired circuit system can be used in place of or in combination with software instructions to perform one or more processes described herein. Therefore, the embodiments described herein are not limited to any specific combination of hardware circuit system and software. The term "programmed or configured" used herein refers to the arrangement of software, hardware circuit system or any combination thereof on one or more devices.

[0119] Reference now Figure 5 , a schematic diagram of an example implementation 500 of a system and method for encoding feature interactions based on tabular data and machine learning according to some non-limiting embodiments or aspects is shown. In some non-limiting embodiments or aspects, the implementation 500 can be implemented by the feature encoding system 102 and / or its machine learning model 104 (e.g., performed by, as part of, the same as, similar to, etc.).

[0120] In some non-limiting embodiments or aspects, a data set 502 may be received (e.g., by a feature encoding system 102, etc.). For example, the data set 502 may be in a table format including a plurality of rows (e.g., n rows) and a plurality of columns (e.g., m columns). In some non-limiting embodiments or aspects, each of the n rows may represent a corresponding data instance in a plurality of data instances, and / or each of the m columns may represent a corresponding feature in a plurality of features. For example, the m feature columns of a data instance x may be represented as x=[x1, ..., x2, ..., x3, ..., x4, ..., x5, ..., x6, ..., x7, ..., x8, ..., x9, ..., x10, ..., x11, ..., x12, ..., x13, ..., x14, ..., x15, ..., x16, ..., x17, ..., x18, ..., x19, ..., x219, ..., x22, ..., x31, ..., x4, ..., x5 m ], where x j Indexes the jth column. For illustration purposes, Figure 5 As shown, the four feature columns are x1, x2, x3, and x4. For the i-th row, the corresponding data instance x (i) Can include multiple values And each corresponding value may be associated with a corresponding feature of the m feature clocks. In some non-limiting embodiments or aspects, the plurality of data instances may include a plurality of payment transaction records. For illustration purposes, Figure 5 As shown, x(i) Instantiate using [0.1, Male, Green, 0.5].

[0121] In some non-limiting embodiments or aspects, each value in the plurality of values ​​may be embedded (e.g., by feature encoding system 102, etc.) to generate a dense embedding matrix. In some non-limiting embodiments or aspects, each corresponding dense embedding matrix row of the dense embedding matrix may include a low-dimensional representation of the corresponding value. For example, the data instance Each eigenvalue in can be embedded into a dense low-dimensional vector. In some non-limiting embodiments or aspects, a given eigenvalue can be discrete or continuous in the tabular data set 502. Thus, for example, discrete values ​​can be encoded using independent embeddings, and / or continuous values ​​can be represented by scaling such continuous values ​​using a shared embedding. For illustration purposes, in a given data instance x (i) In the case of dense embedding matrix Can be initialized as where d is the dimension of the low-dimensional representation. For example, Each row of can represent a low-dimensional representation of the corresponding value. In some non-limiting embodiments or aspects, Each feature embedding in can be mapped to initialize the corresponding node in the learned feature interaction graph.

[0122] In some non-limiting embodiments or aspects, the columns of the tabular data set 502 may be indexed (e.g., by the feature encoding system 102, etc.) to generate a position embedding matrix. For example, each of the plurality of columns may be indexed to generate a position embedding matrix comprising a plurality of position embedding vectors. In some non-limiting embodiments or aspects, each position embedding matrix row in the position embedding matrix may include a corresponding position embedding vector of the plurality of position embedding vectors associated with a corresponding column of the plurality of columns. For illustration purposes, in a given data instance x=[x1,…,x m ] In the case of m feature columns, the position embedding matrix X pos It can be expressed as Where each row can represent the spatial position of the corresponding column. In some non-limiting embodiments or aspects, the position embedding can be shared by all data instances.

[0123] In some non-limiting embodiments or aspects, the columns can be grouped (e.g., by the feature encoding system 102, etc.) to generate a domain embedding matrix. For example, each of the multiple columns can be grouped based on at least one tree model to generate a domain embedding matrix including multiple domain embedding vectors. In some non-limiting embodiments or aspects, each of the multiple columns can be grouped based on a gradient boosted decision tree (e.g., XGBoost, etc.). For example, tree-based methods can enable the discovery of nonlinear dependencies between features, so gradient boosted decision trees can group features into trees (e.g., related features within a tree can be considered semantically similar, and domain embeddings can allow learning such explicit feature combinations). For illustration purposes, let T denote the number of trees, and T t ={x j1 ,…,x jm} represents the feature set of the t-th tree. Considering the total number of T trees, the feature column x j The domain embedding of is given by:

[0124]

[0125] in is the domain embedding shared by the features within the t-th tree, and Indicates whether feature j is in tree T t Consider all feature columns x=[x1,…,x m ], the domain embedding matrix can be given by Given, where T is the transpose operator. In some non-limiting embodiments or aspects, the domain embedding matrix can be shared by all samples in the tabular data.

[0126] In some non-limiting embodiments or aspects, an input vector X may be generated (e.g., by feature encoding system 102, etc.). For example, input vector X may be generated based on data set 502, a position embedding matrix, and a domain embedding matrix. In some non-limiting embodiments or aspects, the input vector X may be generated by concatenating at least one row of data set 502, at least one position embedding vector of the position embedding matrix, and at least one domain embedding vector of the domain embedding matrix to generate the input vector. For illustration purposes, the input vector X may be generated based on the corresponding dense embedding matrix X. Position embedding matrix X pos and domain embedding matrix X dom To generate the input vector X of the i-th data instance. For example, the input vector X of the i-th data instance can be generated based on concatenating these matrices as follows:

[0127] In some non-limiting embodiments or aspects, the input vector may be input (e.g., by feature encoding system 102, etc.) to at least one machine learning model (e.g., of machine learning model 104) to generate a first output vector. For illustration purposes, Figure 5 As shown, the input vector X can be input to the layer normalization layer 504, which can generate a normalized input vector based on layer normalization. For illustration purposes, Figure 5 As shown, matrix 506 may include normalized input vectors for all data instances, and matrix 506 may include columns representing feature channels and rows representing data instance tokens (e.g., not to be confused with payment tokens). For example, a feature channel may be associated with a low-dimensional representation of multiple features, and each data instance token may be a representation of a corresponding data instance. In some non-limiting embodiments or aspects, the normalized input vector may be transposed by transposition operation 508. For illustration purposes, as shown in FIG. Figure 5 As shown, after transposition, the transposed matrix 510 may include the transposed normalized input vector, and thus, the transposed matrix 510 may include columns representing data instance tokens and rows representing feature channels. For illustration purposes, Figure 5 As shown, each row of the transposed matrix 510 can be input to a first MLP model (e.g., a token mixing (TM) MLP model 512). In some non-limiting embodiments or aspects, the TM MLP model 512 may include at least one fully connected layer and at least one non-linear activation function. For example, the TMMLP model 512 may include a first fully connected layer represented by a first weight matrix W1, an element-wise non-linear activation function σ (e.g., a Gaussian Error Linear Unit (GELU) activation function, a sigmoid activation function, etc.), and a second fully connected layer represented by a second weight matrix W2. In some non-limiting embodiments or aspects, the TM MLP model 512 may generate at least one first output vector (e.g., an output vector for each row of the transposed matrix 510). For illustration purposes, as Figure 5 As shown, first output matrix 514 may include rows associated with a corresponding first output vector for each corresponding row of transposed matrix 510 .

[0128] In some non-limiting embodiments or aspects, the first output vector may be transposed (e.g., by feature encoding system 102, etc.) to generate a transposed vector. For illustration purposes, Figure 5As shown, the first output matrix 514 (e.g., including rows associated with the first output vector) can be transposed by a transpose operation 516 to generate a transposed first output matrix 518, and thus, the transposed first output matrix 518 can include columns representing feature channels and rows representing data instance tokens.

[0129] In some non-limiting embodiments or aspects, the transposed first output vector(s) may be combined (e.g., added, summed, etc.) with the input vector X (e.g., by the feature encoding system 102, etc.) to generate an intermediate output U. For illustration purposes, Figure 5 As shown, the transposed first output matrix 518 (e.g., including the transposed first output vector) can be added to the input vector X via the skip connection 520 to generate the intermediate output U. For example, the intermediate output U can therefore be represented by the following equation: *,i =X *,i +W2σ(W1LayerNorm(X) *,i ), for i=1,…,C, where U is the matrix of intermediate outputs U, X is the matrix of input vectors X, and C is the number of channels.

[0130] In some non-limiting embodiments or aspects, the intermediate output(s) may be input (e.g., by feature encoding system 102, etc.) to at least one machine learning model (e.g., of machine learning model 104) to generate a second output vector. For purposes of illustration, Figure 5 As shown, the (multiple) intermediate outputs U (e.g., the matrix U of the intermediate outputs U) can be input to the layer normalization layer 522, which can generate at least one normalized intermediate output vector (e.g., the normalized intermediate output matrix LayerNorm(U)) based on the layer normalization. j,* ). For illustration purposes, Figure 5 As shown, the normalized intermediate output matrix LayerNorm(U) j,* It can include columns representing feature channels and rows representing data instance tokens. For illustration purposes, Figure 5 As shown, the normalized intermediate output matrix LayerNorm(U) j,*Each row of can be input to a second MLP model (e.g., a channel mixture (CM) MLP model 524). In some non-limiting embodiments or aspects, the CM MLP model 524 may include at least one fully connected layer and at least one nonlinear activation function. For example, the CM MLP model 524 may include a third fully connected layer represented by a third weight matrix W3, an element-wise nonlinear activation function σ (e.g., a Gaussian error linear unit (GELU) activation function, a sigmoid activation function, etc.), and a fourth fully connected layer represented by a fourth weight matrix W4. In some non-limiting embodiments or aspects, the CM MLP model 524 may generate at least one second intermediate output vector (e.g., an output vector for each row of the matrix input thereto).

[0131] In some non-limiting embodiments or aspects, the second intermediate output vector(s) may be combined (e.g., added, summed, etc.) with the intermediate output(s) U (e.g., by the feature encoding system 102, etc.). For illustration purposes, Figure 5 As shown, the (multiple) second intermediate output vectors can be added to the (multiple) intermediate output U via the skip connection 526 to generate a second output matrix 528 (e.g., feature embedding matrix Y), which can include the second output vectors. For example, the second output matrix 528 (e.g., feature embedding matrix Y) can therefore be represented by the following equation: j,* =U j,* +W4σ(W3LayerNorm(U) j,* ), for j=1,…,S, where S is the number of data instances.

[0132] In some non-limiting embodiments or aspects, all of 504 to 528 together may be collectively referred to as MLP mixer 530 (eg, MLP mixer K). Thus, for a given data instance x (i) In the case of Each feature of is embedded to determine its representation (e.g., by feature encoding system 102, etc.).

[0133] In some non-limiting embodiments or aspects, the prediction model 540 (e.g., at least one classifier model of the machine learning model 104, etc.) can be based on the second output vector (e.g., the second output matrix 528 and / or based on the cascade from Each feature embedding is determined by the data instance x (i) ) to generate at least one prediction (eg, via feature encoding system 102, etc.).

[0134] In some non-limiting embodiments or aspects, the aforementioned machine learning model can be trained (e.g., by feature encoding system 102, etc.). For example, each data instance (and / or at least each data instance in the training set) can be associated with a label y (i) (For example, the i-th data instance x (i) As described in this paper, it can be based on the i-th data instance x (i) Generate predicted classification (e.g., by feature encoding system 102, etc.). (i) and the predicted classification To determine the loss. For illustration purposes, the loss for a classification task can be determined based on the following equation: In some non-limiting embodiments or aspects, parameters of a machine learning model (e.g., weights of a weight matrix, etc.) can be adjusted (e.g., updated) based on the loss (e.g., based on stochastic gradient descent, back propagation, any combination thereof, etc.).

[0135] For illustration purposes, Table 1 shows the area under the curve (AUC) of the disclosed subject matter compared with a transformer model and a graph neural network (GNN) model for two datasets:

[0136] AUC (Dataset 1) AUC (Dataset 2) Converter 91.24 - GNN - 0.8084 Disclosed subject matter 91.16 0.8090

[0137] Table 1

[0138] In Table 1, a dash (-) indicates model failure. As shown in Table 1, the disclosed subject matter has comparable performance to the transformer model on the first data set, and unlike the transformer model, the disclosed subject matter does not fail on the larger second data set (e.g., due to the high complexity of the transformer, the transformer cannot be applied to the large-scale second data set). Additionally, the disclosed subject matter has improved performance compared to the GNN model for the second data set. Thus, the disclosed subject matter achieves comparable or improved performance compared to other models.

[0139] For illustration purposes, Table 2 shows the time complexity of the disclosed subject matter compared to the Transformer model and the GNN model for the second dataset:

[0140] Time complexity (dataset 2) Converter - GNN 66.7s / period Disclosed subject matter 33.2s / period

[0141] Table 2

[0142] In Table 2, a dash (-) indicates a model failure. As shown in Table 2, the disclosed subject matter has a much lower time complexity (e.g., faster in terms of seconds per epoch) than the GNN for the large-scale second dataset, and unlike the transformer model, the disclosed subject matter does not fail in terms of the large-scale second dataset. Thus, the disclosed subject matter achieves improved speed and scalability compared to other models.

[0143] Although the embodiments have been described in detail for the purpose of illustration, it should be understood that such details are used only for that purpose and that the present disclosure is not limited to the disclosed embodiments or aspects, but on the contrary, is intended to cover modifications and equivalent arrangements within the spirit and scope of the appended claims. For example, it should be understood that the present disclosure contemplates that, to the extent possible, one or more features of any embodiment or aspect may be combined with one or more features of any other embodiment or aspect.

Claims

1. A computer-implemented method comprising: receiving, with at least one processor, a data set in a tabular format comprising a plurality of rows and a plurality of columns, wherein each of the plurality of rows represents a respective data instance of a plurality of data instances, wherein each of the plurality of columns represents a respective feature of a plurality of features, wherein each of the plurality of data instances comprises a plurality of values, the plurality of values ​​comprising a respective value associated with each respective feature of the plurality of features; indexing, with at least one processor, each of the plurality of columns to generate a position embedding matrix comprising a plurality of position embedding vectors, wherein each position embedding matrix row in the position embedding matrix comprises a corresponding position embedding vector of the plurality of position embedding vectors associated with a corresponding column of the plurality of columns; grouping, using at least one processor, each of the plurality of columns based on at least one tree model to generate a domain embedding matrix comprising a plurality of domain embedding vectors; generating, with at least one processor, an input vector based on the dataset, the position embedding matrix, and the domain embedding matrix; Inputting the input vector into a first multi-layer perceptron (MLP) model using at least one processor to generate a first output vector; transposing the first output vector using at least one processor to generate a transposed vector; inputting the transposed vector into a second MLP model using at least one processor to generate a second output vector; as well as The second output vector is input into at least one classifier model using at least one processor to generate at least one prediction.

2. The computer-implemented method of claim 1, wherein the at least one prediction comprises at least one predicted label.

3. The computer-implemented method of claim 2, wherein the plurality of data instances comprises a plurality of payment transaction records, wherein the at least one predicted label indicates that a corresponding payment transaction record in the plurality of payment transaction records is predicted to be fraudulent.

4. The computer-implemented method of claim 1 , wherein generating the input vector comprises: At least one row of the data set, at least one position embedding vector of the position embedding matrix, and at least one domain embedding vector of the domain embedding matrix are concatenated to generate the input vector.

5. The computer-implemented method of claim 1 , further comprising: Each value of the plurality of values ​​is embedded, using at least one processor, to generate a dense embedding matrix, each corresponding dense embedding matrix row of the dense embedding matrix comprising a low dimensional representation of the corresponding value.

6. The computer-implemented method of claim 5, wherein generating the input vector comprises generating the input vector based on the dense embedding matrix, the position embedding matrix, and the domain embedding matrix.

7. The computer-implemented method of claim 6, wherein generating the input vector comprises: At least one row of the dense embedding matrix, at least one position embedding vector of the position embedding matrix, and at least one domain embedding vector of the domain embedding matrix are concatenated to generate the input vector.

8. The computer-implemented method of claim 5, wherein each of the plurality of values ​​comprises one of a discrete value or a continuous value, and wherein embedding each discrete value comprises encoding the discrete value using an independent embedding, and wherein embedding each continuous value comprises encoding the continuous value based on scaling the continuous value using a shared embedding.

9. The computer-implemented method of claim 1 , further comprising: modifying, with at least one processor, the input vector by replacing one or more values ​​of the input vector to produce a modified input vector; inputting the modified input vector into the first MLP model using at least one processor to generate a first modified output vector; transposing, with at least one processor, the first modified output vector to generate a modified transposed vector; inputting the modified transposed vector into the second MLP model using at least one processor to generate a second modified output vector; as well as At least one processor is used to adjust parameters of at least one of the first MLP model, the second MLP model, or any combination thereof based on at least one of the first modified output vector, the second modified output vector, the modified input vector, or any combination thereof.

10. The computer-implemented method of claim 1 , further comprising: normalizing the input vector based on layer normalization using at least one processor to generate a normalized input vector, Wherein inputting the input vector into the first MLP model comprises inputting the normalized input vector into the first MLP model.

11. The computer-implemented method of claim 1 , wherein grouping each column based on at least one tree model to generate the domain embedding matrix comprises grouping each column based on a gradient boosted decision tree.

12. A system, comprising: at least one processor; as well as at least one non-transitory computer-readable medium storing instructions that, when executed by the at least one processor, cause the at least one processor to: receiving a data set in a tabular format comprising a plurality of rows and a plurality of columns, wherein each of the plurality of rows represents a respective data instance of a plurality of data instances, wherein each of the plurality of columns represents a respective feature of a plurality of features, wherein each of the plurality of data instances comprises a plurality of values, the plurality of values ​​comprising a respective value associated with each respective feature of the plurality of features; indexing each of the plurality of columns to generate a position embedding matrix comprising a plurality of position embedding vectors, wherein each position embedding matrix row in the position embedding matrix comprises a corresponding position embedding vector of the plurality of position embedding vectors associated with a corresponding column of the plurality of columns; grouping each of the plurality of columns based on at least one tree model to generate a domain embedding matrix including a plurality of domain embedding vectors; generating an input vector based on the dataset, the position embedding matrix, and the domain embedding matrix; Inputting the input vector into a first multi-layer perceptron (MLP) model to generate a first output vector; transposing the first output vector to generate a transposed vector; inputting the transposed vector into a second MLP model to generate a second output vector; as well as The second output vector is input into at least one classifier model to generate at least one prediction.

13. The system of claim 12, wherein the at least one prediction comprises at least one predicted label, wherein the plurality of data instances comprises a plurality of payment transaction records, wherein the at least one predicted label indicates that a corresponding payment transaction record in the plurality of payment transaction records is predicted to be fraudulent.

14. The system of claim 12, wherein generating the input vector comprises: At least one row of the data set, at least one position embedding vector of the position embedding matrix, and at least one domain embedding vector of the domain embedding matrix are concatenated to generate the input vector.

15. The system of claim 12, wherein the instructions, when executed by the at least one processor, further cause the at least one processor to: embedding each value of the plurality of values ​​to generate a dense embedding matrix, each corresponding dense embedding matrix row of the dense embedding matrix comprising a low dimensional representation of the corresponding value, Wherein generating the input vector comprises generating the input vector based on the dense embedding matrix, the position embedding matrix and the domain embedding matrix.

16. The system of claim 15, wherein generating the input vector comprises: At least one row of the dense embedding matrix, at least one position embedding vector of the position embedding matrix, and at least one domain embedding vector of the domain embedding matrix are concatenated to generate the input vector.

17. The system of claim 15, wherein each of the plurality of values ​​comprises one of a discrete value or a continuous value, and wherein embedding each discrete value comprises encoding the discrete value using an independent embedding, and wherein embedding each continuous value comprises encoding the continuous value based on scaling the continuous value using a shared embedding.

18. The system of claim 12, wherein the instructions, when executed by the at least one processor, further cause the at least one processor to: modifying the input vector by replacing one or more values ​​of the input vector to produce a modified input vector; inputting the modified input vector into the first MLP model to generate a first modified output vector; transposing the first modified output vector to generate a modified transposed vector; inputting the modified transposed vector into the second MLP model to generate a second modified output vector; as well as Parameters of at least one of the first MLP model, the second MLP model, or any combination thereof are adjusted based on at least one of the first modified output vector, the second modified output vector, the modified input vector, or any combination thereof.

19. The system of claim 12, wherein grouping each column based on at least one tree model to generate the domain embedding matrix comprises grouping each column based on a gradient boosted decision tree.

20. A computer program product, the computer program product comprising at least one non-transitory computer-readable medium, the at least one non-transitory computer-readable medium comprising program instructions that, when executed by at least one processor, cause the at least one processor to: receiving a data set in a tabular format comprising a plurality of rows and a plurality of columns, wherein each of the plurality of rows represents a respective data instance of a plurality of data instances, wherein each of the plurality of columns represents a respective feature of a plurality of features, wherein each of the plurality of data instances comprises a plurality of values, the plurality of values ​​comprising a respective value associated with each respective feature of the plurality of features; indexing each of the plurality of columns to generate a position embedding matrix comprising a plurality of position embedding vectors, wherein each position embedding matrix row in the position embedding matrix comprises a corresponding position embedding vector of the plurality of position embedding vectors associated with a corresponding column of the plurality of columns; grouping each of the plurality of columns based on at least one tree model to generate a domain embedding matrix including a plurality of domain embedding vectors; generating an input vector based on the dataset, the position embedding matrix, and the domain embedding matrix; Inputting the input vector into a first multi-layer perceptron (MLP) model to generate a first output vector; transposing the first output vector to generate a transposed vector; inputting the transposed vector into a second MLP model to generate a second output vector; as well as The second output vector is input into at least one classifier model to generate at least one prediction.

Images