In-context learning over interactions

Abstract

A computer obtains interaction data and aggregated data for a current interaction. The current interaction relates to a device requesting access to a resource. The computer generates a feature vector using the interaction data and the aggregated data and generates a prompt comprising a set of feature-label pairs. Each feature-label pair comprises a task-specific label and an additional feature vector including additional interaction data and additional aggregated data. The additional interaction data and the additional aggregated data are related to an additional interaction of a same type of classification as the current interaction. The computer loads a pre-trained machine learning model trained to determine predictions for specific tasks. The computer inputs the prompt and a query including the feature vector into the pre-trained machine learning model. The computer determines a task-specific prediction for the query and the prompt, the task-specific prediction specifying a response state to the current interaction.

Description

IN-CONTEXT LEARNING OVER INTERACTIONSCROSS-REFERENCES TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Provisional Application No.63 / 564,835, filed March 13, 2024, which is herein incorporated by reference in its entirety for all purposes.BACKGROUND

[0002] Interaction data for interactions between devices (e.g., data access requests, transactions, etc.) can have many different features and can include different data based on the interaction type. Predictions can be made based on the features using different machine learning models to make the interaction process more efficient. A different machine learning model can be trained for each different t pe of interaction.

[0003] Typically, to improve performance of all the different machine learning models, more data is leveraged and the models are made larger. This has shown success across domains, especially in natural language processing with large language models (LLMs).

[0004] However, due to the interaction data used, more data and larger model may not improve the model performance because it is not realistic to predict arbitrary numbers. Simply using more data and larger models may not improve the prediction performance for interaction data. Further, cunent uses of natural language models need a lot of fine-tuning and many training heads for different tasks, even if the model is pretrained.

[0005] Embodiments of the disclosure address this problem and other problems individually and collectively.SUMMARY

[0006] One embodiment is related to a method that includes a computer obtaining interaction data and aggregated data for a current interaction. The current interaction relates to a device requesting access to a resource. The computer can generate a feature vector using the interaction data and the aggregated data. The computer can then generate a prompt comprising a set of feature-label pairs. Each set of feature-label pairs can include a task-specific label and an additional feature vector. The additional feature vector can include additional interaction data and additional aggregated data. The additional interaction data and the additional aggregated data are related to an additional interaction of a same type of classification as the current interaction. The computer can load, into memory, a pre-trained machine learning model trained to determine predictions for specific tasks. The computer can input the prompt and a query including the feature vector into the pre-trained machine learning model. The computer can then determine, using the pre-trained machine learning model, a task-specific prediction for the query and the prompt. The task-specific prediction can specify a response state to the current interaction.

[0007] In some embodiments, the computer can generate the query' using the feature vector.

[0008] In some embodiments, the computer can obtain a plurality of historical interaction data, a plurality of historical aggregated data, and a plurality of historical task specific labels for a plurality of historical interactions. The computer can generate a plurality of historical feature vectors using the plurality of historical interaction data, the plurality of historical aggregated data, and the plurality’ of historical task specific label. The computer can then generate a historical prompt and a historical query using the plurality of historical feature vectors. The computer can train the pre-trained machine learning model using the historical prompt and the historical query'.

[0009] In some embodiments, the interaction data includes an indication of a type of interaction of a plurality of types of interactions. Generating the feature vector using the interaction data and the aggregated data can include the following steps. The computer can generate an interaction data portion of the feature vector. The interaction data portion includes a plurality' of interaction type entries. The computer can set values of an interaction type entry of the plurality of interaction type entries that corresponds to the type of interaction to values of the interaction data. The computer can set values of other interaction type entries of the plurality of interaction type entries that do not correspond to the type of interaction to a default value.

[0010] Another embodiment is related to a computer comprising a processor and a computer-readable medium coupled to the processor. The computer-readable medium comprises code executable by the processor for implementing any method herein.

[0011] Another embodiment is related to a system comprising an interaction database, which stores a plurality of interaction data and a plurality of aggregated data, and an evaluation computer in communication with the interaction database. The evaluation computer includes a processor, a memory, and a computer-readable medium coupled to the processor. The computer-readable medium comprises code executable by the processor for implementing any method herein.

[0012] Further details regarding embodiments of the disclosure can be found in the Detailed Description and the Figures.BRIEF DESCRIPTION OF THE DRAWINGS

[0013] FIG. 1 shows a block diagram of a system according to embodiments.

[0014] FIG. 2 shows a block diagram illustrating unified data according to embodiments.

[0015] FIG. 3 shows a block diagram illustrating model inputs to a machine learning model according to embodiments.

[0016] FIG. 4 shows a flowchart of training and inference methods according to embodiments.

[0017] FIG. 5 shows a block diagram illustrating a machine learning model according to embodiments.

[0018] FIG. 6 shows a flow diagram illustrating a prediction determination method according to embodiments.

[0019] FIG. 7 shows a block diagram illustrating masked unified data according to embodiments.

[0020] FIG. 8 shows a block diagram of components of an evaluation computer according to embodiments.

[0021] FIG. 9 shows a block diagram of components of a computer according to embodiments.TERMS

[0022] Prior to discussing embodiments of the disclosure, some terms can be described in further detail.

[0023] A “machine learning computer"’ can include a device that creates, trains, and / or otherwise manipulates models. A machine learning computer can train a machine learning model.

[0024] A "‘machine learning model” (ML model) can include a software module configured to be run on one or more processors to provide a classification or numerical value of a property7of one or more samples. An ML model can include various parameters (e.g., for coefficients, weights, thresholds, functional properties of function, such as activation functions). As examples, an ML model can include at least 10, 100. 1,000, 5,000. 10.000, 50,000, 100,000, or one million parameters. An ML model can be generated using sample data (e.g., training samples) to make predictions on test data. Various number of training samples can be used, e.g., at least 10, 100, 1.000, 5,000, 10,000. 50,000, 100,000, or at least 200,000 training samples. One example is an unsupervised learning model such as hidden Markov model (HMM), clustering (e.g., hierarchical clustering, k-means, mixture models, model-based clustering, density-based spatial clustering of applications with noise (DBSCAN), and OPTICS algorithm), approaches for learning latent variable models such as Expectation-maximization algorithm (EM), method of moments, and blind signal separation techniques (e.g., principal component analysis, independent component analysis, nonnegative matrix factorization, singular value decomposition), and anomaly detection (e.g., local outlier factor and isolation forest). Another example ty pe of model is supervised learning that can be used with embodiments of the present disclosure. Example supervised learning models may include different approaches and algorithms including analytical learning, statistical models, artificial neural network (e.g. including convolutional and / or transformer layers) that may have 1-10 layers as examples, recurrent neural network (e.g., long short term memory7, LSTM), boosting (meta-algorithm), bootstrap aggregating (bagging) such as random forests, support vector machine (SVM), support vector (SVR), Bayesian statistics, case-based reasoning, decision tree learning, inductive logic programming, linear regression, logistic regression, Gaussian process regression, genetic programming, group method of data handling, kernel estimators, learning automata, learning classifier systems, minimum message length (decision trees, decision graphs, etc ), multilinear subspacelearning, naive Bayes classifier, maximum entropy classifier, conditional random field, nearest neighbor algorithm, probably approximately correct learning (PAC) learning, ripple down rules, a knowledge acquisition methodology, symbolic machine learning algorithms, subsymbolic machine learning algorithms, minimum complexity machines (MCM), ordinal classification, data pre-processing, handling imbalanced datasets, statistical relational learning, or Proaftn (a multicriteria classification algorithm), or an ensemble of any of these types. Supervised learning models can be trained in various ways using various cost / loss functions that define the error from the known label (e.g., least squares and absolute difference from known classification) and various optimization techniques, e.g., using backpropagation, steepest descent, conjugate gradient, and Newton and quasi-Newton techniques.

[0025] A '‘deep neural network (DNN)” may be a neural network in which there are multiple layers between an input and an output. Each layer of the deep neural network may represent a mathematical manipulation used to turn the input into the output. In particular, a “recurrent neural network (RNN)” may be a deep neural network in which data can move forward and backward between layers of the neural network.

[0026] An “encoder” can process an input sequence to create a vector. Encoders can process input sequences to generate embedding vectors. An encoder can encode data from a higher dimensionality to a lower dimensionality7. A “decoder” can process a vector to create an output sequence. Decoders can process embedding vectors, or vectors modified therefrom, to generate an output sequence. A decoder can decode data from a lower dimensionality to a higher dimensionality'. Both encoders and decoders can be separate, fully connected neural networks. Encoders and decoders may be recurrent neural networks (RNNs) or variants thereof (e.g., long-short term memory7(LSTM), gated recurrent units (GRUs), etc.) and convolutional neural networks (CNNs), as well as transformer models. An encoder-decoder model can include several encoders and several decoders.

[0027] A “model database” may include a database that can store machine learning models. Machine learning models can be stored in a model database in a variety7of forms, such as collections of parameters or other values defining the machine learning model. Models in a model database may be stored in association with keywords that communicate some aspect of the model. For example, a model used to evaluate news articles may be stored in a model database in association with the keywords “news,” “propaganda,” and“information.” A machine learning computer can access a model database and retrieve models from the model database, modify models in the model database, delete models from the model database, or add new models to the model database.

[0028] A “feature vector” may include a set of measurable properties (or “features”) that represent some object or entity. A feature vector can include collections of data represented digitally in an array or vector structure. A feature vector can also include collections of data that can be represented as a mathematical vector, on which vector operations such as the scalar product can be performed. A feature vector can be determined or generated from input data. A feature vector can be used as the input to a machine learning model, such that the machine learning model produces some output or classification. The construction of a feature vector can be accomplished in a variety of ways, based on the nature of the input data. For example, for a machine learning classifier that classifies words as correctly spelled or incorrectly spelled, a feature vector corresponding to a word such as “LOVE” could be represented as the vector (12, 15, 22, 5), corresponding to the alphabetical index of each letter in the input data word. For a more complex “input,” such as a human entity, an exemplary feature vector could include features such as the human's age, height, weight, a quantitative representation of relative happiness, etc. Feature vectors can be represented and stored electronically in a feature store. Further, a feature vector can be normalized (i.e., be made to have unit magnitude). As an example, the feature vector (12. 15, 22, 5) corresponding to “LOVE” could be normalized to approximately (0.40, 0.51, 0.74. 0.17).

[0029] An “interaction” may include a reciprocal action or influence. An interaction can include a communication, contact, or exchange between parties, devices, and / or entities. Example interactions include a transaction between two parties and a data exchange between two devices. In some embodiments, an interaction can include a user requesting access to secure data, a secure webpage, a secure location, and the like. In other embodiments, an interaction can include a payment transaction in which two devices can interact to facilitate a payment.

[0030] An “access device” may be any suitable device that provides access to a remote system. An access device may also be used for communicating with a coordination computer, a communication network, or any other suitable system. An access device may generally be located in any suitable location, such as at the location of a merchant. An accessdevice may be in any suitable form. Some examples of access devices include POS or point of sale devices (e.g., POS terminals), cellular phones, personal digital assistants (PDAs), personal computers (PCs), tablet PCs, hand-held specialized readers, set-top boxes, electronic cash registers (ECRs), vending machines, automated teller machines (ATMs), virtual cash registers (VCRs), kiosks, security' systems, access systems, and the like.

[0031] An access device may use any suitable contact or contactless mode of operation to send or receive data from, or associated with, a mobile communication or payment device. For example, access devices can have card readers that can include electrical contacts, radio frequency (RF) antennas, optical scanners, bar code readers, or magnetic stripe readers to interact with portable devices such as payment cards.

[0032] The term ‘‘resource'’ can include any asset that may be used or consumed. For example, the resource may be an electronic resource (e.g.. stored data, received data, a computer account, a network-based account, an email inbox), a physical resource (e g., a tangible object, a building, a safe, or a physical location), or other electronic communications between computers (e.g., a communication signal corresponding to an account for performing a transaction).

[0033] A “resource provider” may be an entity that can provide a resource such as goods, services, information, and / or access. Examples of resource providers includes merchants, data providers, transit agencies, governmental entities, venue and dwelling operators, etc.

[0034] An “authorization request message” may be an electronic message that requests authorization for an interaction. In some embodiments, it is sent to a transaction processing computer and / or an issuer of a payment card to request authorization for a transaction. An authorization request message according to some embodiments may comply with International Organization for Standardization (ISO) 8583, which is a standard for systems that exchange electronic transaction information associated with a payment made by a user using a payment device or payment account. The authorization request message may include an issuer account identifier that may be associated with a payment device or payment account. An authorization request message may also comprise additional data elements corresponding to “identification information” including, by way of example only: a service code, a CVV (card verification value), a dCVV (dynamic card verification value), a PAN (primary account number or “account number”), a payment token, a user name, an expirationdate, etc. An authorization request message may also comprise “transaction information,"’ such as any information associated with a current transaction, such as the transaction value, merchant identifier, merchant location, acquirer bank identification number (BIN), card acceptor ID, information identifying items being purchased, etc., as well as any other information that may be utilized in determining whether to identify and / or authorize a transaction.

[0035] An “authorization response message” may be a message that responds to an authorization request. In some cases, it may be an electronic message reply to an authorization request message generated by an issuing financial institution or a transaction processing computer. The authorization response message may include, by way of example only, one or more of the following status indicators: Approval - transaction was approved; Decline - transaction was not approved; or Call Center -- response pending more information, resource provider is to call the toll-free authorization phone number. The authorization response message may also include an authorization code, which may be a code that a credit card issuing bank returns in response to an authorization request message in an electronic message (either directly or through the transaction processing computer) to the merchant's access device (e.g., POS equipment) that indicates approval of the transaction. The code may serve as proof of authorization.

[0036] An “authorizing entity” may be an entity7that authorizes a request. Examples of an authorizing entity may be an issuer, a governmental agency, a document repository, an access administrator, etc. An authorizing entity may operate an authorizing entity computer. An “issuer” may refer to a business entity (e.g., a bank) that issues and optionalfy maintains an account for a user. An issuer may also issue payment credentials stored on a user device, such as a cellular telephone, smart card, tablet, or laptop to the consumer, or in some embodiments, a portable device.

[0037] “Interaction data” can include data related to an interaction and / or related to processing of the interaction. Interaction data can include data for authorization interactions, clearing interactions, and any other type of interaction. Interaction data can include amounts, time stamps, user identifiers, resource provider identifiers, transporter computer identifiers, network processing computer identifiers, authorizing entity identifiers, interaction types, interaction outcomes, and / or any other data related to an interaction between a user operatinga user device and resource provider operating a resource provider computer for the user to obtain a resource from the resource provider.

[0038] “Aggregated data” can include data related to entities, computers, and accounts that are associated with an interaction. Aggregated data can relate to things involved in an interaction as indicated by interaction data. Aggregated data can include data that relates to a user account (e.g., a primary user account), a resource provider, a transport computer, and an authorized entity, that are involved in an interaction. For example, aggregated data related to a user account can include historical data associated with the user account involved in the interaction. Aggregated data related to a resource provider can include data related to the resource provider and / or the resource provider computer, such as a number of interactions performed by the resource provider in the past day. Aggregated data related to a transport computer can include data related to an acquirer and / or the transport computer, such as a number of interactions processed by the transport computer in the past month. Aggregated data related to an authorizing entity7can include data related to the authorizing entity and / or an authorizing entity computer, such as data that was used in the determination of whether or not to authorize the interaction.

[0039] A “prompt” can include an input to a machine learning model. A prompt can include a submission to a machine learning model, such as a neural network or a large language model, to modify processing to determine a response to a query7. A prompt can be crafted to guide the machine learning model’s behavior. Prompts can includes questions, instructions, contextual information, few-shot examples, and partial input for a machine learning model to complete or continue.

[0040] A “query” can include a request. A query7can include a request for information. A query7can seek information retrieval. A query7can be a submission to a machine learning model, such as a neural network or a large language model, to receive a response about the query.

[0041] A “task” can include a specific problem or prediction that is to be evaluated. A task can be performed by7a machine learning model. A task can be a process with a particular goal. Example tasks can include 1) predicting whether or not an interaction is authorized, 2) predicting whether or not an interaction is cleared. 3) predicting whether or not an interaction is fraudulent, 4) determining a predicted score (e.g., a credit score) for a user involved in an interaction, 5) determining a category classification for an interaction (e.g., a spending-category ), 6) determining a category classification for a user involved in an interaction (e.g., a user architype), and 7) determining an anomaly score for an interaction.

[0042] A "label” can include data that identifies associated data. A label can indicate information about an associated feature vector. A label can indicate a particular classification for an associated feature vector. For example, a label can indicate a classification of ‘‘not fraudulent” for a feature vector that includes interaction data for an interaction. As another example, a label can indicate a classification for a spending-category of “transportation” or “entertainment” that is to be posted to a user’s account when the associated interaction is posted to the user’s account.

[0043] A “task-specific label” can include identification data that is specific to a particular problem or prediction. A task-specific label can include a label that is associated with a particular task. Each task can relate to a plurality of task-specific labels. A taskspecific label can indicate both the label for the data as well as the task itself. For example, a task-specific label can be “authorized” which can be specific to a task of determining whether or not to authorize an interaction.

[0044] A “processor” may include a device that processes something. In some embodiments, a processor can include any suitable data computation device or devices. A processor may comprise one or more microprocessors working together to accomplish a desired function. The processor may include a CPU comprising at least one high-speed data processor adequate to execute program components for executing user and / or systemgenerated requests. The CPU may be a microprocessor such as AMD's Athlon, Duron and / or Opteron; IBM and / or Motorola's PowerPC; IBM's and Sony's Cell processor; Intel's Celeron, Itanium, Pentium, Xeon, and / or XScale; and / or the like processor(s).

[0045] A “memory” may be any suitable device or devices that can store electronic data. A suitable memory may comprise a non-transitory computer readable medium that stores instructions that can be executed by a processor to implement a desired method. Examples of memories may comprise one or more memory chips, disk drives, etc. Such memories may operate using any suitable electrical, optical, and / or magnetic mode of operation.

[0046] A “server computer” may include a pow erful computer or cluster of computers. For example, the server computer can be a large mainframe, a minicomputercluster, or a group of servers functioning as a unit. In one example, the server computer may be a database server coupled to a Web server. The server computer may comprise one or more computational apparatuses and may use any of a variety of computing structures, arrangements, and compilations for servicing the requests from one or more client computers.DETAILED DESCRIPTION

[0047] Embodiments of the disclosure can provide for an in-context learning framework to 1) improve model performance in various tasks and 2) make a trained model applicable to different tasks without fine-tuning to the different tasks by leverage data from different stages of interactions. To do so, embodiments can provide for 1) a method to unify interactions from different sources and stages into unified data, 2) a model capable of processing the unified data (e.g., represented as a feature vector), and 3) a set of in-context learning objectives so that different tasks can be performed by the same machine learning model.

[0048] By utilizing this framework, embodiments can leverage a large number of interaction data to pretrain a single larger model (e.g., larger than current task-specific models). The model can have improved performance compared to single task mased models, such as Smarter Posting, and does not require fine-tuning and can apply to any new tasks with a hint of a few examples rather than many examples to train new tasks.

[0049] Some previous methods utilize few-shot learning. Few-shot learning can allow a model to learn to predict from only a few examples (e.g., a limited amount of training data). An idea behind few-shot learning is learn to leam. Rather than training on various samples from a task and then perform inference on samples from the same task, few shot learning trams a model on various tasks, and learns how to leam from the few samples in a task. Its success shows the possibility to train one model for different tasks and generate task-specific predictions from few examples of the task. Embodiments provide for advantages over previous methods that utilize few-shot learning.

[0050] Machine learning models, according to embodiments, can utilize in-context learning, rather than few-shot learning. A few-shot learning model needs to update model weights based on the few examples provided to the few-shot learning model. However, in the in-context learning utilized in some embodiments, a machine learning model does not need to update model weights based on the few examples, rather the machine learning model canpredict based on the few examples and the current example by considering both as the input. Not needing to update the machine learning model in such a manner during use is advantageous since a computer can save computational resources each time the machine learning model is used.

[0051] Large language models (LLM) are based on transformer models and are pretrained on large amounts of text data. A training objective in a typical large language model is next word prediction. For few-shot learning, a large language model can accept a few examples and a question together and can generate a correct response, which is referred to as in-context learning.

[0052] Embodiments can provide for a computer that can generate unified data structures for feature vectors for an in-context learning model. The feature vectors can represent features for interaction data for a plurality of different learning tasks and types of interactions. The computer can generate the feature vectors from interaction data and aggregated data. The interaction data relates to data in the current interaction, while aggregated data relates to data peripherally related to the interaction data.

[0053] The computer can utilize a feature vector for the current interaction and feature vectors from additional interactions that relate to the current interaction in some manner. The computer can utilize a pre-trained machine learning model to determine a taskspecific prediction that specifying a response state (e.g., authorized or not authorized, fraudulent or not fraudulent, a risk score, etc.) to the current interaction.I. EXAMPLE NETWORK ARCHITECTURE

[0054] Embodiments can utilize the systems described herein to train, maintain, and utilize a machine learning model capable of generating a prediction (e.g., a task-specific label) for a current interaction based on the current interaction and additional interactions.

[0055] FIG. 1 shows a system 100 according to embodiments of the disclosure. The system 100 comprises an evaluation computer 102, a interaction database 104, a downstream device 106, a network processing computer 108, a user device 110, an access device 112, a resource provider computer 114, a transport computer 116, an authorizing entity computer

[0056] The evaluation computer 102 can be in operative communication with the interaction database 104 and the downstream device 106. The network processing computer 108 can be in operative communication with the interaction database 104, the transport computer 116, and the authorizing entity computer 118. The transport computer 116 can be in operative communication with the resource provider computer 114, which can be in operative communication with the access device 112 and the user device 110. The access device 112 can be in operative communication with the user device 110.

[0057] For simplicity of illustration, a certain number of components are shown in FIG. 1. It is understood, how ever, that embodiments of the invention may include more than one of each component. In addition, some embodiments of the invention may include few er than or greater than all of the components shown in FIG. 1.

[0058] Messages between the devices in the system 100 illustrated in FIG. 1 can be transmitted using a secure communications protocols such as, but not limited to. File Transfer Protocol (FTP); HyperText Transfer Protocol (HTTP); Secure Hypertext Transfer Protocol (HTTPS). SSL, ISO (e.g., ISO 8583) and / or the like. The communications network may include any one and / or the combination of the following: a direct interconnection; the Internet; a Local Area Network (LAN); a Metropolitan Area Network (MAN); an Operating Missions as Nodes on the Internet (OMNI); a secured custom connection; a Wide Area Network (WAN); a wireless network (e.g., employing protocols such as, but not limited to a Wireless Application Protocol (WAP). I-mode, and / or the like); and / or the like. The communications network can use any suitable communications protocol to generate one or more secure communication channels. A communications channel may, in some instances, comprise a secure communication channel, which may be established in any known manner, such as through the use of mutual authentication and a session key, and establishment of a Secure Socket Layer (SSL) session.

[0059] The evaluation computer 102 can create, train, utilize, and / or otherwise manipulate machine learning models to evaluate data. The evaluation computer 102 can obtain interaction data and aggregated data that is stored in the interaction database 104. The evaluation computer 102 can generate a feature vector (e.g., in a unified data format) using the interaction data and the aggregated data for an interaction. The evaluation computer 102 can utilize a pre-trained machine learning model to determine a task-specific prediction for a prompt and a query created from interaction data and aggregated data from the interactiondatabase 104. In some embodiments, the evaluation computer 102 can train the machine learning model using interaction data and aggregated data obtained from the interaction database 104.

[0060] The interaction database 104 can store interaction data and aggregated data obtained from the network processing computer 108. The interaction database 104 can include any suitable database. The database may be a conventional, fault tolerant, relational, scalable, secure database such as those commercially available from Oracle™ or Sybase™.

[0061] The downstream device 106 can include a device that can utilize the machine learning models trained by the evaluation computer 102. The downstream device 106 can obtain inference data from the evaluation computer 102 based on a query or can generate inference data using the machine learning model trained by the evaluation computer 102.

[0062] The user device 110, the access device 112, the resource provider computer 114, the transport computer 116. the network processing computer 108. and the authorizing entity computer 118 can process interactions between users of user devices and resource providers of resource provider computers.

[0063] The user device 110 can include one or more computers, portable computers, laptop computers, tablet computers, mobile devices, cellular phones, wearable devices (e.g., watches, glasses, lenses, clothing, etc.), personal digital assistants (PDAs), Internet of Things (loT) devices, and / or the like. The user device 110 can initiate interactions (e.g., transactions) with resource provider computers and / or access devices. For example, the user device 110 can select one or more items for the interaction at a resource provider location (e.g., a grocery' store). During checkout, the user can be instructed to tap (e.g., bring into near-field communication range) the user device 110 against the access device 112.

[0064] The access device 112 can include a device operated by a resource provider. The access device 112, for example, can include a mobile device, a POS terminal, a laptop, etc. The access device 112 can communicate with another device (e g., a user device 110) to perform an interaction. During the interaction, the access device 112 can receive credentials from the user device and can provide interaction data to the resource provider computer 114 for authorization of the interaction. In some embodiments, the access device 112 can generate an authorization request message comprising at least the interaction data. The access device 112 can provide the authorization request message to the resource provider computer 114.

[0065] The resource provider computer 114 can include any suitable computational apparatus operated by a resource provider (e.g., a merchant). In some embodiments, the resource provider computer 114 may include one or more server computers that may host one or more websites associated with the resource provider (e.g., a merchant). In some embodiments, the resource provider computer 114 may be configured to send data to the network processing computer 108 via the transport computer 116 as part of a payment verification and / or authentication process for a transaction between the user (e.g., consumer) and the resource provider. The resource provider computer 114 may also be configured to generate authorization request messages for transactions between a resource provider and a user, and route the authorization request messages to the authorizing entity computer 118 for transaction processing.

[0066] The transport computer 116 can include a server computer. The transport computer 116 may be associated with an acquirer, which may be an entity (e.g., a commercial bank) that has a business relationship with a particular merchant or other entity. Some entities can perform both issuer and acquirer functions. Some embodiments may encompass such single entity issuer-acquirers.

[0067] The network processing computer 108 can include a sen’ er computer. The network processing computer 108 may be disposed between the transport computer 116 and the authorizing entity' computer 118. The network processing computer 108 may include data processing subsystems, networks, and operations used to support and deliver authorization senices, exception file services, and clearing and settlement services. For example, the network processing computer 108 may7comprise a server coupled to a network interface (e g., by an external communication interface), and databases of information. The network processing computer 108 may be representative of a transaction processing network. An exemplary transaction processing network may include VisaNet™. Transaction processing networks such as VisaNet™ are able to process credit card transactions, debit card transactions, and other ty pes of commercial transactions. VisaNet™, in particular, includes a VIP system (Visa Integrated Payments system) which processes authorization requests and a Base II system which performs clearing and settlement services. The network processing computer 108 may use any suitable wired or wireless network, including the Internet.

[0068] The network processing computer 108 can store interaction data into the interaction database 104 for interactions as the interactions are processed by the network processing computer 108.

[0069] The authorizing entity computer 118 can include a server computer operated by an authorizing entity. The authorizing entity computer 118 may be associated with an authorizing entity, which may be an entity that authorizes a request. An example of an authorizing entity may be an issuer, which may include an entity (e.g., a bank) that maintains an account for a user. An issuer may also issue and manage an account associated with the user device 110.

[0070] Interactions can include different types of interactions. An interaction can be an authorization interaction, a clearing interaction, etc. An interaction can be a transaction such as a payment transaction (e.g., for the purchase of a good or service), an access transaction (e.g., for access to a transit system), or any other suitable transaction.

[0071] As an illustrative example of an interaction that is an authorization interaction, a user of the user device 110 can conduct an interaction at the resource provider (e.g., a merchant) using the user device 110. The user device 110 can interact with the access device 112 at a resource provider location associated with the resource provider computer 114. For example, the user may tap the user device 110 against an NFC reader in the access device 112. Alternately, the user device 110 can indicate payment account information to the resource provider computer 114 electronically, such as in an online transaction. In some cases, the user device 110 may transmit an account identifier, such as a payment token, to the resource provider computer 114 or to the access device 112.

[0072] In order to authorize the interaction, an authorization request message may be generated by the access device 112 or the resource provider computer 114 and then forwarded to the transport computer 116 (e.g., an acquirer computer). After receiving the authorization request message, the authorization request message is then sent to the network processing computer 108. The network processing computer 108 then forwards the authorization request message to the corresponding authorizing entity computer 118 associated with an authorizing entity associated with the user's payment account.

[0073] After the authorizing entity computer 118 receives the authorization request message, the authorizing entity computer 118 can determine whether or not to authorize theinteraction. The authorizing entity computer 118 can generate an indication of whether or not the interaction is authorized. The authorizing entity computer 118 can generate an authorization response message comprising the indication and can provide the authorization response message to the network processing computer 108 to indicate whether the current transaction is authorized (or not authorized). The network processing computer 108 can then provide the authorization response message to the transport computer 116. In some embodiments, network processing computer 108 may decline the interaction even if the authorizing entity computer 118 has authorized the interaction, for example depending on a value of the fraud risk score.

[0074] The authorizing entity computer 118 can store interaction data related to the interaction into the interaction database 104. The interaction data can include an amount, a date, a time, an account number (e.g., a primary account number (PAN)), a user identifier, a user device identifier, a resource provider identifier, an authorizing entity computer identifier, a transport computer identifier, a network processing computer identifier, a list of resources involved in the interaction, the indication of whether or not the interaction is authorized, and / or any other data related to the interaction and / or the processing of the interaction.

[0075] The transport computer 116 can then provide the authorization response message to the resource provider computer 114. After the resource provider computer 114 receives the authorization response message, the resource provider computer 114 can provide the authorization response message to the user of the user device 110. The response message may be displayed by the access device 112 to the user, or may be printed out on a physical receipt. Alternately, if the interaction is an online interaction, the resource provider computer 114 may provide a web page or other indication of the authorization response message as a virtual receipt to the user device 110.

[0076] At the end of the day, a clearing interaction and settlement interaction process can be conducted by the network processing computer 108. A clearing interaction process is a process of exchanging financial details between an acquirer and an authorizing entity to facilitate posting to a user's payment account and reconciliation of the user's settlement position. The clearing interaction is an additional example of an interaction that can be processed by the system. The network processing computer 108 can store data relating to clearing interactions into the interaction database 104.II. UNIFYING DATA INTO A FEATURE VECTOR

[0077] Interactions of different types can have different sets of features. For example, an interaction that is a transaction can be an authorization interaction, a clearing interaction, etc. and can have different features for each stage of processing the transaction. To solve a technical problem of having a machine learning model that is applicable to all interaction data, a computer can unify the interactions into a same format of feature vector. FIG. 2 shows a block diagram illustrating unified data according to embodiments.

[0078] FIG. 2 includes interaction data 202 for an interaction that can be utilized to generate unified data for the machine learning model. The interaction data 202 can be interaction data for an authorization interaction, a clearing interaction, etc. for an interaction that can be processed by the system illustrated in FIG. 1.

[0079] Interaction data 202 can include interaction identifiers, amounts, time stamps, user identifiers, user device identifiers, account numbers, resource provider identifiers, resource provider computer identifiers, acquirer identifiers, transporter computer identifiers, network processor identifiers, network processing computer identifiers, authorizing entity identifiers, authorizing entity computer identifiers, interaction types, interaction outcomes, and / or any other data related to an interaction between a user operating a user device and resource provider operating a resource provider computer for the user to obtain a resource from the resource provider. The interaction data 202 can include data from interactions performed by an interaction processing system and can be stored in an interaction database.

[0080] Aggregated data can include data that relates to entities, devices, and accounts involved in the interaction. Aggregated data can include data that is peripheral to an interaction but related to the interaction. Aggregated data can include data related to historical interactions. Aggregated data can include statistical data from previous interactions. For example, aggregated data can include interaction velocity, interaction volume, average interaction amounts, etc.

[0081] The unified data can be created as a feature vector 200 illustrated in FIG. 2 includes an interaction data portion 220 and an aggregated data portion 230. A computer can generate the feature vector 200 from the interaction data 202 and from aggregated data that is related to the interaction data 202. For example, the computer can obtain aggregated data from the interaction database using an interaction identifier from the interaction data 202.

[0082] The interaction data portion 220 can include data obtained from the interaction data 202. The interaction data portion 220 can include one or more segments that relate to different types of interactions. For example, the interaction data portion 220 can include interaction authorization data 222 and interaction clearing data 224. The interaction data portion 220 can include additional segments (not shown) relating to other types of interactions.

[0083] When generating the feature vector 200, the computer can determine the type of interaction of the interaction data 202. For example, the interaction data 202 can include an indicator that indicates the ty pe of interaction represented by the interaction data 202. The computer can input the interaction data 202 into a segment of the interaction data portion 220 that matches the type of interaction. The computer can also format the interaction data 202 into a format indicated by the segment of the interaction data portion 220, if the formats are different.

[0084] For example, the interaction data 202 can include an indicator that indicates that the interaction data 202 is for the type of interaction of authorization integration. The computer can input the interaction data 202 into the interaction authorization data 222 segment of the interaction data portion 220.

[0085] For the segments of the interaction data portion 220 that do not correspond to the type of interaction of the interaction data 202, computer can assign a default value for each field indicating that a specific value is missing. Whenever there is a value in the feature vector from the interaction data, the value can be kept in the feature vector, otherwise, a default missing value token can be inserted for the field. The default values can also be used in pretraining tasks for masking purposes when the computer trains the machine learning model. For example, an authorization interaction will not have clearing interaction values, so the clearing interaction values can be set to their respective default values, but if some of the values in the authorization are a value of "‘None;’ then the values can be kept as “None.” As such, the default value can act as identifier as well as a null value that both 1) identifies what the data type is and 2) identifies that the data value is null.

[0086] The interaction authorization data 222 can include an indicator that indicates that the interaction is an authorization interaction. The interaction authorization data 222 can include interaction data that is related to authorization interactions. For example, the interaction authorization data 222 can include authorization request message data and / orother data related to requesting and / or obtaining authorization for an interaction. The authorization request message data can include a date, a time, an amount, a user identifier, a resource provider identifier, and a user credential. The user credential can be a password, a passcode, or a secret message. As another example, user credentials may include any information associated with and / or identifying an account (e.g., a payment account and / or payment device associated with the account). Examples of account information may include an account identifier such as a primary account number (PAN), a token, a subtoken, a gift card number or code, a prepaid card number or code, a user name, an expiration date, a CVV (card verification value), a dCVV (dynamic card verification value), a CVV2 (card verification value 2), etc. An example of a PAN is a 16-digit number, such as “4147 0900 0000 1234”.

[0087] The interaction clearing data 224 can include an indicator that indicates that the interaction is a clearing interaction. The interaction clearing data 224 can include interaction data that is related to clearing interactions. For example, the interaction clearing data 224 can include clearing related data, settlement related data, batch data, etc.

[0088] Examples, of interaction authorization data and interaction clearing data are provided as segments of the interaction data portion 220, but it is understood that the interaction data portion 220 can include fields for other types of interactions.

[0089] The aggregated data portion 230 can include data obtained from the aggregated data that is related to the interaction data 202 in some manner. The aggregated data portion 230 can include one or more segments that relate to different categories of aggregated data. For example, the aggregated data portion 230 can include aggregated account data 232, aggregated resource provider data 234, aggregated transport data 236, and aggregated authorizing entity data 238. The aggregated data portion 230 can include additional segments (not shown) relating to other categories of aggregated data.

[0090] When generating the feature vector 200, the computer can obtain aggregated data from the interaction database that can include aggregated data for the different categories of aggregated data in the aggregated data portion 230.

[0091] Each category' of aggregated data can include information relating to a particular aspect of the interaction data 202. For example, the aggregated data portion 230 can include the aggregated account data 232, which can include data relating to an accountthat is included in the interaction data 202. The data relating to the account can include interaction history data, account balance data, account numbers, account age, account type, account holder data, account manager data, etc. The aggregated account data 232 can include historical data associated with the account (e.g., a primary account number (PAN)) involved in the interaction. The aggregated account data 232 can include, for example, a total amount spent from the account by the user associated with the account in the last week.

[0092] The aggregated data portion 230 can also include aggregated resource provider data 234. The aggregated resource provider data 234 can include data associated with the resource provider that is involved in the interaction. For example, the aggregated resource provider data 234 can include a number of interactions performed by the resource provider in the past day.

[0093] The aggregated data portion 230 can also include aggregated transport data 236. The aggregated transport data 236 can include data associated with a transport computer that may be operated by an acquirer that is associated with the resource provider. The aggregated transport data 236 can include, for example, a number of interactions processed by the transport computer in the past month.

[0094] The aggregated data portion 230 can also include aggregated authorizing entity data 238. The aggregated authorizing entity data 238 can include data associated with an authorizing entity computer that can determine whether or not to authorize the interaction between the user and the resource provider. The aggregated authorizing entity data 238 can include, for example, data that was used in the determination of whether or not to authorize the interaction.

[0095] As an example, since an interaction can involve multiple parties to come to a final decision (e.g., an authorizing entity7may need to decide whether to authorize the interaction request or post an authorized interaction, and an acquirer may need to decide whether to clear an interaction). The decision may also need additional aggregated data, such as historical data, related to the primary account number, the resource provider computer, the transport computer, and the authorizing entity computer. The computer can unify each interaction for different interaction ty pes into the same feature vector format containing all information.III. MACHINE LEARNING MODEL

[0096] Predictions can be generated using interaction data and aggregated data. A computer can generate predictions about a specific task (e.g., predict whether or not an interaction is authorized, for authorization interactions) using the interaction data and the aggregated data. A machine learning model can generate task-specific predictions based on the interaction data and the aggregated data.A. Model Overview

[0097] The machine learning model can be a neural network model, a large language model, or other suitable type of model capable of generating predictions based on an input. The machine learning model can be, for example, a neural network model that can accept a prompt and a query' as input to determine a task-specific prediction.

[0098] FIG. 3 shows a block diagram illustrating model inputs to a machine learning model according to embodiments. FIG. 3 includes a model input 300 comprising a prompt 302 and a query 304. FIG. 3 also includes a model 306 and task-specific prediction 308.

[0099] The model input 300 can include data that is input to the model 306 to generate the task-specific prediction 308. A computer, such as the evaluation computer 102, can generate the model input 300. The model input 300 can include data that is in a unified format, as described in reference to the feature vector of FIG. 2.

[0100] The prompt 302 can include a set of feature-label pairs. Each feature-label pair can include a task-specific label and a feature vector. The task-specific label can be a label for the associated feature vector that is specific to a particular task (e.g., a task in the machine learning model). Each feature vector of the set of feature-label pairs can include a feature vector that includes an interaction data portion and an aggregated data portion.

[0101] For example, the prompt 302 can include a plurality of interaction data portions including a first interaction data portion 310, a second interaction data portion 312, a third interaction data portion 314, e.g., for different historical interactions. The prompt 302 can also include a plurality' of aggregated data portions including a first aggregated data portion 316, a second aggregated data portion 318, and a third aggregated data portion 320. Each interaction data portion can be combined with an aggregated data portion to form a feature vector. For example, the first interaction data portion 310 can be combined with thefirst aggregated data portion 316 to form a first feature vector. Each feature vector in the prompt 302 can correspond to a task-specific label. The prompt 302 can include a first taskspecific label 322, a second task-specific label 324, and a third task-specific label 326. The prompt 302 can include any suitable number of feature vector entries and labels.

[0102] The first interaction data portion 310 and the first aggregated data portion 316 can correspond to the first task-specific label 322. The second interaction data portion 312 and the second aggregated data portion 318 can correspond to the second task-specific label 324. The third interaction data portion 314 can correspond to the third task-specific label 326.

[0103] The model input 300 also includes the query 304. The query 304 can include data that the computer is generating a prediction about. The query 304 can include a fourth interaction data portion 328 that is combined with a fourth aggregated data portion 330 to form a fourth feature vector. The fourth feature vector can correspond to an unknown taskspecific label 332. The query 304 does not include a task-specific label for the feature vector. The unknow n task-specific label 332 can be a task-specific label that the computer is using the model 306 to predict.

[0104] The interaction data in the query 304 can include data involved in a target interaction. The interaction data in the prompt 302 can correspond to a plurality of interactions that are different than the target interaction. The plurality of interactions can be for an interaction of a same type of interaction as the target interaction. For example, if the target interaction is an authorization interaction, then the plurality of interactions can be a plurality’ of authorization interactions. If the target interaction is a clearing interaction, then the plurality' of interactions can be a plurality' of clearing interactions.

[0105] In some embodiments, the prompt 302 can include data that relates to the same user as the data in the query 304. In other embodiments, the prompt 302 can include data that relates to a same resource as the resource indicated in the query 304.

[0106] The model 306 can be a machine learning model (e.g., a neural network, a large language model, etc.) that is trained to generate inferences such as the task-specific prediction 308. The model 306 can generate predictions based on the prompt 302 and the query 304 for the query 304. The model 306 provides for an in-context learning based framework. The context for the model input 300 to the model 306 in the in-context learning framework can be multiple interactions, which may be different types of interactions, withlabels in the prompt 302. The model 306 can also accept the query 304 without a label as input.

[0107] The model 306 can leam from the prompt 302. which can be a context with labels, to produce labels for the query 304. Based on the prompt 302 and the query 304, the model 306 can generate the task-specific prediction 308. Further details of the model 306 are described in further detail in reference to FIG. 4 and FIG. 5.

[0108] In some embodiments, the task-specific prediction 308 can be a prediction for Smarter STIP (e.g.. predicting whether or not the interaction will be approved or accepted). Smarter Posting (e.g., determining whether or not to post the interaction into the account), Smarter Card Verification (e.g., predicting whether or not account verification will succeed), or other interaction prediction

[0109] Such a model framework applies to various new tasks without adaption. While in previous methods, to be applicable to different tasks, different models were trained for different tasks.B. Training and Inference

[0110] The machine learning model can be trained and used for inference to generate task-specific predictions. A computer, such as the evaluation computer 102, can train, maintain, and utilize the machine learning model.[OHl] FIG. 4 shows a flowchart of training and inference methods according to embodiments. FIG. 4 includes two phases for processing a machine learning model 418 including a training phase 400 and an inference phase 410.

[0112] For the training phase 400, at step 402 the computer can obtain data corresponding to a plurality of interactions. The computer can obtain a plurality' of interaction data from a database. Each interaction data can be associated with a task-specific label. The interaction data can be stored in the database in association with the task-specific label.

[0113] As an illustrative example, the plurality of interaction data can include four instances of interaction data, each interaction data corresponding to a task-specific label. Each interaction data can correspond to an interaction of a particular category of interaction. For example each of the four interaction data can correspond to authorization interactions. A first interaction data can include data for an interaction between a user and a resourceprovider for the resource provider to provide the user access to a resource. Each task-specific label can relate to a response state (e.g., an outcome) of the corresponding interaction. For example, each task-specific label for authorization interactions can be a value of ‘'authorized” or a value of “not authorized.” As an example, two of the four task-specific labels can indicate “authorized,” while the other two task-specific labels can indicate “not authorized.”

[0114] As another example, an interaction can include a user requesting access to a secure webpage. The interaction data can include data related to the user accessing the secure webpage (e g., user name, password, IP address, machine identifier code, etc.). Each of the four interaction data can correspond to the category of interaction of webpage access interactions. A first interaction data can include data for an interaction where a user attempted to access a secure webpage. Each task-specific label can relate to a response state (e.g., an outcome) of the corresponding interaction. For example, each task-specific label for webpage access interactions can be a value of “access granted” or a value of “access denied.” As an example, three of the four task-specific labels can indicate “access granted,” while the fourth task-specific label can indicate “access denied.” For the webpage access request interactions, the task can be to determine whether or not to grant the user access to the secure webpage.

[0115] As an additional example, during an interaction, or during any other image classification task, an image can be classified. Given an image, one set of task-specific labels can be “has dog inside” and “does not have dog inside,” which can indicate whether or not a dog is in the image. Further, for the same image data, another set of task-specific labels can include “is colorful” and “is black-and-white.” For example, in some embodiments, the interaction can include a user requesting access to an image, transferring an image from one computer to another computer, uploading an image to a website, etc. The computer can utilize the machine learning model to evaluate the image data to determine tags to apply to the image, to determine a text description to apply to the image, to determine a text title to apply to the image, to determine whether or not the image violates one or more rules, to determine whether or not the image has accessibility problems, etc.

[0116] At step 404, after obtaining the plurality7of interaction data and corresponding task-specific labels, the computer can combine each instance of interaction data with aggregated data to create feature vectors. The plurality of feature vectors can correspond to the plurality' of task-specific labels for training.

[0117] The computer can obtain the aggregated data from the interaction database and / or from other computers and / or databases. The aggregated data can include data related to the interaction data in some manner. For example, for the first interaction data, the aggregated data can include information related to a user account and the resource provider and / or the resource provider computer.

[0118] For the webpage access example, the aggregated data can include data that is representative of historical webpage access attempts. The aggregated data can include a total number of access attempts from all users to access the webpage, a number of access attempts from the user to access the webpage, an indication of where the w ebpage is hosted on the Internet, a number of denied access attempts, a number of granted access attempts, and / or any other data relating to the webpage, accessing the webpage, the host of the webpage (e.g., which may be a resource provider computer), and / or user attempts to access the webpage.

[0119] The computer can combine the w ebpage access interaction data with webpage access aggregated data to form w ebpage access feature vectors that can be utilized in the machine learning model 418.

[0120] At step 406, after combining the interaction data and the aggregated data together using a unified formatting for the feature vectors, the computer can create a training sample that includes a prompt 407, a query 408, and a target label 409 (e.g., a label to be predicted) as an input for a current training step. The prompt 407 can include the plurality of feature vectors and the plurality of task-specific labels for training. The query 408 can include interaction data for an interaction for which the computer is training the machine learning model 418 to predict a task-specific label.

[0121] As an example, the prompt 407 can include four feature vectors created from interaction data, aggregated data, and task-specific labels related to the w ebpage access interactions. The query 408 can include a feature vector created from the webpage access interaction data and aggregated data and can correspond to the target label 409. The input to the model in one data sample can include the five feature vectors.

[0122] In some embodiments, training data can be batched. For example, if there is a batch size of 100, then there can be 500 (5 x 100) feature vectors representing interactions in total for the batch.

[0123] The computer can input the training sample into the model to train the machine learning model 418. The machine learning model 418 can accept the training sample as input, where the training sample includes the prompt 407, the query 408, and the target label 409.

[0124] The computer can iteratively train the machine learning model 418 using training sets of training samples. The computer can create a first training set comprising one or more training samples. The computer can train the machine learning model 418 in a first stage using the first training set. The computer can then create a second training set for a second stage of training. The computer can train the neural network in a second stage using the second training set.

[0125] In some embodiments, rather than training the machine learning model 418 on some of the data samples and perform inference on some other data samples from the same tasks, the model 418 can accept the training samples as well as inference samples altogether as input and can output the prediction (e.g., a few-shot training process). Few-shot learning is a machine learning framework in which the machine learning model 418 can learn to make accurate predictions by training on a small number of labeled examples.

[0126] For the inference phase 410, the computer can use the machine learning model 418, which can be trained during the training phase 400 to generate predictions (e.g., perform inference).

[0127] At step 412, the computer can obtain current interaction data and current aggregated data. The computer can obtain the current interaction data and the current aggregated data from the interaction database. The current interaction data can correspond to a current interaction for which the computer can generate a prediction.

[0128] In some embodiments, the current interaction can be a most recent interaction stored in the interaction database. In other embodiments, the current interaction can correspond to an interaction that is indicated by an interaction identifier in a received prediction request message from another device (e.g., a downstream device, a network processing computer, etc ).

[0129] As an illustrative example, the current interaction data can be a current secure webpage access interaction data, where a user is attempting to access a secure webpage. The current secure webpage access interaction data can include a username and a password.which can be hashed or otherwise obfuscated for security. The current aggregated data can be current secure webpage access aggregated data that can indicate aggregated data about historical access requests for the webpage.

[0130] At step 414, the computer can create a current feature vector using the current interaction data and the current aggregated data. The current feature vector can be a query7for the machine learning model 418.

[0131] For example, the computer can combine the current secure w ebpage access interaction data and the current secure webpage access aggregated data to form the query.

[0132] At step 416, the computer can determine obtain and / or generate one or more additional feature vectors to utilize as the prompt. In some embodiments, the computer can determine which feature vector(s) to utilize as the prompt using a context window7. The computer can obtain a prompt comprising a plurality of obtained feature vectors.

[0133] For example, the computer can obtain additional secure webpage access interaction data and additional secure webpage access aggregated data. The additional data can relate to other secure webpage access interactions. The computer can form additional feature vectors using the additional secure webpage access interaction data and additional secure webpage access aggregated data. The computer can generate the prompt using the additional feature vectors that relate to the additional secure webpage access interactions.

[0134] After obtaining the prompt and the query, the computer can provide the prompt and the query to the machine learning model 418 to determine a task-specific prediction for the query and the prompt. The task-specific prediction can specify a response state to the current interaction. For example, the task-specific prediction can indicate “access granted7’ or “access denied” for the current secure webpage access interaction.C. Context Window

[0135] The computer can obtain additional interaction data for the additional feature vectors. The additional feature vectors can be obtained from interactions that are included in the context window7. The context window can inform the computer on how to determine w hich additional interaction data to utilize w hen determining a prediction for the current interaction. For example, the context w indow7can indicate a length of time that the computer can utilize when determining the additional interaction data. The context window can indicatea length of time of 1 hour, 1 day, 3 days, 1 week, 1 month, 4 months, 1 year, etc. The context window can indicate a number of interactions to include (e.g.. 2 interactions, 3 interactions, 4 interactions, 7 interactions, 10 interactions, 20 interactions, etc ).

[0136] During model training and inference the computer can determine and obtain data for additional interactions with corresponding interaction data using the context window. The additional interaction data can be input, along with additional aggregated data, into the machine learning model.

[0137] In some embodiments, the context window can be small compared to the dataset size (e g., 100 interactions within a context window’s size as compared to 100 million total interactions stored in a database). The computer can select more informative interactions to put into the context window. However, it may be difficult or impossible search through all interaction data, or even 1% of the interaction data, to select a few comparative additional interactions that are similar to the current interaction. Embodiments can provide for an effective and efficient method to select the interactions for the context window.

[0138] The computer can obtain additional interaction data and additional aggregated data based on the criteria of the context window. For example, the computer can obtain k samples of interactions for each of n categories to best provide hints for the in-context learning model to make a prediction for the target sample (e.g., interaction data).

[0139] The sampling method can have two steps 1) feature-label pair rule set determination and 2) context window creation and use.

[0140] During feature-label pair rule set determination, for each task, the computer can compute a feature-label pair rule. To generate a rule, the computer can 1) evaluate the relationship between features and labels. 2) select the most relevant features, and 3) generate a rule for each relevant feature.

[0141] To evaluate the relationship between features and labels, the computer can conduct a feature-label correlation analysis do determine any correlations between the feature vectors and the labels. For example, the computer can evaluate the strength of the relationship (e.g., linear relationship) between two variables and compute their association. During correlation analysis the computer can evaluate the changes in one variable due to the changes in another variable.

[0142] After determining the strength of the relationships between the features and the labels, the computer can select a threshold number of features for the rules. The computer can select the most related features and labels. For example, the computer can select the top 25%, or other suitable threshold percentage, of feature-label relationships for which to generate rules for the rule set.

[0143] After selecting which features to generate rules for, the computer can generate rules for the selected features. The selected features can be of different types. For example, a feature can be a numeric feature or a nominal feature. The computer can generate rules for numeric features (e.g., greater than mean + standard deviation, less than mean - standard deviation, etc.). The computer can generate rules for nominal features (e.g., feature type set. including only types that are less frequent than 30%).

[0144] Table 1. below, illustrates an example task feature-label rule set that includes a number of rules. For example, the computer can identify the five most correlated featurelabel pairs. The most correlated features can be a numerical feature 1, a nominal feature 2, a numerical feature 3, a numerical feature 4, and a numerical feature 5.

[0145] The computer can generate a rule for each selected feature that indicates allowable features for selection during obtaining additional interaction data using the context window. For example, the numerical feature 1 can have a rule that the value is less than 0.9 or greater than 1.4. The nominal feature 2 can have a rule that the value is equal to a category of “red” or a category of “green. ” The numerical feature 3 can have a rule that the value is less than 100 or greater than 400. The numerical feature 4 can have a rule that the value is less than 9 or greater than 14. The numerical feature 5 can have a rule that the value is equal to 2, 5 or 6.

[0146] As an illustrative example, the rules in Table 1 can relate to flower classification for example: feature 1 relates to a petal size, feature 2 relates to a color, feature 3 relates to a petal weight in milligrams, feature 4 relates to a number of flowers on each main stem, and feature 5 relates to a number of petals on each flower.Table 1: Task feature-label rule set

[0147] After generating the rule set, the computer can proceed using the context window based on the rule set. To use the context window, the computer can sample samples (e.g., interaction data and / or aggregated data) from the database according to the rule set. The computer can randomly sample interaction data from the database and can determine whether or not to keep the interaction data based on the rule set.

[0148] The computer can continue to sample interaction data until the computer obtains a predetermined number (k) of interaction data samples (e.g., 50 interaction data samples, 100 interaction data samples, 200 interaction data samples, etc.).D. Model Components

[0149] The machine learning model can accept input data that includes a prompt and a query and can determine an output that includes a task-specific prediction. The internal processing of the machine learning model can include encoders and decoders that can modify the input data to determine the output. The machine learning model can be a neural network or a large language model that includes a number of encoders and decoders as well as a selfattention mechanism.

[0150] The machine learning model can receive an input from the computer. The input can include the prompt and the query. The output can be the predicted label for the queried interaction. Since the machine learning model is learning from a previous task, it can be advantageous to have diverse tasks to backup and retrain the model. For example, the machine learning model can include a classification head and a regression head to aid in performing different tasks. The machine learning model can be trained using both the classification head and the regression head.

[0151] FIG. 5 shows a block diagram illustrating a machine learning model according to embodiments. FIG. 5 illustrates input data 502, a machine learning model 504. and an output 506.

[0152] The machine learning model 504 can accept the input data 502 as input. The input data 502 includes a prompt 508 and a query 510. The prompt 508 can include data related to multiple interactions (e.g., in the example depicted in FIG. 5, two interactions) along with labels that correspond to the interactions. The prompt 508 can include feature vectors that include interaction data, aggregated data, and a corresponding task-specific label.

[0153] The query 510 can include data related to an interaction for which the query is being made. For example, the computer can utilize the machine learning model 504 to generate a predicted label that is related to the interaction. The query 510 can include a feature vector that includes interaction data and aggregated data.

[0154] The output 506 can be the predicted label for the queried interaction. The output 506 can be a value that indicates a prediction related to the queried interaction. The output 506 can be a task-specific prediction. The task-specific prediction can specify a response state to the queried interaction. The response state can be, for example, “authorized,” “not authorized,” “cleared,” “not cleared,” “fraudulent,” “not fraudulent,” etc. depending on a classification of the interaction.

[0155] The machine learning model 504 can include a plurality of interaction encoders including a first interaction encoder 512. a second interaction encoder 514, and a third interaction encoder 516. In some embodiments, each encoder can be pre-trained. In other embodiments, each encoder may not be pre-trained. The interaction encoders can process interaction data. The machine learning model 504 also includes a first label encoder 518 and a second label encoder 520. The label encoders can process task-specific labels. The machine learning model 504 can include any number of interaction encoders and label encoders. For example, the machine learning model 504 can include 2 interaction encoders, 4 interaction encoders, 8 interaction encoders, 15 interaction encoders, etc. The machine learning model 504 can include a same number of label encoders as the number of interaction encoders. The interaction encoders can generate interaction embeddings and the label encoders can generate label embeddings.

[0156] Each encoder of the plurality of encoders can generate a vector (e.g., an embedding) from an input sequence (e.g., the feature vector, the label, etc.). Each encoder can include a fully -connected layer and can be a feed-forward neural network. Each encoder can be pre-trained to generate an embedding based on an input vector.

[0157] The machine learning model 504 can also include a first mixer encoder 522 and a second mixer encoder 524. The machine learning model 504 can include any number of mixer encoders. The machine learning model 504 can include the same number of mixer encoders as the number of interaction encoders. Each mixer encoder can accept interaction embedding and label embedding pairs that correspond to feature-label pairs from the input data 502. The mixer encoder can generate a mixed embedding based on the interaction embedding and the label embedding. In some embodiments, the interaction embedding and the label embedding can be concatenated with one another, or otherwise combined, to form a single input vector that can be input into the mixer encoder.

[0158] The machine learning model 504 also includes a self-attention module 526 and a decoder 528.

[0159] The machine learning model 504 also includes a classification head 530. The classification head 530 can be a classification machine learning model that is trained to determine a classification based on an input vector (e.g., the output vector from the decoder 528). The classification machine learning model of the classification head 530 can perform any classification process to classify input data (e.g., decision trees, support vector machine, random forest, k-nearest neighbors, etc.).

[0160] The machine learning model 504 also includes a regression head 532. The regression head 532 can be a regression machine learning model that is trained to perform regression tasks based on an input vector (e.g., the output vector from the decoder 528). The regression machine learning model of the regression head 532 can perform any regression process to determine an output value based on the input data (e.g., linear regression, polynomial regression, ridge regression, elastic net regression, etc.).

[0161] As an illustrative example the computer can obtain the prompt 508 and the query 510. The query 510 can include a feature vector for a target interaction. The prompt 508 can include a first additional feature vector and a second additional feature vector. The computer can input the feature vectors from the input data 502 into interaction encoders in the machine learning model 504. For example, the computer can input first additional feature vector from the prompt 508 into the first interaction encoder 512. The computer can input the second additional feature vector from the prompt 508 into the second interaction encoder 514. The computer can input the feature vector from the query 510 into the third interaction encoder 51 .

[0162] Each interaction encoder can encode the interaction data and the aggregated data from the input feature vector. Each interaction encoder can generate an interaction embedding. Each interaction encoder can generate a vector that represents the interaction data and the aggregated data. Each interaction encoder can be trained during a training phase to generate an interaction embedding that represents a feature vector for an interaction.

[0163] The first interaction encoder 512 can generate a first interaction embedding from the first additional feature vector. The second interaction encoder 514 can generate a second interaction embedding from the second additional feature vector. The third interaction encoder 516 can generate a third interaction embedding from the feature vector corresponding to the target interaction.

[0164] The computer can also input the task-specific labels from the prompt 508 into label encoders in the machine learning model 504. For example, the computer can input the first task specific label, which corresponds to the first additional feature vector, into the first label encoder 518. The computer can input the second task specific label, which corresponds to the second additional feature vector, into the second label encoder 520. The label encoders can encode the task-specific labels. The label encoders can generate label embeddings. Each label encoders can be trained during a training phase to generate a label embedding that represents a task-specific label for an interaction.

[0165] Each interaction encoder in the machine learning model 504 can be paired with a label encoder. For example, the first interaction encoder 512 can be paired with the first label encoder 518. The first interaction encoder 512 can encode a feature vector from the first feature-label pair of the prompt 508. The first label encoder 518 can encode a label from the first feature-label pair of the prompt 508.

[0166] After generating an interaction embedding an a label embedding for each feature-label pair in the prompt 508, the machine learning model 504 can combine the interaction embedding and the label embedding for each feature-label pair. For example, the machine learning model 504 can provide the first interaction embedding from the first interaction encoder 512 and the first label embedding from the first label encoder 518 into the first mixer encoder 522. The machine learning model 504 can also provide the second interaction embedding from the second interaction encoder 514 and the second label embedding from the second label encoder 520 into the second mixer encoder 524.

[0167] Each mixer encoder can accept two input vectors (e.g., an interaction embedding and a label embedding) and can generate a vector that represents a combination of the two input vectors. Each mixer encoder can generate a mixed embedding from a received interaction embedding and a received label embedding. Each mixer encoder can be trained during a training phase to generate a mixed embedding that represents a combination of an interaction embedding and a label embedding.

[0168] After obtaining the mixer embeddings (e.g., output encodings) from each mixer encoder (e.g., the first mixer encoder 522 and the second mixer encoder 524), the machine learning model 504 can provide the mixer embeddings to the self-attention module 526. The machine learning model 504 can provide the same number of mixer embeddings as there are feature-label pairs in the prompt 508 to the self-attention module 526. The selfattention process can be performed on each mixer embedding.

[0169] The self-attention module 526 can include a self-attention mechanism for neural networks such that each processed mixer embedding is self-attended to across each element in the mixer embedding vector. The self-attention module 526 can examine the correlation of each element in a mixer embedding to other elements in the mixer embedding.

[0170] Attention, in general, is a machine learning method that determines the relative importance of each component in a sequence relative to the other components in that sequence. The sequence provided to the self-attention module 526 can include the elements of each mixer embedding. The self-attention module 526 can determine a query matrix, a key matrix, and a value matrix based on the input mixer embedding. The self-attention module 526 can utilize the query matrix, the key matrix, and the value matrix to determine an attention score for each element in the input mixer embedding. For each input mixer embedding, the self-attention module 526 can output a vector that represents the input mixer embedding modified by attention scores (e.g., be weighting elements in the input mixer embedding). The outputs of the self-attention module 526 can be self-attended mixer embeddings.

[0171] After generating the self-attended mixer embeddings, the machine learning model 504 can provide the self-attended mixer embeddings and the third interaction embedding, which corresponds to the target interaction, into the decoder 528. The decoder 528 can accept a plurality of vectors as input and can generate a single output vector that represents the inputs. The decoder 528 can generate an output vector that represents the self-atended mixer embeddings and the interaction embedding for the query' (e.g., the third interaction embedding). In some embodiments, the decoder 528 can be a flexible decoder.

[0172] The output vector can fit either into the classification head 530 or the regression head 532 to make a final prediction. The machine learning model 504 can determine which head to utilize based on the input data 502. If the task-specific labels from the prompt 508 are binary' labels (e.g., authorized or not authorized), the output vector can be input into the classification head 530. If the task-specific labels from the prompt 508 are an amount or other value (e.g., a likelihood of something, an interaction amount, a risk score, etc.), then the output vector can be provided to the regression head 532 for the final label.

[0173] The classification head 530 can be trained to determine a classification based on a vector. The classification head 530 can obtain the output vector from the decoder 528. The classification head 530 can determine a classification based on the output vector. For example, the classification head 530 can determine a probability that the output vector corresponds to one or more classifications. The one or more classifications can include, for example, authorized, not authorized, fraudulent, not fraudulent, cleared, not cleared, posted, not posted, etc. The classification head 530 can output the classification that has the highest probability. The classification can be a task-specific prediction. The task-specific prediction can specify a response state to the current interaction.

[0174] The regression head 532 can be trained to determine a value based on a vector. The regression head 532 can obtain the output vector from the decoder 528. The regression head 530 can determine an output value that represents the output vector. The output value can be a value that represents something related to the current interaction. The output value can be a task-specific prediction. The output value can be, for example, a risk score, a likelihood score, or other predicted value that relates to the current interaction and / or a response state of the current interaction.

[0175] Embodiments can provide for the advantage of utilizing the same machine learning model for different tasks rather than needing to create custom machine learning models for each different task. If there is a classification task, then the computer can reuse the whole model structure the same as regression head for any task, either classification or regression. A computer can reuse or retrain the model, but can still utilize the overall model structure with a small difference for each head to perform each task.E. Prediction Determination

[0176] A computer can determine predictions for task-specific labels for interactions using the machine learning model. The predictions can predict a response state for a current interaction. For example, a current interaction can be an authorization interaction for an interaction between a user and a resource provider, where the user is requesting authorization to access a resource provided by the resource provider. The response state can indicate whether or not the authorization interaction is authorized.

[0177] FIG. 6 shows a flow diagram illustrating a prediction determination method according to embodiments. The method illustrated in FIG. 6 can be performed by a computer such as the evaluation computer 102.

[0178] In some embodiments, prior to step 602, the evaluation computer 102 can receive a prediction request from the downstream device 106. The prediction request can be a request for a task-specific request for a particular interaction. The prediction request can include an interaction identifier that uniquely identifies an interaction. The evaluation computer 102 can utilize the interaction identifier to identify’ interaction data and related data in the interaction database 104.

[0179] At step 602, the evaluation computer 102 can obtain interaction data and aggregated data for a current interaction. The evaluation computer 102 can obtain the interaction data and the aggregated data from the interaction database 104. The interaction data and the aggregated data can be for a current interaction. The current interaction can include a device (e.g., a user device) requesting access to a resource from a resource provider computer.

[0180] The evaluation computer 102 can obtain interaction data related to a particular interaction identifier. The evaluation computer 102 can also search the interaction database 104 for data related to devices, entities, and / or accounts that are indicated in the interaction data to obtain aggregated data for the devices, entities, and / or accounts.

[0181] At step 604, after obtaining the interaction data and the aggregated data, the evaluation computer 102 can generate a feature vector using the interaction data and the aggregated data. The feature vector can be in a unified format such that the feature vector can be generated for any type of interaction. The evaluation computer 102 can generate thefeature vector using the interaction data and the aggregated data as described in further detail in reference to FIG. 2.

[0182] At step 606. after generating the feature vector, the evaluation computer 102 can generate a prompt comprising a set of feature-label pairs. The evaluation computer 102 can obtain a plurality of additional interaction data and a plurality' of additional aggregated data from the interaction database 104.

[0183] Each feature-label pair can include a task-specific label and an additional feature vector. The additional feature vector can include additional interaction data and additional aggregated data. The additional interaction data and the additional aggregated data are related to an additional interaction of a same ty pe of classification as the current interaction.

[0184] At step 608. after generating the prompt, the evaluation computer 102 can load, into memory, a machine learning model (e.g., the machine learning model 504 depicted in FIG. 5) trained to determine predictions for specific tasks. The machine learning model can be a neural network or a large language model that is to generate predictions based on input prompts and queries. The machine learning model can be a pre-trained machine learning model.

[0185] In some embodiments, the evaluation computer 102 can store the machine learning model in a data storage device and can load the machine learning model from the data storage device. In other embodiments, the machine learning model can be stored in a model database. The evaluation computer 102 can retrieve the machine learning model from the model database.

[0186] At step 610. after loading the machine learning model into memory, the evaluation computer 102 can input the prompt and a query including the feature vector into the machine learning model.

[0187] At step 612, the evaluation computer 102, using the machine learning model, can determine a task-specific prediction for the query and the prompt. The task-specific prediction can specify a response state to the interaction. For example, the current interaction can be an authorization interaction. The interactions in the prompt can also be authorization interactions. The response state to the interaction can indicate whether or not the current interaction is authorized. As such, the evaluation computer 102 can generate a prediction ofwhether or not the current interaction is authorized based on the current interaction and the additional interactions.Training Masks

[0188] Masks can be applied to training data. Training data can relate to an interaction of a particular interaction classification (e.g.. authorization interaction, clearing interaction, etc.) and can relate to one of many different learning tasks and can be a regression task or a classification task. An instance of training data can represent a learning task for the machine learning model. Masks can aid the machine learning model in understanding a current task during a training phase.

[0189] FIG. 7 shows a block diagram illustrating masked unified data (e.g., masked feature vectors) according to embodiments. FIG. 7 will be described in reference to incontext learning objectives and pretraining the machine learning model.

[0190] FIG. 7 illustrates unified data 702 (e.g., a feature vector) that can be masked in two different manners. The unified data 702 can be masked into a first masked unified data 704 and a second masked unified data 706. Particular fields in the unified data 702 can be masked to create the first masked unified data 704 and the second masked unified data 706. A first field 708 can be masked in the unified data 702 to obtain the first masked unified data 704. A second field 710 can be masked in the unified data 702 to obtain the second masked unified data 706. The computer can mask fields the interaction data portion of the feature vector.

[0191] The computer can mask fields of the feature vector prior to being input into the machine learning model for training. Prior to training, the computer can have the masked interaction data, the aggregated data, and the task-specific label for a specific instance of training data.

[0192] The computer can mask a particular interaction data in a plurality of ways (e.g., mask different portions of the interaction data). For example, the computer can generate the first masked unified data 704 and the second masked unified data 706 from the same interaction data. The computer can train the machine learning model using both instances of masked data, such that the model optimizes a loss function that takes the masks into account. The computer can verify that the output of the machine learning model for the first masked unified data 704 is similar to the second masked unified data 706. In some embodiments, thecomputer can also train the machine learning model on the unified data 702 and can compare an output from the machine learning model, based on the unified data 702, to the output for the masked inputs. As such, the computer can determine a loss value from the unified data 702 and can determine a masked loss value from the masked data to compare the loss values in order to help optimize the loss functions.

[0193] As an illustrative example, the computer can train the machine learning model using masks according to the pseudocode illustrated in Table 2.Table 2: Pretraining pseudocode

[0194] is a traditional prediction loss andand a label y. r can be expanded into a unified data format u that includes the interaction data x. the aggregated data xexpand, and the label y. m is a number of masks. CE represents crossentropy loss. yUmodeiindicates a predicted label for the feature vector u by the model, yuindicates the predicted label for the feature vector u, yUmasklindicates a predicted label for afirst masked feature vector u. and y„, indicates a predicted label for a second masked feature vector u.

[0195] The labels can correspond to particular tasks and can be associated with interaction data. Tasks can be separated into two different groups of tasks: classification tasks and regression tasks.

[0196] Classification tasks can include tasks such as smarter VisaNet series tasks (e.g., Smarter Posting, Smarter Account Verification, Smarter STIP, etc.). For the classification tasks the computer can select a set of nominal fields, use the fields as labels, and change the fields into default values in the unified data part.

[0197] For the regression tasks the computer can select a set of numerical fields, use the fields as labels, and change the fields into default values in the unified data part.

[0198] Referring to Table 2, for each training step of a training phase, the computer can train the machine learning model using a number of batches of training data. During each training batch, the computer can sample a particular task t to train. The computer can randomly sample the task t from a plurality of tasks.

[0199] The computer can obtain multiple samples of data for a particular interaction (r=(%, y)) from the interaction database. The computer can obtain interaction data x and a task-specific label y for the interaction data x. The computer can obtain any number of interaction data x and task-specific label y pairs from the interaction database that relate to the task t.

[0200] After obtaining the data for the sampled interactions, the computer can generate a feature vector u for each sampled interaction. The computer can generate the feature vector u using the interaction data x, the task-specific label y, as well as aggregated data xexpandthat is related to the sampled interaction. The computer can form a feature vector dataset U from the feature vectors.

[0201] After forming the feature vector dataset U. the computer can generate one or more masked feature vectors for each feature vector u in the feature vector dataset U. For example, the computer can generate two masked feature vectors (e.g., umaskiand umask2) from each feature vector u.

[0202] After generating the one or more masked feature vectors for each feature vector u in the feature vector dataset U to form a set of masked feature vectors, the computer can train the machine learning model using the set of masked feature vectors. The computer can use the machine learning model to generate a predicted label for the feature vector u (e.g., y^-modei^apredicted label for a first masked feature vector u (e.g.. yUmaskl)- and a predicted label for a second masked feature vector u (e.g., y„, ).

[0203] The computer can use the different predicted labels to optimize a loss function that can indicate how accurately the machine learning model generated the predicted label.Based on the loss function, the computer can update weights throughout the machine learning model to more accurately predict a next predicted label. The computer can iteratively train the machine learning model using the feature vectors and masked feature vectors.IV. ADVANTAGES

[0204] Embodiments of the disclosure have a number of advantages. For example, rather than learning from historical data from data of the same type of task (e.g., only learn from authorization interactions), the machine learning model can leam from few examples from the same tasks and makes predictions for the tasks. By doing so, the model can seamlessly be used in various downstream tasks, which can include existing tasks or new tasks.

[0205] Embodiments of the disclosure have a number of additional advantages. For example, the machine learning model can be trained on different stages of interactions by generating and utilizing the unified data as feature vectors, thus allowing for more data of differing interactions to be utilized in the machine learning model. This also eliminates the need to train many different machine learning models for each different stage of interaction.

[0206] Furthermore, current methods require different machine learning models to perform different types of predictions (e.g., predicting whether or not an interaction is fraudulent, predicting whether or not an interaction is going to be approved, predicting whether or not the interaction will be posted to the user's account as pending or as another status, etc.). Typically, to improve performance of all the different machine learning models, more data is leveraged and the models are made larger. However, due to the interaction data used, more data and larger models may not improve the model performance because it is not realistic to predict arbitrary numbers. As an illustrative example, in US restaurantinteractions, no matter what information a model might have so far regarding the restaurant, it can be difficult to predict what a clearing amount might be even after authorization, since the model might not have any information about tips at the restaurant. Therefore, even if a large amount of authorization-clearing interactions are used to train a large model, it could be difficult to get a model with good performance in such a scenario. Therefore, simply using more data and larger models may not improve the performance on interaction data. Embodiments can provide for a single machine learning model that can perform a plurality of different tasks rather than needing to train many different models on large amounts of data. Embodiments do not simply rely on the use of large amounts of data to improve prediction accuracy as in previous methods.

[0207] The machine learning model can also moderate data distribution shift, especially label distribution shift. For example, when the few examples in the prompt can demonstrate the pattern, the machine learning model can make good predictions even if label shift occurs over time.V. COMPUTER SYSTEMS

[0208] Processing and methods described herein can be performed by computer systems.

[0209] FIG. 8 shows a block diagram of the evaluation computer 102 according to embodiments. The exemplary evaluation computer 102 may comprise a processor 804. The processor 804 may be coupled to a memory' 802, a network interface 806, and a computer readable medium 808. The computer readable medium 808 can include any number of modules. The computer readable medium 808 can include a feature vector module 808A. a prompt module 808B, a query module 808C, and a machine learning module 808D.

[0210] The memory 802 can be used to store data and code. For example, the memory 802 can store training data, machine learning models, machine learning model weights, inference data, etc. The memory 802 may be coupled to the processor 804 internally or externally (e.g.. cloud based data storage), and may comprise any combination of volatile and / or non-volatile memory, such as RAM, DRAM, ROM, flash, or any other suitable memory' device.

[0211] The computer readable medium 808 may comprise code, executable by the processor 804, for performing methods. For example, one method can include the evaluationcomputer 102 obtaining interaction data and aggregated data for a current interaction. The current interaction can relate to a device requesting access to a resource. The evaluation computer 102 can generate a feature vector using the interaction data and the aggregated data. The evaluation computer 102 can then generate a prompt comprising a set of feature-label pairs, each comprising: a task-specific label and an additional feature vector. The additional feature vector can include additional interaction data and additional aggregated data. The additional interaction data and the additional aggregated data can be related to an additional interaction of a same type of classification as the current interaction. The evaluation computer 102 can load, into a memory, a pre-trained machine learning model trained to determine predictions for specific tasks. The evaluation computer 102 can input the prompt and a query including the feature vector into the pre-trained machine learning model. The evaluation computer 102, using the pre-trained machine learning model, can determine a task-specific prediction for the query and the prompt. The task-specific prediction can specify a response state to the current interaction (e.g., authorized or not authorized, cleared or not cleared, fraudulent or not fraudulent, etc.).

[0212] The feature vector module 808 A may comprise code or software, executable by the processor 804, for creating feature vectors. The feature vector module 808A, in conjunction with the processor 804, can generate feature vectors from interaction data and aggregated data.

[0213] The prompt module 808B can include may comprise code or software, executable by the processor 804. for creating prompts. The prompt module 808B. in conjunction with the processor 804, can create prompts from additional interaction data and additional aggregated data to form a set of feature-label pairs.

[0214] The query' module 808C can include may comprise code or software, executable by the processor 804, for creating queries. The query module 808C, in conjunction with the processor 804. can generate queries from feature vectors.

[0215] The machine learning module 808D can include may comprise code or software, executable by the processor 804, for training, maintaining, and utilizing machine learning models. The machine learning module 808D, in conjunction with the processor 804, can train and utilize a machine learning model, which can be a neural network or a large language model, that can generate task-specific predictions for current interactions based on the query and the prompt.

[0216] The network interface 806 may include an interface that can allow the evaluation computer 102 to communicate with external computers. The network interface 806 may enable the evaluation computer 102 to communicate data to and from another device (e.g., the interaction database 104, the downstream device 106, the network processing computer 108, etc.). Some examples of the network interface 806 may include a modem, a physical network interface (such as an Ethernet card or other Network Interface Card (NIC)), a virtual network interface, a communications port, a Personal Computer Memory Card International Association (PCMCIA) slot and card, or the like. The wireless protocols enabled by the network interface 806 may include Wi-Fi™. Data transferred via the network interface 806 may be in the form of signals which may be electrical, electromagnetic, optical, or any other signal capable of being received by the external communications interface (collectively referred to as “electronic signals” or “electronic messages”). These electronic messages that may comprise data or instructions may be provided betw een the network interface 806 and other devices via a communications path or channel. As noted above, any suitable communication path or channel may be used such as, for instance, a wire or cable, fiber optics, a telephone line, a cellular link, a radio frequency (RF) link, a WAN or LAN network, the Internet, or any other suitable medium.

[0217] Any of the computer systems mentioned herein may utilize any suitable number of subsystems. Examples of such subsystems are shown in FIG. 9 in computer system 900. In some embodiments, a computer system includes a single computer apparatus, where the subsystems can be the components of the computer apparatus. In other embodiments, a computer system can include multiple computer apparatuses, each being a subsystem, with internal components. A computer system can include desktop and laptop computers, tablets, mobile phones and other mobile devices.

[0218] The subsystems shown in FIG. 9 are interconnected via a system bus 924. Additional subsystems such as a printer 908, keyboard 916, storage device(s) 918, monitor 922 (e.g., a display screen, such as an LED), which is coupled to display adapter 912, and others are shown. Peripherals and input / output (I / O) devices, which couple to I / O controller 902, can be connected to the computer system by any number of means known in the art such as input / output (I / O) port 914 (e.g., USB, FireWire®). For example, I / O port 914 or external interface 920 (e.g., Ethernet, Wi-Fi, etc.) can be used to connect computer system 900 to a wide area netw ork such as the Internet, a mouse input device, or a scanner. Theinterconnection via system bus 924 allows the central processor 906 to communicate with each subsystem and to control the execution of a plurality of instructions from system memory 904 or the storage device(s) 918 (e.g., a fixed disk, such as a hard drive, or optical disk), as well as the exchange of information between subsystems. The system memory 904 and / or the storage device(s) 918 may embody a computer readable medium. Another subsystem is a data collection device 910, such as a camera, microphone, accelerometer, and the like. Any of the data mentioned herein can be output from one component to another component and can be output to the user.

[0219] A computer system can include a plurality of the same components or subsystems, for example, connected together by external interface 920, by an internal interface, or via removable storage devices that can be connected and removed from one component to another component. In some embodiments, computer systems, subsystem, or apparatuses can communicate over a network. In such instances, one computer can be considered a client and another computer a server, where each can be part of a same computer system. A client and a server can each include multiple systems, subsystems, or components. In various embodiments, methods may involve various numbers of clients and / or servers, including at least 10, 20, 50, 100, 200, 500, 1,000, or 10,000 devices. Methods can include various numbers of communication messages between devices, including at least 100, 200, 500, 1,000, 10,000, 50,000, 100,000, 500,00, or one million communication messages. Such communications can involve at least 1 MB, 10 MB, 100 MB, 1 GB. 10 GB. or 100 GB of data.

[0220] Although the steps in the flowcharts and process flows described above are illustrated or described in a specific order, it is understood that embodiments of the invention may include methods that have the steps in different orders. In addition, steps may be omitted or added and may still be within embodiments of the invention.

[0221] Aspects of embodiments can be implemented in the form of control logic using hardware circuitry (e g., an application specific integrated circuit or field programmable gate array) and / or using computer software stored in a memory with a generally programmable processor in a modular or integrated manner, and thus a processor can include memory’ storing software instructions that configure hardware circuitry, as well as an FPGA with configuration instructions or an ASIC. As used herein, a processor can include a singlecore processor, multi-core processor on a same integrated chip, or multiple processing unitson a single circuit board or networked, as well as dedicated hardware. Based on the disclosure and teachings provided herein, a person of ordinary skill in the art will know and appreciate other ways and / or methods to implement embodiments of the present disclosure using hardware and a combination of hardware and software.

[0222] Any of the software components or functions described in this application may be implemented as software code to be executed by a processor using any suitable computer language such as. for example. Java, C, C++, C#, Objective-C, Swift, or scripting language such as Perl or Python using, for example, conventional or object-oriented techniques. The software code may be stored as a series of instructions or commands on a computer readable medium for storage and / or transmission. A suitable non- transitory computer readable medium can include random access memory (RAM), a read only memory (ROM), a magnetic medium such as a hard-drive or a floppy disk, or an optical medium such as a compact disk (CD) or DVD (digital versatile disk) or Blu-ray disk, flash memory, and the like. The computer readable medium may be any combination of such devices. In addition, the order of operations may be re-arranged. A process can be terminated when its operations are completed but could have additional steps not included in a figure. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination may correspond to a return of the function to the calling function or the main function.

[0223] Such programs may also be encoded and transmitted using carrier signals adapted for transmission via wired, optical, and / or wireless networks conforming to a variety of protocols, including the Internet As such, a computer readable medium may be created using a data signal encoded with such programs. Computer readable media encoded with the program code may be packaged with a compatible device (e.g., as firmware) or provided separately from other devices (e.g., via Internet download). Any such computer readable medium may reside on or within a single computer product (e.g., a hard drive, a CD, or an entire computer system), and may be present on or within different computer products within a system or network. A computer system may include a monitor, printer, or other suitable display for providing any of the results mentioned herein to a user.

[0224] Any of the methods described herein may be totally or partially performed with a computer system including one or more processors, which can be configured to perform the steps. Any operations performed with a processor may be performed in real-time.The term “real-time’' may refer to computing operations or processes that are completed within a certain time constraint. As examples, a time constraint may be 30 seconds. 1 minute, 10 minutes, 30 minutes, 1 hour, 4 hours, 1 day, or 7 days. Thus, embodiments can be directed to computer systems configured to perform the steps of any of the methods described herein, potentially with different components performing a respective step or a respective group of steps. Although presented as numbered steps, steps of methods herein can be performed at a same time or at different times or in a different order. Additionally, portions of these steps may be used with portions of other steps from other methods. Also, all or portions of a step may be optional. Additionally, any of the steps of any of the methods can be performed with modules, units, circuits, or other means of a system for performing these steps.

[0225] The specific details of particular embodiments may be combined in any suitable manner without departing from the spirit and scope of embodiments of the disclosure. However, other embodiments of the disclosure may be directed to specific embodiments relating to each individual aspect, or specific combinations of these individual aspects.

[0226] The above description of example embodiments of the present disclosure has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the disclosure to the precise form described, and many modifications and variations are possible in light of the teaching above.

[0227] A recitation of "a", "an" or "the" is intended to mean "one or more" unless specifically indicated to the contrary. The use of “or” is intended to mean an “inclusive or,” and not an “exclusive or” unless specifically indicated to the contrary. Reference to a “first” component does not necessarily require that a second component be provided. Moreover, reference to a “first” or a “second” component does not limit the referenced component to a particular location unless expressly stated. The term “based on” is intended to mean “based at least in part on.”

[0228] The claims may be drafted to exclude any element which may be optional. As such, this statement is intended to sen e as antecedent basis for use of such exclusive terminology as “solely”, “only”, and the like in connection with the recitation of claim elements, or the use of a “negative” limitation.

[0229] All patents, patent applications, publications, and descriptions mentioned herein are incorporated by reference in their entirety for all purposes. None is admitted as prior art. Where a conflict exists between the instant application and a reference provided herein, the instant application shall dominate.

Claims

WHAT IS CLAIMED IS:

1. A method comprising: obtaining, by a computer, interaction data and aggregated data for a current interaction, wherein the current interaction relates to a device requesting access to a resource; generating, by the computer, a feature vector using the interaction data and the aggregated data; generating, by the computer, a prompt comprising a set of feature-label pairs, each comprising: a task-specific label, and an additional feature vector including additional interaction data and additional aggregated data, wherein the additional interaction data and the additional aggregated data are related to an additional interaction of a same type of classification as the current interaction; loading, into a memory of the computer, a pre-trained machine learning model trained to determine predictions for specific tasks; inputting the prompt and a query including the feature vector into the pretrained machine learning model; and determining, by the computer using the pre-trained machine learning model, a task-specific prediction for the query and the prompt, the task-specific prediction specifying a response state to the current interaction.

2. The method of claim 1, wherein the pre-trained machine learning model comprises a neural network model.

3. The method of claim 1 further comprising: generating, by the computer, the query using the feature vector.

4. The method of claim 1 further comprising: obtaining, by the computer, a plurality of historical interaction data, a plurality of historical aggregated data, and a plurality of historical task specific labels for a plurality of historical interactions; generating, by the computer, a plurality' of historical feature vectors using the plurality of historical interaction data, the plurality of historical aggregated data, and the plurality of historical task specific labels;generating, by the computer, a historical prompt and a historical query using the plurality of historical feature vectors; and training, by the computer, the pre-trained machine learning model using the historical prompt and the historical query.

5. The method of claim 1, wherein the pre-trained machine learning model includes a plurality of interaction encoders, a plurality of task specific label encoders, and a plurality of mixer encoders.

6. The method of claim 5, wherein each mixer encoder of the plurality of mixer encoders combine encodings created by a subset of the plurality of interaction encoders and a subset of the plurality of task specific label encoders.

7. The method of claim 5, wherein determining the task-specific prediction comprises: generating, by the computer using the pre-trained machine learning model, a current interaction encoding from the feature vector; generating, by the computer using the pre-trained machine learning model, a set of interaction encodings from the additional feature vector of each feature-label pair of the set of feature-label pairs; generating, by the computer using the pre-trained machine learning model, a set of label encodings from the task-specific label of each feature-label pair of the set of feature-label pairs; and generating, by the computer using the pre-trained machine learning model, a set of mixed encodings from the set of interaction encodings and the set of label encodings that correspond to the same feature-label pair of the set of feature-label pairs.

8. The method of claim 7 further comprising: determining, by the computer using the pre-trained machine learning model, a set of attention scores based on the set of mixed encodings using a self-attention process; modifying, by the computer using the pre-trained machine learning model, the set of mixed encodings using the set of attention scores using the self-attention process; determining, by the computer using the pre-trained machine learning model, an output vector based on the current interaction encoding and the modified set of mixed encodings; anddetermining, by the computer using the pre-trained machine learning model, the task-specific prediction using a regression head or a classification head based on a current task.

9. The method of claim 8, wherein the current task is indicated by the task-specific label of each feature-label pair of the set of feature-label pairs.

10. The method of claim 1, wherein the response state is authorized or is not authorized.

11. The method of claim 1 , wherein the interaction data includes an indication of a type of interaction of a plurality of types of interactions.

12. The method of claim 1, wherein the aggregated data includes aggregated account data and aggregated resource provider data.

13. The method of claim 1, wherein the interaction data includes an indication of a type of interaction of a plurality of types of interactions, wherein generating the feature vector using the interaction data and the aggregated data comprises: generating, by the computer, an interaction data portion of the feature vector, wherein the interaction data portion includes a plurality7of interaction type entries; setting, by the computer, values of an interaction type entry of the plurality of interaction type entries that corresponds to the type of interaction to values of the interaction data; and setting, by the computer, values of other interaction type entries of the plurality of interaction type entries that do not correspond to the type of interaction to a default value.

14. The method of claim 1, wherein the pre-trained machine learning model includes a plurality of interaction encoders, a plurality7of task specific label encoders, and a plurality of mixer encoders, wherein the number of mixer encoders in the plurality of mixer encoders is the same as the number of feature-label pairs in the set of feature-label pairs.

15. The method of claim 1, wherein the method further comprises:iteratively training, by the computer, the pre-trained machine learning model using historical interaction data, historical aggregated data, historical task-specific labels, and training masks.

16. The method of claim 1, wherein the current interaction is of a classification of authorization interaction, and wherein the additional interaction is of the classification of authorization interaction.

17. The method of claim 1, wherein obtaining the interaction data and the aggregated data for the current interaction comprises: obtaining, by the computer the interaction data and the aggregated data from an interaction database, wherein a network processing computer stores the interaction data into the interaction database.

18. The method of claim 1, wherein after determining the task-specific prediction, the method further comprises: generating, by the evaluation computer, a prediction message comprising the task-specific prediction; and providing, by the evaluation computer, the prediction message to the downstream device.

19. A computer product comprising a computer readable medium storing a plurality7of instructions for controlling a computer system to perform operations of any of the methods above.

20. A system comprising: the computer product of claim 19; and one or more processors for executing instructions stored on the computer readable medium.

Images