System, method, and computer program product for interfacing with a plurality of machine-learning models

Abstract

Systems, methods, and computer program products are provided for interfacing with machine-learning models. The system includes at least one processor configured to: receive, from a plurality of remote issuer systems via an interface with the at least one processor, a plurality of model outputs from a plurality of machine-learning models, each model output of the plurality of model outputs resulting from a same input processed by a separate machine-learning model trained by an issuer system of the plurality of remote issuer systems; and aggregate the plurality of model outputs to generate an aggregated output.

Description

SYSTEM, METHOD, AND COMPUTER PROGRAM PRODUCT FOR INTERFACING WITH A PLURALITY OF MACHINE-LEARNING MODELSCROSS REFERENCE TO RELATED APPLICATION

[0001] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 550,643, filed on February 7, 2024, the disclosure of which is hereby incorporated by reference in its entirety.BACKGROUND1 . Technical Field

[0002] This disclosure relates generally to machine-learning and, in non-limiting embodiments or aspects, to systems, methods, and computer program products for interfacing with a plurality of machine-learning models.2. Technical Considerations

[0003] Building advanced artificial intelligence and / or machine-learning solutions using distributed datasets presents issues of privacy protection and security governance. Such technical limitations can prevent a party, such as a financial institution, a centralized entity, and / or the like, to train machine-learning model(s) with data from heterogenous sources. However, it is desirable to obtain the output from ensembling models across sources and / or jurisdictions to achieve more robust and better results.SUMMARY

[0004] According to non-limiting embodiments or aspects, provided is a system comprising: at least one processor configured to: receive, from a plurality of remote issuer systems via an interface with the at least one processor, a plurality of model outputs from a plurality of machine-learning models, each model output of the plurality of model outputs resulting from a same input processed by a separate machinelearning model trained by an issuer system of the plurality of remote issuer systems; and aggregate the plurality of model outputs to generate an aggregated output.

[0005] In non-limiting embodiments or aspects, the system further includes: a transaction processing system comprising the at least one processor, the transaction processing system configured to host the interface. In non-limiting embodiments or aspects, the at least one processor is further configured to: generate a graphical user interface comprising a global profile for an account holder, the same input comprises an identifier of the account holder, and wherein each model output of the plurality of model outputs further results from an input of transaction data of the issuer system ofthe plurality of remote issuer systems. In non-limiting embodiments or aspects, the plurality of model outputs comprises a plurality of scores, and the aggregated output is based on each score of the plurality of scores. In non-limiting embodiments or aspects, each machine-learning model of the plurality of machine-learning models comprises a different large language model (LLM). In non-limiting embodiments or aspects, the at least one processor is configured to aggregate the plurality of model outputs to generate the aggregated output comprises at least one of the following: averaging the plurality of model outputs, stacking the plurality of model outputs, weighing the plurality of model outputs, or any combination thereof.

[0006] According to non-limiting embodiments or aspects, provided is a method comprising: receiving, with at least one processor, from a plurality of remote issuer systems via an interface with the at least one processor, a plurality of model outputs from a plurality of machine-learning models, each model output of the plurality of model outputs resulting from a same input processed by a separate machine-learning model trained by an issuer system of the plurality of remote issuer systems; and aggregating, with the at least one processor, the plurality of model outputs to generate an aggregated output.

[0007] In non-limiting embodiments or aspects, the methods includes: hosting the interface with a transaction processing system. In non-limiting embodiments or aspects, the methods includes: generating a graphical user interface comprising a global profile for an account holder, the same input comprises an identifier of the account holder, and wherein each model output of the plurality of model outputs further results from an input of transaction data of the issuer system of the plurality of remote issuer systems. In non-limiting embodiments or aspects, the plurality of model outputs comprises a plurality of scores, and the aggregated output is based on each score of the plurality of scores. In non-limiting embodiments or aspects, each machine-learning model of the plurality of machine-learning models comprises a different large language model. In non-limiting embodiments or aspects, aggregating the plurality of model outputs to generate the aggregated output comprises at least one of the following: averaging the plurality of model outputs, stacking the plurality of model outputs, weighing the plurality of model outputs, or any combination thereof.

[0008] According to non-limiting embodiments or aspects, provided is a computer program product, comprising at least one non-transitory computer-readable medium including program instructions that, when executed by at least one processor, causethe at least one processor to: receive, from a plurality of remote issuer systems via an interface with the at least one processor, a plurality of model outputs from a plurality of machine-learning models, each model output of the plurality of model outputs resulting from a same input processed by a separate machine-learning model trained by an issuer system of the plurality of remote issuer systems; and aggregate the plurality of model outputs to generate an aggregated output.

[0009] In non-limiting embodiments or aspects, the at least one processor is part of a transaction processing system configured to host the interface. In non-limiting embodiments or aspects, the at least one processor is further caused to: generate a graphical user interface comprising a global profile for an account holder, the same input comprises an identifier of the account holder, and wherein each model output of the plurality of model outputs further results from an input of transaction data of the issuer system of the plurality of remote issuer systems. In non-limiting embodiments or aspects, the plurality of model outputs comprises a plurality of scores, and the aggregated output is based on each score of the plurality of scores. In non-limiting embodiments or aspects, wherein each machine-learning model of the plurality of machine-learning models comprises a different large language model (LLM). In nonlimiting embodiments or aspects, wherein aggregating the plurality of model outputs to generate the aggregated output comprises at least one of the following: averaging the plurality of model outputs, stacking the plurality of model outputs, weighing the plurality of model outputs, or any combination thereof.

[0010] Clause 1 : A system comprising: at least one processor configured to: receive, from a plurality of remote issuer systems via an interface with the at least one processor, a plurality of model outputs from a plurality of machine-learning models, each model output of the plurality of model outputs resulting from a same input processed by a separate machine-learning model trained by an issuer system of the plurality of remote issuer systems; and aggregate the plurality of model outputs to generate an aggregated output.

[0011] Clause 2: The system of clause 1 , further comprising: a transaction processing system comprising the at least one processor, the transaction processing system configured to host the interface.

[0012] Clause 3: The system of clause 1 or 2, wherein the at least one processor is further configured to: generate a graphical user interface comprising a global profile for an account holder, wherein the same input comprises an identifier of the accountholder, and wherein each model output of the plurality of model outputs further results from an input of transaction data of the issuer system of the plurality of remote issuer systems.

[0013] Clause 4: The system of any of clauses 1 -3, wherein the plurality of model outputs comprises a plurality of scores, and wherein the aggregated output is based on each score of the plurality of scores.

[0014] Clause 5: The system of any of clauses 1 -4, wherein each machine-learning model of the plurality of machine-learning models comprises a different large language model (LLM).

[0015] Clause 6: The system of any of clauses 1 -5, wherein the at least one processor is configured to aggregate the plurality of model outputs to generate the aggregated output comprises at least one of the following: averaging the plurality of model outputs, stacking the plurality of model outputs, weighing the plurality of model outputs, or any combination thereof.

[0016] Clause 7: A method comprising: receiving, with at least one processor, from a plurality of remote issuer systems via an interface with the at least one processor, a plurality of model outputs from a plurality of machine-learning models, each model output of the plurality of model outputs resulting from a same input processed by a separate machine-learning model trained by an issuer system of the plurality of remote issuer systems; and aggregating, with the at least one processor, the plurality of model outputs to generate an aggregated output.

[0017] Clause 8: The method of clause 7, further comprising: hosting the interface with a transaction processing system.

[0018] Clause 9: The method of clause 7 or 8, further comprising: generating a graphical user interface comprising a global profile for an account holder, wherein the same input comprises an identifier of the account holder, and wherein each model output of the plurality of model outputs further results from an input of transaction data of the issuer system of the plurality of remote issuer systems.

[0019] Clause 10: The method of any of clauses 7-9, wherein the plurality of model outputs comprises a plurality of scores, and wherein the aggregated output is based on each score of the plurality of scores.

[0020] Clause 11 : The method of any of clauses 7-10, wherein each machinelearning model of the plurality of machine-learning models comprises a different large language model.

[0021] Clause 12: The method of any of clauses 7-1 1 , wherein aggregating the plurality of model outputs to generate the aggregated output comprises at least one of the following: averaging the plurality of model outputs, stacking the plurality of model outputs, weighing the plurality of model outputs, or any combination thereof.

[0022] Clause 13: A computer program product, comprising at least one non- transitory computer-readable medium including program instructions that, when executed by at least one processor, cause the at least one processor to: receive, from a plurality of remote issuer systems via an interface with the at least one processor, a plurality of model outputs from a plurality of machine-learning models, each model output of the plurality of model outputs resulting from a same input processed by a separate machine-learning model trained by an issuer system of the plurality of remote issuer systems; and aggregate the plurality of model outputs to generate an aggregated output.

[0023] Clause 14: The computer program product of clause 13, wherein the at least one processor is part of a transaction processing system configured to host the interface.

[0024] Clause 15: The computer program product of clause 13 or 14, wherein the at least one processor is further caused to: generate a graphical user interface comprising a global profile for an account holder, wherein the same input comprises an identifier of the account holder, and wherein each model output of the plurality of model outputs further results from an input of transaction data of the issuer system of the plurality of remote issuer systems.

[0025] Clause 16: The computer program product of any of clauses 13-15, wherein the plurality of model outputs comprises a plurality of scores, and wherein the aggregated output is based on each score of the plurality of scores.

[0026] Clause 17: The computer program product of any of clauses 13-16, wherein each machine-learning model of the plurality of machine-learning models comprises a different large language model (LLM).

[0027] Clause 18: The computer program product of any of clauses 13-17, wherein aggregating the plurality of model outputs to generate the aggregated output comprises at least one of the following: averaging the plurality of model outputs, stacking the plurality of model outputs, weighing the plurality of model outputs, or any combination thereof.

[0028] These and other features and characteristics of the present disclosure, as well as the methods of operation and functions of the related elements of structures and the combination of parts and economies of manufacture, will become more apparent upon consideration of the following description and the appended claims with reference to the accompanying drawings, all of which form a part of this specification, wherein like reference numerals designate corresponding parts in the various figures. It is to be expressly understood, however, that the drawings are for the purpose of illustration and description only and are not intended as a definition of the limits of the disclosed subject matter.BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Additional advantages and details are explained in greater detail below with reference to the non-limiting, exemplary embodiments that are illustrated in the accompanying schematic figures and appendix, in which:

[0030] FIG. 1 is a schematic diagram of a system for interfacing with a plurality of machine-learning models, according to some non-limiting embodiments or aspects;

[0031] FIG. 2 is a flow diagram of a method for interfacing with a plurality of machine-learning models, according to some non-limiting embodiments or aspects;

[0032] FIGS. 3A and 3B illustrate a system for interfacing with a plurality of machine-learning models, according to some non-limiting embodiments or aspects;

[0033] FIG. 4 is a schematic diagram of example components of one or more devices of FIG. 1 , according to some non-limiting embodiments or aspects; and

[0034] FIG. 5 is a schematic diagram of an electronic payment processing network used in connection with non-limiting embodiments or aspects.DETAILED DESCRIPTION

[0035] For purposes of the description hereinafter, the terms “end,” “upper,” “lower,” “right,” “left,” “vertical,” “horizontal,” “top,” “bottom,” “lateral,” “longitudinal,” and derivatives thereof shall relate to the embodiments as they are oriented in the drawing figures. However, it is to be understood that the embodiments may assume various alternative variations and step sequences, except where expressly specified to the contrary. It is also to be understood that the specific devices and processes illustrated in the attached drawings, and described in the following specification, are simply exemplary embodiments or aspects of the disclosed subject matter. Hence, specific dimensions and other physical characteristics related to the embodiments or aspects disclosed herein are not to be considered as limiting.

[0036] It is to be understood that the present disclosure may assume various alternative variations and step sequences, except where expressly specified to the contrary. It is also to be understood that the specific devices and processes illustrated in the attached drawings, and described in the following specification, are simply exemplary and non-limiting embodiments or aspects. Hence, specific dimensions and other physical characteristics related to the embodiments or aspects disclosed herein are not to be considered as limiting.

[0037] Some non-limiting embodiments or aspects are described herein in connection with thresholds. As used herein, satisfying a threshold may refer to a value being greater than the threshold, more than the threshold, higher than the threshold, greater than or equal to the threshold, less than the threshold, fewer than the threshold, lower than the threshold, less than or equal to the threshold, equal to the threshold, etc.

[0038] No aspect, component, element, structure, act, step, function, instruction, and / or the like used herein should be construed as critical or essential unless explicitly described as such. Also, as used herein, the articles “a” and “an” are intended to include one or more items and may be used interchangeably with “one or more” and “at least one.” Furthermore, as used herein, the term “set” is intended to include one or more items (e.g., related items, unrelated items, a combination of related and unrelated items, and / or the like) and may be used interchangeably with “one or more” or “at least one.” Where only one item is intended, the term “one” or similar language is used. Also, as used herein, the terms “has,” “have,” “having,” or the like are intended to be open-ended terms. Further, the phrase “based on” is intended to mean “based at least partially on” unless explicitly stated otherwise. In addition, reference to an action being “based on” a condition may refer to the action being “in response to” the condition. For example, the phrases “based on” and “in response to” may, in some non-limiting embodiments or aspects, refer to a condition for automatically triggering an action (e.g., a specific operation of an electronic device, such as a computing device, a processor, and / or the like).

[0039] As used herein, the term “communication” may refer to the reception, receipt, transmission, transfer, provision, and / or the like of data (e.g., information, signals, messages, instructions, commands, and / or the like). For one unit (e.g., a device, a system, a component of a device or system, combinations thereof, and / or the like) to be in communication with another unit means that the one unit is able todirectly or indirectly receive information from and / or transmit information to the other unit. This may refer to a direct or indirect connection (e.g., a direct communication connection, an indirect communication connection, and / or the like) that is wired and / or wireless in nature. Additionally, two units may be in communication with each other even though the information transmitted may be modified, processed, relayed, and / or routed between the first and second unit. For example, a first unit may be in communication with a second unit even though the first unit passively receives information and does not actively transmit information to the second unit. As another example, a first unit may be in communication with a second unit if at least one intermediary unit processes information received from the first unit and communicates the processed information to the second unit. In some non-limiting embodiments or aspects, a message may refer to a network packet (e.g., a data packet and / or the like) that includes data. It will be appreciated that numerous other arrangements are possible.

[0040] As used herein, the term “computing device” may refer to one or more electronic devices configured to process data. A computing device may, in some examples, include the necessary components to receive, process, and output data, such as a processor, a display, a memory, an input device, a network interface, and / or the like. A computing device may be a mobile device. As an example, a mobile device may include a cellular phone (e.g., a smartphone or standard cellular phone), a portable computer, a wearable device (e.g., watches, glasses, lenses, clothing, and / or the like), a personal digital assistant (PDA), and / or other like devices. A computing device may also be a desktop computer or other form of non-mobile computer.

[0041] As used herein, the term “issuer institution” may refer to one or more entities, such as a bank, that provide accounts to customers for conducting transactions (e.g., payment transactions), such as initiating credit and / or debit payments. For example, an issuer institution may provide an account identifier, such as a primary account number (PAN), to a customer that uniquely identifies one or more accounts associated with that customer. The account identifier may be embodied on a portable financial device, such as a physical financial instrument, e.g., a payment card, and / or may be electronic and used for electronic payments. The term “issuer system” refers to one or more computer devices operated by or on behalf of an issuer institution, such as a server computer executing one or more software applications. For example, an issuer system may include one or more authorization servers for authorizing a transaction.

[0042] As used herein, the term “merchant” may refer to one or more entities (e.g., operators of retail businesses that provide goods and / or services, and / or access to goods and / or services, to a user (e.g., a customer, a consumer, a customer of the merchant, and / or the like) based on a transaction (e.g., a payment transaction)). As used herein, the term “merchant system” may refer to one or more computer systems operated by or on behalf of a merchant, such as a server computer executing one or more software applications. As used herein, the term “product” may refer to one or more goods and / or services offered by a merchant.

[0043] As used herein, the term “server” may refer to or include one or more computing devices that are operated by or facilitate communication and processing for multiple parties in a network environment, such as the Internet, although it will be appreciated that communication may be facilitated over one or more public or private network environments and that various other arrangements are possible. Further, multiple computing devices (e.g., servers, point-of-sale (POS) devices, mobile devices, etc.) directly or indirectly communicating in the network environment may constitute a “system.”

[0044] As used herein, the term “system” may refer to one or more computing devices or combinations of computing devices and / or components of such (e.g., processors, servers, client devices, software applications, and / or the like). Reference to “a device,” “a server,” “a processor,” and / or the like, as used herein, may refer to a previously-recited device, server, or processor that is recited as performing a previous step or function, a different device, server, or processor, and / or a combination of devices, servers, and / or processors. For example, as used in the specification and the claims, a first device, a first server, or a first processor that is recited as performing a first step or a first function may refer to the same or different device, server, or processor recited as performing a second step or a second function.

[0045] As used herein, the term “transaction service provider” may refer to an entity that receives transaction authorization requests from merchants or other entities and provides guarantees of payment, in some cases through an agreement between the transaction service provider and an issuer institution. For example, a transaction service provider may include a payment network such as Visa® or any other entity that processes transactions. The term “transaction processing system” may refer to one or more computer systems operated by or on behalf of a transaction service provider, such as a transaction processing server executing one or more softwareapplications. A transaction processing server may include one or more processors and, in some non-limiting embodiments or aspects, may be operated by or on behalf of a transaction service provider.

[0046] Non-limiting embodiments provide for systems, methods, and computer program products for interfacing with machine-learning models that are trained with distributed datasets. Individual models trained with different datasets may be maintained by one or more entities for improving security, privacy, and / or the like. However, having decentralized training datasets results in different models that may produce different results, and such disparate results may not accurately reflect the desired modeled parameters. For example, in electronic payment networks, different issuer systems may maintain separate models and / or training datasets for models, but such model outputs only reflect payments that the specific issuer processed and does not reflect other activity by the same entity, individual, and / or account holder with respect to other financial institutions. Thus, non-limiting embodiments address this limitation of existing payment networks by providing for an interface to multiple different models that are trained with decentralized datasets, such that model outputs from several different models can be obtained and aggregated based on a same or similar input. Non-limiting embodiments provide for a computationally efficient way to leverage multiple different models without having to maintain and / or train all of the different models. The decentralized nature of the training of the machine-learning models distributes the usage of computational resources across several different systems.

[0047] In non-limiting embodiments, a machine-learning model programmable interface is provided to enable a new model to be developed based on other models that are separately trained in silos (e.g., trained in a decentralized manner with limited and / or specific data). This allows for individual models to be trained in their own space, at a pace chosen by an entity associated with each model, and under the primary control of an entity associated with each model. Moreover, model data, such as gradients, model architecture, confidential training data, and / or the like, may be protected and not shared by an entity associated with the model.

[0048] In existing modeling systems, models and / or certain data may be hosted on-premises with an entity for security purposes. Non-limiting embodiments improve upon existing systems by allowing for pre-trained models to be hosted in a network environment, without exposing the data itself, and for such models to be modified (e.g.,fine-tuned and / or adjusted) and used to create and / or train new models. For example, using a Model Development Kit (MDK) based on the non-limiting embodiments described herein, model developers (e.g., users and / or automated processes / agents) may further specify target variables, add one or more layers (e.g., such as a dense layer) for fine-tuning, train new models for downstream tasks, specify how outputs should be combined, and / or the like. As a further example, in non-limiting embodiments, a payment model may be hosted by a transaction processing system and / or issuer system that can be used to predict activity and / or determine actions to be performed with respect to payment transactions (e.g., authorization, denial, risk detection, and / or the like).

[0049] FIG. 1 shows a system 1000 for interfacing with machine-learning models according to some non-limiting embodiments or aspects. A new model system 100 may include one or more computing devices, such as a server computer, arranged in a network environment. In some non-limiting examples, the new model system 100 may be part of and / or hosted by a transaction processing system. However, it will be appreciated that other systems and / or entities may incorporate and / or host the new model system 100.

[0050] With continued reference to FIG. 1 , the system 1000 includes an interface 110 configured to communicate between the new model system 100 and a plurality of different machine-learning models 115, 117, 119. The interface 110 may include one or more Application Programming Interfaces (APIs) in which requests and responses are communicated between systems (e.g., new model system and machine-learning models 115, 117, 119) in a predetermined format. In non-limiting embodiments, the interface 110 represents program logic for communicating between the new model system 100 and the models 115, 117, 119. Such interfacing may be conducted based on functions invoked from a development kit, such as an MDK that includes functions and / or tools for specifying one or more parameters relating to the models 115, 117, 119. As an example, the MDK may be used to fine-tune one or more of the models, specify how the outputs should be combined, specify weights to be given to each output / model, and / or the like.

[0051] In non-limiting embodiments, the interface 110 may communicate with the machine-learning models 115, 117, 119 and / or issuer systems 114, 116, 118 via a network protocol (e.g., via TCP / IP and predetermined ports or the like). The interface 110 may be configured to communicate in different ways (e.g., different messageformats, different network addresses, different ports, and / or the like) with each of the different models 1 15, 1 17, 119. In this manner, a single request message 103 can be generated by a requesting entity (e.g., the new model system 100, a transaction processing system, an authentication system, an issuer system, and / or any other like system or entity). The single request message 103 may represent a uniform input that is provided to each machine-learning model 1 15, 117, 119 such that each model is executed with the same or substantially the same input. The single request message 103 may include, as an example, an account identifier, user identifier, transaction data (e.g., data representing previous transactions, such as transaction amount, transacting parties, time / date, etc.), and / or the like.

[0052] Still referring to FIG. 1 , in some non-limiting embodiments, the machinelearning models 1 15, 1 17,1 19 may be hosted and / or maintained by different entities (e.g., issuer systems 114, 1 16, 1 18). It will be appreciated that any type of system or entity may host and / or maintain a machine-learning model in the system 1000 and that issuer systems are shown as one example. In non-limiting embodiments, the machinelearning models 1 15, 117, 1 19 may be trained in a decentralized manner with different sets of training data 104, 106, 108. For example, in non-limiting embodiments, different issuer systems 114, 1 16, 1 18 may separately train respective models 1 15, 1 17, 1 19 based on training data (e.g., such as but not limited to transaction data) that the respective issuer systems have access to.

[0053] In non-limiting embodiments, the interface 110 returns a plurality of model outputs 105 in response to the single request message 103. The new model system 100, which may include an ensemble model, may collect the plurality of model outputs 105, store the outputs 105 in a database 102, and aggregate and / or combine the outputs 105. The aggregated output may also be stored in the database 102 and / or be provided to a requesting entity. It will be appreciated that the outputs 105 may be aggregated in different ways. For example, for numerical outputs (e.g., such as fraud scores, risk scores, and / or the like), the values may be averaged, weighed and averaged, added, subject to majority voting protocol(s) (e.g., based on a comparison of outputs), and / or the like. For textual outputs, the text may be concatenated, processed and summarized, and / or the like.

[0054] In non-limiting embodiments, the machine-learning models 1 15, 1 17, 1 19 may include fraud scoring models and / or risk scoring models configured to determine a fraud score or risk score (e.g., representing the likelihood of potential fraud or therelative amount of risk associated with a transaction). In this manner, a requesting entity (e.g., a transaction processing system, an issuer system, an authentication system, and / or the like) may use the aggregated score to make a determination (e.g., to determine if the score satisfies a threshold associated with approving or declining a transaction).

[0055] In non-limiting embodiments, the machine-learning models 1 15, 1 17, 1 19 may include different large language models (LLMs) that provide textual and / or visual outputs. The outputs can be aggregated via concatenation, processing each output and generating a summary, and / or the like. For example, non-limiting embodiments may be used to understand a complaint submitted by a consumer and to provide a personalized response. In such an example, different LLMs may be used to generate a narrative that describes the customer’s spending history. The prompt for the LLM may be chosen, inputted, generated, and / or the like in connection with the initial model inference request. Each output may include a description of parameters, such as payment amounts, volumes / frequency, categories (e.g., low spender, high spender, etc.), and / or the like, which may be combined during aggregation (e.g., average payment amount, average frequency, frequent merchant categories, and / or the like). Other parameters, such as merchant city / location (e.g., number of locations visited, most recently visited, most frequently visited, and / or the like), may also be identified, described, and / or combined. It will be appreciated that other types of narratives may be generated and aggregated.

[0056] In some non-limiting embodiments, the system 1000 for interfacing with machine-learning models may be used to predict a probable destination for an account holder if they were to travel in an upcoming period of time (e.g., 60 to 90 days or the like). In such an instance, each model may be instructed to determine a predicted city and / or country based on the data it was trained with. The new model aggregation system may then determine the median or most predicted city and / or country. The new model system may also weigh the different predictions based on how many other models predicted the same city and / or country.

[0057] Non-limiting embodiments of the systems and methods described herein may be used by model developers to improve and / or customize one or more pretrained models (e.g., a deep learning model trained on large datasets to accomplish a specific task). As an example, for implementing downstream tasks like fraud detection, an MDK may be utilized by model developers to append a linear layer, specify a lossfunction, and / or configure an optimization algorithm. Following this, the training process can be initiated to fine-tune the model.

[0058] Non-limiting embodiments of the systems and methods described herein may be used for model ensembling (e.g., combining the predictions from multiple models). As an example, an MDK may be used by model developers to specify weights for each model output.

[0059] Non-limiting embodiments of the systems and methods described herein may be used by model developers to develop a new model that selects an output from multiple, different generative models (e.g., such as LLMs) based on content qualities and / or generates a new output (e.g., such as a new text sequence and / or the like) based on one or more of the outputs. This may then provide a single textual output, as an example.

[0060] In non-limiting embodiments, one or more graphical user interfaces (GUIs) 130 may be generated and displayed to a user of the system. For example, a GU1 130 may display a global profile for an account holder, including an indentifier of the account holder (e.g., account identifier, user identifier, and / or the like). One or more GUIs 130 may be used to input or modify data into the models and / or to specify the format of the model output, to identify each of the plurality of models to interface with (e.g., by network address and / or the like).

[0061] The number and arrangement of systems and devices shown in FIG. 1 are provided as an example. There may be additional systems and / or devices, fewer systems and / or devices, different systems and / or devices, and / or differently arranged systems and / or devices than those shown in FIG. 1 . Furthermore, two or more systems or devices shown in FIG. 1 may be implemented within a single system or device, or a single system or device shown in FIG. 1 may be implemented as multiple, distributed systems or devices. Additionally or alternatively, a set of systems (e.g., one or more systems) or a set of devices (e.g., one or more devices) of the system 1000 may perform one or more functions described as being performed by another set of systems or another set of devices of the system 1000.

[0062] Referring now to FIG. 2, shown is a flow diagram for a method for interfacing with machine-learning models according to some non-limiting embodiments or aspects. The steps shown in FIG. 2 are for example purposes only. It will be appreciated that additional, fewer, different, and / or a different order of steps may be used in some non-limiting embodiments or aspects. In some non-limiting embodimentsor aspects, a step may be automatically performed in response to performance and / or completion of a prior step. At a first step 200, a plurality of separate machine-learning models may be separately trained with separate, decentralized datasets. For example, each of a plurality of systems (e.g., issuer systems) may maintain a decentralized dataset of training data unique to each system. This decentralized dataset may be used to train a machine-learning model that is hosted by the system that maintains the dataset or another system on behalf of such system. Step 200 may occur over a period of time, where each separate machine-learning model is trained over time by a respective system.

[0063] With continued reference to FIG. 2, at step 202, a model inference request is generated. The model inference request may be generated in response to user input or automatically (e.g., at predetermined intervals, in response to an event, and / or the like). The model inference request may include a message containing input for the models. The input may include, for example, an account identifier, a user identifier, transaction data for one or more account holders, and / or the like. At step 204 the model inference request is communicated to each model of the plurality of machinelearning models trained at step 200. For example, an interface may receive a single model inference request and generate separate individual model inference requests for each model of the plurality of models. The separate individual model inference requests may be customized by the interface 1 10 for the specific model.

[0064] Still referring to FIG. 2, at step 206 a plurality of model outputs are received. The outputs may be received in a single batch or may be received separately as the outputs are made available by a respective machine-learning model. At step 208, the outputs are aggregated. For example, the values may be averaged, weighed and averaged, added, subjected to majority voting protocol(s) (e.g., based on a comparison of outputs), and / or the like. In non-limiting embodiments, masking logic may be applied to mask and / or de-identify the data being aggregated so that it can be shared without concern of privacy violations or security risks.

[0065] Non-limiting embodiments of systems and methods for interfacing with machine-learning models may be used for different purposes. Non-limiting embodiments may be configured and implemented to provide customized (e.g., personalized) responses to customers of a merchant (e.g., shoppers). In a customer service environment, for example, the context may be established with comprehensive customer information (e.g., customer profile), purchase history, interaction history(e.g., interaction log), and any pending issues. The output (e.g., target variable) may be a personalized response generated by artificial intelligence, which includes a tailored, dynamically generated runbook (e.g., steps and / or a plan for completing one or more tasks) and suggested actions to efficiently resolve one or more customer issues. This approach ensures that a customer support agent has all the necessary information and guidance to provide exceptional customer service.

[0066] Non-limiting embodiments of systems and methods for interfacing with machine-learning models may be configured and implemented to predict a most likely destination for an account holder if they travel in an upcoming time period (e.g., the next 60 to 90 days or the like). Such cross-border destination model(s) may be configured to output a target location (e.g., such as a country and / or city). The input features for such an implementation may include, for example, time frame features such as quarterly, bi-quarterly, tri-quarterly, one year and / or the like, base line features such as payment volume, transaction volume, type of transactions, average ticket (e.g., average spend), customer category (e.g., high spender, average spender, low spender, and / or the like), and / or the like, and aggregated features such as a number of merchant locations visited, the most recent merchant location visited, the most frequently visited merchant location, and / or the like. Different cross-broder destination models under the control of different entities may have access to different training data and therefore product different results. For example, an account holder may have accounts with several different issuers and / or payment platforms, each of which may have an internal and / or private model.

[0067] Referring now to FIG. 3A, a system 3000 for interfacting with machinelearning models is shown according to non-limiting embodiments. An API 300 may be hosted by a transaction processing system and / or other system in a payment processing network. In this example, an issuer database Db is in communication with an internal model Mb that an issuer system associated with the issuer database Db uses to process data stored by the database Db, such as but not limited to transaction data, account holder data, and / or the like, that may represent one or more parameters about purchases made by account holders, demographic information of the account holders, and / or the like. The details of model Mb may be only known to that issuer system (or other model host) such that model gradients are not published outside of the issuer system and / or are accessible from outside of the issuer system (or other model host). Thus, Mb may be trained internally to an issuer.

[0068] With continued reference to FIG. 3A, a transaction database DT may be hosted by a transaction processing system, a payment gateway, and / or the like, and may include transaction data from transactions processed and / or authorized by the transaction processing system, payment gateway, and / or other system in communication with the issuer system hosting database Db. The transaction database may be in communication with a different model Mvfor processing transaction data. Model Mv may be trained internal to a transaction processing system and / or payment gateway, for example, such that model gradients are not published outside the system. The outputs of model Mv and model Mb may be combined via ensemble learning techniques to form model output ME. For example, techniques such as averaging, voting, and / or stacking may be used to combine outputs from models Mb and Mv. It will be appreciated that additional models may also be used with respective outputs also being ensembled together with the outputs of Mv and Mb. The ensembled model outputs may be processed with one or more models Mn configured to generate a narrative output 302 that is provided to the issuer system via the API 300. Model Mn may be a large language model (LLM) and / or may be the same as model Mv.

[0069] With reference to FIG. 3B, a system 3001 for interfacting with machinelearning models is shown according to non-limiting embodiments. In this example, the outputs of models Mv and Mb, as discussed above in connection with FIG. 3A, may be ensembled into model output Mv+b. This may include combined outputs from a first issuer system (e.g., first bank) and a transaction processing system, as an example. A second issuer system (e.g., “Silver Bank”) may have its own internal model Ms. The system 3001 faciliates the combination of the output of model Ms with the model output Mv+b to form Mv+b+s. In this manner, the ensemble learning may be implemented iteratively, starting with combining two or more model outputs and then iteratively combining additional model outputs. In non-limiting embodiments, issuer systems associated with models Mb and Ms may train their models at their own pace and / or intervals. The example shown in FIGS. 3A and 3B involve models hosted by issuer systems, but it will be appreciated that different types of models may be used in nonlimiting embodiments.

[0070] Referring now to FIG. 4, shown is a diagram of example components of a device 400 according to non-limiting embodiments. Device 400 may correspond to the new model system 100, the interface 1 10, and / or issuer systems 1 14, 1 16, 1 18 in FIG. 1 , as an example. In some non-limiting embodiments, such systems or devices mayinclude at least one device 400 and / or at least one component of device 400. The number and arrangement of components shown are provided as an example. In some non-limiting embodiments, device 400 may include additional components, fewer components, different components, or differently arranged components than those shown. Additionally, or alternatively, a set of components (e.g., one or more components) of device 400 may perform one or more functions described as being performed by another set of components of device 400.

[0071] As shown in FIG. 4, device 400 may include a bus 402, a processor 404, memory 406, a storage component 408, an input component 410, an output component 412, and a communication interface 414. Bus 402 may include a component that permits communication among the components of device 400. In some non-limiting embodiments, processor 404 may be implemented in hardware, firmware, or a combination of hardware and software. For example, processor 404 may include a processor (e.g., a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), etc.), a microprocessor, a digital signal processor (DSP), and / or any processing component (e.g., a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), etc.) that can be programmed to perform a function. Memory 406 may include random access memory (RAM), read only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic memory, optical memory, etc.) that stores information and / or instructions for use by processor 404.

[0072] With continued reference to FIG. 4, storage component 408 may store information and / or software related to the operation and use of device 400. For example, storage component 408 may include a hard disk (e.g., a magnetic disk, an optical disk, a magneto-optic disk, a solid-state disk, etc.) and / or another type of computer-readable medium. Input component 410 may include a component that permits device 400 to receive information, such as via user input (e.g., a touch screen display, a keyboard, a keypad, a mouse, a button, a switch, a microphone, etc.). Additionally, or alternatively, input component 410 may include a sensor for sensing information (e.g., a global positioning system (GPS) component, an accelerometer, a gyroscope, an actuator, etc.). Output component 412 may include a component that provides output information from device 400 (e.g., a display, a speaker, one or more light-emitting diodes (LEDs), etc.). Communication interface 414 may include a transceiver-like component (e.g., a transceiver, a separate receiver and transmitter,etc.) that enables device 400 to communicate with other devices, such as via a wired connection, a wireless connection, or a combination of wired and wireless connections. Communication interface 414 may permit device 400 to receive information from another device and / or provide information to another device. For example, communication interface 414 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi® interface, a cellular network interface, and / or the like.

[0073] Device 400 may perform one or more processes described herein. Device 400 may perform these processes based on processor 404 executing software instructions stored by a computer-readable medium, such as memory 406 and / or storage component 408. A computer-readable medium may include any non-transitory memory device. A memory device includes memory space located inside of a single physical storage device or memory space spread across multiple physical storage devices. Software instructions may be read into memory 406 and / or storage component 408 from another computer-readable medium or from another device via communication interface 414. When executed, software instructions stored in memory 406 and / or storage component 408 may cause processor 404 to perform one or more processes described herein. Additionally, or alternatively, hardwired circuitry may be used in place of or in combination with software instructions to perform one or more processes described herein. Thus, embodiments described herein are not limited to any specific combination of hardware circuitry and software. The term “configured to,” as used herein, may refer to an arrangement of software, device(s), and / or hardware for performing and / or enabling one or more functions (e.g., actions, processes, steps of a process, and / or the like). For example, “a processor configured to” may refer to a processor that executes software instructions (e.g., program code) that cause the processor to perform one or more functions.

[0074] FIG. 5 shows an electronic payment processing network 1 100 according to non-limiting embodiments or aspects. The payment processing network may be used in conjunction with the systems and methods described herein. It will be appreciated that the particular arrangement of the electronic payment processing network 1 100 shown is for example purposes only, and that various arrangements are possible. A transaction processing system 1101 (e.g., a transaction handler) is shown to be in communication with one or more issuer systems (e.g., such as an issuer system 1 106)and one or more acquirer systems (e.g., such as an acquirer system 1 108). Although only a single issuer system 1 106 and single acquirer system 1108 are shown, it will be appreciated that the transaction processing system 1101 may be in communication with a plurality of issuer systems and / or acquirer systems. In some embodiments, the transaction processing system 1 101 may also operate as an issuer system such that both the transaction processing system 1 101 and issuer system 1106 are a single system and / or controlled by a single entity.

[0075] In some non-limiting embodiments or aspects, the transaction processing system 1 101 may communicate with a merchant system 1104 directly through a public or private network connection. Additionally or alternatively, the transaction processing system 1 101 may communicate with the merchant system 1104 through the payment gateway 1102 and / or acquirer system 1 108. In some non-limiting embodiments or aspects, an acquirer system 1 108 associated with the merchant system 1 104 may operate as the payment gateway 1 102 to facilitate the communication of transaction requests from the merchant system 1 104 to the transaction processing system 1 101. The merchant system 1 104 may communicate with the payment gateway 1 102 through a public or private network connection. For example, a merchant system 1 104 that includes a physical POS device may communicate with the payment gateway 1 102 through a public or private network to conduct card-present transactions. As another example, a merchant system 1104 that includes a server (e.g., a web server) may communicate with the payment gateway 1 102 through a public or private network, such as a public Internet connection, to conduct card-not-present transactions.

[0076] In some non-limiting embodiments or aspects, the transaction processing system 1101 , after receiving a transaction request from the merchant system 1 104 that identifies an account identifier of a payor (e.g., such as an account holder) associated with an issued consumer device 1 110, may generate an authorization request message to be communicated to the issuer system 1 106 that issued the consumer device 1 1 10 and / or account identifier. The issuer system 1 106 may then approve or decline the authorization request and, based on the approval or denial, generate an authorization response message that is communicated to the transaction processing system 1 101. The transaction processing system 1 101 may communicate an approval or denial to the merchant system 1 104. When the issuer system 1 106 approves the authorization request message, it may then clear and settle the payment transaction between the the issuer system 1 106 and acquirer system 1 108.

[0077] Although embodiments have been described in detail for the purpose of illustration, it is to be understood that such detail is solely for that purpose and that the disclosure is not limited to the disclosed embodiments or aspects, but, on the contrary, is intended to cover modifications and equivalent arrangements that are within the spirit and scope of the appended claims. For example, it is to be understood that the present disclosure contemplates that, to the extent possible, one or more features of any embodiment or aspect can be combined with one or more features of any other embodiment or aspect. In fact, any of these features can be combined in ways not specifically recited in the claims and / or disclosed in the specification. Although each dependent claim listed below may directly depend on only one claim, the disclosure of possible implementations includes each dependent claim in combination with every other claim in the claim set.

Claims

WHAT IS CLAIMED IS:1 . A system comprising: at least one processor configured to: receive, from a plurality of remote issuer systems via an interface with the at least one processor, a plurality of model outputs from a plurality of machine-learning models, each model output of the plurality of model outputs resulting from a same input processed by a separate machine-learning model trained by an issuer system of the plurality of remote issuer systems; and aggregate the plurality of model outputs to generate an aggregated output.

2. The system of claim 1 , further comprising: a transaction processing system comprising the at least one processor, the transaction processing system configured to host the interface.

3. The system of claim 1 , wherein the at least one processor is further configured to: generate a graphical user interface comprising a global profile for an account holder, wherein the same input comprises an identifier of the account holder, and wherein each model output of the plurality of model outputs further results from an input of transaction data of the issuer system of the plurality of remote issuer systems.

4. The system of claim 1 , wherein the plurality of model outputs comprises a plurality of scores, and wherein the aggregated output is based on each score of the plurality of scores.

5. The system of claim 1 , wherein each machine-learning model of the plurality of machine-learning models comprises a different large language model (LLM).

6. The system of claim 1 , wherein the at least one processor is configured to aggregate the plurality of model outputs to generate the aggregatedoutput by at least one of the following: averaging the plurality of model outputs, stacking the plurality of model outputs, weighing the plurality of model outputs, or any combination thereof.

7. A method comprising: receiving, with at least one processor, from a plurality of remote issuer systems via an interface with the at least one processor, a plurality of model outputs from a plurality of machine-learning models, each model output of the plurality of model outputs resulting from a same input processed by a separate machine-learning model trained by an issuer system of the plurality of remote issuer systems; and aggregating, with the at least one processor, the plurality of model outputs to generate an aggregated output.

8. The method of claim 7, further comprising: hosting the interface with a transaction processing system.

9. The method of claim 7, further comprising: generating a graphical user interface comprising a global profile for an account holder, wherein the same input comprises an identifier of the account holder, and wherein each model output of the plurality of model outputs further results from an input of transaction data of the issuer system of the plurality of remote issuer systems.

10. The method of claim 7, wherein the plurality of model outputs comprises a plurality of scores, and wherein the aggregated output is based on each score of the plurality of scores.1 1 . The method of claim 7, wherein each machine-learning model of the plurality of machine-learning models comprises a different large language model.

12. The method of claim 7, wherein aggregating the plurality of model outputs to generate the aggregated output comprises at least one of the following: averaging the plurality of model outputs, stacking the plurality of model outputs, weighing the plurality of model outputs, or any combination thereof.

13. A computer program product, comprising at least one non- transitory computer-readable medium including program instructions that, when executed by at least one processor, cause the at least one processor to: receive, from a plurality of remote issuer systems via an interface with the at least one processor, a plurality of model outputs from a plurality of machinelearning models, each model output of the plurality of model outputs resulting from a same input processed by a separate machine-learning model trained by an issuer system of the plurality of remote issuer systems; and aggregate the plurality of model outputs to generate an aggregated output.

14. The computer program product of claim 13, wherein the at least one processor is part of a transaction processing system configured to host the interface.

15. The computer program product of claim 13, wherein the at least one processor is further caused to: generate a graphical user interface comprising a global profile for an account holder, wherein the same input comprises an identifier of the account holder, and wherein each model output of the plurality of model outputs further results from an input of transaction data of the issuer system of the plurality of remote issuer systems.

16. The computer program product of claim 13, wherein the plurality of model outputs comprises a plurality of scores, and wherein the aggregated output is based on each score of the plurality of scores.

17. The computer program product of claim 13, wherein each machine-learning model of the plurality of machine-learning models comprises a different large language model (LLM).

18. The computer program product of claim 13, wherein aggregating the plurality of model outputs to generate the aggregated output comprises at leastone of the following: averaging the plurality of model outputs, stacking the plurality of model outputs, weighing the plurality of model outputs, or any combination thereof.

Images