Systems, methods, and computer program products for incremental learning

Abstract

The present disclosure provides systems, methods, and computer program products for incremental learning. A system includes at least one processor programmed or configured to: execute, in a production environment, a first machine learning model for each of a plurality of inputs associated with a plurality of requests; determining to train the first machine learning model based on at least one rule; in response to determining to train the first machine learning model, creating a second machine learning model including weights from the first machine learning model; training the second machine learning model with model data stored in at least one data storage; determining whether to replace the first machine learning model with the second machine learning model; in response to determining that the first machine learning model is replaced with the second machine learning model, and the first machine learning model is replaced with the second machine learning model in the production environment.

Description

Cross Reference to Related Applications

[0001] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 523,956, filed June 29, 2023, the disclosure of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present disclosure relates generally to machine learning, and in some non-limiting embodiments or aspects, to systems, methods, and computer program products for incremental learning of models in production environments. BACKGROUND

[0003] There are several technical limitations to monitoring inference platforms of machine learning models in production. For example, training is a separate process, and retraining a model can take a considerable amount of resources (e.g., memory, processing, time, etc.). Both production and training environments are associated with workloads and overhead. Further, there are limited opportunities for production testing during training of such models. Further, the training process is typically slow in response to data distributions during inference production generation. SUMMARY

[0004] According to non-limiting embodiments or aspects, a system is provided, the system comprising: at least one data storage device; and at least one processor programmed or configured to: execute, in a production environment, a first machine learning model for each input of a plurality of inputs associated with a plurality of requests, the first machine learning model configured to output an inference for each input; store, in the at least one data storage device, model data for each execution of the first machine learning model; determine, based on at least one rule, to train the first machine learning model; responsive to determining to train the first machine learning model, create a second machine learning model comprising weights from the first machine learning model; train the second machine learning model with the model data stored in the at least one data storage device; determine whether to replace the first machine learning model with the second machine learning model; and responsive to determining to replace the first machine learning model with the second machine learning model, replace the first machine learning model with the second machine learning model in the production environment, such that a next plurality of requests are input to the second machine learning model. In non-limiting embodiments or aspects, wherein the plurality of requests are processed as a batch by executing the first machine learning model for each input of the plurality of inputs associated with the plurality of requests.

[0005] In non-limiting embodiments or aspects, the plurality of requests are processed by a batch processor, and the at least one processor is further programmed or configured to generate a dashboard interface configured to communicate with the batch processor. In non-limiting embodiments or aspects, determining whether to replace the first machine learning model with the second machine learning model includes comparing the first machine learning model to the second machine learning model. In non-limiting embodiments or aspects, determining whether to replace the first machine learning model with the second machine learning model is based on at least one of computational efficiency and accuracy. In non-limiting embodiments or aspects, the at least one rule is based on at least one of model scores and feature distributions. In non-limiting embodiments or aspects, the inference includes a score.

[0006] According to non-limiting embodiments or aspects, there is provided a computer- implemented method comprising: executing, with at least one processor, a first machine learning model in a production environment for each input of a plurality of inputs associated with a plurality of requests, the first machine learning model configured to output an inference for each input; storing, in at least one data store, model data for each execution of the first machine learning model; determining, with at least one processor, to train the first machine learning model based on at least one rule; responsive to determining to train the first machine learning model, creating a second machine learning model comprising weights from the first machine learning model; training, with at least one processor, the second machine learning model with the model data stored in the at least one data store; determining, with at least one processor, whether to replace the first machine learning model with the second machine learning model; and responsive to determining to replace the first machine learning model with the second machine learning model, replacing the first machine learning model with the second machine learning model in the production environment such that subsequent plurality of requests are input to the second machine learning model.

[0007] In non-limiting embodiments or aspects, the plurality of requests are processed as a batch by performing the first machine learning model for each of the plurality of inputs associated with the plurality of requests. In non-limiting embodiments or aspects, the plurality of requests are processed by a batch processor, the method comprising generating a dashboard interface configured to communicate with the batch processor. In non-limiting embodiments or aspects, determining whether to replace the first machine learning model with the second machine learning model comprises comparing the first machine learning model to the second machine learning model. In non-limiting embodiments or aspects, determining whether to replace the first machine learning model with the second machine learning model is based on at least one of computational efficiency and accuracy. In non-limiting embodiments or aspects, the at least one rule is based on at least one of model scores and feature distributions. In non-limiting embodiments or aspects, the inference comprises a score.

[0008] According to non-limiting embodiments or aspects, a computer program product is provided, the computer program product comprising at least one non-transitory computer- readable medium including program instructions, the program instructions, when executed by at least one processor, cause the at least one processor to: execute, in a production environment, a first machine learning model for each of a plurality of inputs associated with a plurality of requests, the first machine learning model configured to output an inference for each input; store, in at least one data store, model data for each execution of the first machine learning model; determine, based on at least one rule, to train the first machine learning model; responsive to determining to train the first machine learning model, create a second machine learning model comprising weights from the first machine learning model; train the second machine learning model with the model data stored in the at least one data store; determine whether to replace the first machine learning model with the second machine learning model; and responsive to determining to replace the first machine learning model with the second machine learning model, replace the first machine learning model with the second machine learning model in the production environment such that a next plurality of requests are input to the second machine learning model.

[0009] Other non-limiting embodiments or aspects will be set forth in the following numbered clauses: Clause 1 : A system comprising: at least one data storage device; and at least one processor programmed or configured to: execute, in a production environment, a first machine learning model for each input of a plurality of inputs associated with a plurality of requests, the first machine learning model configured to output an inference for each input; store, in the at least one data storage device, model data for each execution of the first machine learning model; determine, based on at least one rule, to train the first machine learning model; responsive to determining to train the first machine learning model, create a second machine learning model comprising weights from the first machine learning model; train the second machine learning model with the model data stored in the at least one data storage device; determine whether to replace the first machine learning model with the second machine learning model; and responsive to determining to replace the first machine learning model with the second machine learning model, replace the first machine learning model with the second machine learning model in the production environment such that subsequent plurality of requests are input to the second machine learning model.

[0010] Clause 2: The system of clause 1, wherein the plurality of requests are processed as a batch by executing the first machine learning model for each input of the plurality of inputs associated with the plurality of requests.

[0011] Clause 3: The system of clause 1 or 2, wherein the plurality of requests are processed by a batch processor, and wherein the at least one processor is further programmed or configured to generate a dashboard interface configured to communicate with the batch processor.

[0012] Clause 4: The system of any of clauses 1 to 3, wherein determining whether to replace the first machine learning model with the second machine learning model comprises comparing the first machine learning model to the second machine learning model.

[0013] Clause 5: The system of any of clauses 1 to 4, wherein determining whether to replace the first machine learning model with the second machine learning model is based on at least one of computational efficiency and accuracy.

[0014] Clause 6: The system of any of clauses 1 to 5, wherein the at least one rule is based on at least one of a model score and a feature distribution.

[0015] Clause 7: The system of any of clauses 1 to 6, wherein the inference comprises a score.

[0016] Clause 8: A computer-implemented method comprising: executing, with at least one processor, a first machine learning model in a production environment for each input of a plurality of inputs associated with a plurality of requests, the first machine learning model configured to output an inference for each input; storing, in at least one data storage device, model data for each execution of the first machine learning model; determining, with at least one processor, based on at least one rule, to train the first machine learning model; responsive to determining to train the first machine learning model, creating a second machine learning model comprising weights from the first machine learning model; training, with at least one processor, the second machine learning model with the model data stored in the at least one data storage device; determining, with at least one processor, whether to replace the first machine learning model with the second machine learning model; and responsive to determining to replace the first machine learning model with the second machine learning model, replacing the first machine learning model with the second machine learning model in the production environment such that subsequent plurality of requests are input to the second machine learning model.

[0017] Clause 9: The method of clause 8, wherein the plurality of requests are processed as a batch by executing the first machine learning model for each input of the plurality of inputs associated with the plurality of requests.

[0018] Clause 10: The method of clause 8 or 9, wherein the plurality of requests are processed by a batch processor, the method further comprising generating a dashboard interface configured to communicate with the batch processor.

[0019] Clause 11 : The method of any one of clauses 8 to 10, wherein determining whether to replace the first machine learning model with the second machine learning model comprises comparing the first machine learning model to the second machine learning model.

[0020] Clause 12: The method of any one of clauses 8 to 11, wherein determining whether to replace the first machine learning model with the second machine learning model is based on at least one of computational efficiency and accuracy.

[0021] Clause 13: The method of any one of clauses 8 to 12, wherein the at least one rule is based on at least one of a model score and a feature distribution.

[0022] Clause 14: The method of any one of clauses 8 to 13, wherein the inference comprises a score.

[0023] Clause 15: A computer program product comprising at least one non-transitory computer-readable medium comprising program instructions, the program instructions, when executed by at least one processor, cause the at least one processor to: execute, in a production environment, a first machine learning model for each input of a plurality of inputs associated with a plurality of requests, the first machine learning model configured to output an inference for each input; store, in at least one data store, model data for each execution of the first machine learning model; determine, based on at least one rule, to train the first machine learning model; responsive to determining to train the first machine learning model, create a second machine learning model comprising weights from the first machine learning model; train the second machine learning model with the model data stored in the at least one data store; determine whether to replace the first machine learning model with the second machine learning model; and responsive to determining to replace the first machine learning model with the second machine learning model, replace the first machine learning model with the second machine learning model in the production environment such that subsequent plurality of requests are input to the second machine learning model.

[0024] Clause 16: The computer program product of clause 15, wherein the plurality of requests are processed as a batch by executing the first machine learning model for each input of the plurality of inputs associated with the plurality of requests.

[0025] Clause 17: The computer program product of clause 15 or 16, wherein the plurality of requests are processed by a batch processor, and wherein the at least one processor is further programmed or configured to generate a dashboard interface configured to communicate with the batch processor.

[0026] Clause 18: The computer program product of any of clauses 15 to 17, wherein determining whether to replace the first machine learning model with the second machine learning model comprises comparing the first machine learning model to the second machine learning model.

[0027] Clause 19: The computer program product of any of clauses 15 to 18, wherein determining whether to replace the first machine learning model with the second machine learning model is based on at least one of computational efficiency and accuracy.

[0028] Clause 20: The computer program product of any of clauses 15 to 19, wherein the at least one rule is based on at least one of a model score and a feature distribution.

[0029] The operational methods and manufacturing economics of these and other features and characteristics of this disclosure, as well as the combinations of related structural elements and parts, will become more apparent when considered with reference to the accompanying drawings, all of which form part of this specification, wherein similar reference numerals denote corresponding parts in the figures. However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to be a definition of limitation on the disclosed subject matter. Attached Figure Description

[0030] Additional advantages and details are explained in more detail below with reference to the non-limiting exemplary embodiments shown in the illustrative accompanying drawings, in which: Figure 1 This is a schematic diagram of a system for incremental learning according to some non-limiting embodiments or aspects; Figure 2 This is a schematic diagram of a system for incremental learning according to some non-limiting embodiments or aspects; Figure 3 This is a flowchart of a method for incremental learning according to some non-limiting embodiments or aspects; and Figure 4 This is a schematic diagram of example components of one or more devices according to some non-limiting embodiments or aspects. Detailed Implementation

[0031] For the purposes of the following description, the terms “end,” “upper,” “lower,” “right,” “left,” “vertical,” “horizontal,” “top,” “bottom,” “lateral,” “longitudinal,” and their derivatives should be associated with the orientation of the embodiments in the accompanying drawings. However, it should be understood that embodiments may take various alternative variations and sequences of steps, except where explicitly specified as the opposite. It should also be understood that the specific apparatus and processes shown in the drawings and described in the following specification are merely exemplary embodiments or aspects of the disclosed subject matter. Therefore, specific dimensions and other physical characteristics relating to the embodiments or aspects disclosed herein should not be considered limiting.

[0032] No aspect, component, element, structure, act, step, function, instruction, etc. used in the description should be understood as critical or essential unless explicitly described as such. Also, as used in the description, the article "a" and "an" are intended to include one or more items, and can be used interchangeably with "one or more" and "at least one." Furthermore, as used in the description, the term "set" is intended to include one or more items (e.g., related items, unrelated items, a combination of related and unrelated items, etc.), and can be used interchangeably with "one or more" or "at least one." Where only one item is intended, the term "one" or similar language is used. Also, as used in the description, the terms "has," "have," "having," or the like are intended to be open-ended terms. Further, the phrase "based on" is intended to mean "based, at least in part, on" unless explicitly stated otherwise.

[0033] As used herein, the term "account identifier" can include one or more primary account numbers (PANs), tokens, or other identifiers associated with a customer account. The term "token" can refer to an identifier that is used as a substitute or replacement identifier for an original account identifier, such as a PAN. An account identifier can be any combination of alphanumeric or character and / or symbol. A token can be associated with a PAN or other original account identifier in one or more data structures (e.g., one or more databases, etc.) such that the token can be used to conduct transactions without the need to directly use the original account identifier. In some examples, an original account identifier, such as a PAN, can be associated with multiple tokens for different individuals or purposes.

[0034] An "application programming interface" (API) refers to computer code or other data ordered on a computer readable medium that can be executed by a processor to facilitate interaction between software components, such as between a client-side front end and / or a server-side back end for receiving data from a client. An "interface" refers to a generated display, such as one or more graphical user interfaces (GUIs) with which a user can interact, directly or indirectly (e.g., through a keyboard, mouse, etc.).

[0035] As used herein, the term "communication" can refer to receipt, receipt of, transmission, transfer, provision, and / or the like of data (e.g., information, signals, messages, instructions, commands, and / or the like). That one unit (e.g., a device, a system, a component of a device or system, a combination thereof, and / or the like) is in communication with another unit means that the one unit is capable of directly or indirectly receiving a signal from and / or transmitting a signal to the other unit. This can include a direct or indirect connection (e.g., direct communication connection, indirect communication connection, and / or the like) that is wired and / or wireless in nature. Additionally, two units can be in communication with each other even though the information sent can be modified, processed, relayed, and / or routed between the first and second unit. For example, a first unit can be in communication with a second unit even though the first unit passively receives information and does not actively transmit information to the second unit. As another example, a first unit can be in communication with a second unit if at least one intermediary unit processes information received from the first unit and communicates the processed information to the second unit.

[0036] As used herein, the term "computing device" can refer to one or more electronic devices configured to process data. In some examples, a computing device can include the necessary components for receiving, processing, and outputting data, such as a processor, a display, a memory, an input device, a network interface, and / or the like. A computing device can be a mobile device. For example, a mobile device can include a cellular phone (e.g., a smartphone or a standard cellular phone), a portable computer, a wearable device (e.g., a watch, glasses, lenses, clothing, and / or the like), a personal digital assistant (PDA), and / or other similar devices. A computing device can also be a desktop computer or other form of non-mobile computer.

[0037] As used herein, the term "issuer institution" can refer to one or more entities, such as a bank, that provide accounts for customers to conduct transactions, such as payment transactions, such as initiating credit and / or debit payments. For example, an issuer institution can provide a customer with an account identifier, such as a PAN, that uniquely identifies one or more accounts associated with the customer. The account identifier can be embodied on a portable financial device, such as a physical financial instrument (e.g., a payment card), and / or can be electronic and used for electronic payments. The term "issuer system" refers to one or more computer devices operated by or on behalf of an issuer institution, such as a server computer executing one or more software applications. For example, an issuer system can include one or more authorization servers for authorizing transactions.

[0038] As used herein, the term "merchant" can refer to an individual or entity that offers goods and / or services or access to goods and / or services to a customer based on, for example, a payment transaction. The term "merchant" or "merchant system" can also refer to one or more computer systems operated by or on behalf of a merchant, such as a server computer executing one or more software applications.

[0039] As used herein, the terms "client" and "client device" can refer to one or more client-side devices or systems (e.g., remote from a transaction service provider) used to initiate or facilitate a transaction (e.g., a payment transaction). For example, a "client device" can refer to one or more POS devices used by a merchant, one or more acquirer host computers used by an acquirer, one or more mobile devices used by a user, and the like. In some non-limiting embodiments or aspects, a client device can be an electronic device configured to communicate with one or more networks and initiate or facilitate a transaction. By way of example, a client device can include one or more computers, portable computers, laptop computers, tablet computers, mobile devices, cellular telephones, wearable devices (e.g., watches, glasses, lenses, clothing, and the like), PDAs, and the like. Further, a "client" can also refer to an entity (e.g., a merchant, an acquirer, and the like) that owns, utilizes, and / or operates a client device to initiate a transaction (e.g., initiate a transaction with a transaction service provider).

[0040] As used herein, the term "payment device" can refer to a payment card (e.g., a credit or debit card), a gift card, a smart card, a smart media, a payroll card, a healthcare card, a wristband, a machine-readable medium containing account information, a keychain device or fob, an RFID transponder, a retailer discount or loyalty card, a cellular telephone, an electronic wallet mobile application, a personal digital assistant (PDA), a pager, a security card, a computing device, an access card, a wireless terminal, a transponder, and the like. In some non-limiting embodiments or aspects, a payment device can include volatile or non-volatile memory to store information (e.g., an account identifier, an account holder name, and the like).

[0041] As used herein, the term "payment gateway" can refer to an entity and / or a payment processing system operated by or on behalf of such an entity (e.g., a merchant service provider, a payment service provider, a payment facilitator, a payment facilitator contracted with an acquirer, a payment aggregator, and the like) that provides payment services (e.g., transaction service provider payment services, payment processing services, and the like) to one or more merchants. The payment services can be associated with the use of payment devices managed by a transaction service provider. As used herein, the term "payment gateway system" can refer to one or more computer systems, computer devices, servers, groups of servers, and the like operated by or on behalf of a payment gateway.

[0042] As used herein, the term“server” can refer to or include one or more computing devices operated by or facilitating the communication and processing of a plurality of parties in a network environment, such as the Internet, although it will be appreciated that communication can be facilitated through one or more public or private network environments, and that various other arrangements are possible. Moreover, a plurality of computing devices (e.g., servers, point-of-sale (POS) devices, mobile devices, etc.) that communicate directly or indirectly in a network environment can constitute a“system.” As used herein, a reference to a“server” or“processor” can refer to the previously recited server and / or processor stated as performing a previous step or function, a different server and / or processor, and / or a combination of servers and / or processors. For example, as used in the specification and claims, a first server and / or a first processor stated as performing a first step or function can refer to the same or different server and / or processor stated as performing a second step or function.

[0043] As used herein, the term“transaction service provider” can refer to an entity that receives transaction authorization requests from merchants or other entities and in some cases provides payment guarantees through an agreement between the transaction service provider and an issuer institution. For example, a transaction service provider can include a payment network such as Visa®, or any other entity that processes transactions. The term“transaction processing system” can refer to one or more computer systems operated by or on behalf of a transaction service provider, such as a transaction processing server that executes one or more software applications. A transaction processing server can include one or more processors, and in some non-limiting embodiments or aspects, can be operated by or on behalf of a transaction service provider.

[0044] Non-limiting embodiments or aspects of the disclosed subject matter are directed to systems, methods, and computer program products for incremental learning. Non-limiting embodiments allow training of an inference model while the inference model is being used to perform real-time and / or batched processing requests. For example, non-limiting embodiments allow training of a model while deployed in a production environment, using smaller subsets of data each time, rather than training with larger amounts of data at longer intervals. Non-limiting embodiments provide model improvement in a production environment, avoiding service interruptions, delays, and other undesirable effects of training a model. Non-limiting embodiments can be used in conjunction with a batch inference platform as described in PCT Application No. PCT / US2024 / 35341, filed June 25, 2024, the entirety of which is incorporated herein by reference.

[0045] Non-limiting embodiments can be implemented within an electronic payment processing network, although it should be understood that non-limiting embodiments can be implemented within various different types of environments, not limited to payment networks, in which one or more models are executed as a service and benefit from periodic and / or periodic training.

[0046] Referring to Figure 1 , a system 1000 for incremental learning is shown in accordance with some non-limiting embodiments or aspects. A server computer 100 can include one or more computing devices, such as a web server, a private server, an authentication server (e.g., such as a 3D Secure system), and / or any other computing device configured to communicate with one or more client devices (e.g., 114, 116, 118). The client devices 114, 116, 118 can include computing devices that make requests from the server computer 100, such as processing requests (e.g., transaction processing, etc.). For example, the client devices 114, 116, 118 can each include an issuer system, and the server computer 100 can be associated with a transaction processing system and / or a payment gateway. However, it should be understood that various entities can control and / or operate the devices and systems shown in Figure 1 , in various non-limiting embodiments. As an example, the client devices 114, 116, 118 can communicate with the server computer 100 via one or more APIs, although various communication methods can be used.

[0047] With continued reference to Figure 1 , the server computer 100 can be in communication with an inference engine 102. The inference engine 102 can include one or more computing devices and / or software applications executed by one or more computing devices. In some examples, the inference engine 102 can be part of the server computer 100. In some examples, the inference engine 102 can be separate and / or remote from the server computer 100. The inference engine 102 can be configured to control execution of one or more machine learning models stored as model data 104 in one or more data storage devices. The inference engine 102 can also be in communication with a database 107, which can include audit logs for each execution of a model, model metric data, etc. In some examples in which the client devices 114, 116, 118 are issuer systems, the machine learning models can include scoring models (e.g., fraud or risk scoring models) used by the issuer systems to make decisions (e.g., authorization decisions).

[0048] An incremental training engine 101 is also in communication with the server computer 100. The incremental training engine 101 can include one or more computing devices and / or software applications executed by one or more computing devices. In operation, the inference engine 102 can execute a first machine learning model (e.g., a production model) based on requests from the client devices 114, 116, 118 for input received from the server computer 100 in real-time or in batches. Inferences and model data resulting from the execution by the model can be stored in the database 107. The server computer 100 determines when to train the first machine learning model. For example, training can be performed based on a predetermined time interval, a number of transactions processed, user input, etc. In non-limiting embodiments, determining to train the model is based on one or more rules, which can include a threshold value for a model score, a feature distribution, etc.

[0049] As used herein, the term “real-time” can refer to performing one or more tasks during another process or prior to completion of another process. For example, the inference engine 102 can execute the first machine learning model in real-time with respect to processing a transaction (e.g., prior to or during transaction authorization, etc.), during processing and / or communicating a message related to an event, when making a decision related to an event (e.g., an authorization decision, an authentication decision, etc.) (e.g., receiving an authorization request, at least a portion of which is included in a data sample; and determining an authorization decision based thereon), etc.

[0050] Non-limiting embodiments provide a faster, computationally more efficient training process by adding a smaller (e.g., incremental) subset of data and training on top of previous (e.g., current) model weights compared to non-incremental methods. By using a smaller subset of data and training incrementally, an accurate model can be maintained in a real-time production environment.

[0051] With continued reference to Figure 1 When the server computer 100 determines to train the first machine learning model while the model is in production, it can communicate with the incremental training engine 101. The incremental training engine 101 can be separate from and / or part of the server computer 100. The incremental training engine 101 can then create a second machine learning model based on the first machine learning model that is in production. The second machine learning model can be stored as model data 104 and / or in temporary memory. The second machine learning model can include weights from the first machine learning model. The second machine learning model can be copied from the first machine learning model. The incremental training engine 101 can then train the second machine learning model based on audit logs from previous executions stored in the database 107 for a period of time. In this way, the second machine learning model can be trained while the first machine learning model is being executed in a production environment.

[0052] Still referring toFigure 1 In response to the training of the second machine learning model, the incremental training engine 101 and / or the server computer 100 can then determine whether to replace the first machine learning model with the second machine learning model. In response to determining to replace the first machine learning model with the second machine learning model, the incremental training engine 101 and / or the server computer 100 can then replace the first machine learning model with the second machine learning model in the production environment such that subsequent requests are input to the second machine learning model. For example, the first machine learning model in the model data 104 can be replaced, removed, etc. such that the second machine learning model is used going forward.

[0053] In non-limiting embodiments, determining whether to replace the first machine learning model with the second machine learning model involves comparing the two models (e.g., comparing efficiency, accuracy, etc.). Replacement of the first model can occur if the difference in model metrics between the newly trained model and the production model meets a predetermined threshold (e.g., reaches or exceeds the threshold). It should be appreciated that other parameters can be considered to determine when to replace an existing model with a newly trained model.

[0054] With reference to Figure 2 A system 2000 for incremental learning is illustrated in accordance with some non-limiting embodiments or aspects. An inference engine 202 can include one or more computing devices and / or software applications executed by one or more computing devices for executing one or more machine learning models in response to requests. The inference engine 202 can be a real-time inference engine and / or a batch inference engine. For example, a payload can include input data for a risk scoring model. In non-limiting embodiments, the model inference engine 202 can execute one or more models according to requests. In non-limiting embodiments, in response to receiving a plurality of requests to provide inferences associated with outputs (e.g., requests for explainable machine learning or artificial intelligence metrics), the model inference engine 202 can process the requests in batch. An audit log 201 can store model metadata resulting from the execution of the machine learning models such that the model metadata is available to the inference engine 202.

[0055] With continued reference to Figure 2The inference engine 202 can output interpretable machine learning or artificial intelligence metrics related to the model and the inference (e.g., impact) on the model output. The monitoring system dashboard 206 can present one or more GUIs to a user to showcase graphical representations of the data in the audit log 201 in addition to the interpretable metrics. For example, the monitoring system dashboard 206 can display alerts, notifications, etc. based on real-time and / or batched execution of the machine learning model. In non-limiting embodiments, the monitoring system dashboard 206 is in communication with the inference engine 202 and the audit log 201. The dashboard 206 can display some or all of the model metric data and / or facilitate user interaction with the inference engine 202 and / or the audit log 201. For example, a user can access the dashboard 206 via a web browser or application and use it to configure training interval times, review model metric data, etc. In response to determining to train one or more models used by the inference engine, the inference engine and / or the dashboard 206 can communicate with the feature generation engine 209. The feature generation engine 209 can include one or more computing devices and / or software applications executed by one or more computing devices to generate features (e.g., feature vectors) from model data stored in the audit log 201 from previous executions.

[0056] With continued reference to Figure 2 The incremental training engine 208 can be configured to train a second model based on one or more of the models in production by the inference engine 202. The incremental training engine 208 outputs a challenger model 205 that the model evaluation engine 207 analyzes to compare it to the models currently in production. The model evaluation engine 207 can include one or more computing devices and / or software applications executed by one or more computing devices to perform this comparison. In response to the model evaluation engine determining to replace a production model executed by the inference engine 202 based on the comparison of the model metrics, the production model can be replaced with the challenger model 205.

[0057] In non-limiting embodiments, the machine learning models discussed herein can be unsupervised models. However, it should be appreciated that supervised learning models can also be used in non-limiting embodiments, for example, where model data (e.g., labels, observations, etc. generated with the model) is updated.

[0058] Non-limiting embodiments provide for incremental learning that avoids the need to retrain a model from a starting state (e.g., “from scratch”) each time it is trained. Incremental training is provided for each stage and / or threshold by iterating through multiple versions of the model over time as the model data grows and provides more training data.

[0059] Reference is now made to Figure 3FIG. 1 illustrates a flow diagram of a method for incremental learning, in accordance with some non-limiting embodiments or aspects. Figure 3 The illustrated steps are for exemplary purposes only. It will be appreciated that additional, fewer, different, and / or different order steps can be used in some non-limiting embodiments or aspects. In some non-limiting embodiments or aspects, steps can be performed automatically in response to performance and / or completion of a previous step. At step 300, a first machine learning model can be executed in a production environment. For example, the first machine learning model can be configured in an electronic payment network to provide real-time inferences used by the payment network, such as but not limited to fraud determinations for authorizations, etc.

[0060] At step 302, model data from inferences processed at step 300 can be stored in a data structure such as a model database, an audit log, etc. The model data can include model inputs, model outputs, model parameters (e.g., weights of nodes and / or edges of a network, transformations, etc.), and / or model metrics (e.g., a Shapley value, or other metric representing how one or more features affect individual predictions and / or the extent to which the feature affects the prediction score compared to other features).

[0061] At step 304, a determination is made whether to train a machine learning model based on the model data stored at step 302. This determination can be based on one or more rules, such as one or more thresholds associated with model performance metrics. For example, scores and / or feature distributions can be compared to one or more thresholds to determine at step 304 whether training and / or retraining should be triggered. In some examples, feature distribution drift, score distribution drift, etc. can be used to trigger training and / or retraining at step 304. In some non-limiting embodiments, a Shapley value, or other metric representing how one or more features affect individual predictions (e.g., scores) and / or the extent to which the feature affects (e.g., increases and / or decreases) the prediction score compared to other features can be used to monitor the model and trigger and / or retrigger retraining. The first machine learning model running in the production environment is continued to be provided with inputs.

[0062] Once triggered for training and / or retraining, the method can proceed to step 306 and a second model (e.g., a replica model) can be created (e.g., generated) based on the first model. For example, a replica of the model can be generated with the same weights and other parameters, such that the first model can be trained while it is still being executed in the production environment. At step 308, the second model is trained based on the model data stored at step 302. Once the second model is trained using the model data, at step 310, it can be determined whether to replace the first model currently running in the production environment with the second model. The trained second model becomes a challenger model compared to the first model. For example, at step 310, the models can be evaluated by computing efficiency and / or accuracy based on a cross-validation dataset. Other comparison methods can be performed. In response to determining to replace the model, the method proceeds to step 312 and the first model is replaced with the second model as the real-time model in the production environment. If it is determined not to replace the model, for example if the performance of the second model is not better than the first model, the method can return to step 300 and continue with the first model still in the production environment.

[0063] Referring now to Figure 4 , a diagram of example components of a device 400 is shown in accordance with non-limiting embodiments or aspects. The device 400 can correspond to at least one of the computing devices (e.g., server computer 100, inference engine 102, delta training engine 101, etc.) in Figure 1 In some non-limiting embodiments or aspects, such a system or device can include at least one device 400 and / or at least one component of the device 400. The number and arrangement of components shown in Figure 4 are provided by way of example. In some non-limiting embodiments or aspects, the device 400 can include additional components, fewer components, different components, or differently arranged components than those shown in Figure 4 Additionally or alternatively, a set of components (e.g., one or more components) of the device 400 can perform one or more functions described as being performed by another set of components of the device 400.

[0064] As Figure 4As shown, the apparatus 400 can include a bus 402, a processor 404, a memory 406, a storage component 408, an input component 410, an output component 412, and a communication interface 414. The bus 402 can include a component that permits communication among the components of the apparatus 400. In some non-limiting embodiments or aspects, the processor 404 can be implemented in hardware, firmware, or a combination of hardware and software. For example, the processor 404 can include a processor (e.g., a central processing unit (CPU), a GPU, an accelerated processing unit (APU), etc.), a microprocessor, a digital signal processor (DSP), and / or any processing component (e.g., a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), etc.) that can be programmed to perform a function. The memory 406 can include random access memory (RAM), read only memory (ROM), and / or another type of dynamic or static storage (e.g., flash memory, magnetic computer storage medium, optical computer storage medium, and / or the like) that stores information and / or instructions for use by the processor 404.

[0065] With continued reference to Figure 4 The storage component 408 can store information and / or software related to the operation and use of the apparatus 400. For example, the storage component 408 can include a hard disk (e.g., a magnetic disk, an optical disk, a magneto-optic disk, a solid state disk, etc.) and / or another type of computer readable medium. The input component 410 can include a component that permits the apparatus 400 to receive information, such as via a user input (e.g., a touch screen display, a keyboard, a keypad, a mouse, a button, a switch, a microphone, etc.). Additionally, or in the alternative, the input component 410 can include a sensor (e.g., a global positioning system (GPS) component, an accelerometer, a gyroscope, an actuator, etc.) to sense information. The output component 412 can include a component that provides output information from the apparatus 400 (e.g., a display, a speaker, one or more light-emitting diodes (LEDs), etc.). The communication interface 414 can include a transceiver-like component (e.g., a transceiver, a separate receiver and transmitter, etc.) that enables the apparatus 400 to communicate with other devices, such as via a wired connection, a wireless connection, or a combination of wired and wireless connections. The communication interface 414 can permit the apparatus 400 to receive information from another device and / or provide information to another device. For example, the communication interface 414 can include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi® interface, a cellular network interface, etc.

[0066] Device 400 can perform one or more processes described herein. Device 400 can perform these processes based on processor 404 executing software instructions stored by a computer-readable medium, such as memory 406 and / or storage component 408. A computer-readable medium can include any non-transitory memory. A memory device includes a memory space located within a single physical storage device or a memory space spread across multiple physical storage devices. Software instructions can be read into memory 406 and / or storage component 408 from another computer-readable medium or from another device via communications interface 414. When executed, software instructions stored in memory 406 and / or storage component 408 can cause processor 404 to perform one or more processes described herein. Additionally or alternatively, hardwired circuitry can be used in place of or in combination with software instructions to perform one or more processes described herein. Thus, embodiments described herein are not limited to any specific combination of hardware circuitry and software. The term “configured to” as used herein can refer to an arrangement of software, hardware, and / or firmware for performing and / or implementing one or more functions (e.g., acts, processes, steps of processes, etc.). For example, a “processor configured to” can refer to a processor executing software instructions (e.g., program code) that cause the processor to perform one or more functions.

[0067] While embodiments have been described in detail, it should be understood that the description is for illustrative purposes only and that modifications and alterations are possible within the scope of the present disclosure. For example, it is understood that one or more features of any embodiment or aspect can be combined with one or more features of any other embodiment or aspect, as much as is possible.

Claims

1. A system comprising: At least one data storage device; as well as At least one processor, which is programmed or configured to: In a production environment, a first machine learning model is executed for each of a plurality of inputs associated with a plurality of requests, the first machine learning model being configured to output an inference for each input; The model data for each execution of the first machine learning model is stored in the at least one data storage device; The training of the first machine learning model is determined based on at least one rule; In response to determining that the first machine learning model is to be trained, a second machine learning model is created that includes weights from the first machine learning model; The second machine learning model is trained using the model data stored in the at least one data storage device; Determine whether to replace the first machine learning model with the second machine learning model; and In response to determining to replace the first machine learning model with the second machine learning model, the first machine learning model is replaced with the second machine learning model in the production environment, such that subsequent multiple requests are input to the second machine learning model.

2. The system of claim 1, wherein the plurality of requests are processed as a batch by executing the first machine learning model for each of the plurality of inputs associated with the plurality of requests.

3. The system of claim 2, wherein the plurality of requests are processed by a batch processor, and wherein at least one processor is further programmed or configured to generate a dashboard interface configured to communicate with the batch processor.

4. The system of claim 1, wherein determining whether to replace the first machine learning model with the second machine learning model includes comparing the first machine learning model with the second machine learning model.

5. The system of claim 4, wherein determining whether to replace the first machine learning model with the second machine learning model is based on at least one of computational efficiency and accuracy.

6. The system of claim 1, wherein the at least one rule is based on at least one of model score and feature distribution.

7. The system of claim 1, wherein the inference includes a score.

8. A computer-implemented method comprising: A first machine learning model is executed in a production environment using at least one processor for each of a plurality of inputs associated with a plurality of requests, the first machine learning model being configured to output an inference for each input; The model data for each execution of the first machine learning model is stored in at least one data storage device; The first machine learning model is trained using at least one processor based on at least one rule. In response to determining that the first machine learning model is to be trained, a second machine learning model is created that includes weights from the first machine learning model; The second machine learning model is trained using at least one processor with the model data stored in the at least one data storage device; At least one processor is used to determine whether to replace the first machine learning model with the second machine learning model; as well as In response to determining to replace the first machine learning model with the second machine learning model, the first machine learning model is replaced with the second machine learning model in the production environment, such that subsequent multiple requests are input to the second machine learning model.

9. The method of claim 8, wherein the plurality of requests are processed as a batch by executing the first machine learning model for each of the plurality of inputs associated with the plurality of requests.

10. The method of claim 9, wherein the plurality of requests are processed by a batch processor, the method further comprising generating a dashboard interface configured to communicate with the batch processor.

11. The method of claim 8, wherein determining whether to replace the first machine learning model with the second machine learning model comprises comparing the first machine learning model with the second machine learning model.

12. The method of claim 11, wherein determining whether to replace the first machine learning model with the second machine learning model is based on at least one of computational efficiency and accuracy.

13. The method of claim 8, wherein the at least one rule is based on at least one of model score and feature distribution.

14. The method of claim 8, wherein the inference includes a score.

15. A computer program product comprising at least one non-transitory computer-readable medium, said at least one non-transitory computer-readable medium comprising program instructions that, when executed by at least one processor, cause the at least one processor to: In a production environment, a first machine learning model is executed for each of a plurality of inputs associated with a plurality of requests, the first machine learning model being configured to output an inference for each input; The model data for each execution of the first machine learning model is stored in at least one data storage device; The training of the first machine learning model is determined based on at least one rule; In response to determining that the first machine learning model is to be trained, a second machine learning model is created that includes weights from the first machine learning model; The second machine learning model is trained using the model data stored in the at least one data storage device; Determine whether to replace the first machine learning model with the second machine learning model; and In response to determining to replace the first machine learning model with the second machine learning model, the first machine learning model is replaced with the second machine learning model in the production environment, such that subsequent multiple requests are input to the second machine learning model.

16. The computer program product of claim 15, wherein the plurality of requests are processed as a batch by executing the first machine learning model for each of the plurality of inputs associated with the plurality of requests.

17. The computer program product of claim 16, wherein the plurality of requests are processed by a batch processor, and wherein the at least one processor is further programmed or configured to generate a dashboard interface configured to communicate with the batch processor.

18. The computer program product of claim 15, wherein determining whether to replace the first machine learning model with the second machine learning model includes comparing the first machine learning model with the second machine learning model.

19. The computer program product of claim 18, wherein determining whether to replace the first machine learning model with the second machine learning model is based on at least one of computational efficiency and accuracy.

20. The computer program product of claim 15, wherein the at least one rule is based on at least one of model score and feature distribution.

Images