Machine Learning Systems

Abstract

A machine learning system (700) is provided for processing incoming event data (702) associated with a transaction. A state delta generator (708) retrieves a version of state data (712) from a database (704) and receives the incoming event data (702). The generator (708) uses a model to process the event data (702) and the state data (712) to generate state deltas (710) including a set of pending updates to the state data, which are stored in a queue. An updater (714) retrieves the event state deltas (710) from the queue and the state data (712) from the database (704). The updater (714) applies the pending updates from each state delta (710) to generate updated state data (716) used to update the database (704). A cache (718) stores a copy of each state delta (710), where the copy of the state delta is stored in the cache along with a value indicating the version of the state data used to generate the state delta.

Description

[Technical Field]

[0001] The present invention relates to machine learning systems and methods for processing incoming event data, such as machine learning systems for detecting anomalies in patterns of data, e.g., for detecting fraudulent transactions. Some examples relate to machine learning systems for use in real-time transaction processing. [Background technology]

[0002] Digital payments have exploded over the last two decades, with over three-quarters of global payments now using some form of payment card or electronic wallet. Point-of-sale systems are increasingly becoming digital, rather than cash-based. Simply put, the global system of commerce now relies heavily on electronic data processing platforms. This presents many engineering challenges that are largely hidden from the average user. For example, digital transactions must be completed in real time, i.e., with minimal latency experienced by computing devices at the point of purchase. Digital transactions must also be secure and resistant to attack and exploitation. Digital transaction processing is also constrained by the historical development of global electronic systems for payments. For example, much infrastructure is still built around a model designed for mainframe architectures that were in use more than 50 years ago.

[0003] As digital transactions increase, new security risks are also becoming apparent. Digital transactions present new opportunities for fraud and malicious activity. In 2015, it was estimated that 7% of digital transactions were fraudulent, and that number will only increase as more economic activity moves online. Fraud losses are estimated to be four times the world's population (in US dollars, for example) and are growing.

[0004] Financial services institutions are coming under increased regulatory scrutiny as traditional methods of fraud prevention, such as authenticating identities (e.g., passwords, digital biometrics, national IDs, etc.), have proven ineffective at preventing fraud vectors such as synthetic identities and fraud. These much more complex threat vectors for fraud require significantly more analysis in extremely short (sub-50 millisecond) times, often based on much smaller data sampling sizes for the fraud or fraudulent activity itself. This poses significant technical challenges.

[0005] While risks such as fraud are an economic issue for businesses involved in commerce, implementing technical systems to process transactions is an engineering challenge. Traditionally, banks, merchants, and card issuers developed “paper” rules or procedures, manually implemented by clerical staff, to flag or block some transactions. As transactions became digital, one approach to building technical systems to process transactions was to provide computer engineers with these established sets of standards and ask them to implement them using a digital representation of the transaction—that is, to translate the handwritten rules into coded logic statements that can be applied to electronic transaction data. This traditional approach ran into several problems as digital transaction volumes grew. First, any processing applied needed to occur in “real time,” e.g., with millisecond latency. Second, thousands of transactions need to be processed per second (e.g., a typical "load" can be 1,000-2,000 per second), and the load fluctuates unpredictably over time (e.g., the launch of a new product or set of tickets can easily increase average load levels several times). Third, transaction processors and banks' digital storage systems are often siloed or compartmentalized for security reasons, and digital transactions often involve an interconnected web of merchant systems. Fourth, large-scale analysis of actual reported and predicted fraud is now possible. This shows that traditional methods of fraud detection are not up to par, being found to have low accuracy and high false positives. This, in turn, has a physical impact on digital transaction processing, with many more genuine point-of-sale and online purchases being rejected, and those attempting to exploit the new digital systems often running off with them.

[0006] In recent years, more machine learning techniques have been incorporated into transactional data processing. As machine learning models mature in academia, engineers have begun to apply them to transactional data processing. However, this again runs into problems. Even when engineers are provided with an academic or theoretical machine learning model and asked to implement it, this is not easy. For example, the problem of large-scale transaction processing systems arises. Machine learning models do not have the luxury of unlimited inference time that they do in a lab. This means that some models are simply impractical to implement in a real-time setting, or they require significant adaptation to enable real-time processing at the volume levels encountered by real-world servers. Furthermore, engineers must address the problem of implementing machine learning models on siloed or compartmentalized data based on access security and in situations where the rate of data update is extreme. Thus, the problems faced by engineers building transactional processing systems can be seen as similar to those faced by network or database engineers, where machine learning models must be applied while meeting system throughput and query response time constraints set by the processing infrastructure. There are no easy solutions to these problems. Indeed, the fact that many transaction processing systems are confidential, proprietary, and based on older technologies means that engineers lack a developed body of knowledge in these neighboring fields and often face challenges that are unique to the transaction processing domain. Moreover, the field of large-scale, practical machine learning is still young, with few established design patterns or textbooks on which engineers can draw.

[0007] Fraud detection systems generate risk scores for new transactions in real time (i.e., latency ≦100 ms), where a high risk score indicates a likelihood of fraud, error, or unauthorized use of an account. Financial institutions use these scores in their decision logic when determining whether to approve a transaction (typically, this will result in rejecting the transaction if the risk score is higher than some threshold). Financial institutions generally want the set of transactions above a certain risk threshold to contain the highest possible rate of fraud (i.e., to provide a "maximum detection rate"), along with the lowest possible rate of false positive classifications (i.e., to provide "maximum accuracy").

[0008] One approach, known per se in the art, involves using machine learning techniques, such as recurrent neural networks (RNNs), to detect anomalies in patterns of behavior. Those skilled in the art will appreciate that the output of an RNN is a next state, which depends on the previous state and the inputs provided to the RNN. In this way, the state of the RNN is based not only on momentary information related to a given transaction, but also on the history of transactions that came before, and therefore maintains a memory of observed patterns. This can be used to detect anomalous behavior that may indicate fraudulent transactions. The terms "behavior" and "behavioral" are used herein to refer to patterns of activity or action.

[0009] This detection of anomalous activity may provide increased security for the system compared to "traditional" methods of fraud prevention (e.g., passwords, digital biometrics, national IDs, etc.) It will of course be appreciated that behavioral pattern analysis techniques may be used in combination with one or more of these "traditional" methods, where appropriate.

[0010] In general, a rule may require that historical data (eg, state data) relating to a particular entity be stored, for example, for a particular duration.

[0011] Systems that utilize state data stored in and retrieved from a database (e.g., machine learning systems using RNNs) will generally require that the data stored in that database be able to be updated in response to incoming events, e.g., new transactions in a transaction processing system. A transaction or deposit made with a card or merchant will typically result in one or more events being sent to the system.

[0012] Typically, the state data may be stored in a database using any suitable structured numerical representation, for example as a table of vectors or tensors.

[0013] The use of "state deltas" for applying updates to a database is known per se in the art. Each state delta contains information about changes to be applied to the state, but the updates are not made immediately but instead at a later time (and thus multiple state deltas can be processed together). Such state deltas may utilize conflict-free replicated data types (CRDTs) to improve event processing capabilities. Each state delta may exist as a vector or tensor of values ​​to be applied to the state data using suitable mathematical or logical functions (e.g., addition, averaging, Boolean operators such as "AND" or "OR"), or may be constructed as a set of instructions for how the state data should be updated. State deltas may include histogram updates, items to add to an existing set of items, or replacement values ​​for such a set.

[0014] The use of these state deltas can improve the performance of an event processing system at the expense of reduced "state liveness." Specifically, if the state data stored in a database is updated only intermittently, rather than immediately after an event that would have caused an update to the state data, the state data retrieved from the database at any given time may be "stale," i.e., not the most "live" version of that state data. The most live version of the state data is referred to herein as the "canonical" state data. The time delay before a change is properly applied to the state data is referred to herein as the "state lag."

[0015] State lag can affect the performance of machine learning models. State lag can also vary over time - in times of high transaction rates, state lag will typically increase until the transaction rate drops, at which point the system catches up with the state lag over time.

[0016] In particular, if the state lag becomes sufficient that incoming requests are then handled using old state data, it can be problematic that this time delay in changes to state data becomes visible to users of that state data. Summary of the Invention [Means for solving the problem]

[0017] According to a first aspect, an embodiment of the present invention provides a machine learning system for processing incoming event data associated with a transaction, the system comprising: a) a first memory store comprising a database configured to store state data; b) a state delta generator, i) retrieving a version of the state data from a database; ii) receiving each incoming event data; and iii) processing each incoming event data and the retrieved version of the state data using each model to generate a respective event state delta that includes a set of pending updates to the state data; iv) storing the event state delta in a queue; and a state delta generator configured to: c) an updater, i) removing an event state delta from a queue; ii) retrieving state data from the database; and iii) applying pending updates from each event state delta to the state data to generate updated state data; iv) updating the database with the updated status data; and an updater configured to: d) a second memory store comprising a cache configured to store a copy of each event state delta in the queue having a pending update to state data, wherein each copy of the event state delta is stored in the cache along with a value indicating the version of state data used to generate the event state delta; and A machine learning system is provided.

[0018] A first aspect of the present invention also provides a machine learning method for processing incoming event data associated with a transaction, the method comprising: a) storing state data in a database in a first memory store; b) In the state delta generation step, i) retrieving a version of state data from a database; ii) receiving each incoming event data; iii) processing each incoming event data and the retrieved version of the state data using the respective model to generate a respective event state delta that includes a set of pending updates to the state data; iv) storing the event state delta in a queue; c) In the update step, i) removing an event state delta from a queue; ii) retrieving state data from the database; iii) applying pending updates from each event state delta to the state data to generate updated state data; iv) updating the database with the updated status data; d) storing a copy of each of the queued event state deltas having pending updates to state data in a second memory store comprising a cache, wherein each of the copies of the event state deltas is stored in the cache along with a value indicating the version of the state data used to generate the event state delta; This ranges from machine learning methods, including

[0019] It will therefore be appreciated that embodiments of the present invention provide improved arrangements that address several shortcomings associated with conventional machine learning systems and methods for processing events associated with transactions.

[0020] First, the present invention can significantly reduce (and potentially completely eliminate) the adverse effects of state lag associated with using state deltas. Those skilled in the art will appreciate that embodiments of the present invention utilize a separate cache of pending state deltas waiting to be applied to the standard state, along with information about what version of state the deltas have been applied to. As state deltas are created and queued for application to the database, they are also placed in the cache and keyed with the underlying state version on which they were created.

[0021] Applicant realizes that by using a separate cache to store copies of pending state deltas, state data can be read from the database (e.g., by a state delta generator or by any other component or system requesting the state data) along with any pending state updates stored in the cache that are applicable to the version of the state data being retrieved. This can significantly reduce state lag, potentially making it negligible.

[0022] It will be appreciated that the term "version," as used herein with respect to state data, should be understood to mean the state data stored in the database at a particular point in time. The database need not simultaneously store multiple versions of the state data; generally, only a single version of the state data may exist in the database at any given time, and that state data is updated when necessary, as outlined above. When a state delta-generator (or any other component) reads state data from a database, it may simply read "the" state data that exists in the database at that instant in time, and so that "snapshot" of the state data will be referred to as a particular "version" of the state data for ease of reference.

[0023] A timestamp and / or version number may be associated with the state data, where each time the state data in the database is updated, the timestamp is updated and / or the version number is incremented. Such a timestamp or version number may be used as a value indicating the version of the state data used in generating the state delta.

[0024] The event state deltas generated by the state delta-generator may be stored in a cache using a dedicated process, or the state delta-generator itself may, in some embodiments, be configured to store a copy of each state delta it generates in a cache.

[0025] In some embodiments, the system is configured such that when the system receives a request to read a given version of state data stored in the database, the system queries the cache to determine whether there are any event state deltas that should be applied to the given version of state data, and in some such embodiments, when the system determines that there are event state deltas that should be applied to the given version of state data, the system supplies those event state deltas so determined along with the given version of state data.

[0026] In a particular set of embodiments, the system generates the transient updated state data using a given version of the state data and the applicable event state deltas before providing the transient updated state data. It will be appreciated that generating the transient state data does not result in the state data stored in the database being updated in time, and the transient updated state data may be discarded after the associated event has been processed. Alternatively, requestors of state data may be provided with the applicable state deltas directly (along with the state data from the database) so that they can apply the associated state deltas to the given version of the state data provided with them.

[0027] Thus, in some embodiments, the system: i) when the system receives a request to read a given version of state data stored in a database, the system queries the cache to determine whether there are any event state deltas applicable to the given version of the state data; ii) When the system determines that there are event state deltas that are applicable to a given version of the state data, the system supplies those event state deltas so determined along with the given version of the state data. It is configured as follows.

[0028] As outlined above, a request to read state data stored in a database may be responded to with the state data and any applicable state deltas. In a particular set of embodiments, state deltas applicable to the retrieved state data are applied to the retrieved state data by a state delta-generator prior to processing the respective incoming event data and the retrieved version of the state data using the respective models. The state delta-generator may be configured to apply the state deltas itself, or this may be done by a separate component within the system, e.g., a dedicated pre-processing unit, which applies the state deltas to the retrieved state data before it is used by the state delta-generator.

[0029] When updated state data is written to the database, copies of event state deltas corresponding to older versions of the state data may be removed from the cache after the updated state data is written to the database. Thus, in some embodiments, copies of event state deltas having values ​​that indicate versions of state data that are older than that of the state data being updated are removed from the cache after the updated state data is written to the database.

[0030] In one set of embodiments, the system is configured for use in real-time anomaly detection, particularly in transaction processing systems. In a particular set of embodiments, the system is a machine learning transaction processing system. Thus, in some embodiments, the incoming event data includes or is derived from transaction data. The incoming event data may additionally or alternatively include data related to deposits, address updates, anti-money laundering (AML) workflow actions, or any other events associated with digital payment or transaction processing systems.

[0031] In some embodiments, the transaction data includes electronic data associated with one or more transactions, each transaction including a series of communications between different electronic systems to effect a payment or exchange.

[0032] The architecture provided by embodiments of the present invention allows for concurrent access, in which multiple state delta-generators may be used and work independently of each other using the same shared database of state data. Thus, in some embodiments, the system may: a plurality of state delta generators, i) retrieving a version of the state data from a database; ii) receiving each incoming event data; and iii) processing each incoming event data and the retrieved version of the state data using the respective model to generate a respective event state delta that includes a set of pending updates to the state data; iv) storing the event state delta in a queue; and a plurality of state delta-generators each configured to: Equipped with A copy of each event state delta generated by each of the state delta generators is stored in a cache along with a value indicating the version of the state data used to generate the respective event state delta.

[0033] In general, there may be multiple state delta-generators and updaters. In a particular set of embodiments where there are multiple state delta-generators and multiple updaters, one or more of the multiple updaters may service more than one state delta-generator.

[0034] Generally, more than one cache may be provided, e.g., when multiple updaters are used, one cache is provided per updater. Additionally or alternatively, one or more updaters may share a cache.

[0035] The model applied by the state delta-generator may include a machine learning model and / or an adaptive rule set. When multiple state delta-generators are provided in accordance with some embodiments of the present invention, each may apply a respective machine learning model and / or adaptive rule set, as appropriate.

[0036] Additionally or alternatively, the (or each) state delta-generator may be configured to use multiple models when processing incoming event data, with each model generating a respective set of pending updates to the state data, a state delta including each set of pending updates to the state data, the or each such state delta being stored in a queue, and a cache configured to store copies of the or each such state delta. Each model may be different, and each model may include a respective machine learning model and / or adaptive rule set, as appropriate. It will be appreciated that a separate respective state delta may be generated for each model, including a respective set of updates to the state data resulting from that model. Additionally or alternatively, a state delta may include sets of updates resulting from more than one model, and potentially all of the updates from all models within a particular state delta-generator may be included in a single state delta generated by that state delta-generator.

[0037] The "event" state deltas applied by the state delta generator are generated from incoming event data and stored state data using a model (e.g., a suitable machine learning model). However, applicant recognizes that there may be updates to state data (hereinafter referred to as "batch updates") resulting from "batch" processes as well as from live transactions. Examples of such batch processes include changes in details, such as a change in a cardholder's address or a change in the details of a particular entity, such as a merchant. Additionally or alternatively, some transaction data may be sent in batches rather than in real time, for example, for retrospective analysis (e.g., scoring batches of past transactions for anomalies).

[0038] In some embodiments, the updater is configured to receive one or more batch state deltas containing a set of pending batch updates to the state data, and to apply the pending batch updates from each batch state delta to the state data when generating updated state data. When multiple updaters are provided, one or more of such multiple updaters may be configured to apply the updates associated with a batch state delta, as in some embodiments of the invention described below.

[0039] The configuration of which state delta-generators (for live events) and batch state delta-generators (for batch events) are associated with which updaters may require consideration. When both live and batch events are updating the same set of data, it may be advantageous to allocate updaters by entity so that live updates for a first customer (customer1) and batch updates for customer1 go to the same updater (when there may be multiple entities in each event, e.g., different account holders or merchants). In such a configuration, a single block of data for customer1 in the database is updated consistently. When batch and live update different sets of state data (both can be read by both systems), a different set of updaters may be used for batch, again allocated by entity id within the block of state for which that set of updaters is responsible. Live updates and batch updates for a second customer (customer2) can go to a different updater than the one used for customer1, as long as the live updates and batch updates for customer2 go to the same updater as each other.

[0040] Thus, in such embodiments, the respective updates associated with the event state deltas and the batch state deltas may be applied simultaneously. Some or all of the event state deltas may be applied simultaneously with some or all of the batch state deltas; i.e., “batch” and “live” updates may be applied in parallel. In traditional machine learning systems for processing transactional data, updates from batch processing may “delay” the processing of live events while the batch updates are applied. However, embodiments of the present invention enable such types of updates to be applied when ready by holding such updates in a queue until the updater is ready to apply them and having a copy in a cache. Advantageously, the cached copy of the state deltas with their pending updates eliminates the issues with state liveness as outlined above—a request to read state data from a database may be responded to with both the stored data and any pending updates applicable to that version, which may include updates associated with both live and batch events.

[0041] Although the batch state deltas may be generated externally and provided to the updater, in some embodiments the system comprises a batch state delta generator configured to generate the batch state deltas. Thus, in some embodiments the system comprises a batch state delta generator, the batch state delta generator comprising: i) generating a respective batch state delta containing a set of pending batch updates to the state data; ii) storing the batch state delta in a queue; configured to: The cache is further configured to store a copy of each batch state delta having pending batch updates to the state data in the queue, and each of the copies of the batch state delta is stored in the cache along with a value indicating the version of the state data to which the batch state delta is applicable.

[0042] The batch generator may generate batch state deltas based on received batch request data. Such batch request data may include information about required batch changes, such as changes in details about entities for which the database maintains information. Such batch request data may be received from an external device or system, or may be generated elsewhere within the system, as appropriate.

[0043] The batch state deltas generated by the batch state delta generator may be stored in a cache using a dedicated process, or the batch state delta generator itself may, in some embodiments, be configured to store a copy of each batch state delta it generates in a cache.

[0044] In some embodiments, the system is configured such that when the system receives a request to read a given version of state data stored in the database, the system queries the cache to determine whether there are any batch state deltas that should be applied to the given version of state data. In some such embodiments, when the system determines that there are batch state deltas that should be applied to the given version of state data, the system supplies those batch state deltas so determined along with the given version of state data. Thus, in a particular set of embodiments, a request to read state data may be replied to with the state data along with one or more event state deltas and / or one or more batch state deltas.

[0045] In a set of embodiments, there may be more than one such batch state delta-generator.

[0046] The system may, in some embodiments, employ multiple updaters, working independently of each other, to update a shared database of state data. Thus, in some embodiments, the system may: A plurality of updaters, i) retrieving one or more of the event state deltas from a queue; ii) retrieving state data from the database; and iii) applying pending updates from each respective event state delta to the state data to generate updated state data; iv) updating the database with the updated status data; and a plurality of updaters each configured to: Equipped with.

[0047] In some embodiments, the state data is indexed by an identifier. In some such embodiments, the identifier is an entity identifier. In a particular set of embodiments, each entity identifier corresponds to an entity for which real-time anomaly detection is being performed.

[0048] The state data retrieved from the database or stored in the database (in the case of updated state data) may, in some embodiments, be associated with a particular identifier, where the identifier is associated with the transaction. For example, a transaction may involve a particular cardholder with a particular entity identifier (or simply "entity id"). The state data retrieved from the database may then be the state data stored for that particular entity id, and the final updated state data will also be stored in the database for that entity id (typically overwriting the previous data for that entity id).

[0049] If multiple updaters are provided, each may be assigned on an entity-by-entity or entity group basis. For example, each updater may be configured to handle state delta and database updates for transactions associated with a particular entity or a particular group of entities. This grouping may be assigned based on a common characteristic, such as that of a supermarket, although in some embodiments, entities may be grouped to ease system load, i.e., to spread the number of updates more evenly across multiple updaters. Entities may be assigned to updaters using a consistent partitioning scheme. Examples include partitioning by entity type, using expected or observed frequency of entity updates for load balancing across updaters, or distributing entities using hashes of entity IDs and / or other uniquely identifying data. Those skilled in the art will appreciate that other partitioning schemes may be used within the scope of the claimed invention.

[0050] It will be appreciated that there are several different mechanisms, known per se in the art, that may be used to implement a cache. However, in some embodiments, the cache is random access. In a particular set of embodiments, the second memory store comprises random access memory. Applicant has appreciated that the use of random access is particularly advantageous because various state deltas applicable to a particular version of the database may be stored and retrieved non-contiguously from memory.

[0051] In some embodiments, the system comprises an event store, e.g., a distributed event store such as Apache Kafka®. It will be appreciated that Kafka® (and similar platforms known per se in the art) uses processes (called "producers" in Kafka®) that submit messages to "topics," which may be divided into different "partitions." Messages within a topic (and partition) are ordered by their respective "offsets," which define the position of each message within its respective partition. Messages are also typically provided with a timestamp. Other processes (called "consumers" in Kafka®) then read messages from the partitions.

[0052] Incoming event data may be received through any suitable interface. However, in some embodiments, the system includes an event application programming interface (API) through which the event data is received. In a particular set of embodiments, the event API includes the state delta-generator. In other words, the state delta-generator may reside within the API that handles the incoming event data. Such an event API may be exposed so that external systems and software can interface with the state delta-generator.

[0053] If multiple state delta-generators are used, a single event API may include multiple state delta-generators. Additionally or alternatively, multiple event APIs may be provided, each comprising one or more state delta-generators.

[0054] Applicant recognizes that providing a cache can lead to “bottlenecking” in the time it takes for updates to be applied when there are a large number of state deltas for the updater (or a particular updater when multiple are provided). While the overall degree of bottlenecking will generally be less than that associated with prior art configurations, in some embodiments, the system further comprises a state delta combiner configured to combine at least two state deltas before updated state data is generated. In other words, pending updates associated with multiple state deltas can be combined to reduce the load on the updater (or any particular updater). In some configurations, state deltas can be simply added to combine them; for example, if a first state delta causes a particular state data value to change by +7 and a second state delta causes that same state value to change by −2, the combination of the two state deltas will cause that state value to change by +5 (i.e., a net change of +7 and −2). A state delta combiner may advantageously provide a reduction in load if the combined deltas can be applied in less time than applying the original deltas sequentially.

[0055] Additionally or alternatively, the bottlenecking mentioned above can be addressed by keeping entities in an internal cache of the updater. The system can be configured to batch multiple reads for the same entity and apply them simultaneously. Thus, when the updater has one or more “high-frequency” entities (i.e., updated frequently), rather than reading from the database for every state delta for that entity, it keeps the most recent state stored in memory so that it does not read from the database every time—instead, it can simply overwrite the state data in the database with what is in memory after each state delta (or on a timer, or after every number of state deltas, such as after every five state deltas). This avoids the need to read from the database every time, and therefore can improve performance and reduce database load when there are several high-frequency entities in the incoming state delta queue.

[0056] According to a second aspect, an embodiment of the present invention provides a machine learning system for processing incoming event data associated with a transaction, the system comprising: a) a database configured to store state data; b) a processing arrangement configured to generate, using the model, one or more event state deltas with pending updates to the state data based on a version of the state data and the incoming event data; and c) a cache configured to store a copy of each event state delta having a pending update to state data, wherein each copy of the event state delta is stored in the cache along with a value indicating the version of state data used to generate the event state delta; and Equipped with d) The system is i) when the system receives a request to read a given version of state data stored in a database, the system queries the cache to determine whether there are any event state deltas applicable to the given version of the state data; ii) When the system determines that there are event state deltas that are applicable to a given version of the state data, the system supplies those event state deltas so determined along with the given version of the state data. The present invention provides a machine learning system configured to:

[0057] A second aspect of the present invention is a machine learning method for processing incoming event data associated with a transaction, the method comprising: a) storing state data in a database; b) using the model to generate one or more event state deltas with pending updates to the state data based on the version of the state data and the incoming event data; c) storing in a cache a copy of each event state delta having a pending update to state data, each said copy of an event state delta being stored in the cache along with a value indicating the version of state data used to generate said event state delta; d) receiving a request to read a given version of the state data stored in the database; e) querying the cache to determine whether there are any event state deltas applicable to the given version of the state data; f) when there are event state deltas that are applicable to a given version of the state data, providing those event state deltas so determined along with the given version of the state data; This ranges from machine learning methods, including

[0058] According to a third aspect, an embodiment of the present invention provides a machine learning system for processing incoming event data associated with a transaction, the system comprising: a) a first memory store comprising a database configured to store state data, the state data for use with a model configured to receive incoming event data and generate one or more event state deltas with pending updates to the state data; b) a second memory store comprising a cache configured to store copies of one or more event state deltas having pending updates to state data received from the model, each copy of the event state delta being stored in the cache along with a value indicating the version of the state data used to generate the event state delta; and A machine learning system is provided.

[0059] A third aspect of the present invention also provides a machine learning method for processing incoming event data associated with a transaction, the method comprising: a) storing state data in a database in a first memory store, the state data for use with a model configured to receive incoming event data and generate one or more event state deltas with pending updates to the state data; b) storing in a second memory store comprising a cache copies of one or more event state deltas having pending updates to state data received from the model, each copy of the event state delta being stored in the cache along with a value indicating the version of the state data used to generate the event state delta; This ranges from machine learning methods, including

[0060] Applicant recognizes that the use of state deltas to process all incoming event data associated with a transaction is novel and inventive in itself, and therefore, viewed from a fourth aspect, an embodiment of the present invention provides a machine learning system for processing incoming event data associated with a transaction, the system comprising: a) a first memory store comprising a database configured to store state data; b) at least one state delta generator, i) retrieving a version of the state data from a database; ii) receiving each incoming event data; and iii) processing each incoming event data and the retrieved version of the state data using the respective model to generate a respective event state delta that includes a set of pending updates to the state data; iv) storing the event state delta in a queue; and at least one state delta generator configured to: c) an updater, i) removing an event state delta from a queue; ii) retrieving state data from the database; and iii) applying pending updates from each event state delta to the state data to generate updated state data; iv) updating the database with the updated status data; and The updater and Equipped with A machine learning system is provided, in which the system is configured such that all updates to state data associated with incoming event data are made by applying one or more event state deltas.

[0061] A fourth aspect of the present invention also provides a machine learning method for processing incoming event data associated with a transaction, the method comprising: a) storing state data in a database in a first memory store; b) In the state delta generation step, i) retrieving a version of state data from a database; ii) receiving each incoming event data; iii) processing each incoming event data and the retrieved version of the state data using the respective model to generate a respective event state delta that includes a set of pending updates to the state data; iv) storing the event state delta in a queue; c) In the update step, i) removing an event state delta from a queue; ii) retrieving state data from the database; iii) applying pending updates from each event state delta to the state data to generate updated state data; iv) updating the database with the updated status data; Including, It extends to machine learning methods in which all updates to state data associated with incoming event data are made by applying one or more event state deltas.

[0062] The applicant recognises that a cache is novel and inventive in itself, and therefore viewed from a fifth aspect, an embodiment of the present invention provides a cache containing copies of one or more event state deltas having pending updates to state data for use with a model, each said copy of an event state delta being stored in the cache together with a value indicating the version of state data used by the model to generate said event state delta.

[0063] This fifth aspect of the invention extends to a method of operating a cache comprising the step of storing copies of one or more event state deltas having pending updates to state data for use with a model, each said copy of an event state delta being stored in the cache together with a value indicating the version of state data used by the model to generate said event state delta.

[0064] Embodiments of the present invention may be applied to a wide variety of digital transactions, including, but not limited to, card payments, so-called "wire" transfers, peer-to-peer payments, Bankers' Automated Clearing System (BACS) payments, and Automated Clearing House (ACH) payments. The output of machine learning systems may be used to prevent a wide variety of fraudulent and criminal behavior, such as card fraud, application fraud, payment fraud, merchant fraud, gaming fraud, and money laundering.

[0065] It will be appreciated that optional features described above with respect to embodiments of any aspect of the invention apply equally to other aspects of the invention, where technically appropriate.

[0066] Where technically appropriate, embodiments of the invention may be combined. Embodiments are described herein as comprising several features / elements. The present disclosure also extends to separate embodiments consisting of, or consisting essentially of, said features / elements.

[0067] One or more methods according to any embodiment of any aspect of the present invention may be computer-implemented.

[0068] The aspects of the invention described above each extend to a non-transitory computer-readable medium containing instructions that, when executed by a processor, cause the processor to perform the method of such aspect of the invention.

[0069] The aspects of the invention described above each further extend to a computer software product comprising instructions that, when executed by a processor, cause the processor to perform the method of such aspect of the invention.

[0070] Technical references, such as patents and applications, are incorporated herein by reference.

[0071] Any embodiment specifically and explicitly described herein may form the basis of a disclaimer, either alone or in combination with one or more additional embodiments.

[0072] In the context of this specification, "comprising" should be interpreted as "including." Aspects of the invention that comprise several elements are also intended to cover alternative embodiments that "consist" or "consist essentially" of the associated elements.

[0073] The terms "memory" or "memory store" should be understood to mean any means suitable for storing data, and include both volatile and non-volatile memory as appropriate for the intended application. Those skilled in the art will appreciate that this includes, but is not limited to, random access memory (RAM), static RAM (SRAM), dynamic RAM (DRAM), magnetic storage, solid-state storage, and flash memory. It will be appreciated that, where technically appropriate, combinations of one or more of these may also be used for data storage (e.g., using faster-access volatile memory for frequently accessed data).

[0074] The term "data" is used in different contexts herein to refer to digital information, such as that represented by known bit structures in one or more programming languages. In use, data may refer to digital information stored as a bit sequence in computer memory. Some machine learning models may operate on structured arrays of data in a predefined bit format. Those skilled in the art will readily appreciate that these may be referred to as arrays, multidimensional arrays, matrices, vectors, tensors, or other such similar terms. For machine learning methods, it should be noted that a multidimensional array or tensor, e.g., having defined ranges in multiple dimensions, may be "flattened" so as to be represented (e.g., in memory) as a sequence or vector of values ​​stored according to a predefined format (e.g., n-bit integers or floating-point numbers, signed or unsigned). Thus, the term "tensor" as used herein covers multidimensional arrays with one or more dimensions (e.g., vectors, matrices, volumetric arrays, etc.).

[0075] The principles of the present invention apply regardless of the particular data format chosen. Data may be represented as an array, vector, tensor, or any other suitable format. For ease of reference, these terms are used interchangeably herein, and references to a "vector" or "vectors" of values ​​should be understood to extend to any one or more n-dimensional tensors of values, as appropriate. Similarly, references to a "tensor" or "tensors" of values ​​should be understood to extend to vectors, which are understood by those skilled in the art to be simply one-dimensional tensors. The principles of the present invention may apply regardless of the formatting of the data structure used for these arrays of values. For example, those skilled in the art will readily understand that state data may be stored in memory as one-dimensional tensors (i.e., vectors) or as tensors with two or more dimensions (i.e., tensors), and that suitable modifications may be made to data processing elements to handle the selected data format. The relative positions between various state values, e.g., how they are ordered within a vector or tensor, generally do not matter, and the scope of the present invention is not limited to any particular data format or structure.

[0076] The term "structured numeric representation" is used to refer to numeric data in a structured form, such as a one or more dimensional array that stores numeric values ​​having a common data type, such as integers or floating-point values. Structured numeric representations may include vectors or tensors (as used within machine learning terminology). Structured numeric representations are typically stored as an indexed and / or contiguous set of memory locations; for example, a one-dimensional array of 64-bit floating-point numbers may be represented in computer memory as a contiguous sequence of 64-bit memory locations in a 64-bit computing system.

[0077] The term "transaction data" is used herein to refer to electronic data associated with a transaction. A transaction includes a series of communications between different electronic systems to effect a payment or exchange. Generally, transaction data may include data indicating events (e.g., actions undertaken in a timely manner) that are related to and may be beneficial to transaction processing. Transaction data may include structured data, unstructured data, semi-structured data, or any combination thereof. Transaction data may also include data associated with a transaction, such as data used to process the transaction. In some cases, transaction data may be used broadly to refer to actions taken with respect to one or more electronic devices. Transaction data may take various forms depending on the exact implementation. However, different data types and formats may be converted appropriately by pre- or post-processing.

[0078] The term “interface” is used herein to refer to any physical and / or logical interface that enables one or more of data input and data output. An interface may be implemented by a network interface adapted to send and / or receive data or by retrieving data from one or more memory locations, as performed by a processor executing a set of instructions. An interface may also include a physical (network) connection through which data is received, such as hardware for enabling wired or wireless communication over a particular medium. An interface may include an application programming interface and / or a method call or method return. For example, in a software implementation, an interface may include data passes and / or memory references to functions initiated via method calls, where the functions include computer program code executed by one or more processors, while in a hardware implementation, an interface may include wired interconnections between different chips, chipsets, or portions of chips. In drawings, an interface may be indicated by a processing block boundary with inward and / or outward arrows representing data transfer.

[0079] The terms "component" and "module" are used interchangeably to refer to either a hardware structure having a specific function (e.g., in the form of mapping input data to output data) or a combination of general hardware and specific software (e.g., specific computer program code running on one or more general-purpose processors). A component or module may be implemented as a specific packaged chipset, e.g., an application-specific integrated circuit (ASIC) or programmed field-programmable gate array (FPGA), and / or as a software object, class, class instance, script, code portion, etc., as executed in use by a processor.

[0080] The term "machine learning model" is used herein to refer to at least a hardware-implemented implementation of a machine learning model or function. Known models within the field of machine learning include logistic regression models, naive Bayes models, random forests, support vector machines, and artificial neural networks. Classifier implementations may be provided within one or more machine learning programming libraries, including, but not limited to, scikit-learn, TensorFlow, and PyTorch.

[0081] The term "mapping" is used herein to refer to the transformation or conversion of a first set of data values ​​to a second set of data values. These two sets of data values ​​may be arrays of different sizes, with the output array having a smaller dimensionality than the input array. The input and output arrays may have a common or different data type. In some examples, the mapping is a one-way mapping to a scalar value.

[0082] The term "neural network architecture" refers to a set of one or more artificial neural networks configured to perform a specific data processing task. For example, a "neural network architecture" may include a specific configuration of one or more neural network layers of one or more neural network types. Neural network types include convolutional neural networks, recurrent neural networks, and feedforward neural networks. Convolutional neural networks involve the application of one or more convolution operations. Recurrent neural networks involve internal states that are updated during a sequence of inputs. Thus, recurrent neural networks are considered to include a form of recurrent or feedback connection, whereby the state of a recurrent neural network at a given time or iteration (e.g., t) is updated using the state of the recurrent neural network at the previous time or iteration (e.g., t-1). Feedforward neural networks involve transformation operations without feedback, e.g., operations are applied in a unidirectional sequence from input to output. Feedforward neural networks include simple "neural networks" and "fully connected" neural networks. Those skilled in the art will appreciate that "multilayer perceptron" is a term used to describe a fully connected layer and is a special case of a feedforward neural network.

[0083] The term "deep" neural network is used to indicate that the neural network includes multiple neural network layers in series (the term "deep" is used with both feedforward and recurrent neural networks). Some examples described herein utilize recurrent and fully connected neural networks.

[0084] A "neural network layer," as typically defined within machine learning programming tools and libraries, can be considered an operation that maps input data to output data. A "neural network layer" may apply one or more parameters, such as weights, to map input data to output data. One or more bias terms may also be applied. The weights and biases of a neural network layer may be applied using one or more multidimensional arrays or matrices. Generally, a neural network layer has multiple parameters whose values ​​affect how the layer maps input data to output data. These parameters may be trained in a supervised manner by optimizing an objective function, which typically involves minimizing a loss function. Some parameters may also be pre-trained or fixed in another manner. Fixed parameters may be considered configuration data that control the operation of the neural network layer. A neural network layer or neural network architecture may include a mixture of fixed and learnable parameters. A recurrent neural network layer may apply a series of operations to update the recurrent state and transform input data. The update of the recurrent state and the transformation of the input data may involve one or more transformations of the previous recurrent state and the input data. A recurrent neural network layer may be trained by deploying modeled recurrent units so that it can be applied within machine learning programming tools and libraries. Although a recurrent neural network may appear to include several (sub)layers for applying different gating operations, most machine learning programming tools and libraries refer to the application of a recurrent neural network as a whole as a "neural network layer," and this convention will be followed here. Finally, a feedforward neural network layer may apply one or more sets of weights and biases to input data to generate output data. This operation may be expressed as a matrix operation (e.g., where a bias term can be included by adding a value of 1 onto the input data).Alternatively, the bias may be applied through a separate addition operation. As explained above, the term "tensor" is used to refer to an array that may have multiple dimensions, such as in machine learning libraries; for example, a tensor may include a vector, a matrix, or a data structure with a larger number of dimensions. In a preferred example, the described tensor may include a vector with a predefined number of elements.

[0085] To model complex nonlinear functions, neural network layers as described above may be followed by nonlinear activation functions. Common activation functions include the sigmoid function, the hyperbolic tangent function, and the rectified linear unit (RELU). Many other activation functions exist and can be applied. The activation function may be selected based on testing and preference. The activation function may be omitted in some situations and / or may form part of the internal structure of the neural network layer.

[0086] The exemplary neural network architectures described herein may be configured through training. In some cases, “learnable” or “trainable” parameters may be trained using a technique called backpropagation. During backpropagation, the neural network layers that make up each neural network architecture are initialized (e.g., with randomized weights) and then used to make predictions using a set of input data from a training set (e.g., a so-called “forward” pass). The predictions are used to evaluate a loss function. For example, “ground truth” outputs may be compared to predicted outputs, and the difference may form part of the loss function. In some examples, the loss function may be based on the absolute difference between a predicted scalar value and a binary ground truth label. The training set may include a set of transactions. When gradient descent is used, the loss function is used to determine the gradient of the loss function with respect to the parameters of the neural network architecture, where the gradient is then used to backpropagate updates to the parameter values ​​of the neural network architecture. Typically, the updates are propagated according to the derivatives of the neural network layer weights. For example, the gradient of the loss function with respect to the weights of a neural network layer may be determined and used to determine updates to the weights that minimize the loss function. In this case, optimization techniques such as gradient descent, stochastic gradient descent, Adam, etc. may be used to adjust the weights. Chain rules and auto-differentiation functions may be applied to efficiently compute the gradient of the loss function, working backward in sequence through the neural network layers.

[0087] Some embodiments of the present invention will now be described with reference to the accompanying drawings. [Brief explanation of the drawings]

[0088] [Figure 1A] 1A-1C are schematic diagrams illustrating different exemplary electronic infrastructures for transaction processing. [Figure 1B] 1A-1C are schematic diagrams illustrating different exemplary electronic infrastructures for transaction processing. [Figure 1C] 1A-1C are schematic diagrams illustrating different exemplary electronic infrastructures for transaction processing. [Figure 2A] 1A-1C are schematic diagrams illustrating different examples of data storage systems for use by a machine learning transaction processing system. [Figure 2B] 1A-1C are schematic diagrams illustrating different examples of data storage systems for use by a machine learning transaction processing system. [Figure 3A] 3A and 3B are schematic diagrams illustrating different examples of transaction data. [Figure 3B] 3A and 3B are schematic diagrams illustrating different examples of transaction data. [Figure 4] FIG. 1 is a schematic diagram illustrating example components of a machine learning transaction processing system. [Figure 5A] FIG. 2 is a sequence diagram illustrating an example set of processes performed on transaction data by different computing entities. [Figure 5B] FIG. 2 is a sequence diagram illustrating an example set of processes performed on transaction data by different computing entities. [Figure 6] FIG. 1 is a block diagram illustrating a machine learning system for processing incoming event data. [Figure 7] FIG. 1 is a block diagram illustrating a machine learning system for processing incoming event data in which a copy of the state delta is stored in a cache, according to one embodiment of the present invention. [Figure 8] FIG. 1 is a block diagram illustrating a machine learning system that utilizes state deltas for live and batch events. DETAILED DESCRIPTION OF THE INVENTION

[0089] Described herein are several example embodiments of machine learning systems for use in transaction processing. In some embodiments, a machine learning system is applied in a real-time, high-volume transaction processing pipeline to provide an indication of whether a transaction or entity matches a previously observed and / or predicted pattern of activity or action, e.g., whether the transaction or entity is “normal” or “anomalous.” The term “behavioral” is used herein to refer to this pattern of activity or action. The indication may include a scalar value that is normalized within a predefined range (e.g., 0 to 1) and can then be used to prevent fraud and other misuse of the payment system. The machine learning system may apply a machine learning model that is updated as more transaction data is acquired, e.g., continually trained based on new data, to reduce false positives and maintain the accuracy of output metrics. This example may be particularly useful for preventing fraud when the physical presence of a payment card cannot be confirmed (e.g., online transactions known as “card-not-present”), or for commerce transactions where high-value transactions may be routine and where it may be difficult to classify a pattern of behavior as “unexpected.” Thus, the present example facilitates the processing of transactions as these transactions become primarily "online," ie, conducted digitally over one or more public communications networks.

[0090] Some embodiments described herein enable machine learning models to be tuned to be specific to patterns of behavior between entities (e.g., account holders) and categories (e.g., merchants, transaction amounts, time of day, etc.). For example, a machine learning model may model entity-category pair-specific patterns of behavior. The machine learning systems described herein can provide machine learning models that dynamically update despite large transaction flows and / or the need for isolation of different data sources.

[0091] As outlined above, embodiments of the present invention may be applied to a wide variety of digital transactions, including, but not limited to, card payments, so-called "wire" transfers, peer-to-peer payments, Bank Automated Clearing System (BACS) payments, and Automated Clearing House (ACH) payments. The output of machine learning systems may be used to prevent a wide variety of fraudulent and criminal behavior, such as card fraud, application fraud, payment fraud, merchant fraud, gaming fraud, and money laundering.

[0092] A non-limiting exemplary embodiment of a machine learning system according to an embodiment of the present invention is described below: Figures 1A-5B provide a context for the machine learning system.

[0093] Exemplary Transaction Processing System 1A-1C illustrate a set of exemplary transaction processing systems 100, 102, 104. These exemplary transaction processing systems are described to provide a context for the inventions described herein but should not be considered limiting, and the configuration of any one implementation may vary based on the particular requirements of that implementation. However, the exemplary transaction processing systems described will enable one skilled in the art to identify several high-level technical features that are pertinent for the following discussion. These three exemplary transaction processing systems 100, 102, 104 illustrate different areas in which variations may occur.

[0094] 1A-1C illustrate a set of client devices 110 configured to initiate a transaction. In this example, the set of client devices 110 includes a smartphone 110-A, a computer 110-B, a point-of-sale (POS) system 110-C, and a portable merchant device 110-D. These client devices 110 provide a non-exhaustive set of examples. In general, any electronic device or set of devices may be used to initiate a transaction. In some cases, the transaction includes a purchase or payment. For example, the purchase or payment may be an online or mobile purchase or payment made through the smartphone 110-A or computer 110-B, or may be a purchase or payment made at a merchant store, such as via the POS system 110-C or portable merchant device 110-D. The purchase or payment may be for goods and / or services.

[0095] 1A-1C, client device 110 is communicatively coupled to one or more computer networks 120. Client device 110 may be communicatively coupled in various ways, including via one or more wired and / or wireless networks, including telecommunications networks. In a preferred example, all communications across one or more computer networks are secured, for example, using the Transport Layer Security (TLS) protocol. In FIG. 1A, two computer networks are shown as 120-A and 120-B. These may be separate networks or different parts of a common network. First computer network 120-A communicatively couples client device 110 to merchant server 130. Merchant server 130 may execute computer processes that implement process flows for transactions. For example, merchant server 130 may be a backend server that handles transaction requests received from POS system 110-C or portable merchant device 110-D, or may be used by an online merchant to implement a website where purchases can be made. It will be appreciated that the examples of Figures 1A-1C are necessary simplifications of actual architectures, and that there may be several interacting server devices implementing an online merchant, including, for example, separate server devices for providing HyperText Markup Language (HTML) pages detailing products and / or services and for handling payment processes.

[0096] In FIG. 1A, merchant server 130 is communicatively coupled to an additional set of backend server devices for processing transactions. In FIG. 1A, merchant server 130 is communicatively coupled to payment processor server 140 via second network 120-B. Payment processor server 140 is communicatively coupled to a first data storage device 142 that stores transaction data 146 and a second data storage device 144 that stores auxiliary data 148. Transaction data 146 may include batches of transaction data related to different transactions undertaken over a period of time. Ancillary data 148 may include data associated with transactions, such as records storing merchant data and / or end-user data. In FIG. 1A, payment processor server 140 is communicatively coupled to machine learning server 150 via second network 120-B. Machine learning server 150 implements machine learning system 160 for processing transaction data. The machine learning system 160 is configured to receive input data 162 and map it to output data 164 used by the payment processor server 140 to process a particular transaction, such as a transaction originating from a client device 110. In some cases, the machine learning system 160 receives at least transaction data associated with a particular transaction and provides an alert or numerical output used by the payment processor server 140 to determine whether the transaction should be permitted (i.e., approved) or denied. Thus, the output of the machine learning system 160 may include a label, alert, or other indication of fraudulent activity, or general malicious or anomalous activity. The output may include a probabilistic indication, such as a score or probability. In some cases, the output data 164 may include a scalar numeric value. The input data 162 may further include data derived from one or more of the transaction data 146 and the auxiliary data 148.In some cases, output data 164 indicates a level of deviation from a particular expected pattern of behavior based on past observations or measurements. For example, this can often indicate fraudulent or criminal behavior when it differs significantly, especially on a large scale, from the observed pattern of behavior. Output data 164 may form behavioral measurements. Expected patterns of behavior may be defined either explicitly or implicitly based on observed interactions between different entities in a transaction process flow, such as an end user or customer, a merchant (including point-of-sale and back-end locations or entities, if these may be different), and a bank.

[0097] The machine learning system 160 may be implemented as part of a transaction processing pipeline. An exemplary transaction processing pipeline is described below with respect to FIGS. 5A and 5B. The transaction processing pipeline may include electronic communications between the client device 110, the merchant server 130, the payment processor server 140, and the machine learning server 150. Other server devices may also be involved, such as a banking server that provides authorization from the issuing bank. In some cases, the client device 110 may communicate directly with the payment processor server 140. In use, the transaction processing pipeline typically needs to be completed within 100 or 200 milliseconds. Generally, processing times of less than one second may be considered real-time (e.g., humans typically perceive events in a 400 ms time span). Furthermore, 100-200 ms may be the desired maximum latency of the total round-trip time for transaction processing. Within this time span, the time allocated for machine learning system 160 may be a small fraction of this total, such as 10 ms (i.e., less than 5-10% of the target processing time), when most of that time may be reserved for other operations in the transaction processing flow. This presents technical constraints for the implementation of machine learning system 160. Furthermore, in real-world implementations, the average processing rate may be on the order of 1000-2000 transactions per second. This means that most "off-the-shelf" machine learning systems are not suitable for implementing machine learning system 160. This further means that most machine learning techniques described in academic papers cannot be implemented within the above-mentioned transaction processing pipeline without nontrivial adaptation. Another problem is that anomalies are inherently rare events, and therefore accurate machine learning systems are difficult to train.

[0098] Figure 1B shows a variation 102 of the exemplary transaction processing system 100 of Figure 1A. In this variation 102, the machine learning system 160 is implemented within the payment processor computer infrastructure, e.g., executed by the payment processor server 140 and / or on a locally coupled server in the same local network as the payment processor server 140. The variation 102 of Figure 1B may be preferred for larger payment processors because it allows for faster response times, greater control, and improved security. Functionally, however, its transaction processing pipeline may be similar to that of Figure 1A. For example, in the example of FIG. 1A, the machine learning system 160 may be initiated by a secure external application programming interface (API) call, such as a Representation State Transfer (REST) ​​API call using Hypertext Transfer Protocol Secure (HTTPS), while in FIG. 1B, the machine learning system 160 may be initiated by an internal API call, where a common end API may handle both requests (e.g., a REST HTTPS API may provide an external wrapper for the internal API).

[0099] FIG. 1C shows another variation 104 of the example transaction processing system 100 of FIG. 1A. In this variation 104, the machine learning system 160 is communicatively coupled to a local data storage device 170. For example, the data storage device 170 may be on the same local network as the machine learning server 150 or may comprise a local storage network accessible to the machine learning server 150. In this case, there are multiple local data storage devices 170-A through 170-N, where each data storage device stores partitioned auxiliary data 172. The partitioned auxiliary data 172 may include parameters for one or more machine learning models. In some cases, the auxiliary data 172 may include state for the machine learning models, where the state may relate to a particular entity, such as a user or a merchant. The partitioning of the auxiliary data 172 may need to be applied to meet security requirements set by a third party, such as a payment processor, one or more banks, and / or one or more merchants. During use, machine learning system 160 accesses auxiliary data 172-A-172-N via multiple local data storage devices 170-A-170-N based on input data 162. For example, input data 162 may be received through an API request from a particular source and / or may include data identifying that a particular classification should be used to handle the API request. More details of different storage systems that may be applied to meet security requirements are set forth in Figures 2A and 2B.

[0100] Exemplary Data Storage Configurations 2A and 2B show two exemplary data storage configurations 200 and 202 that may be used by an exemplary machine learning system 210 for processing transaction data. The examples of FIGS. 2A and 2B are two non-limiting examples illustrating different options available for implementation, and a specific configuration may be selected according to individual circumstances. The machine learning system 210 may comprise an implementation of the machine learning system 160 described in the previous examples of FIGS. 1A-1C. The examples of FIGS. 2A and 2B, for example, enable the machine learning system 210 to process transaction data secured using heterogeneous cryptographic parameters to securely process transaction data for heterogeneous entities. It will be appreciated that the configurations of FIGS. 2A and 2B may not be used if the machine learning system 160 is implemented, for example, within an internal transaction processing system, for a single set of secure transaction data and auxiliary data, or as a hosted system for use by a single payment processor.

[0101] FIG. 2A shows a machine learning system 210 communicatively coupled to a data bus 220. The data bus 220 may include an internal data bus of the machine learning server 150 or may form part of a storage area network. The data bus 220 communicatively couples the machine learning system 210 to multiple data storage devices 230, 232. The data storage devices 230, 232 may comprise any known data storage device, such as magnetic hard disks and solid-state devices. Although the data storage devices 230, 232 are shown as different devices in FIG. 2A, they may alternatively form different physical areas or portions of storage within a common data storage device. In FIG. 2A, the multiple data storage devices 230, 232 store historical transaction data 240 and auxiliary data 242. In FIG. 2A, a first set of data storage devices 230 stores historical transaction data 240, and a second set of data storage devices 232 stores auxiliary data 242. Ancillary data 242 may include one or more of model parameters for a set of machine learning models (such as trained parameters for a neural network architecture and / or configuration parameters for a random forest model) and state data for those models. In some cases, different sets of historical transaction data 240-A-N and auxiliary data 242-A-N are associated with different entities that securely and collectively use the services provided by machine learning system 210; for example, they may represent data for different banks that need to be kept separate as part of the terms of providing machine learning services to those entities.

[0102] 2B illustrates another manner in which different sets of historical transaction data 240-A-N and auxiliary data 242-A-N may be stored. In FIG. 2B, machine learning system 210 is communicatively coupled to at least one data storage device 260 via data transfer channel 250. Data transfer channel 250 may include a local storage bus, a local storage area network, and / or a remote secure storage coupling (e.g., overlaid on an insecure network such as the Internet). In FIG. 2B, secure logical storage layer 270 is provided using physical data storage device 260. Secure logical storage layer 270 may be a virtualized system that is actually implemented independently on at least one data storage device 260 but appears to machine learning system 210 as a separate physical storage device. Logical storage layer 270 may provide separate encrypted partitions 280 for data relating to groups of entities (e.g., relating to different issuing banks, etc.), and different sets of historical transaction data 240-A-N and auxiliary data 242-A-N may be stored in corresponding partitions 280-A-N. In some cases, entities may be created dynamically as transactions are received for processing based on data stored by one or more of the server systems shown in Figures 1A-1C.

[0103] Example Transaction Data 3A and 3B show examples of transaction data that may be processed by a machine learning system such as 160 or 210. FIG. 3A shows how the transaction data may include a chronological set of records 300, where each record has a timestamp and comprises multiple transaction fields. In some cases, the transaction data may be grouped and / or filtered based on the timestamps. For example, FIG. 3A shows the partitioning of the transaction data into current transaction data 310, which is associated with the current transaction, and “older” or historical transaction data 320, which falls within a predefined time range of the current transaction. The time range may be set as a hyperparameter of any machine learning system. Alternatively, the “older” or historical transaction data 320 may be set as some number of transactions. A mixture of the two approaches is also possible.

[0104] FIG. 3B illustrates how transaction data 330 for a particular transaction may be stored in a numeric format for processing by one or more machine learning models. For example, in FIG. 3B, the transaction data has at least the following fields: transaction amount, timestamp (e.g., as a Unix epoch), transaction type (e.g., card payment or direct debit), product description or product identifier (i.e., relating to the item being purchased), merchant identifier, issuing bank identifier, character set (e.g., Unicode characters in a field of predefined character length), country identifier, etc. Note that a wide variety of data types and formats may be received and preprocessed into appropriate numeric representations. In some cases, incoming transaction data, such as that generated by a client device and sent to merchant server 130, is preprocessed to convert alphanumeric data types to numeric data types for application of one or more machine learning models. Other fields present in the transaction data may include, but are not limited to, an account number (e.g., a credit card number), the location where the transaction is being performed, and the manner in which the transaction is performed (e.g., in person, over the phone, on a website).

[0105] Exemplary Machine Learning System FIG. 4 shows an example 400 of a machine learning system 402 that may be used to process transaction data. The machine learning system 402 may implement one or more of the machine learning systems 160 and 210. The machine learning system 402 receives input data 410. The format of the input data 410 may depend on which machine learning model is being applied by the machine learning system 402. When the machine learning system 402 is configured to perform fraud or anomaly detection with respect to a transaction, e.g., an ongoing transaction as described above, the input data 410 may include transaction data (i.e., data that forms part of the data package for the transaction), such as 330, as well as data derived from historical transaction data (e.g., 300 in FIG. 3A) and / or auxiliary data (e.g., 148 in FIGS. 1A and 1B or 242 in FIGS. 2A and 2B). The auxiliary data may include secondary data linked to one or more entities identified in the primary data associated with the transaction. For example, if the transaction data for an ongoing transaction identifies a user, a merchant, and one or more banks (such as an issuing bank for the user and a merchant bank) associated with the transaction, such as through unique identifiers present in the transaction data, the auxiliary data may include data related to these transaction entities. The auxiliary data may also include data derived from activity records, such as interaction logs and / or authentication records. In some cases, the auxiliary data is stored in one or more static data records and is retrieved from these records based on the received transaction data. Additionally or alternatively, the auxiliary data may include machine learning model parameters that are retrieved based on the content of the transaction data. For example, the machine learning model may have parameters that are specific to one or more of the user, merchant, and issuing bank, and these parameters may be retrieved based on which of these are identified in the transaction data.For example, one or more of the users, merchants, and issuing banks may have corresponding embeddings that may include retrievable or mappable tensor representations for the entities. For example, each user or merchant may have a tensor representation (e.g., a floating-point vector of size 128-1024) that may either be retrieved from a database or other data storage or generated by an embedding layer, e.g., based on a user index or merchant index.

[0106] Input data 410 is received at an input data interface 412. The input data interface 412 may include an API interface, such as an internal or external API interface as described above. This API interface is also referred to as the "EventAPI" below with reference to Figures 6-8.

[0107] In some cases, a payment processor server 140, such as shown in FIGS. 1A-1C, makes requests to this interface, where the request payload includes transaction data. The API interface may be defined to be agnostic with respect to the format or source of the transaction data. The input data interface 412 is communicatively coupled to a machine learning model platform 414. In some cases, requests made to the input data interface 412 trigger execution of the machine learning model platform 414 using the transaction data provided to the interface. The machine learning model platform 414 is configured as an execution environment for the application of one or more machine learning models to the input data 410. In some cases, the machine learning model platform 414 is configured as an execution wrapper for multiple different selectable machine learning models. For example, machine learning models may be defined using a model definition language (e.g., similar to or using a markup language such as Extensible Markup Language—XML). Model definition languages ​​may include SQL, TensorFlow, Caffe, Thinc, and PyTorch (independently or in combination, among others). In some cases, the model definition language includes computer program code that is executable to perform one or more of the training and inference of the defined machine learning models. The machine learning models may include, for example, artificial neural network architectures, ensemble models, regression models, decision trees such as random forests, graph models, and Bayesian networks, among others. The machine learning model platform 414 may define common (i.e., shared) input and output definitions such that different machine learning models are applied in a common (i.e., shared) manner.

[0108] In this example, the machine learning model platform 414 is configured to provide at least a single scalar output 416, which may be normalized within a predefined range, such as between 0 and 1. Once normalized, the scalar output 416 may be viewed as a probability that a transaction associated with the input data 410 is fraudulent or anomalous. In this case, a value of “0” may represent a transaction that matches a normal pattern of activity for one or more of the user, merchant, and issuing bank, while a value of “1” may indicate that the transaction is fraudulent or anomalous, i.e., does not match an expected pattern of activity. (Although, one skilled in the art will recognize that the normalized range may be different, such as inverted or within different boundaries, and have the same functional impact.) Note that while the range of values ​​may be defined as between 0 and 1, the output values ​​may not be uniformly distributed within this range; for example, a value of “0.2” may be a common output for a “normal” event, and a value of “0.8” may be considered above a threshold for a typical “anomalous” or fraudulent event. Thus, the machine learning models implemented by the machine learning model platform 414 may implement some form of mapping between high-dimensional input data (e.g., transaction data and any retrieved auxiliary data) and single-value output. In some cases, for example, the machine learning model platform 414 may be configured to receive input data for the machine learning models in a numeric format, and each defined machine learning model is configured to map the defined input data in a similar manner. The exact machine learning model applied by the machine learning model platform 414, and the parameters for that model, may be determined based on configuration data. The configuration data may be included in and / or identified using the input data 410, and / or set based on one or more configuration files parsed by the machine learning model platform 414.

[0109] In some cases, the machine learning model platform 414 may provide additional outputs depending on the context. In some implementations, the machine learning model platform 414 may be configured to return a “reason code” that captures a human-friendly explanation of the machine learning model's output with respect to the suspicious input attributes. For example, the machine learning model platform 414 may indicate which of one or more input elements or units in the input representation influenced the model output—e.g., the combination of the “amount” channel being above a learned threshold and the set of “merchant” elements or units (e.g., embeddings or indexes) being outside a given cluster. If the machine learning model platform 414 implements decision trees, these additional outputs may include paths through the decision trees or aggregate feature importance based on an ensemble of trees. In the case of neural network architectures, this may include layer output activations and / or layer filters with positive activations.

[0110] In FIG. 4 , some implementations may include an optional alert system 418 that receives the scalar output 416. In other implementations, the scalar output 416 may be passed directly to the output data interface 420 without post-processing. In this latter case, the scalar output 416 may be packaged in a response to the original request to the input data interface 412. In either case, output data 422 derived from the scalar output 416 is provided as an output of the machine learning system 402. The output data 422 is returned to enable final processing of the transaction data. For example, the output data 422 may be returned to the payment processor server 140 and used as the basis for a decision to approve or deny the transaction. Depending on implementation requirements, in some cases, the alert system 418 may process the scalar output 416 to return a binary value indicating whether the transaction should be approved or denied (e.g., “1” equals deny). In some cases, the decision may be made by applying a threshold to the scalar output 416. This threshold may be context-dependent. In some cases, the alarm system 418 and / or the output data interface 420 may also receive additional inputs, such as explanatory data (e.g., the "reason code" described above) and / or the original input data. The output data interface 420 may generate an output data package for the output data 422 that combines these inputs with the scalar output 416 (e.g., at least for logging and / or later review). Similarly, the alerts generated by the alarm system 418 may include and / or be based on the additional inputs described above, in addition to the scalar output 416, for example.

[0111] The machine learning system 402 is typically used in an “online” mode to process large volumes of transactions within a narrowly defined time range. For example, under normal processing conditions, the machine learning system 402 may process requests within 7–12 ms and be capable of managing 1,000–2,000 requests per second (these are moderate constraints from real-world operating conditions). However, the machine learning system 402 may also be used in an “offline” mode, for example, by providing selected historical transactions to the input data interface 412. In offline mode, input data may be passed to the input data interface in batches (i.e., groups). The machine learning system 402 may also be capable of implementing machine learning models that provide scalar outputs for entities and transactions, or entities on behalf of transactions. For example, the machine learning system 402 may receive requests associated with an identified user (e.g., cardholder or payment account holder) or an identified merchant and may be configured to provide a scalar output 416 indicating the likelihood that the user or merchant is fraudulent, malicious, or anomalous (i.e., a general threat or risk). For example, this may form part of a continuous or periodic monitoring process, or a one-time request (e.g., as part of an application for a service). The provision of a scalar output for a particular entity may be based on a set of transaction data up to and including the last approved transaction in a sequence of transaction data (e.g., transaction data for an entity similar to that shown in FIG. 3A).

[0112] As described below, both "live" and "batch" data can be received and used by the machine learning system. These can be received by the same set of Event APIs with different REST endpoints, or by a different set of Event APIs dedicated to reading batch data from another source. Additionally or alternatively, the Event API can read events from another source (rather than receiving events via REST), for example, by reading events from a comma-separated value (CSV) file or any other suitable file format.

[0113] Exemplary Transaction Process Flow 5A and 5B illustrate two possible exemplary transaction process flows 500 and 550. These process flows may take place in the context of the exemplary transaction processing systems 100, 102, and 104 illustrated in FIGS. 1A-1C, as well as other systems. Process flows 500 and 550 are provided as an example of a context in which a machine learning transaction processing system may be applied; however, not all transaction process flows necessarily follow the process illustrated in FIGS. 5A and 5B, and process flows may vary between implementations, systems, and over time. The exemplary transaction process flows 500 and 550 reflect two possible cases: a first case, represented by transaction process flow 500, in which the transaction is approved; and a second case, represented by transaction process flow 550, in which the transaction is rejected. Each transaction process flow 500, 550 involves the same set of five interacting systems and devices: a POS or user device 502, a merchant system 504, a payment processor (PP) system 506, a machine learning (ML) system 508, and an issuing bank system 510. The POS or user device 502 may comprise one of the client devices 110, the merchant system 504 may comprise a merchant server 130, the payment processor system 506 may comprise a payment processor server 140, and the machine learning system 508 may comprise an implementation of machine learning systems 160, 210, and / or 402. The issuing bank system 510 may comprise one or more server devices that perform transaction functions on behalf of the issuing bank. The five interacting systems and devices 502-510 may be communicatively coupled by one or more internal or external communication channels, such as network 120. In some cases, some of these systems may be combined, for example, the issuing bank may also act as a payment processor, so that systems 506 and 510 may be implemented together in a common system.In other cases, a similar process flow may be performed specifically for the merchant (e.g., without involving a payment processor or issuing bank), in which case the machine learning system 508 may communicate directly with the merchant system 504. In these variations, the general functional transaction process flow may remain similar to that described below.

[0114] The transaction process flows in both Figures 5A and 5B include several common (i.e., shared) processes 512-528. In block 512, the POS or user device 502 initiates the transaction. In the case of a POS device, this may involve a cashier attempting to make an electronic payment using a front-end device; in the case of a user device 502, this may involve a user making an online purchase (e.g., clicking "Done" in an online basket) using a credit or debit card or online payment account. In block 514, payment details are received as electronic data by the merchant system 504. In block 516, the transaction is processed by the merchant system 504, and a request is made to the payment processor system 506 to authorize the payment. In block 518, the payment processor system 506 receives a request from the merchant system 504. The request may be made over a proprietary communication channel or as a secure request over a public network (e.g., an HTTPS request over the Internet). The payment processor system 506 then makes a request to the machine learning system 508 for a score or probability for use in processing the transaction. Block 518 may further include retrieving auxiliary data for combination with the transaction data sent to the machine learning system 508 as part of the request. In other cases, the machine learning system 508 may have access to a data storage device that stores auxiliary data (e.g., similar to the configuration of FIGS. 2A and 2B ) and thus may retrieve this data as part of its internal operations (e.g., based on an identifier provided within the transaction data and / or as defined as part of an implemented machine learning model).

[0115] Block 520 illustrates a model initialization operation that occurs prior to any request from the payment processor system 506. For example, the model initialization operation may include loading a defined machine learning model and parameters that instantiate the defined machine learning model. At block 522, the machine learning system 508 receives a request from the payment processor system 506 (e.g., via a data input interface such as 412 in FIG. 4). At block 522, the machine learning system 508 may perform any defined preprocessing prior to applying the machine learning model initialized at block 520. For example, if the transaction data still holds character data, such as a merchant identified by a string of characters or a character transaction description, this may be converted into suitable structured numeric data (e.g., by converting string categorical data to an identifier and / or by mapping characters or groups of characters to a vector embedding via a lookup operation or other mapping). Then, at block 524, the machine learning system 508 applies the instantiated machine learning model by feeding the input data derived from the received request to the model. This may include applying the machine learning model platform 414 as described with reference to FIG. 4. At block 526, a scalar output is generated by the instantiated machine learning model, which may be processed in the machine learning system 508 to determine a binary "approve" or "reject" decision, or, if preferred, returned to the payment processor system 506 in response to the request made in block 518.

[0116] In block 528, the output of the machine learning system 508 is received by the payment processor system 506 and used to approve or deny the transaction. Figure 5A shows a process in which a transaction is approved based on the output of the machine learning system 508, and Figure 5B shows a process in which a transaction is denied based on the output of the machine learning system 508. In Figure 5A, the transaction is approved in block 528. Then, in block 530, a request is made to the issuing bank system 510. In block 534, the issuing bank system 510 approves or denies the request. For example, the issuing bank system 510 may approve the request if the end user or cardholder has sufficient funds and authorization to cover the transaction costs. In some cases, the issuing bank system 510 may apply a second level of security, although this may not be required if the issuing bank relies on anomaly detection performed by the payment processor using the machine learning system 508. In block 536, authorization from the issuing bank system 510 is returned to the payment processor system 506, which sends a response in block 538 to the merchant system 504, which responds in block 540 to the POS or user device 502. If the issuing bank system 510 approves the transaction in block 534, the transaction may be completed and a positive response may be returned to the POS or user device 502 via the merchant system 504. The end user may receive this as an "authorized" message on the screen of the POS or user device 502. The merchant system 504 may then complete the purchase (e.g., initiate internal processing to fulfill the purchase).

[0117] At a later point in time, one or more of the payment processor system 506 and the machine learning system 508 may store data related to the transaction, for example, as part of the transaction data 146, 240, or 300 in the previous example. This is indicated by dashed blocks 542 and 544. This process is performed by updaters 614, 714, 814, which are described below with reference to Figures 6-8.

[0118] The transaction data may be stored along with one or more of the output of the machine learning system 508 (e.g., scalar fraud or anomaly probability) and the final outcome of the transaction (e.g., whether the transaction was approved or rejected). The stored data may be stored for use as training data for the machine learning model implemented by the machine learning system 508.

[0119] The stored data may also be accessed as part of future iterations of block 524 and may, for example, form part of future auxiliary data. In some cases, the final outcome or outcome of a transaction may not be known at the time of the transaction. For example, a transaction may only be labeled as anomalous through later review by analysts and / or automated systems or based on feedback from a user (e.g., when a user reports fraudulent activity or indicates that a payment card or account has been compromised since a certain date). In these cases, ground truth labels for purposes of training the machine learning system 508 may be collected over time following the transaction itself.

[0120] Referring now to the alternative process flow of FIG. 5B , in this case, one or more of the machine learning system 508 and the payment processor system 506 reject the transaction based on the output of the machine learning system 508. For example, if the scalar output of the machine learning system 508 exceeds a retrieved threshold, the transaction may be rejected. At block 552, the payment processor system 506 issues a response to the merchant system 504, which is received at block 554. At block 554, the merchant system 504 takes steps to prevent the transaction from completing and returns an appropriate response to the POS or user device 502. This response is received at block 556, and the end user or customer may be notified that the end user's or customer's payment has been rejected, for example, via an on-screen “rejected” message. The end user or customer may be prompted to use a different payment method. While not shown in FIG. 5B , in some cases, the issuing bank system 510 may be notified that a transaction related to a particular account holder has been rejected. The issuing bank system 510 may be notified as part of the process shown in Figure 5B or as part of a periodic (e.g., daily) update. The transaction may not become part of the transaction data 146, 240, or 300 (when not approved), but may still be logged by at least the machine learning system 508, as indicated by block 544. For example, with respect to Figure 5A, the transaction data may be stored along with the output of the machine learning system 508 (e.g., scalar fraud or anomaly probability) and the final outcome of the transaction (e.g., that the transaction was rejected).

[0121] FIG. 6 is a block diagram illustrating a machine learning system 600 for processing incoming event data 602 associated with a transaction.

[0122] The system has a database 604 configured to store state data, which is stored in a first memory store, which may (by way of example only) be on any suitable magnetic or solid-state storage device.

[0123] The system uses several event APIs 606 (two are shown in FIG. 6, but there could be more or fewer), each of which includes a state delta generator 608 (see below) configured to generate a state delta 610 with pending updates to the state data for later application. Each state delta generator 608 can retrieve the most recent version of state data 612 from the database 604, as well as a particular input of incoming event data 602.

[0124] Each state delta generator 608 applies one or more machine learning models and / or adaptive rule sets to process the respective incoming event data 602 and the retrieved version of state data 612. The result of this processing is the generation of an event state delta 610 that includes one or more updates to be made to the state data following the most recent operation. The operation of the state delta generators 608 is "synchronous," meaning that they process the event data associated with a transaction "live" as it is received.

[0125] State deltas 610 with pending updates are added to a queue for processing by an updater 614, which generates updated state data 616 and stores it in database 604. The operation of the updater 614 is "asynchronous," meaning that it occurs at a suitable time, but not necessarily live, in response to receiving event data 602. Updates may occur according to some particular schedule, in response to a threshold number of state deltas in a queue, in response to a certain amount of event data being received, in response to a command from an external system, or any other such suitable trigger.

[0126] 7 is a block diagram illustrating a machine learning system 700 for processing incoming event data, according to one embodiment of the present invention. The state delta generator 708 and the updater 714 may be instantiated within the machine learning model platform 414 described above with reference to FIG.

[0127] Unless the context dictates otherwise, it will be appreciated that elements in FIG. 7 having reference numbers beginning with "7" correspond in form and function to those elements in FIG. 6 having corresponding reference numbers beginning with "6."

[0128] For simplicity, FIG. 7 shows only a single state delta generator 708 and a single updater 714, but it will be appreciated that in practice multiple ones of one or both of these components may be used, and they may share the database 704 and cache 718 with those shown in FIG. 7.

[0129] Generally, all EventAPIs may share the same cache so that events involving the same entity may be processed by multiple EventAPIs simultaneously. Similarly, multiple state deltas may be cached simultaneously for the same entity. On the other hand, if events are instead processed serially, the second event will see the state delta update from the first event, even if they are in a different EventAPI (because when the state data is read, the state delta from the first event will be returned from the cache along with the state data from the database).

[0130] State delta-generator 708 runs inside EventAPI 706 (which may be input data interface 412 described above with reference to FIG. 4). However, it will be appreciated that the architecture shown in FIG. 7 is exemplary, and in practice state delta-generator 708 may exist as a daemon separate from that of EventAPI 706, e.g., state delta-generator 708 communicates with EventAPI 706 using remote procedure calls (RPC) or representational state transfer (REST) ​​functions.

[0131] For each event 702, state delta generator 708 reads entity states 712 from database 704 and passes them through a model (e.g., a machine learning model such as those described above) to generate an engine response. State delta generator 708 also calculates the state updates that each model wants to apply to each entity-partition as state deltas 710 and places state deltas 710 on appropriate Kafka® topics, which in this particular example are partitioned by entity & state id. It will be appreciated that the Kafka® topics used to implement the queues may be partitioned by a mix of entity id, entity id and other identifiers, or according to any other suitable identification scheme that allows distinct state data in the database to be pointed to.

[0132] In the configuration of FIG. 7, in addition to state deltas 710 with pending state updates being held in a queue for processing by updater 714, copies of these state deltas 710 are also stored in a separate cache 718.

[0133] Cache 718 is maintained by a second memory store, which is separate from the first memory store that maintains database 704. The second memory store may be (by way of example only) any suitable magnetic or solid-state storage device. Typically, cache 718 is random access, meaning that its various memory locations can be written to and read from in any order, rather than being required to be accessed sequentially.

[0134] If a request for access to state data stored in database 704 (e.g., a request from state delta generator 708 or some external component not shown) is received by system 700, system 700 queries its cache to determine whether there are any state deltas 710 that are applicable to the version of state data 712 that it intends to return from database 704. If such state deltas 710 exist in cache 718, they are returned along with the state data 712 from database 704. This ensures that the recipient has the most up-to-date version of the state data, thereby minimizing and potentially eliminating the state lag problem that would otherwise arise from utilizing state delta-based processing in this manner.

[0135] 7, the only output of the engine is from the state delta generator 708 internal to the EventAPI 706. This means that by default, the state lag is the time it takes for the updater 714 to read a state delta update 710 from the queue and apply it to the database 704 (i.e., to store the updated state data 716), which could potentially be several seconds, for example, depending on the load. However, because all state updates are now made using state deltas 710, pending state deltas 710 that have not yet been added to the database 704 may be stored in a separate state delta cache 718.

[0136] Cache 718 stores state deltas 710 that have not yet been applied to entity state, keyed on the entity-partition and a "generation id" that indicates the version of the state data to which the state delta 710 should be applied. When cache 718 is being used and state delta updates are being placed on a Kafka® queue, state delta generator 708 will also add the state delta 710 to a list in cache 718 for that particular partition-entity and generation that was read. Then, when reading entity state from database 704, state delta generator 708 will also read any pending state deltas 710 from cache 718 that are applicable to the version of the state data read from database 704 and apply them to the database state before passing the state to the model for processing.

[0137] This cache 718 can be implemented either using a separate process such as Redis® or using an integrated part of the Event API 706 .

[0138] When the updater 714 updates the state in the database 704, the updated state data 716 for the entity is written to the database 704 along with the incremented generation id, and then the updater 714 issues a clear command 719 to the cache 718, which clears all entries in the cache 718 for that partition-entity with a lower generation id.

[0139] The approach outlined above can generally result in tens of milliseconds of state lag, determined by the time it takes for generator 708 to cache state delta 710. The increased overhead in managing cache 718 and storing state delta 710 therein comes at the cost of increased resource usage and synchronization latency, but this is favorably traded off for a significant reduction in state lag.

[0140] The use of cache 718 to store copies of state delta 710 can also provide significant leeway in handling failures. In particular, if updater 714 goes down, either due to a failure in the daemon or a failure in another component, the EventAPI 706 node can continue to read and respond to events 702, and the state lag slack will be picked up (up to a point) by state delta cache 718. When updater 714 comes back up, it will catch up with pending updates in the state delta updater queue and will empty state delta cache 718 as it goes.

[0141] In this configuration, a (synchronous) state delta generator 708 and an (asynchronous) updater 714 are coupled together by a partitioned Kafka® topic.

[0142] The updaters 714 run as a set of standalone daemons, implemented as Kafka® consumers, that will read state delta updates 710 from their assigned Kafka® partition, read the state from the database 704, apply the state delta updates 710 to the state, and write it back to the database.

[0143] All of the daemons run independently and do not need to communicate with each other. Instead, they communicate only with Kafka® via the producer & consumer interface, and all distribution is handled by the Kafka® consumer group.

[0144] 8 is a block diagram illustrating a machine learning system 800 that utilizes state deltas for live and batch events. The separation between the state delta generator 808 and the updater 814 enables another optional feature: the state deltas 810, 822 sent to the updater 814 do not necessarily have to come (exclusively) from the state delta generator 808 in the Event API 806.

[0145] Unless the context dictates otherwise, it will be appreciated that elements in FIG. 8 having reference numbers beginning with "8" correspond in form and function to those elements in FIG. 6 having corresponding reference numbers beginning with "6" and / or to elements in FIG. 7 having corresponding reference numbers beginning with "7."

[0146] For simplicity, FIG. 8 shows only a single state delta generator 808, a single updater 814, and a single batch state delta generator 820, but it will be appreciated that in practice multiple instances of one or more of these components may be used.

[0147] The state delta generator 808, the updater 814, and the batch state delta generator 820 may be instantiated within the machine learning model platform 414 described above with reference to FIG. 4.

[0148] The batch state delta generator 820 acts as an additional state delta generator that runs as an independent process for some period of time, processes a different set of events (e.g., batch events), and sends state deltas 822 to the updater 814 for integration into the database state. It runs in parallel with the "live" generator 808, thus allowing batch events to be processed simultaneously with live events.

[0149] Database 804 serves both live state delta generator 808 and batch state delta generator 820, providing state data 812 to each when requested.

[0150] The updater 814 can be sized to handle state delta updates 810, 822 from both the live event state delta generator 808 (which operates on incoming event data) and the batch generator 820 without significant updater lag.

[0151] The batch generator 820 may operate on a snapshot of the database state when the batch starts, or read the state from the database as it progresses. Because batch events can be processed faster than real time, state liveness within the batch generator should be considered.

[0152] Parallel processing can be achieved by using a separate set of batch processes that can either run constantly or be created just for the duration of a batch, and in either case, such batch processes can read batch events and send batch state delta updates 822 for onward processing by one or more updaters 814.

[0153] As previously described with reference to FIG. 7, when the updater 814 updates the state in the database 804, the state data for the entity is written to the database 804 along with the incremented generation id, and then the updater 814 issues a clear command 819 to the cache 818, which clears all entries in the cache 818 for that partition-entity with a lower generation id.

[0154] It will be appreciated that embodiments of the present invention may provide an arrangement in which copies of state deltas with pending updates are maintained in a separate cache, which may advantageously offset the "state liveness" problem associated with state delta-based processing, because when a request to read state data is received, the state data may be supplied along with copies of any applicable state deltas stored in the cache, or the state data may be updated with the applicable state deltas prior to its supply to the entity that requested it.

[0155] Although particular embodiments of the present invention have been described in detail, those skilled in the art will appreciate that the embodiments described in detail are not limitations on the scope of the invention as claimed. [Explanation of symbols]

[0156] 100 Transaction Processing Systems 102, 104 Transaction processing system, variants of transaction processing system 110 client devices 110-A Smartphone 110-B Computer 110-C Point of Sale (POS) System 110-D Portable Merchant Device 120 Computer Networks, Networks 120-A First Computer Network 120-B Second Network 130 Merchant Server 140 Payment Processor Server 142 first data storage device 144 second data storage device 146,240,300,330 transaction data 148, 172-A~172-N, 242, 242-A~242-N Auxiliary Data 150 Machine Learning Servers 160,210,402 Machine Learning Systems 162,410 input data 164,422 output data 170 Local Data Storage Device 170-A~170-N Local Data Storage Devices 172 Ancillary Data 200,202 Data storage configuration 220 Data Bus 230,232,260 Data storage devices 240 Entity Transaction Data 240-A~240-N Entity Transaction Data 250 data transfer channels 270 Secure Logical Storage Layer 280,280-A~280-N classification 300 chronological records, transactional data 310 Current Transaction Data 320 "older" or historical transaction data 400 Example 412 Input Data Interface 414 Machine Learning Model Platform 416 Scalar Output 418 Alarm System 420 Output Data Interface 502 POS or user device 504 Merchant System 506 Payment Processor (PP) System 508,600,800 Machine Learning (ML) Systems 510 Issuing bank system, system 602 Incoming event data, event data 604,704,804 databases 606 Event API 608 State Delta Generator 610 State Delta, Event State Delta 612,812 state data 614,714,814 Updater 616,716 updated state data 700 Machine Learning Systems, Systems 702 Incoming Event Data 706,806 EventAPI 708 State Delta Generator 710 State Delta, State Delta Update, Event State Delta 712 Entity State, State Data 718 cache, state delta cache 719,819 Clear Command 808 State Delta Generator, "Live" Generator, Live State Delta Generator, Live Event State Delta Generator 810 State Delta, State Delta Update 818 Cache 820 Batch State Delta Generator, Batch Generator 822 State Delta, State Delta Update, Batch State Delta Update

Claims

1. 1. A machine learning system for processing incoming event data associated with a transaction, the machine learning system comprising: a) a first memory store comprising a database configured to store state data; b) a state delta generator, i) retrieving a version of said state data from said database; ii) receiving each incoming event data; and iii) processing the respective incoming event data and the retrieved version of the state data using a respective model to generate a respective event state delta comprising a set of pending updates to the state data; iv) storing the event state delta in a queue; and a state delta generator configured to: c) an updater, i) removing the event state delta from the queue; ii) retrieving said status data from said database; iii) applying the pending updates from each event state delta to the state data to generate updated state data; and iv) updating the database with the updated status data; and an updater configured to: d) a second memory store comprising a cache configured to store a copy of each of the event state deltas in the queue having pending updates to the state data, wherein each of the copies of an event state delta is stored in the cache along with a value indicating the version of the state data used to generate the event state delta; and A machine learning system comprising:

2. 2. The machine learning system of claim 1, wherein when the machine learning system receives a request to read a given version of the state data stored in the database, the machine learning system is configured to query the cache to determine whether there are any event state deltas that should be applied to the given version of the state data.

3. 3. The machine learning system of claim 2, wherein when the machine learning system determines that there is an event state delta that should be applied to the given version of the state data, the machine learning system is further configured to supply the determined event state delta along with the given version of the state data.

4. 4. The machine learning system of claim 1, wherein a copy of an event state delta having a value indicating an older version of the state data than that of the state data being updated is configured to be removed from the cache after the updated state data is written to the database.

5. 5. The machine learning system of any one of claims 1 to 4, configured for use in real-time anomaly detection, in particular in transaction processing systems.

6. 6. The machine learning system of claim 1, wherein the incoming event data is transactional data or is derived from transactional data.

7. 7. The machine learning system of claim 6, wherein the transaction data includes electronic data associated with one or more transactions, each transaction including a series of communications between different electronic systems to effect a payment or currency exchange.

8. the machine learning system, a plurality of state delta generators, i) retrieving a version of said state data from said database; ii) receiving each incoming event data; and iii) processing the respective incoming event data and the retrieved version of the state data using a respective model to generate a respective event state delta comprising a set of pending updates to the state data; iv) storing the event state delta in the queue; and a plurality of state delta-generators each configured to: Equipped with 8. The machine learning system of claim 1, wherein a copy of each of the event state deltas generated by each of the plurality of state delta generators is stored in the cache along with a value indicating the version of the state data used to generate the respective event state delta.

9. 9. The machine learning system of claim 1, wherein the model applied by the state delta-generator comprises a machine learning model and / or an adaptive rule set.

10. The updater: receiving one or more batch state deltas comprising a set of pending batch updates to the state data; applying the pending batch updates from each batch state delta to the state data when generating the updated state data; 10. The machine learning system of claim 1, further configured to:

11. The machine learning system further comprises a batch state delta generator, the batch state delta generator comprising: i) generating a respective batch state delta that includes a set of pending batch updates to said state data; ii) storing the batch state delta in the queue; and configured to:

11. The machine learning system of claim 1, wherein the cache is further configured to store a copy of each of the batch state deltas in the queue that have pending batch updates to the state data, and each of the copies of a batch state delta is stored in the cache along with a value indicating the version of the state data to which the batch state delta is applicable.

12. 12. The machine learning system of claim 11, wherein when the machine learning system receives a request to read a given version of the state data stored in the database, the machine learning system is configured to query the cache to determine whether there are any batch state deltas that should be applied to the given version of the state data.

13. 13. The machine learning system of claim 12, wherein when the machine learning system determines that there is a batch state delta that should be applied to the given version of the state data, the machine learning system is configured to supply the determined batch state delta along with the given version of the state data.

14. the machine learning system, A plurality of updaters, i) retrieving one or more of said event state deltas from said queue; ii) retrieving said status data from said database; iii) applying the pending updates from each respective event state delta to the state data to generate updated state data; iv) updating the database with the updated status data; and a plurality of updaters each configured to:

14. The machine learning system of claim 1, comprising:

15. 15. The machine learning system of claim 1, wherein the state data is indexed by identifiers, optionally the identifiers are entity identifiers, and further optionally each entity identifier corresponds to an entity on which real-time anomaly detection is being performed.

16. 16. The machine learning system of claim 1, wherein the cache is random access and / or the second memory store comprises random access memory.

17. 17. The machine learning system of claim 1, wherein the machine learning system comprises an event application programming interface (API) through which event data is received, and optionally the event API includes the state delta-generator.

18. the machine learning system, a state delta combiner configured to combine at least two state deltas before the updated state data is generated.

18. The machine learning system of claim 1, further comprising:

19. 19. The machine learning system of claim 1, wherein the state delta-generator is configured to use a plurality of models when processing the incoming event data, the use of each model generating a respective set of pending updates to the state data, a state delta including each set of pending updates to the state data, the or each state delta being stored in the queue, and the cache is configured to store a copy of the or each state delta.

20. 1. A machine learning system for processing incoming event data associated with a transaction, the machine learning system comprising: a) a database configured to store state data; b) a processing arrangement configured to generate, using the model, one or more event state deltas having pending updates to the state data based on a version of the state data and incoming event data; and c) a cache configured to store a copy of each of the event state deltas having pending updates to the state data, wherein each of the copies of an event state delta is stored in the cache along with a value indicating the version of the state data used to generate the event state delta; and Equipped with d) the machine learning system is i) when the machine learning system receives a request to read a given version of the state data stored in the database, the machine learning system queries the cache to determine whether there are any event state deltas applicable to the given version of the state data; ii) when the machine learning system determines that there is an event state delta that is applicable to the given version of the state data, the machine learning system provides the determined event state delta along with the given version of the state data. A machine learning system configured to:

21. 1. A machine learning system for processing incoming event data associated with a transaction, the machine learning system comprising: a) a first memory store comprising a database configured to store state data, the state data for use with a model configured to receive the incoming event data and generate one or more event state deltas with pending updates to the state data; b) a second memory store comprising a cache configured to store copies of one or more event state deltas having pending updates to the state data received from the model, each copy of an event state delta being stored in the cache along with a value indicating the version of the state data used to generate the event state delta; and A machine learning system comprising:

22. 1. A machine learning system for processing incoming event data associated with a transaction, the machine learning system comprising: a) a first memory store comprising a database configured to store state data; b) at least one state delta generator, i) retrieving a version of said state data from said database; ii) receiving each incoming event data; and iii) processing the respective incoming event data and the retrieved version of the state data using a respective model to generate a respective event state delta comprising a set of pending updates to the state data; iv) storing the event state delta in a queue; and at least one state delta generator configured to: c) an updater, i) removing the event state delta from the queue; ii) retrieving said status data from said database; iii) applying the pending updates from each event state delta to the state data to generate updated state data; and iv) updating the database with the updated status data; and The updater and Equipped with 10. A machine learning system, wherein the machine learning system is configured such that all updates to the state data associated with incoming event data are made by applying one or more event state deltas.

23. A cache including copies of one or more event state deltas having pending updates to state data for use with a model, each copy of an event state delta being stored in the cache along with a value indicating the version of the state data used by the model to generate the event state delta.

24. 1. A machine learning method for processing incoming event data associated with a transaction, the machine learning method comprising: a) storing state data in a database in a first memory store; b) a state delta generation step, i) retrieving a version of said state data from said database; ii) receiving each incoming event data; and iii) processing the respective incoming event data and the retrieved version of the state data using a respective model to generate a respective event state delta comprising a set of pending updates to the state data; iv) storing the event state delta in a queue; and a state delta generation step including: c) an updating step, i) removing the event state delta from the queue; ii) retrieving said status data from said database; iii) applying the pending updates from each event state delta to the state data to generate updated state data; iv) updating the database with the updated status data; and an update step, including: d) storing a copy of each of the event state deltas in the queue with pending updates to the state data in a second memory store comprising a cache, wherein each of the copies of an event state delta is stored in the cache along with a value indicating the version of the state data used to generate the event state delta; machine learning methods, including

25. receiving a request to read a given version of the state data stored in the database; querying the cache to determine whether there are any event state deltas applicable to the given version of the state data; 25. The machine learning method of claim 24, further comprising:

26. providing, when there is an event state delta that is applicable to the given version of the state data, the determined event state delta along with the given version of the state data.

26. The machine learning method of claim 25, further comprising:

27. removing from the cache a copy of an event state delta having a value indicating a version of the state data that is older than that of the state data being updated after the updated state data has been written to the database.

26. The machine learning method of claim 25, further comprising:

28. 28. The machine learning method of claim 24, wherein the incoming event data includes or is derived from transaction data.

29. 30. The machine learning method of claim 28, wherein the transaction data includes electronic data associated with one or more transactions, each transaction including a respective series of communications between different electronic systems to effect a payment or currency exchange.

30. 30. The machine learning method of claim 28, wherein the model applied by the state delta-generator comprises a machine learning model and / or an adaptive rule set.

31. receiving one or more batch state deltas comprising a set of pending batch updates to the state data; applying the pending batch updates from each batch state delta to the state data when generating the updated state data; 31. The machine learning method of any one of claims 24 to 30, further comprising:

32. i) generating a respective batch state delta that includes a set of pending batch updates to said state data; ii) storing the batch state delta in the queue; iii) storing in the cache a copy of each of the batch state deltas in the queue having pending batch updates to the state data, each of the copies of a batch state delta being stored in the cache along with a value indicating the version of the state data to which the batch state delta is applicable; 31. The machine learning method of any one of claims 24 to 30, further comprising:

33. receiving a request to read a given version of the state data stored in the database; querying the cache to determine whether there are any batch state deltas applicable to the given version of the state data; 33. The machine learning method of claim 32, further comprising:

34. providing, when there is a batch state delta that is applicable to the given version of the state data, the determined batch state delta along with the given version of the state data.

34. The machine learning method of claim 33, further comprising:

35. combining at least two state deltas prior to generating the updated state data.

34. The machine learning method of claim 33, further comprising:

36. using a plurality of models when processing the incoming event data, the use of each model generating a respective set of pending updates to the state data, a state delta including each set of pending updates to the state data; storing the or each state delta in the queue; storing a copy of the or each state delta in the cache; 36. The machine learning method of any one of claims 24 to 35, comprising:

37. 1. A machine learning method for processing incoming event data associated with a transaction, the machine learning method comprising: a) storing state data in a database; b) using a model to generate one or more event state deltas having pending updates to the state data based on a version of the state data and incoming event data; c) storing in a cache a copy of each of the event state deltas having pending updates to the state data, each of the copies of an event state delta being stored in the cache along with a value indicating the version of the state data used to generate the event state delta; d) receiving a request to read a given version of said state data stored in said database; e) querying the cache to determine whether there are any event state deltas applicable to the given version of the state data; f) if there are event state deltas applicable to the given version of the state data, providing the determined event state deltas along with the given version of the state data; machine learning methods, including

38. 1. A machine learning method for processing incoming event data associated with a transaction, the machine learning method comprising: a) storing state data in a database in a first memory store, the state data for use with a model configured to receive the incoming event data and generate one or more event state deltas with pending updates to the state data; b) storing in a second memory store comprising a cache copies of one or more event state deltas having pending updates to the state data received from the model, each copy of an event state delta being stored in the cache along with a value indicating the version of the state data used to generate the event state delta; machine learning methods, including

39. 1. A machine learning method for processing incoming event data associated with a transaction, the machine learning method comprising: a) storing state data in a database in a first memory store; b) a state delta generation step, i) retrieving a version of said state data from said database; ii) receiving each incoming event data; and iii) processing the respective incoming event data and the retrieved version of the state data using a respective model to generate a respective event state delta comprising a set of pending updates to the state data; iv) storing the event state delta in a queue; and a state delta generation step including: c) an updating step, i) removing the event state delta from the queue; ii) retrieving said status data from said database; iii) applying the pending updates from each event state delta to the state data to generate updated state data; and iv) updating the database with the updated status data; and update step, including Including, 1. A machine learning method, wherein all updates to said state data associated with incoming event data are made by applying one or more event state deltas.

40. 1. A method of operating a cache, comprising:

1. A method of operating a cache, comprising: storing copies of one or more event state deltas having pending updates to state data for use with a model, each copy of an event state delta being stored in the cache along with a value indicating the version of the state data used by the model to generate the event state delta.

41. 41. A non-transitory computer readable medium or computer software product comprising instructions that, when executed by a processor, cause the processor to perform the method of any one of claims 24 to 40.

Images