Access Control

Abstract

A method for controlling access to machine learning model data in a multi-tenant machine learning system having a machine learning model data access hierarchy 700. The hierarchy 700 includes (i) a tenant level 602 with first and second tenants 606, 608 associated with first and second tenant machine learning model data 610, 612, respectively, and (ii) a tenant group level 604 with a tenant group 614 to which the first and second tenants 606, 608 belong. The tenant group 614 is associated with tenant group machine learning model data 620. The tenant group machine learning model data 620 is based on the first and / or second tenant machine learning model data 610, 612. The method includes (i) allowing 724, 726 access to the tenant group machine learning model data 620 and the first tenant machine learning model data 610, and (ii) prohibiting 728 access to the second tenant machine learning model data 612, for a machine learning model 722 applied for the first tenant 606 at the tenant group level 604.

Description

[Technical Field]

[0001] The present disclosure relates to access control.

[0002] In particular, but not exclusively, the present disclosure relates to a method for controlling access to machine learning model data in a multi-tenant machine learning system having a machine learning model data access hierarchy, and to the multi-tenant machine learning system.

[0003] In embodiments, the methods and systems are used to detect anomalies in patterns of data, for example to detect fraudulent transactions. Particular examples relate to real-time transaction processing methods and systems. [Background technology]

[0004] Digital payments have exploded over the past two decades, with over three-quarters of global payments using some form of payment card or electronic wallet. Point-of-sale systems are gradually becoming less cash-based and more digital. Simply put, global commerce systems now rely heavily on electronic data processing platforms. This presents many engineering challenges that are largely hidden from the average user. For example, digital transactions must be completed in real time, i.e., with minimal latency experienced by the computing device at the point of purchase. Digital transactions must also be secure and resistant to attacks and exploitation. Digital transaction processing is also constrained by the historical development of global electronic systems for payments. For example, much infrastructure is still structured around a model designed for mainframe architectures that were in use more than 50 years ago.

[0005] As digital transactions increase, new security risks also become apparent. Digital transactions present new opportunities for fraud and malicious activity. In 2015, it was estimated that 7% of digital transactions were fraudulent, and that number has only increased as more economic activity has moved online. Fraud losses are estimated to be four times the size of the world's population (e.g., in US dollars) and growing.

[0006] Financial services institutions are coming under increased regulatory scrutiny as traditional fraud prevention methods, such as identity authentication (e.g., passwords, digital biometrics, national IDs, etc.), have proven ineffective in preventing fraud vectors such as synthetic identities and fraud. These much more complex threat vectors for fraud require much more analysis in very short periods of time (less than 50 milliseconds (ms)) and are often based on much smaller data sampling sizes than the fraud or fraud itself. This poses significant technical challenges.

[0007] While risks like fraud are an economic issue for businesses involved in commerce, implementing technology systems to process transactions is an engineering challenge. Traditionally, banks, merchants, and card issuers developed "paper" rules or procedures that were manually implemented by store clerks to flag or block certain transactions. As transactions have become digital, one approach to building technology systems to process transactions has been to provide these developed sets of standards to computer engineers and ask them to implement them using a digital representation of the transaction—that is, to translate the handwritten rules into coded logic statements that can be applied to electronic transaction data. This traditional approach has encountered several problems as digital transaction volumes have grown. First, any applied processing must occur in "real time," e.g., with millisecond latency. Second, thousands of transactions must be processed per second (e.g., a typical "load" can be 1,000–2,000 per second), and the load varies unpredictably over time (e.g., the launch of a new product or set of tickets can easily increase average load levels several-fold). Third, transaction processors' and banks' digital storage systems are often siloed or partitioned for security reasons, while digital transactions often involve an interconnected web of merchant systems. Fourth, large-scale analysis of actual reported and predicted fraud is now possible, demonstrating that traditional approaches to fraud detection are inadequate, with low accuracy and high false positives. This, in turn, has a physical impact on digital transaction processing, with more genuine in-store and online purchases being rejected and those attempting to use the new digital systems often escaping.

[0008] In recent years, more and more machine learning approaches have been applied to transactional data processing. As machine learning models mature in academia, engineers have begun to attempt to apply them to transactional data processing. However, this also presents challenges. Even when engineers are provided with an academic or theoretical machine learning model and asked to implement it, this is not easy. For example, problems arise in large-scale transaction processing systems. Machine learning models do not have the luxury of unlimited inference time as in a laboratory. This means that implementing a particular model in a real-time setting is simply impractical or requires significant adaptation to enable real-time processing at volume levels experienced by real-world servers. Furthermore, engineers must address the problem of implementing machine learning models on data that is siloed or partitioned based on access security and in situations where the rate of data update is extreme. Thus, the problems faced by engineers building transactional processing systems can be considered similar to those faced by network or database engineers. Machine learning models need to be applied, but meet the system throughput and query response time constraints set by the processing infrastructure. There are no easy solutions to these problems. Indeed, the fact that many transaction processing systems are confidential, proprietary, and based on older technologies means that engineers lack the body of knowledge developed in these adjacent fields and often face challenges unique to the domain of transaction processing. Furthermore, the field of large-scale, practical machine learning is still young, with few established design patterns or textbooks on which engineers can rely.

[0009] Fraud detection systems generate risk scores for new transactions in real time (i.e., latency <= 100 milliseconds), with a high risk score indicating the likelihood of fraud, error, or unauthorized use of an account. Financial institutions use these scores in their decision logic when deciding whether to approve a transaction (typically, this is to reject the transaction if the risk score is higher than a certain threshold). Financial institutions generally want the set of transactions above a certain risk threshold to contain the highest possible percentage of fraudulent (i.e., provide a "maximum detection rate") along with the smallest possible percentage of false positive classifications (i.e., provide a "maximum accuracy").

[0010] One approach, known per se in the art, involves using machine learning techniques, such as recurrent neural networks (RNNs), to detect anomalies in patterns of behavior. Those skilled in the art will understand that the output of an RNN is a next state, which depends on the previous state and the input provided to the RNN. In this way, the state of the RNN is based not only on moment-to-moment information about a given transaction, but also on the history of transactions that came before, and therefore maintains a memory of observed patterns. This can be used to detect anomalous behavior that may be indicative of fraudulent transactions. The terms "behavior" and "behavioral" are used herein to refer to a pattern of activity or action.

[0011] This detection of anomalous activity may provide an improvement to the security of the system compared to "traditional" methods of fraud prevention (e.g., passwords, digital biometrics, national IDs, etc.) It will of course be understood that behavioral pattern analysis techniques may be used in combination with one or more of these "traditional" methods, if desired.

[0012] In general, a rule may require that historical data (eg, state data) about a particular entity be stored, for example, for a particular duration.

[0013] Systems that utilize state data stored in and retrieved from a database (e.g., machine learning systems using RNNs) generally require that the data stored in that database can be updated in response to an event that occurs, e.g., a new transaction in a transaction processing system. A transaction or deposit that occurs on a card or merchant typically results in one or more events being sent to the system.

[0014] Typically, the state data may be stored in a database using any suitable structured numerical representation, for example as a table of vectors or tensors.

[0015] In practice, tenants (e.g., direct or indirect customers of a fraud detection platform) may want one platform to process multiple different types of data, not only to simplify deployment but also to harmonize third-party components across their organizations and minimize running license costs.

[0016] One way to deploy such a system is to have separate installations running on the same software, with namespaces and external connections that make it appear to the outside world as one system.

[0017] However, this can lead to a complex final solution with multiple components all interacting in different ways. If you want to share data between different installations, combining such data into a coherent view can be challenging when trying to get these separate pieces of data to work together. Summary of the Invention [Means for solving the problem]

[0018] According to a first aspect, there is provided a method for controlling access to machine learning model data in a multi-tenant machine learning system having a machine learning model data access hierarchy, the machine learning model data access hierarchy comprising: a tenant level comprising first and second tenants associated with first and second tenant machine learning model data, respectively; a tenant group level comprising a tenant group to which a first and a second tenant belong, the tenant group being associated with tenant group machine learning model data, the tenant group machine learning model data being based on the first and / or second tenant machine learning model data; The method relates to a machine learning model being applied with respect to a first tenant at a tenant group level, allowing access to the tenant group machine learning model data and the first tenant machine learning model data; prohibiting access to the second tenant machine learning model data.

[0019] Multi-tenant machine learning systems allow a single system to be deployed that supports multiple tenants, which can significantly improve deployment and maintenance efficiency compared to separate installations for separate tenants and / or separate types of data.

[0020] The machine learning model data access hierarchy again provides a computationally efficient mechanism for separating first and second tenant machine learning model data (e.g., with respect to direct tenant access) within a single system, as compared to, for example, having separate installations for separate tenants. The separation can be particularly effective for machine learning model data. Additionally, the computational efficiency gains are effective in the context of machine learning, where relatively large amounts of machine learning model data may be used.

[0021] The tenant group machine learning model data may be based on the first and second tenant machine learning model data, for example, if the first and second tenants have been in the tenant group for an extended period of time.

[0022] Alternatively, the tenant group machine learning model data may be based on only one of the first and second tenant machine learning model data, for example, if one of the first and second tenants recently joined the tenant group.

[0023] In either case, the tenant group machine learning model data provides abstracted data regarding the first and second tenant machine learning model data, which may be used, for example, for abstracted analysis.

[0024] In some examples, the tenant group machine learning model data is based on first tenant machine learning model data, and the tenant group machine learning model data is not based on second tenant machine learning model data or is based on a restricted version of the second tenant machine learning model data.

[0025] In some instances, the second tenant joined the tenant group after the first tenant joined the tenant group.

[0026] In such an example, the second tenant machine learning model data does not contribute, or only minimally contributes, to the tenant group machine learning model data. This may be particularly useful if the second tenant recently joined the tenant group, for example, from another tenant group. The contribution of the second tenant machine learning model data to the tenant group machine learning model data may be lower than the first tenant machine learning model data to the tenant group machine learning model data, which is consistent with the amount of time the first and second tenants have belonged to the tenant group.

[0027] In some cases, the longer a given tenant belongs to a given tenant group, the stronger the characteristic match between the given tenant and the given tenant group, and therefore the greater the influence of the given tenant machine learning model data associated with the given tenant on the given tenant group machine learning model data associated with the given tenant group.

[0028] In some examples, the tenant group machine learning model data is based on a historical set of at least a first tenant machine learning model data, and the tenant group machine learning model data is not based on a historical set of at least a second tenant machine learning model data or is based on a limited version of the historical set of the second tenant machine learning model data.

[0029] In some examples, the historical set of second tenant machine learning model data was generated before the second tenant joined the tenant group.

[0030] In such an example, the historical set of second tenant machine learning model data may be less representative of the current state associated with the second tenant than the more recent set of second tenant machine learning model data, which may limit the contribution of this less representative, historical second tenant machine learning model data to the tenant group machine learning model data, which may in turn result in more accurate tenant group machine learning model data.

[0031] In some examples, the tenant group machine learning model data is updated to generate updated tenant group machine learning model data, where the updated tenant group machine learning model data is based on both the first and second tenant machine learning model data.

[0032] In such an example, the influence of the second tenant machine learning model data on the tenant group machine learning model data may, for example, increase over time while the second tenant remains part of the tenant group. Such contribution may not be limited.

[0033] In some examples, the tenant group machine learning model data is based on at least some of each of the first and second machine learning model data.

[0034] Thus, the tenant group machine learning model data is based, at least to some extent, on both the first and second machine learning model data, which may be particularly useful when the first and second tenants have been in the tenant group for an extended period of time.

[0035] In some examples, the tenant group machine learning model data is derived based on one or more operations involving at least some of each of the first and second machine learning model data.

[0036] In such examples, some degree of combination and / or aggregation may be performed to provide a higher level of abstraction of the first and second machine learning model data, which may, for example, facilitate a higher level of analysis of the first and second machine learning model data.

[0037] In some examples, the method includes allowing access to tenant group machine learning model data and second tenant machine learning model data and prohibiting access to the first tenant machine learning model data for a machine learning model applied for a second tenant at the tenant group level.

[0038] In such an example, the first and second tenants may benefit from the machine learning model applications described herein while strong data separation and isolation is maintained between the first and second tenants.

[0039] In some examples, the method includes prohibiting access to tenant group machine learning model data for machine learning models that have been applied with respect to the first and / or second tenant at the tenant level.

[0040] In such an example, access control can prevent tenant-level models from accessing tenant group machine learning model data, thereby improving data isolation between a first tenant and a second tenant.

[0041] In some examples, the method includes selectively allowing access to tenant group machine learning model data for machine learning models applied with respect to the first and / or second tenant at a tenant level based on one or more permission settings enabled for the machine learning models applied with respect to the first and / or second tenant at the tenant level.

[0042] In such instances, access to tenant group machine learning model data can be selectively enabled, for example, by enabling associated permissions, which can improve anomaly detection, for example, by having a more holistic view of behavior in the hierarchy.

[0043] In some examples, the machine learning model data access hierarchy includes an administrator level to which the machine learning model is applicable for a first tenant, the administrator level includes an administrator associated with the administrator machine learning model data, and the tenant group belongs to the administrator.

[0044] In some examples, the method includes allowing access to administrator machine learning model data, tenant group machine learning model data, and first tenant machine learning model data for a machine learning model applied for a first tenant at an administrator level, and prohibiting access to a second tenant machine learning model data.

[0045] In some examples, further levels of control, analysis, and / or hierarchy may be provided.

[0046] In some examples, the tenant group level includes a further tenant group to which at least one further tenant belongs and to which the first and second tenants do not belong, the further tenant group belongs to an administrator, the further tenant group is associated with further tenant group machine learning model data, and the at least one further tenant is associated with further tenant machine learning model data.

[0047] In such an example, different abstractions in the form of tenant groups may be available within the hierarchy, and the different abstractions may be associated with different tenant group characteristics and / or different tenant characteristics.

[0048] In some examples, the method includes prohibiting access to further tenant group machine learning model data and / or further tenant machine learning model data for the machine learning model being applied with respect to the first tenant at the tenant group level.

[0049] In some examples, the method includes prohibiting access to further tenant group machine learning model data and / or further tenant machine learning model data for the machine learning model being applied for the first tenant at an administrator level.

[0050] In such instances, a more flexible hierarchy is provided while still providing strong data separation.

[0051] In some examples, the machine learning model data access hierarchy includes an additional tenant associated with the additional tenant machine learning model data, the additional tenant belonging to an administrator.

[0052] In such an example, the additional tenant may not exhibit strong matching characteristics with the tenant group and / or another tenant and may belong directly to the administrator. Requiring all tenants to belong to a tenant group may lead to reduced accuracy of the machine learning model data in that, when a given tenant joins a given tenant group, a given additional tenant that does not exhibit strong matching characteristics with the given tenant group may skew the given tenant group machine learning model data associated with the given tenant group.

[0053] In some examples, the method includes prohibiting access to additional tenant machine learning model data for a machine learning model being applied with respect to the first tenant at a tenant group level.

[0054] In some examples, the method includes prohibiting access to additional tenant machine learning model data for a machine learning mode applied for the first tenant at an administrator level.

[0055] In such instances, a more flexible hierarchy is provided while still providing strong data separation.

[0056] In some examples, the method includes performing a predetermined action with respect to the tenant group machine learning model data in response to the first and / or second tenant no longer belonging to the tenant group.

[0057] In such an example, when the first and / or second tenant leaves the tenant group, certain actions may be taken, such as immediately deleting the machine learning model data associated with the departing tenant, or reducing the data immediately or over time.

[0058] In some examples, the machine learning model data includes machine learning model state data and / or machine learning model parameter data.

[0059] In such instances, the machine learning model data access hierarchy provides particularly effective access control for machine learning data, which typically has large amounts of data. Efficient resource usage in such a context can be particularly beneficial.

[0060] In some examples, the tenant group machine learning model data and / or the first tenant machine learning model data are used to perform real-time anomaly detection associated with the first tenant.

[0061] In such instances involving real-time detection, the efficiencies provided by the machine learning model data access hierarchy are particularly effective.

[0062] According to a second aspect, there is provided a method for controlling access to learning model data in a multi-tenant system having a learning model data access hierarchy, the learning model data access hierarchy comprising: a tenant level comprising first and second tenants associated with first and second tenant learning model data, respectively; a tenant group level comprising a tenant group to which at least a first tenant belongs, the tenant group being associated with tenant group learning model data, and the tenant group learning model data being based on at least the first tenant learning model data; The method includes, with respect to a machine learning model being applied with respect to a first tenant at a tenant group level, and / or with respect to a learning model being applied with respect to a first tenant at a tenant level, Allowing access to the tenant group learning model data and the first tenant learning model data; prohibiting access to the second tenant learning model data.

[0063] Thus, greater applicability, functionality, and efficiency may be provided with respect to the other exemplary methods, systems, and hierarchies described herein.

[0064] The learning model may comprise a machine learning model and / or another type of learning model described herein.

[0065] According to a third aspect, there is provided a method for controlling access to machine learning model data in a multi-tenant machine learning system having a machine learning model data access hierarchy, the machine learning model data access hierarchy comprising: a first tenant associated with the first tenant machine learning model data; a second tenant associated with second tenant machine learning model data; and a tenant group associated with tenant group machine learning model data, wherein the method comprises: At the first time, The first tenant belongs to a tenant group, and the second tenant does not belong to a tenant group; The tenant group machine learning model data is based on the first tenant machine learning model data, and the tenant group machine learning model data is not based on the second tenant machine learning model data; At a second later time, the first and second tenants belong to a tenant group, and the tenant group machine learning model data is based on at least the first tenant machine learning model data; At a time after the third, A method is provided that includes controlling access to machine learning model data such that the tenant group machine learning model data is based on first and second tenant machine learning model data.

[0066] Thus, delayed or staggered use of tenant machine learning model data may be provided, which may allow for accurate tenant group machine learning model data according to the characteristics of different tenants at different times.

[0067] According to a fourth aspect, there is provided a multi-tenant machine learning system configured to perform the method provided according to the first, second and / or third embodiment.

[0068] According to a fifth aspect, there is provided a multi-tenant machine learning system having access to a machine learning model data access hierarchy comprising a plurality of tenants and one or more tenant groups, the multi-tenant machine learning system comprising a machine learning model data access controller configured to control access to machine learning model data in accordance with the machine learning model data access hierarchy.

[0069] Such examples leverage the efficiency of machine learning model data access hierarchies in the context of machine learning data.

[0070] Embodiments of the present invention may be applied to a wide variety of digital transactions, including, but not limited to, card payments, so-called "wire" transfers, peer-to-peer payments, Bankers' Automated Clearing System (BACS) payments, and Automated Clearing House (ACH) payments. The output of machine learning systems may be used to prevent a wide variety of fraud and criminal activity, such as card fraud, application fraud, payment fraud, merchant fraud, game fraud, and money laundering.

[0071] It will be understood that optional features discussed above in relation to embodiments of any aspect of the invention apply equally to other aspects of the invention, where technically appropriate.

[0072] Embodiments of the invention may be combined where technically appropriate. Embodiments are described herein as comprising particular features / elements. The disclosure also extends to separate embodiments consisting of, or consisting essentially of, those features / elements.

[0073] The method or methods according to any embodiment of any aspect of the present invention may be computer-implemented.

[0074] Each of the aspects of the invention described above extends to a non-transitory computer-readable medium comprising instructions that, when executed by a processor, cause the processor to perform the method of such aspect of the invention.

[0075] Each of the aspects of the invention described above further extends to a computer software product comprising instructions which, when executed by a processor, cause the processor to perform the method of such aspect of the invention.

[0076] Technical references, such as patents and applications, are incorporated herein by reference.

[0077] Any embodiment specifically and explicitly described herein may form the basis of a waiver, either alone or in combination with one or more additional embodiments.

[0078] In the context of this specification, "comprising" should be interpreted as "including".

[0079] Aspects of the invention comprising particular elements are also intended to extend to alternative embodiments "consisting of" or "consisting essentially of" the associated elements.

[0080] The term "memory" or "memory store" should be understood to mean any means suitable for storing data, and includes both volatile and non-volatile memory appropriate for the intended application. Those skilled in the art will understand that this includes, but is not limited to, random access memory (RAM), static RAM (SRAM), dynamic RAM (DRAM), magnetic storage, solid-state storage, and flash memory. It will be understood that combinations of one or more of these may also be used for data storage, as technically appropriate (e.g., using faster-access volatile memory for frequently accessed data).

[0081] The term "data" is used in various contexts herein to refer to digital information, such as that represented by known bit structures in one or more programming languages. In use, data can refer to digital information stored as a bit sequence in computer memory. Certain machine learning models may operate on structured arrays of data in a predefined bit format. Those skilled in the art will readily understand that these may be referred to as arrays, multidimensional arrays, matrices, vectors, tensors, or other such similar terms. Note that for machine learning methods, for example, multidimensional arrays or tensors having defined ranges in multiple dimensions may be "flattened" to be represented (e.g., in memory) as a sequence or vector of values ​​stored according to a predetermined format (e.g., n-bit integers or floating-point numbers, signed or unsigned). Thus, the term "tensor" as used herein covers multidimensional arrays (e.g., vectors, matrices, volumetric arrays, etc.) having one or more dimensions.

[0082] The principles of the present invention apply regardless of the particular data format selected. Data may be represented as an array, vector, tensor, or any other suitable format. For ease of reference, these terms are used interchangeably herein, and references to a "vector" or "vectors" of values ​​should be understood to extend to any n-dimensional tensor or tensors of values, as appropriate. Similarly, references to a "tensor" or "tensors" of values ​​should be understood to extend to a vector, and a vector will be understood by those skilled in the art to simply be a one-dimensional tensor. The principles of the present invention can be applied regardless of the formatting of the data structure used for arrays of these values. For example, state data may be stored in memory as a one-dimensional tensor (i.e., a vector) or as a tensor having two or more dimensions (i.e., a tensor), and those skilled in the art will readily understand that they can make appropriate modifications to data processing elements to process the selected data format. The relative positions of various state values, e.g., how they are ordered within a vector or tensor, are generally not important, and the scope of the present invention is not limited to any particular data format or structure.

[0083] The term "structured numeric representation" is used to refer to numerical data in a structured format, such as a one- or multi-dimensional array that stores numerical values ​​having a common data type, such as integers or float values. Structured numeric representations may include vectors or tensors (as used within machine learning terminology). Structured numeric representations are typically stored as an indexed and / or contiguous set of memory locations; for example, a one-dimensional array of 64-bit floats may be represented in computer memory as a contiguous sequence of 64-bit memory locations in a 64-bit computing system.

[0084] The term "transaction data" is used herein to refer to electronic data associated with a transaction. A transaction includes a series of communications between different electronic systems to effect a payment or exchange. Generally, transaction data may include data that indicates events (e.g., actions taken in time) related to and that may be beneficial for transaction processing. Transaction data may include structured data, unstructured data, and semi-structured data, or any combination thereof. Transaction data may also include data associated with a transaction, such as data used to process the transaction. In certain cases, transaction data may be used broadly to refer to actions taken with respect to one or more electronic devices. Transaction data can take various forms depending on the exact implementation. However, different data types and formats may be converted by pre- or post-processing, as needed.

[0085] The term “interface” is used herein to refer to any physical and / or logical interface that allows one or more of data input and data output. An interface may be implemented by a network interface adapted to send and / or receive data or by retrieving data from one or more memory locations, as implemented by a processor executing a set of instructions. An interface may also comprise a physical (network) connection through which data is received, such as hardware enabling wired or wireless communication over a particular medium. An interface may comprise an application programming interface and / or a method call or return. For example, in a software implementation, an interface may include passing data and / or memory references to a function initiated via a method call, where the function comprises computer program code executed by one or more processors, while in a hardware implementation, an interface may include a wired interconnection between different chips, chipsets, or portions of a chip. In drawings, an interface may be indicated by a processing block boundary with inward and / or outward arrows representing data transfer.

[0086] The terms "component" and "module" are used interchangeably to refer to either a hardware structure having a specific function (e.g., a way of mapping input data to output data) or a combination of general-purpose hardware and specific software (e.g., specific computer program code running on one or more general-purpose processors). A component or module may be implemented as a specific packaged chipset, e.g., an application-specific integrated circuit (ASIC) or a programmed field-programmable gate array (FPGA), and / or as a software object, class, class instance, script, code portion, etc., that is executed in use by a processor.

[0087] The term "machine learning model" is used herein to refer to at least a hardware-implemented implementation of a machine learning model or function. Known models in the field of machine learning include logistic regression models, naive Bayes models, random forests, support vector machines, and artificial neural networks. Implementations of classifiers may be provided in one or more machine learning programming libraries, including, but not limited to, scikit-learn, TensorFlow, and PyTorch.

[0088] The term "map" is used herein to refer to the transformation or conversion of a first set of data values ​​to a second set of data values. The two sets of data values ​​may be arrays of different sizes, with the output array having a lower dimensionality than the input array. The input and output arrays may have common or different data types. In certain examples, the mapping is a one-way mapping to a scalar value.

[0089] The term "neural network architecture" refers to a set of one or more artificial neural networks configured to perform a specific data processing task. For example, a "neural network architecture" may comprise a specific arrangement of one or more neural network layers of one or more neural network types. Neural network types include convolutional neural networks, recurrent neural networks, and feedforward neural networks. Convolutional neural networks involve the application of one or more convolution operations. Recurrent neural networks involve internal states that are updated over a series of inputs. Thus, recurrent neural networks can be viewed as including a form of recurrent or feedback connection, whereby the state of a recurrent neural network at a given time or iteration (e.g., t) is updated using the state of the recurrent neural network at the previous time or iteration (e.g., t-1). Feedforward neural networks involve transformation operations without feedback; e.g., operations are applied in a unidirectional sequence from input to output. Feedforward neural networks include plain "neural networks" and "fully connected" neural networks. Those skilled in the art will understand that "multilayer perceptron" is a term used to describe a fully connected layer, and is a special case of a feedforward neural network.

[0090] The term "deep" neural network is used to indicate that the neural network comprises multiple neural network layers in series (note that the term "deep" is used in conjunction with both feedforward and recurrent neural networks). The particular examples described herein utilize recurrent and fully connected neural networks.

[0091] A "neural network layer," as typically defined within machine learning programming tools and libraries, can be considered an operation that maps input data to output data. A "neural network layer" can apply one or more parameters, such as weights, to map input data to output data. One or more bias terms can also be applied. The weights and biases of a neural network layer can be applied using one or more multidimensional arrays or matrices. Generally, a neural network layer has multiple parameters, the values ​​of which affect how the layer maps input data to output data. These parameters can be trained in a supervised manner by optimizing an objective function, which typically involves minimizing a loss function. Certain parameters can also be pre-trained in other ways or fixed. Fixed parameters can be considered configuration data that control the operation of the neural network layer. A neural network layer or architecture can comprise a mixture of fixed and learnable parameters. A recurrent neural network layer can apply a series of operations to update a recurrent state and transform input data. The update of the recurrent state and the transformation of the input data may involve one or more transformations of the previous recurrent state and the input data. A recurrent neural network layer may be trained by deploying modeled recurrent units so that it can be applied within machine learning programming tools and libraries. Although a recurrent neural network may appear to have several (sub)layers to apply different gating operations, most machine learning programming tools and libraries refer to the application of a recurrent neural network as a whole as a "neural network layer," and this convention will be followed herein. Finally, a feedforward neural network layer may apply one or more sets of weights and biases to the input data to generate output data. This operation may be expressed as a matrix operation (e.g., a bias term may be included by adding a value of 1 to the input data).Alternatively, the bias may be applied through a separate addition operation. As mentioned above, the term "tensor" is used to refer to an array that may have multiple dimensions according to the machine learning library; for example, a tensor may include a vector, a matrix, or a higher-dimensional data structure. In a preferred example, the described tensor may include a vector having a predetermined number of elements.

[0092] To model complex nonlinear functions, neural network layers such as those described above may be followed by nonlinear activation functions. Common activation functions include the sigmoid function, the tanh function, and the rectified linear unit (RELU). Many other activation functions exist and can be applied. The activation function may be selected based on testing and preference. The activation function may be omitted in certain situations and / or may form part of the internal structure of the neural network layer.

[0093] The exemplary neural network architectures described herein may be configured through training. In some cases, “learnable” or “trainable” parameters may be trained using an approach called backpropagation. During backpropagation, the neural network layers that make up each neural network architecture are initialized (e.g., with randomized weights) and then used to make predictions using a set of input data from a training set (e.g., a so-called “forward” pass). The predictions are used to evaluate a loss function. For example, “ground truth” outputs may be compared to predicted outputs, and the difference may form part of the loss function. In one example, the loss function may be based on the absolute difference between a predicted scalar value and a binary ground truth label. The training set may include a set of transactions. When gradient descent is used, the loss function is used to determine the gradient of the loss function with respect to the parameters of the neural network architecture, and the gradient is then used to backpropagate updates to the parameter values ​​of the neural network architecture. Typically, updates are propagated according to the derivatives of the neural network layer weights. For example, the gradient of the loss function with respect to the weights of the neural network layers may be determined and used to determine updates to the weights that minimize the loss function. In this case, optimization techniques such as gradient descent, stochastic gradient descent, Adam, etc. may be used to adjust the weights. Chain rules and automatic differentiation functions may be applied to efficiently calculate the gradient of the loss function, which in turn works its way back through the neural network layers. [Brief explanation of the drawings]

[0094] Specific embodiments of the present invention will now be described with reference to the accompanying drawings.

[0095] [Figure 1A] 1A-1C are schematic diagrams illustrating different exemplary electronic infrastructures for transaction processing. [Figure 1B]1A-1C are schematic diagrams illustrating different exemplary electronic infrastructures for transaction processing. [Figure 1C] 1A-1C are schematic diagrams illustrating different exemplary electronic infrastructures for transaction processing. [Figure 2A] 1A-1C are schematic diagrams illustrating different examples of data storage systems used by a machine learning transaction processing system. [Figure 2B] 1A-1C are schematic diagrams illustrating different examples of data storage systems used by a machine learning transaction processing system. [Figure 3A] 3A and 3B are schematic diagrams illustrating different examples of transaction data. [Figure 3B] 3A and 3B are schematic diagrams illustrating different examples of transaction data. [Figure 4] FIG. 1 is a schematic diagram illustrating example components of a machine learning transaction processing system. [Figure 5A] FIG. 2 is a sequence diagram illustrating an example set of processes performed by different computing entities on transaction data. [Figure 5B] FIG. 2 is a sequence diagram illustrating an example set of processes performed by different computing entities on transaction data. [Figure 6] FIG. 1 is a schematic diagram illustrating an example of a machine learning model data access hierarchy. [Figure 7] FIG. 1 is a schematic diagram illustrating another example of a machine learning model data access hierarchy. [Figure 8] FIG. 1 is a schematic diagram illustrating another example of a machine learning model data access hierarchy. [Figure 9] FIG. 1 is a schematic diagram illustrating another example of a machine learning model data access hierarchy. [Figure 10] FIG. 1 is a schematic diagram illustrating another example of a machine learning model data access hierarchy. [Figure 11] FIG. 1 is a schematic diagram illustrating another example of a machine learning model data access hierarchy. [Figure 12]FIG. 1 is a schematic diagram illustrating another example of a machine learning model data access hierarchy. [Figure 13] FIG. 1 is a schematic diagram illustrating another example of a machine learning model data access hierarchy. [Figure 14] FIG. 1 is a schematic diagram illustrating another example of a machine learning model data access hierarchy. [Figure 15] FIG. 1 is a schematic diagram illustrating another example of a machine learning model data access hierarchy. [Figure 16] FIG. 1 is a schematic diagram illustrating another example of a machine learning model data access hierarchy. [Figure 17] FIG. 1 is a schematic diagram illustrating another example of a machine learning model data access hierarchy. [Figure 18] FIG. 1 is a schematic diagram illustrating another example of a machine learning model data access hierarchy. [Figure 19] FIG. 1 is a schematic diagram illustrating another example of a machine learning model data access hierarchy. [Figure 20] FIG. 1 is a schematic diagram illustrating another example of a machine learning model data access hierarchy. [Figure 21] FIG. 1 is a flow diagram illustrating an example method for controlling access to machine learning model data. [Figure 22] FIG. 1 is a block diagram illustrating an example of a multi-tenant machine learning system. [Figure 23] FIG. 1 is a schematic diagram illustrating another example of a machine learning model data access hierarchy. [Figure 24] FIG. 1 is a schematic diagram illustrating another example of a machine learning model data access hierarchy. [Figure 25] FIG. 1 is a schematic diagram illustrating another example of a machine learning model data access hierarchy. DETAILED DESCRIPTION OF THE INVENTION

[0096] Certain exemplary embodiments of machine learning systems for use in transaction processing are described herein. In certain embodiments, the machine learning system is applied to real-time, high-volume transaction processing pipelines to provide notifications of whether a transaction or entity matches a previously observed and / or predicted pattern of activity or action, e.g., whether the transaction or entity is “normal” or “anomalous.” The term “behavioral” is used herein to refer to a pattern of activity or action. The notification may include a normalized scalar value within a predetermined range (e.g., 0 to 1), which can be used to prevent fraud and other abuse of payment systems. The machine learning system may apply machine learning models that are updated as more transaction data is acquired, e.g., continually trained on new data, to reduce false positives and maintain accuracy of output metrics. This embodiment may be particularly useful in preventing fraud when the physical presence of a payment card cannot be confirmed (e.g., online transactions known as “card not present”) or in commercial transactions where high-value transactions are common and it may be difficult to classify behavioral patterns as “unexpected.” Thus, the present embodiment facilitates the processing of transactions as these transactions are primarily "online," ie, conducted digitally over one or more public communications networks.

[0097] Certain embodiments described herein enable machine learning models to be tuned to be specific to patterns of behavior between particular pairs of entities (e.g., account holders) and categories (e.g., merchants, transaction amounts, time of day, etc.). For example, the machine learning models can model behavior patterns that are specific to entity-category pairs. The machine learning systems described herein can provide for dynamically updating machine learning models despite large transaction flows and / or the need for segregation of different data sources.

[0098] As outlined above, embodiments of the present invention may be applied to a wide variety of digital transactions, including, but not limited to, card payments, so-called "wire" transfers, peer-to-peer payments, Bankers' Automated Clearing System (BACS) payments, and Automated Clearing House (ACH) payments. The output of machine learning systems may be used to prevent a wide variety of fraud and criminal activity, such as card fraud, application fraud, payment fraud, merchant fraud, game fraud, and money laundering.

[0099] A non-limiting exemplary embodiment of a machine learning system according to an embodiment of the present invention is described below: Figures 1A through 5B provide context for the machine learning system.

[0100] Transaction Processing System Example 1A through 1C illustrate a set of exemplary transaction processing systems 100, 102, 104. These exemplary transaction processing systems are described to provide a context for the inventions discussed herein but should not be considered limiting, and the configuration of any one implementation may vary based on the particular requirements of that implementation. However, the exemplary transaction processing systems described will enable one skilled in the art to identify certain high-level technical features relevant to the following description. The three exemplary transaction processing systems 100, 102, 104 illustrate different areas in which variations may occur.

[0101] 1A through 1C illustrate a set of client devices 110 configured to initiate a transaction. In this example, the set of client devices 110 includes a smartphone 110-A, a computer 110-B, a point-of-sale (POS) system 110-C, and a portable merchant device 110-D. These client devices 110 provide a non-exhaustive set of examples. In general, any electronic device or set of devices can be used to conduct a transaction. In some cases, the transaction includes a purchase or payment. For example, the purchase or payment can be an online or mobile purchase or payment made via smartphone 110-A or computer 110-B, or can be a purchase or payment made at a merchant's premises, such as via POS system 110-C or portable merchant device 110-D. The purchase or payment can be for goods and / or services.

[0102] 1A-1C, client device 110 is communicatively coupled to one or more computer networks 120. Client device 110 may be communicatively coupled in a variety of ways, including by one or more wired and / or wireless networks, including telecommunications networks. In a preferred example, all communications across one or more computer networks are secured using, for example, the Transport Layer Security (TLS) protocol. In FIG. 1A, two computer networks 120-A and 120-B are shown. These may be separate networks or different portions of a common network. The first computer network 120-A communicatively couples client device 110 to a merchant server 130. Merchant server 130 may execute computer processes that implement a process flow for a transaction. For example, merchant server 130 may be a back-end server that processes transaction requests received from POS system 110-C or portable merchant device 110-D, or may be used by an online merchant to implement a website where purchases may be made. It should be understood that the layout of Figures 1A-1C is a necessary simplification of the actual architecture. For example, there may be several interacting server devices implementing an online merchant, including separate server devices for serving HyperText Markup Language (HTML) pages detailing products and / or services and for handling payment processes.

[0103] In FIG. 1A, the merchant server 130 is communicatively coupled to an additional set of backend server devices for processing transactions. In FIG. 1A, the merchant server 130 is communicatively coupled to a payment processor server 140 via a second network 120-B. The payment processor server 140 is communicatively coupled to a first data storage device 142 that stores transaction data 146 and a second data storage device 144 that stores auxiliary data 148. The transaction data 146 may include batches of transaction data for different transactions conducted over a period of time. The auxiliary data 148 may include data associated with transactions, such as records storing merchant and / or end-user data. In FIG. 1A, the payment processor server 140 is communicatively coupled to a machine learning server 150 via the second network 120-B. The machine learning server 150 implements a machine learning system 160 for processing the transaction data. The machine learning system 160 is configured to receive input data 162 and map it to output data 164 used by the payment processor server 140 to process a particular transaction, such as one originating from a client device 110. In some cases, the machine learning system 160 receives at least transaction data associated with a particular transaction and provides an alert or numerical output used by the payment processor server 140 to determine whether the transaction is permitted (i.e., approved) or denied. Thus, the output of the machine learning system 160 may include a label, an alert or other indication of fraud, or general malicious or anomalous activity. The output may include a probabilistic indication, such as a score or probability. In some cases, the output data 164 may include a scalar numeric value. The input data 162 may further include data derived from one or more of the transaction data 146 and the auxiliary data 148. In some cases, the output data 164 indicates a level of deviation from a particular expected pattern of behavior based on past observations or measurements.For example, this can indicate fraudulent or criminal behavior, as it is often significantly different from observed behavior patterns, especially at large scale. The output data 164 can form a behavioral measurement. Expected behavior patterns can be explicitly or implicitly defined based on observed interactions between different entities in the transaction process flow, such as the end user or customer, the merchant (including storefront and backend locations or entities, which may be different), and the bank.

[0104] The machine learning system 160 may be implemented as part of a transaction processing pipeline. An exemplary transaction processing pipeline is described below with respect to FIGS. 5A and 5B . The transaction processing pipeline may include electronic communications between the client device 110, the merchant server 130, the payment processor server 140, and the machine learning server 150. Other server devices, such as a bank server that provides authorization from the issuing bank, may also be involved. In some cases, the client device 110 may communicate directly with the payment processor server 140. In use, the transaction processing pipeline typically needs to complete within 100 or 200 milliseconds. Generally, processing times of less than one second may be considered real-time (e.g., humans typically perceive events in a 400 ms time span). Furthermore, 100-200 ms may be the desired maximum latency of the total round-trip time for transaction processing. Within this time span, most of the time may be reserved for other operations within the transaction processing flow, and the time allocated to machine learning system 160 may be a small fraction of this total, such as 10 ms (i.e., less than 5-10% of the target processing time). This presents technical constraints for the implementation of machine learning system 160. Furthermore, in real-world implementations, average processing rates may be on the order of 1000-2000 per second. This means that most "off-the-shelf" machine learning systems are not suitable for implementing machine learning system 160. This further means that most machine learning techniques described in academic papers cannot be implemented within the aforementioned transaction processing pipeline without nontrivial adaptation. Another problem is that anomalies are inherently rare events, making it difficult to train accurate machine learning systems.

[0105] FIG. 1B illustrates a variation 102 of the exemplary transaction processing system 100 of FIG. 1A. In this variation 102, the machine learning system 160 is implemented within the payment processor computer infrastructure, e.g., executed by the payment processor server 140 and / or executed on a locally coupled server in the same local network as the payment processor server 140. The variation 102 of FIG. 1B may be preferred for larger payment processors because it allows for faster response times, greater control, and improved security. However, functionally, the transaction processing pipeline may be similar to that of FIG. 1A. For example, in the example of FIG. 1A, the machine learning system 160 may be initiated by a secure external application programming interface (API) call, such as a Representation State Transfer (REST) ​​API call using Hypertext Transfer Protocol Secure (HTTPS), whereas in FIG. 1B, the machine learning system 160 may be initiated by an internal API call, where a common end API may handle both requests (e.g., a REST HTTPS API may provide an external wrapper for the internal API).

[0106] FIG. 1C illustrates another variation 104 of the exemplary transaction processing system 100 of FIG. 1A. In this variation 104, the machine learning system 160 is communicatively coupled to a local data storage device 170. For example, the data storage device 170 may be on the same local network as the machine learning server 150 or may comprise a local storage network accessible to the machine learning server 150. In this case, there are multiple local data storage devices 170-A through 170-N, each storing partitioned auxiliary data 172. The partitioned auxiliary data 172 may comprise parameters for one or more machine learning models. In some cases, the auxiliary data 172 may comprise state for the machine learning models, and the state may be associated with a particular entity, such as a user or a merchant. The partitioning of the auxiliary data 172 may need to be applied to meet security requirements set by third parties, such as a payment processor, one or more banks, and / or one or more merchants. In use, machine learning system 160 accesses auxiliary data 172-A-172-N via multiple local data storage devices 170-A-170-N based on input data 162. For example, input data 162 may be received via an API request from a particular source and / or may include data identifying a particular partition to be used to process the API request. Further details of different storage systems that may be employed to meet security requirements are described in Figures 2A and 2B.

[0107] Exemplary Data Storage Configurations 2A and 2B illustrate two exemplary data storage configurations 200 and 202 that may be used by an exemplary machine learning system 210 for processing transaction data. The examples of FIGS. 2A and 2B are two non-limiting examples illustrating different options available for implementation, and a specific configuration may be selected according to individual circumstances. The machine learning system 210 may comprise an implementation of the machine learning system 160 described in the previous examples of FIGS. 1A through 1C. The examples of FIGS. 2A and 2B, for example, enable the machine learning system 210 to process transaction data secured using heterogeneous cryptographic parameters to securely process transaction data for heterogeneous entities. It should be understood that the configurations of FIGS. 2A and 2B may not be used when the machine learning system 160 is implemented for one set of secured transactions and ancillary data, for example, within an internal transaction processing system or as a host system for use by a single payment processor.

[0108] FIG. 2A shows a machine learning system 210 communicatively coupled to a data bus 220. The data bus 220 may comprise an internal data bus of the machine learning server 150 or may form part of a storage area network. The data bus 220 communicatively couples the machine learning system 210 to multiple data storage devices 230, 232. The data storage devices 230, 232 may include any known data storage devices, such as magnetic hard disks and solid-state devices. While the data storage devices 230, 232 are shown as separate devices in FIG. 2A, they may alternatively form different physical areas or storage portions of a common data storage device. In FIG. 2A, the multiple data storage devices 230, 232 store historical transaction data 240 and auxiliary data 242. In FIG. 2A, a first set of data storage devices 230 stores historical transaction data 240, and a second set of data storage devices 232 stores auxiliary data 242. Ancillary data 242 may comprise one or more of model parameters for a set of machine learning models (such as trained parameters for a neural network architecture and / or configuration parameters for a random forest model) and state data for those models. In some cases, different sets of historical transaction data 240-A-N and auxiliary data 242-A-N are associated with different entities that securely and collectively use the services provided by machine learning system 210; for example, they may represent data for different banks that need to be kept separate as part of the terms of providing machine learning services to those entities.

[0109] 2B illustrates another manner in which different sets of historical transaction data 240-A-N and auxiliary data 242-A-N may be stored. In FIG. 2B, machine learning system 210 is communicatively coupled to at least one data storage device 260 via data transfer channel 250. Data transfer channel 250 may include a local storage bus, a local storage area network, and / or a remote secure storage connection (e.g., overlaid on a non-secure network such as the Internet). In FIG. 2B, physical data storage device 260 is used to provide secure logical storage layer 270. Secure logical storage layer 270 may be a virtualized system that is actually implemented independently on at least one data storage device 260, but appears to machine learning system 210 as a separate physical storage device. Logical storage layer 270 may provide separate encrypted partitions 280 for data related to groups of entities (e.g., related to different issuing banks, etc.), and different sets of historical transaction data 240-A-N and auxiliary data 242-A-N may be stored in corresponding partitions 280-A-N. In certain cases, entities may be dynamically created as transactions are received for processing based on data stored by one or more of the server systems shown in Figures 1A-1C.

[0110] Example Transaction Data 3A and 3B illustrate example transaction data that may be processed by a machine learning system such as 160 or 210. FIG. 3A illustrates how the transaction data may include a set of chronologically ordered records 300, each with a timestamp and including multiple transaction fields. In some cases, the transaction data may be grouped and / or filtered based on the timestamps. For example, FIG. 3A illustrates partitioning of the transaction data into current transaction data 310 associated with the current transaction and "older" or historical transaction data 320 that falls within a predetermined time range of the current transaction. The time range may be set as a hyperparameter of any machine learning system. Alternatively, the "older" or historical transaction data 320 may be set as a specific number of transactions. A mixture of the two approaches is also possible.

[0111] FIG. 3B illustrates how transaction data 330 for a particular transaction may be stored in a numeric format for processing by one or more machine learning models. For example, in FIG. 3B, the transaction data has fields such as, at least, the transaction amount, a timestamp (e.g., as a Unix epoch), a transaction type (e.g., card payment or direct debit), a product description or identifier (i.e., relating to the item being purchased), a merchant identifier, an issuing bank identifier, a character set (e.g., Unicode characters in a field of a predetermined character length), a country identifier, etc. Note that a wide range of data types and formats may be accepted and preprocessed into appropriate numeric representations. In some cases, outgoing transaction data, such as that generated by a client device and sent to the merchant server 130, is preprocessed to convert alphanumeric data types into numeric data types for application of one or more machine learning models. Other fields present in the transaction data may include, but are not limited to, an account number (e.g., a credit card number), the location where the transaction is being performed, and the method by which the transaction is performed (e.g., in person, over the phone, on a website).

[0112] Exemplary Machine Learning System FIG. 4 shows one example 400 of a machine learning system 402 that may be used to process transaction data. The machine learning system 402 may implement one or more of the machine learning systems 160 and 210. The machine learning system 402 receives input data 410. The format of the input data 410 may depend on which machine learning model is being applied by the machine learning system 402. When the machine learning system 402 is configured to perform fraud or anomaly detection on a transaction, e.g., an ongoing transaction as described above, the input data 410 may include transaction data (i.e., data that forms part of a data package for the transaction), such as 330, as well as data derived from historical transaction data (e.g., 300 in FIG. 3A) and / or auxiliary data (e.g., 148 in FIGS. 1A-1C or 242 in FIGS. 2A and 2B). The auxiliary data may include secondary data linked to one or more entities identified in the primary data associated with the transaction. For example, if the transaction data for an ongoing transaction identifies the user, the merchant, and one or more banks associated with the transaction (such as the issuing bank and merchant bank for the user), such as through unique identifiers present in the transaction data, the auxiliary data may include data about these transaction entities. The auxiliary data may also include data derived from activity records, such as interaction logs and / or authentication records. In some cases, the auxiliary data is stored in one or more static data records and retrieved from these records based on the received transaction data. Additionally or alternatively, the auxiliary data may include machine learning model parameters that are retrieved based on the content of the transaction data. For example, the machine learning model may have parameters specific to one or more of the user, merchant, and issuing bank, and these parameters may be retrieved based on which of these are identified in the transaction data.For example, one or more of the users, merchants, and issuing banks may have corresponding embeddings, which may comprise retrievable or mappable tensor representations for such entities. For example, each user or merchant may have a tensor representation (e.g., a floating-point vector of size 128-1024), which may be retrieved from a database or other data store or may be generated by an embedding layer based on, for example, a user or merchant index.

[0113] Input data 410 is received at an input data interface 412. The input data interface 412 may comprise an API interface, such as an internal or external API interface as described above.

[0114] In some cases, the payment processor server 140 shown in FIGS. 1A-1C forms requests to this interface, where the request payload includes transaction data. The API interface may be defined to be agnostic with respect to the format of the transaction data or its source. The input data interface 412 is communicatively coupled to the machine learning model platform 414. In some cases, requests made to the input data interface 412 trigger execution of the machine learning model platform 414 using the transaction data provided to the interface. The machine learning model platform 414 is configured as an execution environment for applying one or more machine learning models to the input data 410. In some cases, the machine learning model platform 414 is deployed as an execution wrapper for multiple different selectable machine learning models. For example, machine learning models may be defined using a model definition language (e.g., similar to or using a markup language such as Extensible Markup Language—XML). Model definition languages ​​may include SQL, TensorFlow, Caffe, Thinge, and PyTorch (among others, independently or in combination). In some cases, the model definition language includes computer program code executable to implement one or more of the training and inference of the defined machine learning models. The machine learning models can include, for example, artificial neural network architectures, ensemble models, regression models, decision trees such as random forests, graph models, and Bayesian networks, among others. The machine learning model platform 414 can define common (i.e., shared) input and output definitions so that different machine learning models can be applied in a common (i.e., shared) manner.

[0115] In this example, the machine learning model platform 414 is configured to provide at least a single scalar output 416, which may be normalized within a predetermined range, such as from 0 to 1. Once normalized, the scalar output 416 can be viewed as a probability that a transaction associated with the input data 410 is fraudulent or anomalous. In this case, a value of “0” may represent a transaction that matches the normal activity pattern of one or more of the user, merchant, and issuing bank, while a value of “1” may indicate that the transaction is fraudulent or anomalous, i.e., does not match the expected activity pattern (however, one skilled in the art will recognize that the normalized range may be different, such as inverted or within different boundaries, and have the same functional effect). Note that while the range of values ​​may be defined as 0 to 1, the output values ​​may not be uniformly distributed within this range; for example, a value of “0.2” may be a common output for a “normal” event, while a value of “0.8” may be considered above a threshold for a typical “anomalous” or fraudulent event. The machine learning models implemented by machine learning platform 414 may therefore implement a form of mapping between high-dimensional input data (e.g., transaction data and any retrieved auxiliary data) and single-value output. In some cases, for example, machine learning platform 414 may be configured to receive input data for the machine learning models in numeric format, and each defined machine learning model is configured to map the defined input data in the same way. The exact machine learning model applied by machine learning model platform 414, and the parameters for that model, may be determined based on configuration data. The configuration data may be included in and / or identified using input data 410 and / or may be set based on one or more configuration files parsed by machine learning platform 414.

[0116] In some cases, the machine learning model platform 414 can provide additional outputs depending on the context. In some implementations, the machine learning model platform 414 can be configured to return a "reason code" that captures a human-friendly explanation of the machine learning model's output regarding the suspicious input attribute. For example, the machine learning model platform 414 may indicate which of one or more input elements or units in the input representation influenced the model output, such as the combination of a "volume" channel above a learned threshold and a set of "seller" elements or units (e.g., embeddings or indexes) that are outside a given cluster. If the machine learning model platform 414 implements decision trees, these additional outputs may include paths through the decision trees or aggregate feature importances based on an ensemble of trees. In the case of a neural network architecture, this may include layer output activations and / or layer filters with positive activations.

[0117] In FIG. 4 , certain implementations may include an optional alert system 418 that receives the scalar output 416. In other implementations, the scalar output 416 may be passed directly to the output data interface 420 without post-processing. In this latter case, the scalar output 416 may be packaged in a response to the original request to the input data interface 412. In both cases, output data 422 derived from the scalar output 416 is provided as an output of the machine learning system 402. The output data 422 is returned to enable final processing of the transaction data. For example, the output data 422 may be returned to the payment processor server 140 and used as the basis for a decision to approve or deny the transaction. Depending on implementation requirements, in some cases, the alert system 418 may process the scalar output 416 and return a binary value indicating whether the transaction should be approved or denied (e.g., “1” equals deny). In some cases, the decision may be made by applying a threshold to the scalar output 416. This threshold may be context-dependent. In some cases, the alarm system 418 and / or the output data interface 420 may also receive additional inputs, such as explanatory data (e.g., the "reason code" discussed above) and / or original input data. The output data interface 420 may generate an output data package for the output data 422 that combines these inputs with the scalar output 416 (e.g., at least for logging and / or later review). Similarly, the alerts generated by the alarm system 418 may include and / or be further based on the aforementioned additional inputs in addition to the scalar output 416, for example.

[0118] The machine learning system 402 is typically used in an “online” mode to process a large volume of transactions within a narrowly defined time range. For example, under normal processing conditions, the machine learning system 402 may process a request within 7–12 ms and manage 1,000–2,000 requests per second (these are median constraints from real-world operating conditions). However, the machine learning system 402 may also be used in an “offline” mode, for example, by providing selected historical transactions to the input data interface 412. In offline mode, input data may be passed to the input data interface in batches (i.e., groups). The machine learning system 402 may also be capable of implementing machine learning models that provide a scalar output for an entity in addition to or instead of a transaction. For example, the machine learning system 402 may be configured to receive a request associated with an identified user (e.g., card or payment account holder) or an identified merchant and provide a scalar output 416 indicating the likelihood that the user or merchant is fraudulent, malicious, or anomalous (i.e., a general threat or risk). For example, this may form part of a continuous or periodic monitoring process, or a one-off request (e.g., as part of an application for a service). The provision of a scalar output for a particular entity may be based on a set of transaction data up to and including the most recently approved transaction in the sequence of transaction data (e.g., transaction data for an entity similar to that in Figure 3A).

[0119] Both "live" and "batch" data can be received and used by the machine learning system, either by the same set of Event APIs with different REST endpoints, or by a different set of Event APIs dedicated to reading batch data from another source.

[0120] Additionally or alternatively, the Event API may read events from another source (rather than receiving events via REST), for example, by reading events from a comma-separated value (CSV) file or any other suitable file format.

[0121] Exemplary Transaction Process Flow 5A and 5B illustrate two possible exemplary transaction process flows 500 and 550. These process flows may take place in the context of the exemplary transaction processing systems 100, 102, and 104 shown in FIGS. 1A through 1C and other systems. While process flows 500 and 550 are provided as an example of a context in which a machine learning transaction processing system may be applied, not all transaction process flows necessarily follow the process shown in FIGS. 5A and 5B, and process flows may vary across implementations, systems, and over time. The exemplary transaction process flows 500 and 550 reflect two possible cases: a first case represented by transaction process flow 500 in which the transaction is approved, and a second case represented by transaction process flow 550 in which the transaction is denied. Each transaction process flow 500, 550 involves the same set of five interacting systems and devices: a POS or user device 502, a merchant system 504, a payment processor (PP) system 506, a machine learning (ML) system 508, and an issuing bank system 510. The POS or user device 502 may comprise one of the client devices 110, the merchant system 504 may comprise the merchant server 130, the payment processor system 506 may comprise the payment processor server 140, and the machine learning system 508 may comprise an implementation of the machine learning systems 160, 210, and / or 402. The issuing bank system 510 may comprise one or more server devices that implement transaction functionality on behalf of the issuing bank. The five interacting systems and devices 502-510 may be communicatively coupled by one or more internal or external communication channels, such as the network 120. In some cases, some of these systems may be combined; for example, the issuing bank may also act as the payment processor; thus, systems 506 and 510 may be implemented using a common system. In other cases, a similar process flow may be performed specifically for the merchant (e.g., without a payment processor or issuing bank).In this case, the machine learning system 508 may communicate directly with the merchant system 504. In these variations, the general functional transaction process flow may remain similar to that described below.

[0122] The transaction process flow in both Figures 5A and 5B includes several common (i.e., shared) processes 512 through 528. In block 512, the POS or user device 502 initiates the transaction. In the case of a POS device, this may include a cashier using a front-end device to attempt to make an electronic payment, and in the case of a user device 502, this may include a user making an online purchase (e.g., clicking "Done" in an online basket) using a credit or debit card or online payment account. In block 514, payment details are received as electronic data by the merchant system 504. In block 516, the transaction is processed by the merchant system 504, and a request is made to the payment processor system 506 to authorize the payment. In block 518, the payment processor system 506 receives a request from the merchant system 504. The request may be made over a proprietary communication channel or as a secure request over a public network (e.g., an HTTPS request over the Internet). The payment processor system 506 then makes a request to the machine learning system 508 for a score or probability to use in processing the transaction. Block 518 may further include retrieving the auxiliary data to combine with the transaction data sent as part of the request to the machine learning system 508. In other cases, the machine learning system 508 may access a data storage device (e.g., similar to the configuration of FIGS. 2A and 2B) that stores the auxiliary data and thus retrieve this data as part of its internal operations (e.g., based on an identifier provided in the transaction data and / or as defined as part of an implemented machine learning model).

[0123] Block 520 illustrates a model initialization operation that occurs before any request from the payment processor system 506. For example, the model initialization operation may include loading a defined machine learning model and parameters that instantiate the defined machine learning model. In block 522, the machine learning system 508 receives a request from the payment processor system 506 (e.g., via a data input interface such as 412 in FIG. 4). In block 522, the machine learning system 508 may perform any defined preprocessing before applying the machine learning model initialized in block 520. For example, if the transaction data still holds character data, such as a merchant identified by a string of characters or a character transaction description, this may be converted into suitable structured numeric data (e.g., by converting string categorical data to an identifier via a lookup operation or other mapping and / or by mapping characters or groups of characters to a vector embedding). Then, in block 524, the machine learning system 506 applies the instantiated machine learning model and supplies the model with input data derived from the received request. This may include applying the machine learning model platform 414 described with reference to FIG. 4. At block 526, a scalar output is generated by the instantiated machine learning model, which may be processed in the machine learning system 508 to determine a binary "approve" or "reject" decision, or, if preferred, returned to the payment processor system 506 in response to the request made at block 518.

[0124] In block 528, the output of the machine learning system 508 is received by the payment processor system 506 and used to approve or deny the transaction. Figure 5A illustrates a process by which a transaction is approved based on the output of the machine learning system 508. Figure 5B illustrates a process by which a transaction is denied based on the output of the machine learning system 508. In Figure 5A, the transaction is approved in block 528. Next, in block 530, a request is made to the issuing bank system 532. In block 534, the issuing bank system 532 approves or denies the request. For example, the issuing bank system 532 can approve the request if the end user or cardholder has sufficient funds and authorization to cover the transaction costs. In certain cases, the issuing bank system 532 can apply a second level of security, although this may not be required if the issuing bank relies on anomaly detection performed by the payment processor using the machine learning system 508. In block 536, authorization from the issuing bank system 510 is returned to the payment processor system 506, which then sends a response in block 538 to the merchant system 504, which then responds to the POS or user device 502 in block 540. If the issuing bank system 510 approves the transaction in block 534, the transaction may be completed and an affirmative response may be returned to the POS or user device 502 via the merchant system 504. The end user may experience this as an "authorized" message on the screen of the POS or user device 502. The merchant system 504 may then complete the purchase (e.g., initiate internal processing to fulfill the purchase).

[0125] At a later point in time, one or more of the merchant system 504 and the machine learning system 508 may store data about the transaction, for example, as part of the transaction data 146, 240, or 300 in the previous example. This is indicated by dashed blocks 542 and 544.

[0126] The transaction data may be stored along with one or more of the output of the machine learning system 508 (e.g., scalar fraud or anomaly probability) and the final outcome of the transaction (e.g., whether the transaction was approved or denied). The stored data may be stored for use as training data for the machine learning models implemented by the machine learning system 508.

[0127] The stored data may also be accessed as part of a future iteration of block 524, and may, for example, form part of future auxiliary data. In certain cases, the final result or outcome of a transaction may not be known at the time of the transaction. For example, a transaction may only be labeled as anomalous through later review by analysts and / or automated systems, or based on feedback from a user (e.g., if the user reports fraud or indicates that a payment card or account has been compromised since a particular date). In these cases, ground truth labels for purposes of training the machine learning system 508 may be collected over time following the transaction itself.

[0128] Turning now to the alternative process flow of FIG. 5B , in this case, the one or more machine learning systems 508 and the payment processor system 506 deny the transaction based on the output of the machine learning system 508. For example, if the scalar output of the machine learning system 508 exceeds a derived threshold, the transaction may be denied. At block 552, the payment processor system 506 issues a response to the merchant system 504, which is received at block 554. At block 554, the merchant system 504 performs steps to prevent the transaction from completing and returns an appropriate response to the POS or user device 502. This response is received at block 556, and the end user or customer may be notified that the end user's or customer's payment has been denied, for example, via an on-screen "denied" message. The end user or customer may be prompted to use a different payment method. While not shown in FIG. 5B , in some cases, the issuing bank system 510 may be notified that a transaction for a particular account holder has been denied. The issuing bank system 510 may be notified as part of the process shown in Figure 5B, or may be notified as part of a periodic (e.g., daily) update. The transaction may not become part of transaction data 146, 240, or 300 (because it was not approved), but may still be logged by at least the machine learning system 508, as indicated by block 544. For example, with respect to Figure 5A, the transaction data may be stored along with the output of the machine learning system 508 (e.g., scalar fraud or anomaly probability) and the final outcome of the transaction (e.g., that it was rejected).

[0129] Multi-tenant Processing - Technical Effects As explained above, having separate installations for different tenants and / or data types running on the same software, while appearing as one system, can lead to complex final solutions.

[0130] The examples described herein relate to different types of multi-tenant solutions. A "tenant" may be a customer of the platform administrator. The administrator may be referred to as the "landlord." The term "multi-tenant" is used herein to mean at least two tenants.

[0131] The exemplary multi-tenant solution described herein can securely serve all of an administrator's tenants with data isolation between different tenants.

[0132] Without loss of generality, the examples described herein provide a single platform that separates data by tenant. A given tenant may be identified in an event by a "tenant ID."

[0133] A tenant can be configured to be part of a hierarchy that includes an analysis configuration group (ACG). An ACG is generally referred to herein as a "tenant group." A tenant group can include one tenant, or two or more tenants.

[0134] Tenant groups may themselves be included in different solutions, and therefore a solution may be thought of as a collection of one or more tenant groups.

[0135] The solution can be structured in a variety of different ways.

[0136] A user interface (UI) can be used to configure where each tenant sits in the hierarchy.

[0137] Data may be segregated, for example, by tenant and tenant group: each tenant and each tenant group may have its own area of ​​the database where models executed for the tenant and tenant group can store data.

[0138] When an event (e.g., a transaction event) is processed, data is loaded from the database and made available to one or more models. Depending on access control rights, the data available to a model(s) may include: (i) data for the tenant, (ii) data for each tenant group the tenant is in, (iii) data for the containing solution (if any), and / or (iv) platform-wide administrator-level data.

[0139] The model can then decide how it wants to process the data. If the model is a consortium model created by an administrator to run across all events, the model can consider the event as a whole, potentially using and storing data at each level in the hierarchy appropriate to what that particular tenant group represents. If the model is only configured to run at a particular tenant group or tenant level, the model can only use data from that tenant group down.

[0140] The example system described herein allows data to be used in both hierarchical directions (i.e., up and down) in the configuration hierarchy. Rules at the tenant group or administrator level can use data and events from all levels below them. Tenants and tenant groups may be able to use data exposed to them by higher levels in the hierarchy in their own rules. Data sharing rules, learnings, and data applied within business units, client sets, and other entities adapt to changing events and circumstances more quickly and efficiently than existing systems, and without manual intervention. This universal data sharing and hierarchical referencing is new in the world of machine learning platforms, especially in fraud and risk.

[0141] Because the entire platform is a single system, the tenant hierarchy can be modified on the fly. New solutions and / or tenants can be easily added to an existing platform without, for example, having to deploy a new instance of the platform to run the new solution. Models can be run within a given solution or across all solutions. Data can be accessed up and down the hierarchy by tenant.

[0142] Within the UI, each tenant and / or solution can be configured to allow access only to that tenant, or that solution and all tenant groups below it. In an example, only administrators have access to the entire system and can control any tenant and / or tenant group on the platform and modify its analysis as needed.

[0143] The example platform described herein provides a high level of fine-grained control, allowing models to be isolated by solution, tenant group, and / or tenant, or to be accessed system-wide as a consortium model.

[0144] The exemplary platform described herein simplifies deployment because the platform is deployed and configured once, each solution runs within the same platform, and data separation is managed by the platform and model.

[0145] The exemplary platform described herein is also highly flexible, with analytical configuration and data separated to best suit a particular use case. This architecture allows differentiated and even overlapping analytics to be performed to increase compatibility and flexibility between tenant groups and tiers.

[0146] The examples described herein allow several individual instances to be provisioned from one multi-tenant configuration, providing instant access to cutting-edge fraud and financial crime prevention technology while maintaining accessibility to various combinations of access controls and features. Furthermore, the multi-tenant architecture allows providers to perform updates only once to distribute upgrades across all tenants.

[0147] In principle, the computational overhead of a single multi-tenant platform can be prohibitive compared to individual instances per tenant. There can also, in principle, be a risk of data "pollution" in a single multi-tenant platform. However, the examples described herein provide a computationally efficient, data-isolated multi-tenant platform that is easily deployable, updatable, can be used for real-time anomaly detection, and allows configuration changes (such as adding new tenants) to be made efficiently and effectively.

[0148] Multi-tenant processing - Machine learning model data access hierarchy - Example 1 6 illustrates an exemplary machine learning model data access hierarchy 600. For convenience and brevity, the machine learning model data access hierarchy is generally referred to herein as the "hierarchy."

[0149] In this example, the hierarchy 600 includes first and second levels 602,604.

[0150] A "level" can be thought of as a logical association of a set of elements in hierarchy 600. The association can be based at least in part on one or more characteristics of the elements in question. While elements belonging to the same level as each other are shown in the same horizontal region of hierarchy 600 in Figure 6, this is merely for ease of understanding the present disclosure. A level may also be referred to as a "layer," etc.

[0151] In this example, the first level 602 includes a tenant level 602. In this example, the tenant level 602 includes any and all tenants. In this particular example, the tenant level 602 includes first and second tenants 606, 608.

[0152] The first and second tenants 606, 608 are associated with first and second tenant machine learning model data 610, 612, respectively. For convenience and brevity, the machine learning model data is generally referred to herein as "ML model data." The ML model data may be associated with a tenant by being stored in association with a tenant ID for that tenant. The term "ML model data" should be understood to encompass data, such as state data and parameter data, particularly related to an ML model. The ML model data may correspond, for example, to the auxiliary data 172 described above.

[0153] In this example, the second level 604 includes a tenant group level 604. In this example, the tenant group level 604 includes any and all tenant groups. In this particular example, the tenant group level 604 includes exactly one tenant group 614.

[0154] In this example, first and second tenants 606, 608 belong to tenant group 614. This is indicated in FIG. 6 by arrows 616, 618 pointing from the first and second tenants 606, 608, respectively, to tenant group 614. The term "belonging to" is used generally herein to mean contained in, encompassed by, joined to, part of, a child of, etc.

[0155] In this example, tenant group 614 is associated with tenant group ML model data 620. Tenant group 614 may be associated with tenant group ML model data 620 via a tenant group ID or otherwise.

[0156] Data such as metadata indicating the association between the tenant group 614 and the first and second tenants 606, 608 may be stored.

[0157] In this example, the tenant group ML model data 620 is based on the first and / or second tenant ML model data 610, 612. In other words, in this example, the tenant group ML model data 620 is based on (i) only the first tenant ML model data 610, (ii) only the second tenant ML model data 612, or (iii) both the first and / or second tenant ML model data 610, 612.

[0158] Multi-tenant processing - Access control at the tenant group level - Example 1 FIG. 7 shows another example hierarchy 700 .

[0159] The example hierarchy 700 shown in Figure 7 corresponds to the example hierarchy 600 shown in Figure 6, but additionally illustrates an active ML model. Specifically, Figure 7 illustrates an ML model 722 (denoted as "ML model {606, 604}") being applied with respect to the first tenant 606 at the tenant group level 604. The notation "ML model {A, B}" is used herein to indicate the ML model applied with respect to the Ath tenant at the Bth level.

[0160] In this example, access controls are applied in hierarchy 700 such that ML model 722 is (i) allowed to access tenant group ML model data 620 (as shown in item 724), (ii) allowed to access first tenant ML model data 610 (as shown in item 726), and (iii) prohibited from accessing second tenant ML model data 612 (as shown in item 728). The term "prohibited" is used generally herein to mean prevented, not allowed, etc.

[0161] Thus, the first and second tenant ML model data 610, 612 are separated in that the ML model 722 being applied with respect to the first tenant 606 has access to the first tenant ML model data 610 but not the second tenant ML model data 612. In this example, the ML model 722 can perform real-time anomaly detection associated with the first tenant 606 using the tenant group ML model data 620 and the first tenant ML model data 610. In this example, the ML model 722 does not directly use the second tenant ML model data 612 to perform real-time anomaly detection associated with the first tenant 606. More specifically, as will become more apparent from the description below, in this example, the ML model 722 does not have direct access to the second tenant ML model data 612. However, in some examples, the tenant group ML model data 620 is based at least in part on the second tenant ML model data 612. In such an example, the ML model 722 may be able to directly access the tenant group ML model data 620 and therefore may be able to access data that is affected by the second tenant ML model data 612.

[0162] Multi-tenant processing - Zero or limited contribution from tenants to tenant group ML model data As explained above, in this example, the tenant group ML model data 620 is based on the first and / or second tenant ML model data 610, 612.

[0163] In a particular example, the tenant group ML model data 620 is (i) based on the first tenant ML model data 610 and (ii) not based on the second tenant ML model data 612 or based on a limited version of the second tenant ML model data 612.

[0164] The restricted version of the second tenant ML model data 612 may be restricted in a variety of different ways relative to the second tenant ML model data 612.

[0165] In some examples, the restricted version of the second tenant ML model data 612 includes a subset of the second tenant ML model data 612. The subset may include a historical subset. The historical subset may include all elements of the second tenant ML model data 612 prior to a given threshold time. In some examples, the restricted version of the second tenant ML model data 612 includes some or all of the second tenant ML model data 612 with one or more weighting factors applied. The weighting factors may correspond to percentages, for example.

[0166] Thus, in some examples, the tenant group ML model data 620 is (i) based on at least the history set of the first tenant ML model data 610 and (ii) not based on at least the history set of the second tenant ML model data 612, or is based on a limited version of the history set of the second tenant ML model data 612.

[0167] In this particular example, the second tenant 608 joined the tenant group 614 after the first tenant 606 joined the tenant group 614 .

[0168] The historical set of second tenant ML model data 612 may have been generated before the second tenant 608 joined the tenant group 614. Thus, by not using the second tenant ML model data 612 or by using a limited version of the second tenant ML model data 612, the contribution of historical data that may not be particularly reflective of the current ML model data of the second tenant 608 can be limited. This, in turn, can result in more accurate tenant group ML model data 620.

[0169] According to this particular example, the contribution of the second tenant ML model data 612 to the tenant group ML model data 620 may be zero or limited when the second tenant 608 joins the tenant group 614.

[0170] For example, second tenant 608 may have previously belonged to another tenant group (not shown) and may have been moved to tenant group 614 because tenant group 614 is a better match for second tenant 608 than the other tenant groups (not shown). Updating tenant group ML model data 620 with all historical second tenant ML model data 612 may skew tenant group ML model data 620 as a result of the historical second tenant ML model data 612.

[0171] However, the contribution of the second tenant ML model data 612 to the tenant group ML model data 620 may increase over time while the second tenant 608 belongs to the tenant group 614. The contribution of the second tenant ML model data 612 to the tenant group ML model data 620 may increase over time because the second tenant 608 likely moved to the tenant group 614 because the second tenant 608 was a better fit for the tenant group 614 than another tenant group.

[0172] For example, the tenant group ML model data 620 may be updated to generate updated tenant group ML model data (not shown). The updated tenant group ML model data may be based on both the first and second tenant ML model data 610, 612. With respect to the second tenant ML model data 612, the updated tenant group ML model data may be derived from the second tenant ML model data 612.

[0173] Multi-tenant processing - non-zero contributions from tenants to tenant group ML model data As explained above, in this example, the tenant group ML model data 620 is based on the first and / or second tenant ML model data 610, 612.

[0174] In another particular example, the tenant group ML model data 620 is based on at least some of each of the first and second ML model data 610, 612. The term “at least some of” is used herein to mean some or all of.

[0175] In some examples, the tenant group ML model data 620 is derived based on one or more operations involving at least some of each of the first and second ML model data 610, 612.

[0176] The one or more operations may include a combination operation. Examples of combination operations include, but are not limited to, sum, average, minimum, maximum, standard deviation, and combine into sets. A "combination operation" may also be referred to as an "aggregation operation," "functional operation," etc.

[0177] Multi-tenant processing - Access control at the tenant group level - Example 2 FIG. 8 shows another example hierarchy 800 .

[0178] FIG. 8 shows an ML model 830 (denoted as “ML model {608,604}”) being applied for the second tenant 608 at the tenant group level 604.

[0179] The type of ML model 830 may be the same as or different from the type of ML model 722 described above with reference to Figure 7. However, the ML model 830 is applied with respect to the second tenant 608 instead of the first tenant 606.

[0180] In this example, the ML model 830 is (i) allowed to access the tenant group ML model data 620 (as shown in item 724), (ii) prohibited from accessing the first tenant ML model data 610 (as shown in item 726), and (iii) allowed to access the second tenant ML model data 612 (as shown in item 728).

[0181] Thus, the first and second tenant ML model data 610, 612 are again separated in that the ML model 830 being applied with respect to the second tenant 608 has access to the second tenant ML model data 612 and does not have access to the first tenant ML model data 610. In this example, the ML model 830 can perform real-time anomaly detection associated with the second tenant 608 using the tenant group ML model data 614 and the second tenant ML model data 612.

[0182] Multi-tenant processing - Tenant-level access control - Example 1 FIG. 9 shows another example hierarchy 900 .

[0183] FIG. 9 shows an ML model 932 (denoted as “ML model {606, 602}”) being applied for the first tenant 606 at the tenant level 602.

[0184] In this example, ML model 932 is (i) prohibited from accessing tenant group ML model data 620 (as shown in item 724), (ii) permitted to access first tenant ML model data 610 (as shown in item 726), and (iii) prohibited from accessing second tenant ML model data 612 (as shown in item 728).

[0185] Thus, in this example, the ML model 932 can use the first tenant ML model data 610 to perform real-time anomaly detection associated with the first tenant 606. However, in this example, the ML model 932 being applied at the tenant level 602 cannot access either (i) the tenant group ML model data 620 at the (higher) tenant group level 604 or (ii) the second tenant ML model data 612 at the (same) tenant level 602.

[0186] Multi-tenant processing - Tenant-level access control - Example 2 FIG. 10 shows another example hierarchy 1000 .

[0187] FIG. 10 shows an ML model 1034 (denoted as “ML model {608, 602}”) being applied for the second tenant 608 at the tenant level 602.

[0188] In this example, the ML model 1034 is (i) prohibited from accessing the tenant group ML model data 620 (as shown in item 724), (ii) prohibited from accessing the first tenant ML model data 610 (as shown in item 726), and (iii) permitted to access the second tenant ML model data 612 (as shown in item 728).

[0189] Thus, in this example, the ML model 1034 can use the second tenant ML model data 612 to perform real-time anomaly detection associated with the second tenant 608. However, in this example, the ML model 1034 being applied at the tenant level 602 cannot access either (i) the tenant group ML model data 620 at the (higher) tenant group level 604 or (ii) the first tenant ML model data 610 at the (same) tenant level 602.

[0190] Multi-tenant processing - permission settings(multiple allowed) FIG. 11 shows another example hierarchy 1100.

[0191] FIG. 11 shows the ML model 932 shown in FIG. 9 (denoted as “ML model {606, 602}”) being applied for the first tenant 606 at the tenant level 602.

[0192] However, in this example, the ML model 932 (at the tenant level 602 ) is granted access to the (higher level) tenant group ML model data 620 , as shown in item 724 .

[0193] In this particular example, the ML model 932 is selectively permitted to access the tenant group ML model data 620 based on one or more permission settings that are enabled for the ML model 932 .

[0194] For example, one or more permission settings may have previously been disabled for ML model 932, thereby prohibiting ML model 932 from accessing tenant group ML model data 620, as shown in FIG. 9 .

[0195] One or more permission settings may then be enabled for the ML model 932 such that the ML model 932 has access to the tenant group ML model data 620.

[0196] The one or more permission settings may be per tenant, per model, and / or per level.

[0197] For example, one or more permission settings may be enabled for ML model 932 (with respect to the first tenant 606) such that ML model 932 has access to both the first tenant ML model data 610 and the tenant group ML model data 620, and may be disabled for ML model 1034 (with respect to the second tenant 608) such that ML model 1034 has access to the second tenant ML model data 612 but is prohibited from accessing the tenant group ML model data 620.

[0198] In this example, the ML model 932 can perform real-time anomaly detection associated with the first tenant 606 using the tenant group ML model data 620 and the first tenant ML model data 610.

[0199] Multi-tenant processing - Access control at administrator and administrative levels - Example 1 FIG. 12 shows another example hierarchy 1200.

[0200] In this example, hierarchy 1200 includes a third level 1236. In this example, third level 1236 includes an administrator level 1236. In this example, administrator level 1236 includes an administrator 1238.

[0201] In this example, administrator 1238 is associated with administrator ML model data 1240. Administrator 1238 may be associated with administrator ML model data 1240 via an administrator ID or otherwise.

[0202] In this example, tenant group 614 belongs to administrator 1238. This is indicated in FIG. 12 by arrow 1242 pointing from tenant group 614 to administrator 1238.

[0203] FIG. 12 shows an ML model 1244 (denoted as “ML model {606, 1236}”) being applied for the first tenant 606 at the administrator level 1236.

[0204] In this example, the ML model 1244 is (i) allowed to access the administrator ML model data 1240 (as shown in item 1246), (ii) prohibited from accessing the tenant group ML model data 620 (as shown in item 724), (iii) allowed to access the first tenant ML model data 610 (as shown in item 726), and (iv) prohibited from accessing the second tenant ML model data 612 (as shown in item 728).

[0205] In this example, the ML model 1244 can perform real-time anomaly detection associated with the first tenant 606 using the administrator ML model data 1240, the tenant group ML model data 620, and the first tenant ML model data 610.

[0206] In a similar manner, access control can be applied with respect to the second tenant 608 at the administrator level 1236, such that the ML model being applied with respect to the second tenant 608 at the administrator level 1236 can perform real-time anomaly detection associated with the second tenant 608 using the administrator ML model data 1240, the tenant group ML model data 620, and the second tenant ML model data 612.

[0207] Multi-tenant processing - Access control at the tenant group level - Example 3 FIG. 13 shows another example hierarchy 1300.

[0208] FIG. 13 shows an ML model 1348 (denoted as “ML Model {606, 604}”) being applied for the first tenant 606 at the tenant group level 604.

[0209] In this example, the ML model 1348 is (i) prohibited from accessing the administrator ML model data 1240 (as shown in item 1246), (ii) prohibited from accessing the tenant group ML model data 620 (as shown in item 724), (iii) allowed to access the first tenant ML model data 610 (as shown in item 726), and (iv) prohibited from accessing the second tenant ML model data 612 (as shown in item 728).

[0210] In this example, the ML model 1348 can perform real-time anomaly detection associated with the first tenant 606 using the tenant group ML model data 620 and the first tenant ML model data 610.

[0211] However, the ML model 1348 may be given permission to access the administrator ML model data 1240, as described above.

[0212] Multi-tenant processing - further tenant group(s) FIG. 14 shows another example hierarchy 1400.

[0213] For ease of presentation, some elements of the exemplary hierarchy described above with reference to FIGS. 6 through 13 are represented differently in FIG. 14 and certain subsequent figures.

[0214] In this example, the tenant group level 604 includes a further tenant group 1450 .

[0215] Although a single further tenant group 1450 is shown in FIG. 14, the tenant group level 604 may include two or more further tenant groups 1450 in other examples.

[0216] In this example, an additional tenant 1452 belongs to an additional tenant group 1450. This is indicated in FIG. 14 by an arrow 1454 pointing from the additional tenant 1452 to the additional tenant group 1450. Although a single additional tenant 1452 is shown in FIG. 14, two or more additional tenants 1452 may belong to the additional tenant group 1450 in other examples.

[0217] In general, a hierarchy according to the examples described herein can include one or more tenant groups, and a tenant group can include one or more tenants.

[0218] In this example, the first and second tenants 606 , 608 do not belong to the further tenant group 1450 , and the further tenant 1452 does not belong to the tenant group 614 .

[0219] In this example, further tenant group 1450 belongs to administrator 1238. This is illustrated in FIG. 14 by arrow 1454 pointing from further tenant 1452 to further tenant group 1450.

[0220] In this example, further tenant group 1450 is associated with further tenant group ML model data 1458. For example, further tenant group 1450 may be associated with further tenant group ML model data 1458 via a (further) tenant group ID or otherwise.

[0221] In this example, further tenant 1452 is associated with further tenant ML model data 1460. For example, further tenant 1452 may be associated with further tenant ML model data 1460 via a (further) tenant ID or otherwise.

[0222] Multi-tenant processing - Access control at the tenant group level - Example 4 FIG. 15 shows another example hierarchy 1500.

[0223] In this example, the ML model 722 is applied for the first tenant 606 at the tenant group level 604 .

[0224] In this example, the ML model 722 is (i) prohibited from accessing further tenant group ML model data 1458 (as shown in item 1562) and (ii) prohibited from accessing further tenant ML model data 1460 (as shown in item 1564).

[0225] Additionally, in this particular example, ML model 722 (i) is permitted access to first tenant ML model data 610 and tenant group ML model data 620 (as shown in items 726 and 724, respectively), and (ii) is prohibited from accessing second tenant ML model data 612 and administrator ML model data 1240, as shown in items 728 and 1246, respectively.

[0226] In this example, the ML model 722 can perform real-time anomaly detection associated with the first tenant 606 using the tenant group ML model data 614 and the first tenant ML model data 610.

[0227] Multi-tenant processing - Access control at administrator level - Example 2 FIG. 16 shows another example hierarchy 1600.

[0228] In this example, the ML model 1244 is applied for the first tenant 606 at the administrator level 1236 .

[0229] In this example, ML model 1244 is (i) prohibited from accessing further tenant group ML model data 1458 (as shown in item 1562) and (i) prohibited from accessing further tenant ML model data 1460 (as shown in item 1564).

[0230] Additionally, in this particular example, ML model 1244 is (i) permitted to access first tenant ML model data 610, tenant group ML model data 620, and administrator ML model data 1240 (as shown in items 726, 724, and 1246, respectively), and (ii) prohibited from accessing second tenant ML model data 612 (as shown in item 728).

[0231] In this example, the ML model 1244 can perform real-time anomaly detection associated with the first tenant 606 using the administrator ML model data 1240, the tenant group ML model data 614, and the first tenant ML model data 610.

[0232] Multi-tenant processing - Access control at additional tenant(s) and tenant group level - Example 5 FIG. 17 shows another example hierarchy 1700.

[0233] In this example, hierarchy 1700 includes additional tenant 1766 .

[0234] In this example, additional tenant 1766 is shown as being included at the same horizontal level as first and second tenants 606 , 608 , namely tenant level 602 .

[0235] However, it is again emphasized that this is for ease of explanation. Additional tenants 1766 may be depicted at the same horizontal level as tenant groups 614, but still at the logical tenant level 602.

[0236] In this example, additional tenant 1766 is associated with additional tenant ML model data 1768. For example, additional tenant 1766 may be associated with additional tenant ML model data 1768 via an (additional) tenant ID or otherwise.

[0237] In this example, additional tenant 1766 belongs to administrator 1238. This is indicated in FIG. 17 by arrow 1770 pointing from further tenant 1766 to administrator 1238.

[0238] In this example, additional tenant 1766 can be said to belong “directly” to administrator 1238 in that there are no intermediate elements of hierarchy 1700 between additional tenant 1766 and administrator 1238 .

[0239] Similarly, (i) tenant group 614 can be said to belong “directly” to administrator 1238 in that there are no intermediate elements of hierarchy 1700 between tenant group 614 and administrator 1238, and (ii) first and second tenants 606, 608 can be said to belong “directly” to tenant group 614 in that there are no intermediate elements of hierarchy 1700 between tenant group 614 and administrator 1238.

[0240] However, the first and second tenants 606, 608 can be said to belong “indirectly” to the administrator 1238 in that there is at least one intermediate element of the hierarchy 1700 between the first and second tenants 606, 608 and the administrator 1238. In this example, the at least one intermediate element includes the tenant group 614.

[0241] In this example, the ML model 722 is applied for the first tenant 606 at the tenant group level 604 .

[0242] In this example, the ML model 722 is prohibited from accessing the additional tenant ML model data 1768 (as shown in item 1772).

[0243] Additionally, in this particular example, ML model 722 is (i) allowed access to first tenant ML model data 610 and tenant group ML model data 620 (as shown in items 726 and 724, respectively), and (ii) prohibited from accessing second tenant ML model data 612 and administrator ML model data 1240 (as shown in items 728 and 1246, respectively).

[0244] Multi-tenant processing - Access control at administrator level - Example 3 FIG. 18 shows another example hierarchy 1800.

[0245] In this example, the ML model 1244 is applied for the first tenant 606 at the administrator level 1238 .

[0246] In this example, the ML model 1244 is prohibited from accessing the additional tenant ML model data 1768 (as shown in item 1772).

[0247] Additionally, in this particular example, ML model 1244 is (i) allowed access to first tenant ML model data 610, tenant group ML model data 620, and administrator ML model data 1240 (as shown in items 726, 724, and 1244, respectively), and (ii) prohibited from accessing second tenant ML model data 612 (as shown in item 728).

[0248] Although Figures 14-16 relate to exemplary hierarchies 1400, 1500, 1600 having at least one additional access group 1450 with at least one additional tenant 1452, and Figures 17 and 18 relate separately to exemplary hierarchies 1700, 1800 having at least one additional tenant 1766, other exemplary hierarchies may include both at least one additional access group 1450 with at least one additional tenant 1452 and at least one additional tenant 1766.

[0249] Multi-tenant processing - Predetermined action(s) in response to a tenant leaving the tenant group FIG. 19 shows another example hierarchy 1900.

[0250] FIG. 19 illustrates a scenario in which the first tenant 606 no longer belongs to the tenant group 614.

[0251] The first tenant 606 may have been removed from the tenant group 614 based on one or more removal factors. Examples of removal factors include, but are not limited to, a user request, an insufficient strength of match between the first tenant 606 and the tenant group 614, etc. For example, the first tenant ML model data 610 may represent an average transaction volume associated with the first tenant 606. If the average transaction volume is outside the standard deviation of the average transaction volumes of the other tenants belonging to the tenant group 614, an insufficient strength of match between the first tenant 606 and the tenant group 614 may be determined.

[0252] Here, the first tenant 606 may belong to an administrator (not shown) or may belong to another tenant group (not shown). Additionally, while Figure 19 shows an example in which the first tenant 606 is still part of the hierarchy 1900, in other examples, the first tenant 606 may no longer be part of the hierarchy 1900.

[0253] In some examples, in response to the first and / or second tenant 606, 608 no longer belonging to the tenant group 614, a predetermined action is performed with respect to the tenant group ML model data 620.

[0254] An example of such a predetermined action is updating the tenant group ML model data 620 so that the tenant group ML model data 620 is no longer based on the first tenant ML model data 610 or is based on a limited version of the first tenant ML model data 610.

[0255] For example, if the tenant group ML model data 620 includes a compilation, such as a list of first and second tenant ML model data 610, 612, the first tenant ML model data 610 may be deleted from the compilation.

[0256] For example, if the tenant group ML model data 620 is based on a historical average of the first and second tenant ML model data 610, 612, the first tenant ML model data 610 may no longer be considered for future averaging, but contributions from the historical first tenant ML model data 610 in the tenant group ML model data 620 may still remain for some time.

[0257] Although this particular example concerns a tenant leaving a tenant group, one or more corresponding predetermined actions may be taken in response to a tenant no longer directly belonging to an administrator and / or a tenant group no longer directly belonging to an administrator.

[0258] Multi-tenant processing - Access control at the tenant group level - Example 6 FIG. 20 shows another example hierarchy 2000.

[0259] In this example, the hierarchy 2000 includes a tenant level 602 that includes first and second tenants 606, 608 associated with first and second tenant ML model data 610, 612, respectively.

[0260] In this example, the hierarchy 2000 includes a tenant group level 604 that includes a tenant group 614 to which at least a first tenant 606 belongs.

[0261] The tenant group 614 is associated with tenant group ML model data 620. The tenant group ML model data 620 is based on at least the first tenant ML model data 610.

[0262] In this example, the second tenant 608 may or may not belong to the tenant group 614. This is indicated in FIG. 20 by the dashed arrow 2074 pointing from the second tenant 608 to the tenant group 614.

[0263] For example, the second tenant 608 may belong to the tenant group 614, may belong to an administrator (not shown), may belong to another tenant group (not shown), or may be other.

[0264] In this example, the tenant group ML model data 620 may (i) be based on the first tenant ML model data 610, and (ii) may not be based on the second tenant ML model data 612, or may be based on a limited version of the second tenant ML model data 612.

[0265] In this example, ML model 722 is (i) allowed to access tenant group ML model data 620 and first tenant ML model data 610 (as shown in items 724 and 726, respectively), and (ii) prohibited from accessing second tenant ML model data 612 (as shown in item 728).

[0266] While this particular example relates to ML model 722, in another example, a machine learning model (not shown) being applied at the tenant level 602 for the first tenant 606 may similarly (i) be permitted to access the tenant group ML model data 724 and the first tenant ML model data 610 (as shown in items 724 and 726, respectively), and (ii) be prohibited from accessing the second tenant ML model data 612 (as shown in item 728). The machine learning model (not shown) being applied at the tenant level 602 for the first tenant 606 may not initially be permitted to access the tenant group ML model data 620, but may be granted permission by, for example, enabling one or more permission settings.

[0267] Multitenant Processing - Access Control Methods FIG. 21 illustrates an exemplary access control method 2100.

[0268] The example method 2100 controls access to ML model data in a multi-tenant ML system having an ML model data access hierarchy, which can correspond to one or more example hierarchies described herein or otherwise.

[0269] In this example, the ML model data access hierarchy includes (i) a first tenant 606 associated with first tenant ML model data 610, (ii) a second tenant 608 associated with second tenant ML model data 612, and (iii) a tenant group 614 associated with tenant group ML model data 620.

[0270] As shown by item 2102, at a first time (i) the first tenant 606 belongs to the tenant group 614 and the second tenant 608 does not belong to the tenant group 614, and (ii) the tenant group ML model data 620 is based on the first tenant ML model data 610 and the tenant group ML model data 620 is not based on the second tenant ML model data 612.

[0271] As shown by item 2104, at a second later time (i.e., later than the first time), (i) the first and second tenants 606, 608 belong to a tenant group 614, and (ii) the tenant group ML model data 620 is based on at least the first tenant ML model data 610.

[0272] As indicated by item 2106, at a third later time (ie, later than the second time), the tenant group ML model data 620 is based on the first and second tenant ML model data 610, 612.

[0273] Multitenant Processing - Multitenant Machine Learning System FIG. 22 illustrates an example multi-tenant ML system 2200.

[0274] In this example, system 2200 has access to an ML model data access hierarchy 2202. ML model data access hierarchy 2202 may correspond to one or more example hierarchies described herein or otherwise.

[0275] The hierarchy 2202 includes multiple tenants 2204 and includes one or more tenant groups 2206 .

[0276] The system 2200 includes an ML model data access controller 2208. The ML model data access controller 2208 is configured to control access to the ML model data 2210 according to the ML model data access hierarchy 2202.

[0277] The ML model data access controller 2208 may comprise, be implemented using, or otherwise employ one or more processors and one or more memories to perform the ML model data access control. The one or more memories may comprise instructions that, when executed by the one or more processors, cause the one or more processors to perform the ML model data access control.

[0278] Multi-tenant processing - Machine learning model data access hierarchy - Example 2 FIG. 23 is a diagram illustrating an example hierarchy 2300.

[0279] In this example, the hierarchy includes (i) a tenant group 614 belonging to the administrator 1238 and including first and second tenants 606, 608, (ii) an additional tenant 1766 belonging directly to the administrator 1238, and (iii) a further tenant group 1450 belonging to the administrator 1238 and including a further tenant 1452.

[0280] Multi-tenant processing - Multi-level access control - Example 1 FIG. 24 is a diagram illustrating an example hierarchy 2400.

[0281] The example hierarchy 2400 corresponds to the example hierarchy 2300 described above with reference to FIG. 23, except that the example hierarchy 2400 further illustrates access control for the second tenant 608 using a dashed curve 2476.

[0282] In this example, when an event is processed for the second tenant 608, the active portion of the hierarchy 2400 is indicated by the dashed curve 2476.

[0283] In this particular example, for a model active for the second tenant 608 at the administrator level, the model (i) has access to the administrator ML model data, the tenant group ML model data, and the second tenant ML model data, and (ii) does not have access to the first tenant ML model data, the additional tenant ML model data, the further tenant group ML model data, or the further tenant ML model data.

[0284] In this particular example, for a model active for the second tenant 608 at the tenant group level, the model may (i) have access to the tenant group ML model data and the second tenant ML model data, (ii) have no access to the first tenant ML model data, additional tenant ML model data, further tenant group ML model data, or further tenant ML model data, and (iii) have access to the administrator ML model data.

[0285] In this particular example, for a model active for the second tenant 608 at the tenant level, the model may (i) have access to the second tenant ML model data, (ii) have no access to the first tenant ML model data, additional tenant ML model data, further tenant group ML model data, or further tenant ML model data, and (iii) have access to administrator ML model data and / or tenant group ML model data.

[0286] Thus, in this particular example, none of the models active for the second tenant 608 have access to the first tenant ML model data, the additional tenant ML model data, the further tenant group ML model data, or the further tenant ML model data. As explained above, this provides a high degree of data isolation between different tenants.

[0287] However, in other examples, such a model may be given full or partial access to the first tenant ML model data, additional tenant ML model data, further tenant group ML model data, or further tenant ML model data.

[0288] For example, the business relationship may change between the second tenant 608 and the additional tenant 1766 such that the second tenant 608 may have full or partial access to the additional tenant ML model data. This can provide relaxed data isolation such as between the second tenant 608 and the additional tenant 1766 while still providing strong data isolation for other tenants.

[0289] This possibility of relaxed data separation applies to other examples described herein as well.

[0290] Multi-tenant processing - Multi-level access control - Example 2 FIG. 25 is a diagram illustrating an example hierarchy 2500.

[0291] The example hierarchy 2500 corresponds to the example hierarchy 2300 described above with reference to FIG. 23, except that the example hierarchy 2500 further illustrates access control for an additional tenant 14520 using a dashed curve 2578.

[0292] In this example, the active portion of hierarchy 2500 is indicated by dashed curve 2578 as events are processed for additional tenants 1452 .

[0293] In this particular example, for a model active for further tenant 1452 at the administrator level, the model (i) has access to the administrator ML model data, the further tenant group ML model data, and the further tenant ML model data, and (ii) does not have access to the first tenant ML model data, the second tenant ML model data, the tenant group ML model data, or the additional tenant ML model data.

[0294] In this particular example, for a model active for further tenant 1452 at the tenant group level, the model may (i) have access to further tenant group ML model data and further tenant ML model data, (ii) have no access to the first tenant ML model data, the second tenant ML model data, the tenant group ML model data, or the additional tenant ML model data, and (iii) have access to administrator ML model data.

[0295] In this particular example, for a model active for further tenant 1452 at the tenant level, the model may (i) have access to further tenant ML model data, (ii) have no access to first tenant ML model data, second tenant ML model data, tenant group ML model data, or additional tenant ML model data, and (iii) have access to administrator ML model data and / or further tenant group ML model data.

[0296] Multi-tenant processing - additional technical effects As explained above, dividing systems by use case or internal organization can involve multiple integrations and multiple systems being stitched together, which can lead to difficulties in the initial implementation and when new tenants are added.

[0297] In contrast, the example described here keeps analysis and composition clearly separated.

[0298] Additionally, in contrast to systems where rules must directly reference specific subsets of data, the tenant groups described herein provide clean logical buckets under which tenants reside.

[0299] The embodiment also effectively provides a layer of analytical strategy to the tenant group.

[0300] The exemplary multi-tenancy solution described herein is particularly flexible: some users may have view-only privileges, while other users (e.g., with their own fraud teams) may be able to create and edit their own rules, all on the same solution.

[0301] Additionally, the solution can be adapted to leverage consortium data across other users, or, where applicable, to exclude them if they may affect the performance of the model.

[0302] Furthermore, a given user can offer the services described herein to other users (e.g., using tenant groups), leveraging their own fraud solutions with respect to other users, and not just using their own fraud solutions to solve their own fraud-related issues.

[0303] Therefore, learning to create tenant groups with multiple tenants and cascade across groups without having to create separate instances of the system is an effective, targeted, and configurable solution.

[0304] Machine learning generally involves large amounts of data, and the hierarchy described herein allows for efficient access to different portions of such data.

[0305] Data isolation may also be particularly relevant to machine learning data, especially in the context of fraud detection. Examples described herein use hierarchies to leverage the efficient data access they provide, while also providing controlled access to machine learning data. In particular, the hierarchies described herein may enable complete data isolation between different tenants while allowing higher-level entity access to ML model data for multiple different tenants.

[0306] Multi-tenant processing - example use cases Non-limiting example use cases of the systems and methods described herein are provided.

[0307] In this example, the hierarchy includes an administrator, a first tenant group including first and second tenants, and a second tenant group including third and fourth tenants.

[0308] In this example, the first and second tenants are businesses in the same business sector, and the first tenant group is associated with the business sector to which the first and second tenants are associated. Similarly, the third and fourth tenants are businesses in the same business sector, and the second tenant group is associated with the business sector to which the third and fourth tenants are associated.

[0309] Different business sectors may be associated with different levels of fraud risk, may have different rules, may have different data maturity characteristics, and so on.

[0310] The first tenant group tenant ML model data associated with the first tenant group can be based on the first and second tenant ML model data associated with the first and second tenants, respectively. Similarly, the second tenant group tenant ML model data associated with the second tenant group can be based on the third and fourth tenant ML model data associated with the third and fourth tenants, respectively.

[0311] In an example, a model running for a given tenant cannot directly access another tenant's ML model data, but the model may be able to access the ML model data of other tenants and / or tenant groups, including administrator ML model data, providing effective data isolation within a single multi-tenant platform.

[0312] For example, an analyst associated with a first tenant may be able to perform analysis based on the first tenant ML model data, but may not be able to perform analysis based on the second tenant ML model data.

[0313] However, an analyst associated with the first tenant group (e.g., an account manager for the business sector to which the first and second tenants are associated) may be able to perform analysis based on the first tenant ML model data, the second tenant ML model data, and the first tenant group ML model data.

[0314] In this example, over time the characteristics of the third tenant business change to become more similar to the first and second tenant businesses.

[0315] Thus, the third tenant may be moved to the first tenant group such that the first, second, and third tenants belong to the first tenant group, and the fourth tenant belongs to the fourth tenant group.

[0316] It should be appreciated that a third tenant can be easily moved from the second tenant group to the first tenant group by updating data such as metadata so that the third tenant is associated with the first tenant group instead of the second tenant group. This movement occurs within a single platform as described herein, which is not possible when different installations are used for different tenants.

[0317] The third tenant may now have characteristics that are more closely aligned with the characteristics of the first and second tenants, but the third tenant ML model data may comprise at least some historical data. If the first tenant group ML model data is updated immediately upon the third tenant joining to include all of the third tenant ML model data (particularly the historical data), the first tenant group ML model data may be less representative of the (current) first, second, and third tenant ML model data than if the historical third tenant ML model data had not been used or had been used in a limited way.

[0318] Over time, assuming the characteristics of the third tenant become more similar to the characteristics of the first and second tenants, a higher (e.g., complete) contribution from the third tenant ML model data may be used for the first tenant group ML model data.

[0319] Similarly, when a third tenant is no longer in the second tenant group, the contribution of the third tenant ML model data to the second tenant group ML model data may be removed or reduced. For example, if the third tenant ML model data was {m_31, m_32, m_33,...}, the fourth tenant ML model data was {m_41, m_42, m_43,...}, and the second tenant group ML model data was a compilation of the third and fourth tenant ML model data, e.g., {m_31, m_32, m_33,...; m_41, m_42, m_43,...}, some or all of the third tenant ML model data could be easily deleted from the second tenant group ML model data, with immediate effect when the third tenant leaves the second tenant group.

[0320] Multi-tenant processing - additional features In the above example, a given tenant either belongs to a single tenant group or directly to the administrator, in other words, in such an example, a tenant can only have one parent.

[0321] If multiple types of analysis are being performed, each tenant may be represented using multiple different tenant IDs for different analytical hierarchies within the system.

[0322] More generally, however, a given tenant may belong to one or more tenant groups and / or may belong directly to an administrator (e.g., a given tenant may belong to two different tenant groups, may belong to one tenant group, and may also belong directly to an administrator, etc.).

[0323] In the above examples, the hierarchy includes two, three, or four (logical) levels, however, the hierarchy may include a different number of levels in other examples.

[0324] The above examples relate to models in the form of machine learning models.

[0325] In other examples, another type of model, more specifically, a learning model, may be used. An example of another type of learning model is a rule-based learning model. An example of a rule-based learning model is an adaptive rule set model, which may also be referred to as a rule-based adaptive model. Thus, although the examples described above relate to machine learning models, they may also be applied to other types of learning models, such as, but not limited to, adaptive rule sets. For example, references to a "machine learning model," including in terms such as "machine learning model data," should be understood accordingly.

[0326] Although particular embodiments of the present invention have been described in detail, those skilled in the art will appreciate that the embodiments described in detail do not limit the scope of the invention as set forth in the claims. [Explanation of symbols]

[0327] 100 Transaction Processing Systems 102 Transaction Processing System 104 Transaction Processing Systems 110 client devices 110-A Smartphone 110-B Computer 110-C Point of Sale (POS) System 110-D Portable Merchant Device 120 Computer Networks 120-A First Computer Network 120-B Second Network 128 size 130 Merchant Server 140 processor server 142 first data storage device 144 second data storage device 146 Transaction Data 148 Ancillary Data 150 Machine Learning Servers 160 Machine Learning Systems 162 input data 164 output data 170 Data storage device 170-A~170-N Local Data Storage Devices 172 Ancillary Data 172-A~172-N Auxiliary Data 200 Data Storage Configuration 202 Data Storage Configuration 210 Machine Learning Systems 220 Data Bus 230 Data storage device 232 Data Storage Devices 240 historical transaction data 240-A~N Historical Transaction Data 242 Ancillary Data 242-A~N Auxiliary Data 250 data transfer channels 260 Data Storage Device 270 Secure Logical Storage Layer 280 Encrypted Partition 300 chronological records 310 Transaction Data 320 Historical Transaction Data 330 Transaction Data 402 Machine Learning Systems 410 Input Data 412 Input Data Interface 414 Machine Learning Model Platform 416 Scalar Output 418 Alarm System 420 Output Data Interface 422 Output Data 500 Transaction Process Flow 502 User Device 503 Devices 504 Seller System 505 devices 506 Machine Learning Systems 506 Payment Processor (PP) System 507 devices 508 Machine Learning (ML) Systems 509 devices 510 Issuing Bank System 532 Issuing Bank System 550 Transaction Process Flow 600 Machine Learning Model Data Access Hierarchy 602 First Level 604 Second Level 606 First Tenant 608 Second Tenant 610 First Tenant ML Model Data 612 Second Tenant ML Model Data 614 Tenant Group ML Model Data 616 Arrow 618 Arrow 620 Tenant Group ML Model Data 700 layers 700 Machine Learning Model Data Access Hierarchy 722 ML model 724 items 726 items 728 items 800 layers 830 ML model 900 layers 932 ML model 1000 layers 1034 ML model 1100 layers 1200 layers 1236 Third Level 1238 Administrator 1240 Administrator ML Model Data 1242 Arrow 1244 ML model 1246 items 1300 layers 1348 ML model 1400 layers 1450 Tenant Group 1452 Tenant 1454 Arrow 1458 Tenant Group ML Model Data 1460 Tenant ML model data 1500 layers 1562 items 1564 items 1600 layers 1700 layers 1766 Tenant 1768 additional tenant ML model data 1770 Arrow 1772 items 1800 layers 1900 layers 2000 layers 2074 dashed arrow 2100 Access Control Methods 2200 Multi-tenant ML System 2202 ML Model Data Access Hierarchy 2204 Tenant 2206 Tenant Group 2208 ML Model Data Access Controller 2210 ML model data 2300 layers 2400 layers 2476 Curve 2500 layers 2578 Curve

Claims

1. 1. A method for controlling access to machine learning model data in a multi-tenant machine learning system having a machine learning model data access hierarchy, the machine learning model data access hierarchy comprising: a tenant level comprising first and second tenants associated with first and second tenant machine learning model data, respectively; a tenant group level comprising a tenant group to which the first and second tenants belong, the tenant group being associated with tenant group machine learning model data, the tenant group machine learning model data being based on the first and / or second tenant machine learning model data; The method, with respect to a machine learning model being applied with respect to the first tenant at the tenant group level, allowing access to the tenant group machine learning model data and the first tenant machine learning model data; prohibiting access to the second tenant machine learning model data; and A method comprising:

2. the tenant group machine learning model data is based on the first tenant machine learning model data, the tenant group machine learning model data is not based on the second tenant machine learning model data or is based on a restricted version of the second tenant machine learning model data; The method of claim 1.

3. The method of claim 1 or 2, wherein the second tenant joined the tenant group after the first tenant joined the tenant group.

4. the tenant group machine learning model data is based on at least the first tenant machine learning model data historical set; the tenant group machine learning model data is not based on at least the historical set of the second tenant machine learning model data or is based on a limited version of the historical set of the second tenant machine learning model data; 4. The method according to any one of claims 1 to 3.

5. The method of claim 4 , wherein the historical set of the second tenant machine learning model data was generated before the second tenant joined the tenant group.

6. 6. The method of claim 1, wherein the tenant group machine learning model data is updated to generate updated tenant group machine learning model data, the updated tenant group machine learning model data being based on both the first and second tenant machine learning model data.

7. The method of claim 1 , wherein the tenant group machine learning model data is based on at least some of each of the first and second machine learning model data.

8. 8. The method of claim 7, wherein the tenant group machine learning model data is derived based on one or more operations involving the at least some of each of the first and second machine learning model data.

9. The method, with respect to a machine learning model being applied with respect to the second tenant at the tenant group level, allowing access to the tenant group machine learning model data and the second tenant machine learning model data; prohibiting access to the first tenant machine learning model data; 9. The method of claim 1, comprising:

10. 10. The method of claim 1, wherein the method comprises prohibiting access to the tenant group machine learning model data for machine learning models that have been applied with respect to the first and / or second tenant at the tenant level.

11. 10. The method of claim 1, wherein the method comprises selectively granting access to the tenant group machine learning model data for machine learning models applied with respect to the first and / or second tenant at the tenant level based on one or more permission settings enabled for the machine learning models applied with respect to the first and / or second tenant at the tenant level.

12. 12. The method of claim 1, wherein the machine learning model data access hierarchy includes an administrator level at which machine learning models are applicable for the first tenant, the administrator level includes an administrator associated with administrator machine learning model data, and the tenant group belongs to the administrator.

13. The method, with respect to the machine learning model being applied with respect to the first tenant at the administrator level, allowing access to the administrator machine learning model data, the tenant group machine learning model data, and the first tenant machine learning model data; prohibiting access to the second tenant machine learning model data; and 13. The method of claim 12, comprising:

14. 14. The method of claim 12 or 13, wherein the tenant group level includes a further tenant group to which at least one further tenant belongs and to which the first and second tenants do not belong, the further tenant group belonging to the administrator, the further tenant group associated with further tenant group machine learning model data, and the at least one further tenant associated with further tenant machine learning model data.

15. 15. The method of claim 14, wherein the method comprises prohibiting access to the further tenant group machine learning model data and / or the further tenant machine learning model data for the machine learning model being applied with respect to the first tenant at the tenant group level.

16. 16. The method of claim 14 or 15, wherein the method comprises prohibiting access to the further tenant group machine learning model data and / or the further tenant machine learning model data for the machine learning model being applied for the first tenant at the administrator level.

17. 17. The method of claim 12, wherein the machine learning model data access hierarchy includes an additional tenant associated with additional tenant machine learning model data, the additional tenant belonging to the administrator.

18. 20. The method of claim 17, wherein the method includes prohibiting access to the additional tenant machine learning model data for the machine learning model being applied with respect to the first tenant at the tenant group level.

19. 19. The method of claim 17 or 18, wherein the method comprises prohibiting access to the additional tenant machine learning model data for the machine learning mode applied for the first tenant at the administrator level.

20. 20. The method of claim 1, wherein the method comprises performing a predetermined action with respect to the tenant group machine learning model data in response to the first and / or second tenant no longer belonging to the tenant group.

21. 20. The method of claim 1, wherein the machine learning model data comprises machine learning model state data and / or machine learning model parameter data.

22. 21. The method of claim 1, wherein the tenant group machine learning model data and / or the first tenant machine learning model data is used to perform real-time anomaly detection associated with the first tenant.

23. 1. A method for controlling access to learning model data in a multi-tenant system having a learning model data access hierarchy, the learning model data access hierarchy comprising: a tenant level comprising first and second tenants associated with first and second tenant learning model data, respectively; a tenant group level comprising a tenant group to which at least the first tenant belongs, the tenant group being associated with tenant group learning model data, and the tenant group learning model data being based on at least the first tenant learning model data; The method further comprises: with respect to a machine learning model being applied with respect to the first tenant at the tenant group level; and / or with respect to a learning model being applied with respect to the first tenant at the tenant level; allowing access to the tenant group learning model data and the first tenant learning model data; prohibiting access to the second tenant learning model data; A method comprising:

24. 1. A method for controlling access to machine learning model data in a multi-tenant machine learning system having a machine learning model data access hierarchy, the machine learning model data access hierarchy comprising: a first tenant associated with first tenant machine learning model data; a second tenant associated with the second tenant machine learning model data; Tenant Groups associated with machine learning model data; and Equipped with The method comprises: At a first time, the first tenant belongs to the tenant group, and the second tenant does not belong to the tenant group; the tenant group machine learning model data is based on the first tenant machine learning model data, and the tenant group machine learning model data is not based on the second tenant machine learning model data; At a second later time, the first and second tenants belong to the tenant group; the tenant group machine learning model data is based on at least the first tenant machine learning model data; At a time after the third, controlling access to machine learning model data such that the tenant group machine learning model data is based on the first and second tenant machine learning model data.

25. A multi-tenant machine learning system configured to perform the method of any one of claims 1 to 24.

26. 1. A multi-tenant machine learning system having access to a machine learning model data access hierarchy comprising multiple tenants and one or more tenant groups, the multi-tenant machine learning system comprising a machine learning model data access controller configured to control access to machine learning model data in accordance with the machine learning model data access hierarchy.

Images