Access control

Abstract

A method of controlling access to machine learning model data in a multi-tenant machine learning system having a machine learning model data access hierarchy 700. The hierarchy 700 includes: (i) a tenant level 602 including a first tenant 606 and a second tenant 608 associated with first tenant machine learning model data 610 and second tenant machine learning model data 612, respectively; and (ii) a tenant group level 604 including a tenant group 614 to which the first tenant 606 and the second tenant 608 belong. The tenant group 614 is associated with tenant group machine learning model data 620. The tenant group machine learning model data 620 is based on the first tenant machine learning model data 610 and / or the second tenant machine learning model data 612. The method comprises: for a machine learning model 722 applied at the tenant group level 604 with respect to a first tenant 606: i) allowing 724, 726 access to the tenant group machine learning model data 620 and the first tenant machine learning model data 610; and (ii) prohibiting 728 access to the second tenant machine learning model data 612.

Description

Technical Field

[0001] The present disclosure relates to access control.

[0002] In particular, but not exclusively, the present disclosure relates to a method of controlling access to machine learning model data in a multi-tenant machine learning system having a machine learning model data access hierarchy and a multi-tenant machine learning system.

[0003] In various examples, these methods and systems are used to detect anomalies in data patterns, such as detecting fraudulent transactions. Certain examples relate to real-time transaction processing methods and systems. Background Art

[0004] Digital payments have exploded over the past two decades, with more than three-quarters of payments worldwide using some form of payment card or electronic wallet. Point-of-sale systems are increasingly becoming digital and less cash-based. Simply put, the global commercial system now relies heavily on electronic data processing platforms. This presents a number of engineering challenges that are largely hidden from lay users. For example, digital transactions need to be completed in real time, i.e., with minimal levels of latency experienced by computer devices at the time of purchase. Digital transactions also need to be secure and resistant to attacks and exploits. The processing of digital transactions is also constrained by the historical development of the global electronic payment system. For example, much of the infrastructure is still configured around models that were designed for mainframe architectures more than 50 years ago.

[0005] As digital transactions increase, new security risks become apparent. Digital transactions provide new opportunities for fraud and malicious activity. In 2015, an estimated 7% of digital transactions were fraudulent, and as more economic activity moves online, this number will only increase. Fraud losses are estimated to be four times the world's population (e.g., in US dollars) and are growing.

[0006] Financial institutions are becoming subject to more regulatory scrutiny because traditional methods of fraud prevention, such as identity authentication (e.g., passwords, digital biometrics, national IDs, etc.), have proven to be ineffective in preventing fraud tactics such as synthetic identities and spoofing. These much more sophisticated threat tactics regarding fraud require significantly more analysis in a very short time (less than 50 milliseconds (ms)) and are often based on a much smaller data sample size regarding the fraud or scam itself. This presents significant technical challenges.

[0007] While risks like fraud are economic issues for companies involved in business, the specific implementation of technical systems for processing transactions is an engineering challenge. Traditionally, banks, merchants, and card issuers have developed "paper" rules or procedures that are manually implemented by staff to mark or block certain transactions. As transactions become digital, one approach to building technical systems for processing transactions is to provide a collection of these developed criteria to computer engineers and require computer engineers to implement these developed criteria using digital representations of transactions, that is, to convert handwritten rules into decoded logical statements that can be applied to electronic transaction data. With the growth of digital transaction volumes, this traditional approach has encountered several problems. First, any application processing needs to occur in "real time", for example, where latency is measured in milliseconds. Second, thousands of transactions need to be processed per second (for example, a common "load" can be 1000 to 2000 per second), where the load changes unexpectedly over time (for example, the introduction of a new product or a set of tickets can easily increase the average load level several times). Third, for security reasons, transaction processors and banks' digital storage systems are usually separated or divided, but digital transactions usually involve an interconnected network of merchant systems. Fourth, large-scale analysis of both actual reported fraud and predicted fraud is now possible. This shows that traditional methods for fraud detection are subpar; accuracy is low and false positives are high. This has a real impact on digital transaction processing, with more genuine point-of-sale and online purchases being rejected, while those who try to exploit new digital systems are often successful.

[0008] In recent years, stronger machine learning approaches have been adopted to process transaction data. As machine learning models mature in academia, engineers have begun to try to apply them to the processing of transaction data. However, this has once again encountered problems. Even if an academic or theoretical machine learning model is given to engineers and they are asked to implement it, it is not a simple matter. For example, the problems of large-scale transaction processing systems begin to emerge. Machine learning models do not enjoy unlimited inference time as in the laboratory. This means that some models are impractical to implement in a real-time environment, or they need to be significantly adapted to allow real-time processing at the magnitude experienced by real-world servers. In addition, engineers need to deal with the problem of implementing machine learning models on data that is separated or partitioned based on access security and when the data is updated at a very fast speed. Therefore, the problems faced by engineers building transaction processing systems can be regarded as similar to those faced by network or database engineers; machine learning models need to be applied, but the system throughput and query response time constraints set by the processing infrastructure must be met. There are no easy solutions to these problems. The fact that many transaction processing systems are confidential, proprietary, and based on old technologies means that engineers do not have the knowledge system developed in these adjacent fields and often face challenges unique to the field of transaction processing. Furthermore, the field of large-scale practical machine learning is still young, and there are few established design patterns or textbooks that engineers can rely on.

[0009] Fraud detection systems generate risk scores for new transactions in real time (i.e., latency <= 100 milliseconds), where a high risk score indicates the likelihood of fraud, error, or unauthorized use of an account. Financial institutions use these scores within their decision logic when deciding whether to approve a transaction (typically, this will reject the transaction if the risk score is above a certain threshold). Financial institutions typically want the set of transactions above a certain risk threshold to contain as high a proportion of fraud as possible (i.e., providing a "maximum detection rate"), as well as as low a false positive classification rate as possible (i.e., providing a "maximum precision").

[0010] One method known in the art involves using machine learning techniques such as recurrent neural networks (RNNs) to detect anomalies in behavioral patterns. Those skilled in the art will appreciate that the output of an RNN is the next state, and that this next state depends on the previous state and the inputs provided to the RNN. In this way, the state of the RNN is based not only on instantaneous information related to a given transaction, but also on the history of transactions that have been conducted before, thereby retaining a memory of patterns that have been observed. This can be used to detect abnormal behavior that may indicate fraudulent transactions. The term "behavior" is used herein to refer to a pattern of activity or action.

[0011] Detection of such anomalous activity may provide improvements to system security when compared to "traditional" methods of fraud prevention (e.g., passwords, digital biometrics, national IDs, etc.) Of course, it should be understood that behavioral pattern analysis techniques may be used in combination with one or more of these "traditional" methods, where appropriate.

[0012] Generally speaking, a rule may require that historical data (eg, state data) related to a particular entity be stored for, for example, a particular duration.

[0013] Systems that utilize state data stored in and retrieved from a database (e.g., a machine learning system using an RNN) will typically require that the data stored in that database be updateable in response to incoming events (e.g., a new transaction in a transaction processing system). A transaction or deposit occurring on a card or merchant typically causes one or more events to be transmitted to the system.

[0014] In general, state data may be stored in a database using any suitable structured digital representation, such as a table of vectors or tensors.

[0015] In practice, tenants (e.g., direct or indirect customers of a fraud detection platform) may want a platform to handle multiple different types of data. This is not only to simplify deployment, but also to coordinate third-party components across their organization and to minimize operational licensing costs.

[0016] One way to deploy such a system is to have multiple separate installations of the same software running, with namespaces and external integrations making them appear to be a single system to the outside world.

[0017] However, this can lead to a complex final solution with multiple components all interacting in different ways. When wanting to share data between different installations, combining such data into a coherent view is fraught with difficulties in trying to make these individual data elements work together. Summary of the invention

[0018] According to a first aspect, there is provided a method for controlling access to machine learning model data in a multi-tenant machine learning system having a machine learning model data access hierarchy, the machine learning model data access hierarchy comprising:

[0019] a tenant level, the tenant level comprising a first tenant and a second tenant associated with the first tenant machine learning model data and the second tenant machine learning model data, respectively; and

[0020] a tenant group level, the tenant group level comprising a tenant group to which the first tenant and the second tenant belong, wherein the tenant group is associated with tenant group machine learning model data, and wherein the tenant group machine learning model data is based on the first tenant machine learning model data and / or the second tenant machine learning model data,

[0021] The method includes: for a machine learning model applied with respect to a first tenant at a tenant group level,

[0022] Allowing access to the tenant group machine learning model data and the first tenant machine learning model data; and

[0023] Access to the second tenant's machine learning model data is prohibited.

[0024] A multi-tenant machine learning system enables deployment of a single system that supports multiple tenants. This can greatly improve deployment and maintenance efficiency compared to separate installers for separate tenants and / or separate data types.

[0025] For example, again compared to having separate installers for separate tenants, the machine learning model data access hierarchy provides a computationally efficient mechanism to separate first tenant machine learning model data and second tenant machine learning model data within a single system (e.g., in terms of direct tenant access). The separation may be particularly effective for machine learning model data. Additionally, the computational efficiency gains are significant in the context of machine learning where relatively large amounts of machine learning model data may be used.

[0026] The tenant group machine learning model data may be based on the first tenant machine learning model data and the second tenant machine learning model data, such as when the first tenant and the second tenant have been in the tenant group for an extended period of time.

[0027] Alternatively, the tenant group machine learning model data may be based only on one of the first tenant machine learning model data and the second tenant machine learning model data, such as when one of the first tenant and the second tenant has recently joined the tenant group.

[0028] In either case, the tenant group machine learning model data provides abstracted data about the first tenant machine learning model data and the second tenant machine learning model data, which can be used, for example, for abstracted analysis.

[0029] In some examples, the tenant group machine learning model data is based on the first tenant machine learning model data; and the tenant group machine learning model data is not based on the second tenant machine learning model data, or the tenant group machine learning model data is based on a restricted version of the second tenant machine learning model data.

[0030] In some examples, after the first tenant joins the tenant group, the second tenant joins the tenant group.

[0031] In such examples, the second tenant machine learning model data makes no contribution or a minimal contribution to the tenant group machine learning model data. This may be particularly valid in situations where the second tenant has recently joined the tenant group (e.g., from another tenant group). The contribution of the second tenant machine learning model data to the tenant group machine learning model data may be lower than the contribution of the first tenant machine learning model data to the tenant group machine learning model data, consistent with the amount of time the first tenant and the second tenant have been members of the tenant group.

[0032] In some cases, the longer a given tenant belongs to a given tenant group, the stronger the characteristic match between the given tenant and the given tenant group, and therefore, the greater the influence of the given tenant machine learning model data associated with the given tenant on the given tenant group machine learning model data associated with the given tenant group.

[0033] In some examples, the tenant group machine learning model data is based on at least a historical set of the first tenant machine learning model data; and the tenant group machine learning model data is not based on at least a historical set of the second tenant machine learning model data, or the tenant group machine learning model data is based on a restricted version of the historical set of the second tenant machine learning model data.

[0034] In some examples, the historical set of second tenant machine learning model data was generated before the second tenant joined the tenant group.

[0035] In such an example, the historical set of the second tenant machine learning model data may be less representative of the current state associated with the second tenant than the most recent set of the second tenant machine learning model data. Thus, the contribution of the less representative historical second tenant machine learning model data to the tenant group machine learning model data may be limited. This in turn may result in more accurate tenant group machine learning model data.

[0036] In some examples, the tenant group machine learning model data is updated to produce updated tenant group machine learning model data, and the updated tenant group machine learning model data is based on both the first tenant machine learning model data and the second tenant machine learning model data.

[0037] In such examples, the influence of the second tenant machine learning model data on the tenant group machine learning model data may increase, for example, over time, while the second tenant remains part of the tenant group. Such contributions may be unrestricted.

[0038] In some examples, the tenant group machine learning model data is based on at least some of each of the first machine learning model data and the second machine learning model data.

[0039] Thus, the tenant group machine learning model data is based at least to some extent on both the first machine learning model data and the second machine learning model data. This can be particularly effective where the first tenant and the second tenant have been in the tenant group for an extended period of time.

[0040] In some examples, the tenant group machine learning model data is derived based on one or more operations involving at least some of each of the first machine learning model data and the second machine learning model data.

[0041] In such examples, a certain degree of combination and / or aggregation may be performed to provide a higher level of abstraction of the first machine learning model data and the second machine learning model data. This may, for example, facilitate a higher level analysis of the first machine learning model data and the second machine learning model data.

[0042] In some examples, the method includes: for a machine learning model applied to a second tenant at a tenant group level, allowing access to a tenant group machine learning model and a second tenant machine learning model; and prohibiting access to a first tenant machine learning model.

[0043] In such examples, the first tenant and the second tenant can benefit from the application of the machine learning model described herein while maintaining strong data separation and isolation between the first tenant and the second tenant.

[0044] In some examples, the method includes: for a machine learning model applied at the tenant level with respect to a first tenant and / or a second tenant, prohibiting access to tenant group machine learning model data.

[0045] In such an example, access control can prevent models at the tenant level from accessing the tenant group machine learning model data. This can enhance the data separation between the first tenant and the second tenant.

[0046] In some examples, the method includes: for a machine learning model applied at the tenant level with respect to a first tenant and / or a second tenant, selectively allowing access to tenant group machine learning model data based on one or more permission settings enabled for the machine learning model applied at the tenant level with respect to the first tenant and / or the second tenant.

[0047] In such examples, access to machine learning model data for a tenant group can be selectively enabled, for example, by enabling relevant permissions. This can improve anomaly detection, for example, by having a more comprehensive view of behavior in the hierarchy.

[0048] In some examples, the machine learning model data access hierarchy includes an administrator level, the machine learning model is applicable with respect to the first tenant at the administrator level, the administrator level includes an administrator associated with the administrator machine learning model data, and the tenant group belongs to the administrator.

[0049] In some examples, the method includes: for a machine learning model applied at an administrator level to a first tenant, allowing access to administrator machine learning model data, tenant group machine learning model data, and first tenant machine learning model data; and prohibiting access to second tenant machine learning model data.

[0050] In some examples, further levels of control, analysis, and / or hierarchy may be provided.

[0051] In some examples, the tenant group level includes at least one additional tenant group to which the first tenant and the second tenant do not belong, the additional tenant group belongs to an administrator, the additional tenant group is associated with the additional tenant group machine learning model data, and at least one additional tenant is associated with the additional tenant machine learning model data.

[0052] In such examples, different abstractions in the form of tenant groups may be available in the hierarchy. Different abstractions may be associated with different tenant group properties and / or different tenant properties.

[0053] In some examples, the method includes: for a machine learning model applied to a first tenant at a tenant group level, prohibiting access to additional tenant group machine learning model data and / or additional tenant machine learning model data.

[0054] In some examples, the method includes: for a machine learning model applied at an administrator level with respect to a first tenant, prohibiting access to additional tenant group machine learning model data and / or additional tenant machine learning model data.

[0055] In such examples, a more flexible hierarchy is provided while still providing strong data separation.

[0056] In some examples, the machine learning model data access hierarchy includes an additional tenant associated with the additional tenant machine learning model data, and the additional tenant belongs to the administrator.

[0057] In such examples, additional tenants may not exhibit strong matching characteristics with the tenant group and / or another tenant, and may belong directly to the administrator. Requiring all tenants to belong to a certain tenant group may result in less accurate machine learning model data because if a given tenant were to join a given tenant group, a given additional tenant that does not exhibit strong matching characteristics with the given tenant group may skew the given tenant group machine learning model data associated with the given tenant group.

[0058] In some examples, the method includes: for a machine learning model applied with respect to a first tenant at a tenant group level, prohibiting access to additional tenant machine learning model data.

[0059] In some examples, the method includes: for a machine learning model applied at an administrator level with respect to a first tenant, disabling access to additional tenant machine learning model data.

[0060] In such examples, a more flexible hierarchy is provided while still providing strong data separation.

[0061] In some examples, the method includes performing a predetermined action with respect to the tenant group machine learning model data in response to the first tenant and / or the second tenant no longer belonging to the tenant group.

[0062] In such an example, when the first tenant and / or the second tenant leaves the tenant group, a predetermined action may be taken. For example, the machine learning model data associated with the leaving tenant may be removed immediately or reduced immediately or over time.

[0063] In some examples, the machine learning model data includes machine learning model state data and / or machine learning model parameter data.

[0064] In such examples, the machine learning model data access hierarchy provides particularly effective access control to machine learning data, which generally exists in large amounts. Efficient resource usage in such contexts can be particularly effective.

[0065] In some examples, the tenant group machine learning model data and / or the first tenant machine learning model data are used to perform real-time anomaly detection associated with the first tenant.

[0066] The efficiencies provided by the machine learning model data access hierarchy are particularly effective in examples like this involving real-time detection.

[0067] According to a second aspect, there is provided a method for controlling access to learning model data in a multi-tenant system having a learning model data access hierarchy, the learning model data access hierarchy comprising:

[0068] a tenant level, the tenant level including a first tenant and a second tenant associated with the first tenant learned model data and the second tenant learned model data, respectively; and

[0069] a tenant group level, the tenant group level comprising a tenant group to which at least the first tenant belongs, wherein the tenant group is associated with tenant group learning model data, and wherein the tenant group learning model data is based on at least the first tenant learning model data,

[0070] The method includes: for a learning model for a first tenant application at a tenant group level and / or for a learning model for a first tenant application at a tenant level,

[0071] allowing access to the tenant group learning model data and the first tenant learning model data; and

[0072] Access to the second tenant's learning model data is prohibited.

[0073] Thus, with respect to other example methods, systems, and hierarchies described herein, even greater applicability, functionality, and efficiency may be provided.

[0074] The learning model may include a machine learning model and / or another type of learning model as described herein.

[0075] According to a third aspect, there is provided a method for controlling access to machine learning model data in a multi-tenant machine learning system having a machine learning model data access hierarchy, the machine learning model data access hierarchy comprising:

[0076] a first tenant, the first tenant being associated with the first tenant machine learning model data;

[0077] a second tenant associated with the second tenant machine learning model data; and

[0078] A tenant group, the tenant group is associated with tenant group machine learning model data, the method comprising controlling

[0079] Restrict access to machine learning model data so that:

[0080] At the first time:

[0081] The first tenant belongs to a tenant group and the second tenant does not belong to the tenant group; and the tenant group machine learning model data is based on the first tenant machine learning model data, and the tenant group machine learning model data is not based on the second tenant machine learning model data;

[0082] At a later second time:

[0083] The first tenant and the second tenant belong to a tenant group; and the tenant group machine learning model data is based on at least the first tenant machine learning model data; and

[0084] At a later third time:

[0085] The tenant group machine learning model data is based on the first tenant machine learning model data and the second tenant machine learning model data.

[0086] Thus, delayed or interleaved use of tenant machine learning model data may be provided. This may enable accurate tenant group machine learning model data depending on the characteristics of different tenants at different times.

[0087] According to a fourth aspect, a multi-tenant machine learning system is provided, which is configured to execute the method provided according to the first embodiment, the second embodiment and / or the third embodiment.

[0088] According to a fifth aspect, a multi-tenant machine learning system having access to a machine learning model data access hierarchy including multiple tenants and one or more tenant groups is provided, the multi-tenant machine learning system including a machine learning model data access controller configured to control access to machine learning model data according to the machine learning model data access hierarchy.

[0089] Examples like these exploit the efficiency of the machine learning model data access hierarchy in the context of machine learning data.

[0090] Embodiments of the present invention may be applied to a wide variety of digital transactions, including but not limited to card payments, so-called "wire" transfers, peer-to-peer payments, Bank Automated Clearing System (BACS) payments, and Automated Clearing House (ACH) payments. The output of the machine learning system may be used to prevent a wide variety of fraud and crime, such as card fraud, application fraud, payment fraud, merchant fraud, gaming fraud, and money laundering.

[0091] It will be appreciated that optional features described above in relation to embodiments of any aspect of the invention are equally applicable to other aspects of the invention where technically appropriate.

[0092] Where technically appropriate, embodiments of the invention may be combined. Embodiments are described herein as comprising certain features / elements. The disclosure also extends to separate embodiments consisting of or essentially consisting of the features / elements.

[0093] One or more methods according to any embodiment of any aspect of the invention may be computer-implemented.

[0094] The various aspects of the invention described above each extend to a non-transitory computer-readable medium comprising instructions which, when executed by a processor, cause the processor to perform the method of such aspect of the invention.

[0095] The aspects of the invention described above each further extend to a computer software product comprising instructions which, when executed by a processor, cause the processor to perform the method of such aspect of the invention.

[0096] Technical references, such as patents and applications, are incorporated herein by reference.

[0097] Any embodiment specifically and explicitly recited herein may form the basis of a disclaimer either alone or in combination with one or more further embodiments.

[0098] In the context of this specification, "comprising" is to be interpreted as "including".

[0099] Aspects of the invention comprising certain elements are also intended to extend to alternative embodiments "consisting of" or "consisting essentially of" the relevant elements.

[0100] The term "memory" or "memory repository" should be understood to mean any component suitable for the storage of data and includes both volatile and non-volatile memory as appropriate for the intended application. Those skilled in the art will appreciate that this includes, but is not limited to, random access memory (RAM), static RAM (SRAM), dynamic RAM (DRAM), magnetic storage devices, solid-state storage devices, and flash memory. It should be understood that combinations of one or more of these may also be used for the storage of data where technically appropriate (e.g., using faster access volatile memory for frequently accessed data).

[0101] The term "data" is used in different contexts herein to refer to digital information, such as digital information represented by a known bit structure within one or more programming languages. In use, data may refer to digital information stored in a computer memory as a sequence of bits. Certain machine learning models may operate on structured arrays of data in a predefined bit format. Those skilled in the art will readily understand that these may be referred to as arrays, multidimensional arrays, matrices, vectors, tensors, or other such similar terms. It should be noted that for machine learning methods, multidimensional arrays or tensors (e.g., having defined ranges in multiple dimensions) may be "flattened" so as to be represented (e.g., in memory) as a sequence or vector of values ​​stored according to a predefined format (e.g., an n-bit integer or floating point number (signed or unsigned)). Therefore, the term "tensor" as used herein encompasses multidimensional arrays (e.g., vectors, matrices, volume arrays, etc.) having one or more dimensions.

[0102] Regardless of the specific data format selected, the principles of the present invention apply. Data can be represented as arrays, vectors, tensors, or any other suitable format. For ease of reference, these terms are used interchangeably herein, and references to one or more "vectors" of values ​​should be understood as any n-dimensional tensor or tensor that is extended to values ​​when appropriate. Similarly, references to one or more "tensors" of values ​​should be understood as extended to vectors, which are understood by those skilled in the art to be simply one-dimensional tensors. Regardless of the formatting of the data structure of these arrays of values, the principles of the present invention can be applied. For example, state data can be stored in a memory as a one-dimensional tensor (i.e., vector) or as a tensor with two or more dimensions (i.e., tensor), and those skilled in the art will readily understand that appropriate modifications can be made to data processing elements to handle the selected data format. The relative position between various state values ​​(e.g., how they are sorted in a vector or tensor) is generally unimportant, and the scope of the present invention is not limited to any specific data format or structure.

[0103] The term "structured digital representation" is used to refer to digital data in a structured form, such as a one-dimensional or multi-dimensional array storing numerical values ​​of a common data type (such as integers or floating point values). Structured digital representations may include vectors or tensors (as used within machine learning terminology). Structured digital representations are typically stored as a set of indexes and / or contiguous memory locations, for example, a one-dimensional array of 64-bit floating points may be represented in computer memory as a contiguous sequence of 64-bit memory locations in a 64-bit computing system.

[0104] The term "transaction data" is used herein to refer to electronic data associated with a transaction. A transaction includes a series of communications between different electronic systems to effectuate a payment or exchange. Generally speaking, transaction data may include data indicating events (e.g., actions taken in a timely manner) that are relevant to the processing of a transaction and that may provide information about the processing of the transaction. Transaction data may include structured, unstructured, and semi-structured data, or any combination thereof. Transaction data may also include data associated with a transaction, such as data used to process the transaction. In some cases, transaction data may be broadly used to refer to actions taken with respect to one or more electronic devices. Transaction data may take a variety of forms depending on the precise implementation. However, different data types and formats may be converted by pre-processing or post-processing when appropriate.

[0105] The term "interface" is used herein to refer to any physical and / or logical interface that allows one or more of data input and data output. The interface may be implemented by a network interface adapted to transmit and / or receive data or by retrieving data from one or more memory locations, such as by a processor that executes a set of instructions. The interface may also include a physical (network) coupling through which data is received, such as hardware that allows wired or wireless communication through a specific medium. The interface may include an application programming interface and / or a method call or return. For example, in a software implementation, the interface may include passing data and / or memory references to a function initiated via a method call, wherein the function includes a computer program code executed by one or more processors; in a hardware implementation, the interface may include wired interconnections between different chips, chipsets, or parts of a chip. In the accompanying drawings, the interface may be indicated by the boundaries of a processing box with inward and / or outward arrows representing data transfer.

[0106] The terms "component" and "module" are used interchangeably to refer to a hardware structure having a specific function (e.g., in the form of mapping input data to output data) or a combination of general hardware and specific software (e.g., a specific computer program code executed on one or more general-purpose processors). A component or module may be implemented as a specific packaged chipset, such as an application specific integrated circuit (ASIC) or a programmed field programmable gate array (FPGA), and / or as a software object, class, class instance, script, code portion, etc., as executed by a processor in use.

[0107] The term "machine learning model" is used herein to refer to at least a specific implementation of a hardware execution of a machine learning model or function. Known models in the field of machine learning include logistic regression models, naive Bayes models, random forests, support vector machines, and artificial neural networks. A specific implementation of a classifier may be provided in one or more machine learning programming libraries, including but not limited to scikit-learn, TensorFlow, and PyTorch.

[0108] The term "mapping" is used herein to refer to a transformation or conversion of a first set of data values ​​to a second set of data values. The two sets of data values ​​can be arrays of different sizes, where the output array has a lower dimension than the input array. The input and output arrays can have common or different data types. In some examples, the mapping is a one-way mapping to a scalar value.

[0109] The term "neural network architecture" refers to a set of one or more artificial neural networks configured to perform a specific data processing task. For example, a "neural network architecture" may include a specific arrangement of one or more neural network layers of one or more neural network types. Neural network types include convolutional neural networks, recurrent neural networks, and feedforward neural networks. Convolutional neural networks involve the application of one or more convolution operations. Recurrent neural networks involve internal states that are updated during an input sequence. Therefore, a recurrent neural network is considered to include a form of loop or feedback connection, whereby the state of the recurrent neural network at a previous time or iteration (e.g., t-1) is used to update the state of the recurrent neural network at a given time or iteration (e.g., t). A feedforward neural network involves transformation operations without feedback, for example, applying operations in a unidirectional sequence from input to output. Feedforward neural networks include ordinary "neural networks" and "fully connected neural networks". Those skilled in the art will understand that "multilayer perceptron" is a term used to describe a fully connected layer and is a special case of a feedforward neural network.

[0110] The term "deep" neural network is used to indicate that the neural network includes multiple neural network layers connected in series (it should be noted that this term "deep" is used with both feedforward neural networks and recurrent neural networks). Certain examples described herein utilize recurrent and fully connected neural networks.

[0111] A "neural network layer" as typically defined in machine learning programming tools and libraries can be considered an operation that maps input data to output data. A "neural network layer" can apply one or more parameters (such as weights) to map input data to output data. One or more bias terms can also be applied. The weights and biases of a neural network layer can be applied using one or more multidimensional arrays or matrices. Generally speaking, a neural network layer has multiple parameters whose values ​​affect how input data is mapped to output data by the layer. These parameters can be trained in a supervised manner by optimizing an objective function. This typically involves minimizing a loss function. Certain parameters can also be pre-trained or fixed in another way. Fixed parameters can be considered configuration data that controls the operation of a neural network layer. A neural network layer or architecture can include a mixture of fixed parameters and learnable parameters. A recurrent neural network layer can apply a series of operations to update a loop state and transform input data. The update of the loop state and the transformation of the input data can involve the transformation of one or more of the previous loop state and the input data. A recurrent neural network layer can be trained by expanding a modeled loop unit, as can be applied in a machine learning programming tool and library. Although a recurrent neural network can be viewed as comprising several (sub) layers to apply different gating operations, most machine learning programming tools and libraries refer to the application of a recurrent neural network as a whole as a "neural network layer", and this convention will be followed here. Finally, a feedforward neural network layer may apply one or more of a set of weights and biases to the input data to generate output data. This operation may be represented as a matrix operation (e.g., where a bias term may be included by appending a value of 1 to the input data). Alternatively, the bias may be applied by a separate add operation. As discussed above, according to the machine learning library, the term "tensor" is used to refer to an array that may have multiple dimensions, for example, a tensor may include a vector, a matrix, or a higher dimensional data structure. In a preferred example, the tensor may include a vector having a predefined number of elements.

[0112] In order to model complex non-linear functions, the neural network layers as described above may be followed by non-linear activation functions. Common activation functions include sigmoid function, tanh function and rectified linear unit (RELU). Many other activation functions exist and may be applied. The activation function may be selected based on testing and preference. The activation function may be omitted in some cases and / or form part of the internal structure of the neural network layer.

[0113] The example neural network architectures described herein may be configured via training. In some cases, a method known as back propagation may be used to train "learnable" or "trainable" parameters. During back propagation, the neural network layers that make up each neural network architecture are initialized (e.g., with randomized weights) and then used to make predictions (e.g., a so-called "forward" pass) using a set of input data from a training set. The predictions are used to evaluate a loss function. For example, a "true value" output may be compared to a predicted output, and the difference may form part of a loss function. In some examples, the loss function may be based on the absolute difference between a predicted scalar value and a binary true value label. The training set may include a set of transactions. If a gradient descent method is used, the loss function is used to determine the gradient of the loss function relative to the parameters of the neural network architecture, where the gradient is then used to back propagate updates to the parameter values ​​of the neural network architecture. Typically, updates are propagated according to the derivatives of the weights of the neural network layers. For example, the gradient of the loss function relative to the weights of the neural network layers may be determined and used to determine updates to the weights that minimize the loss function. In this case, an optimization technique (such as gradient descent, stochastic gradient descent, Adam, etc.) may be used to adjust the weights. The chain rule and automatic differentiation functions can be applied to efficiently compute the gradient of the loss function, thus sequentially going backwards through the neural network layers. BRIEF DESCRIPTION OF THE DRAWINGS

[0114] Certain embodiments of the present invention will now be described with reference to the accompanying drawings, in which:

[0115] Figures 1A to 1C are schematic diagrams illustrating different example electronic infrastructures for transaction processing;

[0116] Figure 2A and Figure 2B is a schematic diagram illustrating different examples of data storage systems for use with a machine learning transaction processing system;

[0117] Figure 3A and Figure 3B is a schematic diagram showing different examples of transaction data;

[0118] Figure 4 is a schematic diagram illustrating example components of a machine learning transaction processing system;

[0119] Figure 5A and Figure 5B is a sequence diagram illustrating an example set of processes performed on transaction data by different computing entities;

[0120] Figure 6 is a schematic diagram showing an example of a data access hierarchy for a machine learning model;

[0121] Figure 7is a diagram illustrating another example of a machine learning model data access hierarchy;

[0122] Figure 7 is a diagram illustrating another example of a machine learning model data access hierarchy;

[0123] Figure 8 is a diagram illustrating another example of a machine learning model data access hierarchy;

[0124] Fig. 9 is a diagram illustrating another example of a machine learning model data access hierarchy;

[0125] Fig.10 is a diagram illustrating another example of a machine learning model data access hierarchy;

[0126] Fig.11 is a diagram illustrating another example of a machine learning model data access hierarchy;

[0127] Fig.12 is a diagram illustrating another example of a machine learning model data access hierarchy;

[0128] Fig.13 is a diagram illustrating another example of a machine learning model data access hierarchy;

[0129] Fig.14 is a diagram illustrating another example of a machine learning model data access hierarchy;

[0130] Fig.15 is a diagram illustrating another example of a machine learning model data access hierarchy;

[0131] Fig.16 is a diagram illustrating another example of a machine learning model data access hierarchy;

[0132] Fig.17 is a diagram illustrating another example of a machine learning model data access hierarchy;

[0133] Fig.18 is a diagram illustrating another example of a machine learning model data access hierarchy;

[0134] Fig.19 is a diagram illustrating another example of a machine learning model data access hierarchy;

[0135] Fig. 20 is a diagram illustrating another example of a machine learning model data access hierarchy;

[0136] Fig.21 is a flow chart illustrating an example of a method of controlling access to machine learning model data;

[0137] Fig. 22 is a block diagram illustrating an example of a multi-tenant machine learning system;

[0138] Fig.23 is a diagram illustrating another example of a machine learning model data access hierarchy;

[0139] Fig.24 is a diagram illustrating another example of a machine learning model data access hierarchy; and

[0140] Fig.25 is a diagram showing another example of a machine learning model data access hierarchy. DETAILED DESCRIPTION

[0141] Certain exemplary embodiments are described herein, which relate to machine learning systems for use in transaction processing. In certain embodiments, a machine learning system is applied in a real-time, high-volume transaction processing pipeline to provide an indication of whether a transaction or entity matches a previously observed and / or predicted activity or action pattern, such as an indication of whether a transaction or entity is "normal" or "abnormal". The term "behavior" is used herein to refer to such an activity or action pattern. The indication may include a scalar value normalized within a predefined range (e.g., 0 to 1), which may then be used to prevent fraud and other abuses of the payment system. The machine learning system may apply machine learning models that are updated as more transaction data is obtained, such as continuously training based on new data, so as to reduce false positives and maintain the accuracy of the output metric. The present invention example is particularly useful for preventing fraud in situations where the physical presence of a payment card cannot be ascertained (e.g., online transactions known as "card-not-present") or for commercial transactions where high-value transactions may be the norm and where it may be difficult to classify behavioral patterns as "unexpected". Thus, examples of the present invention facilitate the processing of transactions because those transactions are primarily "online," ie, conducted digitally over one or more public communications networks.

[0142] Certain embodiments described herein allow machine learning models to be customized to be specific to behavioral patterns between certain pairs of entities (such as account holders) and categories (such as merchants, transaction amounts, time of day, etc.). For example, a machine learning model can model specific behavioral patterns for entity-category pairs. The machine learning systems described herein can provide for dynamically updating machine learning models despite large transaction flows and / or despite the need to separate different data sources.

[0143] As previously outlined, embodiments of the present invention may be applied to a wide variety of digital transactions, including but not limited to card payments, so-called "wire" transfers, peer-to-peer payments, Bank Automated Clearing System (BACS) payments, and Automated Clearing House (ACH) payments. The output of the machine learning system may be used to prevent a wide variety of fraud and crime, such as card fraud, application fraud, payment fraud, merchant fraud, gaming fraud, and money laundering.

[0144] The following describes a non-limiting exemplary embodiment of a machine learning system according to one embodiment of the present invention. Figures 1A to 5B Provides context for machine learning systems.

[0145] Example Transaction Processing System

[0146] Figures 1A to 1C A set of example transaction processing systems 100, 102, 104 are shown. These example transaction processing systems are described to provide context for the invention discussed herein, but should not be considered limiting; the configuration of any one implementation may vary based on the specific requirements of that implementation. However, the example transaction processing systems described allow one skilled in the art to identify certain high-level technical features relevant to the following description. The three example transaction processing systems 100, 102, 104 illustrate different areas where variation may occur.

[0147] Figures 1A to 1C A set of client devices 110 configured to initiate a transaction is shown. In this example, the set of client devices 110 includes a smart phone 110-A, a computer 110-B, a point of sale (POS) system 110-C, and a portable merchant device 110-D. These client devices 110 provide a set of non-exhaustive examples. Generally speaking, any electronic device or set of devices can be used to conduct a transaction. In one case, the transaction includes a purchase or payment. For example, the purchase or payment can be an online or mobile purchase or payment made with the help of a smart phone 110-A or a computer 110-B, or it can be a purchase or payment made at a merchant's location (such as via a POS system 110-C or a portable merchant device 110-D). The purchase or payment can be for goods and / or services.

[0148] exist Figures 1A to 1C In the example, client device 110 is communicatively coupled to one or more computer networks 120. Client device 110 can be communicatively coupled in a variety of ways, including through one or more wired and / or wireless networks, including telecommunications networks. In a preferred example, all communications across the one or more computer networks are secured (e.g., using the Transport Layer Security (TLS) protocol). Figure 1A, two computer networks 120-A and 120-B are shown. These may be separate networks or different parts of a common network. The first computer network 120-A communicatively couples the client device 110 to the merchant server 130. The merchant server 130 may execute computer processes that implement a process flow for a transaction. For example, the merchant server 130 may be a back-end server that handles transaction requests received from a POS system 110-C or a portable merchant device 110-D or may be used by an online merchant to implement a website where purchases may be made. It should be understood that Figures 1A to 1C The examples are necessarily simplifications of actual architectures; for example, there may be several interacting server devices implementing an online merchant, including a separate server device for serving Hypertext Markup Language (HTML) pages detailing products and / or services and for handling payment processing.

[0149] exist Figure 1A In the example, the merchant server 130 is communicatively coupled to another set of back-end server devices to process transactions. Figure 1A In the example embodiment, the merchant server 130 is communicatively coupled to the payment processor server 140 via the second network 120-B. The payment processor server 140 is communicatively coupled to a first data storage device 142 storing transaction data 146 and a second data storage device 144 storing ancillary data 148. The transaction data 146 may include batches of transaction data related to different transactions conducted within a time period. The ancillary data 148 may include data associated with the transaction, such as a record storing merchant and / or end-user data. Figure 1A, the payment processor server 140 may be communicatively coupled to the machine learning server 150 via the second network 120-B. The machine learning server 150 implements a machine learning system 160 for processing transaction data. The machine learning system 160 is arranged to receive input data 162 and map it to output data 164, which is used by the payment processor server 140 to process a particular transaction, such as a transaction generated from the client device 110. In one case, the machine learning system 160 receives at least transaction data associated with a particular transaction and provides an alert or digital output used by the payment processor server 140 to determine whether the transaction is authorized (i.e., approved) or rejected. Thus, the output of the machine learning system 160 may include a label, an alert, or other indication of fraud or general malicious or abnormal activity. The output may include a probability indication, such as a score or probability. In one case, the output data 164 may include a scalar numerical value. The input data 162 may also include data derived from one or more of the transaction data 146 and the ancillary data 148. In one case, the output data 164 indicates a level of deviation from a specific expected behavior pattern based on past observations or measurements. For example, this may indicate fraudulent or criminal behavior, as this is often significantly different from observed behavioral patterns, especially on a large scale. The output data 164 may form behavioral metrics. Expected behavioral patterns may be defined explicitly or implicitly based on observed interactions between different entities within a transaction process flow, such as end users or customers, merchants (including point of sale and back-end locations or entities, where these may be different), and banks.

[0150] The machine learning system 160 may be implemented as part of a transaction processing pipeline. Figure 5A and Figure 5BAn example transaction processing pipeline is described. The transaction processing pipeline may include electronic communications between a client device 110, a merchant server 130, a payment processor server 140, and a machine learning server 150. Other server devices may also be involved, such as a bank server that provides authorization from a card issuer. In some cases, the client device 110 may communicate directly with the payment processor server 140. In use, the transaction processing pipeline is typically required to be completed within one or two hundred milliseconds. Generally speaking, processing times below a second may be considered real-time (e.g., humans typically perceive events over a time span of 400ms). In addition, 100ms to 200ms may be the expected maximum latency for the full round-trip time for transaction processing; within this time span, the time allocated to the machine learning system 160 may be a small fraction of this full amount, such as 10ms (i.e., less than 5% to 10% of the target processing time), because most of the time may be reserved for other operations in the transaction process flow. This imposes technical constraints on the specific implementation of the machine learning system 160. Furthermore, in real-world implementations, the average throughput can be around 1000 to 2000 seconds. This means that most “off-the-shelf” machine learning systems are not suitable for implementing machine learning system 160. It also means that most machine learning methods described in academic papers

[0151] It cannot be implemented within the above transaction processing pipeline without non-obvious adaptations. There is also the problem that anomalies are rare events in nature and therefore difficult to train such an accurate machine learning system.

[0152] Figure 1B Shows Figure 1A A variation 102 of the example transaction processing system 100 is shown. In this variation 102, the machine learning system 160 is implemented within a payment processor computer infrastructure, such as executed by a payment processor server 140 and / or on a locally coupled server within the same local network as the payment processor server 140. Figure 1B The variation 102 may be preferred for larger payment processors because it allows for faster response times, greater control, and improved security. However, functionally, the transaction processing pipeline may be similar to Figure 1A For example, in Figure 1A In the example of , the machine learning system 160 can be initiated by a secure external application programming interface (API) call (such as a representation state transfer (REST) ​​API call using Hypertext Transfer Protocol Secure (HTTPS)), and in Figure 1B In the example, the machine learning system 160 may be initiated by an internal API call, but in this case, a common end API may handle both requests (e.g., a REST HTTPS API may provide an external wrapper for the internal API).

[0153] Figure 1C Shows Figure 1A Another variation 104 of the example transaction processing system 100. In this variation 104, the machine learning system 160 is communicatively coupled to a local data storage device 170. For example, the data storage device 170 may be on the same local network as the machine learning server 150 or may include a local storage network accessible to the machine learning server 150. In this case, there are multiple local data storage devices 170-A to 170-N, each of which stores partitioned ancillary data 172. The partitioned ancillary data 172 may include parameters of one or more machine learning models. In one case, the ancillary data 172 may include the state of the machine learning model, where the state may relate to a specific entity (such as a user or a merchant). The partitioning of the ancillary data 172 may need to be applied to meet security requirements set by a third party (such as a payment processor, one or more banks, and / or one or more merchants). In use, the machine learning system 160 accesses the ancillary data 172-A to 172-N via multiple local data storage devices 170-A to 170-N based on the input data 162. For example, input data 162 may be received via an API request from a particular source and / or may include data identifying a particular partition to use to handle the API request. Figure 2A and Figure 2B Further details of the different storage systems that can be applied to meet security requirements are explained in.

[0154] Example data store configuration

[0155] Figure 2A and Figure 2B Two example data storage configurations 200 and 202 are shown that may be used by an example machine learning system 210 for processing of transaction data. Figure 2A and Figure 2B The examples are two non-limiting examples showing different options available for various specific implementations, and a specific configuration may be selected based on individual circumstances. The machine learning system 210 may include Figures 1A to 1C A specific implementation of the machine learning system 160 described in the previous example. Figure 2A and Figure 2B The example allows for the processing of transaction data that uses heterogeneous cryptographic parameters to ensure security, such as for the machine learning system 210 to securely process transaction data for heterogeneous entities. It should be understood that if the machine learning system 160 is implemented for a single set of secure transaction and ancillary data, such as within an internal transaction processing system or as a hosted system for use by a single payment processor, then it may not be necessary. Figure 2A and Figure 2B Configuration.

[0156] Figure 2A The machine learning system 210 is shown communicatively coupled to a data bus 220. The data bus 220 may include an internal data bus of the machine learning server 150 or may form part of a storage area network. The data bus 220 communicatively couples the machine learning system 210 to a plurality of data storage devices 230, 232. The data storage devices 230, 232 may include any known data storage devices, such as magnetic hard disks and solid-state devices. Although the data storage devices 230, 232 may be stored in a plurality of storage devices, the data storage devices 230, 232 may be stored in a plurality of storage devices. Figure 2A , but they may alternatively form different physical storage areas or portions within a common data storage device. Figure 2A In the embodiment, a plurality of data storage devices 230, 232 store historical transaction data 240 and ancillary data 242. Figure 2A , a first set of data storage devices 230 stores historical transaction data 240, and a second set of data storage devices 232 stores ancillary data 242. Ancillary data 242 may include one or more of the model parameters of a set of machine learning models (such as parameters for training of a neural network architecture and / or configuration parameters of a random forest model) and state data of those models. In one case, different sets of historical transaction data 240-A to 240-N and ancillary data 242-A to 242-N are associated with different entities that securely and collectively use the services provided by the machine learning system 210, for example, these may represent data that needs to be kept separate as part of different banks as a condition for providing machine learning services to those entities.

[0157] Figure 2B Another way in which different sets of historical transaction data 240-A to 240-N and ancillary data 242-A to 242-N may be stored is shown. Figure 2B In the embodiment, the machine learning system 210 is communicatively coupled to at least one data storage device 260 via a data transfer channel 250. The data transfer channel 250 may include a local storage bus, a local storage area network, and / or a remote secure storage coupling (e.g., such as over an unsecured network such as the Internet). Figure 2B In the embodiment of the present invention, a physical data storage device 260 is used to provide a secure logical storage layer 270. The secure logical storage layer 270 can be a virtualized system that appears to the machine learning system 210 as a separate physical storage device while actually being independently implemented on at least one data storage device 260. The logical storage layer 270 can provide separate encrypted partitions 280 for data related to a group of entities (e.g., related to different card issuers, etc.), and different sets of historical transaction data 240-A to 240-N and ancillary data 242-A to 242-N can be stored in corresponding partitions 280-A to 280-N. In some cases, the secure logical storage layer 270 can be encrypted based on the data stored by the machine learning system 210 when the transaction is received for processing. Figures 1A to 1COne or more of the server systems shown store data to dynamically create entities.

[0158] Sample transaction data

[0159] Figure 3A and Figure 3B Examples of transaction data that may be processed by a machine learning system such as 160 or 210 are shown. Figure 3A 300, where each record has a timestamp and includes a plurality of transaction fields. In some cases, the transaction data may be grouped and / or filtered based on the timestamp. For example, Figure 3A The partitioning of transaction data into current transaction data 310 associated with the current transaction and "older" or historical transaction data 320 within a predefined time range of the current transaction is shown. The time range can be set as a hyperparameter of any machine learning system. Alternatively, the "older" or historical transaction data 320 can be set to a certain number of transactions. A hybrid of the two approaches is also possible.

[0160] Figure 3B 3 shows how transaction data 330 for a particular transaction can be stored in digital form for processing by one or more machine learning models. Figure 3B In the example of the present invention, transaction data has at least the fields: transaction amount, timestamp (e.g., as a Unix epoch), transaction type (e.g., card payment or direct debit), product description or identifier (i.e., related to the item being purchased), merchant identifier, issuing bank identifier, set of characters (e.g., Unicode characters within a field of predefined character length), country identifier, etc. It should be noted that a wide variety of data types and formats can be received and pre-processed into appropriate digital representations. In some cases, originating transaction data (such as generated by a client device and sent to the merchant server 130) is pre-processed to convert alphanumeric data types into numeric data types for application of one or more machine learning models. Other fields present in the transaction data may include, but are not limited to, account number (e.g., credit card number), location where the transaction is occurring, and manner of performing the transaction (e.g., in person, by phone, on a website).

[0161] Example Machine Learning System

[0162] Figure 4An example 400 of a machine learning system 402 that can be used to process transaction data is shown. The machine learning system 402 can implement one or more of the machine learning systems 160 and 210. The machine learning system 402 receives input data 410. The form of the input data 410 can depend on which machine learning model is being applied by the machine learning system 402. In the case where the machine learning system 402 is configured to perform fraud or anomaly detection on transactions (e.g., ongoing transactions as described above), the input data 410 can include transaction data such as 330 (i.e., data forming part of a data packet about a transaction) and data from historical transaction data (such as Figure 3A 300) and / or derived from ancillary data (such as Figures 1A to 1C 148 or Figure 2A and Figure 2B The ancillary data may be included in the master data associated with the transaction.

[0163] Ancillary data linked to one or more entities identified. For example, if transaction data about an ongoing transaction identifies a user, a merchant, and one or more banks associated with the transaction (such as the user's card issuer and the merchant's bank) via a unique identifier present in the transaction data, the ancillary data may include data related to these transaction entities. The ancillary data may also include data derived from records of the activity, such as interaction logs and / or authentication records. In one case, the ancillary data is stored in one or more static data records and retrieved from these records based on received transaction data. Additionally or alternatively, the ancillary data may include machine learning model parameters retrieved based on the content of the transaction data. For example, a machine learning model may have parameters specific to one or more of a user, a merchant, and a card issuer, and these parameters may be retrieved based on which of these parameters are identified in the transaction data. For example, one or more of a user, a merchant, and a card issuer may have corresponding embeddings that may include retrievable or mappable tensor representations of the entities. For example, each user or merchant may have a tensor representation (e.g., a floating point vector of size 128 to 1024), which may be retrieved from a database or other data storage device or may be generated by an embedding layer, for example, based on a user or merchant index.

[0164] Input data 410 is received at an input data interface 412. The input data interface 412 may include an API interface, such as an internal or external API interface as described above.

[0165] In one case, if Figures 1A to 1CThe payment processor server 140 shown issues a request to this interface, wherein the request payload contains transaction data. The API interface may be defined as being agnostic about the form of the transaction data or its source. The input data interface 412 is communicatively coupled to the machine learning model platform 414. In one case, a request issued to the input data interface 412 triggers the machine learning model platform 414 to execute using the transaction data supplied to the interface. The machine learning model platform 414 is configured as an execution environment for applying one or more machine learning models to the input data 410. In one case, the machine learning model platform 414 is arranged as an execution wrapper for multiple different selectable machine learning models. For example, a model definition language (e.g., similar to or using a markup language such as an extended markup language XML) may be used to define a machine learning model. The model definition language may include (among others, independently or in combination) SQL, TensorFlow, Caffe, Thine, and PyTorch. In one case, the model definition language includes computer program code that is executable to implement one or more of the training and inference of the defined machine learning model. Machine learning models may include, for example, artificial neural network architectures, ensemble models, regression models, decision trees (such as random forests), graph models, and Bayesian networks, etc. The machine learning model platform 414 may define common (i.e., shared) input and output definitions so that different machine learning models are applied in a common (i.e., shared) manner.

[0166] In the present example, the machine learning model platform 414 is configured to provide at least a single scalar output 416. This can be normalized within a predefined range (such as 0 to 1). When normalized, the scalar output 416 can be considered as the probability that the transaction associated with the input data 410 is fraudulent or abnormal. In this case, the value "0" may represent a transaction that matches the normal pattern of activity of one or more of the user, merchant, and card issuer, while the value "1" may indicate that the transaction is fraudulent or abnormal, that is, it does not match the expected pattern of activity (but those skilled in the art will know that the normalization range may be different, such as reversed or within different boundaries, and have the same functional effect). It should be noted that although the range of values ​​may be defined as 0 to 1, the output values ​​may not be evenly distributed within this range, for example, the value "0.2" may be a common output for "normal" events, and the value "0.8" may be considered to exceed the threshold of typical "abnormal" or fraudulent events. The machine learning model implemented by the machine learning platform 414 can therefore implement a form of mapping between high-dimensional input data (e.g., transaction data and any retrieved ancillary data) and a single value output. In one case, for example, the machine learning platform 414 can be configured to receive input data in a digital format for a machine learning model, wherein each defined machine learning model is configured to map input data defined in the same manner. The exact machine learning model applied by the machine learning model platform 414 and the parameters of the model can be determined based on the configuration data. The configuration data can be included in the input data 410 and / or identified using the input data and / or can be set based on one or more configuration files parsed by the machine learning platform 414.

[0167] In some cases, the machine learning model platform 414 may provide additional outputs depending on the context. In some specific implementations, the machine learning model platform 414 may be configured to return a "reason code" that captures a human-friendly explanation of the output of the machine learning model with respect to suspicious input attributes. For example, the machine learning model platform 414 may indicate which of one or more input elements or units within the input representation affected the model output, such as a combination of a set of "amount" channels and "merchant" elements or units (such as embeddings or indices) above a learned threshold outside a given cluster. In the case where the machine learning model platform 414 implements a decision tree, these additional outputs may include routes through the decision tree or aggregate feature importance based on a tree ensemble. For a neural network architecture, this may include layer output activations and / or layer filters with positive activations.

[0168] exist Figure 4In some implementations, the scalar output 416 may include an optional alarm system 418 that receives the scalar output 416. In other implementations, the scalar output 416 may be passed directly to the output data interface 420 without post-processing. In this latter case, the scalar output 416 may be packaged into a response to the original request to the input data interface 412. In both cases, the output data 422 derived from the scalar output 416 is provided as an output of the machine learning system 402. The output data 422 is returned to allow final processing of the transaction data. For example, the output data 422 may be returned to the payment processor server 140 and used as the basis for a decision to approve or reject the transaction. Depending on the specific implementation requirements, in one case, the alarm system 418 may process the scalar output 416 to return a binary value indicating whether the transaction should be approved or rejected (e.g., "1" is equivalent to rejection). In some cases, a decision may be made by applying a threshold to the scalar output 416. This threshold may be context-dependent. In some cases, the alarm system 418 and / or the output data interface 420 may also receive additional inputs, such as interpretation data (e.g., the "reason codes" discussed above) and / or raw input data. The output data interface 420 may generate an output data packet of output data 422 that combines these inputs with the scalar output 416 (e.g., at least for logging and / or later review). Similarly, the alarms generated by the alarm system 418 may include and / or be additionally based on the aforementioned additional inputs, for example, in addition to the scalar output 416.

[0169] The machine learning system 402 is typically used in an "online" mode to process a large number of transactions within a narrowly defined time frame. For example, under normal processing conditions, the machine learning system 402 may process a request within 7ms to 12ms and be able to manage 1000 to 2000 requests per second (these are median constraints from real-world operating conditions). However, the machine learning system 402 may also be used in an "offline" mode, for example by providing selected historical transactions to the input data interface 412. In the offline mode, the input data may be passed to the input data interface in batches (i.e., in groups). The machine learning system 402 may also be capable of implementing machine learning models that provide scalar outputs for entities and for transactions or in lieu of transactions. For example, the machine learning system 402 may receive a request associated with an identified user (e.g., a card or payment account holder) or an identified merchant and be arranged to provide a scalar output 416 indicating the likelihood that the user or merchant is fraudulent, malicious, or abnormal (i.e., a general threat or risk). For example, this may form part of a continuous or periodic monitoring process, or may form a one-off request (e.g., as part of an application for a service). The deployment of a scalar output for a particular entity may be based on a set of transaction data up to and including the last approved transaction in a sequence of transaction data (e.g., similar to Figure 3AThe transaction data shown is transaction data for the entity).

[0170] "Live" and "batch" data can be received and used by the machine learning system. These can be received by the same set of Event APIs with different REST endpoints, or by a different set of Event APIs dedicated to reading batch data from another source.

[0171] Additionally or alternatively, the Event API may read events from another source (rather than receiving them via REST), such as by reading them from a comma-separated values ​​(CSV) file or any other suitable file format.

[0172] Example transaction process flow

[0173] Figure 5A and Figure 5B Two possible example transaction process flows 500 and 550 are shown. These process flows may be Figures 1A to 1C The example transaction process systems 100, 102, 104 and other systems shown occur in the context of the process flows 500 and 550. The process flows 500 and 550 are provided as an example of a context in which machine learning transaction processing systems may be applied, however not all transaction process flows will necessarily follow Figure 5A and Figure 5BThe process shown, and the process flow may change between specific implementations, between systems, and over time. The example transaction process flows 500 and 550 reflect two possible situations: a first situation represented by the transaction process flow 500, in which the transaction is approved, and a second situation represented by the transaction process flow 550, in which the transaction is rejected. Each transaction process flow 500, 550 involves the same set of five interacting systems and devices: POS or user device 502, merchant system 504, payment processor (PP) system 506, machine learning (ML) system 508, and card issuer system 510. The POS or user device 502 may include one of the client devices 110, the merchant system 504 may include the merchant server 130, the payment processor system 506 may include the payment processor server 140, and the machine learning system 508 may include a specific implementation of the machine learning system 160, 210 and / or 402. The card issuer system 510 may include one or more server devices that implement transaction functions on behalf of the card issuer. The five interacting systems and devices 502 to 510 may be communicatively coupled by one or more internal or external communication channels, such as network 120. In some cases, some of these systems may be combined, for example, a card issuer may also act as a payment processor and thus systems 506 and 510 may be implemented with a common system. In other cases, a similar process flow may be performed specifically for a merchant (e.g., without involving a payment processor or card issuer). In this case, the machine learning system 508 may communicate directly with the merchant system 504. In these variations, the general functional transaction process flow may remain similar to the transaction process flow described below.

[0174] exist Figure 5A and Figure 5BThe transaction process flow in both includes multiple common (i.e., shared) processes 512 to 528. At box 512, a POS or user device 502 initiates a transaction. For a POS device, this may include a cashier using a front-end device to attempt to make an electronic payment; for a user device 502, this may include a user using a credit or debit card or an online payment account to make an online purchase (e.g., clicking "Done" in an online shopping cart). At box 514, payment details as electronic data are received by the merchant system 504. At box 516, the transaction is processed by the merchant system 504, and a request for authorized payment is issued to the payment processor system 506. At box 518, the payment processor system 506 receives the request from the merchant system 504. The request may be issued on a proprietary communication channel or as a secure request on a public network (e.g., an HTTPS request on the Internet). The payment processor system 506 then issues a request to the machine learning system 508 for a score or probability used in processing the transaction. Block 518 may additionally include retrieving the ancillary data to combine with the transaction data sent to the machine learning system 508 as part of the request. In other cases, the machine learning system 508 may have access to a data storage device (e.g., similar to a Figure 2A and Figure 2B configuration) and thus retrieve this data as part of internal operations (e.g., based on identifiers provided within the transaction data and / or as defined as part of an implemented machine learning model).

[0175] Block 520 illustrates model initialization operations that occur prior to any request from the payment processor system 506. For example, the model initialization operations may include loading a defined machine learning model and parameters for instantiating the defined machine learning model. At block 522, the machine learning system 508 receives a request from the payment processor system 506 (e.g., via a data input interface such as Figure 4 412 in block 522). At block 522, the machine learning system 508 may perform any defined pre-processing prior to application of the machine learning model initialized at block 520. For example, where the transaction data still retains character data (such as a merchant identified by a string or character transaction description), this may be converted into suitable structured digital data (e.g., by converting string categorical data into identifiers via a lookup operation or other mapping, and / or by mapping characters or groups of characters to vector embeddings). Then, at block 524, the machine learning system 506 applies the instantiated machine learning model, thereby supplying the model with input data derived from the received request. This may include applying, for example, reference Figure 4The machine learning model platform 414. At block 526, a scalar output is generated by the instantiated machine learning model. This can be processed to determine an "approved" or "rejected" binary decision at the machine learning system 508, or, in a preferred case, returned to the payment processor system 506 as a response to the request issued at block 518.

[0176] At box 528, the output of the machine learning system 508 is received by the payment processor system 506 and used to approve or deny the transaction. Figure 5A A process is shown in which a transaction is approved based on the output of a machine learning system 508; Figure 5B A process is shown in which a transaction is rejected based on the output of the machine learning system 508. Figure 5A , at box 528, the transaction is approved. Then, at box 530, a request is made to the card issuer system 532. At box 534, the card issuer system 532 approves or denies the request. For example, if the end user or card holder has sufficient funds and approval to pay the transaction fee, the card issuer system 532 may approve the request. In some cases, the card issuer system 532 may apply a second level of security; however, if the card issuer relies on anomaly detection performed by a payment processor using the machine learning system 508, this may not be required. At box 536, the authorization from the card issuer system 510 is returned to the payment processor system 506, which in turn transmits a response to the merchant system 504 at box 538, and the merchant system 504 in turn responds to the POS or user device 502 at box 540. If the card issuer system 510 approves the transaction at box 534, the transaction can be completed and a positive response is returned to the POS or user device 502 via the merchant system 504. The end user may experience this as an "authorized" message on the screen of the POS or user device 502. The merchant system 504 may then complete the purchase (e.g., initiate internal processing to effectuate the purchase).

[0177] At a later point in time, one or more of the merchant system 504 and the machine learning system 508 may save data related to the transaction, for example, as part of the transaction data 146, 240, or 300 in the previous examples. This is shown at dashed boxes 542 and 544.

[0178] The transaction data may be saved along with one or more of the output of the machine learning system 508 (e.g., scalar fraud or anomaly probability) and the final outcome of the transaction (e.g., whether the transaction was approved or rejected). The saved data may be stored for use as training data for a machine learning model implemented by the machine learning system 508.

[0179] The saved data may also be accessed as part of future iterations of block 524, e.g., may form part of future ancillary data. In some cases, the ultimate outcome or outcome of a transaction may not be known at the time of the transaction. For example, a transaction may be marked as an anomaly only via later review by an analyst and / or automated system or based on feedback from a user (e.g., when a user reports fraud or indicates that a payment card or account has been compromised since a certain date). In these cases, ground truth labels may be collected over time after the transaction itself for purposes of training the machine learning system 508.

[0180] Now go to Figure 5B , in which case one or more of the machine learning system 508 and the payment processor system 506 declines the transaction based on the output of the machine learning system 508. For example, if the scalar output of the machine learning system 508 is above a retrieved threshold, the transaction may be declined. At box 552, the payment processor system 506 issues a response to the merchant system 504, which is received at box 554. At box 554, the merchant system 504 takes steps to prevent the transaction from being completed and returns an appropriate response to the POS or user device 502. At box 556, this response is received, and the end user or customer may be notified that their payment has been declined, for example via an on-screen “DENIED” message. The end user or customer may be prompted to use a different payment method. Although Figure 5B Not shown, but in some cases, the card issuer system 510 may be notified that a transaction associated with a particular account holder has been declined. Figure 5B The portion of the process shown is notified to the card issuer system 510, or may be notified to the card issuer system 510 as part of a periodic (e.g., daily) update. Although the transaction may not become part of the transaction data 146, 240, or 300 (because it is not approved), it may still be logged by at least the machine learning system 508, as indicated by box 544. For example, Figure 5A Likewise, transaction data may be saved along with the output of the machine learning system 508 (e.g., a scalar fraud or anomaly probability) and the final outcome of the transaction (e.g., transaction declined).

[0181] Multi-tenancy processing - technical effects

[0182] As explained above, having separate installers for different tenants and / or data types running on the same software while appearing to be one system can lead to a complex final solution.

[0183] The examples described herein focus on different types of multi-tenant solutions. A "tenant" may be a customer of an administrator of the platform. An administrator may be referred to as a "landlord." The term "multi-tenant" is used herein to refer to at least two tenants.

[0184] The sample multi-tenant solution described in this article securely serves each tenant of an administrator, with data separation between different tenants.

[0185] Without loss of generality, the examples described herein provide a single platform that separates data by tenant. A given tenant can be identified in an event by a "tenant ID".

[0186] Tenants can be configured as part of a hierarchy containing an Analysis Configuration Group (ACG). ACG is generally referred to herein as a "tenant group." A tenant group includes one or more tenants.

[0187] Tenant groups themselves can be included in different solutions. Therefore, a solution can be considered as a collection of one or more tenant groups.

[0188] Solutions can be structured in a variety of different ways.

[0189] A user interface (UI) is used to configure the position of each tenant in the hierarchy.

[0190] The data may be separated, for example, by tenants and tenant groups. Each tenant and each tenant group may have its own area in the database, and models run on the tenants and tenant groups may store data in the database.

[0191] When an event (e.g., a transaction event) is processed, data is loaded from the database and made available to one or more models. Depending on access control rights, the data available to the model may include: (i) data about the tenant, (ii) data about each tenant group that the tenant is a member of, (iii) data about included solutions (if any), and / or (iv) platform-wide administrator-level data.

[0192] Models can then decide how they want to handle the data. If the model is a federated model created by an administrator to run across all events, the model can treat the events as a whole, potentially using and storing data at every level in the hierarchy, depending on what that specific tenant group represents. If the model is only configured to run at a specific tenant group level or tenant level, the model can only use data in and below that tenant group.

[0193] The example system described herein allows data to be used in both hierarchical directions (i.e., up and down) in the configuration hierarchy. Rules at the tenant group or administrator level can use data and events from all levels below them. Tenants and tenant groups may be able to use data that has been exposed to them by higher levels in the hierarchy in their own rules. Data sharing rules, learning, and data will be applied within business units, client sets, and other entities to adapt to changing events and situations more quickly and efficiently than existing systems without manual intervention. This universal data sharing and hierarchical referencing is new in the world of machine learning platforms, especially in terms of fraud and risk.

[0194] Because the entire platform is a single system, the tenant hierarchy can be modified on the fly. New solutions and / or tenants can be easily added to the existing platform without having to deploy a new instance of the platform to run, for example, a new solution. Models can run either within a given solution or across all solutions. For each tenant, data can be accessed both up and down the hierarchy.

[0195] Within the UI, each tenant and / or solution can be configured to allow access only to that tenant or to that solution and all tenant groups below it. In each example, only administrators have access to the entire system and can control any tenant and / or tenant group on the platform to modify their analytics as required.

[0196] The example platform described in this article provides a high level of fine-grained control, allowing models to be separated by solution, tenant group, and / or tenant, or as a federated model with access to the entire system.

[0197] The sample platform described in this article simplifies deployment because the platform is deployed and configured once. Each solution runs within the same platform, where data separation is managed by the platform and the model.

[0198] The example platform described in this article is also highly flexible, with analytics configuration and data separated in a way that best suits specific use cases. This architecture allows for distinct and even overlapping analytics to be performed, increasing adaptability and flexibility across tenant groups and hierarchies.

[0199] The examples described herein enable provisioning of several individual instances from a multi-tenant configuration, providing immediate access to cutting-edge fraud and financial crime prevention technologies; all while maintaining access control and accessibility to various functional combinations. Additionally, with a multi-tenant architecture, providers can distribute upgrades across all tenants by running updates only once.

[0200] In principle, the computational overhead of a single multi-tenant platform may be large compared to individual instances for each tenant. In principle, there may also be a risk of data "contamination" in a single multi-tenant platform. However, the examples described herein provide a computationally efficient and data-separated multi-tenant platform that is easily deployable, updatable, useful for real-time anomaly detection, and to which configuration changes (such as adding new tenants) can be made efficiently and effectively.

[0201] Multi-tenancy handling - Machine learning model data access hierarchy - Example 1

[0202] Figure 6 An example machine learning model data access hierarchy 600 is shown. For convenience and brevity, the machine learning model data access hierarchy will generally be referred to herein as the "hierarchy."

[0203] In this example, the hierarchy 600 includes a first level 602 and a second level 604 .

[0204] A "level" may be considered a logical association of a set of elements of the hierarchy 600. The association may be based at least in part on one or more characteristics of the element in question. Figure 6 6, elements belonging to the same level as each other are depicted in the same horizontal area of ​​the hierarchy 600, but this is only for the convenience of understanding the present disclosure. The level may also be referred to as a "layer" or the like.

[0205] In this example, the first level 602 includes a tenant level 602. In this example, the tenant level 602 includes any and all tenants. In this specific example, the tenant level 602 includes a first tenant 606 and a second tenant 608.

[0206] The first tenant 606 and the second tenant 608 are associated with first tenant machine learning model data 610 and second tenant machine learning model data 612, respectively. For convenience and brevity, the machine learning model data will generally be referred to as "ML model data" herein. The ML model data may be associated with a tenant by being stored in conjunction with the tenant ID of the tenant. The term "ML model data" should be understood to encompass data, such as state data and parameter data specifically relating to the ML model. For example, the ML model data may correspond to the ancillary data 172 described above.

[0207] In this example, the second level 604 includes a tenant group level 604. In this example, the tenant group level 604 includes any and all tenant groups. In this specific example, the tenant group level 604 includes exactly one tenant group 614.

[0208] In this example, first tenant 606 and second tenant 608 belong to tenant group 614. Figure 6 606 and 608 are depicted by arrows 616, 618 pointing from the first tenant 606 and the second tenant 608 to the tenant group 614, respectively. The term "belongs to" is generally used herein to mean included in, contained in, has joined, is part of, is a child of, etc.

[0209] In this example, tenant group 614 is associated with tenant group ML model data 620. Tenant group 614 can be associated with tenant group ML model data 620 via a tenant group ID or otherwise.

[0210] Data (such as metadata) indicating an association between the tenant group 614 and the first tenant 606 and the second tenant 608 may be stored.

[0211] In this example, the tenant group ML model data 620 is based on the first tenant ML model data 610 and / or the second tenant ML model data 612. In other words, in this example, the tenant group ML model data 620 is based on (i) only the first tenant ML model data 610, (ii) only the second tenant ML model data 612, or (iii) both the first tenant ML model data 610 and / or the second tenant ML model data 612.

[0212] Multi-tenancy - Access control at the tenant group level - Example 1

[0213] Figure 7 Another example hierarchy 700 is shown.

[0214] Figure 7 The example hierarchy 700 shown corresponds to Figure 6 The example hierarchy 600 is shown, but additionally depicts the active ML model. Specifically, Figure 7 Depicted is an ML model 722 (denoted as “ML model {606, 604}”) applied with respect to the first tenant 606 at the tenant group level 604. The notation “ML model {606, 604}” is used herein to indicate an ML model applied with respect to the Ath tenant at the Bth level.

[0215] In this example, access control is applied in the hierarchy 700 such that the ML model 722 is (i) allowed access to the tenant group ML model data 620 (as indicated at item 724), (ii) allowed access to the first tenant ML model data 610 (as indicated at item 726), and (iii) prohibited access to the second tenant ML model data 612 (as indicated at item 728). The term "prohibit" is used generally herein to mean prevent, disallow, and the like.

[0216] Thus, the first tenant ML model data 610 and the second tenant ML model data 612 are separated because the ML model 722 applied with respect to the first tenant 606 has access to the first tenant ML model data 610 but does not have access to the second tenant ML model data 612. In this example, the ML model 722 can use the tenant group ML model data 620 and the first tenant ML model data 610 to perform real-time anomaly detection associated with the first tenant 606. In this example, the ML model 722 does not directly use the second tenant ML model data 612 to perform real-time anomaly detection associated with the first tenant 606. More specifically, and as will become clearer from the following description, in this example, the ML model 722 cannot directly access the second tenant ML model data 612. However, in some examples, the tenant group ML model data 620 is based at least in part on the second tenant ML model data 612. In such examples, the ML model 722 may be able to directly access the tenant group ML model data 620, and therefore, may be able to access data affected by the second tenant ML model data 612.

[0217] Multi-tenant processing - zero or limited contribution of ML model data from a tenant to a group of tenants

[0218] As explained above, in this example, the tenant group ML model data 620 is based on the first tenant ML model data 610 and / or the second tenant ML model data 612 .

[0219] In one specific example, the tenant group ML model data 620 (i) is based on the first tenant ML model data 610 and (ii) is not based on the second tenant ML model data 612 , or the tenant group ML model data is based on a restricted version of the second tenant ML model data 612 .

[0220] The restricted version of the second tenant ML model data 612 can be restricted in a variety of different ways with respect to the second tenant ML model data 612 .

[0221] In some examples, the restricted version of the second tenant ML model data 612 includes a subset of the second tenant ML model data 612. The subset may include a historical subset. The historical subset may include all elements of the second tenant ML model data 612 before a given threshold time. In some examples, the restricted version of the second tenant ML model data 612 includes some or all of the second tenant ML model data in the second tenant ML model data 612 with one or more weighting factors applied. For example, the weighting factor may correspond to a percentage.

[0222] Thus, in some examples, the tenant group ML model data 620 (i) is based on at least the historical set of the first tenant ML model data 610 and (ii) is not based on at least the historical set of the second tenant ML model data 612 , or the tenant group ML model data is based on a restricted version of the historical set of the second tenant ML model data 612 .

[0223] In this specific example, after the first tenant 606 joins the tenant group 614 , the second tenant 608 joins the tenant group 614 .

[0224] The historical set of second tenant ML model data 612 may have been generated before the second tenant 608 joined the tenant group 614. Therefore, by not using the second tenant ML model data 612 or by using a limited version of the second tenant ML model data 612, the contribution of historical data that may not particularly reflect the current ML model data of the second tenant 608 may be limited. This in turn may result in more accurate tenant group ML model data 620.

[0225] According to this specific example, when the second tenant 608 joins the tenant group 614 , the contribution of the second tenant ML model data 612 to the tenant group ML model data 620 may be zero or may be limited.

[0226] For example, the second tenant 608 may have previously belonged to another tenant group (not shown), and may have moved to the tenant group 614 because the tenant group 614 is a better match for the second tenant 608 than the other tenant group (not shown). Updating the tenant group ML model data 620 with all of the historical second tenant ML model data 612 may skew the tenant group ML model data 620 due to the historical second tenant ML model data 612.

[0227] However, the contribution of the second tenant ML model data 612 to the tenant group ML model data 620 may increase over time when the second tenant 608 belongs to the tenant group 614. Since the second tenant 608 is likely to move to the tenant group 614 because the second tenant 608 is more suitable for the tenant group 614 than another tenant group, the contribution of the second tenant ML model data 612 to the tenant group ML model data 620 may increase over time.

[0228] For example, the tenant group ML model data 620 may be updated to generate updated tenant group ML model data (not shown). The updated tenant group ML model data may be based on both the first tenant ML model data 610 and the second tenant ML model data 612. With respect to the second tenant ML model data 612, the updated tenant group ML model data may be derived from the second tenant ML model data 612.

[0229] Multi-tenant processing - non-zero contribution of ML model data from a tenant to a tenant group

[0230] As explained above, in this example, the tenant group ML model data 620 is based on the first tenant ML model data 610 and / or the second tenant ML model data 612 .

[0231] In another specific example, the tenant group ML model data 620 is based on at least some of each of the first ML model data 610 and the second ML model data 612. The term “at least some of” is used herein to mean “some or all of”.

[0232] In some examples, the tenant group ML model data 620 is derived based on one or more operations involving at least some of each of the first ML model data 610 and the second ML model data 612 .

[0233] The one or more operations may include a combination operation. Examples of combination operations include, but are not limited to, summing, averaging, minimum, maximum, standard deviation, and combining into sets. "Combination operation" may also be referred to as "aggregation operation", "functional operation", etc.

[0234] Multi-tenancy - Access control at the tenant group level - Example 2

[0235] Figure 8 Another example hierarchy 800 is shown.

[0236] Figure 8 Depicted is the ML model 830 (denoted as “ML model {608, 604}”) applied with respect to the second tenant 608 at the tenant group level 604 .

[0237] The type of ML model 830 can be referred to above. Figure 7 The ML models 722 may be of the same type or may be of a different type. However, the ML model 830 is applied with respect to the second tenant 608 instead of the first tenant 606.

[0238] In this example, the ML model 830 is (i) allowed access to the tenant group ML model data 620 (as indicated at item 724), (ii) is prohibited from access to the first tenant ML model data 610 (as indicated at item 726), and (iii) is allowed access to the second tenant ML model data 612 (as indicated at item 728).

[0239] Therefore, the first tenant ML model data 610 and the second tenant ML model data 612 are separated again because the ML model 830 applied with respect to the second tenant 608 has access to the second tenant ML model data 612 but not to the first tenant ML model data 610. In this example, the ML model 830 can use the tenant group ML model data 614 and the second tenant ML model data 612 to perform real-time anomaly detection associated with the second tenant 608.

[0240] Multi-tenancy handling - Access control at the tenant level - Example 1

[0241] Fig. 9 Another example hierarchy 900 is shown.

[0242] Fig. 9 Depicted is an ML model 932 (denoted as “ML model {606, 602}”) applied at the tenant level 602 with respect to the first tenant 606 .

[0243] In this example, the ML model 932 is (i) prohibited from accessing the tenant group ML model data 620 (as indicated at item 724), (ii) is allowed access to the first tenant ML model data 610 (as indicated at item 726), and (iii) is prohibited from accessing the second tenant ML model data 612 (as indicated at item 728).

[0244] Thus, in this example, the ML model 932 may use the first tenant ML model data 610 to perform real-time anomaly detection associated with the first tenant 606. However, in this example, the ML model 932 applied at the tenant level 602 cannot access (i) the tenant group ML model data 620 at the (higher) tenant group level 604, or (ii) the second tenant ML model data 612 at the (same) tenant level 602.

[0245] Multi-tenancy handling - Access control at the tenant level - Example 2

[0246] Fig.10 Another example hierarchy 1000 is shown.

[0247] Fig.10 Depicted is the ML model 1034 (denoted as “ML model {608, 602}”) applied at the tenant level 602 with respect to the second tenant 608 .

[0248] In this example, the ML model 1034 is (i) prohibited from accessing the tenant group ML model data 620 (as indicated at item 724), (ii) prohibited from accessing the first tenant ML model data 610 (as indicated at item 726), and (iii) permitted to access the second tenant ML model data 612 (as indicated at item 728).

[0249] Thus, in this example, the ML model 1034 can use the second tenant ML model data 612 to perform real-time anomaly detection associated with the second tenant 608. However, in this example, the ML model 1034 applied at the tenant level 602 cannot access (i) the tenant group ML model data 620 at the (higher) tenant group level 604, or (ii) the first tenant ML model data 610 at the (same) tenant level 602.

[0250] Multi-tenant processing-permission setting

[0251] Fig.11 Another example hierarchy 1100 is shown.

[0252] Fig.11 Depicted at the tenant level 602 for a first tenant 606 application Fig. 9 ML model 932 is shown (denoted as “ML model {606, 602}”).

[0253] However, in this example, ML model 932 (at tenant level 602 ) is allowed access to the (higher level) tenant group ML model data 620 , as indicated at item 724 .

[0254] In this particular example, the ML model 932 has been selectively allowed access to the tenant group ML model data 620 based on one or more permission settings enabled for the ML model 932 .

[0255] For example, one or more permission settings may have been previously disabled for ML model 932, such that Fig. 9 As shown, the ML model 932 is prohibited from accessing the tenant group ML model data 620 .

[0256] One or more permission settings may then have been enabled for the ML model 932 , such that the ML model 932 has access to the tenant group ML model data 620 .

[0257] One or more permission settings can be on a per-tenant, per-model, and / or per-level basis.

[0258] For example, one or more permission settings may be enabled for ML model 932 (regarding first tenant 606 ) such that ML model 932 has access to both first tenant ML model data 610 and tenant group ML model data 620 , and may be disabled for ML model 1034 (regarding second tenant 608 ) such that ML model 1034 has access to second tenant ML model data 612 but is prohibited from accessing tenant group ML model data 620 .

[0259] In this example, the ML model 932 can perform real-time anomaly detection associated with the first tenant 606 using the tenant group ML model data 620 and the first tenant ML model data 610 .

[0260] Multi-tenancy handling - Admin level and access control at admin level -

[0261] Example 1 :

[0262] Fig.12 Another example hierarchy 1200 is shown.

[0263] In this example, hierarchy 1200 includes third level 1236. In this example, third level 1236 includes administrator level 1236. In this example, administrator level 1236 includes administrator 1238.

[0264] In this example, administrator 1238 is associated with administrator ML model data 1240. Administrator 1238 can be associated with administrator ML model data 1240 via an administrator ID or otherwise.

[0265] In this example, tenant group 614 belongs to admins 1238. Fig.12 12 is depicted by an arrow 1242 pointing from the tenant group 614 to the administrator 1238 .

[0266] Fig.12 Depicted is an ML model 1244 (denoted as “ML Model {606, 1236}”) applied at the administrator level 1236 with respect to the first tenant 606 .

[0267] In this example, the ML model 1244 is (i) allowed access to the admin ML model data 1240 (as indicated at item 1246), (ii) allowed access to the tenant group ML model data 620 (as indicated at item 724), (iii) allowed access to the first tenant ML model data 610 (as indicated at item 726), and (iv) prohibited access to the second tenant ML model data 612 (as indicated at item 728).

[0268] In this example, the ML model 1244 can use the administrator ML model data 1240 , the tenant group ML model data 620 , and the first tenant ML model data 610 to perform real-time anomaly detection associated with the first tenant 606 .

[0269] Access control may be applied at the administrator level 1236 with respect to the second tenant 608 in a similar manner, whereby the ML model applied at the administrator level 1236 with respect to the second tenant 608 may perform real-time anomaly detection associated with the second tenant 608 using the administrator ML model data 1240 , the tenant group ML model data 620 , and the second tenant ML model data 612 .

[0270] Multi-tenancy handling - Access control at the tenant group level - Example 3

[0271] Fig.13 Another example hierarchy 1300 is shown.

[0272] Fig.13 Depicted is the ML model 1348 (denoted as “ML model {606, 604}”) applied with respect to the first tenant 606 at the tenant group level 604 .

[0273] In this example, the ML model 1348 is (i) prohibited from accessing the admin ML model data 1240 (as indicated at item 1246 ), (ii) permitted to access the tenant group ML model data 620 (as indicated at item 724 ), (iii) permitted to access the first tenant ML model data 610 (as indicated at item 726 ), and (iv) prohibited from accessing the second tenant ML model data 612 (as indicated at item 728 ).

[0274] In this example, the ML model 1348 can use the tenant group ML model data 620 and the first tenant ML model data 610 to perform real-time anomaly detection associated with the first tenant 606 .

[0275] However, ML model 1348 may be given access to administrator ML model data 1240 as described above.

[0276] Multi-tenancy - Additional tenant groups

[0277] Fig.14 Another example hierarchy 1400 is shown.

[0278] For ease of presentation, the above reference Figures 6 to 13 Some elements of the example hierarchy are described in Fig.14 and are represented differently in some subsequent figures.

[0279] In this example, the tenant group level 604 includes additional tenant groups 1450 .

[0280] although Fig.14 , a single additional tenant group 1450 is shown in FIG. 6 , but in other examples, the tenant group level 604 may include more than one additional tenant group 1450 .

[0281] In this example, additional tenant 1452 belongs to additional tenant group 1450. Fig.14 1450 is depicted by arrow 1454 pointing from another tenant 1452 to another tenant group 1450. Fig.14 A single additional tenant 1452 is shown in FIG. 1 , but in other examples, more than one additional tenant 1452 may belong to the additional tenant group 1450 .

[0282] Generally speaking, a hierarchy according to the examples described herein may include one or more tenant groups, and a tenant group may include one or more tenants.

[0283] In this example, the first tenant 606 and the second tenant 608 do not belong to the additional tenant group 1450 , and the additional tenant 1452 does not belong to the tenant group 614 .

[0284] In this example, additional tenant group 1450 belongs to administrators 1238. Fig.14 This is depicted by an arrow 1456 pointing from another tenant group 1450 to the administrator 1238 .

[0285] In this example, the additional tenant group 1450 is associated with the additional tenant group ML model data 1458. For example, the additional tenant group 1450 can be associated with the additional tenant group ML model data 1458 via a (additional) tenant group ID or otherwise.

[0286] In this example, the additional tenant 1452 is associated with the additional tenant ML model data 1460. For example, the additional tenant 1452 may be associated with the additional tenant ML model data 1460 via a (additional) tenant ID or otherwise.

[0287] Multi-tenancy - Access control at the tenant group level - Example 4

[0288] Fig.15 Another example hierarchy 1500 is shown.

[0289] In this example, the ML model 722 is applied with respect to the first tenant 606 at the tenant group level 604 .

[0290] In this example, ML model 722 is (i) prohibited from accessing additional tenant group ML model data 1458 (as indicated at item 1562 ), and (ii) prohibited from accessing additional tenant ML model data 1460 (as indicated at item 1564 ).

[0291] Additionally, in this particular example, the ML model 722 is (i) allowed access to the first tenant ML model data 610 and the tenant group ML model data 620 (as indicated at items 726 and 724, respectively), and (ii) is prohibited from accessing the second tenant ML model data 612 and the administrator ML model data 1240, as indicated at items 728 and 1246, respectively.

[0292] In this example, the ML model 722 can use the tenant group ML model data 614 and the first tenant ML model data 610 to perform real-time anomaly detection associated with the first tenant 606 .

[0293] Multi-tenancy - Access control at the administrator level - Example 2

[0294] Fig.16 Another example hierarchy 1600 is shown.

[0295] In this example, the ML model 1244 is applied at the administrator level 1236 with respect to the first tenant 606 .

[0296] In this example, ML model 1244 (i) is prohibited from accessing additional tenant group ML model data 1458 (as indicated at item 1562), and (i) is prohibited from accessing additional tenant ML model data 1460 (as indicated at item 1564).

[0297] Additionally, in this particular example, the ML model 1244 is (i) allowed access to the first tenant ML model data 610, the tenant group ML model data 620, and the administrator ML model data 1240 (as indicated at items 726, 724, and 1246, respectively), and (ii) is prohibited from accessing the second tenant ML model data 612 (as indicated at item 728).

[0298] In this example, the ML model 1244 can use the administrator ML model data 1240 , the tenant group ML model data 614 , and the first tenant ML model data 610 to perform real-time anomaly detection associated with the first tenant 606 .

[0299] Multi-tenancy handling - Additional tenants and access control at the tenant group level - Example 5

[0300] Fig.17 Another example hierarchy 1700 is shown.

[0301] In this example, hierarchy 1700 includes additional tenant 1766 .

[0302] In this example, additional tenant 1766 is depicted as being included in the same horizontal level as first tenant 606 and second tenant 608 , ie, included in tenant level 602 .

[0303] However, again, this is for ease of explanation. Additional tenant 1766 can be depicted in the same horizontal level as tenant group 614 while still in logical tenant level 602.

[0304] In this example, additional tenants 1766 are associated with additional tenant ML model data 1768. For example, additional tenants 1766 can be associated with additional tenant ML model data 1768 via (n additional) tenant IDs or otherwise.

[0305] In this example, additional tenant 1766 belongs to administrator 1238. Fig.17 17 is depicted by an arrow 1770 pointing from the additional tenant 1766 to the administrator 1238 .

[0306] In this example, the additional tenant 1766 can be said to belong “directly” to the administrator 1238 because there are no intervening elements of the hierarchy 1700 between the additional tenant 1766 and the administrator 1238 .

[0307] Similarly, (i) tenant group 614 may be said to belong "directly" to administrator 1238 because there are no intermediate elements of hierarchy 1700 between tenant group 614 and administrator 1238, and (ii) first tenant 606 and second tenant 608 may be said to belong "directly" to tenant group 614 because there are no intermediate elements of hierarchy 1700 between first tenant 606 and second tenant 608 and tenant group 614.

[0308] However, first tenant 606 and second tenant 608 may be said to belong to administrator 1238 “indirectly” because there is at least one intermediate element of hierarchy 1700 between first tenant 606 and second tenant 608 and administrator 1238. In this example, the at least one intermediate element includes tenant group 614.

[0309] In this example, the ML model 722 is applied with respect to the first tenant 606 at the tenant group level 604 .

[0310] In this example, ML model 722 is disabled from accessing additional tenant ML model data 1768 (as indicated at item 1772 ).

[0311] Additionally, in this particular example, the ML model 722 is (i) allowed access to the first tenant ML model data 610 and the tenant group ML model data 620 (as indicated at items 726 and 724, respectively), and (ii) is prohibited from accessing the second tenant ML model data 612 and the administrator ML model data 1240 (as indicated at items 728 and 1246, respectively).

[0312] Multi-tenancy - Access control at the administrator level - Example 3

[0313] Fig.18 Another example hierarchy 1800 is shown.

[0314] In this example, the ML model 1244 is applied at the administrator level 1238 with respect to the first tenant 606 .

[0315] In this example, ML model 1244 is disabled from accessing additional tenant ML model data 1768 (as indicated at item 1772 ).

[0316] Additionally, in this particular example, the ML model 1244 is (i) allowed access to the first tenant ML model data 610, the tenant group ML model data 620, and the administrator ML model data 1240 (as indicated at items 726, 724, and 1244, respectively), and (ii) is prohibited from accessing the second tenant ML model data 612 (as indicated at item 728).

[0317] although Figures 14 to 16 relates to example hierarchies 1400, 1500, 1600 including at least one additional access group 1450 having at least one additional tenant 1452, and Fig.17 and Fig.18 The example hierarchies 1700 , 1800 , respectively, relate to at least one additional tenant 1766 , but other example hierarchies may include both at least one additional access group 1450 with at least one additional tenant 1452 and at least one additional tenant 1766 .

[0318] Multi-tenancy handling - Predetermined actions in response to a tenant leaving a tenant group

[0319] Fig.19 Another example hierarchy 1900 is shown.

[0320] Fig.19 A scenario is depicted in which the first tenant 606 no longer belongs to the tenant group 614 .

[0321] The first tenant 606 may have been removed from the tenant group 614 based on one or more removal factors. Examples of removal factors include, but are not limited to, user requests, a weak match between the first tenant 606 and the tenant group 614, etc. For example, the first tenant ML model data 610 may represent an average transaction amount associated with the first tenant 606. If the average transaction amount is outside of a standard deviation of the average transaction amounts of other tenants belonging to the tenant group 614, a weak match between the first tenant 606 and the tenant group 614 may be determined.

[0322] The first tenant 606 may now belong to an administrator (not shown) or may belong to another tenant group (not shown). Fig.19 An example is shown in which the first tenant 606 is still part of the hierarchy 1900 , but in other examples, the first tenant 606 may no longer be part of the hierarchy 1900 .

[0323] In some examples, a predetermined action is performed with respect to tenant group ML model data 620 in response to first tenant 606 and / or second tenant 608 no longer belonging to tenant group 614 .

[0324] An example of such a predetermined action is updating the tenant group ML model data 620 such that the tenant group ML model data 620 is no longer based on the first tenant ML model data 610 , or the tenant group ML model data is based on a restricted version of the first tenant ML model data 610 .

[0325] For example, where the tenant group ML model data 620 includes a compilation (such as a list) of the first tenant ML model data 610 and the second tenant ML model data 612 , the first tenant ML model data 610 may be deleted from the compilation.

[0326] In the case where, for example, the tenant group ML model data 620 is based on a historical average of the first tenant ML model data 610 and the second tenant ML model data 612 , the first tenant ML model data 610 may no longer be considered for future averaging, but the contribution from the historical first tenant ML model data 610 may still be maintained in the tenant group ML model data 620 for a certain time.

[0327] While this specific example focuses on a tenant leaving a tenant group, one or more corresponding predetermined actions may be taken in response to a tenant no longer directly belonging to an administrator and / or a tenant group no longer directly belonging to an administrator.

[0328] Multi-tenancy handling - Access control at the tenant group level - Example 6

[0329] Fig. 20 Another example hierarchy 2000 is shown.

[0330] In this example, the hierarchy 2000 includes a tenant level 602 including a first tenant 606 and a second tenant 608 associated with first tenant ML model data 610 and second tenant ML model data 612 , respectively.

[0331] In this example, the hierarchy 2000 includes a tenant group level 604 that includes a tenant group 614 to which at least a first tenant 606 belongs.

[0332] The tenant group 614 is associated with the tenant group ML model data 620. The tenant group ML model data 620 is based on at least the first tenant ML model data 610.

[0333] In this example, the second tenant 608 may or may not belong to the tenant group 614. Fig. 20 608 is depicted by a dashed arrow 2074 pointing from the second tenant 608 to the tenant group 614.

[0334] For example, the second tenant 608 may belong to the tenant group 614, may belong to an administrator (not shown), may belong to another tenant group (not shown), or in other ways.

[0335] In this example, the tenant group ML model data 620 (i) may be based on the first tenant ML model data 610 and (ii) may not be based on the second tenant ML model data 612 , or the tenant group ML model data may be based on a restricted version of the second tenant ML model data 612 .

[0336] In this example, the ML model 722 is (i) allowed access to the tenant group ML model data 620 and the first tenant ML model data 610 (as indicated at items 724 and 726, respectively), and (ii) is prohibited from accessing the second tenant ML model data 612 (as indicated at item 728).

[0337] While this specific example relates to ML model 722, in another example, a machine learning model (not shown) applied at tenant level 602 with respect to first tenant 606 may also (i) be allowed access to tenant group ML model data 724 and first tenant ML model data 610 (as indicated at items 724 and 726, respectively), and (ii) be prohibited access to second tenant ML model data 612 (as indicated at item 728). The machine learning model (not shown) applied at tenant level 602 with respect to first tenant 606 may not have initially been allowed access to tenant group ML model data 620, but may have been granted permission, such as by enabling one or more permission settings.

[0338] Multi-tenancy processing - access control methods

[0339] Fig.21 An example access control method 2100 is shown.

[0340] Example method 2100 controls access to ML model data in a multi-tenant ML system having an ML model data access hierarchy. The ML model data access hierarchy may correspond to one or more example hierarchies described herein or otherwise.

[0341] In this example, the ML model data access hierarchy includes: (i) a first tenant 606 associated with first tenant ML model data 610 ; (ii) a second tenant 608 associated with second tenant ML model data 612 ; and (iii) a tenant group 614 associated with tenant group ML model data 620 .

[0342] As indicated by item 2102 , at a first time: (i) the first tenant 606 belongs to the tenant group 614 and the second tenant 608 does not belong to the tenant group 614 ; and (ii) the tenant group ML model data 620 is based on the first tenant ML model data 610 and the tenant group ML model data 620 is not based on the second tenant ML model data 612 .

[0343] As indicated by item 2104 , at a later second time (ie, later than the first time): (i) the first tenant 606 and the second tenant 608 belong to the tenant group 614 ; and (ii) the tenant group ML model data 620 is based on at least the first tenant ML model data 610 .

[0344] As indicated by item 2106 , at a later third time (ie, later than the second time): the tenant group ML model data 620 is based on the first tenant ML model data 610 and the second tenant ML model data 612 .

[0345] Multi-tenant processing - multi-tenant machine learning system

[0346] Fig. 22 An example multi-tenant ML system 2200 is shown.

[0347] In this example, system 2200 has access to ML model data access hierarchy 2202. ML model data access hierarchy 2202 may correspond to one or more example hierarchies described herein or otherwise.

[0348] The hierarchy 2202 includes multiple tenants 2204 and includes one or more tenant groups 2206.

[0349] The system 2200 includes an ML model data access controller 2208. The ML model data access controller 2208 is configured to control access to the ML model data 2210 according to the ML model data access hierarchy 2202.

[0350] The ML model data access controller 2208 may include, may be implemented using, or may otherwise employ, one or more processors and one or more memories to perform ML model data access control. The one or more memories may include instructions that, when executed by the one or more processors, cause the one or more processors to perform ML model data access control.

[0351] Multi-tenancy handling - Machine learning model data access hierarchy - Example 2

[0352] Fig.23 An example hierarchy 2300 is shown.

[0353] In this example, the hierarchy includes (i) a tenant group 614 belonging to the administrator 1238 and including the first tenant 606 and the second tenant 608 , (ii) an additional tenant 1766 directly belonging to the administrator 1238 , and (iii) another tenant group 1450 belonging to the administrator 1238 and including another tenant 1452 .

[0354] Multi-tenancy processing - multi-level access control - example 1

[0355] Fig.24 An example hierarchy 2400 is shown.

[0356] The example hierarchy 2400 corresponds to the above reference Fig.23 The example hierarchy 2300 is described above, except that the example hierarchy 2400 additionally uses a dashed curve 2476 to depict access control regarding the second tenant 608.

[0357] In this example, the active portion of the hierarchy 2400 is indicated by the dashed curve 2476 when events are being processed for the second tenant 608 .

[0358] In this specific example, for a model active at the administrator level with respect to the second tenant 608, the model (i) has access to administrator ML model data, tenant group ML model data, and second tenant ML model data, and (ii) does not have access to first tenant ML model data, additional tenant ML model data, additional tenant group ML model data, or additional tenant ML model data.

[0359] In this specific example, for a model active at the tenant group level with respect to the second tenant 608, the model (i) has access to the tenant group ML model data and the second tenant ML model data, (ii) does not have access to the first tenant ML model data, the additional tenant ML model data, the additional tenant group ML model data, or the additional tenant ML model data, and (iii) may have access to the administrator ML model data.

[0360] In this specific example, for a model active at the tenant level with respect to a second tenant 608, the model (i) has access to the second tenant ML model data, (ii) does not have access to the first tenant ML model data, the additional tenant ML model data, the additional tenant group ML model data, or the additional tenant ML model data, and (iii) may have access to the administrator ML model data and / or the tenant group ML model data.

[0361] Thus, in this specific example, none of the models active for the second tenant 608 have access to the first tenant ML model data, the additional tenant ML model data, the additional tenant group ML model data, or the additional tenant ML model data. As explained above, this provides a high degree of data separation between different tenants.

[0362] However, in other examples, such models may be given full or partial access to the first tenant ML model data, the additional tenant ML model data, the additional tenant group ML model data, or the additional tenant ML model data.

[0363] For example, the business relationship may be changed between the second tenant 608 and the additional tenant 1766 so that the second tenant 608 may have full or partial access to the additional tenant ML model data. This may provide a loose data separation between the second tenant 608 and the additional tenant 1766 while still providing a strong data separation relative to other tenants.

[0364] This possibility of relaxed data separation applies similarly to the other examples described herein.

[0365] Multi-tenancy handling - multi-level access control - example 2

[0366] Fig.25 An example hierarchy 2500 is shown.

[0367] The example hierarchy 2500 corresponds to the above reference Fig.23 The example hierarchy 2300 is described above, except that the example hierarchy 2500 additionally uses a dashed curve 2578 to depict access control regarding another tenant 14520.

[0368] In this example, the active portion of the hierarchy 2500 is indicated by the dashed curve 2578 when events are being processed for the additional tenant 1452 .

[0369] In this specific example, for a model active at the administrator level with respect to additional tenant 1452, the model (i) has access to administrator ML model data, additional tenant group ML model data, and additional tenant ML model data, and (ii) does not have access to first tenant ML model data, second tenant ML model data, tenant group ML model data, or additional tenant ML model data.

[0370] In this specific example, for a model active at the tenant group level with respect to additional tenant 1452, the model (i) has access to additional tenant group ML model data and additional tenant ML model data, (ii) does not have access to first tenant ML model data, second tenant ML model data, tenant group ML model data, or additional tenant ML model data, and (iii) may have access to administrator ML model data.

[0371] In this specific example, for a model active at the tenant level with respect to another tenant 1452, the model (i) has access to the other tenant ML model data, (ii) does not have access to the first tenant ML model data, the second tenant ML model data, the tenant group ML model data, or the additional tenant ML model data, and (iii) may have access to the administrator ML model data and / or the other tenant group ML model data.

[0372] Multi-tenant processing - additional technical effects

[0373] As explained above, splitting systems by use case or internal organization may involve doing multiple integrations and combining multiple systems together. This can lead to difficulties in the initial implementation as well as difficulties in situations where new tenants are to be added.

[0374] In contrast, the examples described herein keep analysis and configuration clearly delineated.

[0375] Additionally, in contrast to systems where rules must directly reference specific subsets of data, tenant groups as described herein provide clean logical buckets under which tenants are located.

[0376] The example also effectively provides a layer of analytics policy to groups of tenants.

[0377] The example multi-tenancy solution described herein is particularly flexible. Some users may have view-only permissions, while other users (e.g., with their own fraud team) may be able to create and edit their own rules on the same solution.

[0378] Additionally, the solution can be adapted to leverage federated data across other users, or, where applicable, exclude them if this may impact the performance of the model.

[0379] Furthermore, a given user may provide the services described herein to other users (eg, using tenant groups), thereby leveraging their own fraud solutions with respect to other users, rather than just using their fraud solutions to solve their own fraud-related challenges.

[0380] Therefore, the creation of tenant groups with multi-tenancy and cascading learning across groups without having to create separate instances of the system is an efficient, targeted, and configurable solution.

[0381] Machine learning typically involves large amounts of data. The hierarchical structure described in this article enables efficient access to different parts of such data.

[0382] Data isolation can also be particularly relevant to machine learning data, especially in the context of fraud detection. The examples described in this article use hierarchies to take advantage of the efficient data access they provide as described above, while also providing

[0383] Controlled access to machine learning data. In particular, the hierarchical structure described herein enables complete data separation between different tenants while enabling higher-level entities to access ML model data for multiple different tenants.

[0384] Multi-tenancy handling - example use case

[0385] Non-limiting example use cases of the systems and methods described herein will be provided.

[0386] In this example, the hierarchy includes an administrator, a first tenant group including a first tenant and a second tenant, and a second tenant group including a third tenant and a fourth tenant.

[0387] In this example, the first tenant and the second tenant are merchants in the same business domain as each other, and the first tenant group is associated with the business domain associated with the first tenant and the second tenant. Similarly, the third tenant and the fourth tenant are merchants in the same business domain as each other, and the second tenant group is associated with the business domain associated with the third tenant and the fourth tenant.

[0388] Different business areas may be associated with different levels of fraud risk, may have different rules, may have different data maturity properties, etc.

[0389] The first tenant group tenant ML model data associated with the first tenant group may be based on the first tenant ML model data and the second tenant ML model data associated with the first tenant and the second tenant, respectively. Similarly, the second tenant group tenant ML model data associated with the second tenant group may be based on the third tenant ML model data and the fourth tenant ML model data associated with the third tenant and the fourth tenant, respectively.

[0390] In each example, a model running on a given tenant cannot directly access the ML model data of another tenant, but the model may be able to access the ML model data of a tenant group including the other tenant and / or the administrator ML model data. This provides effective data separation within a single multi-tenant platform.

[0391] An analyst associated with, for example, a first tenant may be able to perform analysis based on the first tenant ML model data but not based on the second tenant ML model data.

[0392] However, an analyst associated with the first tenant group (e.g., an account manager for a business area associated with the first tenant and the second tenant) may be able to perform analysis based on the first tenant ML model data, the second tenant ML model data, and the first tenant group ML model data.

[0393] In this example, over time, the characteristics of the third tenant traffic change to become more like the first tenant traffic and the second tenant traffic.

[0394] Thus, the third tenant may be moved to the first tenant group, such that the first tenant, the second tenant, and the third tenant belong to the first tenant group, and the fourth tenant belongs to the fourth tenant group.

[0395] It should be understood that the third tenant can be easily moved from the second tenant group to the first tenant group by updating data (such as metadata) so that the third tenant is associated with the first tenant group instead of the second tenant group. This move occurs within a single platform as described herein, which is not possible when different installers are used for different tenants.

[0396] Although the third tenant may now have characteristics that are more closely aligned with those of the first and second tenants, the third tenant ML model data may include at least some historical data. If the first tenant group ML model data is updated to include all third tenant ML model data (particularly historical data) as soon as the third tenant joins, the first tenant group ML model data may be less representative of the (current) first tenant ML model data, second tenant ML model data, and third tenant ML model data than if the historical third tenant ML model data is not used or is used in a limited manner.

[0397] Over time, and given that the characteristics of the third tenant are more similar to the characteristics of the first and second tenants, a higher (eg, total) contribution from the third tenant ML model data may be used for the first tenant group ML model data.

[0398] Similarly, once the third tenant is no longer in the second tenant group, the contribution of the third tenant ML model data to the second tenant group ML model data can be removed or reduced. For example, if the third tenant ML model data is {m_31, m_32, m_33, ...}, the fourth tenant ML model data is {m_41, m_42, m_43, ...}, and the second tenant group ML model data is a compilation of the third tenant ML model data and the fourth tenant ML model data, such as {m_31, m_32, m_33, ...; m_41, m_42, m_43, ...}, then some or all of the third tenant ML model data in the third tenant ML model data can be easily deleted from the second tenant group ML model data, which takes effect immediately when the third tenant leaves the second tenant group.

[0399] Multi-tenancy - Additional features

[0400] In the examples described above, a given tenant either belongs to a single tenant group or directly to an administrator. In other words, in such examples, a tenant may have only one parent.

[0401] If multiple types of analytics are being run, multiple different tenant IDs can be used within the system to represent each tenant for different analytics hierarchies.

[0402] However, more generally, a given tenant may belong to one or more tenant groups and / or may belong directly to an administrator. For example, a given tenant may belong to two different tenant groups, may belong to one tenant group, and may also belong directly to an administrator, etc.

[0403] In the examples described above, the hierarchy comprises two, three or four (logical) levels. However, in other examples, the hierarchy may comprise a different number of levels.

[0404] The examples described above involve models in the form of machine learning models.

[0405] In other examples, another type of model, more specifically a learning model, may be used. An example of another type of learning model is a rule-based learning model. An example of a rule-based learning model is an adaptive rule set model, which may also be referred to as a rule-based adaptive model. Therefore, although the examples described above involve machine learning models, they may also be applied to other types of learning models, such as but not limited to adaptive rule sets. References to "machine learning model", including, for example, in terms such as "machine learning model data", should be understood accordingly.

[0406] Although specific embodiments of the present invention have been described in detail, it will be understood by those skilled in the art that the embodiments described in detail do not limit the scope of the invention as claimed.

Claims

1. A method for controlling access to machine learning model data in a multi-tenant machine learning system having a machine learning model data access hierarchy, the machine learning model data access hierarchy comprising: a tenant level, the tenant level comprising a first tenant and a second tenant associated with the first tenant machine learning model data and the second tenant machine learning model data, respectively; and a tenant group level, the tenant group level comprising a tenant group to which the first tenant and the second tenant belong, wherein the tenant group is associated with tenant group machine learning model data, and wherein the tenant group machine learning model data is based on the first tenant machine learning model data and / or the second tenant machine learning model data, The method includes: for a machine learning model applied with respect to the first tenant at the tenant group level, allowing access to the tenant group machine learning model data and the first tenant machine learning model data; and Access to the second tenant's machine learning model data is prohibited.

2. The method according to claim 1, wherein: The tenant group machine learning model data is based on the first tenant machine learning model data; and The tenant group machine learning model data is not based on the second tenant machine learning model data, or the tenant group machine learning model data is based on a restricted version of the second tenant machine learning model data. 3 . The method according to claim 1 , wherein the second tenant joins the tenant group after the first tenant joins the tenant group.

4. The method according to any one of claims 1 to 3, wherein: The tenant group machine learning model data is based on at least a historical set of the first tenant machine learning model data; and The tenant group machine learning model data is at least not based on the historical set of the second tenant machine learning model data, or the tenant group machine learning model data is based on a restricted version of the historical set of the second tenant machine learning model data.

5. The method of claim 4, wherein the historical set of the second tenant machine learning model data is generated before the second tenant joins the tenant group.

6. A method according to any one of claims 1 to 5, wherein the tenant group machine learning model data is updated to generate updated tenant group machine learning model data, and wherein the updated tenant group machine learning model data is based on both the first tenant machine learning model data and the second tenant machine learning model data.

7. The method of claim 1, wherein the tenant group machine learning model data is based on at least some of each of the first machine learning model data and the second machine learning model data.

8. A method according to claim 7, wherein the tenant group machine learning model data is derived based on one or more operations involving at least some of each of the first machine learning model data and the second machine learning model data.

9. The method according to any one of claims 1 to 8, wherein the method comprises: for a machine learning model applied at the tenant group level with respect to the second tenant, allowing access to the tenant group machine learning model data and the second tenant machine learning model data; and Access to the first tenant's machine learning model data is prohibited.

10. The method according to any one of claims 1 to 9, comprising: For the machine learning model applied at the tenant level with respect to the first tenant and / or the second tenant, access to the tenant group machine learning model data is prohibited.

11. The method according to any one of claims 1 to 9, comprising: For the machine learning model applied at the tenant level with respect to the first tenant and / or the second tenant, access to the tenant group machine learning model data is selectively allowed based on one or more permission settings enabled for the machine learning model applied at the tenant level with respect to the first tenant and / or the second tenant.

12. A method according to any one of claims 1 to 11, wherein the machine learning model data access hierarchy includes an administrator level, the machine learning model is applicable to the first tenant at the administrator level, wherein the administrator level includes administrators associated with administrator machine learning model data, and wherein the tenant group belongs to the administrator.

13. The method according to claim 12, comprising: for the machine learning model applied at the administrator level with respect to the first tenant, allowing access to the administrator machine learning model data, the tenant group machine learning model data, and the first tenant machine learning model data; and Access to the second tenant's machine learning model data is prohibited.

14. A method according to claim 12 or 13, wherein the tenant group level includes an additional tenant group to which at least one additional tenant belongs but to which the first tenant and the second tenant do not belong, wherein the additional tenant group belongs to the administrator, wherein the additional tenant group is associated with additional tenant group machine learning model data, and wherein the at least one additional tenant is associated with additional tenant machine learning model data.

15. The method according to claim 14, wherein the method comprises: For the machine learning model applied with respect to the first tenant at the tenant group level, access to the additional tenant group machine learning model data and / or the additional tenant machine learning model data is prohibited.

16. The method according to claim 14 or 15, wherein the method comprises: For the machine learning model applied with respect to the first tenant at the administrator level, access to the additional tenant group machine learning model data and / or the additional tenant machine learning model data is prohibited.

17. A method according to any one of claims 12 to 16, wherein the machine learning model data access hierarchy includes additional tenants associated with additional tenant machine learning model data, and wherein the additional tenants belong to the administrator.

18. The method according to claim 17, wherein the method comprises: For the machine learning model applied with respect to the first tenant at the tenant group level, access to the additional tenant machine learning model data is prohibited.

19. The method according to claim 17 or 18, wherein the method comprises: For the machine learning model applied at the administrator level with respect to the first tenant, access to the additional tenant machine learning model data is prohibited.

20. The method according to any one of claims 1 to 19, wherein the method comprises: In response to the first tenant and / or the second tenant no longer belonging to the tenant group, performing a predetermined action on the machine learning model data of the tenant group.

21. A method according to any one of claims 1 to 19, wherein the machine learning model data includes machine learning model state data and / or machine learning model parameter data.

22. A method according to any one of claims 1 to 20, wherein the tenant group machine learning model data and / or the first tenant machine learning model data are used to perform real-time anomaly detection associated with the first tenant.

23. A method for controlling access to learning model data in a multi-tenant system having a learning model data access hierarchy, the learning model data access hierarchy comprising: a tenant level, the tenant level including a first tenant and a second tenant associated with the first tenant learning model data and the second tenant learning model data, respectively; and a tenant group level, the tenant group level comprising at least a tenant group to which the first tenant belongs, wherein the tenant group is associated with tenant group learning model data, and wherein the tenant group learning model data is based on at least the first tenant learning model data, The method comprises: for a learning model for the first tenant application at the tenant group level and / or for a learning model for the first tenant application at the tenant level, Allowing access to the tenant group learning model data and the first tenant learning model data; as well as Access to the second tenant's learning model data is prohibited.

24. A method for controlling access to machine learning model data in a multi-tenant machine learning system having a machine learning model data access hierarchy, the machine learning model data access hierarchy comprising: a first tenant, the first tenant being associated with first tenant machine learning model data; a second tenant, the second tenant being associated with the second tenant machine learning model data; and A tenant group, the tenant group being associated with tenant group machine learning model data, the method comprising controlling access to the machine learning model data such that: At the first time: The first tenant belongs to the tenant group, and the second tenant does not belong to the tenant group; and The tenant group machine learning model data is based on the first tenant machine learning model data, and the tenant group machine learning model data is not based on the second tenant machine learning model data; At a later second time: The first tenant and the second tenant belong to the tenant group; and The tenant group machine learning model data is based at least on the first tenant machine learning model data; as well as At a later third time: The tenant group machine learning model data is based on the first tenant machine learning model data and the second tenant machine learning model data.

25. A multi-tenant machine learning system, configured to execute the method according to any one of claims 1 to 24.

26. A multi-tenant machine learning system having access to a machine learning model data access hierarchy comprising multiple tenants and one or more tenant groups, the multi-tenant machine learning system comprising a machine learning model data access controller configured to control access to machine learning model data according to the machine learning model data access hierarchy.

Images