Multi-scale self-learning

Abstract

Methods and systems for generating embeddings using a multi-scale machine learning model (also referred to as an "encoder model") and performing various processes (e.g., classification) based on those embeddings are disclosed. Additionally disclosed are contrastive learning methods that can be used to train machine learning models according to embodiments. A data sequence can be separated into one or more sets of data subsequences based on window lengths (e.g., "long", "medium", and "short" window lengths). The machine learning model, comprising a sequence module and one or more window modules, can process the data sequence and the one or more sets of data subsequence to generate a set of partial sequence embeddings and one or more sets of partial window embeddings (which can be generated, in part, using the set of partial sequence embeddings). The one or more sets of window embeddings can be combined to produce an output embedding.

Description

PATENT Attorney Docket No.079900-1472209 Client Ref. No.8209WO01 MULTI-SCALE SELF-LEARNING CROSS-REFERENCES TO RELATED APPLICATIONS

[0001] This application is an international patent application which claims the benefit of the filing date of U.S. Patent Application No.63 / 553,376, filed on February 14, 2024, which is herein incorporated by reference in its entirety for all purposes. BACKGROUND

[0002] Various actions or activities can be performed based on the analysis of series or sequences of data. For example, an access control system can approve or deny access to a computer file or account based on an analysis of a sequence of prior access request data. However, many sequential data analysis techniques may be inaccurate in some applications. In turn, this may cause processes based on such analytics, e.g., access control, forecasting, etc., to become inaccurate or otherwise erroneous. For example, an access control system may fail to identify an anomalous access request because an “average access request” (used as the basis of the classification of the anomalous access request) fails to capture or communicate patterns indicative of anomalous access to a system or resource.

[0003] Embodiments of the present disclosure address this and other problems, individually and collectively. SUMMARY

[0004] This Summary is provided to introduce a selection of concepts in a simplified form that are further described herein in the Detailed Description. This Summary is not intended to identify key factors or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0005] Embodiments of the present disclosures are directed to various machine learning methods and systems (e.g., computer systems) performing such methods and instantiating relevant machine learning models. Such methods and systems can be used to perform various actions based on multi-scale processing of sequential data using a novel hierarchical machine learning model. Methods according to embodiments can be applied to various useful tasks.1 KILPATRICK TOWNSEND 792723011For example, methods according to embodiments can be used to perform forecasting or anomaly detection tasks. As a more specific example, embodiments of the present disclosure can be used to improve the accuracy of an access review system, e.g., a system that reviews access requests to a resource and e.g., automatically approves or denies access requests to that resource. A hierarchical machine learning model according to embodiments can be used to generate an output embedding based on an input data sequence (e.g., corresponding to historical access requests to a resource), which could then be used as the input to a classification model used to classify access requests as legitimate (normal) or invalid (anomalous) as part of an access review system.

[0006] In more detail, a machine learning model according to embodiments can comprise a sequence encoder and one or more window encoders. A computer system can use the sequence encoder to generate a set of sequence partial embeddings. The set of sequence partial embeddings can generally contain or represent information corresponding to the relationships between individual data elements in the data sequence. The data sequence can further be broken up into data subsequences based on window lengths. The computer system can use one or more window encoders and the set of sequence partial embeddings to generate one or more sets of window partial embeddings corresponding to these data subsequences, which can generally contain or represent information corresponding to the relationships between subsequences of data elements, rather than individual data elements. In this way, while the set of sequence partial embeddings may represent more general “high-level” patterns or features of the data sequence, the one or more sets of window partial embeddings may capture patterns that are more specific to corresponding window lengths. For example, if a window length generally corresponds to one week of data (e.g., one week of access requests, one week of precipitation measurements, etc.), then a window partial embedding corresponding to that window length may generally capture patterns or relational information in the data sequence corresponding to weekly variations.

[0007] A machine learning model according to embodiments can be organized in a “hierarchical” or “pyramid” configuration, and can comprise multiple windows encoders, each corresponding to a different window length, and thereby generate multiple window partial embeddings corresponding to patterns or relationships in the data sequence at multiple scales. For example, a first window encoder could correspond to a relatively long window length, and each successive window encoder could correspond to a shorter window length. In this way the computer system can generate window partial embeddings corresponding to2 KILPATRICK TOWNSEND 792723011successively more granular patterns or relationships in the data sequence. Further, partial embeddings generated by the sequence encoder or window encoders can be used by successive window encoders to generate their respective window partial embeddings. For example, a sequence partial embedding can be used to generate a first window partial embedding (corresponding to a first window encoder), which could itself be used to generate a second window partial embedding (corresponding to a second window encoder), etc. In this way, partial embeddings from a previous “scale” can be used to enhance the partial embeddings corresponding to subsequent scales. One or more sets of window partial embeddings can be combined to produce an output embedding, which can generally comprise the output of the machine learning model.

[0008] This novel model architecture enables a computer system according to embodiments to produce output embeddings that more accurately reflect the useful information contained in the sequence of time series data, which could enable, e.g., more accurate predictions, forecasts, classifications of access requests, etc., generated based on those output embeddings.

[0009] Further, some embodiments are directed to methods for training a machine learning model (such as the machine learning model described above) using self-supervised contrastive learning methods. In general, a computer system according to embodiments can mask a training data sequence (e.g., comprising time series data) multiple times, thereby generating a plurality of masked training data sequences. The computer system can then input those masked training data sequences into a machine learning model (such as the machine learning model described above), thereby generating a plurality of training output embeddings. A multi-pair contrastive loss can be generated based on this plurality of training output embeddings, which can be used to train the machine learning model. This process can be repeated for e.g., a number of training rounds or epochs, or until a convergence condition has been met, thereby training the machine learning model. The multi-pair contrastive loss can generally compare the embeddings corresponding to different masked training data sequences and e.g., penalize the machine learning model for generating similar training output embeddings for masked training data sequences generated from different training data sequences or generating dissimilar training output embeddings for masked training data sequences generated from the same training data sequence.3 KILPATRICK TOWNSEND 792723011

[0010] There are several advantages of contrastive learning methods according to embodiments. These methods can be used to train a machine learning model to extract relevant information from data sequences without consideration for the task or eventual application of generated output embeddings (e.g. evaluating access requests, predicting future power demand, etc.). As a result, machine learning models according to embodiments can be trained “offline” and without any label information. Later, some other machine learning model (e.g., a classification model used to classify access requests as legitimate or invalid) could be trained using supervised learning and generated output embeddings and corresponding labels (e.g., indicating that a given embedding corresponded to a legitimate access request). This improves the time-efficiency of training machine learning models, a process that can require significant amounts of time and computing resources.

[0011] In summary, one embodiment is directed to a method performed by a computer system for generating a classification corresponding to a data sequence using a machine learning model comprising a sequence encoder and one or more window encoders. The computer system can retrieve the data sequence. The computer system can generate one or more sets of data subsequences from the data sequence based on one or more window lengths. Each data subsequence can comprise a number of sequential data elements from the data sequence. Each number of sequential data elements can correspond to a window length of one or more window lengths. Further, each set of data subsequences can correspond to a window encoder of the one or more window encoders. The computer system can generate a set of sequence partial embeddings by applying the data sequence to the sequence encoder. The computer system can generate one or more sets of window partial embeddings by applying the one or more sets of data subsequences and the set of sequence partial embeddings to the one or more window encoders. The computer system can generate an output embedding by combining the one or more sets of window partial embeddings. A classification can be generated based on the output embedding.

[0012] Another embodiment is directed to a method performed by a computer system for training a machine learning model (e.g., comprising a sequence encoder and one or more window encoders, as described above) based on contrastive loss. The computer system can train the machine learning model by performing an iterative training process until a terminating condition has been met. The computer system can sample one or more training data sequences. For each training data sequence, the computer system can generate a plurality of masked training data sequences by masking the training data sequence. In this4 KILPATRICK TOWNSEND 792723011way, the computer system can generate one or more pluralities of masked training data sequences. The computer system can generate a plurality of training data subsequences based on the training data sequence or the one or more pluralities of masked training data sequences. Each training data subsequence can comprise a number of sequential data elements from the training data sequence or a number of sequential masked data elements from a masked training data sequence (of the one or more pluralities of masked training data sequences). Each number of sequential data elements or number of sequential masked data elements can correspond to a window length of one or more window lengths. The computer system can use the machine learning model to generate one or more pluralities of training output embeddings based on the one or more pluralities of masked training data sequences and the plurality of training data subsequences. The computer system can identify a plurality of pairs of training output embeddings from the one or more pluralities of training output embeddings. For each pair of training output embeddings, the computer system can determine an embedding distance value. In this way, the computer system can determine a plurality of embedding distance values. The computer system can determine a contrastive loss value based on the plurality of embedding distance values. The computer system can update a parameter set of the machine learning model based on the contrastive loss value, thereby training the machine learning model. The computer system can repeat the iterative training process until the terminating condition has been met.

[0013] Some other embodiments are directed to computer systems or other devices that can be configured to perform the method described above or other methods. For example, one embodiment is directed to a computer system comprising one or more processors and a non- transitory computer readable medium coupled to the one or more processors. The non- transitory computer readable medium can comprise instructions that, when executed by the one or more processors, cause the one or more processors to perform the method described above (or other methods described in the detailed description below). TERMS

[0014] A “server computer” may refer to a computer or cluster of computers. A server computer may be a powerful computing system, such as a large mainframe. Server computers can also include minicomputer clusters or a group of servers functioning as a unit. In one example, a server computer can include a database server coupled to a web server. A server computer may comprise one or more computational apparatuses and may use any of a variety5 KILPATRICK TOWNSEND 792723011of computing structures, arrangements, and compilations for servicing requests from one or more client computers.

[0015] A “client computer” may refer to a computer or cluster of computers that receives some service from a server computer (or another computing system). The client computer may access this service via a communication network such as the Internet or any other appropriate communication network. A client computer may make requests to server computers including requests for data. As an example, a client computer can request a video stream from a server computer associated with a movie streaming service. As another example, a client computer may request data from a database server. A client computer may comprise one or more computational apparatuses and may use a variety of computing structures, arrangements, and compilations for performing its functions, including requesting and receiving data or services from server computers.

[0016] A “memory” may refer to any suitable device or devices that may store electronic data. A suitable memory may comprise a non-transitory computer readable medium that stores instructions that can be executed by a processor to implement a desired method. Examples of memories including one or more memory chips, disk drives, etc. Such memories may operate using any suitable electrical, optical, and / or magnetic mode of operation.

[0017] A “processor” may refer to any suitable data computation device or devices. A processor may comprise one or more microprocessors working together to achieve a desired function. The processor may include a CPU that comprises at least one high-speed data processor adequate to execute program components for executing user and / or system generated requests. The CPU may be a microprocessor such as AMD’s Athlon, Duron and / or Opteron; IBM and / or Motorola’s PowerPC; IBM’s and Sony’s Cell processor; Intel’s Celeron, Itanium, Pentium, Xenon, and / or Xscale; and / or the like processor(s).

[0018] A “feature” can be an individual measurable property or characteristic of a phenomenon. One or more features can be described using a “feature vector,” e.g., a structured list of data (such as numerical data) representing those features. A feature can be input into a model to determine an output. As an example, in pattern recognition and machine learning, a feature vector can comprise an n-dimensional vector of numerical features that represent some object. In some machine learning contexts, a numerical representation of objects can facilitate processing and statistical analysis. For image6 KILPATRICK TOWNSEND 792723011processing, for example, feature values might correspond to the pixels of an image. As another example, when feature vectors represent text, the features may comprise occurrence frequency of textual terms. Feature vectors can be equivalent to the vectors of explanatory variables used in statistical procedures such as linear regression.

[0019] A “data set” may include any set of one or more “observations” or “data values.” A “data value” can include any data element. A “data element” can refer to a set of data that can be grouped into a single unit, enabling comparison between that data element and other data elements. For example, a data element can comprise a single numerical value (e.g., the speed of a vehicle in miles per hour) or could comprise multiple numerical values (e.g., 60 speed recordings of a vehicle corresponding to each minute of an hour-long period). Data elements comprising multiple data values can be organized into various forms or structures, including data vectors, data tables, and “data sequences” comprising e.g., ordered lists of data elements. A data element may comprise the input to a machine learning model, and individual data values within that data element may comprise features. A data value can comprise a “data vector,” one or more values (represented in vector form) corresponding to a data element or observation.

[0020] “Time series data” may comprise a sequence of data (or individual data elements) for which each data value has an associated “time value,” “time stamp,” or “time indicator.” A “time series data value” may include any data value corresponding to time series data. A “time value” can represent or correspond to the time at which the corresponding data value was observed or collected. Time series data is often ordered chronologically based on time values. A “segment” (sometimes referred to as a “sequence”) of time series data can comprise a set of (usually sequential) time series data observations. A “subsegment” (or “subsequence”) of time series data can comprise a subset of (usually sequential) time series data observations derived from a segment.

[0021] “Sampling” may include any process or method used to collect data values. Sampling can be used to collect data values from an existing data set. The act of sampling may result in a “sample,” one or more data values collected from the data set during sampling. Data sets can be sampled via a variety of means. For example, “random sampling” involves sampling data values from a data set randomly. A “window” or “window of data” may include any number of contiguous data elements from a data set. A “window” may be defined by a starting data value and an ending data value, such that the window7 KILPATRICK TOWNSEND 792723011contains all data values between the starting data value and ending data value (and optionally the starting data value and ending data values themselves). “Window sampling” can be used to sample data values contained within a window of data. A “stride” may refer to the rate at which a window “moves” across a sequence of data during e.g., “rolling window sampling” of that sequence of data. For example, with a stride of one, a window may move one data element “forward” in a sequence of data during each sampling operation, while with a stride of three, a window may move three data elements “forward” in a sequence of data during each sampling operation.

[0022] The term “artificial intelligence model” or “machine learning model” can include a model that may be used to predict outcomes to achieve a pre-defined goal. A machine learning model may be developed using a learning process, in which training data is classified based on known or inferred patterns.

[0023] "Machine learning" can include an artificial intelligence process in which software applications may be trained to make accurate predictions through learning. The predictions can be generated by applying input data to a predictive model (or “prediction model”) formed from performing statistical analyses on aggregated data. A model can be trained using training data, such that the model may be used to make accurate predictions. The prediction can be, for example, a classification of an image (e.g., identifying images of cats on the Internet) or as another example, a recommendation (e.g., a movie that a user may like or a restaurant that a consumer might enjoy).

[0024] A “machine learning model” (ML model) can refer to a software module configured to be run on one or more processors to provide a classification or numerical value of a property of one or more samples. An ML model can include various parameters (e.g., for coefficients, weights, thresholds, functional properties of function, such as activation functions). As examples, an ML model can include at least 10, 100, 1,000, 5,000, 10,000, 50,000, 100,000, or one million parameters. An ML model can be generated using sample data (e.g., training samples) to make predictions on test data. Various number of training samples can be used, e.g., at least 10, 100, 1,000, 5,000, 10,000, 50,000, 100,000, or at least 200,000 training samples. One example is an unsupervised learning model. Examples of unsupervised learning models include hidden Markov model (HMM), clustering (e.g., hierarchical clustering, k-means, mixture models, model-based clustering, density-based spatial clustering of applications with noise (DBSCAN), and OPTICS algorithm), approaches8 KILPATRICK TOWNSEND 792723011for learning latent variable models such as Expectation–maximization algorithm (EM), method of moments, and blind signal separation techniques (e.g., principal component analysis, independent component analysis, non-negative matrix factorization, singular value decomposition), and anomaly detection (e.g., local outlier factor and isolation forest). Another example type of model is supervised learning that can be used with embodiments of the present disclosure. Example supervised learning models may include different approaches and algorithms including analytical learning, statistical models, artificial neural network (e.g. including convolutional and / or transformer layers) that may have 1-10 layers as examples, recurrent neural network (e.g., long short term memory, LSTM), boosting (meta- algorithm), bootstrap aggregating (bagging) such as random forests, support vector machine (SVM), support vector (SVR), Bayesian statistics, case-based reasoning, decision tree learning, inductive logic programming, linear regression, logistic regression, Gaussian process regression, genetic programming, group method of data handling, kernel estimators, learning automata, learning classifier systems, minimum message length (decision trees, decision graphs, etc.), multilinear subspace learning, naive Bayes classifier, maximum entropy classifier, conditional random field, nearest neighbor algorithm, probably approximately correct learning (PAC) learning, ripple down rules, a knowledge acquisition methodology, symbolic machine learning algorithms, subsymbolic machine learning algorithms, minimum complexity machines (MCM), ordinal classification, data pre- processing, handling imbalanced datasets, statistical relational learning, or Proaftn (a multicriteria classification algorithm), or an ensemble of any of these types. Supervised learning models can be trained in various ways using various cost / loss functions that define the error from the known label (e.g., least squares and absolute difference from known classification) and various optimization techniques, e.g., using backpropagation, steepest descent, conjugate gradient, and Newton and quasi-Newton techniques.

[0025] The process of “training” a machine learning model may include any steps used to prepare a machine learning model to perform some task. Often training involves determining or optimizing a set of “parameters” (which characterize the machine learning model) that result in acceptable model performance. Training can be performed in a series of “training rounds” during which training data is used to update the parameters of the machine learning model, for example, based on a loss value.

[0026] A “loss value” or “error value” may include any value that indicates the deviation between a result of some process, method, or function and an expected, desired, or correct9 KILPATRICK TOWNSEND 792723011result. For example, if a machine learning model can detect anomalies in a data set comprising 100 data values, 17 of which are anomalous, if the machine learning model only detects 15 of the 17 anomalous data values, the loss value could comprise, e.g., 2 (17 – 15). Loss values can be used to train and evaluate the training of machine learning models, e.g., by optimizing machine learning model parameters by minimizing the loss value, using processes such as stochastic gradient descent or backpropagation.

[0027] A “hyperparameter” can include any value used to configure a machine learning model that is external to the machine learning model. Typically, a hyperparameter is set, and is not estimated or determined from the training data that is used to train the machine learning model.

[0028] A machine learning model may comprise multiple “sub-models”, “layers,” or “modules”, which may refer to parts of a larger machine learning system. For example, a machine learning model could comprise a long short-term memory layer (which itself can comprise multiple layers), in addition to an attention layer and a linear layer. Layers can sometimes be organized in series, such that the input to a machine learning system is processed by a first set of layers, which produces an output that is then processed by a subsequent set of layers, and so forth until the output of the machine learning model is produced by the final layer in the series.

[0029] An “embedding” can refer to a representation of data, usually within an “embedding space”, a theoretical region in which embeddings can be compared via vector operations. For example, an embedding can comprise a vector representation of an image, which can be used to evaluate the similarity of that image to other images, or e.g., determine whether that image contains or depicts a particular subject (e.g., a cat). Embeddings can be used within the field of machine learning, thereby enabling machine learning models to perform certain tasks, particularly tasks that are difficult or subjective. In some cases, an embedding can comprise a lower-dimensional representation of a corresponding element of data, enabling more efficient processing due to the reduced logical size of the embedding relative to the logical size of the original element of data.

[0030] An “encoder” can refer to something (e.g., a physical device, software element, component of a machine learning model, etc.) that produces output data representative of some input, usually in the form of a “code” or signal. An encoder can be used to convert data from one format to another, which may facilitate data processing. For example, a nominal10 KILPATRICK TOWNSEND 792723011data encoder can be used to convert nominal data (such as the name of a city) into a numeric code, facilitating such data to be processed using numerical methods. As another example, a physical rotary encoder can be used to convert the motion or position of a shaft or axle into electrical signals, facilitating computer-based odometry. A code (or other output of an encoder) can be used as an embedding. Some encoders can be implemented via applications of machine learning, e.g., via a “transformer” machine learning model or via another model that uses the principle of machine learning “attention.”

[0031] An “interaction” can refer to a reciprocal action or influence or an event or phenomenon corresponding to such an action or influence. As an example, an individual posting a comment on a social media website could be considered an interaction between that individual (or e.g., a personal computer system associated with that individual) and the social media website (or e.g., a server computer hosting that social media website). As another example, the collision of a fast-moving cold air front and a slow-moving warm air front (e.g., an “occluded front”) could be considered an interaction between the two air fronts. As another example, a credit card transaction between a cardholder and a business could be considered an interaction. In some cases, measurable properties of interactions can be analyzed or evaluated (e.g., using machine learning) to quantify or qualify those interactions.

[0032] An “access device” may be any suitable device that provides access to a remote system. An access device may also be used for communicating with a resource provider computer, an authorizing computer, or any other suitable system. An access device may generally be located in any suitable location, such as at the location of a resource provider or merchant. An access device may be in any suitable form. Some examples of access devices include POS or point of sale devices (e.g., POS terminals), cellular phones, PDAs, personal computers (PCs), table PCs, hand-held specialized readers, set-top boxes, electronic cash registers (ECRs), automated teller machines (ATMs), virtual cash registers (VCRs), kiosks, security systems, access systems, terminals, and the like. An access device may use any suitable contact or contactless mode of operation to send or receive data from, or associated with, a user communication device. In some embodiments, where an access device may comprise a POS terminal, any suitable POS terminal may be used and may include a reader, a processor, and a computer-readable medium. A reader may include any suitable contact or contactless mode of operation. For example, exemplary card readers can include radio frequency (RF) antennas, optical scanners, bar code readers, or magnetic stripe readers to interact with a payment device and / or mobile device. Other examples of access devices11 KILPATRICK TOWNSEND 792723011include devices (e.g., locks, gates, access control boxes, etc.,) that control physical access to locations (e.g., venues, transit stations, homes, offices, buildings, etc.,) as well as software devices that control access to data or information. In embodiments, the access device may be configured to charge or recharge a user device via contact or contactless modes of operation.

[0033] A “resource” can generally refer to something that can be acquired or provided to an entity, e.g., by a “resource provider.” Examples of resources include material resources, such as iron, monetary resources, such as dollars, or other resources such as cleaning supplies, clothing, food, etc. Resources may also refer to services, such as cleaning services. Access to something may also qualify as a resource, e.g., access to a secure building. Resources can also include data stored on a computer, such as video files, audio files, databases, data tables, etc. A resource provider can operate a “resource provider computer”, e.g., an entity associated with a digital streaming service could operate a resource provider computer (e.g., a server computer).

[0034] The term “access request” generally refers to a request to access a resource, e.g., provided by a resource provider. An access request and a transaction can be used interchangeably. The access request may be received from a requesting computer, a user device, or a resource computer, for example. The access request may include authorization information, such as a username, account number, or password. The access request may also include access request parameters, such as an access request identifier, a resource identifier, a timestamp, a date, a device or computer identifier, a geo-location, or any other suitable information. The access requests can be received in real time. For example, a user of a user device (e.g., a client computer such as a laptop, smartphone, or tablet) can use that user device to make an access request to a server computer, e.g., to access a service provided by that server computer (e.g., an image hosting service, a video streaming service, an online banking service, etc.), e.g., by transmitting the access request to the server computer over the Internet. In some cases, an access request may be made via an “access device”, which may comprise any device used to access something (e.g., a network or a computer system). This could include a user device as described above, but could also include, e.g., a point-of-sale terminal used to gain access to a payment processing network. An access device may include a means by which it can interface with other devices. For example, a point-of-sale terminal access device may include a chip card reader that can interface with a smartcard operated by a user. An access request may include a “credential” or “identifier”, which may comprise data used to demonstrate or prove that an entity is authorized to access a resource, or which12 KILPATRICK TOWNSEND 792723011may comprise data used to identify the entity, such that an access device (or another device or computer system, e.g., an “authentication computer” or “authorization computer”) can authenticate the entity (e.g., by verifying its identity) and verify that the entity is authorized to access the resource.

[0035] The term “access request outcome” may include any determination of whether to grant access to the resource. The access request outcomes may include “accept,” “reject,” or “review.” In some embodiments, an access request outcome for an access request may be “reject” if any of the access rules have an access rule outcome of “reject.” In some embodiments, an access request outcome for an access request may be “accept” if any of the access rules have an access rule outcome of “accept,” regardless of any outcome being “reject.” An access request outcome of “accept” may cause the access request to be granted. An access request outcome of “reject” may cause the access request to be denied. The “review” outcome may initiate a review process for the access request. In various embodiments, other outcomes or other meanings of these outcomes may be used. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] FIG.1 shows an exemplary sequence of events and indicates general relationships between those events.

[0037] FIG.2 shows a block diagram of an exemplary multilayer machine learning model including an encoder according to some embodiments.

[0038] FIG.3 shows a model of a scaled dot-product attention layer and a multi-head attention model.

[0039] FIG.4 shows some exemplary machine learning tasks that can be performed using embedding vectors.

[0040] FIG.5 shows an exemplary client-server system that can be used to perform machine learning methods according to some embodiments.

[0041] FIG.6 shows an exemplary machine learning model architecture according to some embodiments.

[0042] FIG.7 shows a flowchart depicting an exemplary machine learning inference method according to some embodiments.13 KILPATRICK TOWNSEND 792723011

[0043] FIG.8 shows an exemplary complete sequence of events separated into subsequences based on window lengths.

[0044] FIG.9 shows an exemplary sequence module according to some embodiments.

[0045] FIG.10 shows an exemplary window module according to some embodiments.

[0046] FIG.11 shows an exemplary contrastive learning process according to some embodiments.

[0047] FIG.12 shows a flowchart depicting an exemplary machine learning training method according to some embodiments.

[0048] FIG.13 shows a resource security system 1300 for authorizing access to resources in accordance with some embodiments.

[0049] FIG.14 shows an exemplary computer system according to some embodiments. DETAILED DESCRIPTION

[0050] As described above in the Background and Summary, embodiments of the present disclosure relate to machine learning methods that can be used to process data sequences, e.g., as part of some larger task. Such data sequences can comprise time series data comprising “velocity features”, e.g., features of some phenomena, event, object, person, etc., that are measured over time, such as a person’s heartrate, the speed of a vehicle, etc. As an example, a machine learning model according to embodiments can be used to generate an output embedding corresponding to a data sequences, which could then be input into a classifier model, thereby generating a classification corresponding to the data sequence, e.g., a classification of a phenomena, event, object, etc., related to the data sequence. Such a classification could then be used for some purpose, e.g., a machine learning model according to embodiments can be used to generate an output embedding corresponding to a data sequence of access requests to a resource, which could then be input to a classifier model that could, e.g., classify a particular access request as legitimate or invalid based on the output embedding and automatically approve or deny the access request based on the classification.

[0051] In some ways, embodiments of the present disclosure address problems associated with the use of measures of central tendency or other statistical techniques commonly used in the evaluation of sequences of data. Features (e.g., velocity features) produced through statistical techniques may fail to capture (or inadvertently erase) useful information from the14 KILPATRICK TOWNSEND 792723011data used to produce those features. Such features may be sensitive to variations in the underlying data as well as noise, outliers, and anomalous data values. For example, an adult patient’s average heartrate over the course of an hour may be 120 BPM, which may be considered unusually high for a resting heartrate. However, if the patient performed high intensity exercise during that one-hour period, e.g., for fifteen minutes, their increased heartrate during that exercise period may have pulled up their average considerably. Their actual resting heartrate could comprise a more reasonable value, such as 90 BPM. The average heartrate feature of 120 BPM however does not capture the variations in heartrate during that period due to exercise. Relatedly, features derived from measures of central tendency (e.g., averages) over time often implicitly assume a linear relationship over time. Further, features may fail to capture contextual (e.g., periodic) relationships between data elements, observations, events, etc. For example, an individual may have a higher resting heartrate during their waking hours, and a lower resting heartrate during their sleeping hours. However, their average heartrate over the course of the day does not capture this variation.

[0052] This issue is generally illustrated by FIG.1, which shows a data sequence 102 comprising data elements 104-114. As illustrated by FIG.1, there are a large number of relationships 118 between data elements 104-114. For ^^ data elements, there may be^^^^^ െ 1^⁄ 2 between those data elements, each of which may indicate some characteristic ofa phenomena, event, person, etc., associated with data sequence 102, which could be useful to e.g., accurately classify that phenomenon, event, person etc., or perform some other task.

[0053] In many convention data sequence analysis techniques, data elements in data sequence 102 could be averaged to produce an average data element 116, which could then be used to perform some task or further analysis, e.g., as the input to a machine learning model used to classify data sequence 102 based on the average data element 116. However, the average data element 116 does not contain or communicate any information related to the relationships 118 between data elements 104-114. Further, measures such as the average may be sensitive to outliers or noisy data, or may “smooth out” large variations in data (e.g., a large increase in electricity usage during a heatwave or a winter storm). Consequently, any sort of analysis or machine learning task performed using average data element 116 (e.g., using linear regression, decision trees, rule-based techniques, comparison to thresholds, etc.) may be less accurate and lead to various undesirable effects, e.g., failure to accurately forecast weather based on a data sequence 102 of historical weather data, failure to identify invalid access requests based on a data sequence 102 of historical access requests, etc.15 KILPATRICK TOWNSEND 792723011

[0054] In embodiments however, a hierarchical machine learning model can be used to generate embeddings that capture patterns or features in the data corresponding to multiple scales. A data sequence can be broken up into sets of data subsequences of various window sizes. A machine learning model according to embodiments can comprise a sequence encoder, which can generally encode the data sequence, thereby generating a set of sequence partial embeddings that can capture patterns, interactions, or relationships between the data elements in the data sequence. The machine learning model can further comprise one or more window encoders that can generate window partial embeddings that can capture patterns, interactions, or relationships between subsequences of data elements. For example, a “long” window encoder can generate a set of window partial embeddings that captures patterns, interactions, or relationships between long data subsequences, while a “short” window encoder can generate a set of window partial embeddings that captures patterns, interactions, or relationships between short data subsequences, etc. Sets of window partial embeddings can be combined to generate an output embedding, which can generally represent information (including patterns or relationships between data elements and subsequences of data elements) from the data sequence at multiple scales. Such an output embedding could be used for some task, e.g., as part of a classifier used for an access review system.

[0055] Because the output embedding is generally a better representation of the data sequence than e.g., an average data element generated from the data sequence or an embedding generated based on an average data element, embodiments of the present disclosure can improve the accuracy and effectiveness of systems that use such embeddings, e.g., a classifier model used by an access review system. Further, machine learning models according to embodiments can learn complex temporal features automatically from input data, and may better generalize to unseen data, further improving the accuracy and effectiveness of systems that use such embeddings. I. TIME SERIES, ATTENTION, AND EMBEDDINGS

[0056] As summarized above, some embodiments of the present disclosure relate to machine learning models, methods and systems for training such machine learning models using novel contrastive loss techniques, encoders, time series data analysis, etc. While it is assumed that a potential practitioner of methods according to embodiments has some general knowledge of the field of machine learning and time series data analysis techniques (e.g.,16 KILPATRICK TOWNSEND 792723011from the field of signal processing), a brief summary of these concepts is provided herein in order to better orient the reader. A. Time Series

[0057] “Time series data” can generally comprise a series of observations or time series data elements related to some phenomena, process, event, system, or object, etc. As an example, periodic outdoor air temperature measurements (e.g., recorded by a weather station), could comprise a time series. Various useful analytics and processes based on those analytics can be performed using time series data or other sequences of data. For example, a weather forecasting service may use historical time series weather data to forecast the weather, e.g., the outdoor temperature, chance of rain, wind speed, etc., over the next 10 days. As another example, a power utility could use historical power demand data to determine whether a particular rapid increase in power demand is anomalous or unusual or is within usual variations in demand over a particular period of time. A time series can comprise a chronologically ordered sequence of data values or “observations,” which can be displayed visually as a line or “waveform” on a graph, e.g., similar to graphs 610 in FIG.6. Typically, such graphs or waveforms are organized chronologically from left to right. For the purpose of simplicity, most depictions of time series in the figures are one-dimensional, that is, a single data value is associated with each point in time. However, in reality, a single observation may comprise multiple time series data values. As an example, a time series observation corresponding to patient health data may comprise a heart rate data value, a pulse oximetry data value, a blood pressure data value, etc. Likewise, a time series observation corresponding to a credit card transaction could comprise a transaction amount, a location associated with that transaction (e.g., a city or country where that transaction took place), an industry indicator (e.g., grocery store), etc. Data in a time series (or more generally, any data sequence) may be organized by timestamps (e.g., indicating the time and date at which a particular observation took place) and / or by indices (indicated, e.g., the relative location of an observation within a data sequence via a numerical value).

[0058] Various statistical metrics and methods are used to analyze sequences of data, such as the average or running average. For example, in order to determine if the current outdoor air temperature is anomalous, a weather forecasting station could compare the current outdoor air temperature to the average outdoor air temperature over the last ten years. If the current outdoor air temperature is larger or smaller than the average outdoor air temperature,17 KILPATRICK TOWNSEND 792723011the weather forecasting station could determine that it is either unusually hot or unusually cold outside.

[0059] While statistical methods such as averages are often useful in various data analysis tasks, they also have some weaknesses. For example, an average taken over a large enough timeframe will “erase” variations (e.g., periodic variations) within that timeframe. For a region with extremely hot summers and extremely cold winters, but long mild falls and springs, a yearly (or e.g., ten year) average outdoor air temperature may reflect the mild temperature of fall and spring. When this average is compared to the outdoor air temperature in summer or winter, it may suggest a particularly hot summer or a particularly cold winter, when in reality it may be a typical summer or winter. Hence, more sophisticated methods (e.g., comparing the outdoor air temperature to the average summer temperature or the average winter temperature) may need to be used instead in order to perform useful analysis.

[0060] Further, statistical metrics such as the average may “erase” information about the interrelationships between individual observations or data values in a data sequence, preventing such information from being used for analysis tasks, such as forecasting or anomaly detection. The current outdoor air temperature may be highly predictive of the outdoor air temperature an hour into the future, but so too might the outdoor air temperature an hour ago, two hours ago, or e.g., 24 hours ago. Even further however, the relationship between the outdoor air temperature two hours ago and three hours ago (e.g., the change in the outdoor air temperature over that time period), or e.g., two hours ago and 26 hours ago may also be predictive of the outdoor air temperature in the future. For even a small number of data elements, there may be a large number of relationships between those data elements (e.g., for only 15 observations there may be over 100 paired relationships between those data elements), and when multiple observations or data elements are combined into a single metric (e.g., an average of some number of observations), that single metric fails to capture those relationships, or may implicitly assume or assign linear relationships between data elements. As such, the average or other statistical measurements may not accurately reflect the relationships between observations or the nature of some measured phenomena or sequence of events.18 KILPATRICK TOWNSEND 792723011B. Machine Learning

[0061] At a high level, a machine learning model generally produces output data responsive to received input data. Such input data may comprise “features”, e.g., of some object, event, or phenomena being analyzed by a machine learning model, which may be quantified or qualified by the output data. For example, for a machine learning model used to evaluate comments posted on a social media website, the length of a comment (measured, e.g., in the number of words or the number of characters) could comprise a feature. The output of such a machine learning model could comprise scores used to describe or define such comments, e.g., a score indicating how “malicious” or “benign” a particular comment is. Such scores could be used as part of an automatic content moderation system, e.g., a system that automatically deletes malicious comments based on their scores.

[0062] Machine learning models are often defined by sets of parameters, which generally control how the machine learning model produces output data responsive to received input data. As an example, a support vector machine (SVM) is a type of machine learning model that divides data points using a hyperplane. Input data positioned on one “side” of the hyperplane is classified as one class (e.g., normal) while input data on the other side of the hyperplane is classified as another class (e.g., anomalous). The parameters of the support vector machine can comprise the coefficients used to define the hyperplane. Changing these parameters changes the shape of the hyperplane, and thus changes which data points the SVM classifies as normal or anomalous.

[0063] In general terms, the process of training a machine learning model can involve determining the set of parameters that achieve the “best” performance, usually based on a loss or error function. A loss function relates the expected or ideal performance of the machine learning model to its actual performance on a training dataset. In “supervised learning”, the training data set may be labeled, and the labels may indicate expected or ideal outputs of the machine learning model corresponding to data in the training data set. For example, for a machine learning model used to classify images, training data may comprise images and labels indicating what is depicted in those images. In “self-supervised learning”, the training data set may not be labeled. The loss function typically decreases in value as the model’s performance improves. As such, training a machine learning model often involves determining the set of parameters that minimize a loss function corresponding to that model. Sometimes a random parameter estimate is generated as an initial parameter “guess,” and19 KILPATRICK TOWNSEND 792723011then a process such as gradient descent is used to iteratively refine the parameter estimate, eventually resulting in a final set of parameters associated with the machine learning model.

[0064] This iterative refinement process can be performed in a series of training “rounds”, “epochs”, or other appropriate divisions. In each round, a machine learning model’s performance can be evaluated using the loss function, and the parameters can be updated based on this evaluation, e.g., with the goal of reducing the result over time. As an example, the gradient of the loss function can be determined in parameter space and can be used to reduce the value of the loss function in successive training rounds. Such a gradient corresponds to a change in model parameters that achieves the greatest immediate reduction in the loss function. By changing the model parameters based on the gradient, the loss function can be reduced during each successive training round. This process can be repeated until a terminating condition has been met. In embodiments of the present disclosure, one type of terminating condition is a defined number of training rounds. This terminating condition can be met if the number of training rounds performed (e.g., by a computer system training the machine learning model) equals or exceeds the defined number of training rounds, at which point the iterative training process has been completed. Another type of terminating condition in embodiments is a convergence condition. This terminating condition can be met if the machine learning model parameters “converge.” In general terms, convergence is achieved when the value of the loss function, and / or the values of the model parameters change in increasingly small amounts with each successive training round. For example, a convergence condition can be achieved if the value of the loss function decreases by less than 0.1% in two successive training rounds.

[0065] It is possible to train multiple machine learning models or multiple trainable components of a machine learning model simultaneously, e.g., by updating parameter sets corresponding to each of those machine learning models simultaneously, e.g., using a single training dataset and / or a combined loss function. Components of a machine learning model (trainable or otherwise) may be organized into "layers” or “modules”, usually based on their relative “proximity” to either the model’s inputs (e.g., features) or the model’s outputs (e.g., predictions based on those features). For example, an initial layer may operate on feature inputs, a final layer may produce the model’s output, and an intermediate layer may operate on (as well as generate) intermediate data used by the model to produce the model output.20 KILPATRICK TOWNSEND 792723011

[0066] Some multilayer machine learning models (including some machine learning models according to embodiments) may include “encoders.” Such components can encode a feature input (or intermediate data) into an “encoding”, “embedding”, or “latent space embedding” corresponding to a “latent space”. Such an embedding can generally represent the information contained in the encoder’s input in a different form (e.g., as a point or vector in a latent space), which may facilitate further processing, e.g., by subsequent layers of a machine learning model, ultimately resulting in more accurate outputs (e.g., more accurate predictions or forecasts). FIG.2 shows a diagram of a general multilayer machine learning model architecture (corresponding to exemplary machine learning model 214), comprising an encoder 204, final layer(s) 208 (which can comprise a “classification model”), intermediate model layers 212, and additional intermediate model layers 216.

[0067] The model inputs 202 (or “features”) to machine learning model 214 can be processed by any number of intermediate model layers 212 prior to being processed by encoder 204. Each of these intermediate model layers 212 may produce its own output, which may comprise the input to a successive intermediate model layer 212 (or encoder 204). Such intermediate model layers 212 (and likewise, additional intermediate model layers 216) may depend on the nature of machine learning model 214, and can comprise any variety of layer that could be included in a machine learning model, such as pooling layers, subsampling layers, normalization layers, other neural network layers (e.g., fully connected layers), other encoders, transformers, etc. A convolutional neural network, for example, can comprise an alternating series of convolution layers and subsampling layers (each of which could be considered intermediate model layers) prior to a final fully connected layer. In some machine learning models, the encoder 204 may comprise either the first layer in the machine learning model 214 or the second to last layer in the machine learning model 214, e.g., such that there are no intermediate model layers 212 preceding the encoder 204, or no additional intermediate model layers 216 following the encoder 204. In other machine learning models, there may be any number of intermediate model layers 212 preceding the encoder 204 and any number of additional intermediate model layers 216 following the encoder.

[0068] As stated above, an encoder (such as encoder 204) can produce embeddings or encodings (e.g., embeddings 206, which may comprise “embedding vectors”) based on its inputs. Such embeddings 206 can generally comprise alternative representations of the encoder’s 204 inputs, which may facilitate processing by subsequent components of the machine learning model 214 (e.g., additional intermediate model layers 216, final layer(s)21 KILPATRICK TOWNSEND 792723011208, etc.). For example, embeddings 206 may have a smaller logical size (e.g., 10 times smaller, 100 times smaller, etc.) than the data used to produce such embeddings (e.g., model inputs 202), and may thereby decrease the computational complexity of processing embeddings 206 (e.g., via intermediate model layers 212 or final layer(s) 208), thereby improving the speed at which the machine learning model 214 produces model outputs 210 (e.g., classifications). As another example, some machine learning models or model layers may only operate on numerical data, and encoder 204 can be used to embed nominal data such as the name of a city (e.g., “San Francisco”) into a numerical embedding such as “941”, enabling such data to be processed by such model layers. Encoding data in this manner can be accomplished, e.g., using a mapping table or lookup table, a predefined data structure that maps the names of cities (encoder inputs) to their corresponding numerical values (embeddings) such as zip codes (or e.g., the first three digits of zip codes) from a city look up table.

[0069] However, encoders can also use more (or less) sophisticated methods for producing various types of embeddings, including the usage of machine learning based methods, such as “transformer” models or other models based on the principle of “attention” (or any other applicable machine learning model). As an example, convolution layers in a convolutional neural network may be thought of as a type of encoder that uses a convolution kernel to encode an input (e.g., an image) into a (typically smaller) embedding. Regardless of the particular methods or models used to implement encoder 204, it should be understood that in some embodiments an encoder can comprise a trainable component of a machine learning model (or may comprise a standalone machine learning model), and an encoder may be defined by an “encoder parameter set” that can be iteratively adjusted during training, e.g., based on a loss function or value. As described in more detail below, in some embodiments such a loss function may include a “contrastive loss”.

[0070] After being processed by any number of additional intermediate model layers 216 (including zero), the final layer(s) 208 (which may comprise a “classification model” in some embodiments) can produce a model output 210 based on embeddings 206 (or based on any intermediate data generated from embeddings 206). Such an output could comprise, e.g., a binary prediction, which can comprise a prediction as to whether a set of model inputs 202 correspond to a particular class or do not correspond to that class. For example, a binary prediction could indicate whether an audio recording contains an endangered bird call or does not contain an endangered bird call (e.g., as part of a rainforest conservation and monitoring22 KILPATRICK TOWNSEND 792723011program). As another example, a model output 210 could comprise a non-binary prediction (e.g., a continuous prediction), such as a percentage indicating the probability that a particular set of model inputs 202 correspond to an invalid (or legitimate) access request (e.g., a payment transaction such as a credit card or debit transaction).

[0071] In some embodiments, the final layer(s) 208 can comprise a fully connected layer (also known as a “dense layer”), a layer in a neural network in which every “neuron” in one layer is connected to each neuron in the next layer. The final layer(s) 208 may be defined by their own parameter set and, in some embodiments, both the encoder 204 and a prediction model (comprising final layer(s) 208) can be trained simultaneously based on a combined loss function, e.g., by updating a parameter set corresponding to the prediction model and the encoder parameter set in each training round of an iterative training process. C. Attention

[0072] As described herein, a machine learning model according to embodiments can use mechanisms such as self-attention in order to generate embeddings representative of data sequences. Because attention is a generally well-understood concept in the field of machine learning, it is assumed that a potential practitioner of embodiments of the present disclosure is familiar with the concept of attention. However, a summary of attention is provided below in order to facilitate a better understanding of embodiments of the present disclosure. More information about attention can be found in the literature, e.g., the article “Attention is all you need” (https: / / arxiv.org / abs / 1706.03762).

[0073] Generally, attention is a “set to set” (or “sequence to sequence”) operation. That is, for a set of e.g., ten input vectors, the output of an attention operation may comprise a set of ten output vectors. Two notable types of attention, self-attention and cross-attention are described below. Generally, in self-attention, the attention can be computed between a single input set and itself. In cross-attention there can be multiple input sets, and the attention can be computed between these input sets to produce the output set.

[0074] Attention is often used in the context of language models, in which each “token” (e.g., data representative of a word or part of a word, such as the suffix “-ing”) in a sequence (e.g., a sentence) can “interact” with each other token in the sequence for the purpose of performing the task associated with the language model. This is in contrast to previous types of machine learning models, such as recurrent neural networks, which have a temporal23 KILPATRICK TOWNSEND 792723011window and incremental interaction. Using attention, language models can learn which tokens interact with which other tokens, to which degree, and how, enabling such language models to learn the relationships between words in sentences. Language models that use attention often achieve better performance than language models that use recurrent neural networks, as they can model more complex relationships between input tokens in sentences.

[0075] In more detail, in self-attention, each output in the output set can comprise a weighted average of the inputs in the input set, e.g., for inputs in input set ^^^and outputs inoutput set ^^^, ^^^ ൌ ∑^ ^^^,^^^^ . However, unlike other machine learning models based onweighted weights are parameters of the system, the weights ^^^,^in self-attention are often a function of the inputs in the input set, often based on similarity, such that the weight corresponding to a pair of similar inputs may be greater than the weight corresponding to a pair of dissimilar inputs. Often, for sets of vector inputs, theweights are derived from the dot product between vector inputs, e.g., ^^^,^ ൌ ^^^ ∙ ^^^, and insome cases, functions such as the softmax function can be used to map the weights to adefined range, such as ^0, 1^, i.e.:^^^^∙^^ೕ^^^,^ൌ∑ ^^ ∙^^ೕ ^ ^ ೕattention is similar to self-attention, except the attention is not computed between members of a single set, but rather between members of multiple (often two) sets. In such cases, the cross-attention weights may be based off the dot products of different sets of inputs, rather than a single set.

[0077] Attention operations are often framed in the context of “queries”, “keys”, and “values” (which may also be referred to as “attention queries”, “attention keys”, and “attention values”). For example, in a case of self-attention, queries ^^^, keys ^^^, and values ^^^can be derived from inputs ^^^via a query matrix ^^^, key matrix ^^^, and value matrix ^^௩, e.g.: ^^^ ൌ ^^^^^^ ^^^ ൌ ^^^^^^ ^^^ ൌ ^^௩^^^^^^ ൌ ^ softmax൫^^^ ∙ ^^^൯^^^^

[0078] These weight matrices can be controlled in order to modify any input vectors based on the machine learning task being performed. In addition, the query, key, value framing of24 KILPATRICK TOWNSEND 792723011attention can be useful for implementing cross-attention, e.g., by deriving the queries, keys, and values from different input sources. Attention and the query, key, value framing may be better understood with reference to FIG.3, which shows an example of a scaled dot-product attention layer 302 and a multi-head attention model 304. FIG.3 is adapted from figures from “Attention is all you need” (https: / / arxiv.org / abs / 1706.03762).

[0079] The scaled dot-product attention layer 302 can compare each input token (query 306) with every token (key 308) in a sequence (for self-attention) or every token in another sequence (for cross-attention) by using a dot product 312. The result can be scaled (314), optionally masked (316), and fed into a softmax function (318) to create weights for each query with respect to each key, which may sum to one as a result of the softmax function. These weights can be applied to the value (310) of each key (matrix multiplication 320), which can then be summed up, creating an output value for each query (^^^, as described above).

[0080] FIG.3 also shows an example of a multi-head attention model 304. In multi-head attention, multiple scaled dot-product attention layers 334 (corresponding to the “heads”) can be used in parallel and the results can be concatenated (336) before being applied to a linear layer 338. Each head can process the same query 322, key 324, and value 326, but can transform these data with its own linear layers (e.g., linear layers 328-332). In general, in single head attention, inputs in the input set can influence outputs by different amounts but cannot influence those outputs in different ways. By using multiple self-attention heads, each with their own linear layers and / or weight matrices, attention-based machine learning models can have greater discrimination and accuracy.

[0081] Attention can be useful in a variety of machine learning tasks, particularly when the output of the attention operation can be mapped to a corresponding task output. For example, an AI customer service chatbot may comprise a sub-model used to determine the emotional sentiment (e.g., happiness, anger, etc.) behind a chat message received from a customer, in order to determine an appropriate response. If the output of a self-attention mechanism can be mapped to a “sentiment values” (e.g., by averaging and down-projecting self-attention outputs), then self-attention may be useful for this machine learning task.

[0082] More generally, attention can be used to reduce the problem of learning to perform a particular task to the problem of learning to generate “embeddings” that, when attention is applied to these embeddings, produce outputs that can be mapped to the desired outputs of25 KILPATRICK TOWNSEND 792723011the particular task. In the context of evaluating the sentiment of customer chat messages, training data could comprise pairs of messages and sentiment scores (e.g., “I am very angry” and “0”), and the machine learning sub-model could learn to generate vector embeddings from the messages (e.g., corresponding to individual words or parts of words) such that, when self-attention is applied to those embeddings, result in outputs that can be mapped to the sentiment score (e.g., by averaging and down-projecting). Such embeddings can be generated using a linear layer (e.g., a neural network). The loss or error can be related to the difference between the actual and expected sentiment scores, which can be used to update the parameters of the linear layer used to generate the embeddings, e.g., such that it produces embeddings that result in attention outputs corresponding to accurate sentence sentiment. In this way, an attention-based machine learning model can be trained to perform tasks such as sentiment classification.

[0083] As described in more detail with reference to FIG.11 further below, in some embodiments, attention-based methods can be used to extract the useful information from inputs, which can be collected or otherwise represented by generated embeddings. In this way, attention-based methods can be used to generally discard or ignore random or irrelevant data contained in inputs. Contrastive loss-based training methods according to embodiments can be used to train attention-based machine learning models in this manner. Interestingly, such models can generally be trained without considering the “end use” of the embeddings generated using such models. For example, for a machine learning application in which machine learning is used to determine whether attempts to access a secure network or other computing resource are valid or invalid, an attention-based machine learning model could be trained to generate embeddings corresponding to access attempt training data without using training labels indicating whether access attempt training data corresponded to a valid or invalid attempt. Then later, a second machine learning model (e.g., a neural network) could be trained to determine whether attempts to access the secure network were valid or invalid based on the generated embeddings and the training labels. D. Anomaly Detection Using Embeddings

[0084] As described above, embeddings can be used in various tasks in place of the data used to generate such embeddings. FIG.4 generally summarizes some anomaly detection tasks that can be performed using embeddings generated using machine learning models according to embodiments. FIG.4 shows representations of embeddings in an embedding26 KILPATRICK TOWNSEND 792723011space 418, which can include embeddings corresponding to normal data 404 and embeddings corresponding to anomalous data 406. In order to perform anomaly detection in an anomaly detection task 402, clusters of embedding vectors in the embedding space can be detected and can be correlated with their respective data type. If a new embedding is generated based on a new data sequence, the new embedding can be analyzed in the context of these clusters. If the new embedding is near embeddings corresponding to normal data 404, then it could be classified as a “normal embedding” corresponding to a normal data sequence. By contrast, if the new embedding is near embeddings corresponding to anomalous data 406, then it could be classified as an “anomalous embedding” corresponding to an anomalous data sequence.

[0085] FIG.4 also shows a fine-tuned anomaly detection application 408. An embedding 410 can be input into a classifier 412 that was trained to produce classifications based on embeddings (e.g., using a labeled training data set and supervised learning). Classifier 412 can comprise any type of classification model (e.g., a decision tree, an XGBoost model, a DNN, etc.) and can generate a classification 414, which could indicate e.g., whether the embedding 410 corresponds to normal input data or anomalous input data. Such a classification could comprise a “score” or other numerical value that can be mapped to classifications. For example, in the context of access review classification, a score of “0.95” could indicate a 95% chance that an embedding 410 corresponds to a legitimate access request.

[0086] There are various useful applications for anomaly detection tasks. For example, an access review system could review access requests to a protected resource and approve normal requests and deny anomalous requests. In the context of e.g., cyber security, an access review system could verify that an attempt to login to a network was a legitimate attempt, e.g., by generating an embedding corresponding to various login features (e.g., time of login, location of login, frequency of login attempts) and determining whether that embedding corresponds to a legitimate login attempt or an invalid login attempt (e.g., using a classifier model such as classifier 412). Such an embedding could be generated using methods according to embodiments. In the context of e.g., credit card transactions, an access review system could review access requests to a payment account or a payment processing network, e.g., by generating an embedding corresponding to various credit card transaction features (e.g., the time of purchase, location of purchase, frequency of recent purchases, etc.), determine whether that embedding corresponds to a legitimate credit card transaction or an invalid credit card transaction (e.g., using clustering techniques as depicted in anomaly27 KILPATRICK TOWNSEND 792723011detection task 402), and automatically approve or deny the credit card transaction based on this determination.

[0087] It should be understood that the examples provided above are intended to be non- limiting, and that an access review system can be used in various other tasks in the fields of fraud detection, system monitoring, cyber security, network monitoring, etc. Moreover, it should be understood that methods according to embodiments can be used in various machine learning tasks, including machine learning tasks that are generally not considered forms of “anomaly detection.” For example, a data sequence could correspond to a user of a video-on- demand streaming service, and could comprise, e.g., data corresponding to movies and TV shows viewed by that user over a period of time. Methods according to embodiments could be used to generate an embedding corresponding to that data sequences, which could then be applied to a recommendation model that could be used to generate recommendations of other movies or TV shows for that user. As another example, a data sequence could correspond to power demand data over time. Methods according to embodiments could be used to generate an embedding corresponding to that data sequence, which could then be applied to a forecast model that could be used to estimate power demand for an individual or region (e.g., a city) in the future. II. CLIENT-SERVER MODEL

[0088] Having described some useful concepts related to embodiments of the present disclosure above, it may now be helpful to describe some systems according to embodiments of the present disclosure, including computer systems that can implement methods according to embodiments.

[0089] FIG.5 shows a computer system 502 that can be used to perform methods according to embodiments. As described in more detail below with reference to FIG.14, a computer system such as computer system 502 can comprise one or more processors (not pictured) and a non-transitory computer readable medium (e.g., a hard drive) coupled to the one or more processors (also not pictured). The non-transitory computer readable medium can comprise code or instructions, executable by the processor for performing methods according to embodiments described herein.

[0090] Computer system 502 can receive data sequences (comprising sequences of data elements, comprising features or other data) and can use a machine learning model 504 to28 KILPATRICK TOWNSEND 792723011generate output embeddings corresponding to those data sequences (as summarized above and described in more detail further below). If necessary, various other processes can be performed based on those embeddings, as described above. For example, in the context of credit card fraud detection, computer system 502 can receive a data sequence corresponding to historical access requests and use machine learning model 504 to produce an output embedding that generally captures patterns or access request behaviors at various scales. Afterwards, computer system 502 could use a classification model 516 to classify the output embedding, thereby producing a fraud score indicating the probability that the access request is an invalid access request. As another example, in the context of an automatic content moderation system for a social networking website, computer system 502 can receive a data sequence corresponding to posts on the social networking website and use machine learning model 504 to produce an output embedding that generally captures “posting patterns” of a user or group of users. Afterwards, computer system 502 could use classification model 516 to classify the output embedding, thereby generating a content score indicating whether a given post (or a user corresponding to a sequence of posts corresponding to the data sequence) is objectionable or not (e.g., whether the post violates community guidelines).

[0091] As another example, in the context of a power plant demand forecasting system, computer system 502 can receive a data sequence corresponding to various historical conditions (e.g., outdoor temperature, previous demand for power) and use machine learning model 504 to produce an output embedding that could be used with classification model 516 to estimate power demand for some period of time in the future. As another example, in the context of a film recommendation service (which may comprise, e.g., part of a video streaming service), computer system 502 can receive a data sequence corresponding to films watched or highly rated by users and use machine learning model 504 to produce an output embedding. The computer system could then use classification model 516 to produce scores corresponding to other films in a film catalog, indicating the likelihood that a particular user would also watch or enjoy those films based on the input data sequence.

[0092] As yet another example, in the context of a machine learning diagnostic aid, computer system 502 can receive features corresponding to symptoms (or vital statistics) of a patient and use machine learning model 504 (and classification model 516) to produce predictions or classifications indicating the estimated likelihood that the patient is afflicted by particular diseases. As another example, in the context of a cybersecurity system, computer system 502 can receive features corresponding to network communications (or code executed29 KILPATRICK TOWNSEND 792723011by other computer systems) and use machine learning model 504 (and classification model 516) to produce a prediction or classification indicating the likelihood that there is some malicious or unauthorized access to those other computer systems.

[0093] In some embodiments, machine learning model 504 can comprise a machine learning model 504, which may comprise an “encoder model”. As described above with reference to FIG.2 and in more detail below with reference to FIG.6, in general terms, an encoder can be used to encode data sequences, thereby generating embeddings. Such embeddings can be used by e.g., classification model 516 for some task. For example, embeddings generated by machine learning model 504 can be used as an input to a classification model 516 in order to produce predictions that can be output by computer system 502. Various types of machine learning models can be used in embodiments of the present disclosure and non-limiting examples are provided herein. As an example, classification model 516 could comprise a gradient-boosted decision tree model (e.g., an XGBoost model), a linear model, or a tree model. In some embodiments, machine learning model 504 can be defined by a set of encoder parameters that can be determined, set, or otherwise established during a training process, thereby training machine learning model 504. Likewise, the classification model 516 can be defined by a set of classification model parameters that can be determined, set, or otherwise established during training. These parameter sets can be updated based on various types of loss functions or loss values, including those determined using cross-entropy or based on a mean squared error between prediction model outputs and training labels corresponding to a training dataset (comprising, e.g., training data sequences 514). In some embodiments, the classification model 516 can comprise a neural network (e.g., a multilayer perceptron) and the classification model parameter set can comprise a plurality of weights associated with the neural network. In such cases, the prediction model parameter set can be updated based using backpropagation and / or stochastic gradient descent.

[0094] In some embodiments, the machine learning model 504 may be part of a larger segmentation model comprising multiple machine learning models. In general terms, for a segmentation model, relevant feature data from an input dataset (e.g., a “test dataset”, “production dataset”, “inference dataset”, etc.) may be segmented into data subsets, each of which may be processed by a different machine learning model. The segmentation model can combine the outputs of the different machine learning models to produce a single output, e.g., a single credit card fraud score from multiple credit card fraud stores, a single power30 KILPATRICK TOWNSEND 792723011demand prediction from multiple power demand predictions, a single comment score from multiple comment scores, etc. Segmentation models may be useful in situations in which it would be computationally slow, difficult, or infeasible to process a large set of features with a single model, as many models scale quadratically (or worse than quadratically) with the number and size of input features.

[0095] Regardless, even though FIG.5 depicts only a single machine learning model 504 and a single classification model 516, in some embodiments the computer system 502 may instantiate, train, and / or utilize multiple machine learning models, each of which may have any number of encoders and classification models. It should be appreciated more generally that the numbers of devices, entities, and components shown in FIG.5 were selected for simplicity of illustration and exposition. It should be understood that systems according to embodiments of the present disclosure can include more than one of each device, entity, component, computer system, etc. In addition, some systems according to embodiments may include a lesser number of devices, entities, and / or components or a greater number of devices, entities, and / or components than those shown in FIG.5. For example, computer system 502 may comprise a distributed computing system comprising several computers collectively performing methods according to embodiments. Likewise, there may be multiple data sources 508 from which computer system 502 retrieves training data sequences 514 for the purpose of training machine learning models, along with multiple communication networks 512 over which computer system 502 communicates with client computers(s) 510.

[0096] After generating scores, classifications, predictions, or other outputs using machine learning model 504 and classification model 516, such scores, predictions, or classifications can be used or analyzed by an operator of computer system 502 or an organization that owns and / or operates computer system 502. For example, an analyst at a power plant could use classifications from classification model 516 to prepare for a sudden projected increase in power demand. Additionally, in some embodiments, scores, predictions, or other outputs of classification model 516 can be used to implement some form of automatic action or interaction based on those scores, predictions, or classifications. For example, computer system 502 could automatically approve or deny an access request or approve or delete a post on a social media website based on a classification (e.g., classifying that access request or post on a social media website) produced by classification model 516. As an alternative however, computer system 502 may not comprise classification model 516, and instead computer system 502 may directly output embeddings produced using machine learning31 KILPATRICK TOWNSEND 792723011model 504, e.g., to another computer system, which may include a prediction model that can be used to process those embeddings. In some cases, it may be more efficient to train and utilize encoder models offline and use such models to generate embeddings corresponding to large numbers of data sequences, which can then be stored (e.g., in a database) for later use. Later, when necessary, such embeddings can be input into a classification model to generate classifications or other outputs (e.g., predictions). Splitting up processing in this way can reduce the amount of “real-time” processing that needs to be performed for a given task, thereby reducing task latency.

[0097] Computer system 502 could also be used to implement machine learning based analysis as a service for others, e.g., on behalf of client computer(s) 510 or users of client computer(s) 510 (which may also be referred to as “requestors”). For example, a client computer 510 could comprise a computer associated with a social networking web company. The social networking web company could outsource their content moderation to an organization associated with computer system 502 (which could comprise a server computer). In such a case, client computer 510 could request scores from computer system 502 (e.g., scores indicating whether a posted comment violates community guidelines or is otherwise malicious). Computer system 502 could use machine learning model 504 to generate embeddings, generate scores based on those embeddings using classification model 516, and transmit those scores back to the client computer 510 (e.g., via communication network 512), which could then use them for some purpose (e.g., removing comments that violate community guidelines).

[0098] In such scenarios, client computer(s) 510 and computer system 502 can communicate over a communication network 512. A communication network such as communication network 512 can take any suitable form, and may include any one and / or the combination of the following: a direct interconnection; the Internet; a Local Area Network (LAN); a Metropolitan Area Network (MAN); an Operating Missions as Nodes on the Internet (OMNI); a secured custom connection; a Wide Area Network (WAN); a wireless network (e.g., employing protocols such as, but not limited to a Wireless Application Protocol (WAP), I-mode, and / or the like); and / or the like. Messages between computers and devices in the system of FIG.5 and / or over communication network 512 may be transmitted using a secure communication protocol, such as, but not limited to, File Transfer Protocol (FTP); HyperText Transfer Protocol (HTTP); Secure HyperText Transfer Protocol (HTTPS); Secure Socket Layer (SSL), ISO (e.g., ISO 8583) and / or the like. Any suitable32 KILPATRICK TOWNSEND 792723011communication protocol can be used to communicate over the communication network 512, e.g., for the purpose of creating one or more communication channels. A communication channel may, in some instances, comprise a secure communication channel, which may be established in any known manner, such as through the use of mutual authentication, a session key, and establishment of a Secure Socket Layer (SSL) session.

[0099] In some embodiments, such requests from client computer(s) 510 may contain all the information (e.g., data sequence(s)) necessary for computer system 502 to generate classifications, predictions, scores, or any other data using machine learning model 504 and classification model 516. However, in other embodiments, computer system 502 may retrieve relevant data, features, etc., from another data source 508, e.g., a memory element (e.g., a hard drive), database, or any other data structure or storage element, in order to service requests from client computer(s) 510. As an example, an analyst at a national weather service may request a machine learning based weather forecast in a distance region of their country. The analyst may use a client computer 510 to connect to computer system 502, which is located at a remote weather monitoring station. Using a data sequence stored on a hard drive located in the remote weather monitoring station, computer system 502 may use machine learning model 504 to generate an output embedding, then use classification model 516 to generate a weather forecast based on that embedding. The forecast can be returned to the client computer 510 and can then be interpreted by the weather analyst. As an alternative however, computer system 502 may not comprise classification model 516, and instead computer system 502 may directly transmit output embeddings produced using machine learning model 504 to the client computer 510, which could then process those embeddings as an owner or operator of client computer 510 sees fit.

[0100] In addition to using machine learning model 504 to generate embeddings and classification model 516 to generate scores, classifications, or predictions, computer system 502 can also train machine learning model 504 and / or classification model 516 for this purpose. Such training methods are described in more detail further below with reference to FIGs.11 and 12. As a general summary however, computer system 502 can train the machine learning model 504 using novel contrastive learning techniques. As part of this training process, computer system 502 can retrieve training data sequences 514 e.g., from data source 508. Such data could comprise, e.g., data corresponding to objects, users, historical events, interactions between entities (e.g., access request such as credit card transaction interactions, computer communication interactions, etc.), etc. For example, for a33 KILPATRICK TOWNSEND 792723011machine learning model 504 used as part of an access review system (e.g., used to classify access requests as normal or invalid), the training data could comprise labelled training data sequences corresponding to historical access requests. These retrieved training data sequences can be used to train machine learning model 504 to generate embeddings using the contrastive loss training methods described further below. Afterwards, the classification model 516 could be trained using the generated embeddings and the corresponding labels, thereby training classification model 516 to detect invalid access requests based on corresponding embeddings. III. MACHINE LEARNING MODELS AND INFERENCE METHODS

[0101] Embodiments can use different window lengths for the subsequences. Use of such different scales is described below for multi-scale techniques. An inference method using a trained model for a new data sequence is then described. A later section describes the training of a model. A. Multi-scale Techniques

[0102] Machine learning models and inference methods using such machine learning models are described in more detail below. For example, a machine learning model 600 (sometimes referred to as a “backbone” model) according to embodiments is depicted in FIG. 6. As summarized above and described in detail herein, machine learning models according to embodiments can comprise multi-scale models that can generate output embeddings (e.g., output embedding 652) that capture and represent patterns or other information from data sequences (e.g., data sequence 612) at multiple scales or timeframes. This can result in better performance by classification models (or other systems) that use such embeddings to generate classifications (or perform other tasks). For example, a machine learning model used to predict electric power demand may produce more accurate predictions based on output embeddings generated using methods and machine learning models according to embodiments.

[0103] Generally, a machine learning model according to embodiments can comprise a sequence encoder (also referred to as a “sequence module”) and one or more window encoders (also referred to as “window modules”). Machine learning model 600 comprises sequence encoder 602 and three window encoders 604-608, however it should be understood34 KILPATRICK TOWNSEND 792723011that machine learning models according to embodiments can comprise more or less than three window encoders.

[0104] Generally, sequence encoder 602 can generate a set of sequence partial embeddings 662 that can capture high-level patterns between data elements in the data sequence 612. By contrast, each window encoder can generate a set of window partial embeddings (e.g., sets of window partial embeddings 664-668) that can capture patterns or relationships between sets of data subsequences (e.g., sets of data subsequences 614-618). Unlike the set of sequence partial embeddings 662, each set of window partial embeddings 664-668 can capture more granular patterns or relationships, which may correspond to window lengths defining the lengths of subsequences in the sets of data subsequences 614-618. For example, if set of data subsequences 618 generally corresponds to 24 hours of electricity usage data, set of window partial embeddings 668 (generated using set of data subsequence 618) may generally capture patterns corresponding to 24-hour periods. Herein and in the figures, the terms “long”, “medium”, and “short” are used to describe different subsequence lengths. However, it should be understood that these are intended only as general descriptors to facilitate a better understanding of embodiments of the present disclosure, and there is no specific length (e.g., in numbers of data elements) associated with short, medium, or long subsequences. Generally, using multiple window lengths may enable machine learning model 600 to more accurately capture a larger variety of different temporal patterns.

[0105] Further, a given window encoder may generate its set of window partial embeddings based not only on a corresponding set of subsequences, but also based on either a set of window partial embeddings generated by another window encoder or based on the set of sequence partial embeddings 662 generated by the sequence encoder 602. For example, window encoder 604 can generate a set of window partial embeddings 664 based on set of sequence partial embeddings 662 in addition to set of subsequences 614. Likewise, set of window partial embeddings 664 can be used by window encoder 606 (in addition to set of subsequences 616) to generate set of window partial embeddings 666, which can be used by window encoder 608 to generate set of window partial embeddings 668, and so on. In this way, the sets of window partial embeddings generated by each successive window encoder may be enhanced by the information contained in the set of window partial embeddings (or the set of sequence partial embeddings 662) produced by the previous layer of machine learning model 600.35 KILPATRICK TOWNSEND 792723011

[0106] Machine learning model 600 may have a “hierarchical” or “pyramidal” structure, such that the window encoders are organized based on ascending or descending “window lengths” of subsequences. For example, set of subsequences 614 corresponding to window encoder 604 may comprise “long” subsequences, set of subsequences 616 corresponding to window encoder 606 may comprise “medium” subsequences, and set of subsequences 618 corresponding to window encoder 608 may comprise “short” subsequences, such that each successive window encoder processes successively shorter subsequences, and such that each successive window encoder uses a set of window partial embeddings generated based on longer subsequences (or the set of sequence partial embeddings 662 in the case of window encoder 604) in order to generate its own set of window partial embeddings. However, it should be understood that alternative configurations are also possible, e.g., such that each successive window encoder processes successively longer subsequences, and such that each successive window encoder uses a set of window partial embeddings generated from shorter subsequences (or the set of sequence partial embeddings 662 in the case of window encoder 604) in order to generate its own set of window partial embeddings.

[0107] The sets of window partial embeddings generated by each window encoder can combined (e.g., via averaging blocks 630, 638, and 646 and combine block 648, which can implement a combination method such as concatenation for window partial embeddings comprising vectors or arrays) to produce output embedding 652, e.g., via a feed-forward layer 650 (e.g., a fully-connected neural network). As described herein (e.g., above with reference to FIG.4), output embedding 652 can be used for various tasks such as anomaly detection. B. Inference Method

[0108] An inference method according to embodiments is described herein with reference the flowchart of FIG.7, as well as FIGs.6 and FIGs.8-10. The inference method can be performed by a computer system that can instantiate and execute a machine learning model according to embodiments. As described above, the machine learning model (which may comprise an “encoder model”) can comprise a sequence encoder and one or more window encoders. The computer system can use the machine learning model to generate an output embedding corresponding to a data sequence. The output embedding can in turn be used to generate a classification corresponding to the data sequence, e.g., using a classification model, as described above with reference to FIG.5.36 KILPATRICK TOWNSEND 792723011

[0109] At step 702 of FIG.7, a computer system can retrieve a data sequence. In some embodiments, the computer system can retrieve the data sequence from a database, data stream, a local memory element such as a hard drive, cloud storage, an I / O interface, or any other appropriate source. In other embodiments, the computer system can comprise a server computer that performs machine learning based services for client computers. In such embodiments, the computer system can receive the data sequence from a client computer (or from any other appropriate source), e.g., over a communication network such as the Internet, as described above with reference to FIG.5.

[0110] The data sequence can comprise a plurality of data elements, each of which can comprise data values (e.g., features). In some embodiments, the data sequence can correspond to a sequence of historical access requests, e.g., for a resource. A data sequence can correspond to a sequence of events or interactions, object(s), entities, person(s), etc., which may be the subject of machine learning analysis.

[0111] In some embodiments, the data sequence can comprise a time-series sequence of data comprising a plurality of data elements. In such embodiments, each data element in the time-series sequence of data can comprise an observed value of an object and a timestamp. As an example, an object could comprise a user device (e.g., a smartphone) and each observed value of the object could comprise, e.g., the battery level, a binary indicator indicating whether the smartphone was in use, etc.

[0112] More generally, the data sequence can comprise a sequence of data values, which may comprise quantifiable or qualifiable measurements or observation about the subject(s) of the data sequence. For example, for a data sequence corresponding to electrical power demand, the data sequence could comprise sets of features predictive of electric power demand (e.g., the time of year, the time of day, the outdoor temperature at given times, precipitation at given times, latitude, longitude, etc.). As another example, if embodiments of the present disclosure are used as part of an access review system that predicts whether credit card transactions are legitimate or invalid (e.g., an instance of credit card fraud), the data sequence could comprise sets of features corresponding to historical credit card transactions, which may be predictive of credit card fraud, e.g., purchase amounts, purchase locations, purchase times of day, credit card balances, etc.). As yet another example, if embodiments of the present disclosure are used to provide cybersecurity for computer systems, the data sequence could comprise sets of features corresponding to attempts to access computer37 KILPATRICK TOWNSEND 792723011systems and could comprise, e.g., the contents of messages transmitted over a network during access attempts, calls to the system kernel, etc.

[0113] Herein, the data sequence (also referred to as a “classification sequence of data”, or in some cases a “time series sequence of data) may be indicated by the notation C. In some embodiments, the computer system can retrieve the data sequence by constructing the data sequence from data stored in one or more data sources. For example, the computer system can generate the data sequence by searching a database (or other data structure) for relevant events or observations corresponding to an entity or account, then chronologically sort those events or observations by associated timestamps, thereby generating the data sequence. For example, in order to generate a data sequence corresponding to an individual’s access requests, the computer system could search a database of recorded access requests (e.g., comprising access requests corresponding to multiple individuals) for access requests corresponding to that individual, then sort data elements corresponding to those access requests using associated timestamps to generate the data sequence.

[0114] A data sequence can comprise historical data corresponding to a particular entity, account, object, or phenomena and can be used by the computer system and the machine learning model to infer high-level information or behavioral patterns (if relevant), high-level temporal patterns, and a basic personal profile of a corresponding entity (if relevant). The data sequence may comprise any number of data elements corresponding to an entity, account, object, phenomena, etc., e.g., comprising at least 10, 100, 1,000, 10,000, 100,000, 1,000,000, or one billion data elements. In some cases it may be preferable for the data sequence to have enough data elements or cover a long enough time range to capture long- term temporal trends (e.g., over the last month, the last year, the last two years, the last five years, the last ten years, etc.) corresponding to an entity, account, object, or phenomena. In some embodiments, the computer system can identify data elements corresponding to a given time frame (e.g., the last five years) using timestamps in order to construct the data sequence.

[0115] Herein, a data sequence C may be represented by the sequence ^^ ൌ^^^௧ି்ା^, … , ^^௧ି^, ^^௧ ^. The variable t may be used to indicate the timestamp associated with aparticular event, observation, or data element in the data sequence, while the variable T may be used to indicate the length of the data sequence. As described above, it may be useful to have a larger value of T in order to capture both short-term and long-term velocity feature trends. However, smaller and larger values of T may be appropriate based on the particular38 KILPATRICK TOWNSEND 792723011use case of embodiments of the present disclosure. The variable x is used to refer to raw attributes of events (e.g., data values and data elements, which may comprise or be represented using a vector), such as an amount or amounts associated with an event, an age of an account associated with the event (e.g., an account corresponding to an entity associated with the event), an account type, an event location, etc.

[0116] As depicted in FIG.6, data sequence 612 may be combined with time encodings 654 prior to generating the set of sequence partial embeddings 662 using sequence encoder 602. In general terms, attention-based encoder systems (such as sequence encoder 602) are typically invariant to the order of data elements in data sequence 612. By including time encodings 654 in data sequence 612, the computer system can remove this sequence invariance, thereby causing the sequence encoder 602 to capture temporal patterns in the set of sequence partial embeddings 662. As such in some embodiments, at step 702 the computer system can additionally determine a plurality of time encodings corresponding to the plurality of data elements in the data sequence. The plurality of time encodings can generally indicate, e.g., a year, month, day, hour, and / or minute associated with a corresponding data element from data sequence 612. In some cases, the time encodings can comprise timestamps associated with the data sequence, e.g., associated with each individual data element. In other cases, the time encodings can comprise encodings derived from such timestamps, using any appropriate derivation technique.

[0117] At step 704, the computer system can generate one or more sets of data subsequences from the data sequence based on one or more window lengths. Each data subsequence can comprise a number of sequential data elements from the data sequence. Each number of sequential data elements can correspond to a window length of one or more window lengths. Each set of data subsequences can correspond to a window encoder of the one or more window encoders. As an illustrative example, a “long” window length could comprise 128 data elements, a “medium” window length could comprise 64 data elements, and a “short” window length could comprise 32 data elements. One set of data subsequences could comprise subsequences comprising 128 data elements (e.g., “long” subsequences corresponding to the “long” window length), another set of data subsequences could comprise subsequences comprising 64 data elements (e.g., “medium” subsequences corresponding to the “medium” window length), and another set of data subsequences could comprise subsequences comprising 32 data elements (e.g., “short” subsequences corresponding to the “short” window length). Referring to FIG.6, the set of “long” data39 KILPATRICK TOWNSEND 792723011subsequences 614 could be processed using window encoder 604, the set of “medium” data subsequences 616 could be processed using window encoder 606, and the set of “short” data subsequences 618 could be processed using window encoder 608.

[0118] Herein, various terms may be used to denote and differentiate between data subsequences and sets of data subsequences. For example, in some embodiments, the machine learning model may comprise a single window encoder and the one or more sets of data subsequences may comprise a single set of data subsequences. In other embodiments, the machine learning model may comprise multiple window encoders, and the one or more sets of data subsequences can comprise a “first set of data subsequences” and one or more “additional sets of data subsequences”, which could themselves comprise a “second set of data subsequences”, a “third set of data subsequences”, and so on.

[0119] Step 704 may be better understood with reference to FIG.8, which shows an exemplary data sequence 802 comprising seven data elements. A computer system according to embodiments can segment data sequence 802 into three subsequences 808-812 based on a window length 804 (Wlong) of five data elements. In FIG.8, segmentation is performed using sliding window segmentation with a stride length of one, i.e., subsequence 808 comprises the first five data elements from data sequence 802, subsequence 810 comprises the second through sixth data element from data sequence 802, and subsequence 812 comprises the third through seventh data element from data sequence 802. Collectively, subsequences 808-812 could comprise a set of data subsequences. It should be understood that window lengths and stride values can comprise hyperparameters and various window lengths and stride values (or even other segmentation techniques) can be used. Likewise in FIG.8, the computer system can segment data sequence 802 into four subsequences 814-820 based on window length 806 (Wmedium) of four data elements. The computer system can further generate other subsequences 822 based on other window lengths (e.g., “short” subsequences, “very long” subsequences, etc.). As described above, there may be various relationships 824 and 826 between subsequences of various lengths, and by using such subsequences to generate an output embedding, a machine learning model according to embodiments may better capture such relationships or patterns, e.g., leading to more accurate classification.

[0120] Herein, Kscan indicate a subsequence length of a subsequence, which can correspond to a window length. A set of subsequences corresponding to a given window40 KILPATRICK TOWNSEND 792723011length can be represented as a matrix, e.g., the set of subsequences corresponding to “long” window lengths can be represented by the matrix: ^^௧ି்ା^^^௧ି்ାଶ … ^^… … ௧ି்ା^^^^^^^^^^^ ൌ ^ … … ^

[0121] In the to Wlong,and each row in Wlongcorresponds to a different subsequence corresponding to that window length. For FIG.8, for example, Wlong could have three rows, corresponding to the three subsequences 808-812. A matrix such as the Wlongmatrix can represent or comprise a set of data subsequences. A medium window length matrix ^^^^ௗ^௨^and a short window length matrix ^^^^^^௧can be defined similarly, e.g.: ^^௧ି்ା^^^௧ି்ାଶ … ^^௧ି்ା^^^^^ௗ^௨^ … … ^^ೠ^^^ ൌ ^ … …ൡ

[0122] Padding can be used to enforce the length of different windows. For example, padding can be used to create subsequences corresponding to specific lengths of time, e.g., by adding padding data elements to a subsequence to make it the appropriate length.

[0123] In some embodiments, the data sequence can comprise all the data corresponding to the data elements in the data sequence. For example, if each data element in a data sequence corresponds to observed weather conditions at a given time, e.g., outdoor temperature, wind speed, humidity, rainfall, etc., the data sequence can comprise a sequence of data elements each containing data values for outdoor temperature, wind speed, humidity, rainfall, etc. Similarly, each data subsequence could comprise all the data contained in the data elements associated with that data subsequence. As an alternative however, each data subsequence could only comprise some of the data values associated with the data elements associated with that data subsequence. For example, if outdoor temperature is the “most important” feature for predicting future weather patterns, the sets of data subsequences could include only the outdoor temperature information and exclude wind speed, humidity, rainfall etc. Because the use of data subsequence generally tends to increase the size of inputs to the41 KILPATRICK TOWNSEND 792723011machine learning model, it may increase processing time and computational complexity. By removing data from the data sequences, thereby reducing their logical size, processing time and computational complexity may be reduced.

[0124] In some embodiments the computer system can determine one or more pluralities of positional encodings corresponding to the one or more sets of data subsequences (e.g., positional encodings 656-660 in FIG.6). Each plurality of positional encodings can indicate a temporal ordering of a corresponding set of data subsequences. For example, referring to FIG.8, positional encodings could indicate that in a set of data subsequences, subsequence 808 precedes subsequence 810, which in turn precedes subsequence 812. Referring back to FIG.6, in general terms, attention-based encoder systems (such as window encoders 604- 608) are generally invariant to the order of data subsequences in the sets of data subsequence 614-616. By including positional encodings 656-660 in the one or more sets of data subsequences 614-616, the computer system can remove this sequence invariance, thereby causing window encoders 604-608 to capture temporal patterns in the one or more sets of data subsequences.

[0125] Referring back to FIG.7, at step 706, the computer system can generate a set of sequence partial embeddings by applying the data sequence to the sequence encoder. As described above, in some embodiments the computer system can determine and include a plurality of time encodings (e.g., time encodings 654) in the data sequence. As such, the set of sequence partial embeddings can be generated based on the plurality of time encodings (in addition to the data sequence). As depicted in FIG.6, the sequence encoder 602 can comprise an inner encoder 620 and a feed-forward block 622. A more detailed depiction of a sequence encoder is presented in FIG.9.

[0126] Sequence encoder 900 of FIG.9 comprises an inner encoder 902 and a feed-forward block 904. The inner encoder 902 can comprise a self-attention layer 906 and an “add and normalize” layer 908. In some embodiments, the self-attention layer 906 can comprise a multi-head self-attention layer, e.g., similar to the multi-head attention model 304 of FIG.3. The feed-forward block 904 can comprise a feed-forward layer 910 (which can comprise a feed-forward neural network, which may comprise a fully-connected neural network layer) and an “add and normalize” layer 912. As described above, the computer system can incorporate time encodings 916 into data sequence 914 prior to applying data sequence 914 to sequence encoder 900. In some embodiments, the computer system can derive one or more42 KILPATRICK TOWNSEND 792723011attention queries, one or more attention keys, and one or more attention values from the data sequence 914. The computer system can then apply the attention queries, keys, and values 920 to the self-attention layer 906 of inner encoder 902 and add and normalize the result with the data sequence 914 (using “add and normalize block” 908), thereby generating a set of intermediate embeddings 922. The computer system can then generate the set of sequence partial embeddings 918 by applying the set of intermediate embeddings 922 to the feed- forward block, e.g., by applying the set of intermediate embeddings 922 to the feed-forward layer 910 (which can comprise a feed-forward neural network) and add and normalize the result with the set of intermediate embeddings 922 (e.g., using “add and normalize” block 912), thereby generating the set of sequence partial embeddings 918. Each sequence partial embedding or the set of sequence partial embeddings 918 can comprise or be represented by embedding vectors or arrays, e.g., vectors of embeddings or vectors of numerical values representative of embeddings.

[0127] Referring back to FIG.7, at step 708, the computer system can generate one or more sets of window partial embeddings (also referred to as “one or more output classification embeddings”, as they can be used to generate an output embedding that can be used to generate a classification, as described herein) by applying the one or more sets of data subsequences and the set of sequence partial embeddings to the one or more window encoders. As described above, in some embodiments the computer system can determine and include one or more pluralities of positional encodings (e.g., positional encodings 656-660) in the one or more sets of data subsequences. As such, the one or more sets of window partial embeddings can be generated based on the one or more pluralities of positional encodings (in addition to the one or more sets of data subsequences and the set of sequence partial embeddings). As depicted in FIG.6, window encoder 604 can comprise a inner encoder 624, a merge encoder 626, and a feed-forward block 628 (window encoders 606 and 608 can comprise similar components). A more detailed depiction of a window encoder is presented in FIG.10.

[0128] Much like sequence encoder 900 of FIG.9, window encoder 1000 of FIG.10 comprises an inner encoder 1002 and a feed-forward block 1006. The inner encoder 1002 can comprise a first self-attention layer 1008 and an “add and normalize” layer 1010. In some embodiments, the first self-attention layer 1008 can comprise a multi-head self- attention layer. The feed-forward block 1006 can comprise a feed-forward layer 1016 (which can comprise a feed-forward neural network, which may comprise a fully-connected neural43 KILPATRICK TOWNSEND 792723011network layer) and an “add and normalize” layer 1018. As described above, the computer system can incorporate positional encodings 1024 into a set of subsequences 1022 prior to applying the set of subsequences 1022 to window encoder 1000. In some embodiments, the computer system can generate a first set of intermediate embeddings 1032 by applying the set of subsequences 1022 to the inner encoder 1002, e.g., by applying the set of subsequences 1022 to the first self-attention layer 1008 then adding and normalizing the result with the set of subsequences 1022 using “add and normalize” layer 1010. This first set of intermediate embeddings 1032 can be used by the computer system to generate a set of window partial embeddings 1028 as described below. In some embodiments, the computer system can derive one or more attention queries, one or more attention keys, and one or more attention values from the set of subsequences 1022. The computer system can then apply the attention queries, keys, and values 1030 to the first self-attention layer 1008 of inner encoder 1002 and add and normalize the result with the set of subsequences 1022 (using “add and normalize” layer 1010), thereby generating the first set of intermediate embeddings 1032.

[0129] However, unlike sequence encoder 900 of FIG.9, window encoder 1000 can also comprise a merge encoder 1004, which can “merge” or combine the first set of intermediate embeddings 1032 with a set of sequence partial embeddings generated using a sequence encoder or a set of window partial embeddings generated using a different window encoder. The computer system can use the merge encoder 1004 to generate a second set of intermediate embeddings 1038 by applying the first set of intermediate embeddings 1032 and the set of sequence partial embeddings or a set of window partial embeddings to the merge encoder 1004, e.g., by applying the first set of intermediate embeddings 1032 and the set of sequence partial embeddings or a set of window partial embeddings to a second self-attention layer 1012 then adding and normalizing the result with the first set of intermediate embeddings 1032 (using e.g., “add and normalize” layer 1014), thereby generating the second set of intermediate embeddings 1038. In some embodiments, the computer system can derive one or more attention keys and one or more attention queries 1034 from the set of sequence partial embeddings or a set of window partial embeddings. The computer system can further derive one or more attention values 1036 from the first set of intermediate embeddings 1032. The computer system can generate the second set of intermediate embeddings 1038 by applying the one or more attention keys and the one or more attention queries 1034 and the one or more attention values 1036 to the second self-attention layer 1012. The computer system can then generate a set of window partial embeddings 1028 by applying the second44 KILPATRICK TOWNSEND 792723011set of intermediate embeddings 1038 to the feed-forward block 1006, e.g., by applying the second set of intermediate embeddings 1038 to the feed-forward layer 1016 (which can comprise a feed-forward neural network) and add and normalize the result with the second set of intermediate embeddings 1038 (e.g., using “add and normalize” layer 1018), thereby generating the set of window partial embeddings 1028.

[0130] By performing this process using each window encoder of the one or more window encoders, the computer system can generate one or more sets of window partial embeddings corresponding to the one or more window encoders. As described further below, the sets of window partial embeddings can be averaged (e.g., via average block 1020) and combined (e.g., via vector or array concatenation for sets of window partial embeddings comprising embedding vectors or embedding arrays) to produce an output embedding.

[0131] As described above with reference to FIG.6, a window encoder can use a set of window partial embeddings from a previous window encoder in a machine learning model to generate its set of window partial embeddings. For example, window encoder 606 can use the set of window partial embeddings 664 from window encoder 604 to generate set of window partial embeddings 666. As such, in some embodiments the one or more data subsequences can comprise a first set of data subsequences and one or more additional sets of data subsequences. Similarly, the one or more window encoders can comprise a first window encoder and one or more additional window encoders. Further, the one or more sets of window partial embeddings can comprise a first set of window partial embeddings and one or more additional sets of window partial embeddings. The computer system can generate the first set of window partial embeddings by applying the first data subsequence and the set of sequence partial embeddings to the first window encoder, e.g., as described above with reference to FIG.10. The computer system can then generate the one or more additional sets of window partial embeddings by applying the one or more additional sets of data subsequences and the first set of window partial embeddings to the one or more additional window encoders. Expanding further, in some embodiments the one or more additional sets of data subsequences can comprise a second set of data subsequences, the one or more additional window encoders comprise a second window encoder, and the one or more additional sets of window partial embeddings comprise a second set of window partial embeddings. In such embodiments, a computer system can generate a first set of intermediate embeddings by applying the second set of data subsequences to the inner encoder, generate a second set of intermediate embeddings by applying the first set of45 KILPATRICK TOWNSEND 792723011intermediate embeddings and the first set of window partial embeddings to the merge encoder, and generate the second set of window partial embeddings by applying the second set of intermediate embeddings to the feed-forward block, e.g., as described above with reference to FIG.10. This process is generally depicted in FIG.6, in which set of data subsequences 614 (e.g., a “first set of data subsequences”) and set of sequence partial embeddings 662 are used by window encoder 604 (e.g., a “first window encoder”) to generate set of window partial embeddings 664 (e.g., a “first set of window partial embeddings”), which are in turn used by window encoder 606 (e.g., a “second window encoder”) along with set of data subsequences 616 (e.g., a “second set of data subsequences”) to generate set of window partial embeddings 666 (e.g., a “second set of window partial embeddings”), and so on. It should be appreciated that this method can be extended to e.g., a third set of data subsequences and a third set of window partial embeddings, a fourth set of data subsequences and a fourth set of window partial embeddings, etc.

[0132] Referring to FIGs.6 and 7, at step 710 the computer system can combine the one or more sets of window partial embeddings, thereby generating an output embedding (e.g., output embedding 652). As each set of window partial embeddings corresponds to a different window length (and e.g., a different time period or frequency), the output embedding can capture or contain information corresponding to various time periods in the data sequence. This extra information may enhance the accuracy or effectiveness of classification tasks or other tasks performed using output embeddings, such as access review systems. As depicted in FIG.6, in some embodiments the computer system can average each set of window partial embeddings (using average blocks 630, 638, and 646) then combine the results (using combination block 648). The resulting combination can be applied to a feed-forward layer 650 (e.g., a feed-forward neural network) in order to produce output embedding 652. In some embodiments, combination block 648 can implement vector or array concatenation, e.g., if the averages of each set of window partial embeddings comprise embedding vectors or embedding arrays, combination block 648 can be used to concatenate those embedding vectors or arrays in order to generate the output embedding 652.

[0133] Optionally, at step 712, if the computer system is performing machine learning based analysis on behalf of a client computer, the computer system can transmit the output embedding to a client computer, which could comprise the client computer from which the computer system initially received the data sequence. The computer system can transmit the output embedding to the client computer over a communication network such as the Internet,46 KILPATRICK TOWNSEND 792723011or via any other appropriate means. Such an output embedding may be much smaller (e.g., 100 or 1000 times) than the data sequence used to produce the output embedding. A client computer may have sufficient computing resources to process an output embedding (e.g., as part of a classification task), but may not have the computing resources needed to process the data sequence or generate the output embedding based on the data sequence. Hence, embodiments of the present disclosure may be useful for providing machine learning based services for client computers.

[0134] Optionally, at step 714, the computer system can generate the classification corresponding to the data sequence based on the output embedding. In some embodiments, in which the data sequence relates to an object, the classification corresponding to the data sequence can comprise a classification of the object. The computer system can generate the classification based on the output embedding by applying the output embedding to a classification model.

[0135] The classification model may be referred to as a “second machine learning model”, in order to differentiate it from other machine learning models described herein (e.g., a machine learning model similar to machine learning model 600 of FIG.6, which may be referred to as a “first machine learning model”). As described above, the computer system can generate the output embedding (which may also be referred to as a “combined output classification embedding”) by combining one or more sets of window partial embeddings (or “one or more output classification embeddings”). The computer system can generate the classification based on the output embedding (“combined output classification embedding”), e.g., by applying the output embedding to the classification model (or “second machine learning model”) as stated above. The classification model can comprise any type of classification model, such as a multilayer perceptron, a fully-connected layer (or other neural network model), or any other machine learning model that can be used for generating classifications (e.g., a clustering model, a random forest model, a gradient-boosted random forest model, a decision tree, or a combination or ensemble thereof).

[0136] In some embodiments, the classification can comprise a prediction corresponding to the data sequence. Such a prediction can comprise a binary prediction, indicating e.g., whether some subject of the data sequence belongs to a class of subjects or does not belong to that class of subjects. For example, a binary fraud prediction may indicate whether a transaction conducted by a user is likely valid or likely invalid, based on a data sequence of47 KILPATRICK TOWNSEND 792723011historical transactions performed by that user. As another example, a binary disease prediction may indicate whether a patient likely suffers from a given disease (e.g., type 2 diabetes) or does not suffer from that disease based on a data sequence of patient health data. However, it should be understood that embodiments of the present disclosure can be used to generate classifications corresponding to different types of predictions, including non-binary predictions such as continuous predictions (e.g., indicating a predicted power demand value in megawatts) or trinary, quaternary, etc., predictions (e.g., indicating whether a subject belongs to one of several mutually exclusive classes). Similar to step 712, at step 714 the computer system can transmit the classification to a client computer, e.g., in order to provide machine learning based services to the client computer.

[0137] Optionally, at step 716, the computer system can perform further processing based on the classification. Various actions, interactions, or further processing can be performed (automatically or otherwise) by either the computer system or a client computer responsive to the classification generated at step 714. For example, if the data sequence corresponds to a sequence of historical access requests, if the classification indicates that an access request is invalid (e.g., corresponding to an instance of credit card fraud) based on the sequence of historical access requests, then further processing can comprise causing the access request to be rejected based on the classification.

[0138] Alternatively, if the classification indicates than an access request is legitimate based on the sequence of historical access requests, then further processing can comprise causing the access request to be accepted based on the classification. As a more specific example, in the context of credit card fraud detection, the computer system or a client computer could automatically approve or deny a transaction based on the classification, e.g., approving the transaction if it is likely valid based on the classification and denying the transaction if it is likely invalid based on the classification. For example, if the classification was a binary prediction (with e.g., “TRUE” corresponding to a valid transaction and “FALSE” corresponding to an invalid transaction), the computer system or client computer could approve the transaction if the classification is “TRUE” and deny the transaction if the classification is “FALSE”. As another example, if the classification was a continuous prediction (e.g., a percentage indicating the likelihood that a transaction was invalid), the computer system or the client computer could approve the transaction if the classification was under a threshold value (e.g., 50%) and approve the transaction if the test prediction was greater than the threshold value.48 KILPATRICK TOWNSEND 792723011

[0139] Similarly, in the context of a context moderation system for a social networking website, the computer system or the client computer (or e.g., an operator of either system) could delete a post on the social networking website if a corresponding classification indicates that the post is likely malicious or violates community guidelines based on a data sequence of historical posts. As another example, in the context of a power demand forecasting system, the computer system or the client computer associated with a hydroelectric dam could perform some action responsive to a classification that indicates increasing electric power demand. For example, a computer system or a client computer associated with a hydroelectric governor could increase the speed of hydraulic turbines at a hydropower plant, thereby increasing electric power supply to meet the forecasted increased demand. As another example, in the context of a cybersecurity system, the computer system or client computer could issue an alert, responsive to a classification that indicates a high likelihood of cyberattack, system intrusion, or the presence of viruses or malware within a computer network. Various other actions, automatic or otherwise, can be performed based on the context or application of machine learning models according to embodiments, and the examples provided herein are intended to be non-limiting. IV. CONTRASTIVE TRAINING METHODS

[0140] As described above, some embodiments of the present disclosure are directed to contrastive learning methods that can be used to train machine learning models according to embodiments (e.g., machine learning model 600 from FIG.6). One such embodiment is directed to a method performed by a computer system for training a machine learning model based on contrastive loss. The method can comprise performing an iterative training process until a terminating condition has been met. This training method is described in more detail below with reference to the flowchart of FIG.12. Prior to describing this method in more detail however, it may be useful to generally summarize contrastive learning methods according to embodiments with reference to FIG.11. A. Architecture

[0141] Contrastive learning methods according to embodiments can comprise a form of self-supervised learning, which can be performed without training labels. This can be useful as it allows machine learning models according to embodiments to be trained “offline” and in a task-agnostic manner. For example, a machine learning model according to embodiments49 KILPATRICK TOWNSEND 792723011does not need to be trained within the context of e.g., an access review system, forecasting power demand, etc., and can instead be trained to more generally generate output embeddings that capture multi-scale sequence information from input data sequences.

[0142] FIG.11 generally depicts a contrastive learning method according to embodiments. A data sequence 1102 can be masked multiple times, thereby generating a plurality of masked data sequences 1104-1108. As an example, the data sequence 1102 can be masked by replacing data values in the data sequence 1102 with random values, zero values, default values, etc. Each masked data sequence 1104-1108 can be masked differently. The computer system can additionally generate various sets of subsequences 1110-1114 from either the data sequence 1102 or the masked data sequences 1104-1108, using methods similar to those described above with reference to FIGs.7 and 8. The computer system can use machine learning model 1116 to generate output embeddings 1122-1126 based on one or more sets of subsequences 1110-1114 and masked data sequences 1104-1108 (e.g., after incorporating time encodings 1130 and positional encodings 1132, if necessary), e.g., using methods similar to those described above with reference to FIGs.6 and 7. In this way the computer system can generate an output embedding for each masked data sequence.

[0143] The computer system can calculate a multi-pair contrastive loss 1128 based on differences between pairs of output embeddings. Generally, the masked data sequences 1104-1108 contain both the information from data sequence 1102 and largely “meaningless” noise introduced by masking process. Preferably, machine learning model 1116 generates output embeddings that contain the useful information content from data sequence 1102 and do not contain the noise introduced by masking. If this is the case, output embeddings 1122- 1126 are generally expected to be similar, even though they were generated using different masked data sequences. Hence the similarity or difference between output embeddings 1122- 1126 can form the basis of multi-pair contrastive loss 1128, such that the multi-pair contrastive loss 1128 is smaller when output embeddings 1122-1126 are more similar and larger when embeddings 1122-1126 are more different. This multi-pair contrastive loss 1128 can be used to update a parameter set of the machine learning model 1116, thereby training the machine learning model 1116. This process can be repeated over a number of training rounds, epochs (or other divisions) or until some other terminating condition has been met. After training is complete, machine learning model 1116 can be used to generate output embeddings corresponding to data sequences, which could then be used for various tasks. For example, clustering methods could be used to differentiate between groups of output50 KILPATRICK TOWNSEND 792723011embeddings, e.g., for the purpose of anomaly detection (e.g., as described above and depicted in FIG.4). As another example, a classifier model could be trained using output embeddings and corresponding data sequence labels, e.g., for the purpose of identifying anomalous data sequences (e.g., also as described above and depicted in FIG.4). B. Training Method

[0144] A training method according to embodiments is described in more detail below with reference to FIG.12. Using such a method, a machine learning model can be trained using contrastive loss. The method can be performed by a computer system (e.g., a computer system instantiating and training machine learning models according to embodiments) and can involve the computer system performing an iterative training process. This iterative training process can be performed until a terminating condition has been met, e.g., model convergence or a predefined number of training rounds or epochs.

[0145] At step 1202, after retrieving training data (comprising e.g., a plurality of training data sequences), e.g., from a database, a data stream, a local memory element such as a hard drive, cloud storage, an I / O interface, or any other appropriate source, the computer system can perform a round of an iterative training process. The round of the iterative training process can comprise steps 1204-1218 of FIG.12. The computer system can perform this iterative training process until a terminating condition has been met, e.g., a set number of training rounds or epochs, a convergence condition, or any other appropriate terminating condition.

[0146] Such training data sequences can comprise sequences of training data elements, each comprising, e.g., sets of features corresponding to some observed event, object, individual, etc., which may relate to a particular task or decision-making application associated with the machine learning model being trained. For example, if a machine learning model is being trained to generate embeddings that can be used to generate predictions related to electrical power demand, such training data elements could comprise sets of features predictive of electrical power demand (e.g., the time of year, the time of day, outdoor temperature, latitude, longitude, etc.).

[0147] As another example, if a machine learning model is being trained to generate embeddings used to generate predictions related to the legitimacy (or fraudulence) of access requests (e.g., credit card transactions), the training data elements could comprise sets of51 KILPATRICK TOWNSEND 792723011features predictive of credit card fraud (e.g., purchase amounts, purchase locations, purchase time of day, frequency of purchases, card balance, etc.). In some embodiments, training data elements may comprise message data (e.g., corresponding to message-based communications between two computer systems over the Internet), transaction data (e.g., corresponding to access requests), or machine performance data (e.g., corresponding to the performance of a computer system such as a server computer, e.g., relative to some benchmark performance or performance metrics).

[0148] In some embodiments, the training data sequences may be organized into a data table, such that each row of the table comprises a training data sequence (itself e.g., corresponding to an object, individual, etc.), and such that each column in the table corresponds to a different training data element (or vis versa). For example, for training data sequences corresponding to access requests, each row could comprise a training data sequence corresponding to a particular cardholder, and column of each row could correspond to a transaction performed by that cardholder.

[0149] At step 1204, the computer system can sample one or more training data sequences, e.g., from among the training data sequences described above. These one or more training data sequences can comprise a batch (or minibatch). The computer system can sample any number of training data sequences, such as 10, 100, 1000, 10,000, 100,000, one million, ten million, one hundred million, one billion, or ten billion training data sequences. The computer system can sample the one or more training data sequences in any appropriate manner, e.g., random sampling, sequential or ordered sampling, etc. Each training data element of the one or more training data sequences can comprise a plurality of training data values. Such training data values can comprise the features of their respective training data sequence. For example, for a training data sequence corresponding to recorded weather data, a training data value could comprise a feature such as the current outdoor temperature. Training data sequences can comprise various types of training data values. For example, a training data sequence can comprise nominal values, such as the names of cities (e.g., the city in which a credit card transaction took place), ordered nominal values, such as “underweight”, “normal weight”, “overweight”, or “obese”, numerical values (including interval values and ratio values), etc.

[0150] At step 1206, the computer system can generate a plurality of masked training data sequences for each training data sequence by masking those training data sequences. In this52 KILPATRICK TOWNSEND 792723011way the computer system can generate one or more pluralities of masked training data sequences. In some embodiments, the computer system can mask a training data sequence by replacing one or more data values in one or more data elements in the training data sequence with predetermined masking data values or random masking data values. For example, a data sequence could comprise data elements corresponding to hourly weather conditions. Each data element could comprise, e.g., an outdoor temperature value, a wind speed value, a humidity value, a location at which the weather data was recorded, etc. To mask such a data sequence, the computer system could randomly select data elements, then replace outdoor temperature values, wind speed values, humidity values, etc., within those data elements with predetermined masking values (including e.g., zero values), or random values. For example, the computer system could set wind speed values in the selected data elements to zero, or e.g., set outdoor temperature values to random values within a reasonable range (e.g., from - 15 degrees Celsius to 40 degrees Celsius). Similarly, the computer system could mask location data values by setting them to a default location value (e.g., corresponding to a major city or capital city in a particular country). As another example, the computer system could mask data values by setting them to NULL or NaN (“not a number”) data values. The computer system can mask the training data sequence differently with each masking operation, such that each masked training data sequences is different. As described above with reference to FIG.11 masking the training data sequences differently can enable the machine learning model to learn to identify the useful information content from the training data sequence and “ignore” irrelevant noise introduced by masking.

[0151] At step 1208, the computer system can generate a plurality of training data subsequences based on the training data sequence or the one or more pluralities of masked training data sequences. Each training data subsequence can comprise a number of sequential data elements from the training data sequence or a number of sequential masked data elements from a masked training data sequences of the one or more pluralities of masked training data sequences. Each number of sequential data elements or number of sequential masked data elements corresponds to a window length of one or more window lengths. The computer system can perform step 1208 using techniques similar to those described above with reference to FIGs.7-8. For example, the computer system could use a sliding window segmentation process (with various window lengths) to segment the training data sequence into sets of subsequences of various lengths. Alternatively, the computer system could use a53 KILPATRICK TOWNSEND 792723011sliding window segmentation process to segment the masked training data sequences into sets of masked subsequences of various lengths.

[0152] At step 1210, the computer system can use the machine learning model to generate one or more pluralities of training output embeddings (e.g., output embeddings 1122-1126 from FIG.11) based on the one or more pluralities of masked training data subsequences. The machine learning model can have a structure or architecture similar to the architecture depicted in FIG.6, and the computer system can generate the one or more pluralities of training output embeddings in a manner similar to the method described above with reference to FIGs.6-10. In more detail and in some embodiments, the machine learning model can comprise a sequence encoder and one or more window encoders. In order to generate the one or more pluralities of training output embeddings based on the one or more pluralities of masked training data sequences and the plurality of training data subsequences, the computer system can generate one or more pluralities of sets of sequence partial embeddings by applying the one or more pluralities of masked training data sequences to the sequence encoder. The computer system can then generate one or more pluralities of sets of window partial embeddings by applying the plurality of training data subsequences and the one or more pluralities of sets of sequence partial embeddings to the one or more window encoders. The computer system can then combine each set of window partial embeddings to generate a corresponding training output embedding, thereby generating the one or more pluralities of training output embeddings.

[0153] At step 1212, the computer system can identify a plurality of pairs of training output embeddings from the one or more pluralities of training output embeddings. The computer system can further determine an embedding distance value for each pair of training output embeddings. In this way the computer system can determine a plurality of embedding distance values. Various metrics can be used as embedding distance values. For example, in some embodiments each embedding distance value can comprise a cosine similarity metric between a first vector representative of a first embedding in a corresponding pair of training embeddings and a second vector representative of a second embedding in the corresponding pair of training output embeddings. As described below, these embedding distance values can be used to determine a contrastive loss value, which can be used to train the machine learning model (e.g., using techniques such as stochastic gradient descent, backpropagation, etc.).54 KILPATRICK TOWNSEND 792723011

[0154] The computer system can identify various sets of pairs of training output in different ways. For example, the computer system could iterate through every possible pairing of training data outputs and computer an embedding distance value for each pairing. Alternatively, the computer system could randomly sample some predefined number of pairs, e.g., by randomly selecting a first training output embedding then randomly selecting a second training output embedding, then designating the first training output embedding and the second training output embedding as a pair. The computer system could repeat this process e.g., some number of times and avoid the selection of redundant pairs (e.g., via selection without replacement) until the computer system has identified the plurality of pairs of training output embeddings.

[0155] At step 1214, the computer system can determine a contrastive loss value based on the plurality of embedding distance values. In some embodiments, the contrastive loss value may comprise a combination of one or more loss values and may be referred to as a “combined loss value”. The computer system can use various formula or methods to determine the contrastive loss values. As an example, the computer system can use the following formula to determine a pairwise contrastive loss value corresponding to a pair of training output embeddings ^^௨௫and ^^௬௨:^〈ுೠ^,ு^ೠ〉 / ఛ ൌെlog^ ^^

[0156] Tocombine the contrastive loss values corresponding to various pairs of training output embeddings e.g., by summing or averaging all the pairwise contrastive loss values. In the formula above, 〈^^௫,^^௬〉 can comprise an embedd௫ ௬௨௨ing distance value between embeddings ^^௨and ^^௨, and ^^ a hyperparameter. The contrastive loss value can generally beto reward the machine learning model for generating similar output embeddings from masked training data sequences generated from the same training data sequence, and penalize the machine learning model to generate similar output embeddings from masked training data sequences generated from different training data sequences in the one or more training data sequences (e.g., a batch or minibatch of training data sequences sampled at step 1204).

[0157] At step 1216, the computer system can update a parameter set of the machine learning model based on the contrastive loss value. In this way the computer system can train the machine learning model based on the contrastive loss value (or “combined loss value”).55 KILPATRICK TOWNSEND 792723011As described above, the machine learning model can comprise an “encoder model” defined by an “encoder parameter set”, which can include, e.g., a sequence encoder parameter set corresponding to the sequence encoder and one or more window encoder parameter sets corresponding to the one or more window encoders. In some embodiments, the computer system can update the parameter set by updating the sequence encoder parameter set and the one or more window encoder parameter sets. The computer system can use any appropriate technique to update the parameter set, e.g., using backpropagation, stochastic gradient descent, etc.

[0158] At step 1218, the computer system can determine if a terminating condition has been met. As describe above, in some embodiments the terminating condition can comprise a defined number of training rounds, and the terminating condition can be met if a total number of training rounds performed equals or exceeds the defined number of training rounds. In other embodiments, the terminating condition can comprise a convergence condition. This terminating condition can be met if the set of model parameters converge, e.g., exhibit little to no change in consecutive training rounds. If the terminating condition has not been met, the computer system can return to step 1202 and repeat the iterative training process until the terminating condition has been met, e.g., by sampling a new batch of pairs of training data elements. Otherwise at step 1220 the computer system can complete the iterative training process. At this point, the machine learning model’s parameter can be fixed for use during inference methods, e.g., as described above with reference to FIGs.6-10.

[0159] It should be understood that various other training methods can be performed, and the example provided above is intended to be non-limiting. For example, the computer system could instead use supervised learning to concurrently train the machine learning model and a classification model. In more detail, in each iterative training round, the computer system could use the machine learning model to generate a plurality of training output embedding based on a plurality of training data sequences. The computer system could then generate a plurality of classifications based on the training output embedding using a classification model. The computer system can determine a classification output loss value by comparing the plurality of classifications to a plurality of labels corresponding to the plurality of training data sequences. The computer system can determine the classification output loss value using e.g., cross-entropy or a mean squared error calculated between the plurality of classifications and the plurality of labels corresponding to the plurality of training data sequences. The computer system can then update the machine learning model parameter56 KILPATRICK TOWNSEND 792723011set and the prediction model parameter set based on the classification output loss value using backpropagation and / or stochastic gradient descent. However, it should be understood that any known machine learning technique can be used to update either the machine learning model parameter set or the classification model parameter set. Moreover, as described above, it should be understood that the machine learning model and a classification model can be trained independently, e.g., the machine learning model can be trained using self-supervised learning (as described above), and then later a classification model can be trained using supervised learning using training output embeddings generated using the trained machine learning model. V. EXAMPLE OF AUTHENTICATION FOR ACCESSING A PROTECTED RESOURCE

[0160] As described further above, machine learning models according to embodiments can be used to improve the accuracy of access review systems. Such access review systems can include machine learning models used to dynamically evaluate requests for a resource and e.g., automatically approve or deny access requests based on the output of a prediction model. An access review system can include a fraud detection system and access requests for a computer resource or account (e.g., transactions over the Internet) can go through such a fraud detection system to determine whether those access requests (e.g., transaction) are authorized or rejected as being invalid (e.g., fraudulent). Thus, a resource security system may receive requests to access a resource. The resource security system may include an access server for determining an outcome for the access request based on access rules.

[0161] FIG.13 shows an exemplary resource security system 1300 for authorizing access to resources in accordance with some embodiments. The resource security system 1300 may be used to provide authorized users access to a resource (e.g., via authentication) while denying access to unauthorized users. In addition, the resource security system 1300 may be used to deny invalid access requests that appear to be legitimate access requests of authorized users. The resource security system 1300 may implement access rules to identify invalid access requests based on parameters of the access request. Such parameter may correspond to fields (nodes) of a data structure that is used to distinguish invalid (e.g., fraudulent) access requests from authentic access requests. Such access rules may be based, wholly or in part on the output of a machine learning model, e.g., a prediction model (or classification model) that produces predictions (or classifications) indicating the likelihood that an access request is57 KILPATRICK TOWNSEND 792723011valid or invalid. Such a machine learning model could generate predictions or classifications based on embeddings which could be generated using machine learning models and methods described herein, e.g., above with reference to FIGs.6 and 7.

[0162] The resource security system 1300 includes a resource computer 1310. The resource computer 1310 may control access to a physical resource 1318, such as a building or a lockbox, or an electronic resource 1316, such as a local computer account, digital files or documents, a network database, an email inbox, a payment account, or a website login. In some embodiments, the resource computer may be a webserver, an email server, or a server of an account issuer. The resource computer 1310 may receive an access request from a user 1340 via a user device 1350 (e.g., a computer or a mobile phone) of the user 1340. The resource computer 1310 may also receive the access request from the user 1340 via a request computer 1370 coupled with an access device 1360 (e.g., a keypad or a terminal). In some embodiments, the request computer 1370 may be a resource provider. For example, the request computer 1370 and the resource computer 1310 may be the same and the access request from the user 1340 can be generated directly at the resource computer 1310.

[0163] The access device 1360 and the user device 1350 may include a user input interface such as a keypad, a keyboard, a fingerprint reader, a retina scanner, any other type of biometric reader, a magnetic stripe reader, a chip card reader, a radio frequency identification reader, or a wireless or contactless communication interface, for example. The user 1340 may input authentication information into the access device 1360 or the user device 1350 to access the resource. Authentication information may also be provided by the access device 1360 and / or the user device 1350. The authentication information may include, for example, one or more data elements of a username, an account number, a token, a password, a personal identification number, a signature, a digital certificate, an email address, a phone number, a physical address, and a network address. The data elements may be labeled as corresponding to a particular field, e.g., that a particular data element is an email address. In response to receiving authentication information input by the user 1340, the user device 1350 or the request computer 1370 may send an access request, including authentication information, to the resource computer 1310 along with one or more parameters of the access request.

[0164] In one example, the user 1340 may enter one or more of an account number, a personal identification number, and password into the access device 1360, to request access to a physical resource (e.g., to open a locked security door in order to access a building or a58 KILPATRICK TOWNSEND 792723011lockbox) and the request computer 1370 may generate and send an access request to the resource computer 1310 to request access to the resource. In another example, the user 1340 may operate the user device 1350 to request that the resource computer 1310 provide access to the electronic resource 1316 (e.g., a website or a file) that is hosted by the resource computer 1310. In another example, the user device 1350 may send an access request (e.g., an email) to the resource computer 1310 (e.g., an email server) in order to provide data to the electronic resource 1316 (e.g., deliver the email to an inbox). In another example, the user 1340 may provide an account number and / or a personal identification number to an access device 1360 in order to request access to a resource (e.g., a payment account) for conducting a transaction.

[0165] In some embodiments, the resource computer 1310 may verify the authentication information of the access request based on information stored at the request computer 1370. In other embodiments, the request computer 1370 may verify the authentication information of the access request based on information stored at the resource computer 1310.

[0166] The resource computer 1310 may receive the request substantially in real-time (e.g., account for delays computer processing and electronic communication). Once the access request is received, the resource computer 1310 may determine parameters of the access request. In some embodiments, the parameters may be provided by the user device 1350 or the request computer 1370. For example, the parameters may include one or more of: a time that the access request was received, a day of the week that the access request was received, the source-location of the access request, the amount of resources requested, an identifier of the resource being request, an identifier of the user 1340, the access device 1360, the user device 1350, the request computer 1370, a location of the user 1340, the access device 1360, the user device 1350, the request computer 1370, an indication of when, where, or how the access request is received by the resource computer 1310, an indication of when, where, or how the access request is sent by the user 1340 or the user device 1350, an indication of the requested use of the electronic resource 1316 or the physical resource 1318, and an indication of the type, status, amount, or form of the resource being requested. In other embodiments, the request computer 1370 or the access server 1320 may determine the parameters of the access request.

[0167] The resource computer 1310 or the request computer 1370 may send the parameters of the access request to the access server 1320 in order to determine whether the access59 KILPATRICK TOWNSEND 792723011request is invalid. The access server 1320 may store one or more access rules 1322 for identifying an invalid access request. Each of the access rules 1322 may include one or more conditions corresponding to one or more parameters of the access request. The access server 1320 may determine an access request outcome indicating whether the access request should be accepted (e.g., access to the resource granted), rejected (e.g., access to the resource denied), or reviewed by comparing the access rules 1322 to the parameters of the access request as further described below. In some embodiments, instead of determining an access request outcome, the access server 1320 may determine an evaluation score based on outcomes of the access rules. The evaluation score may indicate the risk or likelihood of the access require being invalid. If the evaluation score indicates that the access request is likely to be invalid, then the access server 1320 may reject the access request. An access server may determine such an evaluation score using machine learning techniques, e.g., by using a machine learning model to generate an evaluation score based on an embedding, which in turn could be generated based on a data sequence comprising some of the parameters described above (e.g., the time that the access request was received, a day of the week that the access request was received, the source-location of the access request, etc.) using methods and machine learning models according to embodiments, e.g., as described above with reference to FIGs.6 and 7.

[0168] The access server 1320 may send the indication of the access request outcome to the resource computer 1310 (e.g., accept, reject, review, accept and review, or reject and review). In some embodiments, the access server 1320 may send the evaluation score to the resource computer 1310 instead. The resource computer 1310 may then grant or deny access to the resource based on the indication of the access request outcome or based on the evaluation score. The resource computer 1310 may also initiate a review process for the access request.

[0169] In some embodiments, the access server 1320 may be remotely accessed by an administrator for configuration. The access server 1320 may store data in a secure environment and implement user privileges and user role management for accessing different types of stored data. For example, user privileges may be set to enable users to perform one or more of the following operations: view logs of received access request, view logs of access request outcomes, enable or disable the execution of the access rules 1322, update or modify the access rules 1322, change certain access request outcomes. Different privileges may be set for different users.60 KILPATRICK TOWNSEND 792723011

[0170] The resource computer 1310 may store access request information for each access requests that it receives. The access request information may include authentication information and / or the parameters of each of the access requests. The access request information may also include an indication of the access request outcome for the access request, e.g., indicating whether the access request was actually valid or invalid. The resource computer 1310 may also store validity information corresponding to each access request. The validity information for an access request may be initially based on its access request outcome. The validity information may be updated based on whether the access request is reported to be invalid. In some embodiments, the access server 1320 or the request computer 1370 may store the access request information and the validity information. VI. COMPUTER SYSTEM

[0171] Any of the computer systems mentioned herein may utilize any suitable number of subsystems. Examples of such subsystems are shown in FIG.14 in computer system 1400. In some embodiments, a computer system includes a single computer apparatus, where the subsystems can be the components of the computer apparatus. In other embodiments, a computer system can include multiple computer apparatuses, each being a subsystem, with internal components. A computer system can include desktop and laptop computers, tablets, mobile phones and other mobile devices.

[0172] The subsystems shown in FIG.14 are interconnected via a system bus 1412. Additional subsystems such as a printer 1408, keyboard 1418, storage device(s) 1420, monitor 1424 (e.g., a display screen, such as an LED), which is coupled to display adapter 1414, and others are shown. Peripherals and input / output (I / O) devices, which couple to I / O controller 1402, can be connected to the computer system by any number of means known in the art such as input / output (I / O) port 1416 (e.g., USB, FireWire®). For example, I / O port 1416 or external interface 1422 (e.g., Ethernet, Wi-Fi, etc.) can be used to connect computer system 1400 to a wide area network such as the Internet, a mouse input device, or a scanner. The interconnection via system bus 1412 allows the central processor 1406 to communicate with each subsystem and to control the execution of a plurality of instructions from system memory 1404 or the storage device(s) 1420 (e.g., a fixed disk, such as a hard drive, or optical disk), as well as the exchange of information between subsystems. The system memory 1404 and / or the storage device(s) 1420 may embody a computer readable medium. Another subsystem is a data collection device 1410, such as a camera, microphone, accelerometer, and61 KILPATRICK TOWNSEND 792723011the like. Any of the data mentioned herein can be output from one component to another component and can be output to the user.

[0173] A computer system can include a plurality of the same components or subsystems, e.g., connected together by external interface 1422, by an internal interface, or via removable storage devices that can be connected and removed from one component to another component. In some embodiments, computer systems, subsystem, or apparatuses can communicate over a network. In such instances, one computer can be considered a client and another computer a server, where each can be part of a same computer system. A client and a server can each include multiple systems, subsystems, or components. In various embodiments, methods may involve various numbers of clients and / or servers, including at least 10, 20, 50, 100, 200, 500, 1,000, or 10,000 devices. Methods can include various numbers of communication messages between devices, including at least 100, 200, 500, 1,000, 10,000, 50,000, 100,000, 500,00, or one million communication messages. Such communications can involve at least 1 MB, 10 MB, 100 MB, 1 GB, 10 GB, or 100 GB of data.

[0174] Aspects of embodiments can be implemented in the form of control logic using hardware circuitry (e.g., an application specific integrated circuit or field programmable gate array) and / or using computer software stored in a memory with a generally programmable processor in a modular or integrated manner, and thus a processor can include memory storing software instructions that configure hardware circuitry, as well as an FPGA with configuration instructions or an ASIC. As used herein, a processor can include a single-core processor, multi-core processor on a same integrated chip, or multiple processing units on a single circuit board or networked, as well as dedicated hardware. Based on the disclosure and teachings provided herein, a person of ordinary skill in the art will know and appreciate other ways and / or methods to implement embodiments of the present disclosure using hardware and a combination of hardware and software.

[0175] Any of the software components or functions described in this application may be implemented as software code to be executed by a processor using any suitable computer language such as, for example, Java, C, C++, C#, Objective-C, Swift, or scripting language such as Perl or Python using, for example, conventional or object-oriented techniques. The software code may be stored as a series of instructions or commands on a computer readable medium for storage and / or transmission. A suitable non-transitory computer readable62 KILPATRICK TOWNSEND 792723011medium can include random access memory (RAM), a read only memory (ROM), a magnetic medium such as a hard-drive or a floppy disk, or an optical medium such as a compact disk (CD) or DVD (digital versatile disk) or Blu-ray disk, flash memory, and the like. The computer readable medium may be any combination of such devices. In addition, the order of operations may be re-arranged. A process can be terminated when its operations are completed but could have additional steps not included in a figure. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination may correspond to a return of the function to the calling function or the main function.

[0176] Such programs may also be encoded and transmitted using carrier signals adapted for transmission via wired, optical, and / or wireless networks conforming to a variety of protocols, including the Internet. As such, a computer readable medium may be created using a data signal encoded with such programs. Computer readable media encoded with the program code may be packaged with a compatible device (e.g., as firmware) or provided separately from other devices (e.g., via Internet download). Any such computer readable medium may reside on or within a single computer product (e.g., a hard drive, a CD, or an entire computer system), and may be present on or within different computer products within a system or network. A computer system may include a monitor, printer, or other suitable display for providing any of the results mentioned herein to a user.

[0177] Any of the methods described herein may be totally or partially performed with a computer system including one or more processors, which can be configured to perform the steps. Any operations performed with a processor may be performed in real-time. The term “real-time” may refer to computing operations or processes that are completed within a certain time constraint. As examples, a time constraint may be 30 seconds, 1 minute, 10 minutes, 30 minutes, 1 hour, 4 hours, 1 day, or 7 days. Thus, embodiments can be directed to computer systems configured to perform the steps of any of the methods described herein, potentially with different components performing a respective step or a respective group of steps. Although presented as numbered steps, steps of methods herein can be performed at a same time or at different times or in a different order. Additionally, portions of these steps may be used with portions of other steps from other methods. Also, all or portions of a step may be optional. Additionally, any of the steps of any of the methods can be performed with modules, units, circuits, or other means of a system for performing these steps.63 KILPATRICK TOWNSEND 792723011

[0178] The specific details of particular embodiments may be combined in any suitable manner without departing from the spirit and scope of embodiments of the disclosure. However, other embodiments of the disclosure may be directed to specific embodiments relating to each individual aspect, or specific combinations of these individual aspects.

[0179] The above description of example embodiments of the present disclosure has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the disclosure to the precise form described, and many modifications and variations are possible in light of the teaching above.

[0180] The claims may be drafted to exclude any element which may be optional. As such, this statement is intended to serve as antecedent basis for use of such exclusive terminology as “solely”, “only”, and the like in connection with the recitation of claim elements, or the use of a “negative” limitation.

[0181] All patents, patent applications, publications, and descriptions mentioned herein are incorporated by reference in their entirety for all purposes. None is admitted as prior art. Where a conflict exists between the instant application and a reference provided herein, the instant application shall dominate.64 KILPATRICK TOWNSEND 792723011

Claims

WHAT IS CLAIMED IS:

1. A method performed by a computer system for generating a classification corresponding to a data sequence using a machine learning model comprising a sequence encoder and one or more window encoders, the method comprising: retrieving the data sequence; generating one or more sets of data subsequences from the data sequence based on one or more window lengths, wherein each data subsequence comprises a number of sequential data elements from the data sequence, wherein each number of sequential data elements corresponds to a window length of one or more window lengths, and wherein each set of data subsequences corresponds to a window encoder of the one or more window encoders; generating a set of sequence partial embeddings by applying the data sequence to the sequence encoder; generating one or more sets of window partial embeddings by applying the one or more sets of data subsequences and the set of sequence partial embeddings to the one or more window encoders; combining the one or more sets of window partial embeddings, thereby generating an output embedding; and generating the classification based on the output embedding.

2. The method of claim 1, wherein the machine learning model comprises an encoder model, and wherein generating the classification based on the output embedding comprises applying the output embedding to a classification model, thereby generating the classification.

3. The method of claim 2, wherein the classification model comprises a neural network, a clustering model, a random forest model, a gradient-boosted random forest model, a decision tree, or a combination or ensemble thereof.

4. The method of claim 1, wherein the sequence encoder comprises an inner encoder comprising a self-attention layer and a feed-forward block comprising a feed- forward neural network, and wherein generating a set of sequence partial embeddings by applying the data sequence to the sequence encoder comprises:65 KILPATRICK TOWNSEND 792723011deriving one or more attention queries, one or more attention keys, and one or more attention values from the data sequence; generating a set of intermediate embeddings by applying the one or more attention queries, one or more attention keys, and one or more attention values to the self- attention layer; and generating the set of sequence partial embeddings by applying the set of intermediate embeddings to the feed-forward neural network.

5. The method of claim 1, wherein the data sequence comprises a plurality of data elements, and wherein the method further comprises: determining a plurality of time encodings corresponding to the plurality of data elements, wherein the plurality of time encodings indicates a temporal ordering of the plurality of data elements in the data sequence, wherein the set of sequence partial embeddings is generated based on the plurality of time encodings; and determining one or more pluralities of positional encodings corresponding to the one or more sets of data subsequences, wherein each plurality of positional encodings indicates a temporal ordering of a corresponding set of data subsequences, wherein the one or more sets of window partial embeddings are generated based on the one or more pluralities of positional encodings.

6. The method of claim 1, wherein the data sequence corresponds to a sequence of historical access requests, wherein the classification indicates that an access request is invalid based on the sequence of historical access requests, and wherein the method comprises causing the access request to be rejected based on the classification.

7. The method of claim 1, wherein: the data sequence is a time-series sequence of data comprising a plurality of data elements, and each data element in the time-series sequence of data includes an observed value of an object and a timestamp, and the classification corresponding to the data sequence comprises a classification of the object.

8. The method of claim 1, wherein: retrieving the data sequence comprises receiving the data sequence from a client computer; and66 KILPATRICK TOWNSEND 792723011the method further comprises transmitting the classification and / or the output embedding to the client computer.

9. The method of claim 1, wherein: the one or more sets of data subsequences comprise a first set of data subsequences and one or more additional sets of data subsequences; the one or more window encoders comprise a first window encoder and one or more additional window encoders; the one or more sets of window partial embeddings comprise a first set of window partial embeddings and one or more additional sets of window partial embeddings; and wherein generating the one or more sets of window partial embeddings comprises: generating the first set of window partial embeddings by applying the first set of data subsequences and the set of sequence partial embeddings to the first window encoder, and generating the one or more additional sets of window partial embeddings by applying the one or more additional sets of data subsequences and the first set of window partial embeddings to the one or more additional window encoders.

10. The method of claim 9, wherein the first window encoder comprises an inner encoder, a merge encoder, and a feed-forward block, and wherein generating the first set of window partial embeddings comprises: generating a first set of intermediate embeddings by applying the first set of data subsequences to the inner encoder; generating a second set of intermediate embeddings by applying the first set of intermediate embeddings and the set of sequence partial embeddings to the merge encoder; and generating the first set of window partial embeddings by applying the second set of intermediate embeddings to the feed-forward block.

11. The method of claim 10, wherein the inner encoder comprises a first self-attention layer, wherein the merge encoder comprises a second self-attention layer, and wherein the feed-forward block comprises a feed-forward neural network.67 KILPATRICK TOWNSEND 79272301112. The method of claim 11, wherein generating the first set of intermediate embeddings by applying the first set of data subsequences to the inner encoder comprises: deriving one or more attention queries, one or more attention keys, and one or more attention values from the first set of data subsequences; and generating the first set of intermediate embeddings by applying the one or more attention queries, the one or more attention keys, and the one or more attention values to the first self-attention layer.

13. The method of claim 11, wherein generating the second set of intermediate embeddings by applying the first set of intermediate embeddings and the set of sequence partial embeddings to the merge encoder comprises: deriving one or more attention queries and one or more attention keys from the set of sequence partial embeddings; deriving one or more attention values from the first set of intermediate embeddings; and generating the second set of intermediate embeddings by applying the one or more attention queries, the one or more attention keys, and the one or more attention values to the second self-attention layer.

14. The method of claim 9, wherein the one or more additional sets of data subsequences comprise a second set of data subsequences, wherein the one or more additional window encoders comprise a second window encoder, wherein the one or more additional sets of window partial embeddings comprise a second set of window partial embeddings, wherein the second window encoder comprises an inner encoder, a merge encoder, and a feed-forward block, and wherein generating the second set of window partial embeddings comprises: generating a first set of intermediate embeddings by applying the second set of data subsequences to the inner encoder; generating a second set of intermediate embeddings by applying the first set of intermediate embeddings and the first set of window partial embeddings to the merge encoder; and generating the second set of window partial embeddings by applying the second set of intermediate embeddings to the feed-forward block.68 KILPATRICK TOWNSEND 79272301115. A method performed by a computer system for training a machine learning model based on contrastive loss, the method comprising performing an iterative training process until a terminating condition has been met, the method comprising: sampling one or more training data sequences; generating, for each training data sequence, a plurality of masked training data sequences by masking the training data sequence, thereby generating one or more pluralities of masked training data sequences; generating, a plurality of training data subsequences based on the training data sequence or the one or more pluralities of masked training data sequences, wherein each training data subsequence comprises a number of sequential data elements from the training data sequence or a number of sequential masked data elements from a masked training data sequence of the one or more pluralities of masked training data sequences, wherein each number of sequential data elements or number of sequential masked data elements corresponds to a window length of one or more window lengths; generating one or more pluralities of training output embeddings based on the one or more pluralities of masked training data sequences and the plurality of training data subsequences using the machine learning model; identifying a plurality of pairs of training output embeddings from the one or more pluralities of training output embeddings; determining, for each pair of training output embeddings, an embedding distance value, thereby determining a plurality of embedding distance values; determining, based on the plurality of embedding distance values, a contrastive loss value; updating a parameter set of the machine learning model based on the contrastive loss value, thereby training the machine learning model; and repeating the iterative training process until the terminating condition has been met.

16. The method of claim 15, wherein the machine learning model comprises a sequence encoder and one or more window encoders, wherein generating the one or more pluralities of training output embeddings based on the one or more pluralities of masked training data sequences and the plurality of training data subsequences comprises:69 KILPATRICK TOWNSEND 792723011generating one or more pluralities of sets of sequence partial embeddings by applying the one or more pluralities of masked training data sequences to the sequence encoder; generating one or more pluralities of sets of window partial embeddings by applying the plurality of training data subsequences and the one or more pluralities of sets of sequence partial embeddings to the one or more window encoders; and combining each set of window partial embeddings to generate a corresponding training output embedding, thereby generating the one or more pluralities of training output embeddings.

17. The method of claim 16, wherein the parameter set comprises a sequence encoder parameter set corresponding to the sequence encoder and one or more window encoder parameter sets corresponding to the one or more window encoders, and wherein updating the parameter set comprises updating the sequence encoder parameter set and the one or more window encoder parameter sets.

18. The method of claim 15, wherein each embedding distance value comprises a cosine similarity metric between a first vector representative of a first embedding in a corresponding pair of training output embeddings and a second vector representative of a second embedding in the corresponding pair of training output embeddings.

19. The method of claim 15, wherein masking a training data sequence comprises replacing one or more data values in one or more data elements in the training data sequence with predetermined masking data values or random masking data values.

20. A computer system comprising: one or more processors; and a non-transitory computer readable medium coupled to the one or more processors, the non-transitory computer readable medium comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform the method of any of claims 1-19.70 KILPATRICK TOWNSEND 792723011

Images