Techniques for input classification and response using generative neural networks.

Abstract

Disclosed herein is a technique for detecting invalid web traffic. The method includes obtaining an input sequence. The method further includes inputting the input sequence to an encoder model configured to generate an embedding from the input sequence. The method includes generating the embedding as an output of the encoder model. The method includes inputting the embedding to a classifier model configured to predict a vector of probability values ​​corresponding to a plurality of classes using the embedding. A first class of the plurality of classes can be attributed to variable traffic and a second class of the plurality of classes can be attributed to repetitive traffic. The method includes predicting the vector of probability values ​​as an output of the classifier. The method includes determining a machine actor score from the vector of probability values. The method also includes outputting the machine actor score.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of and priority to U.S. Application No. 17 / 492,341, filed October 1, 2021, and U.S. Application No. 17 / 690,953, filed March 9, 2022, each of which is incorporated by reference in its entirety for all purposes.

[0002] Technical Field The present disclosure relates to methods and systems for detecting invalid web interactions, and more particularly to techniques using neural networks to generate embeddings from an input sequence of user interface interaction data, classify the embeddings to determine a machine actor score, and / or authorize or block web interactions based on the machine actor score. [Background technology]

[0003] background Invalid interactions with online resources, such as those resulting from bots or other automated systems, represent a persistent and significant challenge for maintaining control of data and limiting the expense of computational resources associated with responding to requests. Invalid interactions may be or include user interface interaction data generated by an automated system configured to emulate a valid human user by simulating input from a peripheral device, such as mouse cursor trace data, keystroke data, or voice input data. For example, a bot or other automated scraping system may be configured to repeatedly access publicly accessible databases, such as social media platforms, to generate inauthentic interaction statistics and / or access and store data. In another example, an automated system "bot" may be configured to indicate a social network platform user base of all users who are violating that platform's terms of service. In some cases, social media platforms may even be fooled when presenting pop-up tests to ensure that users are not scraping data from the platform. For example, automated systems are often configured to detect and bypass such tests by simulating human interactions with the pop-ups to cheat the tests. Summary of the Invention

[0004] overview In various embodiments, techniques are provided for analyzing interaction data and classifying said data according to characteristics of the interaction. In various further embodiments, the interaction data is collected from web interactions including input sessions, for example, in the form of mouse cursor movement and / or click sequence data. The systems and methods described herein include classifying the interaction data based on a machine actor score associated with a prediction of interaction effectiveness (e.g., as associated with a human user or a bot). The machine actor score may indicate a type and / or classification of the interaction as determined by the machine learning system and may be compared to one or more thresholds of acceptability to determine downstream actions to be taken with respect to the interaction. For example, predicted malicious interactions may be isolated and rejected from further interactions with the web-based item. Other entities that are not predicted to be significantly malicious, but are still suspicious, may receive a lesser response action, such as quarantining the interaction. Finally, predicted acceptable interactions may not be prevented by a response action.

[0005] An invalid interaction is an input that originates from an automated system (e.g., a robot or "bot") or is otherwise fraudulent. Invalid interactions cover a range of interaction types, including fraudulent automated interactions driven by bot activity, fraudulent interactions driven by real users (non-automated) such as incentivized browsing, and non-fraudulent automated activities such as web crawling. In contrast, valid interactions describe interactions generated by human users performing authorized or otherwise genuine activities and inputs. Some techniques disclosed herein include obtaining a cursor trail dataset that includes identification of the successive positions of the cursor to generate an embedding of the cursor trail dataset using a generative machine learning model (e.g., an encoder). The generative machine learning model may be used as part of a technique to distinguish human interactions from automated "bot" interactions. However, it should be understood that the techniques, methods, and systems may be applied to distinguish between different classes of interactions of various origins, including invalid human interactions (e.g., "click farming"). As used herein, the term "interaction" may be used interchangeably and refer to "traffic."

[0006] Attempts to solve these problems include using machine learning models and datasets trained using supervised learning to detect and analyze invalid inputs from bots. Machine learning models trained with supervised learning and training datasets do not require direct manual rule-based refinement and respond more efficiently to adaptations by bots. Thus, machine learning models are well-suited to address the “arms race” in detecting bot behavior. However, machine learning models trained with supervised learning and training datasets may require expensive and cumbersome data labeling for learning and operation. Machine learning models trained with supervised learning and training dataset models cannot efficiently refine and implement detection models when labels are difficult to generate or unavailable. For example, machine learning models trained with supervised learning and training dataset models typically cannot learn to detect inputs from bots of relatively new configurations with the required accuracy unless training labels exist to train the supervised model, putting bot detectors at a disadvantage when bots are refined with newly developed techniques and input patterns.

[0007] To address the invalid traffic problem while removing the resource-intensive requirement for labeled training data, cursor trace data for analysis is grouped and embedded according to one or more generative machine learning models, such as a generative neural network (e.g., a variational autoencoder composed of a recurrent neural network or a transformer model) that can generate unique input embeddings (compact vector representations of the inputs). The resulting embedded inputs can be analyzed using approaches such as an embedding clusterer subsystem (e.g., K-means) to cluster the embeddings, and a cluster membership classifier (which may include standard classifiers such as logistic regression). The generative machine learning models can process the input sequence data (e.g., identify the presence, time, duration, and / or location of cursor movements, clicks, etc.), generate unique embeddings of the inputs, classify the embeddings via a cluster membership classifier (trained on the clustering of embeddings learned by the embedding clusterer subsystem), and predict a machine actor score that measures the likelihood that the input sequence is from a bot (e.g., as opposed to a human user).

[0008] In rule-based approaches, the system may use static rules to predict that a user is a bot. For example, if the input cursor speed exceeds a threshold, the bot evaluates the response from the system to reverse engineer the rule, and then modifies the bot's interactions (e.g., by slowing down) to try to avoid detection. Adapting a supervised detection model typically requires laborious reverse engineering to capture the bot's characteristic behavior.

[0009] For machine learning models trained using supervised learning and training datasets, the requirement to train the machine learning model with labeled training data is a resource-intensive process. For example, training a machine learning model using supervised learning and a training dataset to detect inputs by new bots may require generating and inputting an advanced training dataset into the model. Regularly generating such a form of sufficiently representative training dataset with human annotation for each training sample is either infeasible or too noisy, especially when little is known about bots. As long as a machine learning model trained with supervised learning has not yet been trained to detect inputs from new bots, the new bots may continue to fool the model. By implementing an unsupervised learning model rather than a supervised learning model, the detection system can learn to identify new types of invalid traffic in real time with substantially fewer resources.

[0010] In contrast, the techniques disclosed herein utilize machine learning models trained with unsupervised learning. The trained models can be configured to be trained online or on frequent cycles with more recent data, making them robust to data drift from an evolving bot ecosystem and corresponding evasion attempts. The machine learning models trained with unsupervised learning and training datasets can include an autoencoder (e.g., a variational autoencoder including a recurrent neural network or a transformer) where the encoder is configured to generate embeddings (compact vector representations of the inputs), an embedding clusterer subsystem (e.g., K-means) that clusters the embeddings, and a cluster membership classifier (e.g., logistic regression) trained with the clustering learned by the embedding clusterer subsystem and configured to use the embeddings to predict cluster membership likelihoods that can be used to derive a machine actor score that measures the likelihood that the original input corresponds to a bot. As described herein, the unsupervised machine learning model can be a machine learning mode, such as a generative model, trained using unsupervised machine learning techniques.

[0011] In connection with obtaining raw cursor trail data for downstream analysis, the use of unsupervised machine learning models provides advantages over traditional approaches (e.g., rule-based and supervised learning approaches), including, but not limited to, improved accuracy, robustness, and responsiveness of grouping cursor trail data by using embeddings of cursor trail data generated with a generative machine learning model. Rather than utilizing raw cursor trail data collected from an application or attempting to extract metrics directly from the raw cursor trail data, the use of an unsupervised generative machine learning model can more efficiently group and embed cursor trail data relevant to the input being analyzed. Because generative machine learning models are unsupervised, they can learn from the latest bot interactions without requiring the generation of a labeled training dataset as required by supervised machine learning models.

[0012] The generative machine learning model may be an encoder trained in combination with a decoder as part of a unified autoencoder model configured to receive embeddings of cursor data and generate a predicted reconstruction of the cursor data. The encoder and decoder may be trained using a loss function that penalizes the error between the reconstruction of the input generated by the decoder and the input originally received and processed by the encoder. This training approach facilitates generating a highly informative and efficient representation of the cursor data.

[0013] The above, together with other features and embodiments, will become more apparent with reference to the following specification, claims, and accompanying drawings. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 is a diagram of a bot interaction detection system incorporating a data processor / formatter, an autoencoder subsystem, and a machine actor detection subsystem, according to an embodiment described herein. [Diagram 2] FIG. 2 is a diagram of a training system configured for unsupervised training of an autoencoder of a bot traffic detection system, according to an embodiment described herein. [Diagram 3] FIG. 2 is a diagram of a training system for an embedded clusterer subsystem and a cluster membership classifier (both components of the machine actor detection subsystem) according to an embodiment described herein. [Figure 4] 1 is a schematic diagram of an inference stage method for detecting bot traffic from mouse cursor trace data, according to an embodiment described herein. [Diagram 5] 5 is a block flow diagram of an inference phase method 500 for detecting bot traffic from mouse cursor trace data, according to an embodiment described herein. [Figure 6] FIG. 1 illustrates an exemplary interface for displaying aspects of mouse trace data analysis and data analysis reports. [Figure 7] FIG. 1 is a diagram of a distributed system for implementing certain embodiments described herein. [Figure 8] FIG. 1 is a block diagram of a cloud-based system environment according to an embodiment described herein. [Figure 9] FIG. 1 is a block diagram of an example computer system that can be used to implement certain embodiments described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Detailed Description In the following description, for purposes of explanation, specific details are set forth in order to provide a thorough understanding of certain embodiments. It will be apparent, however, that various embodiments may be practiced without these specific details. The figures and description are not intended to be limiting. The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any embodiment or design described herein as "exemplary" or "example" is not necessarily to be construed as preferred or advantageous over other embodiments or designs.

[0016] A trainable browser interaction processing system is provided that can use a generative machine learning model to store, process, and / or embed cursor trace datasets into an embedded representation. The resulting embedded representation can be used downstream to validate the interaction data. The interaction data can be generated as part of an active application, such as a web browser that detects browser input. The trainable interaction embedding, analysis and processing system may be provided as a standalone device, a cloud-accessible service, or the like. In each of these forms, the system is capable of accessing and / or receiving data generated from interactions with a user interface, using the data to determine sequences characterizing sequential interactions, generating embeddings of the sequences using the trained encoder model, classifying the embeddings using an appropriate cluster membership classifier (e.g., logistic regression) trained on a clustering of the embeddings learned by an embedding clusterer subsystem (e.g., K-means), using the resulting cluster membership probabilities to determine a machine actor score indicating the likelihood that the input is a bot, determining whether an access blocking condition is met based on the machine actor score (e.g., comparing the machine actor score to a threshold score), and outputting a result indicating whether the access blocking condition is determined to be met. In some cases, outputting the result includes outputting the result to the web server or particular code of the web server so that the web server or particular code can determine how to respond to a content request received from a device associated with the user (e.g., returning the requested content if the access blocking condition is determined to be not met and not returning the requested content if the access blocking condition is determined to be met) and / or so that the web server or particular code can push updated web content that lacks the particular content if the access blocking condition is met.

[0017] To that end, the techniques, methods, and systems presented herein implement unsupervised learning to overcome the limitations of obtaining comprehensive cursor trail data for analysis. One model system may be an autoencoder, which may be an unsupervised neural network model. An autoencoder may learn a latent representation of the input data, called an embedding, which may be a reduced-dimensional version of the input data that retains only the most salient features. An autoencoder model may be trained using unsupervised learning, as described by the methods and embodiments presented herein. Unsupervised learning indicates that the model does not use labeled data to guide model training. For example, the unsupervised training framework does not provide class labels indicating a particular interaction type (e.g., bot or human) during training, as is done in supervised learning.

[0018] Training an Encoder to Embed Cursor Trace Dataset An autoencoder may include two components: an encoder and a decoder. The encoder may include a neural network capable of learning to map inputs to embeddings (compressed latent reduced-dimensional vector representations) in an embedding space (a vector space containing all possible embeddings). Distances in the embedding space between embeddings of user inputs are particularly important, since proximity in the embedding space may indicate similarity between the embeddings and, therefore, between the original user inputs. In various embodiments, a set of one or more computed distances between embeddings may be used by an embedding clusterer subsystem to group similar embeddings together, which serves as the basis for the final cluster membership classification and generation of machine actor scores, as described below.

[0019] The decoder portion of an autoencoder may include a neural network that can be trained to reconstruct the uncompressed / unembedded input from the embedding output by the encoder. Different types of neural network models are available to construct the encoder and decoder. The choice of the type of neural network model may depend on the intended use of the model. For example, when processing image data, a feedforward neural network (FNN) or a convolutional neural network (CNN) may be selected and used. When processing sequence data, a recurrent neural network (RNN) or a transformer, among other neural network model types, may be selected and used. FNNs, CNNs, and RNNs may be used in combination to encode and decode multimodal data.

[0020] Autoencoders may be trained "end-to-end", where the encoder and decoder are jointly trained. Joint training of the encoder and decoder may involve defining a reconstruction error, or "loss", that serves as a loss function for the autoencoder model, comparing the input with the decoder's reconstruction of the input from the embedding and penalizing the difference (e.g., via squared loss). Training allows the optimization of model parameters for both the encoder and decoder using stochastic gradient descent (or other gradient-based optimization algorithms such as Adam). The decoder plays an important role during training to formulate the autoencoder's loss function in terms of the reconstruction error, but does not play a role in inference, where the embedding output from the encoder serves as input to a cluster membership classifier, described below.

[0021] A variational autoencoder (VAE) is a specific architectural embodiment of an autoencoder model. In addition to the standard autoencoder formulation described above, a VAE is an autoencoder that learns to output an embedding by sampling the embedding from a Gaussian probability distribution whose parameters are learned and output by the encoder. In this way, the parameters of the sampled Gaussian distribution are a learned function of the input data. Because the embedding is sampled from a probability distribution, the VAE produces non-deterministic outputs for sequence inputs. The loss formulation of a VAE differs from that of a standard autoencoder in that, in addition to the reconstruction error term described above, the VAE includes an additional term in the loss function that encourages the learned Gaussian distribution over the embedding space to resemble a "standard" Gaussian distribution with zero mean and unit variance.

[0022] Recurrent neural networks may include, but are not limited to, gradient recurrent units (GRUs) or long short-term memory (LSTM) units. Long short-term memory models include at least one long short-term memory cell, which may contain multiple interaction layers that can hold parts of information for long or short periods of time during the task and protect the gradients inside the cell from harmful changes during training. For example, an LSTM cell may include three gates (input, forget, output) and a cell unit. The gates may be configured to use sigmoid activation, and the input and cell state may be transformed with a hyperbolic tangent, or tanh, function.

[0023] Gradient recurrent unit cells also regulate the flow of information inside the unit and do so without having a separate storage cell. The activation of a GRU cell at time t can involve linear interpolation between the previous activation and the candidate activation, and an update gate determines how much the unit updates the activation. This procedure of determining a linear sum based on the existing state and the newly computed state is similar to that of an LSTM cell. However, GRU cells have no mechanism to control how much of the state they expose, instead exposing the entire state each time. Although the preceding discussion focused on LSTM and GRU cells, the encoder and decoder components of an autoencoder are not limited to such systems. For example, the RNN architecture may be, but is not limited to, fully recurrent, bidirectional, recurrent, etc.

[0024] Another example of a model that can be used to build the encoder and decoder of an autoencoder may be a Transformer model. A Transformer is a deep learning model that can be used to process and model sequential data for various tasks, e.g., machine translation, or modeling mouse cursor trace data for invalid interaction detection. A Transformer is typically composed of an encoder and a decoder. The encoder of a Transformer converts data into an intermediate output sequence called an encoding, and the decoder decodes it into an output sequence. The encoder and decoder incorporate multiple encoder and decoder layers, respectively. The output of the encoder contains latent information from the input sequence for the decoder to generate the output sequence.

[0025] Each encoding layer of a transform encoder takes as input a sequence of tokens, where a token contains a small meaningful unit of the input. In the context of bot interaction detection, an input sequence could be a vector representation of input data, like the position of a mouse cursor at a particular time, or it could correspond to a particular interaction state (describes one or more state conditions around the interaction, like "clicked" or "hovered"). Each encoding layer takes a list of vector representations of the input sequence and passes it to a self-attention layer and subsequently to a feedforward neural network. The self-attention layer attempts to isolate the most salient parts of the input sequence. The output of the feedforward neural network is passed to the subsequent encoder layer, and the output of the encoder's final encoding layer can represent the encoder's final output encoding. These encodings are token-level encodings, as they are output by the encoder for each token, as opposed to sequence-level embeddings, which represent the user's input sequence as a whole and, by definition, include the output of an RNN-based encoder. To generate such a sequence-level embedding from a transform encoder, the token-level encodings need to be aggregated, which can be achieved, for example, by summing or averaging all token-level encodings to obtain a sequence-level embedding. In order for this sequence-level embedding to be a learning function of the input, it must be characterized by a loss function, which can be achieved, for example, by passing it as an input to a decoder and concatenating it with the input of a standard decoder. Similar to the RNN formulation, the transform decoder is also trained to reconstruct the original input sequence from its input.

[0026] In order for an autoencoder to process user session data that includes various user input sequences (e.g., mouse cursor movements, clicks, etc.), the input data must first be formatted, normalized, and otherwise transformed. A specific embodiment of this data preprocessing is described for the case where the autoencoder is included in an RNN encoder and decoder, but it should be noted that other (similar) schemes are possible and may require modifications when using other models (e.g., transformer-based autoencoders).

[0027] Data pre-processing involves first extracting only a specific subset of sequential type user input signals (e.g., mouse cursor movement and click sequences) from the historical web interactions including user session data, among many other signals (possibly of non-sequential type). The elements of these movement and click sequences are tuples (x,y,c) of the horizontal (x) and vertical (y) position coordinates of the mouse on the web page when the web browser fired the event, and the timestamp offset c when the event was fired relative to the start of the user session. These sequences are first ordered by timestamp c, and duplicate (or otherwise invalid) events are removed. The sequences are then converted into sequences of position-time deltas (dx,dy,dc) by taking the difference between adjacent elements in the original sequence.

[0028] All mouse event types (movement, click, etc.) are assumed to be mutually exclusive at a given position (x,y,c). A mouse state can be defined as the state of the mouse when a given event is fired, the number of possible states is the number of event types tracked, and there is also a "lift" state that corresponds to a large difference in timestamps between consecutive events, which can be controlled, for example, by a numerical threshold. In this way, the sequences of different modalities (movement and click, etc.) are then collated into a unified sequence of vectors, where the first three elements of each vector contain the position and time delta (dx,dy,dc) of the current event, and the later elements contain one-hot vectors that specify the mouse state. More specifically, all positions in the one-hot part of the vector have value 0 (inactive) except for the position representing the current mouse state, which has value 1 (active). This vector representation facilitates processing by downstream machine learning models, in particular autoencoders trained to generate embeddings of each user session input.

[0029] Once the data is cast into the above-mentioned format, further processing may include normalization, such as column-wise scaling of position and time values, and truncation of sequences with uniform length restrictions, to improve the efficiency of the autoencoder. In some embodiments, for some or all of the downstream training stages, stratified samples of user session inputs are used based on whether a given session was flagged by other fraud detection methodologies rather than the methods described herein. Using stratified samples can provide the training process with a better distribution of fraudulent and non-fraudulent interactions, whereas unstratified samples would bias the training data heavily towards non-fraudulent interactions.

[0030] Generating embedding representations of cursor trace datasets Once the encoder, or generative machine learning model utilized, has been trained using unsupervised learning as described above, the encoder is utilized to generate an embedded representation of the cursor trail data that is analyzed in a downstream process to determine a classification of human or non-human interaction. That is, the trained encoder receives as input the formatted / processed embedded cursor trail data. The encoder uses the input formatted / processed cursor trail data to generate an embedded representation that preserves the non-fluid, non-human cursor patterns present in the original raw cursor trail data and the corresponding formatted / processed data. In determining the classification of the cursor trail data, it is important to preserve the non-fluid, non-human cursor patterns present so that downstream analysis systems can properly classify the non-human embeddings.

[0031] Aspects of the cursor trail data may be represented in an embedded representation of the cursor trail data generated by the encoder, e.g., an identification of sequential positions of the cursor. An indication of a binary input, such as a button press, may also be stored in the embedded representation. The binary input may be included in the embedded representation along with the corresponding sequential positions of the cursor at the time the binary input changes (i.e., when the button is pressed or released). By processing the clusters of data stored in the embedded representation, indications / aspects in the embedding may be identified.

[0032] Analysis of embeddings of cursor trace dataset A classifier may be trained to predict machine actor scores from the clustered embeddings. For example, a classifier may be trained to predict machine actor scores based on embeddings generated by an autoencoder after the embeddings have been clustered according to the embedding clusterer subsystem. The embedding clusterer subsystem may be a series of steps for clustering embeddings output by the encoder part of the autoencoder and may be separate from the classifier (e.g., the clusterer may be a known clustering algorithm such as K-means or DBSCAN). During training, the embeddings are clustered by the embedding clusterer subsystem, which assigns a cluster label to each embedding. A training dataset consisting of the embeddings along with the cluster labels learned by the embedding clusterer subsystem may then be used to train a classifier to predict cluster membership. The classifier may use a machine learning model to generate the predicted cluster labels, which may include, for example, a probabilistic classification model (such as logistic regression). In other embodiments, the clusterer and classifier may comprise a single model, as in the case of a mixture model, which learns multi-modal distributions over the input embeddings whose modes can be considered as clusters, and individual embeddings can be assigned cluster labels by associating them with the mode that most likely produced them.

[0033] The classifier may output a probability vector for each input, with each element of the vector being the probability that the input belongs to the corresponding cluster. During inference, the embedding clusterer subsystem may no longer be used. New inputs may be passed to the encoder to create embeddings, which may be passed to the classifier, which may make cluster membership predictions in the form of probability vectors. Machine actor scores for the inputs may then be calculated from the probability vectors, the details of which will be explained further below. The fact that clustering may not be performed during inference applies both to embodiments that use separate clusterers and classifiers, as well as to embodiments that use models that combine clustering and classification functions, such as mixture models. In the latter case, the mixture model may persist during inference, but only use the prediction function that generates the probability vectors for the inputs.

[0034] As mentioned above, the prediction may be based on a vector of probability values ​​output by the cluster membership classifier subsystem, including an indication of the confidence (or probability) that a particular encoded input corresponds to each of a set of clusters (e.g., 25% confidence that the encoded input corresponds to a type of user associated with cluster A, 40% confidence that the encoded input corresponds to a type of user associated with cluster B, and 35% confidence that the encoded input corresponds to a type of user associated with cluster C). The encoded input may be assigned to whichever cluster is associated with the highest assigned confidence in this embodiment. The machine actor score may be further generated by first filtering the determined confidences to exclude confidences associated with the "largest" clusters (where more encoded input data sets were assigned compared to other clusters or where the variance of the cluster is greater than other clusters). In other embodiments, a density measure for each cluster may be calculated and confidences for clusters where the density measure is below a predefined threshold (which itself may be cluster dependent, e.g. the average density of the cluster). The machine actor score may then be (or be generated at least in part based on) the remaining confidences. For example, a machine actor score can be defined to be the maximum confidence of the remaining confidences.

[0035] In various embodiments, a cluster within the overall clustering learned by the clusterer describes a grouping of similarly patterned inputs. For example, a cluster may describe a grouping of mouse cursor trace embeddings that indicate that the original mouse cursor trace inputs exhibit similar patterns. Such groupings of mouse cursor traces may be analyzed to detect suspicious behavior to aid in determining whether the inputs are invalid. For example, mouse cursor traces of similar patterns may be analyzed to determine that the mouse cursor traces exhibit rigid, repetitive behavior, indicating suspicious input behavior, and should be determined to be an invalid interaction.

[0036] In some cases (e.g., instead of using clustering embeddings), a signature can be generated for each of one or more clusters. The signature can include (e.g.) one or more characteristics of the cursor trace represented by the embedding in the cluster. The one or more characteristics can include (e.g.) an average or median characteristic across the cluster. In some cases, generating the signature includes identifying a subset of characteristics where the standard deviation, variance or range of the characteristics across the members of the cluster is below a predefined threshold, and then defining a signature to represent the values ​​of the subset of characteristics. The signature can be used by a human to understand what type of cursor trace activity is represented by a cluster. Additionally or alternatively, the signature can be used to facilitate assigning a given cursor trace data element to a cluster (e.g., by identifying the cluster whose cluster signature most closely matches the embedding of the cursor trace data element).

[0037] Clustering and classification of embeddings may be accomplished by separate embedding clusterer subsystem and classifier algorithms. For example, an invalid interaction detection system may use separate clusterers (e.g., a system including an embedding clusterer subsystem) and classifiers to perform the embodiments described herein. The clusterer may be stored, implemented, and utilized in a different system than the classifier, and the only interaction between the entities may be the sending and receiving of cluster labels assigned to the input embeddings. In various alternative embodiments, the clusterer and classifier may be part of a single model, such as a mixture model.

[0038] A mixture model is a probabilistic model that employs a mixture distribution to represent the existence of subpopulations in an overall population. A mixture distribution is a probability distribution that is composed of a combination of subdistributions called components. For example, a mixture distribution may be a linear combination of the subdistributions, where each term in the sum involves the product of the prior probability of the component (coefficient) and the posterior probability of the component given the input data. There are various types of mixture models depending on the distribution type chosen for the subdistributions, which can be (but are not limited to) Gaussian, Dirichlet, categorical, etc. Rather than explicitly assigning input data to a subdistribution, a mixture model allows for the construction of a probability vector whose ith element is the probability that the input "comes from" the ith component. More precisely, the posterior probability of each component given the input data becomes an element of this probability vector, and the predicted value of the most likely component given the input data becomes the maximum value of the probability vector. From this, it is easy to see that a mixture model can simultaneously play the role of a clusterer and a cluster membership classifier. If the components of the overall distribution are interpreted as clusters, then the calculation of the probability vector above can be interpreted as a cluster membership classification. In this way, the mixture model condenses the roles of the embedding clusterer and the embedding cluster membership classifier into a single model. As an exemplary embodiment, during training, a Gaussian mixture model (GMM) fits input embeddings with Gaussian mixture distributions that are contained in a combination of individual Gaussian distributions. During inference, the GMM builds a probability vector for new input embeddings that is contained in the posterior probability of each Gaussian subdistribution given the input embedding.

[0039] FIG. 1 is a diagram of an exemplary bot traffic detection system 100 (also referred to as a "detection system") according to embodiments described herein. The bot traffic detection system 100 utilizes a data processor / formatter 104, an autoencoder subsystem 107, and a machine actor detection subsystem 110. The machine actor detection subsystem 110 may include multiple subsystems therein, including a model training subsystem, an inference subsystem. The training subsystem 111 may be a subsystem for training model components. The training subsystem may be used separately from other subsystems that perform inference-related operations, such as the inference subsystem. The inference subsystem may be a subsystem for performing inference-related operations, such as generating machine actor scores / decisions from input sequence embeddings. The bot interaction detection system 100 depicted in FIG. 1 may be recognized as a general "separate model" paradigm, where the embedding clusterer subsystem and the cluster membership classifier subsystem 114 are separate entities. In various embodiments, the cluster membership classifier subsystem 114 may also be part of the inference subsystem as a means of generating cluster membership probability vectors for input sequence embeddings from which individual machine actor scores may be derived. In various embodiments, the embedded clusterer subsystem and the cluster membership classifier subsystem 114 may be implemented as a single model / component within the training subsystem (the "single model" paradigm), such as in embodiments in which a mixture model is employed for clustering and cluster membership classification. The detection system 100 is configured to receive, format, and otherwise process user session input using a data processor / formatter 104.

[0040] The input data may be generated by an entity 101, such as a human or a bot, simulating data generated by a data input component of the interaction system 102. For example, the entity 101 may input interactions into a user web interaction system, such as the interaction system 102. The interaction data generated by the interaction system 102 may be transferred to the data processor / formatter 104 as interaction data 103 by recording mouse cursor data from interactions made by the entity 101, and combined with mouse state data for interacting with a browser or console environment. The state data may include designations of different types of mutually exclusive interaction states, such as "click", "unclick", "move", "hover", etc., that occur in real time. The data preparation may further include cleaning and de-densification, for example, by ordering the interaction data 103 according to timestamps and removing duplicate events. In various embodiments, the interaction data 103 is then forwarded to a data processor / formatter 104 in the invalid interaction detection system 100. The data processor / formatter 104 can analyze the input data 103 and process / format the interaction data 103 into another state usable by other subsystems, such as the autoencoder subsystem 107. The resulting processed / formatted sequence data 105 may then be sent to the autoencoder system 107, which may recognize the format of the processed / formatted data sequence data 105 and utilize it according to embodiments described herein. In various embodiments, the data processor / formatter 104 is configured to receive data prepared by an associated system, such as a data aggregation system, in which the detection system 100 is used as part of a security system.

[0041] In some embodiments, the input data includes historical interaction data collected from interactions over a previous and / or current period of time. For example, the interaction data may be collected over a period of time and stored in a data repository. The interaction data may be collected over a period of time and transmitted from the repository to another system as historical interaction data. For example, instead of capturing and forwarding the interaction data in real time, the interaction data may be captured in real time and collected in a repository. At a later point in time, all of the collected interaction data may be published in a newly collected data set that includes the historical interaction data.

[0042] The invalid interaction detection system, or "detection system" 100, can perform or cause to be performed one or more actions using the interaction data 103, for example, by using the data processor / formatter 104, the autoencoder subsystem 107, or the machine actor detection subsystem 110, the cluster membership classifier subsystem 114, the machine actor score generator 116, and the thresholding subsystem 118. Each of these subsystems is a component of the machine actor detection subsystem 110. The cluster membership classifier subsystem 114 can also be part of the training system, by virtue of being trained on the cluster labels to make cluster membership predictions in the inference phase. In this manner, the detection system 100 can process the interaction data 103 using the machine actor detection subsystem 110 and output interaction decisions and interaction authorizations, such as by the output component 120. For example, the output component 120 may be or may include a virtual network interface card or a physical router device that can output data to a network 160, such as to an associated bot interaction detection system or interaction control system.

[0043] In various embodiments, the detection system 100 may be further configured to forward requests indicated by the interaction data 103 for human interaction and block requests for bot interaction. Additionally, the output component 120 may be communicatively coupled to one or more display devices to enable a visualization of the output of one or more subsystems of the detection system 100 to be presented on a display screen of the display device. For example, a reduced dimensional projection (e.g., a cluster plot) of the classifier output may be visually presented to the user, as well as a comparison of the output of bot interaction detection using alternative modalities as a validation technique.

[0044] In various embodiments, the detection system 100 is a specialized computing system that can be used to process large amounts of data by using a large number of computer processing cycles for normal operation. The number of devices depicted in FIG. 1 is provided for illustrative purposes. A different number of devices may be used. For example, although each device, server, and system in FIG. 1 is shown as a single device, multiple devices may instead be used to facilitate the operation of any entity depicted therein.

[0045] In various embodiments, the processing performed by the detection system 100 is implemented by a pipeline of components or subsystems, including a data processor / formatter 104, an autoencoder subsystem 107, and a machine actor detection subsystem 110, and may include an output component 120. The above subsystems may be implemented in software (e.g., using code, programs, or instructions executable by one or more processors or cores), hardware, or a combination of hardware and software. In certain implementations, multiple subsystems described herein may be combined into a single subsystem. Additionally or alternatively, in some implementations, functionality described herein performed by a particular subsystem may be implemented by multiple separate subsystems.

[0046] The data processor / formatter 104 includes hardware and software configured to receive the interaction data 103. In some cases, the data processor / formatter 104 may be part of the detection system 100. In some other cases, the data processor / formatter 104 may be separate from and communicatively coupled to the detection system 100. The processor / formatter 104 may include data input hardware and software communicatively coupled to, for example, a data receiving / processing / formatting / conversion system. In an illustrative example, the processor / formatter 104 may include an application programming interface (API) that communicates with a sessionizer system or other subsystems configured to interface with a computer system over a network. In another illustrative example, the processor / formatter 104 may include a personal computer configured to receive a set of interaction data 103 for processing by the subsystems of the detection system 100.

[0047] The interaction data 103 will be parsed by the data processor / formatter 104 to generate input sequences of processed / formatted sequence data 105 that may be sent to the autoencoder subsystem 107 and associated with distinct user sessions. The input sequences may include, but are not limited to, a finite length vector of mouse cursor position data and interaction state data for a single session. To define a session, start and end events may be recorded or identified for the session such that the input sequence is delimited before being stored and / or sent to subsequent subsystems of the detection system 100. The input sequences may be matched to input sequences from content interactions based on distinct user sessions that may be associated with the presentation of content on a user device (e.g., a tablet or display). The sessions may be further divided into periods during which interactions occur, and the input sequences may be grouped into these periods to form bounded time sequences of input data. In an illustrative example, a distinct session corresponds to a period during which a web page is loaded and displayed on a display, during which mouse cursor event data may be collected.

[0048] Upon generating the input sequence of processed / formatted sequence data 105, the autoencoder subsystem 107 is configured to access or receive the input sequence. For example, the input sequence may be received from a data storage utility in communication with the detection system 100 to generate an output embedding. The output embedding may be an embedded version of the cursor trail data of the input sequence. For example, the embedding may be a low-dimensional representation of the input cursor trail. In some examples, the embedded representation is a fixed-length multi-dimensional vector that represents one or more latent dimensions of the sequential positions of the cursor. The fixed-length multi-dimensional vector represents one or more latent dimensions, as described in more detail with reference to FIG. 4. As part of its processing, the autoencoder subsystem 107 uses the encoder model 108 to generate the output embedding.

[0049] In various embodiments, the autoencoder system employs a variational autoencoder (VAE) architecture to learn the output embeddings by sampling the output embeddings from a Gaussian probability distribution whose parameters are output by the encoder. The autoencoder subsystem 107 may be trained using historical interaction data. For example, a dataset may be provided as part of the interaction data 103, provided as processed / formatted sequence data 105, or accessed by the detection system 100 via a database in communication with the detection system 100, the dataset including historical interaction data of human interactions and bot interactions. The interaction data may include data already classified by a detection system employing alternative modalities. In this manner, the training data may be stratified to address biases introduced in training by the relatively low frequency of bot interaction requests that may occur in typical interaction patterns. Training of the model of the autoencoder subsystem 107 is described in more detail with reference to FIG. 2.

[0050] In various embodiments, the interaction classification is performed by the machine actor detection subsystem 110. The machine actor detection subsystem 110 has access to or receives output embeddings generated by the autoencoder subsystem 107. The embeddings received by the machine actor detection subsystem 110 may include fixed length vectors of multi-dimensional encoder outputs. To that end, in various embodiments, the machine actor detection subsystem 110 may include entities 114-118 depicted in FIG. 1. The machine actor detection subsystem 110 may include a cluster membership classifier subsystem 114. The cluster membership classifier subsystem 114 may implement a classifier that has access to data received by the machine actor detection subsystem 110, such as the output embeddings generated by the encoder model 108, and predicts a vector of probability values ​​corresponding to the number of clusters determined using a system such as the embedding clusterer subsystem 340 as shown in FIG. 3. For example, the cluster membership classifier subsystem 114 may be trained to predict the cluster memberships assigned to embeddings. Cluster membership may correspond to a grouping of similar mouse cursor trace patterns known or learned by the model. For example, cluster membership may be assigned to a group of embeddings generated from an input pattern, where the embeddings grouped into a cluster membership have some similar properties, such as shape, distribution, probability, vector, or any other applicable characteristic. Thus, the cluster membership classifier subsystem 114 may be trained to predict the cluster membership to be assigned to a received embedding.

[0051] The machine actor detection subsystem 110 may further include a machine actor score generator 116. The machine actor score generator 116 may be an entity that uses the predictions formed by the cluster membership classifier subsystem 114 to quantify a score associated with a suspicious level of an embedding received by the machine actor detection subsystem 110. For example, the machine actor score generator 116 may calculate the highest probability value output by the cluster membership classifier subsystem 114, excluding a value corresponding to the "largest" cluster discovered by the embedding clusterer subsystem, and output it as a machine actor score. The machine actor detection subsystem 110 may further include a threshold processing subsystem 118. The threshold processing subsystem 118 may be a system or entity that stores and applies thresholds to generated machine actor scores, such as those generated by the machine actor score generator 116. In various embodiments, the threshold processing subsystem 118 may generate a determination as to whether an embedding received by the machine actor detection subsystem 110 may be classified as a valid input or an invalid input based on one or more specified thresholds.

[0052] As shown in FIG. 1, in various embodiments, an unsupervised training system 150 trains an encoder model 108. For example, the encoder model 108 may be or include a recurrent neural network or a transformer model. The unsupervised training system 150 may access and / or receive one or more training datasets and may train the encoder model 108 simultaneously with a decoder model that learns to reconstruct an input sequence from an output embedding. When trained as part of an autoencoder framework, the autoencoder loss function takes the form of a reconstruction error that penalizes the difference between the input sequence and the decoder reconstruction of the input sequence.

[0053] In various embodiments, additional processes may train the cluster membership classifier subsystem 114 to reduce bias and balance classification techniques. In addition to using training to train the cluster membership classifier subsystem 114, the training system 152 may use these additional processes, such as stratification, to further refine the cluster membership classifier subsystem 114 for operation in the detection system 100. The training data for both the autoencoder subsystem 107 and the cluster membership classifier subsystem 114 may be drawn from stratified samples of impressions based on whether a given input has been flagged as a bot interaction by other detection techniques, possibly based on other modalities. The use of stratified samples improves the training process by providing a distribution of fraudulent and non-fraudulent interactions that reduces bias towards valid inputs. Bias in this sense may result in the classifier predicting false negatives during the classification phase. In this way, the training data may be balanced between valid and invalid interactions. This stratification process, when used to supplement the main training method described herein, may provide the benefits described above.

[0054] In various embodiments, models trained according to the methods described herein may be retrained to improve detection of valid and invalid inputs. As one example, retraining the autoencoder may enable improved embedding generation that captures more salient features of the input sequence. In another example, retraining the embedding clusterer subsystem may enable improved clustering of input embeddings by picking up on finer-grained embedding qualities. In yet another example, retraining the cluster membership classifier, as described with reference to Figures 3-4, may improve cluster membership classification by using additional criteria or subtle features of the clustered embeddings to more accurately predict classification of the clustered embeddings.

[0055] This retraining may improve the performance of various components, including the thresholding subsystem 118, by facilitating retraining and dynamic adjustment of the learned model parameters of the autoencoder subsystem 107 and the cluster membership classifier subsystem 114. Such retraining may improve the robustness of the classifiers in the "arms race" between the bot interaction system and the detection system 100. For example, the autoencoder subsystem 107 and the cluster membership classifier subsystem 114 may be periodically retrained by updating the training data with new input sequences. In this manner, adaptations by bots or other sources of invalid interaction access attempts are learned by the cluster membership classifier subsystem 114.

[0056] As described above, the various subsystems of the detection system 100 cooperate to provide functionality that allows the detection system 100 to receive the interaction data 103 and determine whether the interaction data 103 represents a human interaction or a bot interaction. The various subsystems described above may be implemented using a single computer system or multiple computer systems operating in concert. For example, in the case of a device implementing the detection system 100, the subsystems of the detection system 100 described above may be implemented entirely on the device with which the user interacts. In some other implementations, some components or subsystems of the detection system 100 may be implemented on the device with which the user interacts, while other components may be implemented remotely from the device, possibly on some other computing device, platform, or server.

[0057] FIG. 2 is a diagram of an unsupervised training system 150 configured for unsupervised training of the autoencoder subsystem 107 of the bot interaction detection system 100 through the use of an autoencoder training subsystem 153 according to an embodiment described herein. As described above, the autoencoder subsystem may also implement a transformer model. The training data 210 accessed and / or received by the autoencoder training subsystem 153 may be or include historical data from a human interaction or bot interaction source that has been processed to prepare formatted / processed sequence data 105. The formatted / processed sequence data 105 may be received from a data processor / formatter 104 as described herein. In various embodiments, the training data 210 may be or include synthetic data in addition to or instead of historical data.

[0058] For example, generating synthetic data by a generative neural network model may improve the accuracy and speed of the autoencoder training subsystem 153 when large amounts of processed / formatted sequence data 105 are used to train the encoder model 108. Additionally, the training data 210 may be stratified to provide a balanced dataset between bot and human interactions as an approach to training the encoder 108 where bot interactions may not be adequately reflected in the input data. In various embodiments, the autoencoder training subsystem 153 includes a data input subsystem 220, which may be or include one or more devices and / or virtual devices configured to access and / or receive the processed / formatted sequence data 105 from the training data 210 source and provide the data to one or more constituent subsystems of the unsupervised training system 150.

[0059] In various embodiments, the unsupervised training system 150 is implemented in or as part of a computing device such as a server. The unsupervised training system 150 may be implemented as a dedicated hardware device, or as program code, or as a combination of both. For example, the operations described herein as being performed by the unsupervised training system 150 may be embodied in program code implementing the unsupervised training system 150, such program code being executable by one or more processing units. For example, the data input subsystem 220, the encoder model 108, and the decoder model 230, the objective function 240, and other aspects of the autoencoder training subsystem 153 may each be implemented as one or more software functions or dedicated hardware devices and may operate together to implement the unsupervised training system 150 described herein. The encoder model 108 and the decoder model 230 may each be implemented as separate machine learning models, such as recurrent neural networks. The unsupervised training system 150 may implement separate unsupervised training algorithms for training each subsystem of the invalid interaction detection system 100, such that each subsystem is trained using unlabeled data. In this manner, the cluster membership classifier subsystem 114 may be trained to generate machine actor scores for input embeddings using labels predicted by the embedding clusterer subsystems, and the embedding clusterer subsystems, such as the embedding clusterer subsystem 340, may be trained using unlabeled training data.

[0060] The encoder model 108 may be agnostic to the origin of the formatted / processed sequence data 105. As such, the encoder model 108 may be trained to generate embeddings from input sequences that may correspond to human or bot interactions. The decoder model 230 may be configured to take the output embeddings of the encoder model 108 as input and generate an output sequence that is provided to the objective function 240. The objective function 240, in turn, may access and / or receive the output sequence from the decoder model 230 and the input sequence from the data input subsystem 220. Through a comparison of the output sequence to the input sequence, the objective function 240, which may be realized as, for example, a squared loss, adjusts one or more learned parameters of the encoder model 108 and / or the decoder model 230. In this manner, the encoder model 108 and the decoder model 230 may be jointly trained as part of a unified autoencoder training process.

[0061] In various embodiments, the autoencoder training subsystem 153 processes the formatted / processed sequence data 105 until it minimizes an objective function 240, which, for example, defines the change in the comparison of the output sequence to the input sequence and / or estimates the marginal change in the learning parameters with each training cycle. In this way, the autoencoder training subsystem 153 can provide the classifier with fixed-length vectors that can be effectively clustered and classified while verifying that the output embeddings of the encoder model 108 preserve the salient information of the input sequence. In this context, "salient" information may refer to a reduced-dimensional representation of the input sequence that can serve as an input to a clustering model. After training, the encoder model 108 can be used as part of a bot interaction detection system such as that shown in FIG. 1.

[0062] 3 is a diagram of a training system 152 for training the embedding clusterer subsystem 340 and the cluster membership classifier subsystem 114 of the machine actor detection subsystem 110, according to an embodiment described herein. As mentioned above, the embedding clusterer subsystem 340 may be a standard clusterer, such as K-means, that learns clusterings of sequence embeddings 320, i.e., assigns a cluster label to each input embedding. Since proximity in the embedding space may imply similarity, each cluster of the learned clustering may correspond to a grouping of inputs characterized by similar properties, which in turn may correspond to invalid interaction patterns. As mentioned above, the cluster membership classifier subsystem 114 may be or include a standard classifier, such as a logistic regression model. Given the cluster labels assigned to the input sequence embeddings 320 by the embedding clusterer subsystem 340, the classifier 114 may be trained to predict the probability of belonging to each cluster discovered by the embedding clusterer subsystem 340 for a given embedding. In this manner, the training system 152 may include applying learning techniques using one or more prepared sets of training data 310 that are converted by the autoencoder subsystem 107 into sequence embeddings 320. The training data 310 may be or may include the same training data used for unsupervised training of the autoencoder subsystem 107.

[0063] The sequence embedding data 320 may be data corresponding to embeddings of valid and invalid interaction data used to train the autoencoder subsystem 107. If the classifier is trained by the training approach described above, the cluster membership classifier subsystem 114 is not provided with labeled embeddings. In various embodiments, the training data 310 includes a population of input embeddings generated using the input sequence data for each of several subpopulations representing different types of invalid interactions (e.g., bot interactions) and real human interactions from which the cluster membership classifier subsystem 114 will learn. In various further embodiments, the classification is a binary classification representing whether the interaction is a bot interaction or not a bot interaction. Similar to training the encoder model 108, the cluster membership classifier subsystem 114 may be trained using a stratified dataset that includes a balanced proportion of sequence embeddings 320 as an approach to limit the bias of the classifier to classify input sequences in an unbiased manner. The training data 310 may be run through a data entry subsystem 330. In various embodiments, the data entry subsystem 330 may perform similar processing as the data entry subsystem 220 described above.

[0064] The learning subsystem 155 may implement one or more objective functions. The learning subsystem 155 may include an objective function 350(a) suitable for a learning setting, more specifically, clustering (of embeddings output by the encoder model 108). For example, if the embedding clusterer subsystem 340 is implemented as a K-means clusterer, the objective function 350(a) may employ an objective function that penalizes intra-cluster distances to cluster centroids, summed over all clusters. Training using the objective function 350(a) may include adjusting one or more weights and / or coefficients of the embedding clusterer subsystem over multiple iterations until the value of the objective function 350(a) converges to a local or global minimum. For example, parameters defining each cluster of the embedding clusterer subsystem, such as center locations and distributions, may be adjusted at each training iteration, and the objective function 350(a) may be evaluated using the current values ​​of each cluster.

[0065] The value of the objective function 350(b) suitable for a supervised learning setting may be utilized to tune one or more parameters of the cluster membership classifier subsystem 114 at each training iteration to better recognize the classifiable elements of the clustered embeddings (output by the encoder model 108) and predict the cluster membership of the embeddings. The objective function 350(b) may differ from the objective function 350(a) in that the objective function 350(b) may be suitable for a supervised learning setting for training the cluster membership classifier subsystem 114 to predict embedding cluster memberships using, for example, the predicted values ​​of the embedding clusterer subsystem 340 as labels. For example, if the cluster membership classifier subsystem 114 is realized as a logistic regression model, the objective function 350(b) may include a cross-entropy loss function used to train the cluster membership classifier subsystem 114.

[0066] In various embodiments, for example using a mixture model (such as a Gaussian mixture model), the objective functions 350(a) and 304(b) may be collapsed into a single objective function 340 used to train the concatenated system, i.e., the embedding clusterer subsystem 340 and the cluster membership classifier subsystem 114 are collapsed into a single component that comprises the mixture model. To elaborate, for example, a Gaussian mixture model (GMM) can fit multi-modal Gaussian distributions to the input embeddings (individual Gaussian distributions play the role of clusters), from which a "cluster label" can be inferred as the label of the individual distribution with the highest posterior probability given the input (which covers the role of classification). The objective function in this latter example can be derived from an expectation maximization algorithm that can be used to train the GMM.

[0067] A general parametric learning model may be trained and tested using different variable configurations and / or hyperparameters to determine the optimal configuration for a particular application of the model. The autoencoder training subsystem 153 and the learning subsystem 155 may implement separate hyperparameter tuning algorithms to optimize the autoencoder subsystem 107 in the former case and the embedded clusterer subsystem 340 and the cluster membership classifier subsystem 114 in the latter case. For example, one or more terms of the objective function 340 and / or the cluster membership classifier subsystem 114 and / or the embedded clusterer subsystem 340 may be tuned by varying unlearned parameters, such as scalar weighting coefficients. Because the embedded clusterer subsystem 340 and the cluster membership classifier subsystem 114 are separate entities, the various components of each may be optimized individually at a finer level of granularity than in situations where clustering and classification are combined in one model, such as a mixture model.

[0068] Figure 4 is a schematic diagram of an inference stage for detecting bot interactions from mouse cursor event data, according to an embodiment described herein. As an illustrative example of the techniques described with reference to Figures 1-3, Figure 4 illustrates an encoder 400 that receives mouse cursor event data (e.g., including both movement and click sequences) as an input sequence 402 and as part of an inference process for determining some output according to an embodiment described herein. The encoder 400 may be part of an autoencoder that includes an RNN encoder and a decoder, as described herein.

[0069] Mouse cursor event data 404 is shown in FIG. 4 as an example including position changes according to a sequence of position deltas, although other user interface interaction data may be treated similarly. Position deltas may be constructed from a particular subset of consecutive types of user input signals extracted from past web interactions including user session data. In various embodiments, deltas between positions in a coordinate system (e.g., “x”, “y”, “r”, “θ”, etc.) may be accompanied by corresponding deltas between timestamps (e.g., “t”, or “c”). A state value (“M”) may be used to further indicate the current state of the received input. For example, state value M may include a vector representing the state of the input when a mouse button is pressed, not pressed, clicked, released, etc. In constructing the sequence of deltas, a rule may be enforced (e.g., arbitrarily or by prioritizing privileged event types such as movement) that only a single mouse cursor event may occur at a given position and time, and thus state M may be represented as a one-hot vector taking value 1 at coordinates representing active event types and value 0 at all other coordinates. Using these methods, sequences of different modalities of input (mouse movements, clicks, etc.) can be matched into a unified sequence of vectors containing deltas and state.

[0070] It is important that the embeddings produced by the generative machine learning models preserve the non-fluid, non-human cursor patterns present in the original raw cursor trace data. Non-fluid, non-human cursor patterns are generally indicative of invalid bot-based interactions with applications. For example, humans often interact with input devices using inputs that are non-fluid, imprecise, and non-periodic. In contrast, bots often generate inputs using non-fluid, non-human, precise, and / or periodic inputs (i.e., inputs that change corresponding to regular time intervals, such as steps in a script operated by a bot). In some cases, bots may utilize features that attempt to simulate the fluid, random movements of humans. Nevertheless, these bot interactions contain some patterns that are not fully indicative of human interactions. Thus, proper, thorough, and continuous training of the generative models described herein is a key feature in detecting inputs from these simulated bots.

[0071] As described herein, an input device such as a mouse, tablet, or other electronic input device may be used to generate input interaction data. Concurrent with use of the electronic input device, a cursor may be displayed on the graphical user interface that represents the relative position of the input on the electronic input device. As the electronic input device is manipulated to generate the interaction data, the cursor displayed on the graphical user interface also changes position or state accordingly. This can be used to facilitate interaction with an application that utilizes a graphical user interface. For example, the electronic input device can interact with one or more additional elements on a graphical user interface implemented as part of the application by aligning the cursor with the additional elements on the graphical user interface and triggering some interaction.

[0072] For each entry in the input sequence 402, the position of the mouse cursor may be defined as the difference between the current position and the preceding position (e.g., in Cartesian or polar coordinates). The input sequence 402 may be generated based on formatted / processed sequence data 105 derived from cursor trail data 404. In this manner, the input sequence may include a tuple of position and velocity data in a single set of coordinates by including a differential time value "c" that describes the elapsed time between the current and preceding data points. The state value "M" may be or include a one-hot vector of defined states.

[0073] For an input sequence 402 to the encoder 400, each time point may be provided as an input state 408 to a cell 410 of the encoder 400, which may include any number of cells 410 corresponding to the variable length of the input sequence 402. As previously mentioned, the encoder 400 may be an implementation of an RNN, in which case the cell 410 may include an LSTM cell or a GRU cell, among other approaches to context-sensitive sequence embedding.

[0074] Each cell 410 receives the input state 408, cell state 412, and hidden state 414 from the preceding cell 410 in the neural network. For the first of the cells 410, which corresponds to the first time step in memory, the first cell state and the first hidden state are provided to the first cell. LSTM cells differ from GRU cells in that GRU cells do not pass on cell state 412. The hidden state 414 passed to subsequent cells 410 facilitates information retention and provides the encoder 400 with the ability to identify potential information that allows for bot interaction detection, such as, for example, cursor position movement or change in interaction state. In various embodiments, the final hidden state 414 of the encoder 400, “h n" serves as an embedding 416 that may be populated by the cluster membership classifier subsystem 114. The final hidden state may be selected to include latent information from the entire input sequence 402 and reduce the likelihood that entries that may affect the classification are omitted. However, intermediate hidden states 414 may serve as embeddings 416 that, for example, would improve the classification of the input sequence 402. For example, if a bidirectional encoder is used, the intermediate hidden states 414 may better represent the latent dimensional information than either terminal hidden state 414.

[0075] The resulting embedding data may then be sent to the cluster membership classifier subsystem 114 to predict a probability distribution for each of the multiple clusters 418 using the unlabeled input sequence. As mentioned above, in various embodiments, the encoder 400 is included as part of an autoencoder system (e.g., the autoencoder subsystem 107). In some cases, the clusterer and / or classifier may output a visualization of the clustered data, for example, as a three-dimensional or two-dimensional cluster projection. The two-dimensional (2D) cluster projection 420 may represent the clustering of the data for each of the clusters 418 on a pair of axes selected to account for the separation between the clustered embeddings.

[0076] As illustrated, different clusters 418 may have different locations in a 2D coordinate system and different distribution densities. In some cases, bot and human interactions may be distinguishable by distribution density, where aspects of the input sequences generated by the bot algorithm may contain a high degree of duplication or exhibit other detectable patterns, in which case clustering may produce dense groups of inputs that display similar repeating patterns and sparse groups otherwise. For example, depending on the input dataset, the largest sparse cluster (by number of points) may be more likely to be attributable to a mix of genuine non-repetitive human interactions and bot interactions that are indistinguishable from human interactions, while other smaller sparse clusters may contain inputs with similar patterns that are more likely to be bot interactions. To illustrate, cluster 418-2 may be characterized by a lower cluster density and a higher population size than cluster 418-1. The output of the classifier may be or include a probability distribution describing the likelihood that the input sequence 402 belongs to each of these clusters, among others. For example, in the example 2D cluster projection 420, the output of the classifier includes a four-element vector of values ​​between 0 and 1 describing the probability that the input sequence 402 belongs to cluster 418-2, which exhibits relatively variable behavior, or to a cluster (e.g., cluster 418-1), which exhibits repetitive behavior.

[0077] During inference, the output of the classifier may be processed by the machine actor score generator 116. The machine actor score generator may generate a machine actor score based on the classified clusters 418. In various embodiments, the machine actor score generator 116 may modify the output vector by removing the probability values ​​that belong to the largest cluster. The remaining probability values, such as the highest probability value, may then be used to determine the machine actor score. In an illustrative example, the classifier output may be a four-element vector P=[p1, p2, p3, p4]=[0.01, 0.002, 0.89, 0.098], indicating that the input sequence 402 belongs to class 3 with high probability.

[0078] By way of example, but not by way of generality, if class i=1 corresponds to the largest cluster, the machine actor score generator 116 can remove the first element from the vector P, leaving P=[0.002, 0.89, 0.098]. From the largest probability value p3=0.89, the machine actor score can be predicted. In the simplest example, the machine actor score may simply be p3, such that there is a greater than 89% probability that the input sequence 402 corresponds to a bot interaction. The machine actor score generator 116 can then send the machine actor score to the thresholding subsystem 118. The thresholding subsystem 118 can use a threshold or metric to determine whether the generated machine actor score meets, exceeds, or does not exceed the threshold. This decision may be sent to the output 422.

[0079] The output 422, which may include operations of the output component 120 described with reference to Figures 1 and 5, may include providing validation that the input sequence corresponds to a genuine interaction with a human user. In contrast, if the input sequence is deemed suspicious (i.e., indicative of invalid traffic), the output 422 may include instructions and / or messages to an associated system to flag the corresponding request and / or the source of the request. The output may also include, but is not limited to, generating and / or presenting a visualization of the 2D cluster projection 420, such as on a display of a user device, or communicating the probability values, input sequence, and / or embeddings to an associated security system for use in further bot interaction detection processing.

[0080] 5 is a block flow diagram of an inference stage method 500 for detecting bot interactions from mouse cursor trace data according to an embodiment described herein. Specifically, the detection system 100 can perform this method 500 or a similar method, where the cluster membership classifier subsystem 114 is trained to classify the cluster membership of embeddings generated by the encoder 108 from the input data 105, subject to the clustering learned by the embedding clusterer subsystem during training. Thus, this method 500 or a similar method can be performed to distinguish bot interactions from human interactions and perform a gatekeeping function by blocking or forwarding requests based on the classification of the input data 105.

[0081] The method 500 depicted in FIG. 5, as well as other methods described herein, may be implemented in software (e.g., as code, instructions, or programs) executed by one or more processing units (e.g., processors or processor cores), hardware, or a combination thereof. The software may be stored in a non-transitory storage medium, such as a memory device. The method 500 is intended to be exemplary and non-limiting. Although FIG. 5 depicts various activities occurring in a particular order or sequence, this is not intended to be limiting. In certain embodiments, for example, activities may be performed in a different order, or one or more activities of the method 500 may be performed in parallel. In certain embodiments, the method 500 may be performed by the detection system 100.

[0082] At block 505, the detection system 100 obtains input data 105, which may be or may include mouse cursor trail data generated as part of an interaction with a user interface element such as a button. In various embodiments, the data is collected from past web interactions. For example, the data may be collected from a user web session that includes multiple inputs by an entity interacting with a browser during the duration of the session. This collection process is advantageous because it collects raw data from the web session without the possibility of input being biased by requiring a user or entity to interact with an input system to generate the data. The collection period of data may be continuous or may occur periodically during the web session. The input data 105 may include, but is not limited to, mouse cursor position data, movement data, timestamp data, and state data describing the state of the mouse as a function of time. As described in more detail with reference to FIGS. 1-4, the input data 104 may be sessionized to separate distinct input sessions. In some cases, the input data 104 may be an unbound sequence, for example, collected and stored while the detection system 100 is active. In this manner, components of the subsystem may bind the input data 104 based on identifying a single session.

[0083] At block 510, the detection system 100 generates output embeddings by providing the input sequence to an encoder 108 trained to generate output embeddings from input sequences. The encoder 108 may be or include an RNN including LSTM and / or GRU cells trained to generate fixed-length embeddings from variable-length input sequences. In a specific embodiment, the embeddings may be or include hidden state outputs of cells of the RNN such that the embeddings include latent dimensionality information along with clustering learned by the embedding clusterer subsystem (i.e., the embedding clusterer subsystem 340) during training and the cluster membership classifier subsystem 114 is trained to distinguish between human and bot interactions.

[0084] At block 515, the detection system 100 classifies the embeddings according to cluster membership by predicting a probability distribution. As described in more detail with reference to Figures 1 and 4, the classifier can output a vector of probability values ​​that describe the likelihood that the input data 104 belongs to a particular group of similar patterns.

[0085] At block 520, the detection system 100 generates a machine actor score from the probability distribution output by the cluster membership classifier subsystem 114. In various embodiments, the machine actor score corresponds to the probability of belonging to a group of similar cluster patterns when the input data 104 is embedded and clustered. For example, the machine actor score may correspond to the probability of being similar to a group of clustered inputs displaying a particular repetitive pattern. This indicates the likelihood that the input data is an invalid input because repetitive user input behavior is typical of a bot. More specifically, the machine actor score may be or include an element of the classifier output that corresponds to the highest probability value for a class of clustering excluding the "largest" cluster found during training, as described above. The machine actor score serves as a proxy for the likelihood that the input was generated by a bot, insofar as it captures the likelihood that the input belongs to a group of inputs with similar repetitive patterns.

[0086] At decision block 525, the detection system 100 determines whether the machine actor score meets a threshold for attributing the input data to bot traffic. In various embodiments, the determination includes comparing the machine actor score to a threshold and / or threshold range such that if the score exceeds the threshold and / or does not fall within the threshold range, the detection system 100 attributes the input data to a bot interaction. The threshold can be defined based on input received from a data scientist or a client. The threshold may alternatively be determined by using a machine learning model to identify (for example) an optimal threshold with respect to a performance metric (e.g., precision, recall, sensitivity, specificity, accuracy, etc.) or a weighted sum thereof. In some cases, different thresholds are defined for different use cases. In some cases, different types of models, different algorithms, and / or different prioritized performance metrics (or combinations thereof) may be defined for different use cases.

[0087] In various embodiments, the output of the detection system 100 can include a determination of whether the input data represents a human interaction or a bot interaction. Additionally or alternatively, the detection system 100 can act as an intermediate check on one or more interactions, such as a gatekeeping function, to validate the interaction before allowing access to data targeted by the request. For example, the detection system can process the input data, make a suspicious determination, and then block or serve the request based on the suspicious determination. In this manner, the detection system 100 can significantly improve data security and improve automated efforts to detect and thwart automated data access systems.

[0088] It will be appreciated that variations of process 500 are contemplated. For example, block 525 indicates that the machine actor score is compared to a threshold to predict whether the input data corresponds to bot traffic. In some embodiments, rather than using the machine actor score to generate a binary prediction (which may, e.g., result in a binary decision of whether to grant the access request), the machine actor score may be used to prioritize requests. For example, each of a number of requests (e.g., data access, processing resources, etc.) may be ranked in a queue according to the machine actor score. In some cases, requests associated with a low machine actor score (e.g., corresponding to a prediction that the input is from a human) are prioritized and placed closer to the top of the queue compared to requests associated with a high machine actor score. This approach may facilitate improved experience and performance for the user. In some cases, requests associated with a high machine actor score are prioritized and placed closer to the top of the queue compared to requests associated with a low machine actor score. This approach may facilitate ensuring that critical automated processes (e.g., backup generation, security scanning, etc.) are performed.

[0089] There may also or alternatively be different thresholds that change how responses to machine actor scores are generated. For example, the highest machine actor scores may be immediately discarded or sent for analysis. Less high but important machine actor scores may be quarantined and regular machine actor scores may be periodically reviewed by data scientists. Low machine actor scores may approve the request for processing.

[0090] In various embodiments, the generated machine actor scores may be further utilized and / or transmitted to another system to determine one or more attributes of the machine actor score. For example, in some embodiments, the generated machine actor scores may be incorporated as features into a downstream detection / suspiciousness pipeline that utilizes multiple signals of different types to characterize and verify an input session. In some embodiments, the scores may be used to assist researchers in knowing which input sessions to investigate more thoroughly. Investigation may include manually inspecting mouse cursor signals and / or other signals from the input session to investigate individual elements of valid and / or invalid interactions. In some embodiments, the machine actor scores may be used to generate synthetic input data to further refine a training dataset for training a generative neural network.

[0091] In some embodiments, the determination may operate in real-time (e.g., as the mouse device generates input for an application). The determination may be updated in real-time based on incoming input, and the determination may be changed based on the real-time updates. For example, process 500 may iteratively complete the steps of blocks 505-525 over a period of time. The determination at 525 may be made at the end of the period, at which point the determination proceeds to either block 530 or 535.

[0092] If, at block 530, the detection system 100 determines that the input data was generated by a valid entity, the detection system 100 may forward the request and the output of the detection system 100 to another system for processing, e.g., over the network 160, to allow additional input. In contrast, if the detection system 100 identifies the input data as an invalid interaction, the system may block the request or take other action to identify the originating system and prevent further access attempts 535, such as forwarding the request and the output of the detection system 100 to another system for processing. In some embodiments, the input may be quarantined in a suspension environment until it is manually reviewed by an entity, such as a data scientist.

[0093] Advantageously, the detection system 100 provides an improved technique for bot interaction detection and / or rejection of requests generated by an automated data access system. In various embodiments, the detection system 100 can be trained to process input data from a variety of input modalities, including, but not limited to, touch data, scroll data, mouse cursor trace data, speech / voice input, keystrokes, or other computer input data that can be expressed as a sequence. In particular, the detection system 100 provides a robust system that can be periodically trained and learn to detect bot interactions without explicit reverse engineering of bot interaction algorithms.

[0094] Classification weighting It will be appreciated that there may be multiple clusters corresponding to bot activity. For example, one cluster may correspond to trace activity of quickly navigating the cursor directly to a center pixel, while another cluster may correspond to clicking on a pixel in the corner of an image. Different types of activity may result from (for example) different entities with defined rules for bot control, different contexts (e.g., different browser environments with different event firing mechanisms), different time periods, etc. As another example, device types or other types of data characterizing user devices may have different characteristics between clusters. It will be appreciated that these examples are simplifications to convey the idea that clusters ideally correspond to specific differentiated bot patterns. In reality, clusters may represent more complex patterns and behaviors that may not be easily described or identified, but may still represent bot traffic.

[0095] Some disclosures herein relate to predicting whether a given user is a human or a bot. In a purely binary classification instance, each cluster associated with any type of bot activity can be characterized as a bot cluster. If a given cursor trail dataset is assigned to any of the bot clusters, it can be predicted to have been generated by a bot.

[0096] However, there may be differences between clusters. For example, clusters may differ with respect to the likelihood that the cluster corresponds to bot activity. Thus, while multiple clusters may be much more likely to correspond to bot activity compared to another cluster or compared to unclustered data elements, one may have a higher confidence metric representing the confidence that the represented activity corresponds to bot activity than another. As another example, clusters may differ with respect to the confidence that elements within each cluster correspond to a distinct and uniform type of activity. As yet another example, a content source or other entity may determine that one bot, one type of bot, or one type of bot activity is less desirable than another bot, another type of bot, or another type of bot activity.

[0097] Thus, a weight may be assigned to each cluster. In some cases, the weight may influence the cluster assignment, such that the cluster assignment is made by evaluating both the degree to which characteristics of a given cursor trace match characteristics of the cluster and the cluster's weight. Alternatively or additionally, the weight may influence the machine actor score or a determined threshold (e.g., to determine whether to characterize input activity as being from a bot or to determine whether to allow a given action). For example, the machine actor score may be higher or the threshold lower if a given cursor trace dataset is assigned to a cluster with a higher weight (compared to when the cursor trace dataset is assigned to a cluster with a lower weight).

[0098] Weights can be assigned to clusters either definitively or to specific instances. Additionally or alternatively, weights may be used definitively (e.g., to potentially influence cluster assignment, to influence machine actor scores, or to influence thresholds) or only in specific instances. Exemplary specific instances are when an input (representing a cursor trail) has no nearest neighbors within a certain radius, when an input is tightly clustered with other known inputs of the same type, when classification of other nearby inputs has a certain importance, when an input indicates that the cursor trail was generated by a particular type of input device (e.g., mouse, trackpad, keyboard, VR), when an input indicates that the cursor trail was received during a particular time of day (e.g., at night), when an input indicates that the cursor trail originated from a device in a particular geographic region (e.g., a particular country), etc.

[0099] Regular Sampling In some cases, rather than evaluating each input dataset to predict whether it is bot- or human-generated (or generate another type of prediction), periodic sampling can be used to reduce resource usage for monitoring. Periodic sampling might result in monitoring inputs over a period of time, evaluating every nth input, or evaluating a random or pseudo-random subset of inputs.

[0100] User interface and analytics dashboard In some cases, rather than or in addition to using a machine actor's score (for example) to determine whether to process a request, identify queue placement, allocate resources, etc., the score may be used for analytics. Specifically, analytics may track how many times particular content is viewed, clicked on, etc. The machine actor score may be used to distinguish whether a portion of a view, click, etc., or a view, click, etc., associated with a machine actor score may indicate that such instance is due to a bot interaction.

[0101] For example, a dashboard may show a distribution of machine actor scores across a set of interactions of one or more types (e.g., clicks, views, etc.). As another example, a dashboard may identify the percentage of interactions predicted to be attributable to bots. This data may be presented for each of one or more content objects or in aggregate. In some cases, one or more particular cursor trail data sets are flagged as particularly suspicious or prime for human review (to determine whether attributable to a bot or a human).

[0102] In some cases, a source of content may have committed to paying another entity for placement of the content on a web page and / or for various types of interactions. Thus, the dashboard data can inform the source about the value of the placement and / or potential fraud. Additionally or alternatively, the dashboard can be made available to other entities, who can use the information to identify strategies for reducing bot interaction (e.g., by influencing decisions regarding where content is placed within a web page, various frame or Javascript preferences, etc.).

[0103] Example Implementation 7 is a diagram of a distributed system 700 for implementing an embodiment. In the illustrated embodiment, the distributed system 700 includes one or more client computing devices 702, 704, 706, and 708 coupled to a server 712 via one or more communication networks 710. The client computing devices 702, 704, 706, and 708 may be configured to execute one or more applications.

[0104] In various embodiments, server 712 may be adapted to run one or more services or software applications that enable the use of backpropagation to train cluster membership classifier subsystem 114 of detection system 100, as described herein. For example, server 712 may run some or all aspects of unsupervised training system 150 or some or all aspects of detection system 100.

[0105] In some embodiments, server 712 may also provide other services or software applications, which may include non-virtual and virtual environments. In various embodiments, these services may be provided as web-based or cloud services, such as under a Software as a Service (SaaS) model, to users of client computing devices 702, 704, 706, and / or 708. Users operating client computing devices 702, 704, 706, and / or 708 may in turn utilize one or more client applications that interact with server 712 to utilize the services provided by these components. More specifically, for example, each of client computing devices 702, 704, 706, and / or 708 may be an embedded device configured to execute detection system 100 and further configured to communicate with server 712 to enable server 712 to train cluster membership classifier subsystem 114 of detection system 100 through backpropagation, as described herein.

[0106] In the configuration depicted in Figure 7, server 712 may include one or more components 718, 720, and 722 that implement the functions performed by server 712. These components may include software components that may be executed by one or more processors, hardware components, or combinations thereof. It should be understood that a variety of different system configurations are possible that may differ from distributed system 700. Thus, the embodiment depicted in Figure 7 is one example of a distributed system for implementing an embodiment system and is not intended to be limiting.

[0107] Users can use client computing devices 702, 704, 706, and / or 708 to interact with aspects of the detection system 100 provided by server 712 in accordance with the teachings of this disclosure. The client devices can provide an interface (e.g., a voice interface) that allows a user of the client device to interact with the client device. The client devices can also output information to the user via this interface. Although only four client computing devices are depicted in FIG. 7, any number of client computing devices may be supported.

[0108] The client devices can include various types of computing systems, such as PA devices, portable handheld devices, general purpose computers such as personal computers and laptops, workstation computers, wearable devices, gaming systems, thin clients, various messaging devices, sensors or other sensing devices, etc. These computing devices can run various types and versions of software applications and operating systems (e.g., Microsoft Windows, Apple Macintosh, UNIX or UNIX-like operating systems, Linux or Linux-like operating systems such as Google Chrome OS), including various mobile operating systems (e.g., Microsoft Windows Mobile, iOS, Windows Phone, Android, BlackBerry, Palm OS). Portable handheld devices can include mobile phones, smartphones (e.g., iPhone), tablets (e.g., iPad), personal digital assistants (PDAs), etc. Wearable devices can include Google Glass head-mounted displays, etc. Gaming systems include various handheld gaming consoles, Internet-enabled gaming consoles (e.g., Microsoft Xbox® gaming consoles with or without Kinect® gesture input devices, Sony PlayStation® systems, various gaming systems offered by Nintendo®, etc.) The client devices may run a variety of different applications, such as various Internet-related applications, communication applications (e.g., email applications, short message service (SMS) applications), etc., and may use a variety of communication protocols.

[0109] Network 710 may be any type of network known to those skilled in the art that is capable of supporting data communications using any of a variety of available protocols, including, but not limited to, TCP / IP (Transmission Control Protocol / Internet Protocol), SNA (Systems Network Architecture), IPX (Internet Packet Exchange), Apple Talk, etc. By way of example only, network 710 may be a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., a network operating according to any of the Institute of Electrical and Electronics Engineers (IEEE) 802.11 suite of protocols, Bluetooth, and / or other wireless protocols), and / or any combination of these and / or other networks.

[0110] The servers 712 may be comprised of one or more general purpose computers, dedicated server computers (including, by way of example, PC (personal computer) servers, UNIX servers, mid-range servers, mainframe computers, rack-mounted servers, etc.), server farms, server clusters, or any other suitable arrangement and / or combination. The servers 712 may include one or more virtual machines running a virtual operating system, or other computing architectures involving virtualization, such as one or more flexible pools of logical storage devices that may be virtualized to maintain virtual storage devices for the servers. In various embodiments, the servers 712 may be adapted to run one or more services or software applications that provide the functionality described in the preceding disclosure.

[0111] The computing systems in the servers 712 may run one or more operating systems, including any of those mentioned above, and any commercially available server operating system. The servers 712 may also run any of a variety of other server and / or mid-tier applications, including HTTP (Hypertext Transfer Protocol) servers, FTP (File Transfer Protocol) servers, CGI (Common Gateway Interface) servers, JAVA servers, database servers, and the like. Exemplary database servers include, but are not limited to, those commercially available from Oracle Corporation®, Microsoft Corporation®, Sybase®, IBM® (International Business Machines), and the like.

[0112] In some implementations, the server 712 may include one or more applications for parsing and consolidating data feeds and / or event updates received from users of the client computing devices 702, 704, 706, and 708. By way of example, the data feeds and / or event updates may include, but are not limited to, Twitter® feeds, Facebook® updates, or real-time updates received from one or more third party information sources and continuous data streams, including real-time events related to sensor data applications, financial tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automobile traffic monitoring, and the like. The server 712 may also include one or more applications for displaying the data feeds and / or real-time events via one or more display devices of the client computing devices 702, 704, 706, and 708.

[0113] The distributed system 700 may also include one or more data repositories 714, 716. In an embodiment, these data repositories may be used to store data and other information. For example, one or more of the data repositories 714, 716 may be used to store training data 210 or other data necessary to train the cluster membership classifier subsystem 114 of the detection system 100, as described herein. The data repositories 714, 716 may be in a variety of locations. For example, the data repository used by the server 712 may be local to the server 712 or may be remote from the server 712 and communicate with the server 712 via a network-based or dedicated connection. The data repositories 714, 716 may be of different types. In an embodiment, the data repository used by the server 712 may be a database, such as a relational database, such as databases provided by Oracle Corporation and other vendors. One or more of these databases may be adapted to allow data to be stored, updated, and retrieved from the database in response to SQL-formatted commands.

[0114] In some embodiments, one or more of the data repositories 714, 716 may also be used by an application to store application data. The data repositories used by an application may be of different types, such as a key-value store repository, an object store repository, or a general storage repository supported by a file system.

[0115] In an embodiment, all or a portion of the training of the cluster membership classifier subsystem 114 of the detection system 100, as described herein, may be provided as a service via a cloud environment. FIG. 8 is a block diagram of a cloud-based system environment in which the training of the cluster membership classifier subsystem 114, as described herein, may be provided at least in part as a cloud service, according to an embodiment. In the embodiment shown in FIG. 8, a cloud infrastructure system 802 may provide one or more cloud services that may be requested by users using one or more client computing devices 804, 806, and 808. The cloud infrastructure system 802 may include one or more computers and / or servers, which may include those described above for the server 712. The computers in the cloud infrastructure system 802 may be organized as general purpose computers, dedicated server computers, server farms, server clusters, or any other suitable arrangement and / or combination.

[0116] The network 810 may facilitate communication and data exchange between the client computing devices 804, 806, and 808 and the cloud infrastructure system 802. The network 810 may include one or more networks. These networks may be of the same type or different types. The network 810 may support one or more communication protocols, including wired and / or wireless protocols, to facilitate communication.

[0117] The embodiment shown in Figure 8 is merely one example of a cloud infrastructure system and is not intended to be limiting. It should be understood that in some other embodiments, cloud infrastructure system 802 may have more or fewer components than those shown in Figure 8, may combine two or more components, or may have a different configuration or arrangement of components. For example, while Figure 8 shows three client computing devices, any number of client computing devices may be supported in alternative embodiments.

[0118] The term cloud services is generally used to refer to services made available to users on demand over a communications network such as the Internet by a service provider's system (e.g., cloud infrastructure system 802). Generally, in a public cloud environment, the servers and systems that make up the cloud service provider's system are distinct from the customer's own on-premise servers and systems. The cloud service provider's systems are managed by the cloud service provider. Thus, customers can utilize these services without having to purchase separate licenses, support, or hardware and software resources for the cloud services provided by the cloud service provider. For example, the cloud service provider's system may host an application, and users can order and use this application on demand over the Internet without having to purchase infrastructure resources to run this application. Cloud services are designed to provide easy and scalable access to applications, resources, and services. Several providers offer cloud services. For example, several cloud services, such as middleware services, database services, and Java cloud services, are offered by Oracle Corporation of Redwood Shores, California.

[0119] In some embodiments, cloud infrastructure system 802 may provide one or more cloud services using a variety of models, such as under a software-as-a-service (SaaS) model, a platform-as-a-service (PaaS) model, an infrastructure-as-a-service (IaaS) model, including a hybrid service model. Cloud infrastructure system 802 may include a set of applications, middleware, databases, and other resources that enable the delivery of the various cloud services.

[0120] The SaaS model allows applications or software to be provided to customers as a service over a communications network such as the Internet, without the customer having to purchase hardware or software for the underlying application. For example, the SaaS model may be used to provide customers with access to on-demand applications hosted by cloud infrastructure system 802. Examples of SaaS services offered by Oracle Corporation include, but are not limited to, various services for human capital / capital management, customer relationship management (CRM), enterprise resource planning (ERP), supply chain management (SCM), enterprise performance management (EPM), analytics services, social applications, and the like.

[0121] The IaaS model is commonly used to provide infrastructure resources (e.g., servers, storage, hardware and networking resources) as cloud services to customers to provide elastic compute and storage capabilities. Various IaaS services are offered by Oracle Corporation.

[0122] The PaaS model is commonly used to provide platform and environment resources as a service that allows customers to develop, run and manage applications and services without the need for the customer to procure, build or maintain such resources. Examples of PaaS services provided by Oracle Corporation (registered trademark) include, but are not limited to, Oracle Java Cloud Services (JCS), Oracle Database Cloud Services (DBCS), data management cloud services, various application development solution services, etc.

[0123] Cloud services are generally provided in an on-demand self-service based, subscription based, elastically scalable, reliable, highly available, and secure manner. For example, a customer may order one or more services provided by cloud infrastructure system 802 via a subscription order. Cloud infrastructure system 802 then executes processing to provide the services requested in the customer's subscription order. For example, a customer may conversationally subscribe to information services or other services provided by detection system 100. Cloud infrastructure system 802 may be configured to provide one or more cloud services.

[0124] Cloud infrastructure system 802 may provide cloud services through a variety of deployment models. In a public cloud model, cloud infrastructure system 802 may be owned by a third-party cloud service provider and cloud services are offered to any public customer, which may be an individual or a business. In certain other embodiments, under a private cloud model, cloud infrastructure system 802 may be operated within an organization (e.g., within a corporate organization) and services are provided to customers within the organization. For example, the customers may be various departments of a company, such as human resources, payroll, or may be individuals within the company. In certain other embodiments, under a community cloud model, cloud infrastructure system 802 and the services provided may be shared by several organizations within an associated community. Various other models, such as hybrids of the above models, may also be used.

[0125] Client computing devices 804, 806, and 808 may be of different types (such as client computing devices 702, 704, 706, and 708 shown in FIG. 7) and may be capable of running one or more client applications. Users may use the client computing devices to interact with cloud infrastructure system 802, such as to request services provided by cloud infrastructure system 802. For example, an attacker may use a client device to send malicious requests.

[0126] In various embodiments, the processing performed by cloud infrastructure system 802 may include big data analytics. This analytics may include using, analyzing, and manipulating large data sets to detect and visualize various trends, behaviors, relationships, and the like within the data. This analytics may be performed by one or more processors, possibly including parallel processing of the data, running simulations using the data, and the like. For example, big data analytics may be performed by cloud infrastructure system 802 to provide training for cluster membership classifier subsystem 114, as described herein. The data used for this analytics may include structured data (e.g., data stored in a database or structured according to a structured model) and / or unstructured data (e.g., data blobs (binary large objects)).

[0127] 8, cloud infrastructure system 802 may include infrastructure resources 830 utilized to facilitate the provision of various cloud services offered by cloud infrastructure system 802. Infrastructure resources 830 may include, for example, processing resources, storage or memory resources, networking resources, etc.

[0128] In one embodiment, to facilitate efficient provisioning of these resources to support various cloud services offered by cloud infrastructure system 802 to various customers, infrastructure resources 830 may be bundled into resource sets or resource modules (also referred to as “pods”). Each resource module or pod may include a pre-integrated and optimized combination of one or more types of resources. In one embodiment, different pods may be pre-provisioned for different types of cloud services. For example, a first set of pods may be provisioned for database services and a second set of pods may be provisioned for Java services, which may include a different combination of resources than the pods in the first set of pods. In some services, resources allocated for provisioning of a service may be shared between services.

[0129] Cloud infrastructure system 802 may itself use services 832 internally that are shared by different components of cloud infrastructure system 802 and that facilitate provisioning of services by cloud infrastructure system 802. These internal shared services may include, but are not limited to, security and identity services, integration services, enterprise repository services, enterprise manager services, virus scanning and whitelist services, high availability, backup and recovery services, services to enable cloud support, email services, notification services, file transfer services, etc.

[0130] Cloud infrastructure system 802 may include multiple subsystems. These subsystems may be implemented in software, hardware, or a combination thereof. As depicted in FIG. 8 , the subsystems may include a user interface subsystem 812 that allows users or customers of cloud infrastructure system 802 to interact with cloud infrastructure system 802. User interface subsystem 812 may include a variety of different interfaces, such as a web interface 814, an online store interface 816 where cloud services offered by cloud infrastructure system 802 are advertised and available for purchase by consumers, and other interfaces 818. For example, a customer may use a client device to request one or more services offered by cloud infrastructure system 802 (service request 834) using one or more of interfaces 814, 816, and 818. For example, a customer may access an online store, browse cloud services offered by cloud infrastructure system 802, and place a subscription order for one or more services offered by cloud infrastructure system 802 to which the customer wishes to subscribe. The service request may include information identifying the customer and the one or more services to which the customer wishes to subscribe.

[0131] 8, cloud infrastructure system 802 may include an order management subsystem (OMS) 820 configured to process new orders. As part of this processing, OMS 820 may be configured to create an account for the customer if not already done so, receive billing and / or billing information from the customer that is used to issue a bill to the customer for providing the customer with the requested services, verify the customer information, finalize the customer's order upon verification, and orchestrate various workflows to prepare the order for provisioning.

[0132] Upon proper authentication, OMS 820 may then invoke Order Provisioning Subsystem (OPS) 824. OPS 824 is configured to provision resources for the order, including processing, memory, and networking resources. Provisioning may include allocating resources for the order and configuring these resources to facilitate the service requested by the customer order. The manner of provisioning resources for the order and the type of resources provisioned may depend on the type of cloud service ordered by the customer. For example, according to one workflow, OPS 824 may be configured to determine that a particular cloud service is being requested and identify the number of pods that would have been pre-configured for that particular cloud service. The number of pods allocated to the order may depend on the size / amount / level / scope of the requested service. For example, the number of pods allocated may be determined based on the number of users supported by the service, the period for which the service is being requested, etc. The allocated pods may then be customized to the particular requesting customer to provide the requested service.

[0133] The cloud infrastructure system 802 may send a response or notification 844 to the requesting customer indicating when the requested service is available. In some cases, information (e.g., a link) may be sent to the customer that enables the customer to begin using and taking advantage of the requested service.

[0134] Cloud infrastructure system 802 may provide services to multiple customers. For each customer, cloud infrastructure system 802 is responsible for managing information related to one or more subscription orders received from the customer, maintaining customer data related to the orders, and providing the requested services to the customer. Cloud infrastructure system 802 may also collect usage statistics regarding the customer's use of the services subscribed to. For example, statistics may be collected such as the amount of storage used, the amount of data transferred, the number of users, system uptime and system downtime, etc. This usage information may be used to bill the customer. Billing may occur, for example, on a monthly cycle.

[0135] Cloud infrastructure system 802 may provide services to multiple customers in parallel. Cloud infrastructure system 802 may store information, possibly including proprietary information, of these customers. In an embodiment, cloud infrastructure system 802 includes an identity management subsystem (IMS) 828 configured to manage customer information and provide separation of the managed information such that information related to one customer is not accessed by another customer. IMS 828 may be configured to provide various security-related services, such as information access management, authentication and authorization services, identity services, such as services for managing customer identities and roles and related functions.

[0136] FIG. 9 is a block diagram of an exemplary computer system 900 that may be used to implement certain embodiments. For example, in various embodiments, the computer system 900 may be used to implement any of the systems, subsystems, and components described herein. For example, multiple host machines may provide and perform training of the cluster membership classifier subsystem 114 of the detection system 100 as described herein. A computer system such as the computer system 900 may be used as a host machine. As shown in FIG. 9, the computer system 900 includes various subsystems, including a processing subsystem 904 that communicates with a number of other subsystems via a bus subsystem 902. These other subsystems may include a processing acceleration unit 906, an I / O subsystem 908, a storage subsystem 918, and a communication subsystem 924. The storage subsystem 918 may include a non-transitory computer-readable storage medium, including a storage medium 922 and a system memory 910.

[0137] Bus subsystem 902 provides a mechanism for allowing the various components and subsystems of computer system 900 to communicate with one another as intended. Although bus subsystem 902 is shown diagrammatically as a single bus, alternative embodiments of the bus subsystem may utilize multiple buses. Bus subsystem 902 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, a local bus using any of a variety of bus architectures, and the like. For example, such architectures may include an Industry Standard Architecture (ISA) bus, a MicroChannel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus, which may be implemented as a mezzanine bus manufactured in accordance with the IEEE P1386.1 standard, and the like.

[0138] The processing subsystem 904 controls the operation of the computer system 900 and may include one or more processors, application specific integrated circuits (ASICs), or field programmable gate arrays (FPGAs). The processors may include single-core or multi-core processors. The processing resources of the computer system 900 may be organized into one or more processing units 932, 934, etc. The processing units may include one or more processors, one or more cores from the same or different processors, combinations of cores and processors, or other combinations of cores and processors. In various embodiments, the processing subsystem 904 may include one or more special purpose co-processors, such as a graphics processor, digital signal processor (DSP), etc. In various embodiments, some or all of the processing units of the processing subsystem 904 may be implemented using customized circuitry, such as application specific integrated circuits (ASICs), or field programmable gate arrays (FPGAs).

[0139] In various embodiments, the processing units of the processing subsystem 904 can execute instructions stored in the system memory 910 or the computer-readable storage medium 922. In various embodiments, the processing units can execute various program or code instructions and can maintain multiple simultaneously executing programs or processes. At any given time, some or all of the program code being executed can reside in the system memory 910 and / or the computer-readable storage medium 922, potentially including one or more storage devices. With appropriate programming, the processing subsystem 904 can provide the various functions described above. If the computer system 900 is running one or more virtual machines, one or more processing units can be assigned to each virtual machine.

[0140] In one embodiment, a processing acceleration unit 906 may be optionally provided to perform customized processing to accelerate the overall processing performed by the computer system 900 or to offload some of the processing performed by the processing subsystem 904.

[0141] The I / O subsystem 908 can include devices and mechanisms for inputting information into the computer system 900 and / or outputting information from or through the computer system 900. In general, use of the term input device is intended to include all possible types of devices and mechanisms for inputting information into the computer system 900. User interface input devices can include, for example, keyboards, pointing devices such as mice and trackballs, touchpads or touchscreens integrated into displays, scroll wheels, click wheels, dials, buttons, switches, keypads, voice input devices with voice command recognition systems, microphones, and other types of input devices. User interface input devices can also include motion sensing and / or gesture recognition devices such as Microsoft Kinect® motion sensors that allow a user to control and interact with the input device, Microsoft Xbox® 360 game controllers, and devices that provide an interface for receiving input using gestures and voice commands. The user interface input devices may also include eye gesture recognition devices, such as a Google Glass® blink detector, that detects eye activity from a user (e.g., "blinking" while taking a picture and / or selecting a menu) and translates the eye gestures as input to the input device (e.g., Google Glass®). Additionally, the user interface input devices may include a voice recognition sensing device that allows a user to interact with a voice recognition system (such as the Siri® navigator) through voice commands.

[0142] Other examples of user interface input devices include, but are not limited to, three-dimensional (3D) mice, joysticks or pointing sticks, game pads and graphic tablets, as well as audio / visual devices such as speakers, digital cameras, digital video cameras, portable media players, webcams, image scanners, fingerprint scanners, barcode readers 3D scanners, 3D printers, laser range finders, and eye-tracking devices. Additionally, user interface input devices may include medical imaging input devices such as, for example, computed tomography devices, magnetic resonance imaging devices, positional emission tomography devices, medical ultrasound devices, etc. User interface input devices may also include audio input devices such as, for example, MIDI keyboards, digital musical instruments, etc.

[0143] In general, use of the term output device is intended to include all possible types of devices and mechanisms for outputting information from computer system 900 to a user or to another computer. User interface output devices may include non-visual displays such as a display subsystem, indicator lights, or audio output devices. Display subsystems may be flat panel devices such as those using cathode ray tubes (CRTs), liquid crystal displays (LCDs) or plasma displays, projection devices, touch screens, and the like. For example, user interface output devices may include a variety of display devices that visually convey text, graphics, and audio / video information, such as, but not limited to, monitors, printers, speakers, headphones, automobile navigation systems, plotters, audio output devices, modems, and the like.

[0144] The storage subsystem 918 provides a repository or data store for storing information and data used by the computer system 900. The storage subsystem 918 provides a tangible, non-transitory computer-readable storage medium for storing the basic programming and data constructs that provide the functionality of the various embodiments. The storage subsystem 918 can store software (e.g., programs, code modules, instructions) that, when executed by the processing subsystem 904, provide the functionality described above. The software may be executed by one or more processing units of the processing subsystem 904. The storage subsystem 918 may also provide a repository for storing data used in accordance with the teachings of the present disclosure.

[0145] The storage subsystem 918 may include one or more non-transitory memory devices, including volatile and non-volatile memory devices. As shown in FIG. 9, the storage subsystem 918 includes a system memory 910 and a computer-readable storage medium 922. The system memory 910 may include a number of memories, including a volatile main random access memory (RAM) for storage of instructions and data during program execution, and a non-volatile read-only memory (ROM) or flash memory in which fixed instructions are stored. In some implementations, a basic input / output system (BIOS), containing the basic routines that help to transfer information between elements within the computer system 900, such as during start-up, may typically be stored in a ROM. The RAM typically contains data and / or program modules currently being operated on and executed by the processing subsystem 904. In some implementations, the system memory 910 may include multiple different types of memories, such as static random access memory (SRAM), dynamic random access memory (DRAM), and the like.

[0146] 9, system memory 910 may load running application programs 912, which may include various applications such as a web browser, a mid-tier application, a relational database management system (RDBMS), program data 914, and an operating system 916. By way of example, operating system 916 may include various versions of Microsoft Windows, Apple Macintosh, and / or Linux operating systems, various commercially available UNIX or UNIX-like operating systems (including, but not limited to, various GNU / Linux operating systems, Google Chrome OS, etc.), and / or mobile operating systems such as iOS, Windows Phone, Android OS, BlackBerry OS, Palm OS operating systems, etc.

[0147] In an embodiment, software instructions or code implementing the training of the cluster membership classifier subsystem 114 of the detection system 100 as described herein may be executed in the system memory 910 .

[0148] The computer-readable storage medium 922 may store programming and data configurations that provide functionality of various embodiments. The computer-readable storage medium 922 may provide storage of computer-readable instructions, data structures, program modules, and other data for the computer system 900. Software (programs, code modules, instructions) that, when executed by the processing subsystem 904, provide the functionality described above may be stored in the storage subsystem 918. As an example, the computer-readable storage medium 922 may include non-volatile memory such as a hard disk drive, a magnetic disk drive, a CD ROM, a DVD, an optical disk drive such as a Blu-Ray® disk, or other optical media. The computer-readable storage medium 922 may include, but is not limited to, a Zip® drive, a flash memory card, a Universal Serial Bus (USB) flash drive, a Secure Digital (SD) card, a DVD disk, a digital video tape, and the like. The computer-readable storage medium 922 may also include solid-state drives (SSDs) based on non-volatile memory such as flash memory-based SSDs, enterprise flash drives, solid-state ROM, SSDs based on volatile memory such as solid-state RAM, dynamic RAM, static RAM, DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs using a combination of DRAM and flash memory-based SSDs.

[0149] In some embodiments, storage subsystem 918 may also include a computer readable storage medium reader 920, which may be further connected to a computer readable storage medium 922. Reader 920 may be configured to receive and read data from a memory device, such as a disk, a flash drive, or the like.

[0150] In some embodiments, computer system 900 may support virtualization techniques, including but not limited to virtualization of processing and memory resources. For example, computer system 900 may provide support for running one or more virtual machines. In some embodiments, computer system 900 may execute a program, such as a hypervisor, that facilitates configuration and management of virtual machines. Each virtual machine may be assigned memory, compute (e.g., processors, cores), I / O, and networking resources. Each virtual machine typically runs independently from other virtual machines. A virtual machine typically runs its own operating system, which may be the same or different than the operating system run by other virtual machines executed by computer system 900. Thus, multiple operating systems may potentially be run simultaneously by computer system 900.

[0151] The communications subsystem 924 provides an interface to other computer systems and networks. The communications subsystem 924 serves as an interface for receiving data from the computer system 900 and transmitting data to other systems. For example, the communications subsystem 924 may enable the computer system 900 to establish a communications channel with one or more client devices over the Internet for receiving information from and transmitting information to the client devices.

[0152] The communications subsystem 924 can support both wired and / or wireless communications protocols. For example, in an embodiment, the communications subsystem 924 can include a radio frequency (RF) transceiver component for accessing wireless voice and / or data networks (e.g., using cellular telephone technology, advanced data network technologies such as 3G, 4G, or EDGE (Enhanced Data Rates for Global Evolution), WiFi (IEEE 802.XX family of standards, or other mobile communications technologies, or any combination thereof)), a global positioning system (GPS) receiver component, and / or other components. In various embodiments, the communications subsystem 924 can provide wired network connectivity (e.g., Ethernet) in addition to or instead of a wireless interface.

[0153] The communications subsystem 924 can receive and transmit data in a variety of forms. For example, in various embodiments, in addition to other forms, the communications subsystem 924 can receive incoming communications in the form of structured and / or unstructured data feeds 926, event streams 928, event updates 930, etc. For example, the communications subsystem 924 may be configured to receive (or transmit) data feeds 926 in real time from users of social media networks and / or other communications services, such as web feeds, such as Twitter® feeds, Facebook® updates, Rich Site Summary (RSS) feeds, and / or real-time updates from one or more third party sources.

[0154] In an embodiment, the communications subsystem 924 may be configured to receive data in the form of a continuous data stream, which may include an event stream 928 of real-time events and / or event updates 930, which may be continuous or open-ended in nature with no explicit termination. Examples of applications that generate continuous data may include, for example, sensor data applications, financial tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automobile traffic monitoring, and the like.

[0155] The communications subsystem 924 may also be configured to communicate data from the computer system 900 to other computer systems or networks. The data may be communicated in a variety of different forms, such as structured and / or unstructured data feeds 926, event streams 928, event updates 930, etc., to one or more databases that may be in communication with one or more streaming data source computers coupled to the computer system 900.

[0156] The computer system 900 can be one of a variety of types, including a handheld portable device (e.g., an iPhone® mobile phone, an iPad® computing tablet, a PDA), a wearable device (e.g., a Google Glass® head-mounted display), a personal computer, a workstation, a mainframe, a kiosk, a server rack, or any other data processing system. Because the nature of computers and networks is constantly changing, the description of the computer system 900 depicted in FIG. 9 is intended as a specific example only. Many other configurations are possible having more or fewer components than the system depicted in FIG. 9. Based on the disclosure and teachings provided herein, one of ordinary skill in the art will appreciate other ways and / or techniques of implementing the various embodiments.

[0157] While specific embodiments have been described, various modifications, variations, alternative constructions, and equivalents are possible. The embodiments are not limited to operating in a particular data processing environment, but rather operate freely in multiple data processing environments. Moreover, while certain embodiments are described using a particular sequence of transactions and steps, it will be apparent to one skilled in the art that this is not intended to be limiting. Although some flow charts depict operations as a sequential process, many operations may be performed in parallel or concurrently. Additionally, the order of operations may be rearranged. A process may have additional steps not included in the figures. Various features and aspects of the embodiments described above may be used individually or jointly.

[0158] Additionally, while certain embodiments have been described using particular combinations of hardware and software, it should be recognized that other combinations of hardware and software are possible. Certain embodiments may be implemented exclusively in hardware, exclusively in software, or using a combination thereof. The various processes described herein may be implemented in any combination on the same processor or on different processors.

[0159] Where a device, system, component, or module is described as being configured to perform a certain operation or function, such configuration may be achieved, for example, by designing an electronic circuit to perform the operation, by executing computer instructions or code, or a processor or core programmed to execute code or instructions stored in a non-transitory memory medium, or by programming a programmable electronic circuit (such as a microprocessor) to perform the operation. Processes may communicate using a variety of techniques, including, but not limited to, conventional techniques for inter-process communication, and different sets of processes may use different techniques, or the same set of processes may use different techniques at different times.

[0160] In this disclosure, specific details are given to provide a thorough understanding of the embodiments. However, the embodiments may be practiced without these specific details. For example, well-known circuits, processes, algorithms, structures, and techniques are shown without unnecessary detail to avoid obscuring the embodiments. This description only provides exemplary embodiments and is not intended to limit the scope, applicability, or configuration of other embodiments. Rather, the preceding description of the embodiments provides one of ordinary skill in the art with an enabling description for implementing various embodiments. Various changes may be made in the function and arrangement of elements.

[0161] The specification and drawings are therefore to be regarded in an illustrative rather than restrictive sense. It will be apparent, however, that additions, subtractions, deletions, and other modifications and changes may be made without departing from the broader spirit and scope of the appended claims. Accordingly, while specific embodiments have been described, these are not intended to be limiting. Various modifications and equivalents are within the scope of the following claims. Modifications and / or variations include any relevant combination of the disclosed features.

[0162] According to embodiments, the disclosed features are summarized below. Example 1 is a computer-implemented method that includes obtaining interaction data generated as part of a particular interaction by a user with a computer application; formatting the interaction data into a formatted sequence including a bounded sequence of interaction events over time that is processable by a neural network; inputting the formatted sequence to an encoder model to generate an embedding; inputting the embedding to a classifier model to output a vector of probability values ​​corresponding to a number of classes characterizing the embedding; and determining a machine actor score for the particular interaction based at least in part on the vector of probability values, where the machine actor score indicates that the particular interaction is invalid, and the method includes outputting the machine actor score.

[0163] Example 2 is the method of any preceding or subsequent aspect, wherein the encoder model is trained as part of an autoencoder that includes an encoder model and a decoder model, the encoder model including a recurrent neural network that includes a plurality of gradient recurrent unit cells or long short-term memory cells.

[0164] Example 3 is the method of any preceding or subsequent aspect, wherein the autoencoder is a variational autoencoder configured to learn to output an embedding by sampling from a Gaussian probability distribution whose parameters are learned and output by the encoder model.

[0165] Example 4 is the method of any preceding or subsequent aspect, wherein the encoder model is a deep learning transformation model including a transform encoder and a transform decoder, the transform encoder configured to transform the formatted sequence into a sequence of intermediate outputs, and the transform decoder configured to decode the intermediate outputs into an embedded output sequence.

[0166] Example 5 is the method of any preceding or subsequent embodiment, further including clustering the embeddings with a clustering model separate from the classifier model to form a clustered embedding set, and training the classifier model to learn embedding cluster membership probabilities using the embeddings as input features and corresponding cluster labels from the clustering as labels to form a trained classifier model, where inputting the embeddings into the classifier model includes inputting the embeddings into the trained classifier model.

[0167] Example 6 is the method of any preceding or subsequent embodiment, wherein the cluster model and the classifier model are part of a single mixture model.

[0168] Example 7 is the method of any preceding or subsequent embodiment, where determining the machine actor score includes identifying a first value in a vector of probability values ​​attributable to a first class and identifying a second value in the vector of probability values ​​attributable to a second class, the second value being a maximum value in the vector of probability values ​​other than the first value, and determining the machine actor score includes predicting the machine actor score based at least in part on the second value.

[0169] Example 8 is the method of any preceding or subsequent embodiment, wherein identifying the first value includes evaluating a plurality of variance values ​​of a plurality of clusters generated by the classifier model and assigning a cluster having a maximum variance value among the plurality of variance values ​​to the first class.

[0170] Example 9 is the method of any preceding or subsequent embodiment, further including generating a visualization of the interaction input in the embedding space, possibly as an annotated two-dimensional or three-dimensional projection, and presenting the visualization on a computer display such that the embedding can be qualitatively monitored and addressed as necessary.

[0171] Example 10 is the method of any preceding or subsequent embodiment, wherein the bounded sequence of interaction events includes a sequence of positional and temporal deltas generated by differences in positional and temporal elements, respectively, of an input over a period of time.

[0172] Example 11 is the method of any preceding or subsequent aspect, wherein the bounded sequence further includes mouse click data corresponding to a sequence of mouse clicks over a period of time.

[0173] Example 12 is the method of any preceding or subsequent embodiment, wherein a first class of the plurality of classes is attributable to human traffic and a second class of the plurality of classes is attributable to bot traffic.

[0174] In some examples, the cursor trace embedding system includes one or more processors and non-transitory memory storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including the methods described above.

[0175] In some examples, a computer program product includes a computer-readable storage medium having program instructions executable by one or more processors to cause the one or more processors to perform operations including the methods described above.

Claims

1. 1. A method for detecting automated web traffic, comprising: The method comprises: Obtaining interaction data generated as part of a particular interaction by a user with a computer application; formatting the interaction data into a formatted sequence comprising a bounded sequence of interaction events over time that is processable by a neural network; inputting the formatted sequence into an encoder model to generate an embedding; inputting the embedding into a classifier model and outputting a vector of probability values ​​corresponding to a number of classes characterizing the embedding; determining a machine actor score for the particular interaction based at least in part on the vector of probability values; a high machine actor score corresponds to a prediction that the particular interaction is ineffective; The method further includes outputting the machine actor scores.

2. 2. The method of claim 1 , wherein the encoder model is trained as part of an autoencoder that includes the encoder model and a decoder model, the encoder model comprising a recurrent neural network including a plurality of gradient recurrent unit cells or long short-term memory cells.

3. 3. The method of claim 2, wherein the autoencoder is a variational autoencoder configured to learn to output an embedding by sampling from a Gaussian probability distribution whose parameters are learned and output by the encoder model.

4. 4. The method of claim 1, wherein the encoder model is a deep learning transformation model including a transform encoder and a transform decoder, the transform encoder configured to transform the formatted sequence into a sequence of intermediate outputs, and the transform decoder configured to decode the intermediate outputs into an embedded output sequence.

5. clustering the embeddings by a clustering model separate from the classifier model to form a clustered embedding set; training the classifier model to learn embedded cluster membership probabilities using the embeddings as input features and corresponding cluster labels from the clustering as labels to form a trained classifier model; The method of any of claims 1 to 4, wherein inputting the embeddings into a classifier model comprises inputting the embeddings into a trained classifier model.

6. The method of claim 5 , wherein the clustering model and the classifier model are part of a single mixture model.

7. Determining the machine actor score includes: identifying a first value of a vector of probability values ​​for belonging to a first class; and determining a second value of the vector of probability values ​​for belonging to a second class; the second value being the maximum value of a vector of probability values ​​other than the first value; The method of claim 1 , wherein determining the machine actor score further comprises predicting the machine actor score based at least in part on the second value.

8. Identifying the first value comprises: evaluating a plurality of variance values ​​of a plurality of clusters generated by the classifier model; and attributing to the first class a cluster having a maximum variance value among the plurality of variance values.

9. generating a visualization of the interaction input in the embedding space as an annotated 2D or 3D projection; The method of any of claims 1 to 8, further comprising: presenting a visualization on a computer display so that the embedding can be qualitatively monitored and acted upon if necessary.

10. The method of any of claims 1 to 9, wherein the bounded sequence of interaction events comprises a sequence of positional and temporal deltas generated by differences in positional and temporal components, respectively, of an input over a period of time.

11. The method of claim 10 , wherein the bounded sequence further comprises mouse click data corresponding to a sequence of mouse clicks over the period of time.

12. The method of claim 7 , wherein the first class of the plurality of classes is attributable to human traffic and the second class of the plurality of classes is attributable to bot traffic.

13. 1. An automatic web traffic detection system, comprising: one or more processors; and a non-transitory memory for storing computer-executable instructions. The computer-executable instructions, when executed by the one or more processors, cause the one or more processors to: Obtaining interaction data generated as part of a user's interaction with a computer application; formatting the interaction data as a bounded sequence of interaction events over time that can be modeled by a neural network to form a formatted sequence; inputting the formatted sequence into an encoder model configured to generate an embedding from the bounded sequence; generating an embedding as an output of the encoder model; inputting the embeddings into a classifier model configured to use the embeddings to predict a vector of probability values ​​corresponding to a plurality of classes; predicting the vector of probability values ​​as an output of the classifier model; determining a machine actor score for the interaction from the vector of probability values; and and outputting said machine actor scores.

14. 14. The automatic web traffic detection system of claim 13, wherein the encoder model is trained as part of an autoencoder that includes the encoder model and a decoder model, the encoder model including a recurrent neural network that includes a plurality of gradient recurrent unit cells or long short-term memory cells.

15. clustering the embeddings by a clustering model separate from the classifier model to form a clustered embedding set; training the classifier model using the clustered embedding set to form a trained classifier model; 15. The automatic web traffic detection system of claim 13 or 14, wherein inputting the embeddings into a classifier model comprises inputting the embeddings into a trained classifier model.

16. Determining the machine actor score includes: identifying a first value of a vector of probability values ​​for belonging to a first class; and determining a second value of the vector of probability values ​​for belonging to a second class; the second value being the maximum value of a vector of probability values ​​other than the first value; 16. The automated web traffic detection system of claim 13, wherein determining the machine actor score further comprises predicting the machine actor score based at least in part on the second value.

17. Identifying the first value comprises: evaluating a plurality of variance values ​​of a plurality of clusters generated by the classifier model; and attributing a cluster having a maximum variance value among said plurality of variance values ​​to a first class among said plurality of classes.

18. The computer-executable instructions, when executed by the one or more processors, cause the one or more processors to: generating a visualization of the interaction input in the embedding space as a possibly annotated two-dimensional or three-dimensional projection; and presenting a visualization on a computer display so that said embeddings can be qualitatively monitored and addressed if necessary.

19. The automatic web traffic detection system of any of claims 13 to 18, wherein the bounded sequence of interaction events comprises a sequence of positional and temporal deltas generated by differences in positional elements of an input over a period of time.

20. A program for causing one or more processors to execute a method according to any one of claims 1 to 12.

Images