Detecting Collaborative Abuse Activities Using Graph Neural Networks

Abstract

The present technique uses a graph neural network (GNN) to determine whether a particular entity under consideration, in collaboration with other entities, is engaging in network-related abusive activity on a computing network. In some applications, the particular entity is part of a bot attack aimed at fraudulently engaging in advertising. The present technique trains the GNN by performing machine learning on a training set including multiple nodes, edges, and node labels. In forming the training set, the present technique associates each node in the training set with a feature set that describes the network activity exhibited by the entity at that node. The present technique then connects each pair of nodes in the training set with an edge if the feature sets of the pair satisfy a specified similarity test. The present technique assigns labels to at least some nodes to indicate whether the node is associated with network-related abusive activity.

Description

[Background technology]

[0001] background Conventional systems for detecting malware-infected entities operate by comparing the entity's signature with the signatures of known malicious actors. While useful, these systems do not adequately detect many types of abuse that occur on computing networks. Summary of the Invention

[0002] overview This specification describes a technique for detecting whether a particular entity under consideration, in collaboration with other entities, is engaging in network-related abusive activity on a computing network. The technique uses a graph neural network (GNN) to perform this task. What constitutes abuse is defined differently for different applications. In some applications, a particular entity is part of a group of entities that are engaging in bot attacks on the computing network. In some examples, the bot attacks are specifically aimed at fraudulently engaging in advertising.

[0003] In some implementations, the technique trains a GNN by performing machine learning on a training set that includes multiple nodes, edges, and node labels. In forming the training set, the technique creates nodes for entities that engage in a specified amount of network activity. The technique then associates each node in the training set with a feature set that describes the network activity exhibited by the node's corresponding entity. The technique connects a pair of nodes in the training set with an edge if the feature sets of the nodes satisfy a specified similarity test, as described below. The technique assigns labels to at least some nodes that indicate whether the node is associated with network-related abusive activity.

[0004] Different applications use different criteria in defining what constitutes an entity, for example, different applications may associate an entity with a different device identifier, a different network address identifier (e.g., IP address), a different network cookie identifier, etc., or any combination thereof.

[0005] In some implementations, the feature set associated with a given entity describes characteristics regarding the entity's engagement with content that can be communicated over a computing network via a website. In some examples, the content is an advertisement and the website is hosted by a content publisher.

[0006] In some implementations, the feature set associated with a given entity includes at least some features that describe the distribution of network activity by the given entity with respect to particular categories of activity, e.g., some features that describe the level of engagement of the given entity with different websites.

[0007] In some implementations, the technique determines that a first node (associated with a first entity) is connected to a second node (associated with a second entity) if a first feature vector describing the distribution of network activity by the first entity and a second feature vector describing the distribution of network activity by the second entity satisfy a specified similarity test.

[0008] The above summarized techniques are described herein as being found in various types of systems, devices, components, methods, computer-readable storage media, data structures, graphical user interface presentations, articles of manufacture, and the like.

[0009] This Summary is provided to introduce various concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. [Brief explanation of the drawings]

[0010] BRIEF DESCRIPTION OF THE DRAWINGS [Figure 1] A computing system is shown having a training framework for training a graph neural network (GNN) and an abuse control system that uses the GNN to control network-related abuse activity on a computing network. [Figure 2] A sample training set used by the training framework in Figure 1 to train the GNN is shown. [Figure 3] 2 illustrates components associated with one application of the computing system of FIG. 1. [Figure 4] Figure 2 shows the feature sets associated with each entity in the training set. [Figure 5] We show a function for determining whether any two nodes in the training set of Figure 2 should be connected by an edge. [Figure 6] 1 shows a process illustrating one mode of operation of the training framework of FIG. [Figure 7] We present the process for training a type of GNN. [Figure 8] 1 shows a process illustrating one mode of operation of the abuse control system of FIG. [Figure 9] 2 illustrates an exemplary computing device in which the computing system of FIG. 1 may be implemented. [Figure 10] An exemplary type of computing system is shown that, in some implementations, may be used to implement any aspect of the features shown in the foregoing figures. DETAILED DESCRIPTION OF THE INVENTION

[0011] The same numbers are used throughout this disclosure and the figures to refer to like components and features, with the 100 series numbers referring to features originally found in Figure 1, the 200 series numbers referring to features originally found in Figure 2, the 300 series numbers referring to features originally found in Figure 3, and so on.

[0012] Detailed Description This disclosure is organized as follows: Section A describes an exemplary computing system for detecting and controlling abusive activity on a network. Section B describes an exemplary method that illustrates the operation of the computing system of Section A. Section C describes exemplary computing functionality that, in some implementations, may be used to implement any aspect of the features described in Sections A and B.

[0013] A. Exemplary Computing System FIG. 1 illustrates a computing system 102 having a training framework 104 for training a graph neural network (GNN) 106 and an abuse control system 108 that uses the GNN 106 to control network-related abuse activity on a computing network 110. In some implementations, the computing network 110 is a wide-area network such as the Internet. Different applications define what constitutes "network-related abuse activity" with respect to any given standard specified by a service provider, network administrator, standards body, user group, etc. Generally, the standard defines (a) what network transactions are indicative of abusive behavior and (b) the quantity of that behavior that constitutes abuse. In some examples, network-related abuse activity is activity associated with bot attacks, where multiple entities engage in coordinated network-related activity aimed at achieving a result that is deemed malicious, impermissible, or undesirable with respect to the given standard.

[0014] In some instances, for example, multiple entities involved in bot attacks are configured to fraudulently click on advertisements in order to disrupt the operation of advertising platforms. This type of network-related activity destroys statistics that advertisers use to measure the success of their advertising campaigns. As a further consequence, this type of attack can cause financial damage to advertisers and other entities in the advertising industry. Furthermore, this type of attack disrupts end users' consumption of online resources.

[0015] More generally, any service that solicits user participation over a wide area network is subject to similar attacks. Other examples of bot attacks include fraudulent votes (e.g., "likes"), comments, subscribers, registrations, accounts, contacts (e.g., "friends"), news article selections, etc., or any combination thereof. The goals of bot attacks are similarly diverse. In some cases, bot attacks are aimed simply at swaying public opinion by actively shaping the narrative about an event, for example, through artificial clicks or "likes" on news articles or social network posts.

[0016] Different bot campaigns achieve coordinated behavior in different ways. In a traditional bot attack, an originating entity distributes a malicious program to multiple computing devices, for example, by sending email messages to the computing devices. Once a computing device is infected with the malicious program, it begins participating in the bot attack. In some cases, the malicious program induces the infected computing device to click on advertisements on one or more websites according to a predetermined schedule and / or in response to other cues. Generally, a bot refers to a program running on a computing network with the purpose of impersonating actions that a human might perform on the computing network.

[0017] However, it should be noted that the computing system 102 does not know how bot campaigns technically achieve their objectives. For example, the computing system 102 does not operate by directly detecting malware programs installed on user devices. Similarly, the computing system 102 can detect whether particular entities are engaging in fraudulent concerted activity, whether they are knowingly or unwittingly participating in such activity.

[0018] As used herein, the term "entity" refers to an entity that interacts with the GNN 106. Different applications define what constitutes an entity in different ways. For example, an entity may be associated with a different device address (e.g., a different Media Access Control (MAC) address), a different browser application identifier, a different network address (e.g., a different Internet Protocol (IP) address), a different user identifier, a different group identifier, a different user location identifier, a different network cookie identifier, etc. In other cases, any entity may be defined in terms of two or more identifiers. For example, a single entity may be defined by the combination of a particular IP address and a particular network cookie identifier. In another case, a single entity may be defined by the combination of a modem identifier and a MAC address, and so on. For ease of explanation, it is assumed that different entities in the following examples correspond to different IP addresses.

[0019] The following provides an overview of the operation of the training framework 104, followed by an overview of the operation of the abuse control system 108. The training framework 104 generates a training set 112. In general, the training component 114 operates on the training set 112 to generate the GNN 106. The abuse control system 108 uses the GNN 106 to control interactions by entities with the computing network 110.

[0020] More specifically, the training framework 104 includes a network inspection component or mechanism 116 that collects information about entities interacting with the computing network 110. Consider the merely illustrative case where a search engine or social network application provider implements the training framework 104, or an entity implements the training framework 104 in cooperation with a search engine or social network application. Such an entity maintains a record log that provides information about queries submitted by end users (“users”), search results and advertisements served to users in various contexts, and selections (e.g., “clicks”) made by users in response to the submission of input queries. In some cases, the selections reflect clicks by users on advertisements or landing pages. The network inspection component 116 collects and stores at least a portion of such network-related activity. The training framework 104 further takes appropriate precautions to ensure that sensitive user content is removed from the data collected by the network inspection component 116, for example, by anonymizing the data.

[0021] As a first function, the node identification component 118 identifies a group of eligible entities for inclusion in the training set 112. In some implementations, the node identification component 118 defines eligible entities as entities that have engaged in a specified amount of network activity in a specified time period. In some examples, the node identification component 118 specifically defines eligible entities as (1) entities that have performed at least a specified number of clicks within a specified time period, and (2) entities that have engaged in network-related activity for at least a specified length of time. The testing performed by the node identification component 118 generates a sparse graph that serves as the training set 112. By manipulating the sparse graph, the training component 114 reduces its use of computing resources (e.g., memory and processor resources). The testing performed by the node identification component 118 also reduces noise in the training set 114 by eliminating less active entities from the training set 112, thereby enabling the training component 114 to generate a high-quality trained GNN 106.

[0022] Second, the node identification component 118 associates nodes in the training set 112 with respective eligible entities. The node identification component 118 also creates a feature set for each node. The feature set includes a plurality of features that describe the network-related activities of the entities associated with the node. Exemplary configurations of feature sets are described below in conjunction with the discussion of FIGS. 4 and 5. As a preview of the discussion below, some features in the feature set describe characteristics related to a particular entity's engagement with content that can be communicated over the computing network via websites hosted by the computing network. In some examples, the content corresponds to advertisements that can be communicated over the computing network by websites hosted by respective publishers.

[0023] For each pair of nodes in the training set 112, the edge identification component 120 determines whether the nodes should be connected by an edge. Assume that the node pair under consideration includes a first node associated with a first entity and a second node associated with a second entity. In some implementations, the edge identification component 120 performs this task by comparing a particular feature in the feature set of the first node with a corresponding feature in the feature set of the second node. The edge identification component 120 determines that the nodes should be connected by an edge if the features from the two feature sets satisfy a specified similarity test. As described in more detail below with respect to FIG. 5, in some examples, the edge identification component 120 specifically calculates the cosine similarity between the first vector and the second vector and then determines whether the result of this calculation meets a specified threshold. The first and second vectors represent a portion of the features in the first and second feature sets, respectively. Additionally, when applying its specified similarity test, edge identification component 120 relies on any other evidence to determine whether the first and second nodes under consideration are related. For example, edge identification component 120 may connect a first node and a second node if the corresponding entities belong to a known group, and / or if the entities are members of a single system, and / or if another service identifies the entities as related. In other words, the specified similarity test is not limited to comparing sets of features represented as vectors.

[0024] The label identification component 122 assigns labels to at least some nodes in the training set 112. Each label assigned to a particular node identifies whether the node is associated with network-related abusive activity. The label identification component 122 reaches its conclusion for a particular node based on any of the following: a) assessments made by computer security experts and / or other bot detection services for the particular node; b) behavioral characteristics of the node (particularly if such characteristics exceed a specified threshold); and c) evidence that malware is installed on an entity associated with the particular node. The behavioral characteristics of factor (b) include the conversion rate associated with the node's entity, the click-through rate exhibited by the node's entity, etc. In addition to these considerations, in some implementations, the label identification component 122 applies a label only if the aforementioned evidence has a confidence level above a specified environment-specific value. Some implementations weight the greatest evidence toward factor (a) when making this determination. In some cases, the label identification component 122 cannot associate a label with the node under consideration because the above tests are not met. In that case, the label identification component 122 leaves the node unlabeled.

[0025] The training component 114 iteratively trains the GNN 106 based on the training set 112. Generally, the training component 114 iteratively updates the model weights of the GNN 106. After each training iteration, the training component 114 computes a loss, which represents the difference between the prediction made by the GNN 106 and the ground truth labels in the training set 112. For example, assume the GNN 106 predicts that a particular node under consideration is associated with network-related abuse activity. The training component 114 adjusts the model weights to reinforce this prediction if it matches the ground truth label for the node, and adjusts the model weights to penalize this prediction if it differs from the ground truth label for the node. More formally, in some examples, the training component 114 performs this type of training using stochastic gradient descent combined with backpropagation.

[0026] Referring now to the abuse control system 108, the node classification component 124 uses the GNN 106 to determine a classification result for a particular node under consideration. The classification result indicates whether the particular node is associated with network-related abusive activity with respect to predefined standards for what constitutes network-related abusive activity. In some cases, the node under consideration corresponds to an entity already represented by the training set 112. This node may or may not have been assigned a preliminary label in the training set 112. In other cases, the particular node under consideration corresponds to an entity not represented by the training set 112. For example, this node may correspond to an entity that has recently begun engaging in activity on the computing network 110.

[0027] The node classification component 124 operates by using the trained GNN 106 to map the feature set associated with the node under consideration to a classification result, including any deterministic mapping depending on the particular type of model used by the GNN 106. The step-by-step description of Figure 7 illustrates one way in which a graph neural network propagates node states in a graph to a classification result.

[0028] The action taking component 126 takes any action based on the classification result. For example, suppose the classification result of the node under consideration indicates that the entity associated with the node is the type of node that is engaged in network-related abusive activity (e.g., because it appears to be participating in bot attacks). The action taking component 126 can block the entity from interacting with the selected service (e.g., by adding the entity to a block list), ignore or not consider the entity's clicks, etc.

[0029] FIG. 2 illustrates a training set 202 generated by the training framework 104. The training set 202 includes a set of nodes (represented by circles in FIG. 2) associated with entities that exhibit a specified amount of network-related activity. The training set 202 includes edges connecting some pairs of nodes. For each pair having a first node and a second node, the edge indicates that the feature set of the first node is sufficiently similar to the feature set of the second node with respect to at least some of the features in the feature sets. At least some of the nodes in the training set 202 have labels. A solid star indicates that the node's entity is associated with network-related abusive activity. A hollow star indicates that the node's entity is not associated with network-related abusive behavior. At least some nodes (e.g., node 204) are not classified as having or not having abusive activity.

[0030] The bottom portion 206 of Figure 2 represents the results of the operation of the node classification component 124. Assume the goal in this case is to determine the classification status of the node 204. The node classification component 124 uses the trained GNN 106 to map a feature set associated with the node entity to a classification result. In this particular case, the classification result indicates that the node entity is associated with network-related abusive activity.

[0031] In general, the computing system 102 provides an effective way to detect and respond to collaborative network-related activity that may be considered abusive. In contrast, traditional malware detection engines operate by comparing an entity's signature to the signatures of known malicious entities. Traditional malware detection engines may be successful in detecting an occurrence where an entity is infected with a malicious bot program with a known signature. However, traditional malware detection engines are not successful in detecting instances where the entity's abusive behavior is revealed only by considering the behavioral relationships between different entity-computing networks 220. The computing system 102 is effective in exposing this type of abusive behavior through the use of the GNN 106.

[0032] FIG. 3 illustrates the application of the computing system 102 to an advertising-related platform. Consider first an activity that is not inherently considered abusive. Assume that an advertiser 302 first creates an advertisement 304. An advertising platform 306 receives the advertisement 304 from the advertiser (operation 1) or creates the advertisement on behalf of the advertiser 302. The advertising platform 306 then presents the advertisement 304 on a publisher's website 308 (operation 2) in response to the occurrence of any environment-specific trigger event. A browser application 310 running on a computing device associated with an entity 312 clicks on the advertisement 304 on the website 308 (operation 3). In some cases, this action may direct the entity 312 (operation 4) to a landing page 314 corresponding to another website hosted by the advertiser (operation 5). For example, assume that the advertisement 304 provides an offer for a particular product or service. The landing page 314 is a website that provides additional information about the product or service and potentially enables the user to purchase the product or service. Although not shown in FIG. 3, the advertising platform 306 collects information about the entity's behavior in order to update statistics that the advertiser 302 uses to monitor the success of its advertising campaigns.

[0033] Abuse scenarios attempt to mimic the above interactions. For example, a malicious program may click on advertisement 304 without the knowledge of the user associated with entity 310. In some cases, the malicious program may click on advertisement 304 with a high frequency or timing that reflects some other unusual pattern (with respect to how humans typically click on advertisements). In other cases, the malicious activity of a program running on entity 312 becomes apparent only when considering the behavior of multiple malicious programs acting in concert.

[0034] 4 illustrates feature sets created by the node identification component 118 of FIG. 1 for three nodes associated with three representative entities (402, 404, 406). That is, feature set 408 is associated with entity 402, feature set 410 is associated with entity 404, and feature set 412 is associated with entity 406. The node identification component 118 generates the feature sets based on data collected from the network inspection component 116.

[0035] A non-exhaustive, exemplary set of features includes any of the following features: The term "click" below and variations thereof encompasses when an entity clicks on an item with a mouse device or by touching with a finger / stylus, but more generally includes any mechanism by which a user selects an item. a) Number of Clicks: This feature describes the number of clicks an entity has made in a defined period (e.g., the last 7 days). b) Conversion Rate. This feature describes the percentage of conversions performed by an entity relative to the number of opportunities given to the entity to perform a conversion. For example, a conversion may occur when a user purchases a product associated with an advertisement. The conversion rate describes how many times a user purchases one or more specified products within a defined period of time after being presented with an advertisement for that product. In other advertising campaigns, a conversion may be defined as a subscription, registration, survey completion, etc. c) Time to Click (TTC) Variance. Time to click generally describes the amount of time between when an ad is presented and when an entity clicks on that ad. TTC variance generally describes the statistical dispersion of these TTC measures within a specified time period. Standard deviation (σ) is one possible representation of variance. d) Minute-by-Minute Click Variance. This feature describes the statistical variance between successive clicks on an ad over a gradual time period of one minute. This feature exposes patterns of click behavior that are unnatural and inconsistent with the typical behavior of users who click on ads. Therefore, in some cases, "genuine" entities exhibit a larger variance in clicks per minute than malicious entities. e) Hourly Click Variance: This feature describes the statistical variance between successive clicks on an ad with respect to gradual hourly time variations. f) Variance of mean query entropy. This feature describes the variance of queries that ultimately lead to ad clicks. In some cases, malicious entities draw on a limited set of queries to drive their fraudulent activity. In this case, "real" entities exhibit a greater variance in their query selection compared to malicious entities. g) Median or average daily activity time. A single activity time measure reflects how long a user session lasts. This feature accounts for the median or average of multiple such individual measures within a specified time period (e.g., the last 7 days). h) Distribution of Clicks across Publishers. This portion of the feature set refers to a group of individual features, each describing an entity's engagement with a particular website of a defined group of websites. For example, assume that the group of websites is hosted by a popular content publisher that features advertising. Further, assume that an entity has clicked 500 times across all websites hosted by these publishers in the past seven days. The first feature in this group identifies the normalized number of clicks the entity made on the first publisher (clicks_pub1 / 500), the second feature in this group identifies the normalized number of clicks the entity made on the second publisher (clicks_pub2 / 500), and so on. i) Distribution of Clicks by Advertiser. This component of the feature set refers to a group of individual features, each of which quantifies an entity's engagement with ads by a particular advertiser, where the particular advertiser is one from a predefined list of advertisers. In other words, this group of features is similar to feature (h), but in this case the focus of interest is advertisers rather than publishers. j) Distribution of Clicks in TTC. This component of the feature set refers to a set of individual features, each of which describes a measure of time to click (TTC) for a particular length of time. For example, a first feature describes a normalized measure of the number of times users clicked on an ad within the first 5 seconds after the ad was presented, a second feature describes a normalized measure of the number of times users clicked on an ad within 5-10 seconds after the ad was displayed, and a third feature describes a normalized measure of the number of times users clicked on an ad within 10-15 seconds after the ad was displayed.

[0036] Again, other implementations may introduce additional features other than those mentioned in the above list. Alternatively or additionally, other implementations may omit one or more features mentioned in the above list.

[0037] FIG. 5 illustrates an example of the processing performed by edge identification component 120. Assume that feature vector 502 represents selected features extracted from a feature set for a first entity, and feature vector 504 represents selected features extracted from a feature set for a second entity. In the example of FIG. 5, first vector 502 specifically includes at least three feature groups that describe the behavior of the first entity with respect to three respective distributions. For example, assume that first feature group X1 506 corresponds to the above-mentioned feature (h) corresponding to the number of clicks at the publisher. FIG. 5 shows an expanded view of this feature group X1 506. Second feature group Y1 corresponds to the above-mentioned feature (i) corresponding to the number of clicks at the advertiser. Third feature group Z1 corresponds to the above-mentioned feature (j) corresponding to the number of clicks in TTC (time to click). Similarly, the second vector 504 includes at least three complementary groups of feature groups (X2, Y2, Z2) that describe the behavior of the second entity with respect to the three distributions identified above, e.g., corresponding to the above-mentioned features (h), (i), and (j). Although not depicted in Figure 5, the feature vectors (502, 504) may also include features that describe single-valued information (e.g., corresponding to any of the above-mentioned features (a)-(g)).

[0038] The comparison component 508 measures the similarity between the first vector 502 and the second vector 504 using any distance metric, such as cosine similarity, Euclidean distance, etc. The comparison component 505 concludes that the two entities under consideration are connected if the result of this similarity measure exceeds a specified threshold; otherwise, the two entities are not considered to be connected. The edge identification component 120 generates edges between any two nodes that are determined to be connected.

[0039] B. Exemplary Process 6-8 illustrate exemplary processes illustrating the operation of computing system 102 in Section A in flowchart form. Because the principles underlying the operation of computing system 102 have already been described in Section A, this section will focus on a summary of certain operations. Each flowchart is depicted as a series of operations performed in a particular order. However, the order of these operations is merely representative and may differ in other implementations. Furthermore, any two or more operations described below may be performed in parallel. In one implementation, the blocks shown in the flowcharts relating to processing-related functionality are implemented by hardware logic circuitry described in Section C, which in turn includes one or more processors and / or other logic units that include collections of task-specific logic gates.

[0040] Beginning with FIG. 6, this figure illustrates a process 602 for training a graph neural network (GNN) 106 for use in controlling network activity. At block 604, the training framework 104 identifies a plurality of entities interacting with the computing network 110, each entity determined to exhibit a specified amount of network activity. At block 606, the training framework 104 associates a respective node with the entity. At block 608, the training framework 104 identifies a feature set for each of the plurality of entities, each feature set including features describing the network activity exhibited by the associated entity. At block 610, the training framework 104 identifies a plurality of edges, each of which connects two entities of the plurality of entities having a feature set that satisfies a specified similarity test. At block 612, the training framework 104 labels at least some of the entities, each label indicating whether the corresponding entity is a type of entity that engages in network-related abusive activity in collaboration with other entities. The nodes, edges, and labels define a training set 112 .

[0041] At block 614, the training framework 104 trains a graph neural network (GNN) based on the training set 112 to provide a trained GNN 106. At block 616, the training framework 104 configures the abuse control system 108 to use the trained GNN 106 to detect and address network-related abuse activity in the computing network 110. Block 616 includes installing the GNN 106 in the abuse control system 108.

[0042] A variation of process 602 shown in FIG. 2 includes associating respective nodes with entities engaged in network activity on the computing network 110 (per block 606); identifying a feature set for each of the plurality of entities (per block 608), where each feature set includes features describing characteristics related to engagement by the particular entity with content communicable on the computing network 110 via a website hosted by the computing network 110; identifying a plurality of edges (per block 610), where each edge of the plurality of edges connects two entities of the plurality of entities having a feature set that satisfies a specified similarity test; and labeling at least some of the entities (per block 612), where each label indicates whether the corresponding entity is a bot engaged in collaborative activity with other bots on the computing network 110, the nodes, edges, and labels defining a training set 112; and generating a trained GNN for use by the abuse control system 108. and training (per block 614) the graph neural network based on the training set 112 to provide the graph neural network 106.

[0043] 7 illustrates a process 702 performed by the training component 114 to train the GNN 106. In particular, the process 702 corresponds to steps for training a convolutional graph neural network. However, the training component 114 can train any type of graph neural network. Background information on the general topic of different types of graph neural networks is available from Wu, et al., “A Comprehensive Survey on Graph Neural Networks,” arXiv, Cornell University, arXiv:1901.00596v4[cs.LG], December 4, 2019, 22 pages.

[0044] The training component 114 executes the routine 704 in parallel for all nodes in the training set 112. Figure 7 specifically illustrates the routine 704 executed for a particular node n. More specifically, assume that node n corresponds to node 208 associated with entity identifier ID10, as shown in Figure 2. At any point in time, node 208 has a current state

number

[0045] In block 708, the training component 114 collects the contributions of all nodes connected to node 208. For example, node 208 is connected by three edges to neighboring nodes associated with identifiers ID6, ID11, and ID12. In block 710, the training component 114 sums the current states of the three neighboring nodes and then adds the resulting sum to the current state of node 208. Other implementations perform other types of combinations besides summation, such as multiplication or concatenation. Then, in block 712, the training component 114 multiplies this result by the weight W of the machine-learned model, divides the result by the number of states added together (four states in the example of node 208), and transforms the result using any activation function, such as a ReLU activation function. This results in an updated state 714 for the node 208 under consideration.

number

[0046] In block 716, the training component 114 computes a loss metric based on the current metrics and adjusts the model weights of the GNN 106 based on the loss metric. In the example of Figure 2, the training component 114 computes a loss metric that measures the accuracy with which the GNN 106 predicts node labels at its current state of training with reference to known ground truth labels identified by the label identification component 122.

[0047] Loop 718 indicates that the entire process 702 is repeated one or more additional times. As the iterations are performed, each node in the graph gains deeper and deeper knowledge with respect to the knowledge contributed by other nodes in the graph. The "depth" of knowledge refers to the distance, in terms of the number of node "hops," between the node under consideration and the nodes that contribute information to the node under consideration.

[0048] FIG. 8 shows a process 802 illustrating one mode of operation of the Abuse Control System 108. In block 804, the Abuse Control System 108 identifies a particular feature set associated with a particular entity under consideration whose classification is unknown. The feature set includes features that describe the particular entity's network activity on the computing network 110. In block 806, the Abuse Control System 108 uses the GNN 106 to generate a result indicating whether the particular entity is a type of entity that is engaged in network-related abuse activity in collaboration with other network entities that are engaged in network-related abuse activity, with respect to a predefined standard defining what constitutes network-related abuse activity. In block 808, the Abuse Control System 108 takes action to control the particular entity's interactions with the computing network 110 based on the result generated by the GNN 106.

[0049] C. Representative Computing Functions 9 illustrates an example of a computing device 902 that, in some implementations, may be used to implement aspects of the training framework 104 and / or the abuse control system 108. The computing device 902 includes a set of user devices 904 coupled to a set of servers 906 via a computing network 908. Each user device corresponds to any type of computing device, including any of a desktop computing device, a laptop computing device, any type of handheld computing device (e.g., a smartphone or tablet-type computing device), a mixed reality device, a wearable computing device, an Internet of Things (IoT) device, a gaming system, a media device, an in-vehicle computing system, any type of robotic computing system, a computing system in a manufacturing system, etc. In some implementations, the computing network 908 is implemented as a local area network, a wide area network (e.g., the Internet), one or more point-to-point links, or any combination thereof.

[0050] The dashed boxes in Figure 9 indicate that the functionality of the training framework 104 and the abuse control system 108 can be spread across the user devices 904 and / or the servers 906 in any manner. For example, in some cases, each user device with which a user interacts implements a local version of the abuse control system 108. Here, the servers 906 are not involved in the operation of the abuse control system 108 other than downloading trained GNNs 106 that can be trained by the training framework 104 running on the servers 906. In other implementations, one or more servers 906 implement the entire abuse control system 108. In other cases, the functionality associated with the abuse control system 108 is distributed between the servers 906 and each user device in any manner. Similarly, components of the training framework 104 can be distributed between the user devices 904 and the servers 906 in various ways.

[0051] Figure 10 illustrates a computing system 1002 that, in some implementations, may be used to implement any aspect of the mechanisms illustrated in the previous figures. For example, in some implementations, a computing system 1002 of the type illustrated in Figure 10 may be used to implement any user computing device or any server illustrated in Figure 9. In all cases, computing system 1002 represents a physical and tangible processing mechanism.

[0052] The computing system 1002 includes a processing system 1004 that includes one or more processors. The processors may include one or more central processing units (CPUs), one or more graphics processing units (GPUs), one or more application specific integrated circuits (ASICs), one or more neural processing units (NPUs), etc. More specifically, any processor may represent a general-purpose processing unit or an application-specific processor unit.

[0053] The computing system 1002 also includes a computer-readable storage medium 1006, which represents one or more computer-readable media hardware units. The computer-readable storage medium 1006 holds any type of information 1008, such as machine-readable instructions, settings, and / or data. For example, in some implementations, the computer-readable storage medium 1006 includes one or more solid-state devices, one or more magnetic hard disks, one or more optical disks, magnetic tape, etc. Any example of the computer-readable storage medium 1006 may use any technology for storing and retrieving information. Furthermore, any example of the computer-readable storage medium 1006 may represent a fixed or removable unit of the computing system 1002. Furthermore, any example of the computer-readable storage medium 1006 may provide volatile and / or non-volatile retention of information.

[0054] More generally, any of the storage resources described herein, or any combination of storage resources, shall be considered a computer-readable medium. Often, a computer-readable medium represents some form of physical and tangible entity. The term computer-readable medium also encompasses propagated signals transmitted or received, for example, through physical conduits and / or the air or other wireless media. However, the specific terms "computer-readable storage medium" or "storage device" explicitly exclude propagated signals in transmission per se, while including all other forms of computer-readable media.

[0055] Computing system 1002 utilizes any instance of computer-readable storage medium 1006 in different ways. For example, in some implementations, any instance of computer-readable storage medium 1006 represents a hardware memory unit (such as a random access memory (RAM)) for storing information during execution of a program by computing system 1002, and / or a hardware storage unit (such as a hard disk) for holding / archiving information in a more permanent manner. In the latter case, computing system 1002 also includes one or more drive mechanisms 1010 (such as a hard drive mechanism) for storing and retrieving information from the instance of computer-readable storage medium 1006.

[0056] In some implementations, the computing system 1002 performs any of the above functions when the processing system 1004 executes computer-readable instructions stored on any instance of the computer-readable storage medium 1006. For example, in some implementations, the computing system 1002 executes computer-readable instructions to perform each block of the process described in Section B. Figure 10 generally illustrates that the hardware logic circuitry 1012 includes any combination of the processing system 1004 and the computer-readable storage medium 1006.

[0057] Additionally or alternatively, processing system 1004 may include one or more other configurable logic units that perform operations using a collection of logic gates. For example, in some implementations, processing system 1004 includes a fixed configuration of hardware logic gates that are created and configured, e.g., at the time of manufacture and cannot be changed thereafter. Additionally or alternatively, processing system 1004 may include a collection of programmable hardware logic gates that are configured to perform different application-specific tasks. Devices in the latter category include programmable array logic devices (PALs), general purpose array logic devices (GALs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), etc. In these implementations, processing system 1004 effectively incorporates a memory device that stores computer-readable instructions, provided that the configurable logic units are configured to execute, and thus embody or store, these instructions.

[0058] In some cases (e.g., when computing system 1002 represents a user computing device), computing system 1002 also includes an input / output interface 1014 for receiving various inputs (via input devices 1016) and providing various outputs (via output devices 1018). Exemplary input devices include a keyboard device, a mouse input device, a touchscreen input device, a digitizing pad, one or more still image cameras, one or more video cameras, one or more depth camera systems, one or more microphones, a voice recognition mechanism, any position-determining device (e.g., a GPS device), any motion-detecting mechanism (e.g., an accelerometer and / or gyroscope), etc. In some implementations, a specific output mechanism includes a display device 1020 and an associated graphical user interface presentation (GUI) 1022. The display device 1020 corresponds to a liquid crystal display device, a light-emitting diode display (LED) device, a cathode ray tube device, a projection mechanism, etc. Other output devices include a printer, one or more speakers, a haptic output mechanism, an archiving mechanism (for storing output information), etc. In some implementations, the computing system 1002 also includes one or more network interfaces 1024 for exchanging data with other devices via one or more communication conduits 1026. One or more communication buses 1028 communicatively couple the above-mentioned units together.

[0059] The communications conduit 1026 may be implemented in any manner, for example, by a local area computing network, a wide area computing network (e.g., the Internet), a point-to-point connection, or any combination thereof. The communications conduit 1026 may include any combination of wired links, wireless links, routers, gateway functions, name servers, etc., as defined by any protocol or combination of protocols.

[0060] 10 illustrates computing system 1002 as being comprised of a discrete collection of separate units. In some cases, the collection of units corresponds to individual hardware units provided in a computing device chassis having any form factor. Exemplary form factors are shown at the bottom of FIG. 10. In other cases, computing system 1002 includes a hardware logic unit that integrates the functionality of two or more units shown in FIG. 1. For example, in some implementations, computing system 1002 includes a system-on-chip (SoC or SOC) corresponding to an integrated circuit that combines the functionality of two or more units shown in FIG. 10.

[0061] The following summary provides a series of illustrative examples of the techniques described herein.

[0062] (A1) According to a first aspect, a method (e.g., 602) for training a graph neural network (e.g., 106) for use in controlling interactions with a computing network (e.g., 110) is described. The method includes identifying a plurality of entities interacting with a computing network (e.g., 604), each entity determined to exhibit a specified amount of network activity; associating respective nodes with the entities (e.g., 606); identifying feature sets for each of the plurality of entities (e.g., 608), each feature set including features describing the network activity exhibited by the associated entity; identifying a plurality of edges (e.g., 610), each of the plurality of edges connecting two entities of the plurality of entities having feature sets that satisfy a specified similarity test; and labeling at least some of the entities (e.g., 612), each label indicating whether the corresponding entity is a type of entity engaging in network-related abusive activity in collaboration with other entities engaging in network-related abusive activity, with respect to a given standard defining what constitutes network-related abusive activity. The nodes, edges, and labels define a training set (e.g., 112). The method further includes training (e.g., 614) a graph neural network based on the training set to provide a trained graph neural network (e.g., 106), and configuring (e.g., 616) an abuse control system (e.g., 108) to detect and address network-related abuse activity in the computing network using the trained graph neural network.

[0063] (A2) According to some implementations of the method described in A1, each of the entities is associated with any combination of a particular device identifier, and / or a particular network address identifier, and / or a particular network cookie.

[0064] (A3) According to some implementations of the method described in A1 or A2, the trained neural network is configured to detect network-related abusive activity with respect to different ways of defining entities.

[0065] (A4) According to some implementations of the method described in any one of A1 to A3, a particular feature set associated with a particular entity describes characteristics regarding the particular entity's engagement with content communicable on the computing network via a website hosted by the computing network.

[0066] (A5) According to some implementations of the method described in A4, the content corresponds to advertisements that can be conveyed over the computing network by websites hosted by respective publishers.

[0067] (A6) According to some implementations of the methods described in any one of A1 to A5, the particular feature set associated with a particular entity includes at least some features that describe characteristics of choices made by the particular entity.

[0068] (A7) According to some implementations of the methods described in any one of A1 to A6, the particular feature set associated with the particular entity includes at least some features that describe characteristics of the query-related behavior of the particular entity preceding a selection made by the particular entity.

[0069] (A8) According to some implementations of the methods described in any one of A1 to A7, the particular feature set associated with the first entity includes at least some features that describe the distribution of network activity by the first entity with respect to a particular category of activity.

[0070] (A9) According to some implementations of the method described in A8, the distribution of activity describes a selection by the first entity of content hosted by each different website.

[0071] (A10) According to some implementations of the method described in A8 or A9, the distribution of activity also describes when particular entities made their selections with respect to different lengths of time.

[0072] (A11) According to some implementation forms of the method described in any one of A8 to A10, a particular entity pair includes a first entity and a second entity, and the first entity is determined to be connected to the second entity if a feature describing a distribution of network activity by the first entity and a feature describing a distribution of network activity by the second entity satisfy a specified similarity test.

[0073] (A12) According to some implementations of the methods described in any one of A1 to A10, the particular entity pair includes a first entity associated with a first set of features represented as a first vector and a second entity associated with a second set of features represented as a second vector, and the particular entity pair is determined to be connected by an edge if a distance between the first vector and the second vector satisfies a specified similarity test.

[0074] (A13) According to some implementations of the methods described in any one of A1 to A12, a particular entity engaging in network-related abusive activity is associated with a bot configured to solicit multiple entities, including the particular entity, to engage in network-related abusive activity.

[0075] (A14) According to some implementations of the methods described in any one of A1 to A13, the abuse control system is configured to identify a particular set of features associated with a particular entity under consideration whose abuse classification is unknown, use a graph neural network to generate results indicating whether the particular entity is a type of entity that engages in network-related abuse activity, and take action to control the particular entity's interaction with the computing network based on the results generated by the graph neural network.

[0076] (B1) According to a second aspect, a method (e.g., 802) for controlling network activity is described. The method includes identifying (e.g., 804) a particular feature set associated with a particular entity under consideration, the classification of which is unknown, the feature set including features describing the particular entity's network activity on a computing network (e.g., 110), using a graph neural network (e.g., 106) (e.g., 806) to generate results indicating whether the particular entity is a type of entity engaging in network-related abusive activity in collaboration with other network entities engaging in network-related abusive activity with respect to a predefined standard defining what constitutes network-related abusive activity, and taking action (e.g., 808) to control the particular entity's interactions with the computing network based on the results generated by the graph neural network. The graph neural network is trained by a training framework (e.g., 104) by associating (e.g., 606) each node with a plurality of entities; identifying (608) a feature set for each of the plurality of entities, where each feature set includes features that describe network activity exhibited by the associated entity; identifying a plurality of edges, where each edge of the plurality of edges connects two entities of the plurality of entities having a feature set that satisfies a specified similarity test; labeling (e.g., 612) at least some of the entities, where each label indicates whether the corresponding entity is a type of entity that engages in network-related abuse activity, the nodes, edges, and labels defining a training set (e.g., 112); and training (e.g., 614) the graph neural network based on the training set.

[0077] (C1) According to a third aspect, another method (e.g., 802) for training a graph neural network (e.g., 106) is described. The method includes associating (e.g., 606) entities engaged in network activity on a computing network (e.g., 110) with respective nodes; identifying (e.g., 608) a feature set for each of the plurality of entities, where each feature set includes features that describe characteristics related to engagement by the particular entity with content communicable on the computing network via a website hosted by the computing network; identifying (e.g., 610) a plurality of edges, where each edge of the plurality of edges connects two entities of the plurality of entities having a feature set that satisfies a specified similarity test; labeling (e.g., 612) at least some of the entities, where each label indicates whether the corresponding entity is a bot engaged in collaborative activity with other bots on the computing network, the nodes, edges, and labels defining a training set; and training (e.g., 614) a graph neural network based on the training set to provide a trained graph neural network (e.g., 106) for use by an abuse control system (e.g., 108).

[0078] In yet another aspect, some implementations of the techniques described herein include a computing system (e.g., computing system 1002) that includes a processing system (e.g., processing system 1004) having a processor. The computing system also includes a storage device (e.g., computer-readable storage medium 1006) for storing computer-readable instructions (e.g., information 1008) that, when executed by the processing system, perform any of the methods described herein (e.g., any one of methods A1-A14, B1, or C1).

[0079] In yet another aspect, some implementations of the techniques described herein include a computer-readable storage medium (e.g., computer-readable storage medium 1006) for storing computer-readable instructions (e.g., information 1008). A processing system (e.g., processing system 1004) executes the computer-readable instructions to perform any of the operations described herein (e.g., the operations in any one of methods A1-A14, B1, or C1).

[0080] More generally, any of the individual elements and steps described herein may be combined, for example, into any logically consistent permutation or subset. Furthermore, any such combination may be manifested as a method, device, system, computer-readable storage medium, data structure, article of manufacture, graphical user interface presentation, etc. The technology may also be expressed in the claims as a series of means-plus-formatting elements, but this formatting should not be considered invoked unless the phrase "means for" is explicitly used in the claims.

[0081] With respect to terminology used in this description, the phrase "configured to" encompasses various physical and tangible mechanisms for performing a specified operation. The mechanism is configurable to perform the operation using hardware logic 1012 of Section C. The term "logic" similarly encompasses various physical and tangible mechanisms for performing a task. For example, each processing-related operation shown in the flowchart of Section B corresponds to a logical component for performing that operation.

[0082] This description may identify one or more features as "optional." Such a statement should not be construed as an exhaustive list of features that are considered optional. In general, any feature shall be considered optional unless otherwise stated, even if not explicitly stated in the text. Furthermore, any reference to a single entity is not intended to exclude the use of multiple such entities. Similarly, a description of multiple entities herein is not intended to exclude the use of a single entity. Thus, a description that an apparatus or method has feature X does not exclude the possibility that it may have additional features. Furthermore, any features described as performing a specified function or as alternative ways of implementing a specified mechanism may be combined together in any combination unless otherwise noted. Furthermore, the terms "plurality" or "multiple," or the plural form of any term (unless explicitly using "plurality" or "multiple"), refer to two or more items and do not necessarily refer to "all" items of a particular type unless explicitly specified. The term "at least one of" refers to one or more items. Reference to a singular item is not intended to exclude the inclusion of multiple items unless specifically stated otherwise, absent the explicit recitation "at least one of." Furthermore, descriptors such as "first," "second," and "third" are used to distinguish between different items and do not imply any ordering among the items unless specifically stated otherwise. The phrase "A and / or B" means A, or B, or A and B. Furthermore, the terms "comprising," "including," and "having" are open-ended terms used to identify at least a portion of a larger whole, but not necessarily all parts of the whole. A "set" includes zero members, one member, or multiple members. Finally, the term "exemplary" or "illustrative" refers to one implementation among potentially many implementations.

[0083] Finally, the functionality described herein may employ various mechanisms to ensure that all user data is handled in a manner consistent with applicable laws, societal norms, and individual user expectations and preferences. For example, the functionality may be configurable to allow users to explicitly opt in to (and then explicitly opt out of) the functionality. The functionality may also be configurable to provide appropriate security mechanisms (e.g., data sanitization, encryption, and / or password protection mechanisms) to ensure the privacy of user data.

[0084] Furthermore, this description has presented various concepts in the context of example problems or problems. This style of description is not intended to suggest that others have assessed and / or clarified the problem or problem in the manner specified herein. Furthermore, this style of description is not intended to suggest that the claimed subject matter is limited to solving the specified problem or problem; i.e., the claimed subject matter may be applied in the context of problems or problems other than those described herein.

[0085] Although the patented subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the patented subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. 1. A method for training a graph neural network for use in controlling interaction with a computing network, comprising: identifying a plurality of entities that interact with the computing network, each entity determined to exhibit a predetermined amount of network activity; associating each node with said entity; identifying a feature set for each of the plurality of entities, each feature set including features describing network activity exhibited by the associated entity; identifying a plurality of edges, each edge of the plurality of edges connecting two entities of the plurality of entities having a feature set that satisfies a specified similarity test; labeling at least some of the entities, each label indicating whether the corresponding entity is a type of entity that engages in network-related abusive activity in collaboration with other entities that engage in network-related abusive activity with respect to a given standard defining what constitutes network-related abusive activity, the nodes, edges, and labels defining a training set; training the graph neural network based on the training set to provide a trained graph neural network; configuring an abuse control system to use the trained graph neural network to detect and address network-related abuse activity in the computing network; and A method comprising:

2. Each of the entities a specific device identifier, and / or a specific network address identifier, and / or Certain network cookies The method of claim 1 , wherein the method is associated with any combination of:

3. 10. The method of claim 1, wherein a particular feature set associated with a particular entity describes characteristics regarding the particular entity's engagement with content communicable on the computing network via a website hosted by the computing network.

4. The method of claim 1 , wherein a particular feature set associated with a particular entity includes at least some features that describe characteristics of choices made by the particular entity.

5. 2. The method of claim 1, wherein a particular feature set associated with a particular entity includes at least some features that describe characteristics of the particular entity's query-related behavior preceding a selection made by the particular entity.

6. The method of claim 1 , wherein a particular feature set associated with a first entity includes at least some features that describe a distribution of network activity by the first entity with respect to a particular category of activity.

7. The method of claim 6 , wherein the distribution of activity describes the first entity's selection of content hosted by each different website.

8. The method of claim 6 , wherein the distribution of activity describes when the particular entity made a selection for each different length of time.

9. 7. The method of claim 6, wherein a particular pair of entities includes the first entity and a second entity, and the first entity is determined to be connected to the second entity if a feature describing the distribution of network activity by the first entity and a feature describing the distribution of network activity by the second entity satisfy the specified similarity test.

10. 2. The method of claim 1, wherein a particular pair of entities includes a first entity associated with a first set of features represented as a first vector and a second entity associated with a second set of features represented as a second vector, and the particular pair of entities is determined to be connected by an edge if the distance between the first vector and the second vector satisfies the specified similarity test.

11. 2. The method of claim 1, wherein a particular entity engaging in the network-related abusive activity is associated with a bot configured to solicit multiple entities, including the particular entity, to engage in the network-related abusive activity.

12. 2. The method of claim 1, wherein the abuse control system is configured to: identify a particular set of features associated with a particular entity under consideration whose abuse classification is unknown; use the graph neural network to generate results indicating whether the particular entity is a type of entity that engages in abuse activity related to the network; and take action to control interactions of the particular entity with the computing network based on the results generated by the graph neural network.

13. A processing system having a processor and storage device configured to perform the method of any one of claims 1 to 12.

14. A computer readable storage medium for storing computer readable instructions which, when executed by a processing system, perform the method of any one of claims 1 to 12.

15. 1. A computing system for controlling network activity, comprising: a processing system including a processor; a network inspection mechanism for detecting network activity by a plurality of entities on a computing network; a memory device for storing machine-readable instructions that, when executed by said processing system, perform operations; a computing system including: identifying a particular feature set associated with a particular entity under consideration whose classification is unknown, the feature set including features that describe the network activity of the particular entity on the computing network; using a graph neural network to generate a result indicating whether the particular entity is a type of entity that engages in network-related abusive activity in collaboration with other network entities that engage in network-related abusive activity with respect to a predefined standard that defines what constitutes network-related abusive activity; taking action to control interactions with the computing network by the particular entity based on the results produced by the graph neural network; Including, The graph neural network is trained by a training framework. associating each node with said plurality of entities; identifying a feature set for each of the plurality of entities, each feature set including features describing network activity exhibited by the associated entity; identifying a plurality of edges, each edge of the plurality of edges connecting two entities of the plurality of entities having a set of features that satisfy a specified similarity test; labeling at least some of the entities, each label indicating whether a corresponding entity is a type of entity that engages in abusive activity related to the network, the nodes, edges, and labels defining a training set; training the graph neural network based on the training set; A computing system that is trained by performing

Images