A machine account detection method, system, device and storage medium

Abstract

The present invention discloses a machine account detection method, system, device and storage medium. The method constructs a heterogeneous graph by crawling various types of data from a social platform, uses reinforcement learning to search for a graph neural network architecture based on the local structural information of the social network graph where the target account to be detected is located, and uses the graph neural network architecture obtained by the search to aggregate the data of neighbor nodes in the heterogeneous graph (social network graph) to obtain a representation vector of the target user and perform machine account detection. The method utilizes information such as interactions and relationships between accounts in the social platform, and uses a suitable graph neural network architecture to aggregate the rich semantic information provided by the heterogeneous graph, fully retaining the account's own discriminable pattern while also aggregating the classification information provided by the neighbors, greatly improving the detection capability of machine accounts.

Description

Technical Field

[0001] The present invention relates to the technical field of social platform machine account detection, and in particular to a machine account detection method, system, device and storage medium. Background Art

[0002] Since 2016, the third generation of machine accounts has been discovered. These accounts have a deeper mix of human operation and automation, and even steal information from other real accounts. They use artificial intelligence technology to generate highly credible text or pictures. Their behavior is more like that of real human accounts, making machine accounts more difficult to detect and identify.

[0003] Many patents for machine account detection methods have been proposed. The Chinese patent with authorization announcement number CN102571485B, "A method for identifying robot users on the Weibo platform," distinguishes between normal accounts and robot accounts by analyzing the social relationships of user friends. The Chinese patent with authorization announcement number CN102571484B, "A method for detecting and finding online water armies," uses the posting and following strategies of honeypot accounts to collect accounts, detects robot accounts from the collected accounts, and uses the detected robot accounts to find more robot accounts and water armies. The Chinese patent with authorization announcement number CN104518930B, "A method for simultaneous detection of abnormal users and messages for Weibo," starts from two types of entities, users and messages, and uses the non-negative matrix tri-factorization method to simultaneously give the partition indicator matrix of users and messages to detect abnormal users and messages. Chinese patent application publication number CN106296422A, "A Method for Detecting Spam Users on Social Networks by Integrating Multiple Algorithms," uses a clustering algorithm that combines the K-means algorithm and the DBSCAN algorithm (a density-based clustering algorithm) to cluster accounts on social networks. Data near the cluster centers and cluster boundaries are used to train a support vector machine classifier, which is then used to detect machine accounts. Chinese patent application publication number CN106682118A, "A Method for Detecting Fake Followers on Social Networks Based on Web Crawler and Machine Learning," uses a web crawler to automatically acquire Weibo and other social network user data, divides it into training and test sets, uses the training set to train a support vector machine classifier, and uses the trained classifier to detect social network users.

[0004] However, these methods suffer from the following technical issues: 1) The specific architecture of the model is closely tied to the dataset, and the model design is highly dependent on the dataset. This requires expertise in the relevant field and significant manual effort to refine, raising the bar for model design. 2) They ignore the rich semantic information provided by social network relationships such as following and friendships, as well as actions such as replies and comments, reducing detection effectiveness. 3) Account structure (such as number of friends and followers) in social network graphs follows a power-law distribution, and different accounts vary greatly. Fixed model architectures are ineffective in detecting highly diverse machine accounts. Summary of the Invention

[0005] The purpose of the present invention is to provide a machine account detection method, system, device and storage medium, which utilizes information such as interactions and relationships between accounts on social platforms, greatly improving the detection capabilities of machine accounts.

[0006] The purpose of the present invention is achieved through the following technical solutions:

[0007] A machine account detection method, comprising:

[0008] Obtain accounts on the social platform, the content published by each account, and the social network relationships between accounts. Treat the accounts and the content published by each account as different types of nodes, and the social network relationships between accounts and the relationships between accounts and their published content as edges of the corresponding nodes. Construct a heterogeneous graph, extract features from the account-related information and the content-related information respectively, and use the extracted features as the node features of the corresponding nodes.

[0009] For the target node to be detected, neighbor nodes of the corresponding initial neighbor order are extracted from the heterogeneous graph with each target node as the center according to the preset initial neighbor order to construct a corresponding subgraph, and the initial subgraph representation vector is determined using the features of all nodes in the corresponding subgraph; wherein, the target nodes are all nodes corresponding to the account;

[0010] Using reinforcement learning methods combined with all initial subgraph representation vectors, we search for the optimal neighbor order and the optimal number of layers of the graph neural network corresponding to each target node;

[0011] A graph neural network with a residual structure is constructed according to different optimal numbers of layers. For each target node, the neighbor nodes of the corresponding optimal neighbor order are extracted from the heterogeneous graph with the corresponding target node as the center according to the corresponding optimal neighbor order to construct an optimal subgraph. The trained graph neural network with the corresponding optimal number of layers is combined with the attention mechanism to obtain the final representation vector of the optimal subgraph corresponding to the corresponding target node, and then classification detection is performed to obtain the detection result of the corresponding target node.

[0012] A machine account detection system, comprising:

[0013] An information acquisition and heterogeneous graph construction unit is used to obtain accounts on the social platform, the content published by each account, and the social network relationships between accounts. The accounts and the content published by each account are treated as different types of nodes, and the social network relationships between accounts and the relationships between accounts and their published content are used as edges of corresponding nodes to construct a heterogeneous graph. Feature extraction is performed on the relevant information of the account and the relevant information of the content respectively, and the extracted features are used as the node features of the corresponding nodes.

[0014] A subgraph initialization unit, for a target node to be detected, extracts neighbor nodes of the corresponding initial neighbor order from the heterogeneous graph with each target node as the center according to a preset initial neighbor order to construct a corresponding subgraph, and determines an initial subgraph representation vector using all node features in the corresponding subgraph; wherein the target nodes are all account corresponding nodes;

[0015] Reinforcement learning and optimal parameter acquisition unit, which is used to use reinforcement learning methods to combine all initial subgraph representation vectors to search for the optimal neighbor order and the optimal number of layers of the graph neural network corresponding to each target node;

[0016] The network construction and detection unit is used to build a graph neural network with a residual structure according to different optimal numbers of layers; for each target node, according to the corresponding optimal neighbor order, the neighbor nodes of the corresponding optimal neighbor order are extracted from the heterogeneous graph with the corresponding target node as the center to construct an optimal subgraph, and the trained graph neural network with the corresponding optimal number of layers is combined with the attention mechanism to obtain the final representation vector of the optimal subgraph corresponding to the corresponding target node, and then classification detection is performed to obtain the detection result of the corresponding target node.

[0017] A processing device comprising: one or more processors; a memory for storing one or more programs;

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned method.

[0019] A readable storage medium stores a computer program, which implements the aforementioned method when the computer program is executed by a processor.

[0020] It can be seen from the technical solution provided by the present invention that a heterogeneous graph is constructed by crawling various types of data from the social platform, and reinforcement learning is used to search for a graph neural network architecture based on the local structural information of the social network graph where the target account to be detected is located. The graph neural network architecture obtained by the search is used to aggregate the data of the neighbor nodes in the heterogeneous graph (social network graph) to obtain the representation vector of the target user and perform machine account detection. This method utilizes the interaction, relationship and other information between accounts in the social platform, and uses a suitable graph neural network architecture to aggregate the rich semantic information provided by the heterogeneous graph, fully retaining the account's own discriminable pattern while also aggregating the classification information provided by the neighbors, greatly improving the detection capability of machine accounts. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 A flowchart of a machine account detection method provided by an embodiment of the present invention;

[0023] Figure 2 Schematic diagram of the attention mechanism provided by an embodiment of the present invention;

[0024] Figure 3 A reinforcement learning graph neural network architecture search framework diagram provided by an embodiment of the present invention;

[0025] Figure 4 A schematic diagram of a machine account detection system provided by an embodiment of the present invention;

[0026] Figure 5 A schematic diagram of a processing device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The following is a clear and complete description of the technical solutions in the embodiments of the present invention, in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0028] First, the following terms may be used in this article:

[0029] The term “and / or” means that either or both of them can be realized at the same time. For example, X and / or Y includes both “X” or “Y” and “X and Y”.

[0030] The terms "include," "comprises," "contains," "has," or other similar expressions should be interpreted as non-exclusive. For example, "including certain technical features (such as raw materials, components, ingredients, carriers, dosage forms, materials, dimensions, parts, components, mechanisms, devices, steps, procedures, methods, reaction conditions, processing conditions, parameters, algorithms, signals, data, products, or manufactured articles, etc.) should be interpreted as including not only the technical features explicitly listed, but also other technical features known in the art that are not explicitly listed.

[0031] The following describes in detail the machine account detection method, system, device, and storage medium provided by the present invention. Any information not described in detail in the embodiments of the present invention is prior art known to those skilled in the art. For any unspecified conditions in the embodiments of the present invention, the conditions generally used in the art or recommended by the manufacturer shall prevail.

[0032] Example 1

[0033] An embodiment of the present invention provides a machine account detection method. When detecting a machine account, a heterogeneous graph is constructed using the association relationships such as attention, friends, replies, and comments on the social platform itself, as well as entities extracted from the content published by the account. The local structure of the target node in the heterogeneous graph is used as a search basis, and reinforcement learning is used to search the optimal local structure (i.e., the optimal neighbor order) and the architecture of the graph neural network to obtain a suitable graph neural network structure. The information between accounts is aggregated to obtain the representation vector of the account, and classification detection is performed. Since a suitable neural network architecture is used to aggregate the rich semantic information provided by the heterogeneous information network, the discriminable pattern of the account itself is fully retained while the classification information provided by the neighbors is also aggregated, thereby improving the machine account detection effect. Figure 1 As shown, the method mainly includes the following steps:

[0034] Step 1: Obtain the accounts on the social platform, the content published by each account, and the social network relationships between the accounts. Treat the accounts and the content published by each account as different types of nodes, and the social network relationships between the accounts and the relationships between the accounts and their published content as the edges of the corresponding nodes. Construct a heterogeneous graph, and perform feature extraction on the account-related information (corresponding user information and published content information) and the content-related information (content text). The extracted features are used as the node features of the corresponding nodes.

[0035] In an embodiment of the present invention, the present invention crawls account data, account friend relationships and friend account data, and published content (including articles, replies, comments, and other data) from a social platform to construct a heterogeneous graph.

[0036] In an embodiment of the present invention, the constructed heterogeneous graph (heterogeneous graph) includes multiple different types of nodes, and the relationships corresponding to the edges between different types of nodes are also different. For example, the edges between the nodes corresponding to the accounts are social relationships, and the edges between the nodes corresponding to the content and the nodes corresponding to the accounts are subordinate relationships; by taking the content as the nodes, extracting the nodes from the heterogeneous graph according to the defined meta-path (the meta-path can contain certain semantic information) to perform information aggregation, it is possible to discover machine accounts that are useful for detection.

[0037] In this embodiment of the present invention, account-related information primarily includes: user information corresponding to the account and published content, from which features are extracted to form node features for the account-related node. Content-related information primarily refers to relevant text information, from which relevant features are extracted to form node features for the content-related node. Node features for different nodes have the same dimensions.

[0038] Step 2: For the target node to be detected, extract the neighbor nodes of the corresponding initial neighbor order from the heterogeneous graph with each target node as the center according to the preset initial neighbor order to construct the corresponding subgraph, and use the features of all nodes in the corresponding subgraph to determine the initial subgraph representation vector; wherein, the target nodes are all account corresponding nodes.

[0039] In an embodiment of the present invention, the initial neighbor order can be set to 1, and the corresponding first-order neighbor nodes are extracted from the heterogeneous graph with each target node as the center to construct a corresponding subgraph, and the initial subgraph representation vector is determined using the features of all nodes in the corresponding subgraph.

[0040] For example, the subgraph corresponding to the i-th target node is recorded as The initial subgraph representation vector is expressed as:

[0041] Among them, v j For subgraph The feature of the jth node in V i It is a subgraph The set of nodes contained in , e(.) represents the encoding function (for example, mean pooling, etc.), that is, the node features are encoded to obtain the initial subgraph representation vector.

[0042] Step 3: Use reinforcement learning methods to combine all initial subgraph representation vectors to search for the optimal neighbor order and the optimal number of layers of the graph neural network corresponding to each target node.

[0043] In an embodiment of the present invention, all initial subgraph representation vectors are used as input and a reinforcement learning method is used to train the intelligent agent. After the intelligent agent is trained, all initial subgraph representation vectors are input into the intelligent agent respectively to obtain the optimal neighbor order corresponding to each target node and the optimal number of layers of the graph neural network.

[0044] The training process is as follows: a subgraph is randomly selected at the initial time step, and its initial subgraph representation vector is used as the state at the initial time step. Afterwards, the transition probability is calculated based on the connection relationship between the corresponding subgraph of the previous time step and other subgraphs, and the corresponding subgraph of the next time step is determined according to the transition probability, and the initial subgraph representation vector of the corresponding subgraph is used as the state of the next time step. Under the state of the current time step, different actions are selected from the action space, and the corresponding reward values ​​(environmental feedback) are obtained after executing different actions and the corresponding state-action values ​​are calculated. The final action is then selected in combination with the strategy function, and the record data of the current time step, the final action, the reward value and the state of the next time step are combined to form the record data of the current time step; the record data of multiple time steps are used to train the agent until convergence, and the training of the agent is completed; wherein, each action contains a sub-action for selecting the neighbor order value and a sub-action for selecting the layer value of the graph neural network. Specifically:

[0045] The state of the next time step is determined as follows: for the current time step t, the number of connections (i.e., the number of edges) between each other subgraph and the subgraph corresponding to the current time step t accounts for the proportion of the number of connections between the subgraph corresponding to the current time step t and the external nodes as the transition probability P i , and then according to the transition probability P i Select a subgraph from all other subgraphs and use the initial subgraph representation vector of the selected subgraph as the state s in the next time step t+1 t+1 .

[0046] The final action selection method is: each action a in the current time step t t By binary action Composition, by executing the first sub-action Get the neighbor order value by executing the second sub-action Get the layer values ​​of the graph neural network and select different sub-actions from the given action space With sub-actions After executing them separately in the state of the current time step, the reward feedback of each action is determined according to the classification accuracy of the subgraph corresponding to the current time step t, and a reward value is determined based on the reward feedback. Combined with the reward value and the cumulative state-action value from the given next time step t+1 to the maximum time step T, the state-action value obtained after executing each action at the current time step t is determined. The strategy function is used to determine whether the current time step t meets the set conditions. If so, an action is randomly selected as the final action of the current time step t. If not, the action corresponding to the maximum state-action value is selected as the final action of the current time step t.

[0047] Step 4. Build a graph neural network with a residual structure according to different optimal numbers of layers. For each target node, extract neighbor nodes of the corresponding optimal neighbor order from the heterogeneous graph with the corresponding target node as the center according to the corresponding optimal neighbor order to construct an optimal subgraph. Use the trained graph neural network with the corresponding optimal number of layers combined with the attention mechanism to obtain the final representation vector of the optimal subgraph corresponding to the corresponding target node, and then perform classification detection to obtain the detection result of the corresponding target node.

[0048] In the embodiment of the present invention, for the i-th target node, the optimal neighbor order is recorded as k, and the optimal subgraph is constructed using its 1st to k-th stage neighbor nodes.

[0049] In an embodiment of the present invention, building a graph neural network with a residual structure in combination with the corresponding optimal number of layers includes:

[0050] For the i-th target node, the optimal number of layers is set to L, a graph neural network with L layers is built, and a residual structure is introduced to connect the input of the graph neural network with the output of the L-th layer. For the feature of the j-th node in the optimal subgraph corresponding to the i-th target node, the node feature is recorded as x ij (that is, the node features that have not been processed by the graph neural network), the corresponding output features of the Lth layer are recorded as The final feature is expressed as:

[0051]

[0052] in, represents the final feature of the j-th node in the optimal subgraph corresponding to the i-th target node, and ADD(.) represents the addition operation.

[0053] Furthermore, through graph pooling, the vector representation of the optimal subgraph corresponding to the i-th target node is obtained. The graph pooling is expressed as:

[0054]

[0055] Where J represents the number of nodes in the optimal subgraph corresponding to the i-th target node, R(.) represents the read operation, The vector representation of the optimal subgraph corresponding to the i-th target node.

[0056] All target nodes obtain the vector representation of the corresponding optimal subgraph through the above method.

[0057] Afterwards, the attention mechanism is combined to obtain the final representation vector of the optimal subgraph corresponding to each target node, which is expressed as:

[0058]

[0059] Where M represents the number of attention heads and m is the sequence number of the attention head; W represents the attention coefficient between the optimal subgraph corresponding to the i-th target node and the u-th target node in the m-th attention head; m represents the weight matrix of the mth attention head, Represents the vector representation of the optimal subgraph corresponding to the u-th target node, represents the optimal subgraph corresponding to the u-th target node, Represents a heterogeneous graph.

[0060] Finally, the final representation vector of the optimal subgraph is input into the detector for classification detection. The detection result is whether the corresponding target node is a normal account or a machine account.

[0061] In addition, graph neural networks need to be trained in advance. Graph neural networks with the same optimal number of layers are considered the same graph neural network. During training, the processing flow of each graph neural network refers to the previous introduction. The input training data is the optimal subgraph corresponding to the corresponding target node. The predicted detection results are combined with the given labels to calculate the loss function, and the parameters of each graph neural network are updated. The loss function for all graph neural network training is the same, expressed as:

[0062]

[0063] in, Represents the target node set during training, including several target nodes corresponding to the current graph neural network. Different graph neural networks have different target node sets. l represents the lth target node, y l Represents the label of the given l-th target node, z l represents the final representation vector of the optimal subgraph feature corresponding to the lth target node, ||Θ||2 is the L2 norm of the single graph neural network parameter Θ, λ is the weight parameter, MLP is the multi-layer perceptron, and σ is the normalization function (for example, the sigmoid function).

[0064] In order to more clearly demonstrate the technical solution and technical effects provided by the present invention, a machine account detection method provided by an embodiment of the present invention is described in detail below with reference to a specific embodiment.

[0065] 1. Construction of heterogeneous graph and problem definition.

[0066] For accounts on social platforms or other platforms, we can use the social network relationships between accounts to build a heterogeneous graph. By using the accounts as nodes and the existing friend and follow relationships, we can connect these accounts. Then, we analyze the content published by the accounts (including articles, replies, comments, etc.), and add these content data as nodes to the graph to form a heterogeneous graph. Suppose there is a set of target nodes to be detected Each target node in the set is the node corresponding to the account, and the label y of each target node in the set is v ∈{0, 1}∈Y, where 0 represents a normal account, 1 represents a bot account, and Y represents the label space. Using a reinforcement learning algorithm, we search for an appropriate graph neural network structure based on the target user's local information within a heterogeneous graph. We then aggregate information from other neighbors using the graph neural network to obtain the final graph representation vector. We then calculate the loss using a defined loss function and backpropagate to update the model parameters, accurately detecting the target node category (normal or bot).

[0067] 2. Subgraph representation vector initialization.

[0068] In the embodiment of the present invention, a subgraph is constructed for each target user. Specifically, the nodes of the initial neighbor order of the target user are extracted to construct a subgraph, and the subgraph is encoded to obtain the initial subgraph representation vector. Specifically, for the subgraph corresponding to the i-th target node The initial subgraph representation vector is expressed as:

[0069]

[0070] Among them, v j For subgraph The feature of the jth node in V i It is a subgraph The set of nodes contained in , e(.) represents the encoding function. The initial subgraph obtained represents the vector In fact, it is a vector obtained by encoding the set of feature vectors of the nodes contained in the subgraph.

[0071] 3. Obtain the optimal neighbor order and number of graph neural network layers based on reinforcement learning.

[0072] The encoded subgraph representation vector is used as the input of reinforcement learning, and the agent in reinforcement learning learns two optimal strategies. and in Used to select the optimal neighbor order k, To select the optimal number of layers in a graph neural network (GNN), the base layer can be chosen from the most widely used GNN models, including GCN (graph convolutional neural network), GAT (graph attention neural network), and GraphSAGE (a network model that samples and aggregates neighboring nodes). In the initial subgraph vector representation, nodes with an initial neighbor order are selected for each target node to construct the subgraph. This may not be the most appropriate neighbor order. Therefore, reinforcement learning is used to search for the optimal neighbor order and the optimal number of layers in the graph neural network, thereby improving detection performance.

[0073] Specifically, the process of searching for the optimal neighbor order and number of graph neural network layers based on reinforcement learning is modeled as a Markov decision process (MDP):

[0074] State Space: The state s corresponding to the current time step t t ∈S is defined as the subgraph The subgraph representation vector extracted from Each time step uses the initial subgraph representation vector corresponding to a different subgraph. t = 0, ..., T, where T is the maximum time step. The maximum time step T is not less than the number of target nodes. T is a preset hyperparameter that reflects the change in accuracy over the most recent time step, thereby indicating whether the current optimization direction is towards the optimal strategy.

[0075] Action Space: the action a corresponding to the current time step t t Contains two sub-actions and The two-tuple composition is: Reinforcement learning agent performs sub-actions To obtain the value of the neighbor order, perform the sub-action To obtain the value of the number of network layers. Therefore, the overall sequence of MDP can be described as That is, starting from the initial state, the agent will get a corresponding reward feedback for each action it performs, and then move to the next state. In order to generate this sequence, the agent will The number of connections between the extracted subgraph and other subgraphs related to the nodes within the target node multi-hop is used to form a probability distribution, and the next state s is reached according to the probability distribution. t+1state transfer.

[0076] Reward Function: This function determines whether the subgraph representation at the current time step t is good enough by judging whether the model can achieve better detection results. To this end, we first define the metric for accuracy improvement:

[0077]

[0078] Among them, b is a hyperparameter that reflects the window size of historical results involved in the comparison. is the detection accuracy on the validation set at time step i′; Represents the state s corresponding to the current time step t t Next, perform action a t The detection accuracy of the corresponding target node after . In fact, It reflects the average accuracy of the most recent b time windows (consisting of multiple consecutive time steps). Intuitively, the training strategy is to find a positive increasing reward feedback To ensure the continuous improvement of model accuracy. In addition, a binary reward value r(s t , a t ) to guide the direction of training:

[0079]

[0080] in, Represents the reward feedback obtained at the previous time step t-1.

[0081] If action a t Compared with the previous time step, the reward feedback can be increased, then r(s t , a t ) is set to 1, otherwise it is set to -1, which means that the purpose of the binary reward function is to ensure that the model training is in the direction of improving the detection accuracy.

[0082] State transition: As mentioned before, the state s at the current time step t t Will transfer to the state s of the next time step according to the probability distribution t+1 .

[0083] Termination: Model-free deep reinforcement learning is used to solve the MDP problem (i.e., to find the optimal neighbor order and optimal number of layers). The basic idea is to learn to take the optimal action through exploration. Due to the limited number of target nodes, the current round ends once the agent makes a decision for each target. The DQN algorithm is primarily used to reduce the number of states in the search space. More specifically, DQN approximates the state-action value of the Bellman optimality equation:

[0084]

[0085] Among them, a t and a′ both represent actions in the action space a; As a whole, it represents the cumulative state action value from the next time step t+1 to the maximum time step T, and γ is the set weight coefficient (the discount factor of future cumulative returns); Represents the state s corresponding to the current time step t t Next, perform action a t The state action value after .

[0086] In the embodiment of the present invention, a greedy strategy (∈-greedy) is used to select Action a t , and then get the strategy function π based on this:

[0087]

[0088] Among them, ∈ is a hyperparameter, which is a setting, p is the random value corresponding to the current time step t, wp∈ means if p < ∈, random action means selecting a random action as the final action; Represents the state s corresponding to the current time step t t The maximum state-action value obtained by executing an action in the action space a; Represents the state s corresponding to the given current time step t t Under the condition of the maximum state action value, select the final action a t .

[0089] During the training phase, at the current time step t, after each action is executed, the aforementioned and r(s t , a t ) calculates the binary reward value, and then passes Calculate the state action value and finally decide the final action a through the strategy function π t , according to the final action Record the reward value obtained by executing the final action at the current time step Form the recorded data of the current time step t Continuously iterate and obtain the recorded data of each time step, use all the records to train the intelligent agent, and optimize the network parameters of the intelligent agent until convergence (when the training reaches the specified number of epochs and the test accuracy reaches the peak).

[0090] After training, all initial subgraph representation vectors are input into the agent to obtain the optimal neighbor order and the optimal number of layers of the graph neural network corresponding to each target node.

[0091] 4. Graph neural network construction, training and testing.

[0092] In an embodiment of the present invention, for each target node, the reinforcement learning search in the aforementioned third part is used to select the optimal neighbor order k and the optimal number of layers L of the graph neural network. After that, the optimal subgraph structure can be constructed for the target node and the optimal graph neural network structure can be designed for detection.

[0093] In the embodiment of the present invention, for the i-th target node, a graph neural network is constructed according to the optimal number of layers L, and a residual structure is introduced, which is expressed as:

[0094]

[0095] in, It represents the final feature of the j-th node in the optimal subgraph corresponding to the i-th target node. ADD(.) represents the addition operation, that is, adding the output feature of the L-th layer of the graph neural network to the original node feature to obtain the final feature. Feature fusion in the above way can alleviate the problems of gradient explosion / diffusion and network degradation during training.

[0096] The subgraph representation vector obtained after processing by the L-layer graph neural network structure is actually a set, in which the elements are the final features of the nodes contained in the subgraph. Therefore, the subgraph needs to be pooled to aggregate the final feature vector representations of the nodes in the subgraph into a feature vector of a super node for vector representation of the subgraph:

[0097]

[0098] Among them, J represents the number of nodes in the optimal subgraph corresponding to the i-th target node, It represents the vector representation of the optimal subgraph corresponding to the i-th target node, and R(.) represents the readout operation (READOUT operation), for example, it can be max, sum, or mean operation, that is, the maximum value / sum / mean of the final features of all nodes is calculated.

[0099] All target nodes obtain the vector representation of the corresponding optimal subgraph through the above method.

[0100] After this, an attention mechanism is employed to integrate the influence of subgraphs belonging to relevant neighbors into the final representation vector:

[0101]

[0102] in, represents the weight matrix, represents the set of real numbers, d L represents the dimension of the output feature of the Lth layer of the graph neural network, d1 represents the dimension of the input node feature of the graph neural network; M represents the number of attention heads, and m is the sequence number of the attention head; W represents the attention coefficient between the optimal subgraph corresponding to the i-th target node and the u-th target node in the m-th attention head; m represents the weight matrix of the mth attention head, Represents the vector representation of the optimal subgraph corresponding to the u-th target node, represents the optimal subgraph corresponding to the u-th target node, Represents a heterogeneous graph.

[0103] like Figure 2 As shown in the figure, a single attention head is used as an example to show the schematic diagram of the attention mechanism. Considering the length of the paper, only three examples of subgraphs corresponding to target nodes are drawn in the figure. The weight vector of each target node in a single attention head is Contains the attention coefficient of each target node and all target nodes.

[0104] The final representation vector z i The input to the detector predicts the probability that the corresponding i-th node is a normal account and a machine account, and uses this to determine the detection result (i.e. the one with a higher probability).

[0105] As mentioned earlier, the graph neural network needs to be trained in advance, and the training process is the same as the above process. The difference is that during training, it is necessary to use the detector's prediction results and the given labels to calculate the loss to update the parameters of the graph neural network. The parameter update process involved can refer to conventional technology and will not be elaborated in this invention.

[0106] In the embodiment of the present invention, different target nodes may correspond to different optimal numbers of layers. Therefore, there may eventually be multiple graph neural networks with different optimal numbers of layers. The graph neural networks with the same optimal number of layers are treated as the same graph neural network. The input training data is the optimal subgraph corresponding to the corresponding target node. Different graph neural networks are trained separately, and the loss function during training is the same, which is expressed as:

[0107]

[0108] in, Represents the target node set during training, including several target nodes corresponding to the graph neural network, v l Indicates the lth target node, the first item (i.e. -log(y l ·σ(MLP(z l )))) represents the cross entropy loss function, ||Θ||2 is the L2 norm of the parameters of a single graph neural network model, λ is the weight parameter, MLP is a multi-layer perceptron, and the final embedding vector z can be represented i The dimension is reduced to the number of classification categories (for example, two categories), and then the probability of each category is obtained through the normalization function σ, thereby achieving the purpose of classification.

[0109] Taking c target nodes as an example, if the optimal number of layers of their corresponding graph neural networks is the same, then the c target nodes will be detected using the same graph neural network, and the c target nodes will form a target node set. c is less than or equal to the number of target nodes to be detected. The input training data is the optimal subgraph extracted from the heterogeneous graph using the optimal neighbor order of each target node. The optimal neighbor orders of the c target nodes can be the same or different.

[0110] At the same time, since different target nodes correspond to different optimal numbers of layers, in order to improve training efficiency, during training, each graph neural network is trained in order from low to high optimal number of layers. The graph neural network with the lowest optimal number of layers is trained first. Based on the trained graph neural network with the lowest optimal number of layers, the graph neural network with the second lowest optimal number of layers is trained until the training of all graph neural networks is completed.

[0111] For example: assuming that all target nodes correspond to graph neural networks with three optimal numbers of layers, three graph neural networks need to be trained, which are recorded as the first, second, and third graph neural networks from low to high; first train the first graph neural network, and use the initialization parameters during training. After training, obtain the first graph neural network that can be used for corresponding target detection; on this basis, stack several layers to form the second graph neural network, and the stacked layers can use the initialization parameters, and the remaining layers belonging to the first graph neural network use the parameters of the trained first graph neural network, and train the second graph neural network according to the loss function; similarly, after the second graph neural network is trained, stack several layers to form the third graph neural network, and use the initialization parameters, and the remaining layers belonging to the second graph neural network use the parameters of the trained second graph neural network, and train the third graph neural network according to the loss function.

[0112] Figure 3The reinforcement learning graph neural network architecture search framework is demonstrated. The graph structure on the far left is the heterogeneous graph provided by the present invention, and three subgraph examples are exemplarily provided. In each subgraph, v represents the target node, and v1 to v6 are the neighbor nodes of the target node; 1 to 3 in the figure represent different actions. The figure only provides examples of some actions. The specific content and number of actions in actual applications can be set according to actual conditions; the subgraph search module and the GNN layer search module are both modules that execute the reinforcement learning part, and the Attention on the right is the attention mechanism.

[0113] In the above scheme provided by the embodiment of the present invention, the machine account detection task is converted into a subgraph classification with the target node to be detected as the central node. By utilizing the account (target node)'s own data and the interaction data with other accounts to construct a heterogeneous graph, reinforcement learning is used to search and determine the neighbor order extracted for the target node and the graph neural network structure used for subgraph classification. Compared with the existing technology, the present invention has the following main advantages: 1) By extracting the target node's own data features and the interaction information between accounts to construct a subgraph, the rich semantic information in the heterogeneous graph is fully aggregated, thereby improving the detection effect. 2) The reinforcement learning method is used to adaptively search and determine the design details of the graph neural network structure according to the characteristics of the machine account detection task, which greatly reduces the workload of the model designer. 3) The residual structure and attention mechanism are used to more comprehensively capture the feature information of nodes and subgraphs, thereby better serving the classification detection task and effectively improving the accuracy of the detection results.

[0114] Through the description of the above embodiments, those skilled in the art will clearly understand that the above embodiments can be implemented through software or by using software plus a necessary general-purpose hardware platform. Based on this understanding, the technical solutions of the above embodiments can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, a USB flash drive, a mobile hard disk, etc.) and includes a number of instructions for causing a computer device (such as a personal computer, a server, or a network device) to execute the methods described in the various embodiments of the present invention.

[0115] Example 2

[0116] The present invention also provides a machine account detection system, which is mainly implemented based on the method provided in the above embodiment. Figure 4 As shown, the system mainly includes:

[0117] An information acquisition and heterogeneous graph construction unit is used to obtain accounts on the social platform, the content published by each account, and the social network relationships between accounts. The accounts and the content published by each account are treated as different types of nodes, and the social network relationships between accounts and the relationships between accounts and their published content are used as edges of corresponding nodes to construct a heterogeneous graph. Feature extraction is performed on the relevant information of the account and the relevant information of the content respectively, and the extracted features are used as the node features of the corresponding nodes.

[0118] A subgraph initialization unit, for a target node to be detected, extracts neighbor nodes of the corresponding initial neighbor order from the heterogeneous graph with each target node as the center according to a preset initial neighbor order to construct a corresponding subgraph, and determines an initial subgraph representation vector using all node features in the corresponding subgraph; wherein the target nodes are all account corresponding nodes;

[0119] Reinforcement learning and optimal parameter acquisition unit, which is used to use reinforcement learning methods to combine all initial subgraph representation vectors to search for the optimal neighbor order and the optimal number of layers of the graph neural network corresponding to each target node;

[0120] The network construction and detection unit is used to build a graph neural network with a residual structure according to different optimal numbers of layers; for each target node, according to the corresponding optimal neighbor order, the neighbor nodes of the corresponding optimal neighbor order are extracted from the heterogeneous graph with the corresponding target node as the center to construct an optimal subgraph, and the trained graph neural network with the corresponding optimal number of layers is combined with the attention mechanism to obtain the final representation vector of the optimal subgraph corresponding to the corresponding target node, and then classification detection is performed to obtain the detection result of the corresponding target node.

[0121] Those skilled in the art will clearly understand that for the convenience and brevity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above.

[0122] Example 3

[0123] The present invention also provides a processing device, such as Figure 5 As shown, it mainly includes: one or more processors; a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the method provided by the aforementioned embodiment.

[0124] Furthermore, the processing device further includes at least one input device and at least one output device; in the processing device, the processor, memory, input device, and output device are connected via a bus.

[0125] In the embodiment of the present invention, the specific types of the memory, input device, and output device are not limited; for example:

[0126] The input device can be a touch screen, image acquisition device, physical button or mouse;

[0127] The output device may be a display terminal;

[0128] The memory may be a random access memory (RAM) or a non-volatile memory, such as a disk memory.

[0129] Example 4

[0130] The present invention also provides a readable storage medium storing a computer program, which implements the method provided in the above embodiment when the computer program is executed by a processor.

[0131] In the embodiments of the present invention, the computer-readable storage medium may be provided in the aforementioned processing device, for example, as a memory in the processing device. Alternatively, the computer-readable storage medium may be a USB flash drive, a removable hard drive, a read-only memory (ROM), a magnetic disk, or an optical disk, among other media capable of storing program code.

[0132] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A machine account detection method, characterized in that: include: Obtain accounts on the social platform, the content published by each account, and the social network relationships between accounts. Treat the accounts and the content published by each account as different types of nodes, and the social network relationships between accounts and the relationships between accounts and their published content as edges of the corresponding nodes. Construct a heterogeneous graph, extract features from the account-related information and the content-related information respectively, and use the extracted features as the node features of the corresponding nodes. For the target node to be detected, neighbor nodes of the corresponding initial neighbor order are extracted from the heterogeneous graph with each target node as the center according to the preset initial neighbor order to construct a corresponding subgraph, and the initial subgraph representation vector is determined using the features of all nodes in the corresponding subgraph; wherein, the target nodes are all nodes corresponding to the account; Using reinforcement learning methods combined with all initial subgraph representation vectors, we search for the optimal neighbor order and the optimal number of layers of the graph neural network corresponding to each target node; A graph neural network with a residual structure is constructed according to different optimal numbers of layers. For each target node, the corresponding optimal neighbor order is extracted from the heterogeneous graph with the corresponding target node as the center to construct an optimal subgraph. The trained graph neural network with the corresponding optimal number of layers is combined with the attention mechanism to obtain the final representation vector of the optimal subgraph corresponding to the corresponding target node, and then classification detection is performed to obtain the detection result of the corresponding target node. For each target node, a graph neural network with a residual structure is built based on the corresponding optimal number of layers, including: For the i-th target node, the optimal number of layers is set to L, a graph neural network with L layers is built, and a residual structure is introduced to connect the input of the graph neural network with the output of the L-th layer. For the feature of the j-th node in the optimal subgraph corresponding to the i-th node, the node feature is recorded as x ij , the corresponding output feature of the Lth layer is recorded as The final feature is expressed as: in, represents the final feature of the jth node in the optimal subgraph corresponding to the i-th target node, and ADD(.) represents the addition operation; For the optimal subgraph corresponding to the i-th target node, graph pooling is performed through the graph neural network to obtain the vector representation of the corresponding optimal subgraph. The graph pooling process is expressed as: Where J represents the number of nodes in the optimal subgraph corresponding to the i-th target node, R(.) represents the read operation, The vector representation of the optimal subgraph corresponding to the i-th target node; Combined with the attention mechanism, the final representation vector of the optimal subgraph corresponding to each target node is obtained, which is expressed as: in, W represents the attention coefficient between the optimal subgraph corresponding to the i-th target node and the u-th target node in the m-th attention head; m represents the weight matrix of the mth attention head, Represents the vector representation of the optimal subgraph corresponding to the u-th target node, represents the optimal subgraph corresponding to the u-th target node, Represents a heterogeneous graph.

2. A machine account detection method according to claim 1, characterized in that: The method of using reinforcement learning to combine all initial subgraph representation vectors to search for the optimal neighbor order and the optimal number of layers of the graph neural network corresponding to each target node includes: Using all the initial subgraph representation vectors as input, the agent is trained using reinforcement learning. After the agent is trained, all the initial subgraph representation vectors are input to the agent to obtain the optimal neighbor order and the optimal number of layers of the graph neural network for each target node. The training process is as follows: a subgraph is randomly selected at the initial time step, and its initial subgraph representation vector is used as the state of the initial time step. Afterwards, the transition probability is calculated based on the connection relationship between the corresponding subgraph of the previous time step and other subgraphs, and the corresponding subgraph of the next time step is determined according to the transition probability, and the initial subgraph representation vector of the corresponding subgraph is used as the state of the next time step; under the state of the current time step, different actions are selected from the action space, and the corresponding reward values ​​are obtained after executing different actions and the corresponding state-action values ​​are calculated, and then the final action is selected in combination with the strategy function, and the record data of the current time step is formed by combining the state of the current time step, the final action, the reward value and the state of the next time step; the intelligent agent is described using the recorded data of multiple time steps until convergence, and the training of the intelligent agent is completed; wherein, each action includes a sub-action for selecting the neighbor order value and a sub-action for selecting the layer value of the graph neural network.

3. A machine account detection method according to claim 2, characterized in that: The final action selection methods include: Each action a in the current time step t t By binary action Composition, by executing the first sub-action Get the neighbor order value by executing the second sub-action Get the layer values ​​of the graph neural network and select different sub-actions from the given action space With sub-actions After executing them separately in the state of the current time step, the reward feedback of each action is determined according to the classification accuracy of the subgraph corresponding to the current time step t, and a reward value is determined based on the reward feedback. Combined with the reward value and the cumulative state-action value from the given next time step t+1 to the maximum time step T, the state-action value obtained after executing each action at the current time step t is determined. The strategy function is used to determine whether the current time step t meets the set conditions. If so, an action is randomly selected as the final action of the current time step t. If not, the action corresponding to the maximum state-action value is selected as the final action of the current time step t.

4. A machine account detection method according to claim 1, characterized in that: Graph neural networks with the same optimal number of layers are considered to be the same graph neural network. The input training data is the optimal subgraph corresponding to the corresponding target node. The loss function of all graph neural network training is the same. The loss function of graph neural network training is expressed as: in, Represents the target node set during training, v l represents the lth target node, y l Represents the label of the given l-th target node, z l represents the final representation vector of the optimal subgraph feature corresponding to the lth target node, ||Θ||2 is the L2 norm of the graph neural network parameter Θ, λ is the weight parameter, MLP is the multi-layer perceptron, and σ is the normalization function; During training, each graph neural network is trained in order from the lowest to the highest optimal number of layers. The graph neural network with the lowest optimal number of layers is trained first. Based on the trained graph neural network with the lowest optimal number of layers, the graph neural network with the second lowest optimal number of layers is trained until the training of all graph neural networks is completed.

5. A machine account detection system, characterized in that: The method according to any one of claims 1 to 4 is implemented, and the system comprises: An information acquisition and heterogeneous graph construction unit is used to obtain accounts on the social platform, the content published by each account, and the social network relationships between accounts. The accounts and the content published by each account are treated as different types of nodes, and the social network relationships between accounts and the relationships between accounts and their published content are used as edges of corresponding nodes to construct a heterogeneous graph. Feature extraction is performed on the relevant information of the account and the relevant information of the content respectively, and the extracted features are used as the node features of the corresponding nodes. A subgraph initialization unit, for a target node to be detected, extracts neighbor nodes of the corresponding initial neighbor order from the heterogeneous graph with each target node as the center according to a preset initial neighbor order to construct a corresponding subgraph, and determines an initial subgraph representation vector using all node features in the corresponding subgraph; wherein the target nodes are all account corresponding nodes; Reinforcement learning and optimal parameter acquisition unit, which is used to use reinforcement learning methods to combine all initial subgraph representation vectors to search for the optimal neighbor order and the optimal number of layers of the graph neural network corresponding to each target node; The network construction and detection unit is used to build a graph neural network with a residual structure according to different optimal numbers of layers; for each target node, according to the corresponding optimal neighbor order, the neighbor nodes of the corresponding optimal neighbor order are extracted from the heterogeneous graph with the corresponding target node as the center to construct an optimal subgraph, and the trained graph neural network with the corresponding optimal number of layers is combined with the attention mechanism to obtain the final representation vector of the optimal subgraph corresponding to the corresponding target node, and then classification detection is performed to obtain the detection result of the corresponding target node.

6. A processing device, characterized in that include: one or more processors; a memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.

7. A readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images