A meta-graph based network malicious behavior detection method and device

Abstract

The application discloses a network malicious behavior detection method and device based on a meta graph, comprising: acquiring required user data; inputting the user data into a trained network malicious behavior detection model to output a detection result through the network malicious behavior detection model, wherein the network malicious behavior detection model comprises a social network graph construction unit, a meta graph convolutional recurrent network model, a meta learning regression graph neural network model and a meta graph reinforcement learning framework, and the detection result is output based on the meta graph convolutional recurrent network model, the meta learning regression graph neural network model and the meta graph reinforcement learning framework. The embodiment of the application combines the meta graph convolutional recurrent network and the meta learning regression graph neural network model to jointly realize detection of water army behavior and improve the accuracy and efficiency of water army post behavior detection.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method and device for detecting malicious network behavior based on metagraphs. Background Technology

[0002] Basic machine learning-based malicious behavior detection typically consists of feature engineering models and anomaly detection models. Feature engineering models learn different types of behavioral data features using deep neural networks with varying architectures, or they can map multiple types of behavioral data into a unified feature space using multimodal methods. Anomaly detection models aim to capture outliers in large datasets by training on pre-representational behavioral features using unsupervised or weakly supervised methods. Anomaly detection models can be categorized as clustering-based or classification-based, and can also be fused with feature learning to learn the underlying manifold of normal samples.

[0003] Considering the internet and social network environment, automated network malicious behavior detection needs to consider not only behavioral characteristics but also network structure. Its foundation lies in clearly defining the network topology, network interaction entities, and network behavior, and modeling them for downstream task objectives. Generally, network scenarios can be abstracted using defined graphs, heterogeneous graphs, or multi-relationship graphs, and specific attribute features can be assigned to nodes and edges in the graph, followed by optimization and prediction using graph neural networks. Early graph modeling methods either relied on the natural topology of road networks (i.e., binary adjacency graphs) or on predefined graphs in certain metrics (e.g., Euclidean distance). GSL aims to jointly learn graph structures and their corresponding node (or graph) representations. In short, GSL methods fall into three categories: (1) metric-based methods that derive edge weights from node embeddings based on metric functions; (2) neural-based methods that learn graph structures from node embeddings using neural networks; and (3) direct methods that treat adjacency matrices or node embedding dictionaries as learnable parameters. GSL has been applied to temporal graph data. Some works use learnable node embedding dictionaries to adaptively learn adjacency matrices from data. However, these GSL methods are limited to short sequences with time steps less than 1k and cannot capture dynamically evolving graph structures. Summary of the Invention

[0004] This application provides a method and device for detecting malicious network behavior based on metagraphs. It combines metagraph convolutional recurrent networks with meta-learning regression graph neural network models to jointly detect online troll behavior and improve the accuracy and efficiency of detecting online troll posting behavior.

[0005] This application provides a method for detecting malicious network behavior based on metagraphs, including:

[0006] Obtain the required user data;

[0007] The user data is input into the trained network malicious behavior detection model to output detection results. The network malicious behavior detection model includes: a social network graph construction unit, a metagraph convolutional recurrent network model, a metalearning regression graph neural network model, and a metagraph reinforcement learning framework.

[0008] The social network graph construction unit is used to construct a social network graph based on the user data, and input the constructed social network graph into the metagraph convolutional recurrent network model, the metalearning regression graph neural network model and the metagraph reinforcement learning framework respectively, so as to output the detection result based on the metagraph convolutional recurrent network model, the metalearning regression graph neural network model and the metagraph reinforcement learning framework.

[0009] The metagraph convolutional recurrent network model includes an encoder, a decoder, and a metagraph learner. The encoder and the decoder are constructed using graph convolutional recurrent units as basic units. The metagraph learner is used to input feedback information to the decoder based on the constructed metanode library and the super network, in response to the output of the encoder.

[0010] The meta-learning regression graph neural network model is based on a regression GNN network, which includes two graph convolutional layers and one fully connected layer.

[0011] The metagraph reinforcement learning framework is based on deep reinforcement learning (DRL) and includes a graph pooling module, a candidate node prediction module, and a recognition module. The graph pooling module acts as an environment pool, the candidate node prediction module is used to provide a set of candidate nodes based on the AIM search algorithm, and the recognition module is used to process DRL agents.

[0012] Optionally, after obtaining the required user data, word sense disambiguation may also be performed using the following methods:

[0013] Constructing a monolingual co-occurrence graph G s = <V s E s >;

[0014] Collect each co-occurring noun or adjective pair (cw) from the user data. i cw j And add any co-occurring noun or adjective as a node to the monolingual co-occurrence graph, with each co-occurring word pair connected to an edge (v i v j )∈E s Connect, either side based on the corresponding word cw i and CW j The strength of the association between them is assigned a weight w(v) i vj )satisfy:

[0015] w(v i v j )=1-max[p(cw i |cw j ), p(cw j |cw i )]

[0016] Where p() represents the conditional probability, derived from cw i and CW j The number of contexts that appear simultaneously divided by the number of contexts containing cw j Estimate by the number of contexts;

[0017] Configure a set of target languages ​​and combine the monolingual co-occurrence graph G s Expanded into a multilingual graph G ML = <V ML E ML >, where V ML =V s ∪∪ l∈L V l It indicates that it comes from the source (V) s ) or target (V) l A set of nodes representing the content words of a language, E ML =E s ∪∪ l∈L {E l ∪E s，l} is a set of edges;

[0018] Calculate the root hub in the multilingual graph to distinguish the meaning of the target word in the source language, satisfying:

[0019]

[0020] Where d is the damping factor, deg(v) i ) is node v i The number of adjacent nodes, w ij It is node v i and v j The weights of co-occurrence edges between words, the minimum spanning tree (MST) of the graph with the target word as the root, G ML The root word center constitutes its first level;

[0021] Given the context W of the target word w in the source language, use MST to find the most relevant word in w to eliminate ambiguity.

[0022] Optionally, using MST to find the most relevant words in w to disambiguate includes:

[0023] By calculating the correct central disHub, relevant nodes are found, and only nodes linked to the disHub are retained. W is defined. h It is a set of mapped content words that satisfy:

[0024]

[0025] Where d(cw) is a function that assigns weights to cw based on its distance from w, and dist(cw, h) is given by the number of edges between cw and h in the MST;

[0026] Sum the translation counts to sort each translation;

[0027] The similarity metric between two translations is defined as follows:

[0028]

[0029] in It returns a set of languages. A function for the lexicalization set of word c in Chinese;

[0030] The accuracy of the translation is verified by maximizing the similarity metric to achieve cross-lexical resource alignment.

[0031] Optionally, the graph convolutional recurrent unit of the metagraph convolutional recurrent network model adopts the definition of graph convolution operation and gated recurrent unit (GRU) to represent GCRU as the basic unit, satisfying:

[0032]

[0033]

[0034] Where, X∈R N×C (N represents the number of spatial units), C represents the number of information channels, and H∈R N Represents graph convolution operation ★ G Inputs and outputs, Θ, W K ∈R K×C To approximate the kernel parameters to order K using Chebyshev polynomials, σ represents the activation function, ⊙ represents the product of elements at corresponding positions, and parameters u, r, and C represent the update gate, reset gate, and candidate state in the GCRU unit, respectively. {u，r，C} Indicates the gate parameters;

[0035] The metagraph convolutional recurrent network model uses a canonical formula for spatiotemporal graph learning, satisfying:

[0036]

[0037] This is derived by performing a normalized random walk on the non-negative part of the matrix product of the embeddings of trainable nodes E. To its transpose;

[0038] The metagraph learner is used to learn from the constructed metanode library. And using a super network, feedback information is input to the decoder, wherein d and d represent the number of memory items and the dimension of each item, respectively; the intermediate variables for updating the metagraph learner are stored in a specific metanode library Φ, and the memory operation relationship during the update process is defined as follows:

[0039]

[0040]

[0041] Where the superscript (i) serves as the row index, H represents t The vector of the i-th node in the vector. This indicates that a fully connected layer will hide its state. Projection to localized queries scalar a j Physically representing a vector Similarity with memory item Φ[j], metanode vector This indicates that it can be further restored to a combination of stored items;

[0042] The supernetwork is used to generate GSL node embeddings based on the meta-node library, and is formulated as follows:

[0043]

[0044] Where NN H Indicates a hypernet;

[0045] The metagraph learner uses the Metagraph Convolutional Recurrent Network (MegaCRN) as a general framework.

[0046] Optionally, the meta-learning regression graph neural network model is implemented based on a regression GNN model, which includes two graph convolutional layers and one fully connected layer. During meta-learning, for each task T = {L(g1, s1, ..., g...} H s H ), q(g1), q(g) t+1 |g t s t H} is defined by the loss function L, the distribution q(g1) on the initial observations, and the transition distribution q(g). t+1 |g t s t It consists of the set length H;

[0047] For regression, the loss function satisfies:

[0048]

[0049] Where g (j) s (j) They represent the tasks from task T respectively. i The input and output of the sample;

[0050] The update Θ of the regression GNN model satisfies:

[0051]

[0052]

[0053] Where γ represents the step size hyperparameter and η represents the metastep size.

[0054] Optionally, the candidate node prediction module of the metagraph reinforcement learning framework provides a set of candidate nodes based on the AIM search algorithm, which is completed using a finite Markov decision process.

[0055] Optionally, the agent of the metagraph reinforcement learning framework is trained using double Q-learning, where one estimator determines the best possible action for the next state, and the other estimator provides the Q-value of the selected action, satisfying the following update method:

[0056]

[0057] Among them, Q T Denotes the target network, Q L Indicates the local network, a t X represents the action performed at time t. t The state matrix represents the state executed at time t, θ is the tuning parameter, r and γ are parameters, and G is the state matrix executed at time t. i The sampling pattern corresponding to the i-th set.

[0058] This application also proposes a computer device, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the steps of the aforementioned metagraph-based network malicious behavior detection method.

[0059] This application also proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the aforementioned metagraph-based network malicious behavior detection method.

[0060] This application combines a metagraph convolutional recurrent network with a metalearning regression graph neural network model to jointly detect online troll behavior, thereby improving the accuracy and efficiency of detecting online troll posting behavior.

[0061] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0062] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0063] Figure 1 This is an example of the overall framework for detecting malicious network behavior in this application.

[0064] Figure 2 This is an example of the metagraph convolutional recurrent network model structure in an embodiment of this application;

[0065] Figure 3 This is an example of the meta-learning regression graph neural network model structure in an embodiment of this application;

[0066] Figure 4 This is an example of the metagraph reinforcement learning framework structure in an embodiment of this application. Detailed Implementation

[0067] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0068] 1. Graphical Structure Learning (GSL)

[0069] Early methods either relied on the natural topology of the road network (i.e., binary adjacency graphs) or on predefined graphs in some metric (e.g., Euclidean distance). GSL aims to learn the graph structure and its corresponding node (or graph) representations together. In short, GSL methods fall into three categories: (1) metric-based methods that derive edge weights from node embeddings based on a metric function; (2) neural-based methods that learn the graph structure from node embeddings using neural networks; and (3) direct methods that treat the adjacency matrix or node embedding dictionary as learnable parameters. GSL has been applied to temporal graph data. Some works use learnable node embedding dictionaries to adaptively learn the adjacency matrix from the data. However, these GSL methods are limited to short sequences with time steps less than 1k and cannot capture dynamically evolving graph structures.

[0070] 2. Graph Neural Networks (GNN)

[0071] Graph neural networks (Graph Neural Networks) are a class of deep learning techniques with graph convolutional layers that outperform existing methods in a wide range of computer vision applications. Recently, they have attracted significant attention due to their unique ability to effectively model correlations between samples. They provide an efficient solution for integrating diverse information.

[0072] Generative Neural Networks (GNNs) are optimizable transformations of all properties (nodes, edges, global context) of a graph while preserving its symmetry (permutation invariance). Symmetry refers to the fact that rearranging the vertices does not change the overall result. A GNN takes a graph as input and outputs a graph. It transforms the graph's properties (vertices, edges, global context) but does not alter the graph's connectivity—that is, which edges connect to which vertices remains unchanged.

[0073] 3. Metagraph Convolutional Recurrent Network

[0074] It can easily rely on observed data to remain robust and adapt to normal to non-stationary conditions. Metagraph convolutional recurrent networks mainly consist of graph convolutional recurrent units and meta-learners. The meta-learner learning process includes two steps: (1) querying node-level prototypes from the meta-node library; and (2) dynamically reconstructing node embeddings using the hypernetwork. This local memory capability enables modular metagraph learners to substantially distinguish different spatiotemporal patterns over time, which can even be generalized to accident scenarios.

[0075] 4. Meta-learning regression graphical neural network

[0076] Meta-learning regression graph neural networks (GNNs) appropriately incorporate the graphical structure of social network posting behaviors and effectively model the correlations between them. Furthermore, due to learning, the regression GNN model becomes more flexible while mitigating the influence of certain problems. This network exhibits a unique ability to model correlations between data and combines global and local topological properties to predict behavior scores. Predictive performance is improved by reducing the impact of sample heterogeneity. This network demonstrates a good balance between flexibility and performance and can be used in other application areas suffering from high intra-class variability.

[0077] 5. Meta-learning

[0078] Meta-learning is often understood as "learning-to-learn," referring to the process of improving a learning algorithm across multiple learning stages. In the basic learning process, the internal (or lower-level / basic) learning algorithm solves the task defined by the dataset and the objective. In the meta-learning process, the external (or upper-level / meta) algorithm updates the internal learning algorithm, causing its learned model to improve upon the external objective. Therefore, the core idea of ​​meta-learning is to learn prior knowledge.

[0079] Meta-learning has two layers of meaning. The first layer is to enable machines to learn, so that they have the ability to analyze and solve problems. Machines gain experience by completing tasks and improve their ability to complete tasks. The second layer is to enable machine learning models to better generalize to new domains, thereby completing new tasks that are very different.

[0080] This application provides a method for detecting malicious network behavior based on metagraphs, including:

[0081] Acquire the necessary user data. Gather relevant user data from various social media platforms to construct the required dataset. Subsequent operations will then be used to identify and detect online trolling behavior on social networks. For example, during training, trolling behavior can be labeled to build the dataset, thus completing the training process.

[0082] The user data is input into the trained network malicious behavior detection model, and the model outputs detection results, such as... Figure 1 As shown, the network malicious behavior detection model includes: a social network graph construction unit, a metagraph convolutional recurrent network model, a metalearning regression graph neural network model, and a metagraph reinforcement learning framework.

[0083] The social network graph construction unit is used to construct a social network graph based on the user data, for example, defined as G = (V, E), and inputs the constructed social network graph into the metagraph convolutional recurrent network model, the metalearning regression graph neural network model, and the metagraph reinforcement learning framework, respectively, so as to obtain the detection result through data collaborative processing based on the metagraph convolutional recurrent network model, the metalearning regression graph neural network model, and the metagraph reinforcement learning framework.

[0084] The metagraph convolutional recurrent network model includes an encoder, a decoder, and a metagraph learner. The encoder and the decoder are constructed using graph convolutional recurrent units as basic units. The metagraph learner is used to input feedback information to the decoder based on the constructed metanode library and the super network, in response to the output of the encoder.

[0085] The meta-learning regression graph neural network model is based on a regression GNN network, which includes two graph convolutional layers and a fully connected layer.

[0086] The metagraph reinforcement learning framework is based on deep reinforcement learning (DRL) and includes a graph pooling module, a candidate node prediction module, and a recognition module. The graph pooling module acts as an environment pool, the candidate node prediction module is used to provide a set of candidate nodes based on the AIM search algorithm, and the recognition module is used to process DRL agents.

[0087] The data collaborative processing mechanism utilizes a metagraph convolutional recurrent network model for graph structure and feature learning, a meta-learning regression graph neural network model for network scoring, and a metagraph reinforcement learning framework for enhanced identification of anomalous (internet troll) nodes based on both.

[0088] This application combines a metagraph convolutional recurrent network with a metalearning regression graph neural network model to jointly detect online troll behavior, thereby improving the accuracy and efficiency of detecting online troll posting behavior.

[0089] Due to the complexity of the social network environment and the prevalence of languages, translation errors are possible, and sometimes semantic discrepancies can lead to incorrect judgments. This embodiment performs word meaning disambiguation before detecting online troll behavior to improve detection accuracy. In some embodiments, word meaning disambiguation is performed in the following manner:

[0090] Define C as all contexts of the target word w in the source language s, and construct a monolingual co-occurrence graph G. s = <V s E s >

[0091] Then, collect each co-occurring noun or adjective pair (cw) from the user data. i cw jAnd add any co-occurring noun or adjective as a node to the monolingual co-occurrence graph, with each co-occurring word pair connected to an edge (v i v j )∈E s Connect, either side based on the corresponding word cw i and CW j The strength of the association between them is assigned a weight w(v) i v j )satisfy:

[0092] w(v i v j )=1-max[p(cw i |cw j ), p(cw j |cw i )]

[0093] Where p() represents the conditional probability, given the word cw j The word cw i The conditional probability is given by cw i and CW j The number of contexts that appear simultaneously divided by the number of contexts containing cw i Estimate by the number of contexts;

[0094] Configure a set of target languages ​​L, and then configure the monolingual co-occurrence graph G. s Expanded into a multilingual graph G ML = <V ML E ML >, where V ML =V s ∪∪ l∈L V l It indicates that it comes from the source (V) s ) or target (V) l A set of nodes representing the content words of a language, E ML =E s ∪∪ l∈L {E l ∪E s，l} is a set of edges;

[0095] Calculate the root hub in the multilingual graph to distinguish the meaning of the target word in the source language, satisfying:

[0096]

[0097] Where d is the damping factor, deg(v) i ) is node v i The number of adjacent nodes, w ij It is node v i and v jThe weights of co-occurrence edges between words, the minimum spanning tree (MST) of the graph with the target word as the root, G ML The root centers constitute its first level. By using a multilingual graph, it is possible to obtain the MST containing translation nodes and edges.

[0098] Given the context W of the target word w in the source language, use MST to find the most relevant word in w to eliminate ambiguity.

[0099] In some embodiments, using MST to find the most relevant word in w to disambiguate includes:

[0100] Relevant nodes are found by calculating the correct central disHub (i.e., meaning), and only nodes linked to the disHub are retained. W is defined as follows. h It is a set of mapped content words that satisfy:

[0101]

[0102] Where d(cw) is a function that assigns weights to cw based on its distance from w, and dist(cw, h) is given by the number of edges between cw and h in the MST.

[0103] Sum the translation counts to sort each translation;

[0104] This application proposes a translation-based cross-lexical resource alignment method, which verifies the accuracy of the translation by maximizing the similarity metric to complete the cross-lexical resource alignment.

[0105] By maximizing the lexical intersection between translations, a given source word is mapped to a target word, and the similarity metric between two translations is defined as follows:

[0106]

[0107] in It returns a set of languages. A function for the lexicalization set of word c.

[0108] In some embodiments, such as Figure 2As shown, the meta-graph convolutional recurrent network model references a meta-graph convolutional recurrent network to identify multiple social media accounts. It mainly includes graph convolutional recurrent units and a meta-learner. The graph convolutional recurrent units are inspired by the successful learning of graph-structured data (such as social networks) representations using graph convolutional networks (GCNs), enabling the injection of graph convolution operations into recurrent units (such as LSTMs). Therefore, the derived graph convolutional recurrent unit (GCRU) can simultaneously capture the spatial and temporal dependencies represented by the input graph topology in a sequential manner. In this embodiment, without loss of generality, the definition of graph convolution operations and gated recurrent units (GRUs) is used to represent GCRU as the basic unit, satisfying:

[0109]

[0110]

[0111] Where, X∈R N×C (N represents the number of spatial units), C represents the number of information channels, and H∈R N Represents graph convolution operation ★ G Inputs and outputs, Θ, W K ∈R K×C To approximate the kernel parameters to order K using Chebyshev polynomials, σ represents the activation function, ⊙ represents the product of elements at corresponding positions, and parameters u, r, and C represent the update gate, reset gate, and candidate state in the GCRU unit, respectively. {u，r，C} Indicates the gate parameter.

[0112] For learning graph structures, matrices Typically defined based on certain metrics and empirical laws. However, the choice of metrics can be arbitrary and suboptimal, prompting a series of studies to integrate graph structure learning (GSL) into modeling to achieve simultaneous optimization. In this embodiment, the metagraph convolutional recurrent network model uses a canonical formula for spatiotemporal graph learning, i.e., adaptive graph, satisfying:

[0113]

[0114] This is derived by performing a normalized random walk on the non-negative part of the matrix product of the embeddings of trainable nodes E. Then it is transposed.

[0115] Another GSL strategy is to use the input signal X to create a transient graph. t Or hidden state H t Defined in a similar way. With hidden state H t For example:

[0116]

[0117] The parameter matrix W essentially represents H. t Projected into another embedded space.

[0118] For the metagraph learner, this embodiment proposes a novel spatiotemporal graph (STG) learning module. The term "metagraph" is coined to represent the generation of node embeddings used for graph structure learning, which differs from the definition in Heterogeneous Information Networks (HIN). According to the definitions in the two equations above, the adaptive graph depends only on the parameterized node embeddings E, while the instantaneous graph is actually conditional on the input (projected with parameters W). t or H t Clearly, this generation process determines the properties of the derived graph, as the former is time-invariant, while the latter is sensitive to the input signal. This prompts further enhancement of the node embeddings generated by STG, since real-world networks are more complex, exhibiting spatiotemporal heterogeneity and non-stationarity.

[0119] The aim is to memorize typical features from observed samples for further pattern matching. The goal is to inject memory and discriminative abilities into spatiotemporal graph learning. Embodiments of this application utilize the concept of memory networks, where the metagraph learner is used to learn from a constructed metanode library. And using a super network, feedback information is input to the decoder, wherein Let and d represent the number of memory items and the dimension of each item, respectively. The intermediate variables updated by the metagraph learner are stored in a specific memory group Φ. The memory operation relationships during the update process are defined as follows:

[0120]

[0121]

[0122] Where the superscript (i) serves as the row index, H represents t The vector of the i-th node in the array. The first expression represents complete connectivity (FC parameter W). Q ∈R d The layer will hide the state. Projection to localized queries The second expression represents the process of... To compute the scalar a by matching each memory Φ[j]. j The memory read operation, the scalar a j Physically representing a vector Similarity to memory item Φ[j]. Metanode vector It can be further restored to a combination of stored items. The reconstructed representation M can be utilized. t ∈R N×d With enhanced coding, the hidden representation H t, by H′ t =[H t M t [] indicates ([·] indicates cascading operation). This embodiment further utilizes a super network to generate GSL node embeddings based on a meta-node library. The super network is used to generate GSL node embeddings based on a meta-node library, and the memory-enhanced node embedding generation can be formulated as follows:

[0123]

[0124] Where NN H To represent a hypernetic network; without loss of generality, use an FC layer (parameter W). E ∈R d Then, a metagraph can be constructed. As an alternative to adaptive and transient graphs, it is fed back to the GCRU decoder. In this way, the interaction between the GCRN backbone and the metagraph learner is carried out dynamically.

[0125] The metagraph learner uses the Metagraph Convolutional Recurrent Network (MegaCRN) as a general framework.

[0126] Specifically, this embodiment utilizes the described metagraph learner, referencing the Metagraph Convolutional Recurrent Network (MegaCRN) as a general framework for modeling. It can be optimized end-to-end. MegaCRN learns node-level prototypes of patterns in a meta-node library to adaptively update the auxiliary graph based on observations. To further enhance its ability to distinguish different scenes on different paths, two constraints are used to sparse the attention-based query mechanism, including consistency loss. and contrast loss Represented as:

[0127]

[0128]

[0129] Where T represents the total number of sequences (i.e., samples) in the training set, and p and n represent the results obtained through a given local query. In the case of The first two memory item indices obtained through ranking. By implementing these two constraints, Considered as anchors, the most similar prototype Φ[p] is treated as a positive sample, and the second most similar prototype Φ[n] as a negative sample (λ represents the margin between positive and negative pairs). The idea here is to keep memory items as compact as possible while also being as distinct as possible. These two constraints guide the memory Φ to distinguish different spatiotemporal patterns at the node level. In practice, it has been found that adding them to the objective criterion (i.e., MAE) helps with training convergence (with balancing factors κ1, κ2):

[0130]

[0131] In some embodiments, such as Figure 3 As shown, the meta-learning regression graph neural network model is implemented based on a regression GNN model, denoted as f, which maps the social network graph G to behavior scores s. It is combined with a meta-graph convolutional recurrent network to jointly determine the results. The regression GNN model includes two graph convolutional layers and one fully connected layer. During meta-learning, the regression GNN model is trained to adapt to a large number of tasks, each task T = {L(g1, s1, ..., g...} H s H ), q(g1), q(g) t+1 |g t s t H} is defined by the loss function L, the distribution q(g1) on the initial observations, and the transition distribution q(g). t+1 |g t s t It consists of the set length H;

[0132] For a regression GNN model, the loss function satisfies:

[0133]

[0134] Where g (j) s (j) They represent the tasks from task T respectively. i The input and output of the sample; in the model-agnostic meta-learning scenario, the distribution of the task p(T) that we want the regression GNN model to adapt to is defined.

[0135] The purpose of Meta-RegGNN is to prepare a regression GNN model for rapid adaptation. Therefore, the GNN may understand the internal features of social network posting behavior relevant to all tasks in p(T). To this end, the parameters of the RegGNN model that respond to modifications in a given task are first found, so that small modifications to the parameters can significantly improve the loss function for any task in social network posting behavior. The model consists of a parameterized function f with parameter Θ. Θ Indicates. When adapting to the new task T i When the latter is updated to Θ′, the updated Θ is defined as:

[0136]

[0137] Here, γ represents the step size hyperparameter. Meta-optimization is performed on the regression GNN model parameters Θ, while the objective is computed using the updated regression GNN parameters Θ′. In fact, Meta-RegGNN aims to optimize model parameters so that one or a few gradient steps produce efficient behavior on new tasks.

[0138] Meta-optimization is performed through tasks to update the parameters Θ of the regression GNN model. The updated Θ of the regression GNN model satisfies:

[0139]

[0140]

[0141] Where γ represents the step size hyperparameter and η represents the metastep size.

[0142] To accurately consider the graphical structure of social network troll behavior and effectively model the correlation between data samples, this embodiment uses a regression GNN network, which consists of two graph convolutional layers and one fully connected layer. Given that the correlation matrix of the connection volume C is symmetric and can have zero or positive eigenvalues, it can be easily regularized to be symmetric positive definite, according to:

[0143] I′=C+μI

[0144] Where I represents the identity matrix, μ > 0. In fact, since positive correlation has proven more important in analyzing brain networks, all negative eigenvalues ​​are set to zero to train the regressive GNN. Therefore, the regressive GNN receives the regularized positive adjacency matrix I′ of the connectome and uses graph convolution to predict the corresponding behavior scores. This reduces the size of the connectome and learns the embeddings. After the first graph convolution operation, a dropout layer for regularization is added. Finally, the resulting embeddings are passed through a fully connected layer that produces a scalar output (IQ score).

[0145] In some embodiments, such as Figure 4 As shown, the metagraph reinforcement learning framework is a GNN-fused meta-reinforcement learning framework, which is a deep reinforcement learning (DRL)-based framework for learning the optimal subset of activation-informed influence maximization (AIM), a general influence maximization (IM) formula that incorporates two types of activations. The framework has several novel aspects: (1) meta-learning for prediction across different graph types and sizes; (2) double-Q learning (a reinforcement learning algorithm) to estimate the sequence of child nodes without solving a computationally intensive optimization problem at each test time; and (3) a single-policy multi-objective reward formula for system balancing multiple AIM objectives.

[0146] The candidate node prediction module of the metagraph reinforcement learning framework provides a set of candidate nodes based on the AIM search algorithm, which is completed using a finite Markov decision process.

[0147] Finite Markov Decision Process

[0148] The task of identifying the optimal set of AIM child nodes is a sequential process, adding one node at a time. More importantly, the factors determining node selection at any step of the process depend solely on the last node added to the sequence (the solution set). Therefore, this process follows the Markov property and is thus formulated as a Markov Decision Process (MDP). Furthermore, at any step of the process, the action space is finite; that is, nodes must be selected from a finite set of nodes. Therefore, more precisely, this process can be called a finite MDP. The key elements defining any finite MDP in the context of Deep Reinforcement Learning (DRL) are:

[0149] The state represents the current solution set, where nodes are appended sequentially to form the final AIM solution set. Therefore, the cardinality of the state increases continuously as the process progresses. Thus, a fixed-dimensional state representation vector (X) is required. t ). X t The state of the system at any time step t should be characterized according to the selected nodes. Therefore, X t It can be represented as:

[0150] X t =f(S) t )

[0151] Among them, S t The partial AIM solution is set at time t, and f is the transformation operator. When nodes in the state are sampled from the graph, the appropriate choice of operator f will be based on the transformation of the graph neural network.

[0152] The action refers to adding a new node u to the partial solution set S. t The process.

[0153] The reward quantifies the benefit of taking action. The AIM formula has two objectives, which generate two reward functions. The first reward (R1) belongs to the marginal gain of the diffusion effect when a particular node is added to the solution set. The second reward (R2) corresponds to the intrinsic probability of a node being added to the solution set. The reward can be written as:

[0154]

[0155] R2(X, a)=p s (a)

[0156] Among them, the operator It calculates the diffusion of the influence of the solution set S in graph G, p s This represents the activation probability parameter, and 'a' represents a specific node.

[0157] Environment: It is a world of agents with which we interact, composed of everything outside the agents. Here, the environment is a graph. These interactions occur continuously: agents select nodes, the graphical environment responds to these actions, and presents new situations to the agents.

[0158] Strategy: A strategy or recommended action that the DRL agent should take in each state of the environment to achieve the learning objective. It is a probability distribution on feasible nodes that can be added to the partial solution set S. t To change the state from X t Move to X t+1 Therefore, the strategy π(a|X) t Choose any state X t The node that generates the highest cumulative return.

[0159] Termination: In each set, the search begins with a random node in the candidate set, and estimated nodes are appended to the partial solution set, one at a time (one per step per set). When the solution set S... t The set terminates when the base number reaches the search budget b.

[0160] Metagraph Reinforcement Learning (GraMeR)

[0161] GraMeR consists of three modules. The first module acts as an environment pool containing training graphs from different series and sizes. The second module provides a set of candidate nodes on which the AIM search algorithm will be implemented. Finally, the third module handles the DRL agent. The process of training the agent begins by randomly selecting a graph G from the pool. i A second module generates a set of candidate nodes. Each training graph is an environment that interacts with the agent's MDP. An event begins with a random node from the candidate set of the sampled graph and continues until the budget is exhausted. At each step of the event, the agent selects the next node based on its current policy. Afterward, the agent updates its current policy by training the network with batch-sampled data from the replay buffer. The replay buffer stores the state, actions, and rewards (graph) of all past steps in the event and environment, allowing the network to utilize known information. Once the event meets the termination criteria, the agent samples a new graph and iteratively trains until the policy converges.

[0162] The agent in GraMeR is trained using Q-learning because it is a discrete finite MDP. Q-learning maximizes the cumulative reward of actions taken by the agent during its interaction with the environment. Future rewards depend on the current action. The optimal value of the action (i.e., the Q-value) corresponds to the optimal policy that maximizes the Q-value. The Q-value can be iteratively updated according to the Bellman equation:

[0163]

[0164]

[0165] Using a single estimator, Q-learning based on the above equation is often overestimated in practice. This estimator determines the best action with the highest Q-value in the next state and the Q-value of that best action. To avoid overestimation, embodiments of this application employ a dual-Q learning method, using two different estimators. One estimator determines the best possible action for the next state, while the other (target network Q-value)... T The metagraph reinforcement learning framework provides the Q-value of the selected action. In some embodiments, the agent of the metagraph reinforcement learning framework is trained via double Q-learning, where one estimator determines the best possible action for the next state, and the other estimator provides the Q-value of the selected action. L (X t a t The corrected update equation is:

[0166] Q L (X t a t )+θ×[r t +γQ T (X t+1 a * )-Q L (X t a t )]

[0167]

[0168] θ is the tuning parameter, and r and γ are parameters. Local network (Q L At each step, training is performed by sampling a batch of data from the playback buffer. The predicted Q-value (i.e., Q0) is then used. L (X t a t )) and the expected Q value (r) from the Bellman equation t +γQ T (X t+1 a * Minimize the mean squared error loss between )) to update the local Q L The parameters of the network. However, the target network Q... TIt's not about explicit training at every step, but rather using the entire Q series after a certain number of episodes. L The network's weights are constantly being updated.

[0169] Meta-Q learning, introduced in GraMeR, utilizes meta-learning properties to quickly and efficiently solve unknown tasks. Typically, meta-reinforcement learning methods involve two optimizer loops. The outer optimizer samples the new environment in each iteration and adjusts the parameters that determine the agent's behavior. In the inner loop, the agent interacts with the environment and optimizes to maximize the reward. As with most environments, obtaining the representation vector of the entire environment is infeasible. Therefore, learning across environments is captured by the outer optimizer. However, in the case of AIM, the environment is a graph, which can be represented very accurately by a single graph embedding vector. Therefore, the outer optimizer is skipped, and the entire environment (graph) information (graph embedding vector) along with the state and actions is provided to the training algorithm. This allows Q-learning to capture changes in the environment through a single optimizer. The agent's precise updates then satisfy the following update pattern:

[0170]

[0171] Q L (X t a t G i )=Q L (X t ,at,G i )+θ×[r t +γQ T (X t+1 a * G i )-Q L (X t a t G i )]

[0172] Among them, Q T Denotes the target network, Q L Indicates the local network, a t X represents the action performed at time t. t The state matrix represents the state executed at time t, θ is the tuning parameter, r and γ are parameters, and G is the state matrix executed at time t. i This corresponds to the sampled graph for episode i. The graph will change as the episodes progress, but for a given set, only one graph will be explored.

[0173] GNN encoding: Similar to graph encoding, it also requires vectors representing states (partial solution sets) and actions (nodes). In this regard, GraphSAGE is used to estimate node embeddings. Subsequently, the state S... tThe embedding vectors of the nodes in the array are aggregated using a mean / maximum operation to obtain a single representation vector X for the entire state. t Since an action corresponds to a single node, it is represented by the corresponding node embedding vector. GraphSAGE is more efficient than traditional GCN because the candidate nature of a particular node as part of the AIM solution set depends primarily on its subgraph. Therefore, GraphSAGE, as a learning method based on induced subgraphs, is a suitable choice. Furthermore, unlike GCN, GraphSAGE does not require complete graph information, while GCN requires the complete adjacency matrix and therefore does not scale well with the graph size.

[0174] Multi-objective shaping: Along with state and action, the reward function needs to be explicitly designed for AIM because it involves multiple objectives. The first objective relates to maximizing the diffusion of influence, while another objective corresponds to maximizing the intrinsic probability of child nodes. Therefore, GraMeR falls into the category of multi-objective DRL.

[0175] A single-policy approach is employed to learn an optimal policy by combining two objectives with known preference weights. Specifically, at each step of an event, the reward is computed separately for each objective and accumulated in a buffer along with the state and action. Then, linear scalarization is used to combine the Q-values ​​of the two objectives to generate a single Q-value for selecting the action.

[0176] Therefore, the detection results of online trolls can be determined collaboratively using the three models proposed in this implementation, specifically in the following manner:

[0177] After obtaining a large number of social network graph models, the metagraph convolutional recurrent network model completes node-level feature learning. It captures the feature representation of node sequences through an encoder-decoder based on recurrent graph neural networks, and combines graph structure learning (GSL) to simultaneously optimize and update node representations and social network graph topology.

[0178] The meta-learning regression graph neural network model completes graph-level regression tasks and designs a meta-learning mechanism to enhance adaptability. It performs task-driven scoring for different social network graphs, thereby generating behavioral scores for each social network graph. In this example, the higher the degree of abnormality of social network behavior, the higher the behavioral score. The behavioral score is used as a sample importance weight to train the overall model.

[0179] The metagraph reinforcement learning framework is used to reinforce node classification. It utilizes the optimized graph structure, node features, and behavior scores of the graph, combined with the metagraph reinforcement learning mechanism, to select anomalous nodes (i.e., troll nodes) from the social network graph in the graph pool using a certain learnable strategy.

[0180] This application's embodiments involve learning graphical structures. In addition to sequence modeling, a general graphical structure is used to capture the correlations between variables (road connections in traffic data). The learnable graph here is a time (input) variable, which handles abrupt changes in upcoming data well. The graph is subdivided, where each node embedding is finely customized based on its prototype (meta-node).

[0181] The metagraph convolutional recurrent network of this application mainly includes graph convolutional recurrent units and meta-learners. The metagraph convolutional recurrent network can easily rely on observed data to remain robust and adapt to normal to non-stationary conditions. A metagraph learner driven by a metanode library is inserted into the encoder-decoder of the graph convolutional recurrent unit. The meta-learner learning includes two steps: (1) querying node-level prototypes from the metanode library; (2) dynamically reconstructing node embeddings using a hypernetwork. This local memory capability enables the modular metagraph learner to substantially distinguish different spatiotemporal patterns over time, which can even be generalized to accidental situations. The metagraph learner explicitly unravels spatial and temporal heterogeneity.

[0182] The meta-learning regression graph neural network model of this application embodiment appropriately incorporates the graph structure of social network posting behavior and effectively models the correlations between them. Furthermore, due to learning, the regression GNN model becomes more flexible while mitigating the impact of certain problems. This network exhibits a unique ability to model the correlations between data and combines global and local topological properties to predict behavior scores. Moreover, it ensures the model's flexibility and supports inductive learning, thereby enhancing the model's generalizability for unknown data.

[0183] The graph primitive reinforcement learning framework of this application is a GNN fused with a meta-reinforcement learning framework for identifying influential nodes in the network. First, a GNN-based candidate node predictor is used to reduce the search space of the IM (Influence Context). Then, deep Q-learning is used to identify IM child nodes with a GNN as the environment encoder.

[0184] This application's embodiments obtain data from major social media platforms and utilize word sense disambiguation technology to disambiguate word senses in multiple languages. It also employs a translation-based cross-lexical resource alignment method, which verifies the accuracy of translation by maximizing similarity metrics. Furthermore, it combines metagraph convolutional recurrent networks with metalearning regression graph neural network models and metagraph reinforcement learning frameworks to jointly detect online troll behavior.

[0185] This application also proposes a computer device, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the steps of the aforementioned metagraph-based network malicious behavior detection method.

[0186] This application also proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the aforementioned metagraph-based network malicious behavior detection method.

[0187] It should be noted that, in the embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0188] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0189] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0190] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims. All of these forms are within the protection scope of this application.

Claims

1. A method for detecting malicious network behavior based on metagraphs, characterized in that, include: Obtain the required user data; The user data is input into the trained network malicious behavior detection model to output detection results. The network malicious behavior detection model includes: a social network graph construction unit, a metagraph convolutional recurrent network model, a metalearning regression graph neural network model, and a metagraph reinforcement learning framework. The social network graph construction unit is used to construct a social network graph based on the user data, and input the constructed social network graph into the metagraph convolutional recurrent network model, the metalearning regression graph neural network model and the metagraph reinforcement learning framework respectively, so as to output the detection result based on the metagraph convolutional recurrent network model, the metalearning regression graph neural network model and the metagraph reinforcement learning framework. The metagraph convolutional recurrent network model includes an encoder, a decoder, and a metagraph learner. The encoder and the decoder are constructed using graph convolutional recurrent units as basic units. The metagraph learner is used to input feedback information to the decoder based on the constructed metanode library and the super network, in response to the output of the encoder. The meta-learning regression graph neural network model is based on a regression GNN network, which includes two graph convolutional layers and one fully connected layer. The metagraph reinforcement learning framework is based on deep reinforcement learning (DRL) and includes a graph pooling module, a candidate node prediction module, and a recognition module. The graph pooling module acts as an environment pool, the candidate node prediction module is used to provide a set of candidate nodes based on the AIM search algorithm, and the recognition module is used to process DRL agents.

2. The network malicious behavior detection method based on metagraphs as described in claim 1, characterized in that, After obtaining the required user data, word sense disambiguation is performed using the following methods: Constructing a monolingual co-occurrence graph G s = <V s E s >; Collect each co-occurring noun or adjective pair (cw) from the user data. i ,cw j And add any co-occurring noun or adjective as a node to the monolingual co-occurrence graph, with each co-occurring word pair connected to an edge (v i ,v j )∈E s Connect, either side based on the corresponding word cw i and CW j The strength of the association between them is assigned a weight w(v) i ,v j )satisfy: w(v i ,v j )=1-max[p(cw i |cw j ),p(cw j |cw i )] Where p(·|·) represents the conditional probability, derived from cw i and CW j The number of contexts that appear simultaneously divided by the number of contexts containing cw j The number of contexts is used to estimate; Configure a set of target languages ​​and combine the monolingual co-occurrence graph G s Expanded into a multilingual graph G ML = <V ML E ML >, where V ML =V s ∪∪ l∈L V l It indicates that it comes from the source (V) s ) or target (V) l A set of nodes representing the content words of a language, E ML =E s ∪∪ l∈L {E l ∪E s，l } is a set of edges; Calculate the root hub in the multilingual graph to distinguish the meaning of the target word in the source language, satisfying: Where d is the damping factor, deg(v) i ) is node v i The number of adjacent nodes, w ij It is node v i and v j The weights of co-occurrence edges between words, the minimum spanning tree (MST) of the graph with the target word as the root, G ML The root word center constitutes its first level; Given the context W of the target word w in the source language, use MST to find the most relevant word in w to eliminate ambiguity.

3. The network malicious behavior detection method based on metagraphs as described in claim 2, characterized in that, Using MST to find the most relevant words in w to disambiguate includes: By calculating the correct central disHub, relevant nodes are found, and only nodes linked to the disHub are retained. W is defined. h It is a set of mapped content words that satisfy: Where d(cw) is a function that assigns weights to cw based on its distance from w, and dist(cw, h) is given by the number of edges between cw and h in the MST; Sum the translation counts to sort each translation; The similarity metric between two translations is defined as follows: in It returns a set of languages. A function for the lexicalization set of word c in Chinese; The accuracy of the translation is verified by maximizing the similarity metric to achieve cross-lexical resource alignment.

4. The network malicious behavior detection method based on metagraphs as described in claim 1, characterized in that, The graph convolutional recurrent unit of the metagraph convolutional recurrent network model adopts the graph convolution operation and the definition of gated recurrent unit (GRU) to represent GCRU as the basic unit, satisfying: Where, X∈R N×C (N represents the number of spatial units), C represents the number of information channels, and H∈R N Represents graph convolution operation ★ G Inputs and outputs, Θ, W K ∈R K×C To approximate the kernel parameters to order K using Chebyshev polynomials, σ represents the activation function, ⊙ represents the product of elements at corresponding positions, and parameters u, r, and C represent the update gate, reset gate, and candidate state in the GCRU unit, respectively. {u，r，C} Indicates the gate parameters; The metagraph convolutional recurrent network model uses a canonical formula for spatiotemporal graph learning, satisfying: This is derived by performing a normalized random walk on the non-negative part of the matrix product of the embeddings of trainable nodes E. To its transpose; The metagraph learner is used to learn from the constructed metanode library. And using a super network, feedback information is input to the decoder, wherein d and d represent the number of memory items and the dimension of each item, respectively; the intermediate variables updated by the metagraph learner are stored in a specific metanode library Φ, and the memory operation relationship during the update process is defined as follows: Where the superscript (i) serves as the row index, H represents t The vector of the i-th node in the vector. It consists of a fully connected layer W Q Hide state Projecting onto the local query space yields the scalar a. j Physically representing a vector Similarity with memory item Φ[j], metanode vector This indicates that it can be further restored to a combination of stored items; The supernetwork is used to embed GSL nodes, generating them based on the meta-node library, and is formulated as follows: Where NN H Indicates a hypernet; The metagraph learner uses the Metagraph Convolutional Recurrent Network (MegaCRN) as a general framework.

5. The network malicious behavior detection method based on metagraphs as described in claim 4, characterized in that, The meta-learning regression graph neural network model is implemented based on a regression GNN model, which includes two graph convolutional layers and one fully connected layer. During meta-learning, each task T = {L(g1, s1, ..., g...} H s H ), q(g1), q(g) t+1 |g t s t H} is defined by the loss function L, the initial observation distribution q(g1), and the transition distribution q(g2). t+1 |g t s t It consists of the set length H; For a regression GNN model, the loss function satisfies: Where g (j) s (j) They represent the tasks from task T respectively. i The input and output of the sample; The update Θ of the regression GNN model satisfies: Where γ represents the step size hyperparameter and η represents the metastep size.

6. The network malicious behavior detection method based on metagraphs as described in claim 5, characterized in that, The candidate node prediction module of the metagraph reinforcement learning framework provides a set of candidate nodes based on the AIM search algorithm, which is completed using a finite Markov decision process.

7. The network malicious behavior detection method based on metagraphs as described in claim 6, characterized in that, The agent in the metagraph reinforcement learning framework is trained using double Q-learning, where one estimator determines the best possible action for the next state, and the other estimator provides the Q-value of the selected action, with the following update mechanism: Q L (X t ,a t ,G i )=Q L (X t ,a t ,G i )+θ×[r t +γQ T (X t+1 ,a * ,G i )-Q L (X t ,a t ,G i )] Among them, Q T Denotes the target network, Q L Indicates the local network, a t X represents the action performed at time t. t The state matrix represents the state executed at time t, θ is the tuning parameter, r and γ are parameters, and G is the state matrix executed at time t. i The sampling pattern corresponding to the i-th set.

8. A computer device, characterized in that, include: A processor and a memory, wherein the memory stores a computer program that, when executed by the processor, implements the steps of the metagraph-based network malicious behavior detection method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the metagraph-based network malicious behavior detection method as described in any one of claims 1 to 7.

Images