Social robot collaborative group detection method based on multi-order collaborative knowledge graph

Abstract

The invention relates to the technical field of network security, and discloses a social robot collaborative group detection method based on a multi-order collaborative knowledge graph. According to the method, user data in a Chinese social platform is collected through a crawler, a multi-order collaborative interaction graph is constructed based on the interaction relation of users of the social platform and then input into a graph encoder module, node feature vectors are obtained and input into a time perception module and a graph perception module respectively, and a time sequence prediction task and a side link prediction task are executed. Meanwhile, the model is constructed based on a time-graph semantic consistency perception module of a self-attention mechanism. And finally, inputting a node feature vector obtained by training under the guidance of double tasks into a group detection module constructed based on a Gaussian mixture model and a K-means clustering method for detection. According to the method, the complex interaction relationship among the users in the social robot group and the abnormal mode in the time sequence activity can be effectively captured, and efficient detection of the social robot collaborative group in the social network platform is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network security, and particularly relates to a social robot collaborative group detection method based on a multi-order collaborative knowledge graph. BACKGROUND

[0002] In the current digital era of social life, numerous social activities are emerging in the form of hot public opinions in the network space. Individuals and organizations initiating activities, with the help of powerful information dissemination tools, disregard geographical restrictions, carry out propaganda activities, and reshape people's cognition in the information cocoon effect. More and more evidence shows that malicious media manipulation activities are emerging. In these efficient media manipulation movements, malicious social robot groups play a key role.

[0003] A malicious social robot group is a user cluster composed of multiple automated social accounts. They can spread content containing false opinions at an extremely fast speed through posting, liking, and forwarding social behaviors in a short time, and hold humans in the "information filter" they create. Today, malicious social robot groups have successfully penetrated into many fields, seriously disrupting the normal order of social networks. Malicious social robot groups are emerging as a new weapon of media manipulation, posing a serious threat to social network public opinion security and social stability. Therefore, in order to protect public group cognitive security and maintain national network stability, it is urgent to carry out research on social robot group detection technology.

[0004] In real social networks, in order to maximize the impact in a short time, social robot groups will coordinate with each other to form a huge and complex false information network, publish a large amount of information content, and submerge legitimate users in a huge false information stream. Therefore, social robot groups have the following two main characteristics:

[0005] (1) Hidden association

[0006] Compared with direct interaction, malicious social robot groups will achieve the common goal more efficiently through more organic coordination. For example, social robot groups may participate in the discussion of the same topic at the same time to achieve the purpose of hot topic hype. The coordination behavior of social robots makes their activities have hidden interrelations rather than direct interaction.

[0007] (2) Highly consistent time sequence activities

[0008] In order to cause the maximum impact in a short time, the behavior of the coordinated robot group is collectively abnormal at a certain moment, showing an unreasonable activity frequency in a certain interval of event development, and being distinguished from normal social users. However, due to the complexity of the coordinated behavior of the robot group, how to model the hidden connection between groups, find the abnormal situation in time sequence, and organically combine the two is a challenging task.

[0009] The existing social robot group detection method still faces the following problems:

[0010] (1) The information depth and breadth of the Chinese dataset of social robots are insufficient; the existing social robot detection research dataset mainly comes from English social networks, and there are few Chinese mainstream social platform research datasets. The existing research datasets have deficiencies in user information dimensions, labeled data scale, and user diversity, and cannot effectively support the training and evaluation of social robot detection methods;

[0011] (2) It is difficult to capture the complex interaction between users in the social robot group and the abnormal pattern in the user time sequence activity, which greatly reduces the detection effect; for example, the detection method based on prior knowledge graph only represents the interaction between two users as a weighted edge, which leads to poor ability to capture complex interaction; in the detection method based on representation learning, only the abnormal pattern in the time sequence activity is considered, and there is a lack of ability to jointly model the collaborative prior knowledge graph and the time sequence activity, which cannot organically capture the dynamic consistency between the time sequence activity and the complex collaborative interaction relationship, thus mining deeper abnormal patterns of the collaborative group, resulting in insufficient detection ability of the method.

[0012] Therefore, a social robot collaborative group detection method based on a multi-order collaborative knowledge graph is proposed to solve the above problems. SUMMARY

[0013] To solve the above problems, the application aims to provide a social robot collaborative group detection method based on a multi-order collaborative knowledge graph. According to the direct interaction relationship, collaborative interaction relationship and corresponding time stamp of the user of the social platform, a multi-order collaborative interaction graph is constructed. The multi-order collaborative interaction graph is input into the Graph Transformer encoder module, and after obtaining the node feature vector, it is respectively sent into the time perception module and the graph perception module to perform the time series prediction task and the edge link prediction task, so as to make the Graph Transformer encoder module have the graph and time perception ability through back propagation. In this process, the model constructs a time-graph semantic consistency perception module based on the self-attention mechanism to enhance the perception ability of the model to the graph structure and the time sequence. Finally, the node feature vector obtained by training under the guidance of the double task is input into the group detection module constructed based on the Gaussian Mixture Model (Gaussian Mixture Model, GMM) and the K-means clustering method for detection. This method can effectively capture the complex interaction relationship between users in the social robot group and the abnormal pattern in the time sequence activity, realize efficient detection of the social robot collaborative group in the social network platform, and the technical scheme is as follows:

[0014] A social robot collaborative group detection method based on a multi-order collaborative knowledge graph, comprising the following steps:

[0015] Step S1, collecting social platform user data through a distributed crawler, and constructing a social robot data set;

[0016] Step S2, based on the collected user data, extracting user relationship features and corresponding time stamps; the user relationship features include direct interaction relationship and collaborative interaction relationship;

[0017] Step S3, according to the user relationship features and the corresponding time stamps, constructing a direct interaction weight graph and a collaborative interaction weight graph respectively, and then merging the two to obtain a multi-order collaborative interaction weight graph; with the weight abnormal edges in the multi-order collaborative interaction weight graph, a multi-order collaborative interaction graph is constructed;

[0018] Step S4, constructing a social robot collaborative group detection model; the multi-order collaborative interaction graph is the input data of the social robot collaborative group detection model;

[0019] Step S5, training the social robot collaborative group detection model;

[0020] Step S6, using the social robot collaborative group detection model to detect the collaborative group of the social platform to be detected.

[0021] Further, the step S1 comprises the following steps:

[0022] S11. Build a distributed crawler based on the Celery framework, including message middleware, crawler nodes, node monitoring engine, and storage module:

[0023] The message middleware is implemented based on the third-party component Redis and is used to distribute tasks downwards; the crawler nodes are used to execute anti-crawling strategies, establish HTTP connections with social platform servers, collect user data, and transfer it to the storage module for storage; the node monitoring engine is implemented using Sentry and is used to capture abnormal situations of crawler nodes; the storage module uses a MongoDB document-oriented NoSQL database to store the collected user data.

[0024] S12. Collect user data from social platforms by constructing a distributed crawler, and collect users who participated in trending topics, and expand the collection scale based on the BFS algorithm.

[0025] S13. Manually label the collected user data to construct a social robot dataset with real labels.

[0026] Furthermore, the direct interaction relationships include follow interaction, like interaction, comment interaction, forwarding interaction, and mention interaction; the collaborative interaction relationships include shared topics, shared follows, shared mentions, shared URLs, shared likes, shared forwards, and shared comments.

[0027] Further, step S3 includes the following steps:

[0028] S31. Using users as nodes, if there is a direct interaction relationship between two nodes, an edge is established between them. The timestamp corresponding to the direct interaction relationship is used as the attribute of the edge, and the co-occurrence frequency of the two nodes is used as the weight of the corresponding edge. A direct interaction weight graph is constructed, and the formula for the edge weight is as follows:

[0029] (1);

[0030] in, For edge weights, For the first 1 node For the first 1 node Let be the set of edges. It is an edge that belongs to the edge set;

[0031] S32. Obtain collaborative interaction events based on collaborative interaction relationships; using users as individual nodes and collaborative interaction events as event nodes, if a collaborative interaction relationship exists between an individual node and an event node, establish an edge between the individual node and the event node to construct a bipartite graph; project the bipartite graph, using the timestamp corresponding to the collaborative interaction relationship as the attribute of each edge. weight is the number of co-occurrence of neighbor nodes of two individual nodes and , a collaborative interaction weight graph composed of a set of individual nodes is generated, the neighbor nodes represent other individual nodes connected to the same event node, and the formula is as follows:

[0032] (2);

[0033] wherein, denotes the neighbor node set of individual node denotes the neighbor node set of individual node is the number of co-occurrence of neighbor nodes of individual node and individual node

[0034] S33, the direct interaction weight graph and the collaborative interaction weight graph are merged to generate a multi-order collaborative interaction weight graph, and the edge weight of the multi-order collaborative interaction weight graph is obtained by adding the weights of the same edges of the direct interaction weight graph and the collaborative interaction weight graph;

[0035] S34, the direct interaction weight graph and the collaborative interaction weight graph are merged to generate a multi-order collaborative interaction weight graph, and based on a preset threshold of edge weight, the edges in the multi-order collaborative interaction weight graph are filtered, only the abnormal edges are reserved, and a multi-order collaborative interaction graph is obtained.

[0036] Further, the social robot collaborative group detection model in the step S4 comprises a GraphTransformer encoder module, a graph perception module, a time perception module, a perception interaction module and a group detection module:

[0037] The graph perception module learns the probability distribution function of edge link prediction in the multi-order collaborative interaction graph, so that the model can identify abnormal edges;

[0038] The time perception module learns the conditional probability density function of the time sequence of node interaction in the multi-order collaborative interaction graph at time, so that the model can identify time sequence abnormal nodes;

[0039] The perception interaction module learns the time sequence and graph structure distribution characteristics learned by the graph perception module and the time perception module through the self-attention mechanism, so that the model can share the time sequence and graph structure distribution characteristics learned by the two modules in the two modules;

[0040] The group detection module combines the K-means clustering method of Gaussian Mixture Model (GMM) to identify the collaborative group after the graph perception and time perception interaction, and outputs the social robot collaborative group detection result.​​​

[0041] Further, the social robot coordination group detection model specifically comprises:

[0042] (1) Graph Transformer encoder module:

[0043] The Graph Transformer encoder module comprises a plurality of layers of Graph Transformer encoding layers;

[0044] The initialization node feature sequence of the multi-order collaborative interaction graph is obtained by Node2vec, and is input into the Graph Transformer encoder module, and the node feature vector of the user is obtained by the Graph Transformer layer The node feature vector is input into the graph perception module and the time perception module;

[0045] (2) Graph perception module:

[0046] The node feature vector obtained by the Graph Transformer encoder module is subjected to spatial transformation by a linear layer, and the formula is as follows:

[0047] (3);

[0048] wherein, is the node feature vector, and are learnable parameters, is the graph perception node feature vector;

[0049] The edges in the multi-order collaborative interaction graph are defined as positive edges , the negative sampling edges are generated by batch negative sampling on the positive edges in the graph , and the negative sampling edges are fused with the positive edges, and the fused edges are represented as the concatenation of the graph perception node feature vectors , and then sequentially pass through a linear layer, a Sigmoid nonlinear activation function layer and a Softmax activation function layer to realize edge link prediction, and the formula is as follows:

[0050] (4);

[0051] wherein, is a probability distribution function learned in the graph perception process, , are learnable parameters, is a Sigmoid nonlinear activation function, is a Softmax activation function; and​ Representative node and nodes Graph-aware node feature vectors Represents the node With nodes The prediction result indicates whether there is an edge between them. Positive edges are labeled as 1, and negative edges are labeled as 0.

[0052] (3) Time-aware module:

[0053] The node feature vectors obtained by the Graph Transformer encoder module are transformed into time-aware node feature vectors through a linear layer. ;

[0054] Construct a time series based on the nodes, edges, and timestamps in the multi-level collaborative interaction graph. , indicating that the first User nodes of interactive events and time ;

[0055] Based on translation time-invariant kernel function The time series is converted into a time vector with the same dimension as the feature vector of the time-aware node. The time vector is added to the feature vector of the time-aware node to obtain a hybrid information vector that combines spatial and temporal information. The hybrid information vector is input into the Transformer encoder to obtain historical information at each time point. ;

[0056] For historical information, a log-normal distribution is used for modeling. Conditional probability density function at time t and using a multilayer perceptron (MLP) and Activation function prediction in The formula for user nodes that constantly interact is as follows:

[0057] (5);

[0058] in, , , These represent the mixture weights, mixture variance, and mixture mean, respectively. yes User nodes that interact at any given moment; This represents the number of log-normal distributions used to fit probabilities in the model;

[0059] (4) Perception and Interaction Module:

[0060] Graph-aware node feature vectors and time-aware node feature vectors respectively, and the interaction embedding vectors are generated through cross attention mechanism and respectively, and the interaction embedding vectors are generated through cross attention mechanism and respectively, and the interaction embedding vectors are generated through cross attention mechanism and ; the original graph perception node feature vectors and and time perception node vectors are replaced by the perception interaction vectors for subsequent graph perception and time perception;

[0061] (5) Group detection module:

[0062] After the graph perception and time perception interaction, the Graph Transformer encoder module can generate embedding vectors fitting the distribution characteristics of time series and graph structure ; the embedding vectors are modeled through Gaussian Mixture Model GMM, and the embedding vectors are optimized; the optimized embedding vectors are clustered using K-means clustering method, and the social robot collaborative group detection result is output.

[0063] Further, the step S5 comprises:

[0064] In combination with the graph perception module, the time perception module and the perception interaction module, a joint perception loss function is constructed;

[0065] The loss function of Gaussian Mixture Model GMM is used as the group identification loss function;

[0066] The training target of the social robot collaborative group detection model is to minimize the joint perception loss function to realize the joint perception of graph structure and time series, and to minimize the group identification loss function to realize the collaborative group modeling.

[0067] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0068] (1) The present application constructs a high-performance distributed data acquisition crawler based on Celery framework to acquire user data in Chinese social platforms, and expands the data scale based on hot topics and breadth-first search algorithm, thereby improving the data acquisition efficiency and supporting the construction of Chinese social robot data set;

[0069] (2) The application can integrate the prior knowledge of direct interaction, collaborative interaction and time sequence of user activities into a unified graph structure by constructing a direct interaction weight graph and a collaborative interaction weight graph based on the direct interaction relationship and the collaborative interaction relationship of the user of the social platform and the corresponding time stamp, merging the two into a multi-order collaborative interaction weight graph, and retaining the edges with abnormal weights, thereby more comprehensively representing the complex interaction relationship and dynamic collaboration of the time sequence activities of the user and the social robot;

[0070] (3) The application constructs a social robot collaborative group detection model by using a Graph Transformer, a history information encoder based on a Transformer, a self-attention mechanism and a K-means clustering method combined with a GMM, which is used for group detection of the obtained multi-order collaborative interaction graph, can effectively capture the complex interaction relationship between users in the social robot group and the abnormal patterns in the time sequence activities, and realizes efficient detection of the social robot collaborative group in the social network platform. BRIEF DESCRIPTION OF DRAWINGS

[0071] Figure 1 It is a distributed data acquisition crawler structure diagram.

[0072] Figure 2 It is a strategy diagram for constructing a microblog social robot dataset.

[0073] Figure 3 It is a structure diagram of a social robot collaborative group detection model.

[0074] Figure 4 It is a structure diagram of a time-graph semantic consistency perception module.

[0075] Figure 5 It is a schematic diagram of the social robot group with the highest degree of collaboration. DETAILED DESCRIPTION

[0076] The technical solutions in the application will be described in detail below with reference to the drawings in the application, so that the concept of the application, the technical problems solved, the technical features constituting the technical solutions and the technical effects brought by the application can be further understood.

[0077] The overall framework of the method includes three parts: data acquisition and annotation, multi-order collaborative knowledge graph construction and detection model construction.

[0078] (A) Data acquisition and annotation

[0079] The developed distributed Sina Weibo crawler is used to collect user data in the social network platform according to the data acquisition strategy, and the data is annotated, and then a microblog social robot dataset is constructed, which provides effective data support for the application.

[0080] (B) Multi-order collaborative interaction graph construction

[0081] Using the constructed microblog social robot dataset, a direct interaction weight graph is constructed according to five types of direct interactions: attention, likes, comments, forwards, and mentions. Based on seven types of collaborative interactions: common topics, common attention, common mentions, common URLs, common likes, common forwards, and common comments, a two-part graph is constructed and projected into a collaborative interaction weight graph. The direct interaction weight graph and the collaborative interaction weight graph are merged into a multi-order collaborative interaction graph, and filtered according to a pre-set edge weight threshold, retaining the connections of collaborative groups with abnormal interaction frequencies, to construct a multi-order collaborative interaction graph.

[0082] (C) Detection model construction

[0083] Combining Graph Transformer, a Transformer-based historical information encoder, a self-attention mechanism, and a K-means clustering method that integrates GMM (Gaussian Mixture Module), an MLK-TGCA-MD social robot group detection model is constructed. The constructed multi-order collaborative interaction graph is input into the MLK-TGCA-MD social robot group detection model, effectively capturing the complex interactions between users in the social robot group and the abnormal patterns in the user's time series activities, and achieving efficient detection of social robot collaborative groups on social networking platforms.

[0084] Specifically includes the following steps:

[0085] Step S1, collect user data on social platforms through distributed crawlers, and construct a social robot dataset;

[0086] In recent years, although researchers have established several social robot research datasets, most of these datasets are for English social platforms such as Twitter, and there are few datasets for Chinese social platforms. Most of the datasets for Chinese social platforms have the problem of missing information dimensions. For example, the most representative dataset SWLD-20K in current Chinese social platforms has the following problems: 1) User attribute information is incomplete, lacking key attributes such as creation date and geographic location; 2) The size of historical tweet information is small, containing only 214,506 tweets, which cannot effectively reflect the tweet activity characteristics of social robots; 3) Lack of social relationships and interaction relationships. These problems make advanced social robot group detection methods based on multi-modal information unable to effectively train and evaluate.

[0087] To solve the problem, the application uses a high-performance distributed data collection crawler based on the Celery framework, formulates a data collection strategy to collect user data, and then manually annotates the collected user data, thereby constructing the microblog social robot dataset of the application.

[0088] Step S11, constructing a data collection crawler

[0089] In order to improve the data collection efficiency and support the construction of the microblog social robot dataset, a high-performance distributed data collection crawler is constructed based on the Celery framework to collect various user data. Figure 1 As shown in the structure of the crawler,

[0090] (a) Message middleware

[0091] The main function of the message middleware is to distribute tasks downward and avoid resource waste caused by repeated execution of tasks. The crawler constructed by the application implements the message middleware based on the third-party component Redis. Redis is a Key-Value database supporting network interaction and memory persistence. As a message middleware, it has the following advantages: ① Redis can meet the data integrity and reliability of large-scale collection task storage; ② Redis supports rich data structures, which can meet the needs of large-scale data collection tasks; ③ Redis supports multi-way IO service, which can guarantee distributed task sending in large-scale data collection; ④ Redis provides cluster mode and sentinel mode, which can automatically transfer to normal nodes when a single point fails without any manual intervention.

[0092] (b) Crawler node

[0093] The crawler node is an execution unit for collecting tasks. First, in order to break through the data collection blockade of the social platform, the crawler node will perform various anti-crawling strategies such as proxy IP scheduling and request packet forgery. Then, the crawler node will perform resource scheduling to obtain the corresponding resources required in the data collection process. Next, the collection node will construct a request interface, establish an HTTP link with the social platform server, obtain user data and perform parsing. Finally, the crawler node will transfer the data to the data storage module for storage.

[0094] (c) Node monitoring engine

[0095] During the execution of the crawler node, the Celery framework sends events to the designated consumer (Receiver) in the form of events when the running status of the crawler node (Worker) changes (e.g., when a task is received). Users can view the task execution status based on the Eventid. Within the Celery framework, Celery Flower manages the Celery task process through the Event mechanism. Furthermore, to capture node anomalies, this invention uses Sentry to capture exceptions generated during crawler runtime. Various custom anomalies are defined in Sentry within the collection node (Worker), such as Page Exception (page parsing failure) and NoSpdier Exception (no available crawlers).

[0096] (d) Data storage

[0097] The web crawler uses MongoDB, a document-oriented NoSQL database, and fully utilizes its flexible BSON (similar to JSON) document structure to store the crawled data in a unified collection. To address the issue of interruptions during the crawling process due to abnormal situations, requiring data to be crawled again, but storing some data in the database at this time may lead to data duplication and waste of storage resources, the web crawler built in this invention designs a distributed Bloom filter based on Redis to ensure that data is not stored repeatedly, thereby achieving the goal of saving storage resources.

[0098] Step S12: Use the constructed crawler to collect and expand user data from social network platforms;

[0099] like Figure 2 As shown, in this embodiment, the existing dataset SWLD-20K is selected as the basic user data for collection. Using a constructed crawler, user attribute data, tweet data, and social and interaction relationship data of still-active users in the SWLD-20K dataset are collected to expand the information dimensions of the dataset. Based on two trending social topics: #Russia*War# and #New*Epidemic#, active participants in these discussions are collected, and based on a breadth-first search algorithm, the social and interaction relationships of users discussing these trending topics are continuously expanded to increase the size of the dataset. Specifically, the following steps are included:

[0100] (a) Information Dimension Expansion

[0101] To address the issue of incomplete information dimensions in existing Chinese social robot datasets mentioned earlier, the SWLD-20K dataset was expanded in terms of information dimensions. Specifically, accounts suspended by Weibo were removed from the SWLD-20K dataset to obtain a seed user set. Then, a high-performance distributed data collection crawler was used to supplement the seed user data in the following three aspects:

[0102] ①Attribute data

[0103] Collect almost all available user attributes of Sina Weibo seed users, including account creation time, account credit level and other attributes missing in the previous data set;

[0104] ②Tweet data (text semantic information)

[0105] Collect about 150 tweets recently published by seed users, as well as like count, comment count, retweet count, tweet publishing time and other tweet metadata information contained in the tweets;

[0106] ③Social and interaction relationship data

[0107] Under the privacy restrictions of the microblog platform, all available user fans and followers (social relationships) are collected; at the same time, the likes, retweets, comments and corresponding time stamps of these users in the first 150 tweets are collected.

[0108] (b) Data scale expansion

[0109] In order to better reflect the characteristics of the latest social robots in the Sina Weibo platform, user data participating in the discussion of the hot social topics #Russian war# and #new epidemic# are collected from the microblog platform, which have caused large-scale microblog user discussion. Among them, a large number of highly suspicious accounts are observed, some of which form a botnet, use uniform avatars, publish the same content, express the same views, and have more complex behavior patterns than before.

[0110] The data scale expansion algorithm based on breadth-first search algorithm is used to expand the scale of the data set (user data participating in the discussion). Specifically, based on the search of hot topic keywords, the initial root node of the user is obtained The breadth-first search is used for the users in the root node to collect their attributes, tweets, social relationships and interaction relationships. Each collected user is taken as an initial node, its layer number is set to the 0th layer, and information expansion is carried out based on their friends (social relationships) and interaction relationships, wherein the users in the After three layers of expansion, the complete expanded data set is obtained.

[0111] Step S13, data labeling is performed on the expanded social network platform user data to construct a social robot data set;

[0112] For example Figure 2 ​The expanded SWLD-20K dataset is manually annotated. Based on the data annotation rules of SWLD-20K, the annotation method proposed by SWLD-20K according to the characteristics of user profiles, text content, and behavior characteristics of accounts is used, including the following six annotation indicators for social robot data:

[0113] (1) Completeness of user account information

[0114] Legal users of Sina Weibo usually configure complete attribute information, while the attributes of social robots usually show low level, lack of avatars, and strange character and number combinations as nicknames.

[0115] (2) Rationality of user social relationships

[0116] Social robots will follow a large number of people, but it is difficult to attract fans, resulting in a large difference between the number of fans and followers.

[0117] (3) Interaction frequency with other users

[0118] Since robot tweets are difficult to attract normal users and lack real social relationships, there are few interactive relationships.

[0119] (4) Originality of user posts

[0120] Since robots will be used to hype hot tweets, this presents a large number of forwarding behaviors.

[0121] (5) Regularity of posting time

[0122] Since robots need to hype in a short period of time, their posting time will show an abnormal surge.

[0123] (6) Quality of user tweet content

[0124] Social robot posts often have misuse of punctuation marks, ambiguous semantics, and confusion of context logic.

[0125] According to the above six annotation indicators, the expanded SWLD-20K dataset is assigned a human or social robot label to ensure the quality of the annotation, and a microblog social robot dataset is constructed, as shown in Table 1:

[0126] Table 1 Description of microblog social robot dataset .

[0127] Step S2, based on the collected user data, extracting user relationship features and corresponding time stamps; the user relationship features include direct interaction relationship and collaborative interaction relationship;

[0128] Social bots collaborate in various ways to generate buzz, drive traffic, and manipulate public opinion, resulting in exceptionally close interaction patterns. Furthermore, to achieve efficient manipulation, social bots not only like and retweet each other's posts but also collaborate to generate buzz for specific topics or posts. Therefore, in constructing the prior knowledge graph, we considered two levels of collaborative knowledge: direct interaction relationships and collaborative interaction relationships, and extracted their corresponding timestamps. The multi-level collaborative knowledge (user relationship features) is shown in Table 2.

[0129] Table 2 Multi-level collaborative knowledge .

[0130] Step S3: Based on the user relationship characteristics and their corresponding timestamps, construct a direct interaction weight graph and a collaborative interaction weight graph respectively, and then merge the two to obtain a multi-level collaborative interaction weight graph; construct a multi-level collaborative interaction graph using the edges with abnormal weights in the multi-level collaborative interaction weight graph.

[0131] Step S31: Construct a direct interaction weight graph based on the direct interaction relationships between users and their corresponding timestamps;

[0132] Direct interaction refers to direct interaction between social bots within a collaborative group, without the involvement of a third-party node. The direct interaction relationships considered in this invention are shown in Table 2, including five types: follow interaction, like interaction, comment interaction, share interaction, and mention interaction.

[0133] Using the Weibo social bot dataset, a direct interaction weight graph was constructed based on five direct interaction types. This method is used to capture explicit behavior and reflect direct relationships. Specifically, it uses the user set as the node set of the direct interaction weight graph, user attributes as node attributes, and five direct interaction types as edge types. If one of the five direct interaction relationships exists between two nodes, an edge is constructed between the two nodes. , The co-occurrence frequency is used as the corresponding edge weight :

[0134] (1);

[0135] in, For the first 1 node For the first 1 node Let be the set of edges. Let the edge be a single edge belonging to the edge set. For example, if there is a like relationship and a comment relationship (a direct interaction relationship) between node 1 and node 2 (i.e., user 1 and user 2), then the edge weight between node 1 and node 2 is... It is the sum of the direct interaction relationships between two nodes, i.e. .

[0136] Step S32: Construct a collaborative interaction weight graph based on the collaborative interaction relationships between users and their corresponding timestamps;

[0137] Collaborative interaction is a more efficient method for collaborative group activities, involving the participation of third-party nodes. Through this method, collaborative groups can better form synergy and achieve common goals. The collaborative interaction relationships considered in this invention are shown in Table 2, including seven types: shared topics, shared interests, shared mentions, shared URLs, shared likes, shared shares, and shared comments.

[0138] Using the Weibo social robot dataset, a collaborative interaction weight graph was constructed based on seven types of collaborative interaction relationships:

[0139] (1) Construct a bipartite graph It contains two types of nodes: individual nodes, which use users in the Weibo social robot dataset as individual nodes in the bipartite graph; and event nodes, which obtain seven types of collaborative interaction events based on seven types of collaborative interaction relationships, and use these seven collaborative interaction events as seven types of event nodes in the bipartite graph. If there is a collaborative interaction relationship between an individual node and an event node, then an edge is constructed between the individual node and the event node.

[0140] Taking the HashTag (a topic tag in a "common topic") event as an example, multiple users may use the same HashTag in tweets at the same time. In this case, multiple individual nodes will connect to the same HashTag node, thus constructing a bipartite graph.

[0141] (2) Based on the completed bipartite graph Projecting this data generates a projected weight graph consisting of the target node set, i.e., a collaborative interaction weight graph. The projection process transforms the relationships between individual nodes and event nodes in a bipartite graph into relationships between individual nodes themselves. Then, the edge weights between individual nodes are determined based on the co-occurrence number of their neighboring nodes. Each edge weight There are two body nodes. and The co-occurrence number of neighboring nodes (other individual nodes connected to the same event node) is calculated using the following formula:

[0142] (2);

[0143] wherein, denotes the neighbor node set of individual node , denotes the neighbor node set of individual node , is the co-occurrence number of neighbor nodes of individual node and individual node , i.e. the number of common neighbor nodes. For example, individual node and individual node simultaneously publish a tweet with the same Hashtag (denoted as event node ) and simultaneously forward a tweet (denoted as event node ), at this time, the co-occurrence neighbor nodes of individual node and individual node are and , thus the weight of is 2 (the number of co-occurrence neighbor nodes).

[0144] Step S33, merging the direct interaction weight graph and the collaborative interaction weight graph, and retaining the edges with abnormal weights, to construct a multi-order collaborative interaction graph;

[0145] (1) After obtaining the direct interaction weight graph and the collaborative interaction weight graph , the two graphs are merged to generate a multi-order collaborative interaction weight graph , the edge weight of the merged graph is obtained by adding the weights of the same edges in the two graphs and . For example, the edge weight between node and node in the direct interaction weight graph is , the edge weight between node and node in the collaborative interaction weight graph is , thus the weight between node and node in the multi-order collaborative interaction weight graph obtained after merging the two is

[0146] (2) Considering that the interaction frequency of collaborative groups is significantly higher than that between ordinary users, in order to capture collaborative group accounts with abnormal interaction patterns, a multi-level collaborative interaction weight graph is weighted based on a preset edge weight threshold. edge set in Perform filtering to retain only edges with abnormally high interaction frequencies:

[0147] (8);

[0148] in, This is a set of edges with abnormally high interaction frequency. It is a pre-set edge weight threshold. In the settings of this invention, The 90th quantile of all edge weights is the multi-order collaborative interaction weight graph. After sorting all edge weights in ascending order, the weights at the 90th percentile are selected as the edge weight threshold, and all edges below this threshold are removed, resulting in a multi-order collaborative interaction graph. .

[0149] Step S4: Construct a social robot collaborative group detection model (MLK-TGCA-MD model); the multi-level collaborative interaction graph is the input data for the social robot collaborative group detection model;

[0150] like Figure 3 As shown, the social robot collaborative group detection model consists of a Graph Transformer encoder module, a time-... Figure 1 It consists of three main parts: consistency perception module, group detection module, and time- Figure 1 The consistency perception module includes a graph perception module, a time perception module, and a perception interaction module;

[0151] 1. Graph Transformer Encoder Module

[0152] The Graph Transformer encoder module encodes nodes in the input multi-order collaborative interaction graph into node feature vectors through a self-attention mechanism. Specifically:

[0153] The acquired multi-level collaborative interaction graph Input the Graph Transformer encoder module, and the model passes... The Graph Transformer encoding layer encodes the feature representations of target nodes in the graph structure. Through a self-attention mechanism, the Graph Transformer effectively captures the interactions between nodes and learns richer representations. Since the Graph Transformer requires nodes to initialize features, Node2vec is used to perform feature encoding. The nodes in the sequence are initialized to obtain the node initialization feature sequence. Initialize the number of Graph Transformer encoding layers in the feature sequence. Then, the Graph Transformer encoding layer computes each node. arrive Bullish attention:

[0154] (9);

[0155] in, , , For trainable parameters, It is a feature of edge (i,x) The representation after linear mapping corresponding to c attention heads; It is a dot product function. For the feature dimension of the embedded node, MLK-TGCA-MD sets it to 128; for the th... Size, source node Feature-based query vector generation The key-value vector is generated by the destination node. Generated through linear transformation edge information Add it to the query vector and key-value vector. , and These are trainable parameters. After multi-head attention computation, message aggregation generates the input feature vector for the next layer:

[0156] (10);

[0157] in, and It is a set of trainable parameters, the target node After linear transformation, a value vector is generated. This vector is then added to the edge information and multiplied by the attention weights to obtain the feature input for the next layer. Then, similar to Transformer, After passing through a residual structure, we obtain The final output of the layer ; It is represented by the current layer node Candidate residual representation obtained through linear transformation It is a gating factor used to control and The fusion ratio between them , and It is a set of trainable parameters.

[0158] (11);

[0159] Finally, MLK-TGCA-MD uses a graph encoder to encode nodes in a multi-order collaborative interaction graph into node feature vectors. For later use - Figure 1 Consistency perception and group detection.

[0160] 2. Time-Graph Semantic Consistency Awareness Module

[0161] like Figure 4 As shown, it includes a graph perception module, a time perception module, and a perception interaction module;

[0162] (1) Graph perception module

[0163] The graph awareness module learns the probability distribution function of edge link prediction in a multi-order collaborative interaction graph, enabling the model to identify abnormal edges.

[0164] 1) Generation of negative sampling edges

[0165] Multi-level collaborative interaction graphs contain information about complex collaborative interactions between nodes, i.e., graphs. We define edges that already exist as having positive labels; to better perceive the interaction patterns between nodes, we further refine the existing edges... Perform batch negative sampling to generate a set of negative sampling edges. Marked as a negative label:

[0166] (12);

[0167] In the negative sampling process, to ensure the efficiency and effectiveness of sampling, the nodes in the existing edges are decomposed and edges are generated with other nodes in the graph. The geometric meaning of these edges is to describe the non-intersecting relationship between the nodes.

[0168] 2) Link Predictive Decoding

[0169] negative sampling edge and positive side After fusion, the node feature vectors generated by the Graph Transformer encoder undergo spatial transformation through a linear layer to obtain the graph-aware node feature vectors, as shown in the following formula:

[0170] (13);

[0171] Then, the fused information is represented as a transformed graph-aware node vector. After concatenation and linear transformation, the edges are sequentially passed through a Sigmoid nonlinear activation function layer and a Softmax activation function layer to achieve edge link prediction, as shown in the following formula:

[0172] (14);

[0173] in, Let be the probability distribution function learned during the graph perception process. , For learnable parameters, The Sigmoid non-linear activation function and the Softmax activation function (normalized exponential function) convert the vector representation of edges into probabilities of predicted label types, and the true label set. The positive edge label is 1, and the negative edge label is 0.

[0174] 3) Graph-aware loss function

[0175] Will , Parameters are uniformly defined as follows The input vector is the feature vector of the graph-aware node. The parameters of the encoder for the input vector are defined as follows: The actual category label is The probability distribution function learned based on the graph perception process The loss function for the graph sensing process is calculated using crossover loss entropy:

[0176] (15);

[0177] (2) Time-aware module

[0178] The time-aware module learns the time series of interactions between nodes in a multi-level collaborative interaction graph. The conditional probability density function at time step enables the model to identify nodes that exhibit behavior in a given time series;

[0179] 1) Time series construction

[0180] Loading nodes, edges, and timestamps from a multi-level collaborative interaction graph indicates which node performed the interaction and when. By parsing the timestamps, they are standardized into timezone-free time data for subsequent chronological sorting. Then, the entire event sequence is sorted in ascending order by timestamps to ensure event history. The time sequence must be in chronological order; then the sorted time series will be divided into fixed lengths. (128) Divide into several continuous segments to represent the behavior sequence under multiple local time windows; finally, for each segment, calculate the time interval between events and perform a small smoothing (such as adding 0.001) to avoid the problem that the time series construction will fail because the interval between two events is the same time and cannot be distinguished.

[0181] Finally, the time series was constructed. ,in Indicates the first The node identifier that triggers an event can be understood as which user performed an action (such as tweeting or liking) at that point in time. Indicates the first The time when the event occurred.

[0182] 2) Historical information acquisition based on Transformer historical information encoder

[0183] For event time series The node information and time information are first converted into time-aware node feature vectors by using a linear transformation. Then, based on the translation-invariant time kernel function Encode time information by converting the time series into a format with the same dimension as the feature vector of the time-aware nodes. The time vector is then added to the feature vector of the time-aware node to obtain a hybrid information vector that combines spatial and temporal information. This hybrid information vector is then added to the position encoding vector in the Transformer to generate the input vector. .

[0184] Based on the Transformer mask self-attention mechanism, The output at each moment is used to encode historical information and ensures The generation of time-based historical information relies solely on The information encoded before a certain time is represented by the following formula:

[0185] (16);

[0186] in, , , These are learnable parameters responsible for linearly transforming the input vector into the parameters used in the attention mechanism. , , The Mask (information mask) is an upper triangular matrix, with the upper triangular part set to negative infinity.

[0187] Finally, the historical information at each time point is obtained after passing through the Transformer's Feed Forward Network. .

[0188] 3) Time series decoding

[0189] For historical information, a log-normal distribution is used for modeling. The conditional probability density function at time step 1, and using a multilayer perceptron (MLP) and Activation function prediction in User nodes that constantly engage in interactive behavior are defined as follows:

[0190] (17);

[0191] in, , , These represent the mixture weights, mixture variance, and mixture mean, respectively. yes User nodes that interact at any given moment.

[0192] 4) Time-aware loss function

[0193] In the time-aware component , , The parameters in the MLP, Transformer history encoder are defined as follows: Input time-aware node feature vector The encoder parameters are defined as follows: The probability distribution function learned by the time-aware process based on historical information and The loss function for the time-aware process can be defined as:

[0194] (18);

[0195] (3) Perception and Interaction Module

[0196] The perception and interaction module uses a self-attention mechanism to enable the model to share the time series and graph structure distribution characteristics learned by the graph perception module and the time perception module in the two modules.

[0197] To combine the data distribution learned in time-aware processing with that learned in graph-aware processing, MLK-TGCA-MD uses a self-attention mechanism for perceptual interaction and updates the parameters of the graph encoder, which serves as the common encoding layer, based on a joint loss function, enabling it to optimize the generated embedding vectors. It can fit into the function distribution learned throughout the entire perception process.

[0198] 1) Consistency-aware interaction

[0199] MLK-TGCA-MD uses a self-attention mechanism to allow the perceptual knowledge learned in the graph-aware and time-aware components to interact, enabling knowledge sharing and consistency between the two components. This is achieved in the two vector spaces after linear transformation. and Linear transformations are performed to generate the corresponding values ​​in self-attention. and The values ​​are then used to calculate the interaction attention weights separately:

[0200] (19);

[0201] in, It is a dot product function. It's the embedding space dimension. Then, based on attention weights... Value and The values ​​are used to update the generated interaction embeddings to obtain the interaction vector representation. and Then with the original vector and By adding the residual structures separately to avoid gradient vanishing, the perceptual interaction vector is obtained. and And replace the original graph-aware node feature vectors. and time-aware node feature vectors This is used in subsequent graph perception and time perception processes.

[0202] 2) Joint sensing loss function

[0203] After consistency-aware interaction, the parameters of the shared coding layer graph encoder are jointly updated to ensure that the resulting embedding vectors are updated. The data can be fitted to both perceptual learning data distributions, and the joint loss function is then updated as follows:

[0204] (20);

[0205] in, To determine the loss weight, MLK-TGCA-MD sets it to 0.5, for multi-level collaborative interaction graphs. This is the initial input.

[0206] 3. Group Detection Module

[0207] The group detection module combines the K-means clustering method of Gaussian mixture model (GMM) with graph perception and time perception interaction. After this interaction, it performs collaborative group identification on the embedded vectors of the fitted time series and graph structure distribution characteristics generated by the graph encoder, and outputs the collaborative group detection results of the social robot.

[0208] Based on time - Figure 1 After coherent joint sensing, the graph encoder can generate embedding vectors that, to a certain extent, fit the characteristics of time and graph data distribution. Because the goal of MLK-TGCA-MD is to identify cooperative groups, it requires modeling the cooperative groups and optimizing the embedding vectors. This allows the model to reflect the distribution characteristics of the group. Since methods like K-means cannot be integrated into the model training, a multivariate Gaussian mixture distribution is used for modeling during training. After training, the optimized embedding vectors are used for further analysis. K-means clustering is used to output the final cooperative group detection results. Given user nodes Belonging to The loss function for each cooperative group and multivariate Gaussian mixture distribution is as follows:

[0209] (twenty one);

[0210] in, Represents the probability density function. It is the weight of the distribution. It is the mean vector. It is the covariance matrix.

[0211] Step S5: Train the social robot collaborative group detection model;

[0212] During the training of the group detection and recognition model, the training objective is to minimize the joint perceptual loss function. Achieve graph-temporal semantic consistency awareness while minimizing the group identification loss function (loss function of multivariate Gaussian mixture distribution). Group modeling is implemented. Specifically, the encoder parameters are first pre-trained based on group recognition loss. Parameters in the group recognition model Then, minimize the joint perception loss and group recognition loss, and optimize the parameters. , , and After the model training is completed, based on the trained encoder parameters... Generate optimized node representation vector representation ( (This refers to the number of training rounds), which is used for subsequent group detection.

[0213] experiment

[0214] 1. Performance evaluation experiment

[0215] (1) Dataset selection

[0216] Since existing research lacks datasets with real labels for robot collaborative groups, this experiment uses the Twitter Collaborative Accounts Dataset (IRA) and the Weibo social robot dataset constructed in this invention for experimental evaluation.

[0217] (2) Compare to baseline

[0218] To evaluate the performance of the social robot cooperative group detection method proposed in this invention, this experiment compares it with advanced cooperative group detection methods, including IRL, MalReG, CLSB, Synchronized Action Framework, Transformer Hawkes, AMDN-HAGE, and VigDet.

[0219] (3) Evaluation indicators

[0220] In this experiment, we used the IRA dataset with real labels and the Weibo social bot dataset without real labels. Therefore, we used two sets of metrics to evaluate the detection performance of the method:

[0221] 1) Evaluation metrics based on real labels

[0222] Robotic cooperative group detection is essentially a multi-classification task. This experiment selected common multi-classification human evaluation metrics such as accuracy, precision, recall, F1 score, standardized mutual information, and adjusted Land coefficient to evaluate the performance of the detection method.

[0223] 2) Group density assessment indicators

[0224] For the cooperative groups identified by robot cooperative group detection, average degree, graph density, and clustering coefficient are used to measure the tightness within the group.

[0225] (4) Experimental setup

[0226] This experiment used the Adam optimizer to train the MLK-TGCA-MD model for 50 epochs, with Adam set to a learning rate of 1e-3 and regularization of 1e-5. The embedding vector dimension was 128, and the batch size was 128. Since unsupervised methods cannot predict the potential number of co-optation groups, MLK-TGCA-MD used the Silhouette score to evaluate the optimal number of co-optation groups. The Silhouette score is a metric used to evaluate clustering quality, helping to determine whether the clustering results for a given dataset are appropriate and to determine the optimal number of clusters. After Silhouette score tuning, it was determined that on the IRA dataset, the Silhouette score was optimal with 3 clusters, which is consistent with the actual distribution in the IRA dataset. On the Weibo social robot dataset, MLK-TGCA-MD generated the highest Silhouette score when it produced 12 clusters.

[0227] (5) Experimental results

[0228] Table 3 presents the experimental results of the proposed detection method and baseline methods on the IRA dataset. The proposed detection method, MLK-TGCA-MD, outperforms existing state-of-the-art baseline methods on all metrics. Specifically, compared to the second-best performing baseline method, VigDet, MLK-TGCA-MD improves accuracy by 0.05 and Marco F1 score by 0.15. On the Marco Precision and Marco Recall metrics, the prediction precision and recall for the three classes reach 0.8101 and 0.7332, respectively, demonstrating that MLK-TGCA-MD has strong predictive ability for cooperative groups of different classes. On the NMI and ARI metrics, MLK-TGCA-MD shows significant superiority, indicating that its prediction results are closer to cooperative groups in real space and have better cooperative group identification capabilities.

[0229] Table 3 Experimental results of MLK-TGCA-MD and current state-of-the-art baseline methods on the IRA dataset. .

[0230] 2. Social Robot Collaborative Group Recognition Experiment on Weibo

[0231] The Weibo social robot dataset contains 13,002 social robot nodes and a wealth of multimodal information. This comprehensive information helps reconstruct the collaborative activities of robot nodes throughout the timeline. We used MLK-TGCA-MD to detect collaborative groups of social robots in the Weibo social robot dataset to analyze the characteristics of collaborative groups on Weibo. During the construction of the multi-level knowledge graph, a multi-level collaborative weight graph containing 13,002 robot nodes and 17,559,070 edges was constructed. Through weight filtering (removing edges with low weight values ​​during construction, totaling 90%) and time-graph heterogeneity awareness, MLK-TGCA-MD identified a total of 6,704 highly collaborative nodes and 158,750 edges. The identification results are as follows: Figure 5 As shown in the figure, multiple robots working together are illustrated, with larger nodes indicating larger TF-IDF weights.

[0232] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting collaborative groups of social robots based on multi-level collaborative knowledge graphs, characterized in that, Includes the following steps: Step S1: Collect user data from social platforms using distributed crawlers and construct a social robot dataset; Step S2: Based on the collected user data, extract user relationship features and their corresponding timestamps; the user relationship features include direct interaction relationships and collaborative interaction relationships; Step S3: Based on the user relationship characteristics and their corresponding timestamps, construct a direct interaction weight graph and a collaborative interaction weight graph respectively, and then merge the two to obtain a multi-level collaborative interaction weight graph; construct a multi-level collaborative interaction graph using the edges with abnormal weights in the multi-level collaborative interaction weight graph. Step S4: Construct a social robot collaborative group detection model; the multi-level collaborative interaction graph is the input data for the social robot collaborative group detection model; Step S5: Train the social robot collaborative group detection model; Step S6: Use the social robot collaborative group detection model to perform collaborative group detection on the social platform under test.

2. The social robot collaborative group detection method based on multi-level collaborative knowledge graph as described in claim 1, characterized in that, Step S1 includes the following steps: S11. Build a distributed crawler based on the Celery framework, including message middleware, crawler nodes, node monitoring engine, and storage module: The message middleware is implemented based on the third-party component Redis and is used to distribute tasks downwards; the crawler nodes are used to execute anti-crawling strategies, establish HTTP connections with social platform servers, collect user data, and transfer it to the storage module for storage; the node monitoring engine is implemented using Sentry and is used to capture abnormal situations of crawler nodes; the storage module uses a MongoDB document-oriented NoSQL database to store the collected user data. S12. Collect user data from social platforms by constructing a distributed crawler, and collect users who participated in trending topics, and expand the collection scale based on the BFS algorithm. S13. Manually label the collected user data to construct a social robot dataset with real labels.

3. The social robot collaborative group detection method based on multi-level collaborative knowledge graph as described in claim 1, characterized in that, The direct interaction relationships include following, liking, commenting, forwarding, and mentioning; the collaborative interaction relationships include shared topics, shared follows, shared mentions, shared URLs, shared likes, shared forwards, and shared comments.

4. The social robot collaborative group detection method based on multi-level collaborative knowledge graph as described in claim 1, characterized in that, Step S3 includes the following steps: S31. Using users as nodes, if there is a direct interaction relationship between two nodes, an edge is established between them. The timestamp corresponding to the direct interaction relationship is used as the attribute of the edge, and the co-occurrence frequency of the two nodes is used as the weight of the corresponding edge. A direct interaction weight graph is constructed, and the formula for the edge weight is as follows: （1）； in, For edge weights, For the first 1 node For the first 1 node Let be the set of edges. It is an edge that belongs to the edge set; S32. Obtain collaborative interaction events based on collaborative interaction relationships; using users as individual nodes and collaborative interaction events as event nodes, if a collaborative interaction relationship exists between an individual node and an event node, establish an edge between the individual node and the event node to construct a bipartite graph; project the bipartite graph, using the timestamp corresponding to the collaborative interaction relationship as the attribute of the edge, to generate a collaborative interaction weight graph composed of a set of individual nodes, where each edge... weight There are two body nodes. and The co-occurrence count of neighboring nodes, where neighboring nodes represent other individual nodes connected to the same event node, is calculated using the following formula: （2）； in, Represents individual nodes The set of neighboring nodes, Represents individual nodes The set of neighboring nodes, For individual nodes With individual nodes The number of co-occurrences of neighboring nodes; S33. Merge the direct interaction weight graph and the collaborative interaction weight graph to generate a multi-order collaborative interaction weight graph. The edge weights of the multi-order collaborative interaction weight graph are obtained by adding the weights of the same edges in the direct interaction weight graph and the collaborative interaction weight graph. S34. Merge the direct interaction weight graph and the collaborative interaction weight graph to generate a multi-level collaborative interaction weight graph, and filter the edges in the multi-level collaborative interaction weight graph based on the preset threshold of the edge weight, retaining only the edges with abnormal weights, to obtain the multi-level collaborative interaction graph.

5. The social robot collaborative group detection method based on multi-level collaborative knowledge graph as described in claim 1, characterized in that, The social robot collaborative group detection model in step S4 includes a Graph Transformer encoder module, a graph perception module, a time perception module, a perception interaction module, and a group detection module. The Graph Transformer encoder module encodes nodes in the input multi-order collaborative interaction graph into node feature vectors through a self-attention mechanism; The graph perception module learns the probability distribution function of edge link prediction in a multi-order collaborative interaction graph, enabling the model to identify abnormal edges. The time-aware module learns the time series of interactions between nodes in a multi-level collaborative interaction graph. The conditional probability density function at time step enables the model to identify nodes with temporal anomalies; The perception and interaction module uses a self-attention mechanism to enable the model to share the time series and graph structure distribution characteristics learned by the graph perception module and the time perception module in the two modules. The group detection module combines the K-means clustering method of Gaussian mixture model (GMM) with graph perception and time perception interaction to perform collaborative group identification on the embedded vectors of the fitted time series and graph structure distribution characteristics generated by the Graph Transformer encoder, and outputs the collaborative group detection results of social robots.

6. The method for detecting collaborative groups of social robots based on multi-level collaborative knowledge graphs as described in claim 5, characterized in that, The social robot collaborative group detection model specifically includes: (1) Graph Transformer encoder module: The Graph Transformer encoder module contains several layers of Graph Transformer encoders; The initial node feature sequence of the multi-order collaborative interaction graph is obtained through Node2vec and input into the GraphTransformer encoder module. The Graph Transformer layer obtains the user's node feature vector. The node feature vectors are input into the graph perception module and the time perception module; (2) Graph perception module: The node feature vectors obtained by the Graph Transformer encoder module undergo spatial transformation through a linear layer, as shown in the following formula: （3）； in, For node feature vectors, and For learnable parameters, Graph-aware node feature vectors; In a multi-order collaborative interaction graph, edges are defined as positive edges. Batch negative sampling is performed on the positive edges in the graph to generate negative sampled edges. And merge the negative sampling edges with the positive edges, the merged edges Represented as graph-aware node feature vectors The concatenation of elements is followed by sequential passing through a linear layer, a Sigmoid non-linear activation function layer, and a Softmax activation function layer to achieve edge link prediction, as shown in the following formula: （4）； in, Let be the probability distribution function learned during the graph perception process. , For learnable parameters, It is a Sigmoid non-linear activation function. Use the Softmax activation function; and Representative node and nodes Graph-aware node feature vectors Represents the node With nodes The prediction result indicates whether there is an edge between them. Positive edges are labeled as 1, and negative edges are labeled as 0. (3) Time-aware module: The node feature vectors obtained by the Graph Transformer encoder module are transformed into time-aware node feature vectors through a linear layer. ; Construct a time series based on the nodes, edges, and timestamps in the multi-level collaborative interaction graph. , indicating that the first User nodes of interactive events and time ; Based on translation time-invariant kernel function The time series is converted into a time vector with the same dimension as the feature vector of the time-aware node. The time vector is added to the feature vector of the time-aware node to obtain a hybrid information vector that combines spatial and temporal information. The hybrid information vector is input into the Transformer encoder to obtain historical information at each time point. ; For historical information, a log-normal distribution is used for modeling. Conditional probability density function at time t and using a multilayer perceptron (MLP) and Activation function prediction in The formula for user nodes that constantly interact is as follows: （5）； in, , , These represent the mixture weights, mixture variance, and mixture mean, respectively. yes User nodes that interact at any given moment; This represents the number of log-normal distributions used to fit probabilities in the model; (4) Perception and Interaction Module: Graph-aware node feature vectors and time-aware node feature vectors Linear transformations are performed to generate the corresponding queries, keys, and values, and an interaction embedding vector is generated through a cross-attention mechanism. and , and then with and The perceptual interaction vector is obtained by adding the residual structures separately. and Using perceptual interaction vectors and Replace the original graph-aware node feature vector and time-aware node vectors To perform subsequent graph perception and time perception; (5) Group detection module: After graph-aware and time-aware interaction, the Graph Transformer encoder can generate embedding vectors that fit the time series and the distribution characteristics of the graph structure. Embedded vector Modeling of cooperative groups is achieved using Gaussian Mixture Models (GMMs), and the embedding vectors are optimized. ; for the optimized embedding vector The K-means clustering method is used to perform clustering, and the results of social robot collaborative group detection are output.

7. The social robot collaborative group detection method based on multi-level collaborative knowledge graph as described in claim 6, characterized in that, Step S5 includes: By combining the graph perception module, time perception module, and perception interaction module, a joint perception loss function is constructed. The loss function of Gaussian Mixture Model (GMM) is used as the group identification loss function; The training objective of the social robot collaborative group detection model is to minimize the joint perception loss function to achieve joint perception of graph structure and time series, while minimizing the group identification loss function to achieve collaborative group modeling.

Images